Initial commit

docs/policy/architecture.md

@@ -0,0 +1,98 @@
+# Architecture
+
+This document defines the development principles for this Go project. It is inward-facing: developers and LLM coding agents should use it to preserve the project’s shape, boundaries, and invariants as the code evolves.
+
+## Project Shape
+
+Default to a small, explicit, dependency-light Go application. Keep the design modular enough to test and change safely, but do not add abstraction unless it protects a real boundary or enables a real extension point.
+
+Business/domain logic should live outside CLI, transport, and external-adapter packages.
+
+## Dependency Policy
+
+Prefer the Go standard library where practical.
+
+Use external dependencies only when justified by correctness, security, interoperability, or substantial complexity reduction. Good reasons include complex security-sensitive behavior, such as HTML sanitization, or widely used de facto standards, such as YAML parsing.
+
+Avoid dependencies for small conveniences. Do not let external dependency types leak across internal package boundaries unless the dependency is itself the explicit public contract of that package.
+
+## Package Layout
+
+Use this layout unless the project has a documented reason to differ:
+
+- `internal/app`: application orchestration and top-level use cases.
+- `internal/cli`: CLI command definitions, flags, argument parsing, and command wiring.
+- `internal/config`: configuration structs, defaults, loading, precedence, and validation.
+- `internal/adapters/<name>`: adapters for external CLIs, APIs, databases, object stores, or libraries.
+- `internal/api`: HTTP API handlers and request/response types, when the application exposes an HTTP API.
+- `internal/transport/http`: HTTP client code, when the application calls HTTP services.
+
+Package-private implementation constants may live near the package that owns them, preferably in `constants.go` when useful.
+
+## Configuration
+
+Centralize configuration loading, processing, precedence, defaults, and validation in `internal/config`.
+
+The goal is to make configuration discoverable and avoid implicit or hidden operational values. User-visible defaults and cross-package operational defaults should be defined in `internal/config/defaults.go`.
+
+Unless documented otherwise, precedence is:
+
+1. CLI flags
+2. environment variables
+3. configuration file
+4. built-in defaults
+
+Prefer YAML configuration unless the project has a strong reason to use another format. Config files should be discovered at `/usr/local/etc/<app_name>/config.yml`, with a CLI override via `--config`.
+
+Configuration files should not contain raw secrets unless the application is explicitly designed for that. Prefer environment variables or secret files for secrets.
+
+## Adapters and External Integrations
+
+Use a hexagonal architecture style for external integrations.
+
+External adapters belong under `internal/adapters/<name>`. If an adapter uses an external dependency, that dependency’s interface must not leak outside the adapter package. Other packages should interact only with the adapter’s API, so the dependency can be swapped, upgraded, or removed without touching unrelated code.
+
+Adapters should be thin. Domain decisions belong in application/domain packages, not inside adapter glue.
+
+## Modules, Stages, and Registries
+
+When the application has stages or modules, each major stage/module should live in its own package and have an explicit input/output contract.
+
+The orchestrator should be able to compose, skip, resume, or run individual stages/modules when their prerequisites are satisfied. Ordering should be explicit: use a default sequence, dependency graph, or documented orchestration rule.
+
+If users can select modules, stages, validators, renderers, or adapters, selection should go through a registry or equivalent mechanism rather than scattered conditionals.
+
+## Embedded Assets
+
+Store embedded JSON schemas, Markdown prompts, templates, and similar assets as separate files, not inline string literals, unless there is a strong reason otherwise.
+
+## Errors and Logging
+
+Errors should be actionable and preserve context. Wrap errors with operation and path/resource context. CLI code should convert internal errors into concise user-facing messages.
+
+Errors and logs must not expose secrets.
+
+Use structured logging where practical. Logs should describe operations, paths, external calls, retries, and failure causes, but should not include large user data by default.
+
+## Context, Timeouts, and Cancellation
+
+Long-running operations should accept `context.Context`. External calls, subprocesses, HTTP requests, storage operations, and multi-stage workflows should respect cancellation and timeouts.
+
+## State, Files, and Safety
+
+If the application writes durable state, writes should be atomic where practical. Multi-step workflows should preserve enough state to support inspection, retry, or resume after failure.
+
+Code that deletes, moves, or overwrites files must use narrow, explicit paths. Avoid broad parent-directory operations. Cleanup that can cause data loss must be opt-in.
+
+## Testing
+
+Core logic should be testable without real external services. Use fakes, fixtures, or local test doubles for adapters where practical.
+
+Config examples should be load-tested. Important CLI workflows should have parser or command tests. Stage/module contracts should have focused tests that do not require running the full application unless end-to-end coverage is intentional.
+
+## Documentation
+
+Documentation should follow the project documentation policy. Keep user docs focused on implemented behavior. Put future, planned, or aspirational work only under `docs/roadmap/`.
+
+When changing architecture, config, CLI behavior, adapters, or stage/module contracts, update the relevant docs and examples in the same change.
+

docs/policy/documentation.md

@@ -0,0 +1,356 @@
+# Go Project Documentation Policy
+
+## Purpose
+
+Project documentation must help four audiences:
+
+1. users who need to run the application;
+2. administrators/operators who need to configure and operate it;
+3. developers who need to understand and change it safely;
+4. LLM coding agents that need clear scope, boundaries, and invariants.
+
+Docs should be accurate, concise, task-oriented, and organized by audience. Prefer links to canonical docs over repetition.
+
+## Core Rules
+
+### 1. Keep docs concise
+
+Each document should cover a defined scope and only the essentials for that scope.
+
+Avoid:
+- long background explanations;
+- repeated reference material;
+- implementation detail in user-facing docs;
+- aspirational language outside roadmap docs;
+- verbose examples where one minimal example is clearer.
+
+### 2. Document only implemented behavior outside roadmap files
+
+Unimplemented, planned, aspirational, experimental, or future work may be described only under:
+
+- `docs/roadmap/`
+
+No other documentation file, including `README.md`, should describe code, features, modules, stages, commands, config fields, or behaviors that do not currently exist.
+
+If a feature is partial, non-roadmap docs may describe only the implemented portion and its current boundary.
+
+### 3. Use canonical homes
+
+Each type of information should have one canonical location.
+
+Canonical homes:
+
+- project purpose and quickstart: `README.md`
+- development principles: `docs/policy/architecture.md`
+- configuration reference: `docs/config.md`
+- CLI reference: `docs/cli.md`
+- operations and recovery: `docs/operations.md`
+- troubleshooting: `docs/troubleshooting.md`
+- implemented internals: `docs/internal/`
+- future work: `docs/roadmap/`
+- contributor workflow: `docs/policy/development.md`
+- copyable examples: `examples/`
+
+Other files should summarize briefly and link to the canonical source.
+
+### 4. Keep examples real
+
+Examples should be valid, maintained, and free of secrets.
+
+Where practical:
+- example configs should load successfully;
+- example commands should match real CLI syntax;
+- important examples should be covered by tests.
+
+## Documentation Profiles
+
+All projects require:
+
+- `README.md`
+- `docs/policy/architecture.md`
+
+Additional docs depend on the project.
+
+### Small library
+
+Recommended:
+- `docs/policy/development.md`, if contributor conventions are non-obvious
+
+### Simple CLI
+
+Required:
+- `docs/cli.md`
+
+Recommended:
+- `docs/policy/development.md`
+
+### Config-driven CLI
+
+Required:
+- `docs/cli.md`
+- `docs/config.md`
+
+Recommended:
+- `examples/`
+- `docs/policy/development.md`
+
+### Stateful or operator-facing application
+
+Required:
+- `docs/cli.md`, if CLI-based
+- `docs/config.md`, if config-driven
+- `docs/operations.md`
+
+Recommended:
+- `docs/troubleshooting.md`
+- `examples/`
+- `docs/policy/development.md`
+
+### Modular, staged, service-oriented, or orchestration application
+
+Required:
+- `docs/cli.md`, if CLI-based
+- `docs/config.md`, if config-driven
+- `docs/operations.md`
+- `docs/internal/`
+- `docs/policy/development.md`
+
+Recommended:
+- `docs/troubleshooting.md`
+- validated examples under `examples/`
+
+## Required Documents
+
+### README.md
+
+**Audience:** users, administrators, operators
+
+The README is the outward-facing project orientation page.
+
+It should include, in order:
+
+1. concise description;
+2. elevator pitch;
+3. shortest useful command or usage example;
+4. links to targeted docs.
+
+The README should be short. It is not a manual.
+
+The “shortest useful command” means the simplest command that performs the project’s core use case. (It does not mean `app --help`.)
+
+### docs/policy/architecture.md
+
+**Audience:** developers, LLM coding agents
+
+`docs/policy/architecture.md` is required for every project.
+
+It is an inward-facing development policy document. It should describe how the project is intended to be built and changed.
+
+It should include:
+
+- project shape;
+- core design principles;
+- package and boundary philosophy;
+- state/persistence philosophy, if applicable;
+- external integration philosophy, if applicable;
+- error-handling and logging principles;
+- testing expectations;
+- documentation expectations;
+- architectural invariants;
+- explicit non-goals, if useful.
+
+For small projects, this file may be brief. It may simply state that the project is intentionally narrow, monolithic, and dependency-light.
+
+### docs/policy/development.md
+
+**Audience:** developers, LLM coding agents
+
+Required for projects maintained by humans and LLM coding agents.
+
+It should include:
+
+- repository layout;
+- build/test commands;
+- coding conventions;
+- dependency policy;
+- how to add config fields;
+- how to add CLI flags;
+- how to add stages/modules/adapters, if applicable;
+- how to update examples;
+- documentation update expectations.
+
+### docs/config.md
+
+**Audience:** administrators, operators, advanced users
+
+Required for applications with configuration files.
+
+It should include, in order:
+
+1. config file locations and discovery precedence;
+2. minimal working config;
+3. production-oriented config;
+4. full configuration reference;
+5. secrets handling, if applicable;
+6. links to maintained examples.
+
+The full configuration reference should be canonical.
+
+### docs/cli.md
+
+**Audience:** users, administrators, operators
+
+Required for CLI applications.
+
+It should include, in order:
+
+1. shortest useful command;
+2. command overview;
+3. complete flag reference;
+4. common workflows;
+5. diagnostic or recovery commands, if applicable.
+
+Explain when commands are useful, not just their syntax.
+
+### docs/operations.md
+
+**Audience:** administrators, operators
+
+Required for applications that maintain state, support resume behavior, run multiple stages, write durable artifacts, use remote storage, or require recovery procedures.
+
+It should cover:
+
+- normal workflow;
+- filesystem layout;
+- remote storage layout, if applicable;
+- logs and manifests;
+- resume/retry behavior;
+- cleanup behavior;
+- archive/backup behavior;
+- safe recovery procedures;
+- operational caveats.
+
+### docs/troubleshooting.md
+
+**Audience:** administrators, operators
+
+Recommended once recurring failure modes exist.
+
+Each entry should include:
+
+- symptom;
+- likely cause;
+- diagnostic command or inspection step;
+- safe fix;
+- relevant links.
+
+### docs/internal/
+
+**Audience:** developers, LLM coding agents
+
+Required for modular, staged, service-oriented, or orchestration projects.
+
+This directory describes implemented internal components. It is not the roadmap.
+
+Use one file per major component where useful.
+
+Each component doc should include:
+
+1. purpose;
+2. inputs and outputs;
+3. boundaries;
+4. config fields used;
+5. external adapters used;
+6. state or manifest behavior, if applicable;
+7. skip/resume behavior, if applicable;
+8. failure behavior;
+9. tests to inspect before changing;
+10. architectural invariants.
+
+### docs/roadmap/
+
+**Audience:** maintainers, developers, LLM coding agents
+
+This is the only place for planned, future, aspirational, experimental, or unimplemented work.
+
+Roadmap docs should clearly distinguish:
+
+- proposed work;
+- accepted plans;
+- deferred ideas;
+- rejected ideas;
+- implementation prompts or task breakdowns, if useful.
+
+Roadmap docs should not be confused with current behavior.
+
+### docs/integrations/
+
+**Audience:** developers, LLM coding agents
+
+Required for projects that depend on external CLIs, APIs, services, protocols, or file formats where the integration contract is important to maintain.
+
+This directory contains concise, versioned reference notes for external integration contracts. It should document only the parts of the external system that this project actually uses.
+
+Use one file per integration where useful.
+
+## Examples Directory
+
+Projects with non-trivial configuration or workflows should include `examples/`.
+
+Useful examples include:
+
+- minimal working config;
+- production-oriented config;
+- full annotated config;
+- local development config;
+- remote/object-storage config;
+- minimal session/input file.
+
+Examples should be valid, maintained, tested when practical, and linked from relevant docs.
+
+## Security and Privacy
+
+Docs and examples must not include:
+
+- real API keys;
+- tokens;
+- passwords;
+- private keys;
+- private environment dumps;
+- sensitive user data;
+- raw private transcripts;
+- private infrastructure details unless intentionally public.
+
+Document secret-handling mechanisms, not actual secret values.
+
+## Maintenance Rules
+
+When docs change, verify the affected behavior.
+
+Where practical:
+
+- load example config files in tests;
+- test CLI examples or command parser behavior;
+- validate documented flags against real flags;
+- remove stale references;
+- update links after renames;
+- keep roadmap content out of non-roadmap docs.
+
+If documentation and code disagree, fix the documentation and/or open a roadmap item; do not leave aspirational behavior in current-behavior docs.
+
+Documentation is complete only when it matches the current code.
+
+## Documentation Change Checklist
+
+Before merging documentation changes, verify:
+
+- README is concise and orientation-focused.
+- `docs/policy/architecture.md` describes development principles.
+- Future work appears only under `docs/roadmap/`.
+- User-facing docs avoid unnecessary internals.
+- Developer-facing docs preserve boundaries and invariants.
+- Config examples match the schema.
+- CLI examples match real commands and flags.
+- Defaults appear in the canonical config reference.
+- No secrets or private data are included.
+- Links are accurate.