diff --git a/AGENTS.md b/AGENTS.md index 2829342..34531ca 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,2 @@ -Please review `docs/internal/overview.md` for initial orientation in this repository. - -Additionally, please carefully review the relevant documents in `docs/policy` before making any changes to this repository. - - `development.md` defines the contributor workflow for this application. - - `architecture.md` provides the canonical high-level architecture policy for this repository, and should be reviewed before writing or changing any code. - - `documentation.md` provides the canonical documentation policy for this repository, and should be reviewed before writing or changing any documentation. +Please review `docs/development.md` for initial orientation in this repository +and follow its task-specific reading guide. diff --git a/README.md b/README.md index aaa5379..13845d4 100644 --- a/README.md +++ b/README.md @@ -23,11 +23,10 @@ Useful references: - [CLI reference](docs/cli.md) - [Configuration reference](docs/config.md) - [Operations](docs/operations.md) -- [Troubleshooting](docs/troubleshooting.md) - [Seriatim input contract](docs/integrations/seriatim.md) - [JSON output contract](docs/integrations/json-output.md) - [D&D spell artifact contract](docs/integrations/dnd-spell-artifacts.md) -- [Developer workflow](docs/policy/development.md) +- [Developer guide](docs/development.md) - [Internal architecture docs](docs/internal/overview.md) - [Maintained example config](examples/dnd-spells.config.yml) - [Maintained example input](examples/seriatim-minimal-transcript.json) diff --git a/docs/adr/0001-record-architecture-decisions.md b/docs/adr/0001-record-architecture-decisions.md new file mode 100644 index 0000000..32c9ff6 --- /dev/null +++ b/docs/adr/0001-record-architecture-decisions.md @@ -0,0 +1,23 @@ +# ADR-0001: Record architecture decisions as ADRs + +**Status:** Accepted +**Date:** 2026-07-13 + +## Context +Architectural reasoning made during design (pattern choices, rejected +alternatives, trigger conditions for revisiting) is lost if only the final +state is documented. + +## Decision +We keep a living overview in docs/policy/architecture.md describing current +intended state, and immutable, numbered ADRs (Nygard format) in docs/adr/ +recording each significant decision, its alternatives, and its consequences. +Changed decisions get a new ADR that marks the old one Superseded. + +## Alternatives considered +- Overview doc only: loses the "why" and the rejected options. +- arc42 / RFC-style design docs: heavier than warranted for a solo repo. + +## Consequences +Small ongoing writing cost; durable reasoning trail; cheap onboarding for +future contributors (including future-us). \ No newline at end of file diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..4f7b2d6 --- /dev/null +++ b/docs/development.md @@ -0,0 +1,75 @@ +# Development + +This is the first-read landing page for people and LLM coding agents working on +Notarius. It provides a concise repository orientation and routes each kind of +change to its canonical documentation. + +## Orientation + +Notarius is a small Go application for extracting structured data from source +material. It is a general extraction platform with an initial Seriatim and D&D +implementation. Configured modules run through a fixed workflow: + +```text +input -> chunk -> extract -> merge -> normalize -> output +``` + +The CLI is the application boundary. Core packages own deterministic models and +policy, framework packages own reusable contracts and orchestration, and module +and validator packages own concrete behavior. + +## Repository Map + +- `cmd/notarius`: executable entry point. +- `internal/cli`: CLI behavior and production composition. +- `internal/core`: deterministic source, config, artifact, diagnostics, and + workspace packages. +- `internal/framework`: contracts, pipeline orchestration, validation helpers, + checkpoints, debug recording, and LLM runtime plumbing. +- `internal/modules`: concrete implementations of the six pipeline stages. +- `internal/validators`: concrete output validators. +- `docs`: canonical policy, reference, integration, internal, ADR, and roadmap + documentation. +- `examples`: maintained, secret-free example inputs and configuration. + +See [Internal Overview](internal/overview.md) for the implemented component +map and links to focused internal documentation. + +## What To Read + +| When working on | Read | Why | +| --- | --- | --- | +| Application shape, package boundaries, contracts, dependency direction, runtime guarantees, or safety properties | [Architecture](policy/architecture.md) and relevant [ADRs](adr/) | Architecture defines the intended system and its invariants; ADRs preserve significant decision rationale. | +| Any documentation addition or revision | [Documentation Policy](policy/documentation.md) | It defines canonical homes, audiences, current-behavior rules, and maintenance requirements. | +| Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. | +| Production modules or validators | [Module Internals](internal/modules.md) | It documents implemented module contracts, capabilities, assets, and registration. | +| LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and Scriptorium integration. | +| Diagnostics, workspace state, resume, or debug artifacts | [Diagnostics Internals](internal/diagnostics.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. | +| CLI or user-visible configuration behavior | [CLI Reference](cli.md) and [Configuration](config.md) | These are the canonical user and operator references. | +| External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. | +| Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. | + +For an existing subsystem, also inspect its focused tests and the package-local +types and contracts before changing behavior. + +## Validation + +Use focused package tests while iterating. Run the repository-wide checks when +a change affects shared contracts, application behavior, or maintained +documentation examples: + +```sh +go test ./... +go vet ./... +go build ./cmd/notarius +``` + +## Universal Reminders + +- Keep non-roadmap documentation limited to implemented behavior. +- Update affected documentation and maintained examples in the same change as + behavior. +- Use fakes, fixtures, or local test servers instead of real external services + in tests. +- Do not expose secrets in code, errors, logs, diagnostics, manifests, + documentation, or examples. diff --git a/docs/internal/modules.md b/docs/internal/modules.md index 73e975e..391493e 100644 --- a/docs/internal/modules.md +++ b/docs/internal/modules.md @@ -263,6 +263,6 @@ When adding a module, keep source-format and extraction-domain boundaries clear: - merge and normalize modules own raw output combination and reconciliation; - output modules own serialization, not diagnostics or CLI reporting. -Update [Development](../policy/development.md), [Configuration](../config.md), +Update [Development](../development.md), [Configuration](../config.md), internal docs, integration docs, and examples when the new module becomes implemented production behavior. diff --git a/docs/internal/overview.md b/docs/internal/overview.md index 3fd99bb..1be9452 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -1,105 +1,127 @@ # Internal Overview -This directory documents implemented Notarius internals for developers and LLM -coding agents. It complements [Architecture](../policy/architecture.md), which -is the durable policy for boundaries and invariants. +This document maps the implemented Notarius components and their ownership. It +complements the durable invariants in [Architecture](../policy/architecture.md) +and links to focused internal documentation for deeper behavior. -## Executable And CLI +## Execution Path -`cmd/notarius` calls the CLI package. `internal/cli` owns: +The executable delegates to the CLI, which resolves configuration and wires the +production application around the framework runner: -- command parsing and usage; -- config discovery and loading; -- production module catalog and registry wiring; -- production LLM client construction; -- run directory creation; -- durable output writes; -- user-facing stdout, stderr, and exit codes. +```text +cmd/notarius + -> internal/cli + -> config resolution + production registries + LLM client + -> input -> chunk -> extract -> merge -> normalize -> output + -> durable output writes -The CLI should stay thin around framework contracts. Domain extraction behavior -belongs in modules, not in command handlers. +Pipeline side channels: + diagnostics checkpoints debug artifacts +``` + +Pipeline execution is serial. Configuration selects modules for the fixed stage +shape; registries construct them after profile, capability, validator, and +reference resolution. + +## Application Boundary + +`cmd/notarius` contains the executable entry point and delegates process exit +behavior to `internal/cli`. + +`internal/cli` owns command parsing, configuration discovery, production module +and validator registration, prompt asset collection, production LLM client +construction, reference preparation, workspace recorder setup, durable output +writes, and user-facing stdout, stderr, and exit codes. It is the composition +root for concrete production packages. ## Core Packages -- `internal/core/artifacts`: run manifests and legacy artifact serialization - shapes retained while pipeline handoff contracts use raw outputs. -- `internal/core/config`: defaults, YAML config parsing, environment overrides, - validation, redaction, and resolved pipeline config. -- `internal/core/diagnostics`: per-run diagnostics directory creation, - diagnostics artifact writers, atomic writes, and retention decisions. -- `internal/core/source`: source documents, source units, source references, and - validation. -- `internal/core/workspace`: effective workspace roots, enabled-state helpers, - safe workspace-relative path construction, atomic workspace artifact writes, - checkpoint identities, checkpoint path construction, and checkpoint manifest - types. +| Package | Implemented responsibility | +| --- | --- | +| `internal/core/artifacts` | Run manifests and artifact serialization shapes. | +| `internal/core/config` | Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline configuration. | +| `internal/core/diagnostics` | Diagnostics run directories, artifact writers, atomic writes, and retention decisions. | +| `internal/core/source` | Generic source documents, units, references, and validation. | +| `internal/core/workspace` | Workspace settings, safe paths and writes, checkpoint identities, and checkpoint manifest types. | -Core packages should remain deterministic and concrete. They should not import -production modules. +These packages provide concrete, deterministic models and policy. Production +module registration occurs at the CLI boundary rather than in core packages. ## Framework Packages -- `internal/framework/contracts`: interfaces and request/result structs for - input adapters, chunkers, extractors, mergers, normalizers, validators, output - encoders, and structured LLM clients. -- `internal/framework/checkpoint`: workspace-backed checkpoint recorder and - checkpoint payload envelope serialization. -- `internal/framework/debug`: workspace-backed debug artifact writer. -- `internal/framework/pipeline`: module registries, module specs, profile - resolution, capability checks, run orchestration, checkpoint and debug - recorder boundaries, warnings, validation, and manifest population. -- `internal/framework/llm`: Scriptorium-backed structured-output client, - prompt/schema asset registry, scheduler, schema registry, and secret - redaction. -- `internal/framework/validate`: validator decision helpers and cardinality - enforcement. +| Package | Implemented responsibility | +| --- | --- | +| `internal/framework/contracts` | Stage, validator, reference, output, and structured LLM interfaces and request/result types. | +| `internal/framework/pipeline` | Module registries, profile resolution, capability checks, reference materialization, validation chains, retries, orchestration, warnings, and manifest population. | +| `internal/framework/validate` | Shared validator decision and cardinality helpers. | +| `internal/framework/llm` | Scriptorium-backed structured completions, prompt/schema asset registration, scheduling, profile recording, and secret redaction. | +| `internal/framework/checkpoint` | Workspace-backed checkpoint loading, recording, and payload envelopes. | +| `internal/framework/debug` | Workspace-backed framework and LLM debug artifacts. | -Framework code should stay source-agnostic and domain-agnostic. +Framework contracts carry raw stage outputs between modules. The runner owns +provenance, validation sequencing, rejection handling, checkpoint boundaries, +debug boundaries, and final manifest assembly. -## Module Packages +## Production Modules -Production module packages live under `internal/modules`: +Production implementations live under `internal/modules` and register through +the CLI catalog. -- `input/seriatim` -- `chunk/generic` -- `chunk/dnd/scenes` -- `extract/dnd/spells` -- `merge/appendorder` -- `normalize/noop` -- `output/json` +| Stage | Module key | Package | Role | +| --- | --- | --- | --- | +| Input | `seriatim` | `internal/modules/input/seriatim` | Converts Seriatim transcript JSON into the generic source model. | +| Chunk | `generic` | `internal/modules/chunk/generic` | Splits ordered source units by configured unit counts and overlap. | +| Chunk | `dnd/scenes` | `internal/modules/chunk/dnd/scenes` | Uses structured LLM output to create contiguous D&D scene chunks. | +| Extract | `dnd/spells` | `internal/modules/extract/dnd/spells` | Extracts source-grounded D&D spell-cast artifacts. | +| Merge | `appendorder` | `internal/modules/merge/appendorder` | Combines accepted extract outputs in chunk order. | +| Normalize | `noop` | `internal/modules/normalize/noop` | Preserves accepted merged output unchanged. | +| Output | `json` | `internal/modules/output/json` | Encodes manifests, indexes, warnings, rejections, and accepted lane payloads as logical JSON files. | -Each module package owns its contract implementation, module spec, -registration, options, focused tests, and module-specific errors. +`internal/modules/sharedassets` composes shared prompt filesystems. +`internal/modules/sharedassets/dnd` owns shared D&D prompt fragments, reference +slots, prompt input assembly, and source-unit reference helpers. -Module-owned prompts and schemas live in each module's shallow `assets/prompts` -and `assets/schemas` directories. Generic shared prompt filesystem composition -lives in `internal/modules/sharedassets`; shared D&D prompt fragments and -reference helpers live in `internal/modules/sharedassets/dnd`. +## Validators -Shared asset package: `internal/modules/sharedassets` -Shared D&D helper package: `internal/modules/sharedassets/dnd` +Concrete validators live under `internal/validators` and register separately +from stage modules. Generic validators cover unconditional test decisions, JSON +syntax, and JSON Schema. D&D spell validators cover artifact shape, source +reference validity, and source relatedness. -## Fixtures And Tests +The production default chain for `dnd/spells` extract output is registered +centrally in `internal/cli`; module packages produce raw output but do not own +the production approve/reject policy. -The repository uses focused package tests plus a fixture-driven CLI workflow. +## Files And Run State -- CLI acceptance tests cover maintained examples under `examples/`. -- Pipeline tests cover registry composition and end-to-end framework behavior - with fakes. -- Module tests cover implemented module contracts without requiring real - provider calls. -- LLM tests use local test servers and fakes. +Notarius keeps distinct output and inspection surfaces: -Do not use real external services in tests. Use fakes, fixtures, or local test -servers. +| Surface | Owner | Purpose | +| --- | --- | --- | +| Durable output | Output module and CLI writer | User-consumable run files. | +| Diagnostics | `internal/core/diagnostics` and CLI | Redacted run inspection, reports, warnings, and failures. | +| Checkpoints | `internal/framework/checkpoint` | Validated stage reuse for explicit resume. | +| Debug artifacts | `internal/framework/debug` and pipeline instrumentation | Sensitive framework-boundary and LLM call inspection. | -## Boundary Reminders +Workspace settings determine whether and where diagnostics, checkpoints, and +debug artifacts are written. Concrete stage modules do not receive workspace +paths. -- Source-format details stay in input modules and integration docs. -- Extraction-domain details stay in extract modules and artifact docs. -- Generic shared prompt plumbing stays in `internal/modules/sharedassets`; - domain-specific shared prompt behavior stays with the relevant module helper - package. -- Provider wire details stay in the LLM runtime and provider integration docs. -- Durable output contracts belong in integration docs. -- Operator procedures belong in `docs/operations.md`, not internal docs. +## Focused Internal Documentation + +- [Pipeline Internals](pipeline.md): resolution, execution, validators, + references, retries, checkpoints, outputs, and manifests. +- [Module Internals](modules.md): production module contracts, capabilities, + options, prompts, schemas, and registration. +- [LLM Runtime](llm.md): structured completion contracts, Scriptorium adapter, + assets, scheduling, profile recording, and redaction. +- [Diagnostics Internals](diagnostics.md): diagnostics files, retention, + failure behavior, and path safety. + +## Test Surfaces + +The repository uses focused package tests, registry and pipeline composition +tests, a fake-backed walking skeleton, fixture-driven CLI coverage, and local +test servers for LLM integration behavior. Tests do not require real provider +calls. diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md index 0e7cb83..d55a09a 100644 --- a/docs/policy/architecture.md +++ b/docs/policy/architecture.md @@ -1,216 +1,179 @@ # Architecture -This document defines Notarius development policy. It is inward-facing: -developers and LLM coding agents should use it to preserve the project's shape, -boundaries, and invariants as the code evolves. +This document defines the intended high-level architecture of Notarius and the +invariants that changes must preserve. Implemented component details belong in +[Internal Overview](../internal/overview.md) and its linked documents. The +reasoning behind significant architectural choices belongs in +[ADRs](../adr/). -Keep this document concise. It should describe durable architectural rules, not -CLI syntax, configuration reference material, module catalogs, or roadmap items. +## System Shape -## Project Shape +Notarius is a small, dependency-light Go application for extracting structured +artifacts from source material. It is a general extraction platform whose +source formats, extraction domains, validation policies, LLM providers, and +output formats are isolated behind explicit boundaries. -Notarius is a small, explicit, dependency-light Go application for extracting -structured artifacts from source material using modular pipeline stages. - -The application is contract-first but not abstraction-heavy. Add interfaces and -extension points when they protect a real boundary: - -- external source formats; -- pipeline stage modules; -- validators; -- LLM providers and runtime plumbing; -- output schemas and embedded assets. - -Avoid abstractions that only anticipate hypothetical complexity. Prefer narrow -contracts that can be exercised by tests and real modules. - -## Core Invariants - -The framework must remain source-agnostic and domain-agnostic. - -Source-format details belong in input modules. Transcript-specific concepts such -as segments, speakers, timestamps, and transcript schemas must not spread into -runner, extractor, validator, or LLM framework code. - -Extraction-domain details belong in domain modules. D&D-specific concepts such -as spells, NPCs, items, combat turns, and encounters must not spread into core -source, runner, or LLM framework packages. - -Extracted facts should be grounded with source references. Source references -should point to generic source units, not transcript-only structures. Framework -code should preserve source-reference ranges exactly and should not merge or -rewrite overlapping ranges unless a module explicitly owns that behavior. - -The application workflow is fixed: +The application has one fixed pipeline shape: ```text input -> chunk -> extract -> merge -> normalize -> output ``` -These stages should remain explicit in the architecture. Chunking, merging, and -normalization must not be hidden inside domain extractors when they represent -general pipeline behavior. +Pipelines are configured compositions of this shape. They are not arbitrary +DAGs or a general workflow language. Every stage remains explicit; general +chunking, merging, or normalization behavior must not be hidden inside an +extractor. -Pipelines are fixed-shape templates for this workflow, not arbitrary DAGs or a -general workflow language. Module selection should be configuration- and -registry-driven, not scattered through conditionals. +Input and chunking are pipeline-wide. Each selected artifact lane owns its +extract, merge, and normalize stages, and the output stage aggregates the run's +lane outcomes. -## Package Boundaries +Notarius is contract-first without being abstraction-heavy. Interfaces and +extension points should protect demonstrated boundaries. New abstraction is not +itself an architectural goal. -Prefer fewer, larger framework packages until a boundary proves itself through -import direction, ownership, test seams, or substantial file size. +## Package Layout And Dependency Direction -Core packages should contain deterministic models and policy. Framework -packages should contain reusable orchestration and provider plumbing. Concrete -business logic should live under stage-oriented module packages: +| Area | Ownership | +| --- | --- | +| `cmd/notarius` | Executable entry point; delegates to the CLI. | +| `internal/cli` | Application boundary, production composition, runtime setup, durable writes, and user-facing results. | +| `internal/core` | Generic deterministic models and policy for source material, configuration, manifests, diagnostics, and workspace identity. | +| `internal/framework` | Reusable contracts, registries, pipeline orchestration, validation mechanics, checkpoints, debug boundaries, and LLM runtime plumbing. | +| `internal/modules` | Concrete pipeline stage behavior. | +| `internal/validators` | Concrete approve/reject policies. | -```text -internal/modules/input/... -internal/modules/chunk/... -internal/modules/extract/... -internal/modules/merge/... -internal/modules/normalize/... -internal/modules/output/... -``` +The CLI is the composition root and may import concrete implementations. Core +and framework packages cooperate as generic application layers; neither may +depend on production modules or validators. Concrete implementations may depend +on core models and framework contracts. -Use short, lowercase, idiomatic Go package names. Avoid package names that repeat -parent-stage context. +The following dependency boundaries are mandatory: -Input modules translate external source formats into the core source model. -They may know about external schema details, source-specific metadata, and -format-specific validation rules. They should not own extraction-domain -decisions. +- extractors and validators do not depend on concrete input adapters; +- provider-specific types do not cross the LLM runtime boundary; +- external dependency types do not leak across internal package boundaries + unless that dependency is the package's explicit contract. -Extract modules own artifact semantics, prompt usage, structured response schema -selection, and domain-specific interpretation. They should depend on framework -contracts and core source/artifact types, not concrete input module packages. -Production validation defaults are central catalog policy, not behavior owned by -module packages. +Shared helpers may support demonstrated common needs, but must not move +source-format or extraction-domain knowledge into generic framework packages. +External dependencies require a clear correctness, security, interoperability, +or complexity benefit. -Merge modules combine extracted candidates. Normalize modules reconcile merged -candidates for semantic consistency. Generic behavior may exist for simple -artifact types, but domain-specific behavior belongs in modules for the relevant -stage. +## Source And Domain Boundaries -Output modules serialize final artifacts and may report warnings out of band. -CLI, diagnostics, and reporting layers are responsible for surfacing those -warnings. +Input modules translate external source formats into the generic source model. +Format-specific schemas, fields, and validation remain with the input module +and its integration contract. + +Framework stages operate on source documents, source units, and source +references rather than format-specific structures. A source reference identifies +an ordered range of generic source units. Framework code preserves those ranges +and does not merge or rewrite them unless a stage module explicitly owns that +behavior. + +Extract modules own artifact semantics, prompt use, response schemas, and +domain interpretation. Domain-specific concepts remain in the relevant module, +validator, shared domain helper, and artifact contract. + +Auxiliary references provide context or disambiguation. They are not source +evidence and must not be converted into source references. + +## Pipeline Composition And Ownership + +Module selection is configuration- and registry-driven. The framework resolves +named pipeline definitions, applies explicit defaults and runtime overrides, +and verifies module availability and capabilities before execution. Structural +pipeline choices must not be scattered through conditionals or hidden behind +ad hoc command flags. + +Stage ownership is explicit: + +- input modules convert external material into the generic source model; +- chunk modules partition source material for extraction; +- extract modules produce domain artifacts from chunks; +- merge modules combine accepted extraction outputs; +- normalize modules reconcile merged output; +- output modules encode accepted results and run outcomes into logical files. + +The framework owns orchestration and handoff provenance. Modules return logical +results and warnings; they do not own CLI reporting, workspace paths, durable +file placement, checkpoints, or diagnostics. ## Validation -Validators should be independently testable and composable. +Validation is a framework-managed boundary around raw outputs from chunk, +extract, merge, and normalize stages. Validators receive immutable stage output +and make an explicit whole-output decision: approve, approve with warnings, or +reject. -Validators evaluate immutable module outputs returned by `chunk`, `extract`, -`merge`, and `normalize` stages. Validator decision semantics should be -explicit: each validator call approves, rejects, or approves with warnings for -the whole module output it receives. Validator rejection records rejected raw -output; validator execution errors are framework errors. +Rejection is a recorded pipeline outcome, not a framework execution error. +Validator execution failures are framework errors. Rejected output does not +advance to the next stage. -Default validator chains belong in central production catalog mappings keyed by -stage and module key. Pipeline configuration may override those mappings at the -stage-local module binding. Empty chains are valid and approve by default. +Default validator chains are production composition policy and are registered +centrally by stage and module. Configuration may replace a stage-local default, +including with an explicitly empty chain. Configured validator order is +authoritative; the framework must not silently reorder it. -Deterministic validators should run before LLM-backed validators in production -defaults when both are present. Configured validator order is authoritative and -must not be silently reordered. +## LLM Boundary -Shared validator runtime mechanics belong in framework code. Concrete validator -behavior belongs in module or validator implementation packages. +Modules and validators use transport-neutral structured completion contracts. +Provider request and response types, authentication, transport behavior, and +provider error adaptation remain inside the LLM runtime. -## LLM Runtime +The caller of the LLM owns prompt selection, prompt inputs, response schema, +and interpretation of structured output. Provider adapters do not own source- +or domain-specific prompt logic. -LLM provider details belong behind transport-neutral framework contracts. +LLM calls and other external operations accept cancellation and respect +timeouts. Concurrency control belongs in shared runtime plumbing rather than in +individual modules. -Provider-specific HTTP request and response types should stay inside the LLM -runtime package. Prompt construction should stay in extractors, validators, or -shared prompt helpers; provider adapters should not own domain prompt logic. +## Configuration And Provenance -Errors, diagnostics, reports, manifests, and redacted configuration must not -expose secrets. +Configuration loading, precedence, defaults, environment overrides, redaction, +and validation are centralized. Named pipeline definitions make structural +composition explicit and discoverable. Operational overrides are permitted +when they do not obscure the configured pipeline structure. -## Configuration +Run preparation fails before stage execution when statically discoverable +modules, capabilities, reference bindings, or explicitly selected profiles are +invalid or incompatible. -Configuration should make pipeline composition explicit and discoverable. +Run manifests record enough resolved pipeline, module, source, reference, and +LLM provenance to make a run auditable after configuration changes. Manifests +record identities and summaries rather than secret or large payload content. -Centralize configuration loading, precedence, defaults, and validation. Structural -pipeline choices should come from named pipeline definitions, not ad hoc command -flags. Operational overrides may be handled separately when they do not obscure -the configured pipeline structure. +## State, Output, And Safety -Module registries should expose module metadata and capabilities without -requiring module construction. Configuration validation should fail fast when a -pipeline binds incompatible or unknown modules. +Durable output, diagnostics, checkpoints, and debug artifacts are separate +surfaces with separate ownership: -Run manifests should record enough resolved pipeline provenance to make a run -auditable after named configuration changes over time. +- output modules define logical durable output; the application boundary owns + filesystem placement; +- diagnostics provide redacted run inspection and are not the durable output + contract; +- checkpoints support validated stage reuse and are not diagnostics; +- debug artifacts are opt-in inspection data and may contain sensitive source, + prompt, reference, and model-output content. -## Dependencies +Writes of durable state are atomic where practical. Paths for writes, moves, +overwrites, and deletion must be narrow and explicit. Cleanup that can lose data +is opt-in. -Prefer the Go standard library where practical. +Secrets must not appear in errors, logs, diagnostics, manifests, +documentation, examples, or redacted configuration. Default logs and +diagnostics must not include large source, prompt, reference, or artifact +payloads. -Use external dependencies only when justified by correctness, security, -interoperability, or substantial complexity reduction. Good reasons include -widely used file formats, complex validation behavior, or secure transport -handling. +## Architectural Non-Goals -Avoid dependencies for small conveniences. Do not let external dependency types -leak across internal package boundaries unless the dependency is itself the -explicit contract of that package. +Notarius does not aim to provide: -## State, Files, and Safety - -If the application writes durable state, writes should be atomic where -practical. Multi-step workflows should preserve enough diagnostics to support -inspection after failure. - -Code that deletes, moves, or overwrites files must use narrow, explicit paths. -Avoid broad parent-directory operations. Cleanup that can cause data loss must -be opt-in. - -## Errors and Logging - -Errors should be actionable and preserve context. Wrap errors with operation and -path or resource context. CLI code should convert internal errors into concise -user-facing messages. - -Errors and logs must not expose secrets. Logs should describe operations, -external calls, retries, and failure causes, but should not include large source -or artifact payloads by default. - -Long-running operations should accept `context.Context`. External calls, -subprocesses, HTTP requests, storage operations, LLM calls, and multi-stage -workflows should respect cancellation and timeouts. - -## Testing - -Core logic should be testable without real external services. Use fakes, -fixtures, or local test doubles for input modules, extract modules, validators, -and LLM clients where practical. - -Contract-first work should include fake implementations that prove interfaces -compose before real modules depend on them. - -Maintain a fixture-driven walking skeleton that exercises the full pipeline with -fake modules and fake external clients. This protects stage composition as real -modules evolve. - -Important CLI and configuration workflows should have tests. Adapter, extractor, -validator, and stage contracts should have focused tests that do not require -running the full application unless end-to-end coverage is intentional. - -## Documentation - -Documentation should follow the project documentation policy. Keep user docs -focused on implemented behavior. Put future, planned, or aspirational work only -under `docs/roadmap/`. - -Core documentation should use generic terms such as source document, source -unit, source reference, input adapter, extractor, chunker, merger, normalizer, -artifact, validator, and run manifest. - -Source-format details belong in input module or integration docs. -Domain-specific extraction details belong in extract module or artifact docs. - -When changing architecture, config, CLI behavior, stage modules, extractor -contracts, validator contracts, LLM runtime behavior, or artifact schemas, update -the relevant docs and examples in the same change. +- an arbitrary workflow graph or general workflow language; +- source-format or extraction-domain behavior in generic framework packages; +- provider-specific contracts exposed to modules; +- structural pipeline composition through ad hoc CLI flags; +- implicit cross-stage behavior that bypasses the fixed pipeline; +- abstractions introduced solely for hypothetical future complexity. diff --git a/docs/policy/development.md b/docs/policy/development.md deleted file mode 100644 index fffa23a..0000000 --- a/docs/policy/development.md +++ /dev/null @@ -1,156 +0,0 @@ -# Development - -This document defines contributor workflow for Notarius. For architectural -invariants and package boundaries, read [Architecture](architecture.md) first. - -## Required Reading - -Before changing the repository, review: - -- [Architecture](architecture.md) -- [Documentation Policy](documentation.md) - -Keep current-behavior documentation limited to implemented behavior. Put planned -or deferred behavior under `docs/roadmap/`. - -## Repository Layout - -- `cmd/notarius`: executable entry point. -- `internal/cli`: CLI parsing, production catalog wiring, config loading, run - command orchestration, output writes, and user-facing errors. -- `internal/core`: deterministic models and policy for artifacts, source - documents, config, and diagnostics. -- `internal/framework`: reusable contracts, pipeline orchestration, prompt - helpers, validation helpers, and LLM runtime plumbing. -- `internal/modules`: concrete input, chunk, extract, merge, normalize, and - output modules. -- `docs`: policy, user/operator docs, internal docs, integration docs, and - roadmap files. -- `examples`: maintained, secret-free examples covered by tests where practical. - -## Validation Commands - -Run focused tests for the area changed, then run the broader checks when the -change affects shared contracts, CLI behavior, or documentation examples. - -```sh -go test ./... -go vet ./... -go build ./cmd/notarius -``` - -Useful focused checks: - -```sh -go test ./internal/cli -go test ./internal/core/config -go test ./internal/framework/pipeline -go test ./internal/framework/llm -go test ./internal/modules/input/seriatim -go test ./internal/modules/extract/dnd/spells -go test ./internal/modules/output/json -``` - -## Prompt Asset Tests - -Tests should not assert the exact text of embedded prompt assets for production -modules. Prompt wording is expected to change frequently during development, and -tests should not fail solely because prompt prose was edited. - -Prefer assertions against prompt structure, prompt IDs and versions, declared -inputs, schema wiring, input propagation, diagnostics redaction, and successful -prompt preparation. - -Narrow exceptions: - -- generic or test-only modules may use fixed prompt text assertions when the - text is part of the test surface; -- test fixtures may supply their own prompt text and assert against that fixture - text. - -## Go Conventions - -- Prefer the standard library unless a dependency is justified by correctness, - security, interoperability, or substantial complexity reduction. -- Keep package names short, lowercase, and idiomatic. -- Preserve import direction: framework and core code must not depend on concrete - production modules. -- Use `context.Context` for long-running operations and external calls. -- Return contextual errors that name the operation and relevant module, path, or - resource. -- Do not include secrets in errors, logs, diagnostics, manifests, or docs. - -## Adding Config Fields - -Config behavior is centralized under `internal/core/config`. - -When adding a file config field: - -1. Update file config structs and YAML parsing in `file_config.go`. -2. Apply the field over defaults in config application code. -3. Add validation in `validation.go` when the field has constraints. -4. Add environment override support in `env.go` only for operational overrides. -5. Update redaction if the field can contain secrets. -6. Add focused config tests. -7. Update [Configuration](../config.md) and maintained examples when behavior - changes. - -Pipeline composition should remain config-driven. Do not add command flags that -silently replace structural pipeline definitions. - -## Adding CLI Flags Or Commands - -CLI behavior lives in `internal/cli`. - -When adding CLI surface: - -1. Keep syntax explicit and update usage text. -2. Validate arguments before running expensive work. -3. Convert internal errors into concise user-facing messages. -4. Add CLI tests for success, syntax errors, and failure modes. -5. Update [CLI Reference](../cli.md), and update - [Operations](../operations.md) or [Troubleshooting](../troubleshooting.md) - if run behavior changes. - -## Adding Modules Or Adapters - -Concrete modules live under `internal/modules//...` and implement the -interfaces in `internal/framework/contracts`. - -For a new production module: - -1. Implement the relevant contract. -2. Expose a `ModuleSpec` with the correct module key, module kind, provided - capabilities, and required capabilities. -3. Expose a `Register` function that registers the module with its registry. -4. Add focused module tests for contract behavior, registration, options, - validation, and errors. -5. Register the module in `internal/cli/catalog.go` only when it is production - ready. -6. Update internal docs and user-facing docs only for implemented behavior. - -Source-format behavior belongs in input modules and integration docs. -Extraction-domain behavior belongs in extract modules and artifact docs. - -## Updating Examples - -Examples must be valid, secret-free, and small. - -- Prefer environment-based secret configuration. -- Keep `examples/dnd-spells.config.yml` loadable by CLI tests. -- Keep `examples/seriatim-minimal-transcript.json` compatible with the Seriatim - adapter. -- Do not add expected-output fixtures unless they are validated or have a clear - regeneration procedure. - -## Documentation Updates - -Update docs in the same change when behavior changes. - -- CLI syntax: `docs/cli.md` -- Config fields and defaults: `docs/config.md` -- Output, diagnostics, retention, or recovery: `docs/operations.md` -- Common user-facing failures: `docs/troubleshooting.md` -- Internal architecture and contracts: `docs/internal/` -- External file formats and durable integration contracts: `docs/integrations/` -- Future or planned work only: `docs/roadmap/` diff --git a/docs/policy/documentation.md b/docs/policy/documentation.md index 5bf2683..467871c 100644 --- a/docs/policy/documentation.md +++ b/docs/policy/documentation.md @@ -1,446 +1,143 @@ -# Go Project Documentation Policy +# Documentation Policy ## Purpose -Project documentation must help five audiences: - -1. users who need to run the application; -2. administrators/operators who need to configure and operate it; -3. developers who need to understand and change it safely; -4. LLM coding agents that need clear scope, boundaries, and invariants; -5. developers and LLM coding agents integrating this project from another codebase. - -Docs should be accurate, concise, task-oriented, and organized by audience. Prefer links to canonical docs over repetition. +This policy assigns each documentation topic to one canonical owner. Its goal is +to keep Notarius documentation accurate, concise, discoverable, and resistant +to drift for users, operators, developers, integrators, and LLM coding agents. ## Core Rules -### 1. Keep docs concise +### One Canonical Owner -Each document should cover a defined scope and only the essentials for that scope. +Each authoritative fact belongs in one document. A non-owning document may give +a short, stable summary for orientation, but it must link to the canonical owner +instead of repeating volatile details. -Avoid: -- long background explanations; -- repeated reference material; -- implementation detail in user-facing docs; -- aspirational language outside roadmap docs; -- verbose examples where one minimal example is clearer. +Volatile details include commands, flags, configuration fields and defaults, +module keys, schemas, file names, paths, status codes, retry behavior, and +runtime guarantees. If readers could reasonably treat a statement as a +contract, maintain it only in the owning document. -### 2. Document only implemented behavior outside roadmap files +### Current And Future Behavior -Unimplemented, planned, aspirational, experimental, or future work may be described only under: +Outside `docs/roadmap/`, documentation describes implemented behavior only. +Partial features may be described only to their implemented boundary. + +ADRs are the narrow exception: an ADR may record an accepted architectural +decision before implementation, but acceptance must not be presented as proof +that the behavior exists. The roadmap owns implementation status and sequencing +until the decision is implemented. Current architecture, user, operator, +integration, and internal documentation are updated when the behavior lands. -- `docs/roadmap/` +### Audience And Detail -No other documentation file, including `README.md`, should describe code, features, modules, stages, commands, config fields, or behaviors that do not currently exist. +Write for the document's stated audience and include only the detail needed for +its owned topic. User and operator docs should not expose implementation detail. +Developer docs should link to user-facing and external contracts rather than +restate them. -If a feature is partial, non-roadmap docs may describe only the implemented portion and its current boundary. +### Examples -### 3. Use canonical homes - -Each type of information should have one canonical location. - -Canonical homes: - -- project purpose and quickstart: `README.md` -- development principles: `docs/policy/architecture.md` -- public HTTP API reference: `docs/api.md` -- configuration reference: `docs/config.md` -- CLI reference: `docs/cli.md` -- operations and recovery: `docs/operations.md` -- troubleshooting: `docs/troubleshooting.md` -- public API/package consumer guidance: `docs/consumers/` -- implemented internals: `docs/internal/` -- external protocol, service, and file-format contracts: `docs/integrations/` -- future work: `docs/roadmap/` -- contributor workflow: `docs/policy/development.md` -- copyable examples: `examples/` - -Other files should summarize briefly and link to the canonical source. - -### 4. Keep examples real - -Examples should be valid, maintained, and free of secrets. - -Where practical: -- example configs should load successfully; -- example commands should match real CLI syntax; -- important examples should be covered by tests. - -## Documentation Profiles - -All projects require: - -- `README.md` -- `docs/policy/architecture.md` - -Additional docs depend on the project. - -### Small library - -Recommended: -- `docs/policy/development.md`, if contributor conventions are non-obvious - -### Simple CLI - -Required: -- `docs/cli.md` - -Recommended: -- `docs/policy/development.md` - -### Config-driven CLI - -Required: -- `docs/cli.md` -- `docs/config.md` - -Recommended: -- `examples/` -- `docs/policy/development.md` - -### Stateful or operator-facing application - -Required: -- `docs/cli.md`, if CLI-based -- `docs/config.md`, if config-driven -- `docs/operations.md` - -Recommended: -- `docs/troubleshooting.md` -- `examples/` -- `docs/policy/development.md` - -### Modular, service-oriented, or orchestration application - -Required: -- `docs/cli.md`, if CLI-based -- `docs/config.md`, if config-driven -- `docs/operations.md` -- `docs/internal/` -- `docs/policy/development.md` - -Recommended: -- `docs/troubleshooting.md` -- validated examples under `examples/` - -### Public HTTP API service - -Required: -- `docs/api.md` -- `docs/cli.md`, if CLI-based -- `docs/config.md`, if config-driven -- `docs/operations.md` -- `docs/internal/` -- `docs/policy/development.md` - -Recommended: -- `docs/troubleshooting.md` -- `docs/consumers/`, for task-oriented client integration guides -- `docs/integrations/`, for upstream/downstream service contracts -- validated examples under `examples/` - -### Project with public packages or consumer APIs - -Required: -- `docs/consumers/api.md` -- one `docs/consumers/pkg-.md` file per public package, if public packages exist - -Recommended: -- copyable consumer examples under `examples/`, if practical - -## Required Documents - -### README.md - -**Audience:** users, administrators, operators - -The README is the outward-facing project orientation page. - -It should include, in order: - -1. concise description; -2. elevator pitch; -3. shortest useful command or usage example; -4. links to targeted docs. - -The README should be short. It is not a manual. - -The “shortest useful command” means the simplest command that performs the project’s core use case. (It does not mean `app --help`.) - -### docs/policy/architecture.md - -**Audience:** developers, LLM coding agents - -`docs/policy/architecture.md` is required for every project. - -It is an inward-facing development policy document. It should describe how the project is intended to be built and changed. - -It should include: - -- project shape; -- core design principles; -- package and boundary philosophy; -- state/persistence philosophy, if applicable; -- external integration philosophy, if applicable; -- error-handling and logging principles; -- testing expectations; -- documentation expectations; -- architectural invariants; -- explicit non-goals, if useful. - -Notably, this file should prescribe a core development *policy* that should remain unchanged as the application evolves. It is not a place for details (e.g., CLI flags) that could change over time. - -The contents of `architecture.md` should be trim and concise. LLMs may be directed to review it routinely via AGENTS.md, CLAUDE.md, or similar. - -### docs/api.md - -**Audience:** external HTTP API consumers, developers, LLM coding agents integrating by HTTP - -Required for projects whose primary public interface is HTTP. - -`docs/api.md` is the canonical public HTTP API contract. It should be normative for external consumers and should not be duplicated by README, operations docs, consumer guides, or integration docs. - -It should include: - -1. base URL conventions; -2. authentication and authorization behavior, if implemented; -3. response envelope; -4. supported media types and content negotiation behavior; -5. shared query parameters; -6. endpoint reference grouped by route family; -7. request parameters and validation rules; -8. response fields, units, nullability, and optionality; -9. error response shape and status codes; -10. pagination, caching, rate-limit, idempotency, and retry behavior, if implemented; -11. compact request and response examples. - -It must document only implemented endpoints and behavior. Planned endpoints, proposed fields, future filters, and experimental response shapes belong only under `docs/roadmap/`. - -For HTTP API projects, `docs/consumers/` may provide task-oriented client integration guides, but those guides should link to `docs/api.md` for the authoritative endpoint contract. - -### docs/policy/development.md - -**Audience:** developers, LLM coding agents - -Required for projects maintained by humans and LLM coding agents. - -It should include: - -- repository layout; -- build/test commands; -- coding conventions; -- dependency policy; -- how to add config fields; -- how to add CLI flags; -- how to add modules or adapters, if applicable; -- how to update examples; -- documentation update expectations. - -### docs/config.md - -**Audience:** administrators, operators, advanced users - -Required for applications with configuration files. - -It should include, in order: - -1. config file locations and discovery precedence; -2. minimal working config; -3. production-oriented config; -4. full configuration reference; -5. secrets handling, if applicable; -6. links to maintained examples. - -The full configuration reference should be canonical. - -### docs/cli.md - -**Audience:** users, administrators, operators - -Required for CLI applications. - -It should include, in order: - -1. shortest useful command; -2. command overview; -3. complete flag reference; -4. common workflows; -5. diagnostic or recovery commands, if applicable. - -Explain when commands are useful, not just their syntax. - -### docs/operations.md - -**Audience:** administrators, operators - -Required for applications that maintain state, support resume behavior, run multi-step workflows, write durable artifacts, use remote storage, or require recovery procedures. - -It should cover: - -- normal workflow; -- filesystem layout; -- remote storage layout, if applicable; -- logs and manifests; -- resume/retry behavior; -- cleanup behavior; -- archive/backup behavior; -- safe recovery procedures; -- operational caveats. - -### docs/troubleshooting.md - -**Audience:** administrators, operators - -Recommended once recurring failure modes exist. - -Each entry should include: - -- symptom; -- likely cause; -- diagnostic command or inspection step; -- safe fix; -- relevant links. - -### docs/consumers/ - -**Audience:** developers and LLM coding agents integrating this project from another codebase - -Required for projects with public packages, SDKs, client APIs, plugin APIs, or other application-facing integration surfaces. - -This directory describes how an external codebase should consume the project's public API. It should be task-oriented and copyable where useful. It is not the place for internal implementation details or operator procedures. - -For projects whose public API is HTTP, `docs/consumers/` is not required, and it should not duplicate the endpoint reference in `docs/api.md`. If present, it may provide practical integration workflows, client-specific examples, or migration notes that link back to `docs/api.md`. - -`docs/consumers/api.md` should provide the consumer-facing overview and primary implementation workflow. It should include: - -1. intended consumer audience and use cases; -2. required inputs supplied by operators or deployment configuration; -3. recommended public package or API workflow; -4. minimal copyable example; -5. consumer responsibilities and boundaries; -6. retry, idempotency, or status behavior, if applicable; -7. links to package-specific docs and canonical integration contracts. - -Package-specific docs should be named `pkg-.md` and should include: - -1. import path; -2. intended use cases; -3. primary types and functions needed by consumers; -4. minimal examples; -5. validation, error, retry, and boundary behavior; -6. links to canonical file-format or wire-protocol contracts. - -### docs/internal/ - -**Audience:** developers, LLM coding agents - -Required for modular, service-oriented, or orchestration projects. - -This directory describes implemented internal components. It is not the roadmap. - -Use one file per major component where useful. - -Each component doc should include: - -1. purpose; -2. inputs and outputs; -3. boundaries; -4. config fields used; -5. external adapters used; -6. state or manifest behavior, if applicable; -7. skip/resume behavior, if applicable; -8. failure behavior; -9. tests to inspect before changing; -10. architectural invariants. - -### docs/roadmap/ - -**Audience:** maintainers, developers, LLM coding agents - -This is the only place for planned, future, aspirational, experimental, or unimplemented work. - -Roadmap docs should clearly distinguish: - -- proposed work; -- accepted plans; -- deferred ideas; -- rejected ideas; -- implementation prompts or task breakdowns, if useful. - -Roadmap docs should not be confused with current behavior. - -### docs/integrations/ - -**Audience:** developers, LLM coding agents - -Required for projects that depend on external CLIs, APIs, services, protocols, or file formats where the integration contract is important to maintain. - -This directory contains concise, versioned reference notes for external integration contracts. It should document only the parts of the external system that this project actually uses or exposes. - -For public HTTP API services, `docs/integrations/` should document upstream, downstream, storage, protocol, or runtime contracts that the service depends on or bridges. It should not become a second copy of the public HTTP endpoint reference; that belongs in `docs/api.md`. - -Use one file per integration where useful. - -## Examples Directory - -Projects with non-trivial configuration or workflows should include `examples/`. - -Useful examples include: - -- minimal working config; -- production-oriented config; -- full annotated config; -- local development config; -- remote/object-storage config; -- minimal session/input file. - -Examples should be valid, maintained, tested when practical, and linked from relevant docs. - -## Security and Privacy - -Docs and examples must not include: - -- real API keys; -- tokens; -- passwords; -- private keys; -- private environment dumps; -- sensitive user data; -- raw private transcripts; -- private infrastructure details unless intentionally public. - -Document secret-handling mechanisms, not actual secret values. - -## Maintenance Rules - -When docs change, verify the affected behavior. - -Where practical: - -- load example config files in tests; -- test CLI examples or command parser behavior; -- validate documented flags against real flags; -- remove stale references; -- update links after renames; -- keep roadmap content out of non-roadmap docs. - -If documentation and code disagree, fix the documentation and/or open a roadmap item; do not leave aspirational behavior in current-behavior docs. - -Documentation is complete only when it matches the current code. - -## Documentation Change Checklist - -Before merging documentation changes, verify: - -- README is concise and orientation-focused. -- `docs/policy/architecture.md` describes development principles. -- `docs/api.md` is the canonical HTTP contract for HTTP API services. -- Future work appears only under `docs/roadmap/`. -- User-facing docs avoid unnecessary internals. -- Consumer-facing docs explain public APIs without duplicating HTTP endpoint or integration contracts. -- Developer-facing docs preserve boundaries and invariants. -- Config examples match the schema. -- CLI examples match real commands and flags. -- Defaults appear in the canonical config reference. -- No secrets or private data are included. -- Links are accurate. +Complete copyable files belong in `examples/`. Documentation may use the +smallest illustrative snippet needed to explain its owned topic, but should link +to maintained examples instead of embedding a second complete copy. + +Examples must be valid, secret-free, and tested where practical. Commands and +configuration used in documentation should match the application. + +### Security And Privacy + +Documentation and examples must not contain real credentials, private keys, +private environment dumps, sensitive source material, or private infrastructure +details unless intentionally public. Document secret-handling mechanisms, not +secret values. + +## Canonical Ownership + +| Topic | Canonical owner | Owned content | Content owned elsewhere | +| --- | --- | --- | --- | +| Product orientation and minimal end-to-end quickstart | `README.md` | What Notarius is, why it is useful, one shortest successful invocation, and links onward. | Complete command reference, configuration reference, operational procedures, implementation detail. | +| Contributor entry point | `docs/development.md` | Task-oriented reading guide, minimal contributor orientation, baseline validation commands, and links to canonical docs. | Package inventory, architecture rules, subsystem behavior, detailed change recipes. | +| Current application architecture | `docs/policy/architecture.md` | System shape, normative ownership, dependency direction, architectural boundaries, invariants, safety properties, and non-goals. | Concrete package inventory, implementation mechanics, contributor procedures, decision history, future work. | +| Documentation organization | `docs/policy/documentation.md` | Documentation ownership, audience boundaries, maintenance rules, and ADR/document lifecycle. | Application architecture or product behavior. | +| CLI contract | `docs/cli.md` | Commands, arguments, flags, invocation semantics, and exit codes. | End-to-end operating procedures, configuration field definitions, runtime filesystem layout, module implementation details. | +| Configuration contract | `docs/config.md` | Discovery and precedence, file schema, fields, defaults, environment overrides, validation rules, and user-selectable module or validator keys. | Complete example files, CLI syntax, runtime state lifecycle, module implementation details. | +| Operations | `docs/operations.md` | Runtime workflows, physical filesystem and state layout, diagnostics use, retention, resume, cleanup, permissions, recovery, and operational limits. | CLI flag syntax, configuration field definitions, logical output schemas, implementation mechanics. | +| Public HTTP contract, if introduced | `docs/api.md` | Routes, authentication, media types, request and response schemas, status codes, pagination, caching, idempotency, rate limits, and HTTP retry semantics. | Client walkthroughs, upstream or downstream integration internals, implementation detail. | +| Consumer guidance, if a public package or API is introduced | `docs/consumers/` | Task-oriented use of the public interface, minimal client examples, and consumer responsibilities. | HTTP wire semantics, external protocol contracts, internal implementation detail. | +| External and durable integration contracts | `docs/integrations/` | External file formats and protocols, upstream and downstream contracts, logical output bundle paths and schemas, media types, and compatibility behavior. | Physical runtime placement and lifecycle, internal transformations, CLI syntax, configuration defaults. | +| Implemented component inventory | `docs/internal/overview.md` | Current packages and components, their implemented responsibilities, and links to focused internal docs. | Normative architecture, contributor reading policy, external contracts. | +| Internal component behavior | Other files under `docs/internal/` | Implementation flow, internal collaborators and state transitions, package-local guarantees and failures, and relevant tests. | Global architecture invariants, configuration definitions and defaults, external schemas, operator procedures. | +| Architectural decision history | `docs/adr/` | Significant decisions, context, alternatives, rationale, consequences, and supersession history. | Current behavior reference, implementation status, task sequencing. | +| Future work and implementation status | `docs/roadmap/` | Proposed, accepted, deferred, or rejected work; implementation status; sequencing; and task breakdowns. | Implemented behavior reference and architectural decision rationale. | +| Complete copyable artifacts | `examples/` | Maintained configuration, inputs, and other files intended to be copied or run. | Field-by-field reference, command reference, prose explanation. | + +Documents that do not exist are required only when the corresponding interface +or responsibility exists. Do not create placeholder API, consumer, integration, +or operations documents for behavior the application does not have. + +## Boundary Rules + +### Orientation + +The README owns product orientation. The developer guide routes contributors. +Architecture owns normative structure. Internal overview owns the current +concrete component map. These documents may link to one another but should not +maintain parallel package or behavior descriptions. + +### Commands, Configuration, And Operations + +CLI documentation answers how to invoke the application. Configuration +documentation answers what settings mean. Operations answers what happens to +runtime state and how to operate or recover the application. When a workflow +crosses these topics, choose the document that owns the task and link to the +other contracts. + +### Contracts And Implementation + +Integration and API documents define externally observable shapes and +semantics. Internal documents explain how Notarius implements or consumes those +contracts. Internal docs may name a field, file, or protocol to identify a +dependency, but must link to its canonical contract for the definition. + +### Security Topics + +This policy owns what documentation and examples may contain. Architecture owns +application security invariants. Configuration owns credential-supply +mechanisms. Operations owns permissions and handling of sensitive runtime +artifacts. Internal docs own implementation mechanisms only. + +## Architecture Decision Records + +Use sequentially numbered ADR filenames such as +`0001-record-architecture-decisions.md`. Follow the lightweight Nygard format: + +1. title; +2. status; +3. date; +4. context; +5. decision; +6. alternatives considered; +7. consequences. + +Treat the decision content of an accepted ADR as immutable. When a decision +changes, create a new ADR and update the earlier ADR's status to superseded. +Rejected architectural alternatives belong in the ADR; rejected product ideas +belong in the roadmap. + +## Maintenance + +When behavior changes, update its canonical owner in the same change. If +ownership moves, remove the old definition and replace it with a link where +navigation remains useful. + +Before completing documentation work: + +- verify affected behavior and examples; +- check commands, flags, fields, defaults, schemas, and paths against their + implementation; +- keep unimplemented behavior in the roadmap, subject to the ADR exception; +- remove stale references and validate links; +- confirm that non-owning documents summarize and link rather than redefine; +- confirm that no secrets or sensitive private data were added. diff --git a/docs/roadmap/documentation.md b/docs/roadmap/documentation.md new file mode 100644 index 0000000..e16083c --- /dev/null +++ b/docs/roadmap/documentation.md @@ -0,0 +1,130 @@ +# Documentation Policy Migration + +The documentation ownership policy has been revised to assign each topic to one +canonical document. This roadmap organizes the likely migration work into three +ordered passes. + +This is a planning inventory, not a review of the current documents. Each item +must be verified before content is moved, removed, or rewritten. Complete the +passes in order so canonical content is established before non-owning copies are +removed. + +## Pass 1: User And Operator Contracts + +Establish the canonical externally observable facts first. These documents will +become the targets linked from developer and internal documentation in the next +pass. + +### Product And CLI + +- Audit `README.md` so it owns product orientation and the single minimal + end-to-end quickstart without becoming a command or configuration reference. +- Audit `docs/cli.md` for material owned by the README, configuration reference, + or operations guide. +- Keep commands, arguments, flags, invocation semantics, and exit codes + canonical in `docs/cli.md`. + +### Configuration And Examples + +- Audit `docs/config.md` for complete example files, CLI syntax, runtime state + lifecycle, or implementation detail. +- Keep fields, defaults, precedence, environment overrides, validation, and + user-selectable module and validator keys canonical in `docs/config.md`. +- Audit minimal and production-oriented configuration examples, moving complete + copyable files under `examples/` and leaving only small illustrative fragments + in reference documentation. +- Inventory complete configuration, input, command, and output examples embedded + in prose documents. +- Select one maintained copy for each complete artifact under `examples/` where + practical, and verify important examples through automated tests. + +### Operations And Integrations + +- Audit `docs/operations.md` for duplicated CLI syntax, configuration field + definitions, logical output schemas, or implementation mechanics. +- Keep runtime workflows, physical state, retention, recovery, permissions, and + operational limits canonical in `docs/operations.md`. +- Audit `docs/integrations/` so external formats, protocols, logical output + paths, schemas, media types, and compatibility rules have one canonical home. +- Separate logical output bundle contracts from physical runtime placement and + lifecycle. + +### Pass 1 Completion + +- Confirm that README, CLI, configuration, operations, integrations, and + examples have non-overlapping ownership. +- Validate user-facing commands, fields, defaults, keys, schemas, paths, and + maintained examples against implemented behavior. +- Validate links among the user and operator documents. + +## Pass 2: Developer And Internal Documentation + +Use the canonical contracts established in Pass 1 to remove duplicated facts +from contributor and implementation documentation. + +### Orientation And Architecture + +- Audit `docs/development.md` so it routes contributors without maintaining a + parallel package inventory or architectural description. +- Audit `docs/internal/overview.md` so it owns the implemented component map + without restating normative architecture. +- Audit `docs/policy/architecture.md` so it contains current normative + architecture without implementation inventory, decision history, or future + behavior. + +### Internal Components + +- Audit internal component documents for repeated configuration definitions, + external input or output schemas, operator procedures, and global + architectural invariants. +- Replace duplicated external schemas or field definitions with links to the + canonical configuration or integration contracts from Pass 1. +- Keep implementation flow, internal collaborators, state transitions, + package-local guarantees, failures, and relevant tests in focused internal + documents. +- Identify task-specific contributor recipes that need a focused internal home + rather than the developer landing page. +- Retain production module and validator implementation details in module + internals while linking user-selectable keys to `docs/config.md`. + +### Pass 2 Completion + +- Confirm that development routes, architecture governs, internal overview + inventories, and focused internal docs explain implementation. +- Confirm that developer documents link to configuration, operations, and + integration contracts rather than redefining them. +- Validate developer-facing links and relevant focused tests. + +## Pass 3: Lifecycle And Final Deduplication + +Finish the migration by reconciling historical and future documentation, then +perform a repository-wide ownership audit. + +### ADR And Roadmap Lifecycle + +- Audit roadmap files for implemented behavior or completed status summaries + that should be removed or replaced with links to current canonical docs. +- Ensure ADRs own architectural rationale and supersession history without + becoming current behavior references or implementation trackers. +- Ensure accepted but unimplemented ADR decisions link to roadmap-owned + implementation status where appropriate. +- Distinguish rejected architectural alternatives in ADRs from rejected product + ideas in roadmap files. + +### Repository-Wide Audit + +- Inventory repeated commands, flags, defaults, module keys, validator keys, + file names, paths, schemas, retry semantics, and runtime guarantees. +- Assign each repeated fact to the canonical owner defined by the policy. +- Replace non-owning copies with short summaries and links where navigation is + useful. +- Retain only minimal illustrative snippets in prose and link them to maintained + examples. +- Validate local links and remove references to deleted or relocated material. + +### Pass 3 Completion + +- Confirm that every contractual or volatile fact has one canonical owner. +- Confirm that roadmap files contain future work and implementation status, ADRs + contain decision rationale, and current docs contain implemented behavior. +- Run final link, example, and documentation consistency checks. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 0c1a3a1..b4cd418 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -6,7 +6,6 @@ behavior is documented in the canonical current-behavior docs: - [Configuration](../config.md) - [CLI Reference](../cli.md) - [Operations](../operations.md) -- [Troubleshooting](../troubleshooting.md) - [Diagnostics Internals](../internal/diagnostics.md) - [Pipeline Internals](../internal/pipeline.md) diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md deleted file mode 100644 index cce7326..0000000 --- a/docs/troubleshooting.md +++ /dev/null @@ -1,423 +0,0 @@ -# Troubleshooting - -This guide maps common implemented failure modes to inspection steps and fixes. -For command syntax, see [CLI Reference](cli.md). For YAML fields and -environment overrides, see [Configuration](config.md). For output and -diagnostics layout, see [Operations](operations.md). - -## Config File Not Found - -Symptom: - -```text -notarius: config file not found; pass --config or set NOTARIUS_CONFIG -``` - -Fix: - -- Pass `--config path/to/config.yml`. -- Or set `NOTARIUS_CONFIG` to a readable file. -- Or install a config at `/usr/local/etc/notarius/config.yml`. - -If the message says the config path is a directory or is not available, correct -the path or file permissions. - -## Unsupported Or Invalid Config - -Symptoms include: - -- `unsupported config version` -- `config version is required` -- `field not found` -- `total LLM concurrency must be greater than zero` -- `diagnostics retention "" is not supported` - -Fix: - -- Use `version: 2`. -- Remove unknown YAML fields. -- Validate with: - -```sh -go run ./cmd/notarius config validate --config path/to/config.yml -``` - -## Unknown Pipeline - -Symptom: - -```text -notarius: pipeline "..." is not configured -``` - -Fix: - -- List configured pipeline IDs: - -```sh -go run ./cmd/notarius pipelines list --config path/to/config.yml -``` - -- Use one of those IDs in `notarius run `. -- Check indentation under the top-level `pipelines` map. - -## Unknown Or Incompatible Module - -Symptoms mention a module key, pipeline slot, lane, capability, or `not -registered`. - -Fix: - -- Validate the pipeline against the production module catalog: - -```sh -go run ./cmd/notarius config validate \ - --config path/to/config.yml \ - --pipeline dnd-session -``` - -- Use only implemented production module keys listed in - [Configuration](config.md#implemented-production-modules). -- Check that artifact lanes include an `extract` binding. - -## Invalid `--only` - -Symptoms include: - -- `--only must contain comma-separated non-empty artifact lane IDs` -- `--only requires --pipeline` -- `selected artifact lane` - -Fix: - -- Use comma-separated lane IDs with no empty entries: - -```sh -go run ./cmd/notarius run dnd-session \ - --config path/to/config.yml \ - --input path/to/input.json \ - --only spells -``` - -- For `config validate`, include `--pipeline` when using `--only`. -- Confirm the lane ID exists under `pipelines..artifacts`. - -## Reference Binding Failure - -Symptoms include: - -- `reference slot "..." is not declared` -- `reference slot "..." is declared by multiple selected targets` -- `required reference slot "..." is not bound` -- `--reference must use slot=path` -- `--without-reference must use a reference selector without =path` -- `read "...": no such file` -- `must be UTF-8 text` -- `media type "..." is not accepted` -- `is ... bytes, limit ...` - -Fix: - -- Confirm the selected chunker, extractor, merger, or normalizer declares the - slot. The implemented `dnd/scenes` chunker and `dnd/spells` extractor declare - optional `players`, `party`, and `glossary` slots, plus `roster` as a - deprecated compatibility alias for `party`. -- Use a specific selector when more than one selected target declares the same - slot: `chunk.context=./context.txt`, - `spells.extract.context=./extract-context.txt`, - `spells.merge.context=./merge-context.txt`, or - `spells.normalize.context=./normalize-context.txt`. -- `lane.slot=path` is valid only when exactly one selected extractor, merger, - or normalizer in that lane declares the slot. If more than one does, use - `lane.extract.slot=path`, `lane.merge.slot=path`, or - `lane.normalize.slot=path`. -- Use `--without-reference selector` to remove optional config bindings; do not - pass an empty `--reference selector=`. -- Check whether a path came from config or CLI. Config paths are relative to - the config file. CLI reference paths are relative to the current working - directory. -- Ensure the file is readable UTF-8 text and within any byte limit declared by - the declaring module. -- If the declaring module narrows accepted media types, use a file extension that - infers an accepted type such as `text/markdown`, `application/yaml`, or - `application/json`. - Unknown extensions infer `application/octet-stream`. -- If diagnostics are retained, inspect `resolved-pipeline.json`, - `resolved-references.json`, and `error.log`. - -## Seriatim Input Validation Failure - -Symptoms include `seriatim input`, `parse JSON`, `segments must not be empty`, -or validation errors naming a segment field. - -Fix: - -- Compare the input to - [examples/seriatim-minimal-transcript.json](../examples/seriatim-minimal-transcript.json). -- Ensure the JSON has a `metadata` object and a non-empty `segments` array. -- Each segment needs a non-empty `id`, non-empty `speaker`, non-empty `text`, - non-negative numeric `start`, and non-negative numeric `end`. -- Segment IDs must be unique and must not contain leading or trailing - whitespace. -- `end` must be greater than or equal to `start`. - -## Scriptorium Profile Source Failure - -Symptoms include: - -- `scriptorium profile_dir and profile_file are mutually exclusive` -- `scriptorium.profile_dir must not be empty when set` -- `scriptorium.profile_file must not be empty when set` -- `profile load` -- `profile not found` - -Fix: - -- Configure at most one of `scriptorium.profile_dir` or - `scriptorium.profile_file`. -- Confirm the selected Scriptorium profile ID exists in the configured profile - source or Scriptorium built-in profiles. -- If using `--llm-profile`, pass a Scriptorium profile ID, not a removed - Notarius profile ID. -- Validate the config and selected pipeline: - -```sh -go run ./cmd/notarius config validate \ - --config path/to/config.yml \ - --pipeline dnd-session -``` - -## LLM Profile Override Failure - -Symptom: - -```text -notarius: LLM profile override "..." is not configured -``` - -Fix: - -- Add the profile to the configured Scriptorium profile source. -- Or use an existing Scriptorium profile ID with `--llm-profile`. - -Use `--llm-profile ` when one run should force every LLM-backed binding to -the same Scriptorium profile. The override applies to effective chunk, extract, -merge, and normalize bindings. - -## Missing API Key Environment Variable - -Symptoms include: - -- `api_key_env` -- `unset environment variable` -- provider authentication failures after selecting a profile that needs a key - -Fix: - -- Check the selected Scriptorium profile's `api_key_env` field. -- Set that environment variable before running Notarius. -- Do not put raw API keys in Notarius config or file-backed Scriptorium - profiles. - -## Prompt Or Structured Output Failure - -Symptoms include: - -- `prompt not found` -- `prompt render` -- `schema` -- `validation` -- `decode structured output` - -Fix: - -- Ensure production modules register their embedded Scriptorium prompt and - schema assets. -- If the error names a profile, select a Scriptorium profile that is available - through the configured profile source or built-in catalog. -- If the error names generated output validation, retry with a model that - follows JSON schema instructions reliably. -- Inspect retained diagnostics `error.log`, `resolved-pipeline.json`, and - `run-manifest.json` when available. Prompt text, source text, reference - content, raw schema JSON, and secrets are not written to default diagnostics. -- Provider errors are redacted for bearer tokens and configured API key values. - -## Scene Chunking Failure - -Symptoms include: - -- `dnd scenes chunker` -- `malformed structured output` -- `boundary_caveats` -- `start_unit_id` -- `end_unit_id` -- `gap` -- `overlap` -- `final scene` -- `complete structured output` - -Fix: - -- Validate the pipeline configuration and confirm the input module provides a - transcript source when using `chunk: dnd/scenes`. -- Confirm the selected Scriptorium profile has a working endpoint, model, and - credentials. -- Inspect retained diagnostics for the run error and resolved pipeline. -- If the error names malformed structured output, retry with a model that - follows structured response schemas reliably. -- If the error names `boundary_caveats`, check for blank or whitespace-only - caveat text in the scene response. -- Scene boundaries must resolve to valid source units, cover the full source - document, be contiguous, and not overlap. - -## Session ID - -Symptom: external logs or provider traces cannot be correlated with a Notarius -run. - -Fix: - -- Pass `--session-id ` to `notarius run`. -- Use a stable, non-secret identifier from the external orchestrator. - -## Resume Or Checkpoint Reuse Failure - -Symptoms include: - -- `--resume requires workspace.resume.enabled: true` -- `checkpoint artifact is missing` -- `checkpoint workspace schema version` -- `checkpoint dependency fingerprints do not match` -- a resumed run executes work instead of reusing a checkpoint - -Fix: - -- Set both `workspace.directory` and `workspace.resume.enabled: true`. -- Use `--resume`; plain `notarius run` executes normally and refreshes - checkpoints. -- Confirm the current run uses the same input bytes, resolved pipeline, selected - lanes, runtime LLM profile override, and materialized references as the run - that wrote the checkpoint. -- Inspect retained diagnostics `checkpoint-events.json` to see which workflow - steps were reused or executed. -- If a checkpoint payload is missing or corrupt, rerun without relying on that - checkpoint. Notarius executes invalidated steps normally and writes fresh - checkpoints when checkpointing remains enabled. - -Checkpoint files can contain source text, intermediate outputs, rejected -outputs, metadata, and warnings. Protect the workspace directory accordingly. - -## Debug Output Missing Or Too Verbose - -Symptoms: - -- no files appear under `/debug//`; -- debug files contain more source, reference, prompt, or model-output material - than expected. - -Fix: - -- Set both `workspace.directory` and `workspace.debug.enabled: true`. -- Confirm you are inspecting the current run ID. Debug output is per invocation - and is not used for resume. -- Disable `workspace.debug.enabled` after the inspection run. Debug output may - contain sensitive source material, reference material, prompt inputs, model - outputs, and validation payloads. - -## Output Write Failure - -Symptoms include: - -- `create output directory` -- `write output file` -- `output file name must` -- `unsupported media type` -- `invalid JSON` - -Fix: - -- Ensure `--output-dir` points to a directory path or a path that can be - created. -- Check filesystem permissions and available disk space. -- The production JSON output encoder writes lane payloads under `lanes/` and - accepts only valid `application/json` normalized outputs. If an error names an - unsupported media type or invalid JSON, inspect the lane's merge and normalize - module output. -- If diagnostics were retained, inspect `run-report.json`, `run-manifest.json`, - and `error.log`. - -The CLI rejects unsafe logical output paths before writing files. - -## Raw Output Rejection - -Symptoms include a successful run with: - -- `validation_status` set to `rejected`; -- non-empty `rejected.json`; -- `rejected_outputs` entries in `manifest.json`. - -Explanation and fixes: - -- Validator rejection is a non-fatal run outcome. Rejected module outputs do not - pass to the next pipeline stage. -- Check `rejected.json` for the stage, lane, module, chunk, validator, reason, - message, and attempt count. -- Check `manifest.json` `validator_chains` to see the exact resolved validators - and order used for the rejected validation point. The production `dnd/spells` - extractor runs JSON syntax, JSON schema, D&D spell shape, source-reference, - and source-relatedness validators by default. -- If the configured chain is not what you expected, inspect the selected - binding in config. Omitted `validators` uses production defaults, - `validators: []` disables validators for that binding, and a non-empty list - replaces the default chain in configured order. -- Run `notarius config validate --pipeline ` to catch unknown validator keys - and invalid validator `llm_profile` usage before running the pipeline. -- Increase a module binding's `retries` only when re-running the same module - input can reasonably produce an acceptable output. -- If rejection is deterministic, fix the source input, module configuration, or - validation policy rather than adding retries. - -Common production D&D spell validator reasons: - -- `invalid_json`: the raw output is not valid JSON. -- `json_schema_invalid`: the raw JSON does not match the spell response schema. -- `invalid_spell_shape`: required spell-cast fields are missing or malformed. -- `invalid_source_refs`: source references are missing or do not point to valid - source units. -- `spell_not_near_source`: warning-only; the spell name was not found near the - cited source text. - -## Retry Exhaustion - -Symptoms include: - -- errors containing `failed after ... attempt(s)`; -- rejected output records with `attempt_count` greater than `1`. - -Fix: - -- `retries` is the number of extra attempts after the first attempt for chunk, - extract, merge, and normalize bindings. -- Framework-level errors after the last attempt fail the run. -- Validator rejections after the last attempt are recorded as rejected outputs. -- Check retained `error.log`, `run-manifest.json`, and `rejected.json` for the - operation, module key, lane, chunk, and attempt count. - -## Diagnostics Directory Surprise - -Symptom: the diagnostics directory is missing after a successful run. - -Fix: - -- Check `diagnostics.retention`. -- With `auto`, successful runs without warnings are removed. -- Use `diagnostics.retention: always` when every diagnostics run directory - should be kept. -- Use `--diagnostics-dir` to override the configured work directory for a run. - -Symptom: diagnostics exist even with `retention: never`. - -Explanation: - -- Failed runs are retained so that `error.log` and available context can be - inspected.