1032 lines
60 KiB
Markdown
1032 lines
60 KiB
Markdown
# Documentation Roadmap
|
|
|
|
## Purpose
|
|
|
|
This roadmap defines the work required to bring the project documentation into
|
|
compliance with `docs/policy/documentation.md` and into alignment with the
|
|
current implementation. It is an implementation-planning document: it does not
|
|
replace any canonical documentation, and claims in this file must not be
|
|
treated as product contracts.
|
|
|
|
## Repository Documentation Inventory
|
|
|
|
The classifications below are based on the current code, tests, command
|
|
behavior, and maintained examples.
|
|
|
|
| Path | Classification | Reason |
|
|
| --- | --- | --- |
|
|
| `README.md` | keep and rewrite | It is correctly placed, but still describes a spell-only pipeline and does not orient users to the implemented staged, multi-lane D&D workflow. |
|
|
| `docs/cli.md` | keep and rewrite | It is the correct canonical home for commands, flags, and exit behavior, but contains invalid reference-slot examples, historical removal notes, and incomplete complete-workflow prose. |
|
|
| `docs/config.md` | keep and rewrite | It is the correct configuration reference, but is oversized, includes migration/history material, duplicates internal mechanics, and contains inaccurate validator-chain examples. |
|
|
| `docs/development.md` | keep and lightly update | Its contributor orientation and task-specific reading guide are useful; links and the guide must be updated for the new internal documentation. |
|
|
| `docs/operations.md` | keep and rewrite | It is the correct home for state lifecycle, recovery, and operational procedures, but should stop duplicating pipeline and implementation mechanics. |
|
|
| `docs/policy/architecture.md` | keep and lightly update | It is the controlling architectural policy and currently matches the fixed-stage, ordered-step implementation. Only verify links and terminology; do not dilute its normative role. |
|
|
| `docs/policy/documentation.md` | keep and lightly update | It is the controlling documentation policy. Only verify links after the migration; do not rewrite its rules as part of this work. |
|
|
| `docs/policy/testing.md` | keep and lightly update | It remains the testing policy. Verify links and ensure documentation-related test additions follow its behavioral-testing rules. |
|
|
| `docs/internal/overview.md` | keep and rewrite | It should be a concise implemented-component map, but currently overlaps more detailed internal documents. |
|
|
| `docs/internal/pipeline.md` | keep and rewrite | It is the right home for pipeline resolution and execution mechanics, but is too broad and uses historical wording such as “legacy top-level artifacts.” |
|
|
| `docs/internal/modules.md` | split | Keep it for generic extension mechanics, and move D&D-specific conventions and collaborators into a new `docs/internal/dnd.md`. |
|
|
| `docs/internal/llm.md` | keep and rewrite | The topic is warranted and implemented, but the document should focus on the adapter, scheduler, prompt assembly/caching, validation, and retry boundaries without duplicating config contracts. |
|
|
| `docs/internal/state.md` | keep and lightly update | It appropriately owns state-store and debug-bundle mechanics; verify it against current checkpoint and output behavior and remove duplicated operational instructions. |
|
|
| `docs/integrations/seriatim.md` | keep and lightly update | It documents an actual input contract. Verify the accepted Seriatim shape and compatibility claims against the adapter and fixtures. |
|
|
| `docs/integrations/json-output.md` | keep and rewrite | It documents a durable external output contract, but should clearly separate the output envelope from artifact-specific schemas and internal persistence. |
|
|
| `docs/integrations/chunk-map.md` | keep and lightly update | It is the correct contract document for the exported chunk map; verify identity, range, ordering, and metadata semantics. |
|
|
| `docs/integrations/spell-catalog-overlays.md` | keep and lightly update | It documents an implemented reference-file contract. Verify accepted fields and merge behavior against the loader and tests. |
|
|
| `docs/integrations/dnd-spell.md` | keep and rewrite | Harmonize it with the other artifact contracts and ensure it documents only the current minimal spell schema. |
|
|
| `docs/integrations/dnd-npc.md` | keep and rewrite | Harmonize it and ensure the NPC identity artifact remains distinct from NPC interactions. |
|
|
| `docs/integrations/dnd-npc-interaction.md` | keep and rewrite | Harmonize its event schema, enum semantics, evidence rules, and deterministic fields. |
|
|
| `docs/integrations/dnd-combat-turn.md` | keep and rewrite | Harmonize it and document that extraction is scoped to combat-tagged scenes without duplicating routing internals. |
|
|
| `docs/integrations/dnd-item-event.md` | keep and rewrite | Harmonize its ordered event schema, including item and currency semantics. |
|
|
| `docs/integrations/dnd-scene-description.md` | keep and rewrite | Harmonize its minimal `kind`, `title`, and `summary` contract and distinguish generated values from deterministic scene identity/ranges. |
|
|
| `docs/adr/0001-record-architecture-decisions.md` | keep and lightly update | It establishes the ADR convention. Preserve the decision; verify links and status only. |
|
|
| `docs/adr/0002-use-a-fixed-linear-pipeline.md` | keep and lightly update | It is retained architectural history supporting the current fixed-stage pipeline. |
|
|
| `docs/adr/0003-use-typed-artifacts-with-two-zone-model-output.md` | keep and lightly update | It records a still-relevant artifact boundary. Preserve the decision and verify terminology. |
|
|
| `docs/adr/0004-package-by-domain-and-layer.md` | keep and lightly update | It records the package organization policy reflected in the code. |
|
|
| `docs/adr/0005-cache-the-deterministic-chunk-plan.md` | keep and lightly update | It records the implemented chunk-plan cache decision. |
|
|
| `docs/adr/0006-separate-operational-state-from-published-output.md` | keep and lightly update | Preserve it as a superseded ADR and verify that it points to ADR 0007. |
|
|
| `docs/adr/0007-adopt-recording-and-reuse-state.md` | keep and lightly update | It is the current state model decision; verify links and current terminology. |
|
|
| `docs/adr/0008-support-ordered-pipeline-steps.md` | keep and lightly update | It records the implemented ordered-step model and remains authoritative history. |
|
|
| `docs/adr/0009-minimize-dnd-extraction-contracts.md` | keep and lightly update | It records the durable minimal-extraction policy used by current D&D artifacts. |
|
|
| `docs/roadmap/future.md` | keep and lightly update | It is the canonical home for unimplemented work. Remove anything already implemented, but do not convert it into development history. |
|
|
| `examples/dnd-minimal.config.yml` | keep and lightly update | It is the maintained single-lane configuration and currently resolves successfully. |
|
|
| `examples/dnd-complete.config.yml` | keep and lightly update | It is the maintained staged configuration using all six implemented D&D extraction lanes; keep it synchronized with registries and reference slots. |
|
|
| `examples/seriatim-minimal-transcript.json` | keep and lightly update | It is a valid small Seriatim input, but is best suited to the minimal spell example rather than demonstrating the complete pipeline. |
|
|
| `examples/dnd-party.txt` | keep and lightly update | It is an implemented static-reference example used by the complete configuration. |
|
|
| `examples/dnd-glossary.txt` | keep and lightly update | It is an implemented stable-reference example used by the complete configuration. |
|
|
| `examples/dnd-spell-catalog.json` | keep and lightly update | It is a valid spell-catalog overlay example and should remain aligned with that integration contract. |
|
|
| `examples/dnd-complete-transcript.json` | create new | Add a synthetic Seriatim input that meaningfully exercises the configured D&D lanes without claiming to be a golden model-output fixture. |
|
|
| `docs/internal/configuration.md` | create new | The implemented configuration loader, defaults, environment precedence, effective resolution, redaction, and digest logic warrant a focused internal document. |
|
|
| `docs/internal/cli.md` | create new | The CLI is the composition root for configuration, pipeline execution, state, publication, and debug bundles; those mechanics should not live in the user command reference. |
|
|
| `docs/internal/dnd.md` | create new | Six D&D extractors now share substantial conventions that need one canonical internal home to prevent drift. |
|
|
| `docs/roadmap/documentation.md` | create new, then delete when complete | This is the temporary implementation roadmap for the migration. Retire it only after all stages and final validation succeed. |
|
|
|
|
No current documentation file needs to be moved wholesale or merged wholesale.
|
|
The migration should instead move narrowly scoped material to its canonical
|
|
owner while rewriting the source document to link to that owner.
|
|
|
|
## Policy Compliance Assessment
|
|
|
|
### Required documents
|
|
|
|
The policy-required top-level documents for this CLI/config-driven project
|
|
already exist: `README.md`, `docs/development.md`, `docs/architecture.md` or its
|
|
project equivalent, `docs/cli.md`, `docs/config.md`, and
|
|
`docs/operations.md`. In this repository the normative architecture document
|
|
is deliberately located at `docs/policy/architecture.md`; creating a second
|
|
`docs/architecture.md` would introduce competing authority and is therefore
|
|
not recommended.
|
|
|
|
`docs/api.md` is not applicable. The repository does not implement an HTTP API.
|
|
A `docs/consumers/` tree is also not applicable because the repository does
|
|
not expose a supported public library surface with independent consumer
|
|
workflows.
|
|
|
|
### Recommended additions
|
|
|
|
The following recommended documents should be added because implemented
|
|
components have distinct maintainer-facing boundaries:
|
|
|
|
- `docs/internal/configuration.md`
|
|
- `docs/internal/cli.md`
|
|
- `docs/internal/dnd.md`
|
|
|
|
The existing integration set already covers the implemented external input,
|
|
published output, reference-file, chunk-map, and D&D artifact contracts. A
|
|
separate Scriptorium-provider integration document is not recommended:
|
|
user-selectable profiles and credentials belong in `docs/config.md`, while
|
|
adapter and scheduling mechanics belong in `docs/internal/llm.md`. The
|
|
provider's own API is not a contract maintained by this repository.
|
|
|
|
### Stale, misplaced, or duplicated content
|
|
|
|
- `README.md` describes the application as extracting only D&D spells. The
|
|
code and complete example now implement spells, NPC identities, NPC
|
|
interactions, combat turns, item events, and scene descriptions in ordered
|
|
steps.
|
|
- `docs/cli.md` gives examples using undeclared reference slots such as
|
|
`chunk.scene_guide`, `spells.merge.merge_notes`, and
|
|
`spells.normalize.normalization_notes`. It also mentions the removed
|
|
`--diagnostics-dir` flag as development history.
|
|
- The complete-workflow prose in `docs/cli.md` omits item events and NPC
|
|
interactions even though both are configured and registered.
|
|
- `docs/config.md` contains a version-2-to-version-3 migration section and
|
|
removed field/environment-variable history. Such history belongs in version
|
|
control, an ADR when it explains a durable decision, or a roadmap while
|
|
still planned—not in a current contract reference.
|
|
- Several validator-chain examples in `docs/config.md` do not match
|
|
`internal/modules/dnd/register/chains.go`; one NPC-interaction chain also
|
|
repeats `generic/valid_json`. Future prose should be derived from registry
|
|
definitions and contract tests rather than copied forward.
|
|
- `docs/config.md`, `docs/operations.md`, and `docs/internal/pipeline.md`
|
|
duplicate details about resolution, generated references, provenance,
|
|
checkpoints, and state. Each fact needs one canonical owner with links from
|
|
the other documents.
|
|
- `docs/internal/modules.md` combines generic module-extension rules,
|
|
registered-key inventories, D&D prompt conventions, and artifact details.
|
|
It should be split by responsibility.
|
|
- `docs/internal/pipeline.md` uses historical “legacy” language for an
|
|
implemented implicit single-step configuration form. Document the current
|
|
form neutrally and leave migration history out.
|
|
- Artifact integration documents vary in organization and sometimes blend
|
|
external wire contracts with validator or normalizer implementation. They
|
|
should share a contract-focused outline.
|
|
|
|
### Planned and historical behavior
|
|
|
|
ADR decision context and supersession history are permitted in `docs/adr/`.
|
|
Unimplemented work belongs only in `docs/roadmap/future.md` and this temporary
|
|
roadmap. During rewrite, search all other documentation for “future,”
|
|
“planned,” “deprecated,” “removed,” “migration,” and “legacy”; retain a use
|
|
only when it describes a currently supported compatibility contract or a
|
|
durable ADR, and otherwise delete or relocate it.
|
|
|
|
The currently supported `roster` alias and any supported lane-level
|
|
reference/validator aliases are current compatibility behavior, not stale
|
|
history. If retained in the user reference, state the accepted spelling and
|
|
canonical replacement concisely without recounting its development history.
|
|
|
|
### Examples and links
|
|
|
|
Both maintained configuration files currently load and resolve. The minimal
|
|
example validates without provider credentials. The complete example requires
|
|
the configured OpenRouter credential to be present even for configuration
|
|
validation; an inert placeholder value is sufficient for offline validation
|
|
because no provider request is made.
|
|
|
|
The current minimal transcript contains two spell-cast occurrences and does
|
|
not meaningfully illustrate the complete six-lane workflow. Add one synthetic
|
|
complete transcript, keep the existing transcript for the minimal example,
|
|
and extend existing example contract tests to parse the new input. Do not add
|
|
more configuration examples: current tests intentionally enforce exactly the
|
|
two maintained configurations.
|
|
|
|
All checked local Markdown file targets currently exist. No repository-owned
|
|
Markdown link or anchor checker was found, and no common external checker was
|
|
installed in the inspection environment. Heading-anchor and external-link
|
|
verification therefore remain explicit manual review items unless lightweight
|
|
repository tooling is added in a separate, policy-justified change.
|
|
|
|
## Target Documentation Set
|
|
|
|
Only the following files should remain after the migration. The temporary
|
|
`docs/roadmap/documentation.md` is intentionally absent from the final tree.
|
|
|
|
```text
|
|
README.md
|
|
docs/
|
|
cli.md
|
|
config.md
|
|
development.md
|
|
operations.md
|
|
policy/
|
|
architecture.md
|
|
documentation.md
|
|
testing.md
|
|
internal/
|
|
overview.md
|
|
cli.md
|
|
configuration.md
|
|
pipeline.md
|
|
modules.md
|
|
dnd.md
|
|
llm.md
|
|
state.md
|
|
integrations/
|
|
seriatim.md
|
|
json-output.md
|
|
chunk-map.md
|
|
spell-catalog-overlays.md
|
|
dnd-spell.md
|
|
dnd-npc.md
|
|
dnd-npc-interaction.md
|
|
dnd-combat-turn.md
|
|
dnd-item-event.md
|
|
dnd-scene-description.md
|
|
adr/
|
|
0001-record-architecture-decisions.md
|
|
0002-use-a-fixed-linear-pipeline.md
|
|
0003-use-typed-artifacts-with-two-zone-model-output.md
|
|
0004-package-by-domain-and-layer.md
|
|
0005-cache-the-deterministic-chunk-plan.md
|
|
0006-separate-operational-state-from-published-output.md
|
|
0007-adopt-recording-and-reuse-state.md
|
|
0008-support-ordered-pipeline-steps.md
|
|
0009-minimize-dnd-extraction-contracts.md
|
|
roadmap/
|
|
future.md
|
|
examples/
|
|
dnd-minimal.config.yml
|
|
dnd-complete.config.yml
|
|
seriatim-minimal-transcript.json
|
|
dnd-complete-transcript.json
|
|
dnd-party.txt
|
|
dnd-glossary.txt
|
|
dnd-spell-catalog.json
|
|
```
|
|
|
|
### Public and contributor documents
|
|
|
|
| Path | Audience and purpose | Canonical scope and recommended outline | Sources to inspect | Acceptance criteria |
|
|
| --- | --- | --- | --- | --- |
|
|
| `README.md` | New users and evaluators; explain what the application does and provide the shortest successful start. | Overview; implemented capabilities; prerequisites; minimal quickstart; complete example pointer; documentation map. Own orientation only. | `cmd/notarius`, `internal/cli`, both example configs, CLI example contract tests. | Describes all current capabilities without enumerating internal mechanics; quickstart runs; links point to canonical docs; no planned behavior. |
|
|
| `docs/cli.md` | CLI users and automation authors; command/flag/exit reference. | Invocation; global help; `run`; `config validate`; `pipelines list`; flags by command; stdout/stderr; exit statuses; examples. | `internal/cli/run.go`, command parsing and contract tests, built binary help. | Every documented command and flag exists; defaults and exit behavior match tests; no config-field, state-layout, or removed-flag history. |
|
|
| `docs/config.md` | Pipeline authors and operators; complete current configuration contract. | File/version; top-level fields; defaults; profiles; pipelines; chunker; ordered steps/lanes; references; validators; state/output/debug fields; environment precedence; validation examples; registry-key tables. | `internal/core/config`, effective-config/redaction tests, `internal/framework/pipeline`, module registries, both example configs. | Every accepted field, default, enum, key, alias, and rejection rule matches code/tests; validator examples match production chains; no migration history or internal algorithms. |
|
|
| `docs/development.md` | Contributors; repository orientation and task-specific reading guide. | Prerequisites; build/test commands; package map; reading guide by task; policy/ADR rules. | `go.mod`, package tree, `Makefile` or CI if present, policy docs, internal-doc tree. | Commands work; each task points first to the canonical document; no duplicate architecture or contract prose. |
|
|
| `docs/operations.md` | Operators; safe execution, state, recovery, cleanup, and troubleshooting. | Run lifecycle; filesystem prerequisites; recording/reuse state; resume/retry; selective recomputation; publication/debug bundles; failure recovery; operational checklist. | CLI orchestration, `internal/core/fileio`, debug bundle code, chunk-plan/checkpoint code, runner tests. | Procedures reflect current behavior and reason codes; destructive operations identify exact targets; no duplicated CLI flag reference or pipeline algorithm. |
|
|
|
|
### Policy and roadmap documents
|
|
|
|
| Path | Audience and purpose | Canonical scope and outline | Sources to inspect | Acceptance criteria |
|
|
| --- | --- | --- | --- | --- |
|
|
| `docs/policy/architecture.md` | All contributors; normative development architecture and invariants. | Keep its existing normative organization. | Current package boundaries, resolver, runner, artifact and state code, ADRs. | No implementation contradiction; links valid; changes, if any, preserve its controlling-policy role. |
|
|
| `docs/policy/documentation.md` | Documentation authors; normative layout, ownership, and maintenance rules. | Keep its existing policy organization. | Final documentation tree. | Every canonical home it names exists or is explicitly inapplicable; links valid. |
|
|
| `docs/policy/testing.md` | Contributors; normative testing approach. | Keep its existing policy organization. | Current test suite and any example tests changed by this migration. | New tests assert user-visible or contract behavior, not exact prose or incidental document length. |
|
|
| `docs/roadmap/future.md` | Maintainers planning future work; unimplemented ideas and policy choices. | Current priorities; later considerations; explicit boundaries. | Registries, current module implementations, accepted ADRs. | Contains only unimplemented work; no completed-work log; no claim is presented as current product behavior. |
|
|
|
|
### Internal documents
|
|
|
|
| Path | Audience and purpose | Canonical scope and recommended outline | Sources to inspect | Acceptance criteria |
|
|
| --- | --- | --- | --- | --- |
|
|
| `docs/internal/overview.md` | Maintainers; concise map of implemented components and dependency direction. | Runtime flow; package/component inventory; dependency direction; extension points; links to focused internals. | `cmd/notarius`, `internal/cli`, `internal/core`, `internal/framework`, `internal/modules`, registration composition. | Every named component exists; remains descriptive rather than normative; detailed mechanics are linked, not copied. |
|
|
| `docs/internal/cli.md` | CLI maintainers; composition-root behavior. | Command dispatch; dependency construction; config resolution; run orchestration; publication/debug handling; failure mapping; tests. | `internal/cli` and its tests, composition/registration code. | Explains boundaries and ownership without duplicating public command syntax or operations procedures. |
|
|
| `docs/internal/configuration.md` | Config maintainers; parsing and resolution mechanics. | File model; load/validate phases; default/env precedence; effective config; redaction/digests; pipeline handoff; tests. | `internal/core/config/*.go`, config contract/effective/redaction tests. | Matches phase ordering and error boundaries; links to `docs/config.md` for the user contract; contains no unsupported future design. |
|
|
| `docs/internal/pipeline.md` | Pipeline maintainers; resolution, preparation, and execution mechanics. | Resolver inputs; ordered-step planning; artifact/reference flow; preparation; runner lifecycle; checkpoints; concurrency/failure; tests. | `internal/framework/pipeline`, resolver/runner/checkpoint tests, architecture policy. | Describes the fixed stages and ordered steps accurately; distinguishes artifacts, references, and evidence; avoids historical framing. |
|
|
| `docs/internal/modules.md` | Extension authors; generic module conventions and registration mechanics. | Module taxonomy; interfaces; registry/keys; generic codecs/normalizers/validators; asset embedding; adding a module; test checklist. | `internal/modules/register`, generic and Seriatim modules, registry tests. | D&D-specific details are links to `dnd.md`; adding an extension can be done without copying another module blindly; registered-key inventories are generated from or checked against code. |
|
|
| `docs/internal/dnd.md` | D&D module maintainers; shared extractor conventions and collaborator boundaries. | Implemented lanes; common package layout; prompt-message ordering/cache-stable assets; raw/normalized schemas; source references; deterministic normalization; validation chains; generated-reference use; combat scene routing; tests. | `internal/modules/dnd`, D&D registration chains, shared assets/helpers, codec/normalizer/validator tests. | Covers conventions shared by all six extractors, names intentional exceptions, and links to integration contracts instead of reproducing wire schemas. |
|
|
| `docs/internal/llm.md` | LLM-boundary maintainers; provider-independent execution mechanics. | Boundary and adapters; Scriptorium adapter; profile/model selection handoff; prompt construction/cache ordering; scheduler/concurrency; schema/repair/retry flow; observability; tests. | `internal/framework/llm`, Scriptorium adapter, prompt assets/builders, scheduler and retry tests. | Distinguishes stable prompt assets from variable messages; no credentials or provider API tutorial; config fields are linked to `docs/config.md`. |
|
|
| `docs/internal/state.md` | State/persistence maintainers; recording, reuse, and debug implementation. | State zones; identities/digests; chunk plan; checkpoints; reuse/recompute; publication; debug bundles; tests. | state/checkpoint/chunk-plan/file I/O/debug code and tests, ADRs 0005 and 0007. | Physical and logical boundaries match code; operational procedures remain in `docs/operations.md`. |
|
|
|
|
### Integration documents
|
|
|
|
All integration documents target pipeline authors and downstream consumers.
|
|
Their common outline should be: contract identity and version, producer or
|
|
consumer, wire shape, field semantics, ordering/identity/range rules,
|
|
validation and compatibility notes, minimal example, and links to related
|
|
config/internal docs.
|
|
|
|
| Path | Canonical purpose | Sources to inspect | Acceptance criteria |
|
|
| --- | --- | --- | --- |
|
|
| `docs/integrations/seriatim.md` | Accepted external Seriatim transcript input. | Seriatim adapter/codec, fixtures, parser tests. | Documents only accepted fields and normalization assumptions; sample parses. |
|
|
| `docs/integrations/json-output.md` | Published JSON envelope and artifact grouping contract. | output encoder/publisher, artifact types, golden/contract tests. | Separates envelope fields from lane schemas and operational state; examples match encoding. |
|
|
| `docs/integrations/chunk-map.md` | Exported chunk-map artifact contract. | chunk artifact types, chunkers, normalizer/output tests. | Identity, ordering, segment ranges, tags, and optional metadata match code. |
|
|
| `docs/integrations/spell-catalog-overlays.md` | User-supplied spell-catalog overlay format. | overlay loader/merge code and tests, example catalog. | Accepted keys and precedence are exact; no general spell-catalog tutorial. |
|
|
| `docs/integrations/dnd-spell.md` | Minimal spell occurrence artifact. | spell schema, codec, normalizer, validators, contract tests. | Name, caster, deterministic identity, and source ranges match the current v1 wire contract; no removed synthesis fields. |
|
|
| `docs/integrations/dnd-npc.md` | Minimal NPC identity artifact. | NPC schema, codec, identity normalizer, validators, tests. | Clearly distinguishes identity extraction from interaction events and documents evidence rules. |
|
|
| `docs/integrations/dnd-npc-interaction.md` | Ordered NPC interaction event artifact. | interaction schema/enums, codec, normalizer, validators, tests. | Enum values, actor/NPC semantics, deterministic fields, ordering, and source ranges are exact. |
|
|
| `docs/integrations/dnd-combat-turn.md` | Combat-turn artifact emitted for combat scenes. | combat schema, scene-filter routing, normalizer/validators, tests. | Documents minimal turn fields and combat-scene eligibility; does not imply extraction from non-combat scenes. |
|
|
| `docs/integrations/dnd-item-event.md` | Ordered item and currency event artifact. | item schema/enums, currency handling, codec, normalizer, validators, tests. | `discovered`, acquired, lost, consumed/use, and transferred semantics match code; currency representation is unambiguous. |
|
|
| `docs/integrations/dnd-scene-description.md` | Minimal scene-description artifact. | scene schema/enums, normalizer, chunk-map join logic, tests. | `kind`, `title`, and `summary` are separated from deterministically supplied scene identity and ranges; enum is exact. |
|
|
|
|
### ADRs
|
|
|
|
Every ADR targets maintainers and owns one historical architectural decision,
|
|
not the current user contract. Each must retain the Nygard headings
|
|
`Status`, `Context`, `Decision`, and `Consequences`, with status/date and
|
|
supersession links where applicable. Acceptance for every ADR is limited to
|
|
valid links, an accurate status, and terminology that still identifies the
|
|
decision; do not rewrite past context to resemble current reference docs.
|
|
|
|
| Path | Decision scope | Current implementation evidence |
|
|
| --- | --- | --- |
|
|
| `docs/adr/0001-record-architecture-decisions.md` | ADR process | ADR directory and documentation policy |
|
|
| `docs/adr/0002-use-a-fixed-linear-pipeline.md` | Fixed pipeline stages | architecture policy, resolver, runner |
|
|
| `docs/adr/0003-use-typed-artifacts-with-two-zone-model-output.md` | Typed artifacts and raw/normalized boundary | artifact types, codecs, normalizers |
|
|
| `docs/adr/0004-package-by-domain-and-layer.md` | Package organization | `internal/core`, `framework`, `modules`, `cli` |
|
|
| `docs/adr/0005-cache-the-deterministic-chunk-plan.md` | Chunk-plan persistence | chunk-plan store and tests |
|
|
| `docs/adr/0006-separate-operational-state-from-published-output.md` | Superseded state design | status/supersession link to ADR 0007 |
|
|
| `docs/adr/0007-adopt-recording-and-reuse-state.md` | Current recording/reuse state | state/checkpoint implementation |
|
|
| `docs/adr/0008-support-ordered-pipeline-steps.md` | Ordered pipeline steps | configuration, resolver, complete example |
|
|
| `docs/adr/0009-minimize-dnd-extraction-contracts.md` | Minimal structured D&D extraction | current v1 D&D schemas and prompts |
|
|
|
|
### Maintained examples
|
|
|
|
The audience is users learning configuration and contributors verifying
|
|
contracts. Each example owns only its concrete, copyable input or
|
|
configuration; prose explanation belongs in the README or canonical
|
|
reference.
|
|
|
|
| Path | Purpose and sources | Acceptance criteria |
|
|
| --- | --- | --- |
|
|
| `examples/dnd-minimal.config.yml` | Minimal Seriatim-to-spell pipeline; inspect config loader, registries, and minimal example tests. | Loads, resolves, lists, and validates offline without credentials. |
|
|
| `examples/dnd-complete.config.yml` | Complete ordered-step pipeline using every D&D lane; inspect all registries and generated-reference slots. | Loads and resolves with a placeholder credential; all configured keys and dependencies exist. |
|
|
| `examples/seriatim-minimal-transcript.json` | Small spell-focused input for the minimal pipeline; inspect Seriatim parser. | Parses as accepted Seriatim input and remains small enough to understand by inspection. |
|
|
| `examples/dnd-complete-transcript.json` | Synthetic input covering implemented scene, NPC, interaction, combat, item/currency, and spell evidence. | Parses offline; contains evidence for every lane without prescribing nondeterministic expected model text. |
|
|
| `examples/dnd-party.txt` | Stable party reference for the complete pipeline. | Referenced path exists; content is secret-free and consistent with the complete transcript. |
|
|
| `examples/dnd-glossary.txt` | Stable campaign glossary reference. | Referenced path exists; terms are consistent with the complete transcript. |
|
|
| `examples/dnd-spell-catalog.json` | Spell overlay reference. | Parses with the production overlay loader and contains only supported fields. |
|
|
|
|
## File-by-File Rewrite Guidance
|
|
|
|
### `README.md`
|
|
|
|
- Replace the spell-only description with a compact description of the staged,
|
|
modular extraction pipeline and its current D&D lanes.
|
|
- Keep one minimal copyable quickstart. Link to the complete example rather
|
|
than embedding its configuration.
|
|
- Link command syntax to `docs/cli.md`, configuration to `docs/config.md`,
|
|
operations to `docs/operations.md`, internals to
|
|
`docs/internal/overview.md`, and future work to
|
|
`docs/roadmap/future.md`.
|
|
- Avoid registry inventories, full schemas, state layouts, architecture rules,
|
|
planned features, and development history.
|
|
|
|
### `docs/cli.md`
|
|
|
|
- Regenerate the command inventory from the parser and verify it against
|
|
actual help output for the root command, `run`, `config validate`, and
|
|
`pipelines list`.
|
|
- Document exact flag ownership, defaults, required arguments, output streams,
|
|
and exit behavior. Do not infer a successful subcommand `--help` path when
|
|
the parser returns usage with status 2.
|
|
- Remove undeclared reference-slot examples and the removed
|
|
`--diagnostics-dir` note.
|
|
- Make the complete example describe both ordered steps and all six lanes.
|
|
- Link configuration semantics and registry keys to `docs/config.md`, state and
|
|
recovery to `docs/operations.md`, and output shapes to integration docs.
|
|
|
|
### `docs/config.md`
|
|
|
|
- Rebuild the reference from config structs, loader/default logic, environment
|
|
handling, validation, effective resolution, and production registries.
|
|
- Give every current field its type, required/optional status, default,
|
|
allowed values, precedence, and validation rule.
|
|
- Keep current aliases only when the parser still accepts them. Describe the
|
|
canonical spelling without migration narrative.
|
|
- Replace copied validator chains with the exact registered defaults from
|
|
`internal/modules/dnd/register/chains.go`; specifically verify spell, NPC,
|
|
combat-turn, item-event, scene-description, and NPC-interaction extract and
|
|
normalize chains.
|
|
- Explain ordered steps, lane dependencies, generated references, and
|
|
fail-whole-run behavior at the user-contract level. Link internal resolution
|
|
mechanics to `docs/internal/configuration.md` and
|
|
`docs/internal/pipeline.md`.
|
|
- Delete version migration, removed-field, and removed-environment-variable
|
|
sections. Do not document proposals from `future.md`.
|
|
|
|
### `docs/development.md`
|
|
|
|
- Retain the concise contributor orientation.
|
|
- Update its task-routing table to include internal configuration, CLI, D&D,
|
|
LLM, pipeline, and state documents plus integration contracts.
|
|
- Verify build, test, and formatting commands against repository tooling.
|
|
- Link architecture/testing/documentation policy instead of restating them.
|
|
|
|
### `docs/operations.md`
|
|
|
|
- Verify directory creation, permissions, state roots, recording/reuse,
|
|
checkpoints, reason codes, debug bundles, publication, interruption, and
|
|
cleanup against the CLI and state implementation.
|
|
- Keep commands operational and safe; use explicit example paths for cleanup.
|
|
- Link flag syntax to `docs/cli.md`, fields to `docs/config.md`, state
|
|
implementation to `docs/internal/state.md`, and output contracts to
|
|
integration docs.
|
|
- Avoid pipeline pseudocode, registry inventories, configuration migrations,
|
|
and duplicated physical-schema detail.
|
|
|
|
### `docs/internal/overview.md`
|
|
|
|
- Reduce it to the runtime path and an inventory of actual packages/components
|
|
with their dependency direction.
|
|
- Add links to every focused internal document and explain which one owns each
|
|
mechanic.
|
|
- Do not make new normative architectural rules; link
|
|
`docs/policy/architecture.md`.
|
|
|
|
### `docs/internal/cli.md` and `docs/internal/configuration.md`
|
|
|
|
- Create these from their implemented package boundaries, using the outlines
|
|
and acceptance criteria in the target set.
|
|
- The CLI document must explain composition and error mapping, not public
|
|
invocation syntax.
|
|
- The configuration document must explain load/resolution phases, not repeat
|
|
the field reference.
|
|
- Both should name focused tests a maintainer should run before changing the
|
|
component.
|
|
|
|
### `docs/internal/pipeline.md`
|
|
|
|
- Reconstruct the flow from resolver and runner code, including ordered steps,
|
|
generated artifacts/references, preparation, checkpoints, concurrency, and
|
|
failure propagation.
|
|
- Use “implicit single-step form” for the supported compact configuration;
|
|
remove “legacy” history.
|
|
- Keep the artifact/reference/evidence distinction explicit.
|
|
- Link normative invariants to architecture policy, user configuration to
|
|
`docs/config.md`, and state internals to `docs/internal/state.md`.
|
|
|
|
### `docs/internal/modules.md` and `docs/internal/dnd.md`
|
|
|
|
- Rewrite `modules.md` as the generic extension-author guide: interfaces,
|
|
registry namespaces, package layout, embedded assets, collaborator chains,
|
|
and behavior-focused test expectations.
|
|
- Move D&D-specific conventions into new `dnd.md`. Inspect all six extractors,
|
|
not a single exemplar, and list intentional exceptions.
|
|
- In `dnd.md`, document stable-before-variable prompt-message ordering,
|
|
shared prompt assets, raw-versus-normalized types, deterministic fields,
|
|
source evidence, chain construction, generated references, and routing.
|
|
- Do not copy full artifact schemas into either file; link the applicable
|
|
integration contract.
|
|
|
|
### `docs/internal/llm.md`
|
|
|
|
- Verify the Scriptorium adapter, profile handoff, scheduler, prompt assembly,
|
|
validation, retry/repair, and observability behavior.
|
|
- Explain why byte-identical shared prompt assets and stable message ordering
|
|
matter for backend prompt caching, but do not add “change detector” tests
|
|
for exact prompt-prefix length.
|
|
- Keep provider credential fields in `docs/config.md` and provider usage
|
|
tutorials outside the repository.
|
|
|
|
### `docs/internal/state.md`
|
|
|
|
- Verify chunk-plan identity, artifact/checkpoint identity, state zones,
|
|
recomputation, published output, and debug bundles.
|
|
- Keep storage mechanics here and operator procedures in
|
|
`docs/operations.md`.
|
|
- Preserve the rationale links to ADRs 0005 and 0007 without retelling their
|
|
full history.
|
|
|
|
### Integration documents
|
|
|
|
- Apply the common integration outline to every file.
|
|
- Derive schemas, required fields, enums, ordering, deterministic additions,
|
|
and source-range rules from production types/codecs/validators and contract
|
|
tests.
|
|
- Show minimal valid JSON only. Avoid full-session golden output, internal
|
|
prompt text, validator ordering, normalization algorithms, or roadmap work.
|
|
- Cross-link related contracts: JSON output to lane artifacts; scene
|
|
descriptions and combat turns to chunk maps; NPC interactions to NPC
|
|
identities; spells to catalog overlays; item events to currency semantics.
|
|
- Do not carry forward removed high-synthesis fields or imply backwards
|
|
compatibility that the pre-release v1 contracts do not provide.
|
|
|
|
### ADRs, policies, and `docs/roadmap/future.md`
|
|
|
|
- Do not rewrite ADR decisions or policy content as part of a consistency
|
|
cleanup. Only correct broken links, statuses, and terms that no longer
|
|
identify the relevant code.
|
|
- Confirm ADR 0006 remains explicitly superseded by ADR 0007.
|
|
- Review `future.md` against registries and integration contracts; remove
|
|
completed items and retain only genuinely unimplemented outcomes.
|
|
- Do not add implementation chronology, release notes, or a completed-work
|
|
section.
|
|
|
|
## Examples Plan
|
|
|
|
The `examples/` directory exists and should remain the home of maintained,
|
|
complete, secret-free inputs and configurations. It should contain exactly two
|
|
configuration examples:
|
|
|
|
1. `examples/dnd-minimal.config.yml` demonstrates the smallest useful
|
|
Seriatim-to-spell pipeline. Validate with:
|
|
|
|
```sh
|
|
go run ./cmd/notarius config validate \
|
|
--config examples/dnd-minimal.config.yml \
|
|
--pipeline dnd-session
|
|
```
|
|
|
|
Link it from `README.md`, `docs/cli.md`, and `docs/config.md`.
|
|
|
|
2. `examples/dnd-complete.config.yml` demonstrates ordered steps, generated
|
|
references, all implemented D&D lanes, state/output/debug configuration,
|
|
and bounded LLM concurrency. Validate offline with:
|
|
|
|
```sh
|
|
OPENROUTER_API_KEY=validation-placeholder \
|
|
go run ./cmd/notarius config validate \
|
|
--config examples/dnd-complete.config.yml \
|
|
--pipeline dnd-session
|
|
```
|
|
|
|
Link it from `README.md`, `docs/cli.md`, `docs/config.md`, and
|
|
`docs/operations.md`. State clearly that the placeholder is for validation
|
|
only and cannot run the pipeline.
|
|
|
|
Keep `examples/seriatim-minimal-transcript.json` as the minimal pipeline input.
|
|
Create `examples/dnd-complete-transcript.json` as a synthetic but realistic
|
|
input containing:
|
|
|
|
- narrative and combat scenes plus recap/meta material;
|
|
- a named NPC present in dialogue, a hostile combatant, and a mentioned-only
|
|
NPC;
|
|
- at least one spell cast;
|
|
- an acquired or discovered item, a transfer or consumption/loss event, and a
|
|
currency event;
|
|
- segment ranges sufficient for all evidence-based extractors.
|
|
|
|
The new transcript must exercise implemented behavior only. It must not embed
|
|
expected model-generated summaries or exact extraction output. Extend the
|
|
existing maintained-example test to parse it with the production Seriatim
|
|
adapter. Keep `dnd-party.txt`, `dnd-glossary.txt`, and
|
|
`dnd-spell-catalog.json` consistent with its names and terms, and validate the
|
|
catalog through the production loader if that behavior is not already covered.
|
|
|
|
## Internal Documentation Plan
|
|
|
|
The modular architecture warrants `docs/internal/`. The target internal files
|
|
and their component contracts are:
|
|
|
|
| Component and path | Purpose; inputs and outputs | Boundaries and config | Adapters and failure behavior | Tests and invariants |
|
|
| --- | --- | --- | --- | --- |
|
|
| CLI composition root — `docs/internal/cli.md` | Converts parsed commands plus dependencies into validation, listing, or a completed run and published artifacts. | Uses CLI flags and effective config; owns orchestration, not parsing schemas or module algorithms. | Calls config, registries, pipeline, state, output, and debug components; maps failures to diagnostics/exit status without partial success. | Inspect `internal/cli/*_test.go`; preserve thin composition-root and fail-whole-run behavior. |
|
|
| Configuration — `docs/internal/configuration.md` | Converts YAML, environment, and CLI selections into validated, redacted effective configuration and digests. | Uses all fields in `docs/config.md`; does not own runtime execution. | Hands resolved pipeline/profile data to framework components; rejects unknown/invalid/ambiguous configuration before execution. | Inspect config contract, effective-config, redaction, and validation tests; preserve deterministic precedence and secret redaction. |
|
|
| Pipeline — `docs/internal/pipeline.md` | Resolves configured ordered steps and executes fixed stages, passing typed artifacts/references between lanes. | Uses pipeline, step, lane, profile, reference, validator, state, and concurrency fields. | Uses chunker/extractor/merger/normalizer/validator registries plus state/checkpoint adapters; dependency or stage failure fails the run. | Inspect resolver, runner, preparation, checkpoint, and dependency tests; preserve fixed stages, ordered steps, type safety, and bounded concurrency. |
|
|
| Generic modules — `docs/internal/modules.md` | Defines how registered chunkers, collaborators, codecs, assets, and validators plug into the framework. | Uses selectable registry keys and module-specific config; does not own the overall run. | Registry lookup is the adapter boundary; unknown keys or incompatible artifact kinds fail during resolution/preparation. | Inspect registry and generic-module tests; preserve config-driven selection and explicit registration. |
|
|
| D&D modules — `docs/internal/dnd.md` | Documents common construction of spells, NPCs, NPC interactions, combat turns, item events, and scene descriptions. | Uses D&D module keys, reference slots, default chains, scene routing, and artifact schemas. | Uses shared prompt assets, LLM collaborators, codecs, deterministic normalizers, validators, and generated references; malformed or unsupported model output is rejected/retried according to the LLM boundary. | Inspect every D&D package and registration-chain test; preserve minimal schemas, evidence-only references, prompt-cache ordering, deterministic enrichment, and combat-only routing. |
|
|
| LLM boundary — `docs/internal/llm.md` | Converts collaborator requests into scheduled provider calls and validated raw output. | Uses profile/model credentials, concurrency, retry, and collaborator selections. | Scriptorium is the provider adapter; provider, validation, or exhausted-retry failures propagate as lane/run failures. | Inspect scheduler, adapter, prompt, validation, and retry tests; preserve concurrency bounds, byte-stable assets, and separation of generation from deterministic normalization. |
|
|
| State — `docs/internal/state.md` | Persists deterministic chunk plans, recordings/checkpoints, reuse metadata, debug data, and publication inputs/outputs. | Uses state, output, and debug paths plus run/reuse settings. | File storage and publication are adapters; corrupt, mismatched, or missing reusable state triggers documented rejection/recompute behavior. | Inspect chunk-plan, checkpoint, file I/O, debug, and run-reuse tests; preserve state/output separation, digest identity, and atomic publication assumptions. |
|
|
|
|
## Integration Documentation Plan
|
|
|
|
The repository has four kinds of implemented integration contracts:
|
|
|
|
1. Seriatim transcript input (`docs/integrations/seriatim.md`).
|
|
2. Published JSON and chunk-map output
|
|
(`docs/integrations/json-output.md`,
|
|
`docs/integrations/chunk-map.md`).
|
|
3. Spell-catalog reference input
|
|
(`docs/integrations/spell-catalog-overlays.md`).
|
|
4. Versioned D&D artifact schemas (the six `dnd-*.md` files).
|
|
|
|
All currently use repository-defined v1 contracts where a version is exposed.
|
|
The project is pre-release and intentionally does not promise backward
|
|
compatibility beyond what the current parser or schema explicitly accepts.
|
|
Documentation must describe the current accepted/emitted format and avoid
|
|
inventing migration guarantees.
|
|
|
|
For each contract, document the external system or durable file format, how
|
|
this project consumes or emits it, field and compatibility rules knowable from
|
|
code, and one minimal valid example. Do not document provider APIs, internal
|
|
prompt wording, implementation call graphs, speculative schema versions, or
|
|
unimplemented consumers. The per-file sources and acceptance criteria in
|
|
“Target Documentation Set” are controlling for the rewrite.
|
|
|
|
## Recommended Implementation Sequence
|
|
|
|
### Stage 1 — Establish maintained example coverage
|
|
|
|
**Goal:** Make the two intended user journeys concrete before rewriting prose.
|
|
|
|
**Files:** Create `examples/dnd-complete-transcript.json`; lightly update the
|
|
other `examples/*` assets only as needed for internal consistency; update the
|
|
existing maintained-example test.
|
|
|
|
**Inspect:** Seriatim adapter/fixtures, both config examples, all D&D artifact
|
|
contracts, example contract tests.
|
|
|
|
**Acceptance criteria:**
|
|
|
|
- Exactly two `*.config.yml` examples remain.
|
|
- Both configs load, resolve, and list successfully.
|
|
- Both transcript examples parse through production input code.
|
|
- The complete transcript contains source evidence relevant to every
|
|
configured D&D lane.
|
|
- No secret, provider call, or brittle expected LLM prose is introduced.
|
|
|
|
**Validation:**
|
|
|
|
```sh
|
|
go test ./internal/cli -run 'TestMaintainedExamplesLoadResolveAndList|TestMaintainedConfigurationExampleSet'
|
|
go run ./cmd/notarius config validate --config examples/dnd-minimal.config.yml --pipeline dnd-session
|
|
OPENROUTER_API_KEY=validation-placeholder go run ./cmd/notarius config validate --config examples/dnd-complete.config.yml --pipeline dnd-session
|
|
```
|
|
|
|
**Prompt sizing:** Small enough for one implementation prompt.
|
|
|
|
### Stage 2 — Rewrite orientation and CLI documentation
|
|
|
|
**Goal:** Provide an accurate entry point and command contract.
|
|
|
|
**Files:** Rewrite `README.md` and `docs/cli.md`; lightly update
|
|
`docs/development.md`.
|
|
|
|
**Inspect:** `cmd/notarius`, `internal/cli`, command/help/exit contract tests,
|
|
both examples.
|
|
|
|
**Acceptance criteria:**
|
|
|
|
- README reflects the implemented staged six-lane D&D workflow and has a
|
|
working minimal quickstart.
|
|
- Every CLI command, flag, default, output stream, and exit claim matches code
|
|
or tests.
|
|
- Invalid slot examples and removed-flag history are gone.
|
|
- Contributor routing points to existing or planned canonical documents
|
|
without duplicating them.
|
|
|
|
**Validation:**
|
|
|
|
```sh
|
|
go build ./cmd/notarius
|
|
go test ./internal/cli
|
|
go run ./cmd/notarius help
|
|
go run ./cmd/notarius config validate --help
|
|
go run ./cmd/notarius pipelines list --help
|
|
go run ./cmd/notarius run --help
|
|
```
|
|
|
|
Account for the parser's documented nonzero usage exit when invoking
|
|
subcommand `--help`; do not treat that expected behavior as a failed doc check.
|
|
|
|
**Prompt sizing:** Small enough for one implementation prompt.
|
|
|
|
### Stage 3 — Rewrite the public configuration reference
|
|
|
|
**Goal:** Make `docs/config.md` a complete, current user contract and remove
|
|
implementation/history material.
|
|
|
|
**Files:** Rewrite `docs/config.md`.
|
|
|
|
**Inspect:** `internal/core/config`, effective/redaction tests, pipeline
|
|
configuration/resolution, all production registries and D&D registration
|
|
chains, both configuration examples.
|
|
|
|
**Acceptance criteria:**
|
|
|
|
- All current fields, defaults, precedence rules, aliases, selectable keys,
|
|
reference slots, validator chains, and rejection rules match code.
|
|
- Version migration and removed-option history are absent.
|
|
- Generated references and ordered steps are explained at contract level.
|
|
- Internal algorithms are replaced with links/placeholders to the target
|
|
internal docs, without linking to nonexistent files at stage completion.
|
|
Create links only when their target exists, or land the minimal target stub
|
|
in the same stage.
|
|
|
|
**Validation:**
|
|
|
|
```sh
|
|
go test ./internal/core/config ./internal/framework/pipeline ./internal/modules/dnd/register
|
|
go test ./internal/cli -run 'TestMaintainedExamplesLoadResolveAndList|TestProduction'
|
|
```
|
|
|
|
Also compare every documented key and default chain to registry construction
|
|
in code.
|
|
|
|
**Prompt sizing:** This is the largest single-document stage but is coherent
|
|
and suitable for one focused implementation prompt. Do not combine it with
|
|
another rewrite.
|
|
|
|
### Stage 4 — Document configuration and CLI internals
|
|
|
|
**Goal:** Move maintainer mechanics out of the public references.
|
|
|
|
**Files:** Create `docs/internal/configuration.md` and
|
|
`docs/internal/cli.md`.
|
|
|
|
**Inspect:** `internal/core/config`, `internal/cli`, their tests, registration
|
|
composition, config redaction/digest code.
|
|
|
|
**Acceptance criteria:**
|
|
|
|
- Load/resolution phases, precedence, redaction, digests, composition, and
|
|
failure mapping match implementation.
|
|
- Each document names its inputs, outputs, boundaries, failure behavior,
|
|
invariants, and focused tests.
|
|
- User-facing fields and syntax are linked rather than duplicated.
|
|
|
|
**Validation:**
|
|
|
|
```sh
|
|
go test ./internal/core/config ./internal/cli
|
|
```
|
|
|
|
**Prompt sizing:** Small enough for one implementation prompt because the two
|
|
documents describe adjacent sides of the same composition boundary.
|
|
|
|
### Stage 5 — Rewrite operations and state documentation
|
|
|
|
**Goal:** Give operators one coherent lifecycle/recovery guide and maintainers
|
|
one coherent state implementation guide.
|
|
|
|
**Files:** Rewrite `docs/operations.md`; lightly update
|
|
`docs/internal/state.md`.
|
|
|
|
**Inspect:** CLI run orchestration, file I/O and debug bundles, chunk-plan and
|
|
checkpoint stores, reuse/recompute code, state/runner/CLI tests, ADRs 0005 and
|
|
0007.
|
|
|
|
**Acceptance criteria:**
|
|
|
|
- Paths, lifecycle, reason codes, resume/reuse, recomputation, publication,
|
|
cleanup, and recovery match code and tests.
|
|
- Operator commands and maintainer mechanics have distinct canonical homes.
|
|
- Any cleanup command uses a narrow explicit target and is described as
|
|
destructive.
|
|
|
|
**Validation:**
|
|
|
|
```sh
|
|
go test ./internal/cli ./internal/core/... ./internal/framework/...
|
|
```
|
|
|
|
**Prompt sizing:** Small enough for one implementation prompt if limited to
|
|
state/operations ownership; do not also rewrite pipeline mechanics.
|
|
|
|
### Stage 6 — Rewrite pipeline and component overview
|
|
|
|
**Goal:** Accurately document resolution and execution mechanics and provide a
|
|
concise component map.
|
|
|
|
**Files:** Rewrite `docs/internal/pipeline.md` and
|
|
`docs/internal/overview.md`.
|
|
|
|
**Inspect:** pipeline resolver, preparation, runner, checkpoint hooks,
|
|
artifact/reference types, concurrency code and tests, architecture policy.
|
|
|
|
**Acceptance criteria:**
|
|
|
|
- Fixed stages, ordered steps, dependency/reference flow, preparation,
|
|
checkpoints, concurrency, and fail-whole-run behavior match code.
|
|
- Historical “legacy” framing is absent.
|
|
- Overview links to focused internal documents and does not duplicate them.
|
|
- Descriptive docs do not introduce new architecture policy.
|
|
|
|
**Validation:**
|
|
|
|
```sh
|
|
go test ./internal/framework/pipeline/... ./internal/cli
|
|
```
|
|
|
|
Adjust the package pattern to the actual Go package layout if the ellipsis form
|
|
is not accepted.
|
|
|
|
**Prompt sizing:** Small enough for one implementation prompt.
|
|
|
|
### Stage 7 — Split generic and D&D module documentation
|
|
|
|
**Goal:** Produce maintainable extension guidance and codify cross-extractor
|
|
conventions.
|
|
|
|
**Files:** Rewrite `docs/internal/modules.md`; create
|
|
`docs/internal/dnd.md`.
|
|
|
|
**Inspect:** all module registries, generic helpers/assets, Seriatim modules,
|
|
all six D&D extractors, registration chains, prompt/codec/normalizer/validator
|
|
tests.
|
|
|
|
**Acceptance criteria:**
|
|
|
|
- Generic registration/extension mechanics have one canonical owner.
|
|
- Every convention shared by the D&D extractors is documented, and intentional
|
|
deviations are explicit.
|
|
- Prompt-cache ordering, shared assets, minimal raw schemas, deterministic
|
|
normalization, evidence rules, generated references, and combat routing
|
|
match code.
|
|
- Wire schemas are links to integration docs, not copies.
|
|
|
|
**Validation:**
|
|
|
|
```sh
|
|
go test ./internal/modules/...
|
|
```
|
|
|
|
**Prompt sizing:** This touches a broad code area. Split it into two sequential
|
|
implementation prompts:
|
|
|
|
1. rewrite generic `modules.md`;
|
|
2. create and verify D&D-specific `dnd.md`.
|
|
|
|
### Stage 8 — Rewrite LLM boundary documentation
|
|
|
|
**Goal:** Give maintainers an accurate provider-independent LLM execution and
|
|
prompt-maintenance guide.
|
|
|
|
**Files:** Rewrite `docs/internal/llm.md`.
|
|
|
|
**Inspect:** LLM framework, Scriptorium adapter, scheduler, retry/repair flow,
|
|
prompt builders/assets, validation, observability, and tests.
|
|
|
|
**Acceptance criteria:**
|
|
|
|
- Adapter, scheduling, concurrency, prompt assembly, caching, validation, and
|
|
failure behavior match code.
|
|
- Stable prompt assets and stable-before-variable ordering are documented.
|
|
- No exact-prefix-length change-detector test is proposed.
|
|
- Credentials/config fields and artifact schemas are linked to their canonical
|
|
references.
|
|
|
|
**Validation:**
|
|
|
|
```sh
|
|
go test ./internal/framework/llm/... ./internal/modules/dnd/...
|
|
```
|
|
|
|
Adjust package patterns to the actual layout if necessary.
|
|
|
|
**Prompt sizing:** Small enough for one implementation prompt.
|
|
|
|
### Stage 9 — Harmonize foundational integration contracts
|
|
|
|
**Goal:** Align input, output, chunk-map, and overlay contracts to the common
|
|
integration outline.
|
|
|
|
**Files:** Update `docs/integrations/seriatim.md`,
|
|
`docs/integrations/json-output.md`, `docs/integrations/chunk-map.md`, and
|
|
`docs/integrations/spell-catalog-overlays.md`.
|
|
|
|
**Inspect:** corresponding codecs/loaders/types, output encoder, chunkers,
|
|
fixtures, examples, and contract tests.
|
|
|
|
**Acceptance criteria:**
|
|
|
|
- Every field and compatibility statement is supported by code/tests.
|
|
- Published output is distinguished from operational state.
|
|
- Minimal examples parse or encode correctly.
|
|
- Internal algorithms and future formats are absent.
|
|
|
|
**Validation:**
|
|
|
|
```sh
|
|
go test ./internal/modules/seriatim/... ./internal/framework/... ./internal/cli
|
|
```
|
|
|
|
Use the repository's actual package paths for focused commands.
|
|
|
|
**Prompt sizing:** Small enough for one implementation prompt.
|
|
|
|
### Stage 10 — Harmonize D&D identity and event contracts
|
|
|
|
**Goal:** Apply one external-contract style to all six D&D artifacts.
|
|
|
|
**Files:** Rewrite `docs/integrations/dnd-spell.md`,
|
|
`docs/integrations/dnd-npc.md`,
|
|
`docs/integrations/dnd-npc-interaction.md`,
|
|
`docs/integrations/dnd-combat-turn.md`,
|
|
`docs/integrations/dnd-item-event.md`, and
|
|
`docs/integrations/dnd-scene-description.md`.
|
|
|
|
**Inspect:** each schema/type, codec, normalizer, validator, registration
|
|
chain, routing rule, and contract test.
|
|
|
|
**Acceptance criteria:**
|
|
|
|
- Required fields, enums, source references, ordering, deterministic fields,
|
|
identity, and version semantics are exact.
|
|
- Minimal-extraction policy is preserved.
|
|
- Related artifacts are clearly distinguished and cross-linked.
|
|
- No validator implementation, prompt text, removed synthesis fields, or
|
|
unimplemented behavior is presented as contract.
|
|
|
|
**Validation:**
|
|
|
|
```sh
|
|
go test ./internal/modules/dnd/...
|
|
```
|
|
|
|
**Prompt sizing:** Split into two sequential prompts:
|
|
|
|
1. spell, NPC identity, and scene-description contracts;
|
|
2. NPC-interaction, combat-turn, and item-event contracts.
|
|
|
|
### Stage 11 — Final policy, ADR, roadmap, and link audit
|
|
|
|
**Goal:** Verify the final tree as a coherent whole and retire this completed
|
|
roadmap.
|
|
|
|
**Files:** Lightly update policy files, ADRs, and
|
|
`docs/roadmap/future.md` only when a link/status/current-versus-future error is
|
|
found; delete `docs/roadmap/documentation.md` after all other checks pass.
|
|
|
|
**Inspect:** final documentation tree, all registries, accepted ADRs, future
|
|
roadmap, examples, local links.
|
|
|
|
**Acceptance criteria:**
|
|
|
|
- Every documented fact has one canonical owner.
|
|
- All local file links and manually reviewed heading anchors resolve.
|
|
- No non-roadmap document presents planned or unimplemented behavior as
|
|
current.
|
|
- No current reference contains migration/changelog framing or unsupported
|
|
deprecated terminology.
|
|
- ADR statuses and supersession links are correct.
|
|
- `future.md` contains only unimplemented work.
|
|
- The completed documentation roadmap is deleted last.
|
|
|
|
**Validation:**
|
|
|
|
```sh
|
|
go test ./...
|
|
go vet ./...
|
|
go build ./cmd/notarius
|
|
git diff --check
|
|
rg -n 'Version 2 To Version 3|diagnostics-dir|chunk\.scene_guide|merge_notes|normalization_notes' README.md docs examples
|
|
rg -n '\b(planned|future|deprecated|removed|migration|legacy)\b' README.md docs examples
|
|
```
|
|
|
|
Review every second search result rather than assuming all occurrences are
|
|
wrong: ADRs, the active future roadmap, and currently accepted aliases may use
|
|
such words legitimately. Run a local Markdown target check and manually test
|
|
heading anchors because no repository-owned automated link checker currently
|
|
exists.
|
|
|
|
**Prompt sizing:** Small enough for one implementation prompt. It is a
|
|
validation/cleanup stage, not an invitation to redesign documents already
|
|
accepted in prior stages.
|
|
|
|
## Validation Plan
|
|
|
|
Run validation incrementally, not only after the final rewrite.
|
|
|
|
### Repository and behavior checks
|
|
|
|
```sh
|
|
go test ./...
|
|
go vet ./...
|
|
go build ./cmd/notarius
|
|
git diff --check
|
|
```
|
|
|
|
These catch code/example drift and malformed patches. Documentation-only
|
|
stages should still run the most focused tests named above; the full suite is
|
|
required at the end.
|
|
|
|
### Maintained configuration checks
|
|
|
|
```sh
|
|
go test ./internal/cli -run 'TestMaintainedExamplesLoadResolveAndList|TestMaintainedConfigurationExampleSet|TestProduction'
|
|
go run ./cmd/notarius config validate --config examples/dnd-minimal.config.yml --pipeline dnd-session
|
|
OPENROUTER_API_KEY=validation-placeholder go run ./cmd/notarius config validate --config examples/dnd-complete.config.yml --pipeline dnd-session
|
|
go run ./cmd/notarius pipelines list --config examples/dnd-minimal.config.yml
|
|
OPENROUTER_API_KEY=validation-placeholder go run ./cmd/notarius pipelines list --config examples/dnd-complete.config.yml
|
|
```
|
|
|
|
The placeholder credential is only for offline load/resolution validation.
|
|
Do not run provider-backed extraction as a documentation test.
|
|
|
|
### CLI verification
|
|
|
|
Build a temporary binary and compare its root and subcommand help, error
|
|
messages, and exit statuses with `docs/cli.md`. Include:
|
|
|
|
- no arguments;
|
|
- `help`, `--help`, and `-h`;
|
|
- `run --help`, `config validate --help`, and `pipelines list --help`;
|
|
- missing required flags;
|
|
- missing/invalid config;
|
|
- unknown command.
|
|
|
|
Use the parser's actual contract tests as the authority. Do not normalize
|
|
different exit statuses in prose merely because the displayed usage is
|
|
similar.
|
|
|
|
### Contract and registry checks
|
|
|
|
- Compare every key table in `docs/config.md` to production registry
|
|
registration.
|
|
- Compare every default collaborator/validator chain to
|
|
`internal/modules/dnd/register/chains.go` and its tests.
|
|
- Compare every integration schema and enum to production types, codecs,
|
|
schemas, validators, and contract tests.
|
|
- Parse JSON examples with production loaders where available; do not rely
|
|
only on generic JSON syntax checks.
|
|
- Verify generated-reference producer/consumer slots against resolver tests
|
|
and `examples/dnd-complete.config.yml`.
|
|
|
|
### Documentation ownership and stale-content checks
|
|
|
|
```sh
|
|
rg -n 'Version 2 To Version 3|diagnostics-dir|chunk\.scene_guide|merge_notes|normalization_notes' README.md docs examples
|
|
rg -n '\b(planned|future|deprecated|removed|migration|legacy)\b' README.md docs examples
|
|
rg -n 'spell-only|only.*spell|extracts? D&D spells' README.md docs
|
|
```
|
|
|
|
Manually classify results in ADRs and roadmaps. Also sample repeated topics
|
|
(configuration defaults, state layout, artifact schemas, prompt ordering) to
|
|
confirm that one document owns the detail and other documents link to it.
|
|
|
|
### Link checks
|
|
|
|
No automated documentation checker is currently evident. At minimum:
|
|
|
|
- enumerate every relative Markdown link and verify that its target file
|
|
exists;
|
|
- manually verify heading fragments after headings are rewritten;
|
|
- inspect external links for relevance and prefer stable upstream project
|
|
pages;
|
|
- verify that examples linked from prose exist at the exact path and still
|
|
pass the example tests.
|
|
|
|
Adding a prose-snapshot or exact-heading test is not recommended. If the
|
|
project later adopts a link checker, configure it to validate links rather
|
|
than freeze document wording or length.
|
|
|
|
### Manual policy review
|
|
|
|
Before deleting this roadmap, confirm:
|
|
|
|
- README is orientation, not a reference manual.
|
|
- CLI, config, operations, integration, and internal docs each stay within
|
|
their policy-owned scope.
|
|
- Architecture policy remains normative and internal docs remain descriptive.
|
|
- Current docs contain implemented behavior only.
|
|
- Future work exists only under `docs/roadmap/`.
|
|
- Examples are copyable, maintained, secret-free, and tested.
|
|
- No document uses changelog or development-history framing.
|
|
|
|
## Open Questions
|
|
|
|
None. Repository inspection supports a decision-complete migration plan. The
|
|
new complete transcript and the three focused internal documents are
|
|
recommended additions; the HTTP API, consumer-library, and separate provider
|
|
integration documents are not applicable to the current implementation.
|