Clean up completed documentation roadmaps
This commit is contained in:
@@ -24,6 +24,13 @@ Useful references:
|
||||
|
||||
- [CLI reference](docs/cli.md)
|
||||
- [Configuration reference](docs/config.md)
|
||||
- [Operations](docs/operations.md)
|
||||
- [Troubleshooting](docs/troubleshooting.md)
|
||||
- [Seriatim input contract](docs/integrations/seriatim.md)
|
||||
- [OpenAI-compatible provider contract](docs/integrations/openai-compatible.md)
|
||||
- [JSON output contract](docs/integrations/json-output.md)
|
||||
- [D&D spell artifact contract](docs/integrations/dnd-spell-artifacts.md)
|
||||
- [Developer workflow](docs/policy/development.md)
|
||||
- [Internal architecture docs](docs/internal/overview.md)
|
||||
- [Maintained example config](examples/dnd-spells.config.yml)
|
||||
- [Maintained example input](examples/seriatim-minimal-transcript.json)
|
||||
|
||||
@@ -1,759 +0,0 @@
|
||||
# Documentation Roadmap
|
||||
|
||||
## Purpose
|
||||
|
||||
This roadmap defines the work required to bring Notarius documentation into
|
||||
compliance with [`docs/policy/documentation.md`](../policy/documentation.md) and
|
||||
the current implementation. It is a planning document only. Future implementation
|
||||
agents should use it to rewrite documentation in stages without documenting
|
||||
unimplemented behavior outside `docs/roadmap/`.
|
||||
|
||||
## Repository Documentation Inventory
|
||||
|
||||
- `README.md`: keep and rewrite. It still contains template text
|
||||
(`go-application-template`) and does not describe the implemented Notarius CLI.
|
||||
- `AGENTS.md`: keep and lightly update if needed. It correctly directs agents to
|
||||
read `docs/policy`, and should stay short.
|
||||
- `docs/policy/documentation.md`: keep and do not update. It is the controlling documentation policy.
|
||||
- `docs/policy/architecture.md`: keep and do not update. It is the controlling architecture policy.
|
||||
- `docs/integrations/seriatim.md`: keep and lightly update. It is close to the
|
||||
current Seriatim adapter behavior and belongs in `docs/integrations/` because
|
||||
Seriatim minimal transcript JSON is an external file-format contract.
|
||||
- `docs/integrations/dnd-spells.md`: split. It mixes implemented extractor
|
||||
internals, artifact payload details, and stale roadmap text claiming the CLI
|
||||
`run` workflow does not exist. Move module-internal behavior to
|
||||
`docs/internal/modules.md` or a module-specific internal doc; move durable
|
||||
spell artifact output shape to an integration/artifact contract doc.
|
||||
- `docs/roadmap/documentation.md`: keep during the documentation migration. After
|
||||
the migration is complete, either delete it or reduce it to any remaining
|
||||
documentation work only.
|
||||
- `docs/roadmap/implementation.md`: delete after migration. It is a completed MVP
|
||||
implementation plan and is no longer an active roadmap.
|
||||
- `docs/roadmap/initial-architecture.md`: delete after migration. The implemented
|
||||
architecture now belongs in policy and internal docs; historical planning
|
||||
detail should not remain as active roadmap content.
|
||||
- `docs/roadmap/mvp.md`: delete or rewrite after migration. It currently records
|
||||
implemented MVP behavior and deferred documentation work; once current-behavior
|
||||
docs exist, keep only genuine future work under `docs/roadmap/`.
|
||||
- `examples/dnd-spells.config.yml`: keep and lightly update. It is a maintained
|
||||
example covered by CLI tests, but should be checked for user-facing clarity and
|
||||
secret-free LLM configuration.
|
||||
- `examples/seriatim-minimal-transcript.json`: keep and lightly update. It is a
|
||||
maintained Seriatim input fixture covered by CLI tests.
|
||||
|
||||
## Policy Compliance Assessment
|
||||
|
||||
Required documents missing under the documentation policy:
|
||||
|
||||
- `docs/cli.md`: required because Notarius is a CLI application.
|
||||
- `docs/config.md`: required because Notarius is config-driven.
|
||||
- `docs/operations.md`: required because Notarius runs multi-step workflows,
|
||||
writes durable artifacts, and maintains diagnostics directories.
|
||||
- `docs/internal/`: required because Notarius is modular and orchestration-based.
|
||||
- `docs/policy/development.md`: required because the project is maintained by
|
||||
humans and LLM coding agents.
|
||||
|
||||
Recommended documents that should be added:
|
||||
|
||||
- `docs/troubleshooting.md`: recommended because implemented commands have common
|
||||
failure modes: missing config, unknown pipeline, invalid lane selection,
|
||||
invalid Seriatim input, LLM configuration failures, provider failures, output
|
||||
write failures, and diagnostics retention behavior.
|
||||
- Maintained examples under `examples/`: already present and should remain
|
||||
linked from README, CLI, config, and integration docs.
|
||||
|
||||
Documents that exist but are stale or in the wrong canonical home:
|
||||
|
||||
- `README.md` is stale template content.
|
||||
- `docs/integrations/dnd-spells.md` includes stale limits and belongs partly in
|
||||
internal/module docs and partly in artifact integration docs.
|
||||
- Completed roadmap documents describe implemented behavior and should not be the
|
||||
only source of current-behavior documentation after the migration.
|
||||
|
||||
Content describing deprecated, historical, planned, or unimplemented behavior
|
||||
outside `docs/roadmap/`:
|
||||
|
||||
- `docs/integrations/dnd-spells.md` says the CLI `run` workflow does not exist;
|
||||
that is now false.
|
||||
- No other non-roadmap stale claims were found during this pass, but future
|
||||
implementation agents should grep before rewriting.
|
||||
|
||||
Examples:
|
||||
|
||||
- `examples/dnd-spells.config.yml` and
|
||||
`examples/seriatim-minimal-transcript.json` are covered by
|
||||
`internal/cli/run_test.go`.
|
||||
- There is no maintained expected-output example under `examples/`. Do not add
|
||||
one unless it can be validated or regenerated from implemented behavior.
|
||||
|
||||
Links likely needing verification:
|
||||
|
||||
- New README links to `docs/cli.md`, `docs/config.md`,
|
||||
`docs/operations.md`, `docs/troubleshooting.md`, `docs/internal/`, and
|
||||
`docs/integrations/` will need verification after those files are created.
|
||||
- Existing roadmap links to old implementation documents should be removed or
|
||||
deleted with the roadmap cleanup stage.
|
||||
|
||||
Documents not applicable:
|
||||
|
||||
- `docs/api.md` is not needed because Notarius does not implement a public HTTP
|
||||
API.
|
||||
- `docs/consumers/` is not needed for this pass because the repository exposes a
|
||||
CLI, not a supported public package, SDK, plugin API, or HTTP consumer API.
|
||||
|
||||
## Target Documentation Set
|
||||
|
||||
Desired final tree after the migration:
|
||||
|
||||
```text
|
||||
AGENTS.md
|
||||
README.md
|
||||
docs/cli.md
|
||||
docs/config.md
|
||||
docs/operations.md
|
||||
docs/troubleshooting.md
|
||||
docs/policy/architecture.md
|
||||
docs/policy/development.md
|
||||
docs/policy/documentation.md
|
||||
docs/internal/overview.md
|
||||
docs/internal/pipeline.md
|
||||
docs/internal/modules.md
|
||||
docs/internal/llm.md
|
||||
docs/internal/diagnostics.md
|
||||
docs/integrations/seriatim.md
|
||||
docs/integrations/openai-compatible.md
|
||||
docs/integrations/json-output.md
|
||||
docs/integrations/dnd-spell-artifacts.md
|
||||
docs/roadmap/mvp.md
|
||||
examples/dnd-spells.config.yml
|
||||
examples/seriatim-minimal-transcript.json
|
||||
```
|
||||
|
||||
`docs/roadmap/mvp.md` should exist only if it contains genuine remaining future
|
||||
work after the migration. `docs/roadmap/documentation.md` exists to drive this
|
||||
migration and should be deleted or reduced when the migration is complete.
|
||||
|
||||
### `AGENTS.md`
|
||||
|
||||
- Audience: LLM coding agents.
|
||||
- Purpose: shortest repository-specific agent instruction.
|
||||
- Canonical scope: policy files that agents must read before changes.
|
||||
- Recommended outline: one or two sentences linking to policy docs.
|
||||
- Source-of-truth areas: `docs/policy/architecture.md`,
|
||||
`docs/policy/documentation.md`.
|
||||
- Acceptance criteria: stays short; no duplicated architecture or documentation
|
||||
policy; links remain accurate.
|
||||
|
||||
### `README.md`
|
||||
|
||||
- Audience: users, administrators, operators.
|
||||
- Purpose: concise outward-facing orientation.
|
||||
- Canonical scope: project purpose, shortest useful command, and links to
|
||||
targeted docs.
|
||||
- Recommended outline: description; elevator pitch; shortest useful command
|
||||
using maintained examples; where output appears; links.
|
||||
- Source-of-truth areas: `cmd/notarius/main.go`, `internal/cli/run.go`,
|
||||
`examples/dnd-spells.config.yml`,
|
||||
`examples/seriatim-minimal-transcript.json`,
|
||||
`internal/cli/run_test.go`.
|
||||
- Acceptance criteria: no template text; no long manual content; commands match
|
||||
real CLI syntax; links resolve; no unimplemented features.
|
||||
|
||||
### `docs/policy/architecture.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: durable development architecture policy.
|
||||
- Canonical scope: stable architecture invariants and boundaries.
|
||||
- Recommended outline: keep the current concise policy shape.
|
||||
- Source-of-truth areas: current architecture policy and actual package
|
||||
boundaries.
|
||||
- Acceptance criteria: does not grow into CLI/config/reference documentation;
|
||||
no planned behavior outside roadmap language.
|
||||
|
||||
### `docs/policy/documentation.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: controlling documentation policy.
|
||||
- Canonical scope: documentation layout, audience, maintenance rules, and
|
||||
canonical homes.
|
||||
- Recommended outline: keep current policy unless project-wide documentation
|
||||
rules change.
|
||||
- Source-of-truth areas: current documentation policy and this migration.
|
||||
- Acceptance criteria: remains policy-oriented; no project-specific
|
||||
current-behavior reference material that belongs elsewhere.
|
||||
|
||||
### `docs/cli.md`
|
||||
|
||||
- Audience: users, administrators, operators.
|
||||
- Purpose: canonical CLI reference.
|
||||
- Canonical scope: implemented commands, flags, exit behavior, common workflows,
|
||||
and command output.
|
||||
- Recommended outline: shortest useful command; command overview; `help`;
|
||||
`run`; `config validate`; `pipelines list`; exit codes; common workflows;
|
||||
diagnostics/output pointers.
|
||||
- Source-of-truth areas: `internal/cli/run.go`, `internal/cli/run_test.go`,
|
||||
`internal/cli/catalog.go`.
|
||||
- Acceptance criteria: documents only implemented commands and flags; explains
|
||||
`--config`, `--input`, `--only`, `--output-dir`, `--diagnostics-dir`,
|
||||
`--llm-profile`, `--pipeline`, and `--json`; does not duplicate full config
|
||||
reference; examples are covered by tests or are direct variants of tested
|
||||
commands.
|
||||
|
||||
### `docs/config.md`
|
||||
|
||||
- Audience: administrators, operators, advanced users.
|
||||
- Purpose: canonical configuration reference.
|
||||
- Canonical scope: config discovery, YAML version 1 structure, defaults,
|
||||
environment overrides, pipeline profiles, module bindings, LLM profiles,
|
||||
diagnostics settings, validation behavior, and secrets handling.
|
||||
- Recommended outline: config discovery and precedence; minimal working config;
|
||||
production-oriented config; `version`; `llm_profiles`; `pipelines`;
|
||||
module-binding shorthand and object form; `concurrency`; `diagnostics`;
|
||||
environment overrides; validation and capability checks; examples.
|
||||
- Source-of-truth areas: `internal/core/config/*.go`,
|
||||
`internal/framework/pipeline/profile.go`, `internal/framework/pipeline/module.go`,
|
||||
`internal/cli/run.go`, `examples/dnd-spells.config.yml`, config tests.
|
||||
- Acceptance criteria: full config reference is here and not duplicated
|
||||
elsewhere; lists implemented defaults and supported env vars; explains that
|
||||
raw secrets should not be committed; examples load successfully.
|
||||
|
||||
### `docs/operations.md`
|
||||
|
||||
- Audience: administrators, operators.
|
||||
- Purpose: canonical operational behavior and recovery reference.
|
||||
- Canonical scope: run output directory layout, diagnostics directory layout,
|
||||
retention behavior, manifests, warnings, failure inspection, safe cleanup, and
|
||||
operational caveats.
|
||||
- Recommended outline: normal run workflow; output files; diagnostics files;
|
||||
retention modes; failure behavior; warnings; atomic writes and path safety;
|
||||
safe cleanup; what is not resumable yet.
|
||||
- Source-of-truth areas: `internal/cli/run.go`,
|
||||
`internal/core/diagnostics/*.go`, `internal/modules/output/json/encoder.go`,
|
||||
output and diagnostics tests.
|
||||
- Acceptance criteria: accurately names implemented durable files; states that
|
||||
resume/retry is not implemented as a command; explains when diagnostics are
|
||||
retained; does not describe unimplemented remote storage or archives.
|
||||
|
||||
### `docs/troubleshooting.md`
|
||||
|
||||
- Audience: users, administrators, operators.
|
||||
- Purpose: task-oriented fixes for recurring failure modes.
|
||||
- Canonical scope: symptoms, likely causes, inspection steps, and safe fixes.
|
||||
- Recommended outline: config not found; unsupported config version; unknown
|
||||
pipeline; invalid `--only`; Seriatim validation failure; missing LLM base URL
|
||||
or model; provider HTTP failures; output write failure; diagnostics retention
|
||||
surprise.
|
||||
- Source-of-truth areas: `internal/cli/run.go`, `internal/cli/run_test.go`,
|
||||
`internal/core/config/*_test.go`,
|
||||
`internal/modules/input/seriatim/*_test.go`,
|
||||
`internal/framework/llm/*_test.go`.
|
||||
- Acceptance criteria: every entry maps to implemented behavior; no speculative
|
||||
troubleshooting for future modules; links to CLI/config/operations docs
|
||||
instead of duplicating reference material.
|
||||
|
||||
### `docs/policy/development.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: contributor workflow and change discipline.
|
||||
- Canonical scope: repository layout, build/test commands, coding conventions,
|
||||
dependency policy, how to add config fields, CLI flags, modules, adapters, and
|
||||
examples.
|
||||
- Recommended outline: repository layout; required policy reads; build/test
|
||||
commands; Go conventions; dependency policy; adding CLI flags; adding config;
|
||||
adding stage modules; adding examples; docs update rules.
|
||||
- Source-of-truth areas: `AGENTS.md`, `go.mod`, `internal/cli`,
|
||||
`internal/core/config`, `internal/framework/pipeline`, `internal/modules`,
|
||||
`docs/policy`.
|
||||
- Acceptance criteria: procedural and concise; links to `architecture.md` for
|
||||
invariants; does not restate full CLI/config reference.
|
||||
|
||||
### `docs/internal/overview.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: high-level implemented internals map.
|
||||
- Canonical scope: package roles and import/boundary expectations.
|
||||
- Recommended outline: executable and CLI; core packages; framework packages;
|
||||
module packages; tests and fixtures; where not to put domain/source-specific
|
||||
logic.
|
||||
- Source-of-truth areas: repository tree, `docs/policy/architecture.md`,
|
||||
`internal/cli/catalog.go`.
|
||||
- Acceptance criteria: concise package map; reflects current code; does not
|
||||
become a generated file listing every source file.
|
||||
|
||||
### `docs/internal/pipeline.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: implemented pipeline orchestration reference.
|
||||
- Canonical scope: runner inputs/outputs, stage ordering, registries, resolved
|
||||
profiles, artifact lanes, validation, warnings, and manifest population.
|
||||
- Recommended outline: stage flow; profile resolution; registries and module
|
||||
specs; runner execution; validator behavior; output encoder handoff; failure
|
||||
behavior; tests.
|
||||
- Source-of-truth areas: `internal/framework/contracts/contracts.go`,
|
||||
`internal/framework/pipeline/*.go`, pipeline tests.
|
||||
- Acceptance criteria: explains implemented serial execution; does not promise
|
||||
parallel execution or resume; links to config docs for profile syntax.
|
||||
|
||||
### `docs/internal/modules.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: implemented stage modules and how to add new ones.
|
||||
- Canonical scope: current production modules and their boundaries.
|
||||
- Recommended outline: module contract pattern; `seriatim`; `generic`;
|
||||
`dnd/spells`; `appendorder`; `noop`; `json`; module registration; tests to
|
||||
add for new modules.
|
||||
- Source-of-truth areas: `internal/modules/**`, `internal/cli/catalog.go`,
|
||||
module tests.
|
||||
- Acceptance criteria: documents only implemented modules; moves relevant
|
||||
non-external content from `docs/integrations/dnd-spells.md`; keeps detailed
|
||||
external file-format contracts in `docs/integrations/`.
|
||||
|
||||
### `docs/internal/llm.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: implemented LLM runtime internals.
|
||||
- Canonical scope: structured LLM client contract, OpenAI-compatible adapter,
|
||||
scheduler, schema registry, retry/redaction behavior, and prompt boundary.
|
||||
- Recommended outline: contract; provider request shape at a high level;
|
||||
scheduler; retries and timeouts; structured output decoding; secret redaction;
|
||||
tests.
|
||||
- Source-of-truth areas: `internal/framework/contracts/contracts.go`,
|
||||
`internal/framework/llm/*.go`, `internal/framework/prompt/*.go`,
|
||||
`internal/modules/extract/dnd/spells/assets`.
|
||||
- Acceptance criteria: no provider marketing claims; no API keys; links to
|
||||
OpenAI-compatible integration doc for the external wire contract.
|
||||
|
||||
### `docs/internal/diagnostics.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: implemented diagnostics internals.
|
||||
- Canonical scope: run directory creation, artifact writers, retention decision
|
||||
mechanics, and redacted effective config payloads.
|
||||
- Recommended outline: purpose; inputs and outputs; files written; retention;
|
||||
failure behavior; tests; invariants.
|
||||
- Source-of-truth areas: `internal/core/diagnostics/*.go`,
|
||||
`internal/core/config/redaction.go`, CLI diagnostics tests.
|
||||
- Acceptance criteria: complements `docs/operations.md`; does not duplicate
|
||||
operator procedures.
|
||||
|
||||
### `docs/integrations/seriatim.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: Seriatim minimal transcript input contract.
|
||||
- Canonical scope: accepted JSON shape, validation, source mapping, metadata
|
||||
keys, capabilities, and compatibility limits.
|
||||
- Recommended outline: adapter; accepted shape; validation; source mapping;
|
||||
metadata keys; capabilities; example; limits.
|
||||
- Source-of-truth areas: `internal/modules/input/seriatim/*.go`,
|
||||
`internal/modules/input/seriatim/*_test.go`,
|
||||
`examples/seriatim-minimal-transcript.json`.
|
||||
- Acceptance criteria: matches implemented validation exactly; clearly states
|
||||
only the minimal Seriatim shape is supported.
|
||||
|
||||
### `docs/integrations/openai-compatible.md`
|
||||
|
||||
- Audience: administrators, developers, LLM coding agents.
|
||||
- Purpose: external LLM provider wire contract currently used by Notarius.
|
||||
- Canonical scope: OpenAI-compatible chat completions endpoint assumptions,
|
||||
structured output response format, request/response fields used, retry
|
||||
behavior, and secret handling.
|
||||
- Recommended outline: provider type; base URL handling; endpoint path;
|
||||
request shape summary; response assumptions; retryable statuses; auth header;
|
||||
configuration links; limits.
|
||||
- Source-of-truth areas: `internal/framework/llm/openai_compatible_client.go`,
|
||||
`internal/framework/llm/openai_compatible_client_test.go`,
|
||||
`internal/core/config/effective_config.go`.
|
||||
- Acceptance criteria: documents only what the client implements; avoids
|
||||
copying external API docs wholesale; links to config docs for settings.
|
||||
|
||||
### `docs/integrations/json-output.md`
|
||||
|
||||
- Audience: users, administrators, downstream developers, LLM coding agents.
|
||||
- Purpose: durable JSON output file-format contract.
|
||||
- Canonical scope: output directory contents and JSON file shapes produced by
|
||||
the `json` output module.
|
||||
- Recommended outline: output encoder; file list; `index.json`;
|
||||
`manifest.json`; `artifacts/<type>.json`; `rejected.json`; `warnings.json`;
|
||||
artifact envelope; filename sanitization; limits.
|
||||
- Source-of-truth areas: `internal/modules/output/json/encoder.go`,
|
||||
`internal/core/artifacts/artifacts.go`, output tests,
|
||||
`internal/cli/run_test.go`.
|
||||
- Acceptance criteria: exactly matches implemented file names and shapes; does
|
||||
not document future output formats.
|
||||
|
||||
### `docs/integrations/dnd-spell-artifacts.md`
|
||||
|
||||
- Audience: downstream developers, LLM coding agents.
|
||||
- Purpose: D&D spell-cast artifact payload contract.
|
||||
- Canonical scope: durable `dnd.spell_cast` payload fields, schema version,
|
||||
source-reference expectations, prompt/schema provenance in manifests, and
|
||||
validator rejection reasons.
|
||||
- Recommended outline: artifact type; payload fields; source references;
|
||||
validators; manifest metadata; example payload; limits.
|
||||
- Source-of-truth areas: `internal/modules/extract/dnd/spells/*.go`,
|
||||
`internal/modules/extract/dnd/spells/assets/schemas/dnd_spells.v1.json`,
|
||||
spell extractor tests.
|
||||
- Acceptance criteria: replaces the external-contract parts of
|
||||
`docs/integrations/dnd-spells.md`; no stale statement that CLI run is missing;
|
||||
no item/NPC/combat claims outside limitations.
|
||||
|
||||
### `docs/roadmap/mvp.md`
|
||||
|
||||
- Audience: maintainers, developers, LLM coding agents.
|
||||
- Purpose: remaining roadmap work only, if any remains after the documentation
|
||||
migration.
|
||||
- Canonical scope: future, planned, deferred, or unimplemented work.
|
||||
- Recommended outline: remaining future work; deferred ideas; release checklist,
|
||||
if still useful.
|
||||
- Source-of-truth areas: current code and this documentation roadmap.
|
||||
- Acceptance criteria: contains no current-behavior reference material that
|
||||
belongs in README, CLI, config, operations, internal, or integration docs.
|
||||
|
||||
### `examples/dnd-spells.config.yml`
|
||||
|
||||
- Audience: users, administrators, operators, LLM coding agents.
|
||||
- Purpose: maintained minimal config example for the implemented spell
|
||||
extraction workflow.
|
||||
- Canonical scope: one valid, secret-free config example.
|
||||
- Recommended outline: YAML fixture only, with comments only if they do not
|
||||
interfere with tests.
|
||||
- Source-of-truth areas: config parser/tests and CLI example-fixture tests.
|
||||
- Acceptance criteria: remains loadable by tests; contains no secrets; is linked
|
||||
from README, CLI, and config docs.
|
||||
|
||||
### `examples/seriatim-minimal-transcript.json`
|
||||
|
||||
- Audience: users, developers, LLM coding agents.
|
||||
- Purpose: maintained minimal input example for the Seriatim adapter.
|
||||
- Canonical scope: one valid minimal transcript JSON fixture.
|
||||
- Recommended outline: JSON fixture only.
|
||||
- Source-of-truth areas: Seriatim adapter/tests and CLI example-fixture tests.
|
||||
- Acceptance criteria: remains valid; contains no private transcript data; is
|
||||
linked from README, CLI, and Seriatim integration docs.
|
||||
|
||||
## File-by-File Rewrite Guidance
|
||||
|
||||
- `README.md`: cover what Notarius does, the shortest useful command, where
|
||||
outputs go, and links. Avoid package diagrams, full config reference, and
|
||||
roadmap content. Inspect `internal/cli/run.go`, `examples/`, and CLI tests.
|
||||
Do not carry forward `go-application-template`.
|
||||
- `docs/cli.md`: cover implemented commands and flags. Avoid full YAML field
|
||||
reference and internal package details. Link to `docs/config.md` for config
|
||||
fields and `docs/operations.md` for output/diagnostics.
|
||||
- `docs/config.md`: cover config schema, defaults, discovery, env overrides,
|
||||
validation, and secrets. Avoid command walkthroughs except minimal examples.
|
||||
Link to `docs/cli.md` for command syntax and integration docs for module
|
||||
contracts.
|
||||
- `docs/operations.md`: cover output and diagnostics behavior, retention, and
|
||||
failure inspection. Avoid developer implementation details that belong in
|
||||
`docs/internal/diagnostics.md`.
|
||||
- `docs/troubleshooting.md`: cover recurring implemented failures. Avoid generic
|
||||
LLM advice not grounded in Notarius errors or tests.
|
||||
- `docs/policy/development.md`: cover contributor workflow. Avoid copying
|
||||
`docs/policy/architecture.md`; link to it for invariants.
|
||||
- `docs/internal/overview.md`: cover package roles and boundaries. Avoid
|
||||
user-facing command details.
|
||||
- `docs/internal/pipeline.md`: cover runner and registry mechanics. Avoid config
|
||||
syntax beyond linking to the canonical config doc.
|
||||
- `docs/internal/modules.md`: cover implemented modules and how they fit stage
|
||||
contracts. Avoid external Seriatim schema detail and durable JSON output
|
||||
schemas beyond links to integration docs.
|
||||
- `docs/internal/llm.md`: cover runtime internals. Avoid claiming support for
|
||||
non-implemented providers.
|
||||
- `docs/internal/diagnostics.md`: cover diagnostics implementation. Avoid
|
||||
operator runbooks; link to operations.
|
||||
- `docs/integrations/seriatim.md`: keep focused on Seriatim minimal transcript
|
||||
JSON. Avoid D&D extractor behavior.
|
||||
- `docs/integrations/openai-compatible.md`: document the implemented external
|
||||
LLM contract. Avoid extensive copied provider docs.
|
||||
- `docs/integrations/json-output.md`: document durable output files. Avoid
|
||||
diagnostics run-directory details except a link to operations.
|
||||
- `docs/integrations/dnd-spell-artifacts.md`: document the spell artifact
|
||||
contract. Avoid extractor implementation details better covered internally.
|
||||
- `docs/integrations/dnd-spells.md`: replace by split docs above, then delete or
|
||||
leave a temporary redirect-style note only if that is useful during migration.
|
||||
- `docs/roadmap/*.md`: remove completed implementation history or rewrite it as
|
||||
future work. Do not leave implemented behavior solely in roadmap docs.
|
||||
|
||||
## Examples Plan
|
||||
|
||||
`examples/` exists and should remain because Notarius has non-trivial
|
||||
configuration and workflows.
|
||||
|
||||
- `examples/dnd-spells.config.yml`
|
||||
- Purpose: maintained minimal D&D spell extraction pipeline config.
|
||||
- Expected validity check: `go test ./internal/cli` currently validates this
|
||||
fixture with `config validate`, `pipelines list`, and fake-LLM `run`.
|
||||
- Documentation links: README, `docs/cli.md`, `docs/config.md`,
|
||||
`docs/integrations/seriatim.md`, `docs/integrations/dnd-spell-artifacts.md`.
|
||||
- Update guidance: keep secret-free; prefer environment-based API key
|
||||
behavior; preserve test coverage.
|
||||
|
||||
- `examples/seriatim-minimal-transcript.json`
|
||||
- Purpose: maintained minimal Seriatim transcript input.
|
||||
- Expected validity check: `go test ./internal/cli` uses it in the fake-LLM run
|
||||
acceptance tests; Seriatim adapter tests cover compatible shapes separately.
|
||||
- Documentation links: README, `docs/cli.md`,
|
||||
`docs/integrations/seriatim.md`.
|
||||
|
||||
Do not add an expected-output example unless it can be kept in sync by tests or
|
||||
a small documented regeneration procedure. Do not invent examples for NPC, item,
|
||||
combat, Markdown, or Obsidian extraction until those features exist.
|
||||
|
||||
## Internal Documentation Plan
|
||||
|
||||
### Pipeline Orchestration
|
||||
|
||||
- Path: `docs/internal/pipeline.md`
|
||||
- Purpose: explain implemented runner orchestration.
|
||||
- Inputs and outputs: `pipeline.RunInput`, `pipeline.RunOutput`,
|
||||
`contracts.*Request`, `contracts.*Result`, `artifacts.RunManifest`.
|
||||
- Boundaries: orchestration and registries only; no source-format or D&D logic.
|
||||
- Config fields used: resolved pipeline bindings and lane selection from
|
||||
`internal/core/config`.
|
||||
- Adapters used: input, chunk, extract, merge, normalize, validator, output
|
||||
registries.
|
||||
- Failure behavior: stage errors wrap module keys/lane/chunk context; partial
|
||||
manifest may be returned on failure.
|
||||
- Tests to inspect before changing: `internal/framework/pipeline/*_test.go`,
|
||||
`internal/framework/pipeline/walking_skeleton_test.go`,
|
||||
`internal/cli/run_test.go`.
|
||||
- Architectural invariants: fixed six-stage flow; generic source references;
|
||||
module selection via registries.
|
||||
|
||||
### Stage Modules
|
||||
|
||||
- Path: `docs/internal/modules.md`
|
||||
- Purpose: summarize implemented production modules and extension pattern.
|
||||
- Inputs and outputs: module-specific contract methods under
|
||||
`internal/framework/contracts`.
|
||||
- Boundaries: module logic under `internal/modules/<stage>/...`; CLI only wires
|
||||
production modules.
|
||||
- Config fields used: module bindings, `llm_profile`, and `options`.
|
||||
- Adapters used: Seriatim input, generic chunker, D&D spells extractor,
|
||||
append-order merger, noop normalizer, JSON output encoder.
|
||||
- Failure behavior: module constructors and stage methods return contextual
|
||||
errors; validators reject candidates rather than panicking.
|
||||
- Tests to inspect before changing: module package tests and
|
||||
`internal/cli/catalog.go` coverage in `internal/cli/run_test.go`.
|
||||
- Architectural invariants: source/domain concerns stay in modules; framework
|
||||
remains source- and domain-agnostic.
|
||||
|
||||
### LLM Runtime
|
||||
|
||||
- Path: `docs/internal/llm.md`
|
||||
- Purpose: explain structured LLM execution internals.
|
||||
- Inputs and outputs: `StructuredCompletionRequest`,
|
||||
`StructuredCompletionResponse`, schema registry entries, scheduler calls.
|
||||
- Boundaries: provider wire details stay in `internal/framework/llm`; prompts
|
||||
and schemas are owned by modules or generic prompt helpers.
|
||||
- Config fields used: `llm_profiles`, timeout, retries, concurrency, API key
|
||||
environment handling.
|
||||
- Adapters used: OpenAI-compatible HTTP client and scheduler.
|
||||
- Failure behavior: validation errors before calls; retryable provider errors;
|
||||
redacted error messages.
|
||||
- Tests to inspect before changing: `internal/framework/llm/*_test.go`,
|
||||
`internal/framework/prompt/*_test.go`, spell prompt/schema tests.
|
||||
- Architectural invariants: no secrets in errors/diagnostics; no domain prompt
|
||||
logic in provider adapters.
|
||||
|
||||
### Diagnostics
|
||||
|
||||
- Path: `docs/internal/diagnostics.md`
|
||||
- Purpose: explain diagnostics implementation.
|
||||
- Inputs and outputs: invocation metadata, redacted effective config, resolved
|
||||
pipeline, run manifest, warnings, run report, error log.
|
||||
- Boundaries: diagnostics writes inspection artifacts; durable user output is
|
||||
written by CLI/output modules.
|
||||
- Config fields used: diagnostics work dir and retention mode.
|
||||
- Adapters used: none external.
|
||||
- Failure behavior: diagnostics write failures fail the CLI command with
|
||||
context; retention removes or preserves the run directory based on result and
|
||||
warnings.
|
||||
- Tests to inspect before changing: `internal/core/diagnostics/*_test.go`,
|
||||
CLI diagnostics tests.
|
||||
- Architectural invariants: diagnostics must not expose secrets; cleanup must
|
||||
use narrow paths.
|
||||
|
||||
## Integration Documentation Plan
|
||||
|
||||
### Seriatim Minimal Transcript JSON
|
||||
|
||||
- Path: `docs/integrations/seriatim.md`
|
||||
- External system or contract: Seriatim minimal transcript JSON.
|
||||
- Current usage: parsed by the `seriatim` input adapter into a generic source
|
||||
document.
|
||||
- Version or compatibility notes: repository currently supports only the minimal
|
||||
shape implemented by `internal/modules/input/seriatim`.
|
||||
- What should be documented: accepted fields, validation, source mapping,
|
||||
metadata keys, capabilities, example.
|
||||
- What should not be documented: future Seriatim variants or D&D extraction
|
||||
behavior.
|
||||
|
||||
### OpenAI-Compatible Structured Output
|
||||
|
||||
- Path: `docs/integrations/openai-compatible.md`
|
||||
- External system or contract: OpenAI-compatible chat completions API with
|
||||
`response_format.type=json_schema`.
|
||||
- Current usage: production LLM client posts to `<base_url>/chat/completions`
|
||||
and decodes assistant JSON content.
|
||||
- Version or compatibility notes: compatibility is defined by the implemented
|
||||
request and response fields in `internal/framework/llm`.
|
||||
- What should be documented: endpoint construction, auth header behavior,
|
||||
request shape summary, response assumptions, retry behavior, config links.
|
||||
- What should not be documented: comprehensive provider API reference or
|
||||
unsupported providers.
|
||||
|
||||
### Durable JSON Output
|
||||
|
||||
- Path: `docs/integrations/json-output.md`
|
||||
- External system or contract: file-format contract for Notarius JSON output.
|
||||
- Current usage: produced by the `json` output encoder and written by the CLI.
|
||||
- Version or compatibility notes: manifest includes schema version and resolved
|
||||
pipeline provenance; exact envelope versioning should match implemented
|
||||
fields.
|
||||
- What should be documented: file names, JSON shapes, artifact grouping, source
|
||||
references, rejected artifacts, warnings, filename sanitization.
|
||||
- What should not be documented: future output formats or remote storage.
|
||||
|
||||
### D&D Spell-Cast Artifacts
|
||||
|
||||
- Path: `docs/integrations/dnd-spell-artifacts.md`
|
||||
- External system or contract: durable `dnd.spell_cast` artifact payload and
|
||||
validation contract.
|
||||
- Current usage: produced by the `dnd/spells` extractor and serialized by JSON
|
||||
output.
|
||||
- Version or compatibility notes: artifact type `dnd.spell_cast`, schema version
|
||||
`v1`; response schema asset `dnd_spells.v1.json` is internal to extraction.
|
||||
- What should be documented: payload fields, source references, rejection reason
|
||||
codes, prompt/schema manifest metadata, limits.
|
||||
- What should not be documented: unimplemented D&D item, NPC, combat, or rules
|
||||
extraction.
|
||||
|
||||
## Recommended Implementation Sequence
|
||||
|
||||
### Stage 1: Root Orientation And Core References
|
||||
|
||||
- Goal: make the project immediately understandable and add required public
|
||||
references.
|
||||
- Files to create/update/delete/move: rewrite `README.md`; create
|
||||
`docs/cli.md`; create `docs/config.md`; update examples only if needed for
|
||||
clarity and tests.
|
||||
- Repository areas to inspect: `internal/cli/run.go`, `internal/cli/run_test.go`,
|
||||
`internal/core/config`, `examples/`.
|
||||
- Acceptance criteria: README is concise; CLI and config docs are canonical and
|
||||
describe only implemented behavior; maintained example commands match tests.
|
||||
- Suggested validation commands: `go test ./internal/cli ./internal/core/config`;
|
||||
`go test ./...` if examples changed.
|
||||
- One prompt? Yes, if kept focused on README, CLI, and config only.
|
||||
|
||||
### Stage 2: Operations And Troubleshooting
|
||||
|
||||
- Goal: document durable output, diagnostics, retention, and common recovery
|
||||
paths.
|
||||
- Files to create/update/delete/move: create `docs/operations.md`; create
|
||||
`docs/troubleshooting.md`.
|
||||
- Repository areas to inspect: `internal/cli/run.go`,
|
||||
`internal/core/diagnostics`, `internal/modules/output/json`,
|
||||
`internal/framework/llm`, relevant tests.
|
||||
- Acceptance criteria: operations doc names implemented output and diagnostics
|
||||
files; troubleshooting entries map to implemented errors; no remote storage,
|
||||
resume command, or archive behavior is documented.
|
||||
- Suggested validation commands: `go test ./internal/cli ./internal/core/diagnostics ./internal/modules/output/json ./internal/framework/llm`.
|
||||
- One prompt? Yes.
|
||||
|
||||
### Stage 3: Developer Policy And Internal Docs
|
||||
|
||||
- Goal: add required developer workflow and internal component documentation.
|
||||
- Files to create/update/delete/move: create `docs/policy/development.md`;
|
||||
create `docs/internal/overview.md`; create `docs/internal/pipeline.md`;
|
||||
create `docs/internal/modules.md`; create `docs/internal/llm.md`; create
|
||||
`docs/internal/diagnostics.md`.
|
||||
- Repository areas to inspect: `AGENTS.md`, `go.mod`, `docs/policy`,
|
||||
`internal/framework`, `internal/core`, `internal/modules`,
|
||||
`internal/cli/catalog.go`.
|
||||
- Acceptance criteria: docs preserve architecture-policy boundaries; internal
|
||||
docs cover implemented components only; development doc gives actionable
|
||||
commands and contribution patterns without duplicating reference docs.
|
||||
- Suggested validation commands: `go test ./...`; `go vet ./...`;
|
||||
`go build ./cmd/notarius`.
|
||||
- One prompt? Possibly too large. If needed, split into Stage 3A
|
||||
(`development.md` and `overview.md`) and Stage 3B (component docs).
|
||||
|
||||
### Stage 4: Integration And Artifact Contracts
|
||||
|
||||
- Goal: place external and durable file-format contracts in canonical
|
||||
integration docs.
|
||||
- Files to create/update/delete/move: update `docs/integrations/seriatim.md`;
|
||||
create `docs/integrations/openai-compatible.md`; create
|
||||
`docs/integrations/json-output.md`; create
|
||||
`docs/integrations/dnd-spell-artifacts.md`; delete or replace
|
||||
`docs/integrations/dnd-spells.md` after its content is split.
|
||||
- Repository areas to inspect: `internal/modules/input/seriatim`,
|
||||
`internal/framework/llm`, `internal/modules/output/json`,
|
||||
`internal/core/artifacts`, `internal/modules/extract/dnd/spells`.
|
||||
- Acceptance criteria: integration docs document actual external/file-format
|
||||
contracts; no stale claim that CLI run is unimplemented; D&D details are kept
|
||||
out of generic docs.
|
||||
- Suggested validation commands: `go test ./internal/modules/input/seriatim ./internal/framework/llm ./internal/modules/output/json ./internal/modules/extract/dnd/spells`.
|
||||
- One prompt? Yes.
|
||||
|
||||
### Stage 5: Roadmap Cleanup And Link Verification
|
||||
|
||||
- Goal: remove completed planning clutter and verify the final documentation
|
||||
tree.
|
||||
- Files to create/update/delete/move: delete completed roadmap files or reduce
|
||||
them to genuine future work; update links across README and docs; keep
|
||||
`docs/roadmap/documentation.md` only if remaining documentation work exists.
|
||||
- Repository areas to inspect: all docs, examples, roadmap files.
|
||||
- Acceptance criteria: future/planned behavior appears only under
|
||||
`docs/roadmap/`; completed MVP implementation details are documented in
|
||||
canonical current-behavior docs; links resolve; no stale template text remains.
|
||||
- Suggested validation commands: `go test ./...`; `go vet ./...`;
|
||||
`go build ./cmd/notarius`; grep checks from the validation plan.
|
||||
- One prompt? Yes.
|
||||
|
||||
## Validation Plan
|
||||
|
||||
Run these checks during or after implementation:
|
||||
|
||||
- `go test ./...`
|
||||
- `go vet ./...`
|
||||
- `go build ./cmd/notarius`
|
||||
- `go test ./internal/cli` after CLI docs or examples change.
|
||||
- `go test ./internal/core/config` after config docs or examples change.
|
||||
- `go test ./internal/modules/input/seriatim` after Seriatim docs/examples
|
||||
change.
|
||||
- `go test ./internal/modules/extract/dnd/spells` after spell artifact docs
|
||||
change.
|
||||
- `go test ./internal/modules/output/json` after JSON output docs change.
|
||||
- `go test ./internal/framework/llm` after OpenAI-compatible docs change.
|
||||
|
||||
No automated documentation link checker currently exists in the repository.
|
||||
Future implementation agents should manually verify Markdown links or add a
|
||||
tool only if the project adopts one deliberately.
|
||||
|
||||
Recommended grep checks:
|
||||
|
||||
```sh
|
||||
rg -n 'go-application-template|does not yet|not yet implemented|provisional|Likely future|CLI `run` workflow' README.md docs examples --glob '!docs/policy/**' --glob '!docs/roadmap/**'
|
||||
rg -n 'NPC|item|combat|Obsidian|Markdown' README.md docs examples --glob '!docs/roadmap/**'
|
||||
rg -n 'TODO|FIXME' README.md docs examples
|
||||
```
|
||||
|
||||
Manual review items:
|
||||
|
||||
- Confirm README remains short and orientation-focused.
|
||||
- Confirm `docs/cli.md` contains every implemented command and flag from
|
||||
`internal/cli/run.go`.
|
||||
- Confirm `docs/config.md` contains every implemented file config field and env
|
||||
override from `internal/core/config`.
|
||||
- Confirm examples are secret-free and covered by tests where practical.
|
||||
- Confirm roadmap docs do not masquerade as current-behavior docs.
|
||||
- Confirm no docs mention unimplemented NPC, item, combat, Markdown, Obsidian,
|
||||
remote storage, resume, or DAG behavior except under roadmap limitations or
|
||||
explicit out-of-scope notes.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The repository evidence is sufficient to plan the documentation migration.
|
||||
@@ -1,888 +0,0 @@
|
||||
# Implementation Plan: MVP
|
||||
|
||||
## Status
|
||||
|
||||
This is the staged implementation plan for the active MVP roadmap:
|
||||
[`mvp.md`](mvp.md).
|
||||
|
||||
The target audience is an LLM coding agent. Implement the stages in order.
|
||||
Each stage should leave the repository compiling and tested. Do not skip ahead
|
||||
to later stages unless the current stage's done criteria are satisfied.
|
||||
|
||||
## Policy Context
|
||||
|
||||
Follow:
|
||||
|
||||
- [`../policy/architecture.md`](../policy/architecture.md)
|
||||
- [`../policy/documentation.md`](../policy/documentation.md)
|
||||
- [`mvp.md`](mvp.md)
|
||||
- [`initial-architecture.md`](initial-architecture.md)
|
||||
|
||||
Required boundaries:
|
||||
|
||||
- framework packages must remain source-agnostic and domain-agnostic;
|
||||
- source-format behavior belongs in input modules;
|
||||
- D&D spell behavior, prompt assets, response schema assets, and stable
|
||||
prompt/schema identifiers belong in `internal/modules/extract/dnd/spells`;
|
||||
- stage business logic belongs under `internal/modules/<stage>/...` unless it
|
||||
is genuinely tiny shared framework plumbing;
|
||||
- structural pipeline selection must remain config-driven;
|
||||
- `--only` may select artifact lanes but must not alter pipeline structure;
|
||||
- output-stage warnings are out-of-band from artifact payloads and must be
|
||||
available to CLI/diagnostics;
|
||||
- keep planned documentation in `docs/roadmap/` until MVP behavior exists.
|
||||
|
||||
## Global Implementation Decisions
|
||||
|
||||
- Add no new third-party dependencies.
|
||||
- Keep YAML config version `1` unless a user-visible config syntax change is
|
||||
unavoidable. New module options can use existing binding `options`.
|
||||
- Use the existing six-stage workflow:
|
||||
`input -> chunk -> extract -> merge -> normalize -> output`.
|
||||
- Use production CLI wiring in `internal/cli` for the MVP instead of adding a
|
||||
new app package. The CLI may compose modules, but it must not own module
|
||||
business logic.
|
||||
- Keep `notarius run` serial over chunks for the MVP. The contracts and LLM
|
||||
scheduler should still permit later parallel execution.
|
||||
- Use one effective LLM profile per MVP run. The current runner accepts one
|
||||
`StructuredLLMClient`, so a selected pipeline with multiple distinct effective
|
||||
LLM profile IDs should fail clearly until multi-client runtime support is
|
||||
intentionally added.
|
||||
- Use the existing OpenAI-compatible client for real runs.
|
||||
- Add a scheduled LLM client wrapper so every structured completion passes
|
||||
through the configured scheduler.
|
||||
- Use the existing `seriatim` input module and `dnd/spells` extractor module.
|
||||
- Implement production default modules with these keys:
|
||||
- `generic` chunker;
|
||||
- `appendorder` merger;
|
||||
- `noop` normalizer;
|
||||
- `json` output encoder.
|
||||
- Put production default modules under:
|
||||
- `internal/modules/chunk/generic`;
|
||||
- `internal/modules/merge/appendorder`;
|
||||
- `internal/modules/normalize/noop`;
|
||||
- `internal/modules/output/json`.
|
||||
- The output encoder should return logical output files; the CLI/application
|
||||
layer should write those files to disk. Encoders should not own filesystem
|
||||
side effects.
|
||||
- Use an output directory per run. The MVP default output root should be
|
||||
`./notarius-output`, overrideable by `--output-dir`.
|
||||
- File writes for durable output should be atomic where practical: write to a
|
||||
temporary file in the target directory, then rename.
|
||||
- Use synthetic fixtures only. Do not add private campaign transcript content,
|
||||
real API keys, or private infrastructure values.
|
||||
|
||||
## Stage 1: Move D&D Prompt And Schema Assets Into The Spells Module
|
||||
|
||||
### Goal
|
||||
|
||||
Restore the intended framework/domain boundary before building additional MVP
|
||||
functionality.
|
||||
|
||||
`internal/framework/llm` and `internal/framework/prompt` should provide generic
|
||||
asset loading, metadata, and rendering primitives. They must not define
|
||||
D&D-specific prompt IDs, response schema keys, asset paths, or tests.
|
||||
|
||||
### Files To Update Or Move
|
||||
|
||||
Expected files:
|
||||
|
||||
- `internal/framework/llm/schema_registry.go`
|
||||
- `internal/framework/llm/schema_registry_test.go`
|
||||
- `internal/framework/llm/assets/schemas/dnd_spells.v1.json`
|
||||
- `internal/framework/prompt/registry.go`
|
||||
- `internal/framework/prompt/render.go`
|
||||
- `internal/framework/prompt/render_test.go`
|
||||
- `internal/framework/prompt/assets/dnd/spells/system.md`
|
||||
- `internal/framework/prompt/assets/dnd/spells/user.md`
|
||||
- `internal/modules/extract/dnd/spells/extractor.go`
|
||||
- `internal/modules/extract/dnd/spells/prompt.go`
|
||||
- `internal/modules/extract/dnd/spells/prompt_test.go`
|
||||
- `internal/modules/extract/dnd/spells/schema_test.go`
|
||||
- `docs/integrations/dnd-spells.md`, only if prompt/schema ownership text needs
|
||||
to be corrected.
|
||||
|
||||
### Required Design
|
||||
|
||||
Refactor `internal/framework/llm` so it can load schemas from caller-owned
|
||||
embedded files.
|
||||
|
||||
Add or expose a generic constructor similar to:
|
||||
|
||||
```go
|
||||
func LoadResponseSchema(fsys fs.FS, def ResponseSchemaDefinition) (ResponseSchema, error)
|
||||
```
|
||||
|
||||
where `ResponseSchemaDefinition` carries:
|
||||
|
||||
- key;
|
||||
- ID;
|
||||
- version;
|
||||
- name;
|
||||
- asset path.
|
||||
|
||||
The existing framework registry may keep test schemas, but it must not include
|
||||
`DNDSpellsSchemaKey` or `dnd_spells.v1.json`.
|
||||
|
||||
Refactor `internal/framework/prompt` so it can compile/render prompt pairs from
|
||||
caller-owned embedded files.
|
||||
|
||||
Add or expose a generic constructor/rendering type similar to:
|
||||
|
||||
```go
|
||||
type Bundle struct { ... }
|
||||
func LoadBundle(fsys fs.FS, def Definition) (*Bundle, error)
|
||||
func (b *Bundle) RenderUserSystem(data any) (system string, user string, metadata Metadata, err error)
|
||||
```
|
||||
|
||||
The framework prompt package may continue to own shared hardening text if that
|
||||
is useful, but it must not include `DNDSpellsPromptID` or D&D prompt paths.
|
||||
|
||||
Move D&D prompt and schema assets under `internal/modules/extract/dnd/spells`.
|
||||
Recommended paths:
|
||||
|
||||
```text
|
||||
internal/modules/extract/dnd/spells/assets/prompts/system.md
|
||||
internal/modules/extract/dnd/spells/assets/prompts/user.md
|
||||
internal/modules/extract/dnd/spells/assets/schemas/dnd_spells.v1.json
|
||||
```
|
||||
|
||||
The spells package should define its own stable identifiers:
|
||||
|
||||
```go
|
||||
const PromptID = "dnd.spells"
|
||||
const ResponseSchemaKey = "dnd_spells"
|
||||
const ResponseSchemaID = "notarius.dnd.spells"
|
||||
const ResponseSchemaName = "notarius_dnd_spells_v1"
|
||||
```
|
||||
|
||||
The spells extractor must call module-owned prompt/schema helpers and pass only
|
||||
generic framework values into the LLM client.
|
||||
|
||||
### Required Tests
|
||||
|
||||
- Framework LLM schema tests prove test schemas still load, sort, clone, and
|
||||
omit raw schema content from diagnostics.
|
||||
- Framework LLM schema tests prove looking up `dnd_spells` in the framework
|
||||
registry fails.
|
||||
- Framework prompt tests prove test prompts still render and missing template
|
||||
data still errors.
|
||||
- Framework prompt tests contain no D&D prompt assertions.
|
||||
- Spells package schema tests load the module-owned D&D schema and verify:
|
||||
- key;
|
||||
- ID;
|
||||
- version;
|
||||
- response schema name;
|
||||
- valid JSON;
|
||||
- clone/mutation safety;
|
||||
- diagnostics omit raw schema content.
|
||||
- Spells package prompt tests render the module-owned prompt and verify
|
||||
hardening text and prompt metadata.
|
||||
- Existing spells extractor tests still pass without importing framework-owned
|
||||
D&D constants.
|
||||
|
||||
### Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
gofmt -w internal/framework/llm internal/framework/prompt internal/modules/extract/dnd/spells
|
||||
go test ./internal/framework/llm ./internal/framework/prompt ./internal/modules/extract/dnd/spells
|
||||
go test ./...
|
||||
```
|
||||
|
||||
## Stage 2: Add MVP Manifest And Logical Output File Contracts
|
||||
|
||||
### Goal
|
||||
|
||||
Make output and manifest contracts capable of representing the MVP's durable
|
||||
run output before implementing the production JSON encoder or CLI writing.
|
||||
|
||||
### Files To Update
|
||||
|
||||
Expected files:
|
||||
|
||||
- `internal/framework/contracts/contracts.go`
|
||||
- `internal/core/artifacts/*.go`
|
||||
- `internal/framework/pipeline/runner.go`
|
||||
- `internal/framework/pipeline/runner_test.go`
|
||||
- `internal/framework/contracts/contracts_test.go`
|
||||
- `docs/policy/architecture.md`, only if the implemented output contract
|
||||
requires clarifying policy text.
|
||||
|
||||
### Required Design
|
||||
|
||||
Extend the output contract to support logical files:
|
||||
|
||||
```go
|
||||
type OutputFile struct {
|
||||
Name string `json:"name"`
|
||||
ContentType string `json:"content_type,omitempty"`
|
||||
Bytes []byte `json:"-"`
|
||||
}
|
||||
|
||||
type OutputResult struct {
|
||||
Files []OutputFile `json:"files,omitempty"`
|
||||
Warnings []Warning `json:"warnings,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Remove or stop using the single `OutputResult.Bytes` / `ContentType` path once
|
||||
all tests are updated. If keeping those fields temporarily reduces churn, mark
|
||||
them as legacy in comments and make the runner prefer `Files`.
|
||||
|
||||
Add matching fields to `pipeline.RunOutput`:
|
||||
|
||||
```go
|
||||
OutputFiles []contracts.OutputFile `json:"-"`
|
||||
```
|
||||
|
||||
The runner should collect output-stage warnings exactly as it does now, after
|
||||
calling the output encoder.
|
||||
|
||||
Define safe logical file names:
|
||||
|
||||
- names are slash-separated relative paths;
|
||||
- names must not be empty, absolute, contain `..`, or contain `\`;
|
||||
- names are validated before the runner returns them;
|
||||
- file names are sorted deterministically by the encoder that creates them.
|
||||
|
||||
Extend manifest data enough for MVP provenance:
|
||||
|
||||
- add `RunManifest.LLMProfiles []LLMProfileManifest`;
|
||||
- add `ArtifactLaneManifest.Metadata map[string]any`;
|
||||
- add `RunManifest.StartedAt`, `CompletedAt`, and `RunID` population support
|
||||
in the runner input/output path.
|
||||
|
||||
Recommended structs:
|
||||
|
||||
```go
|
||||
type LLMProfileManifest struct {
|
||||
ID string `json:"id"`
|
||||
Provider string `json:"provider,omitempty"`
|
||||
Model string `json:"model,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Add an optional metadata interface for modules:
|
||||
|
||||
```go
|
||||
type ManifestMetadataProvider interface {
|
||||
ManifestMetadata() map[string]any
|
||||
}
|
||||
```
|
||||
|
||||
When a stage module implements the interface, the runner should include that
|
||||
metadata in the appropriate manifest area. For the MVP, the D&D spells extractor
|
||||
will use this to report prompt and response schema identifiers/hashes on its
|
||||
artifact lane.
|
||||
|
||||
Update `pipeline.RunInput` to accept:
|
||||
|
||||
- `RunID string`;
|
||||
- `StartedAt time.Time`;
|
||||
- `LLMProfiles []artifacts.LLMProfileManifest`.
|
||||
|
||||
The runner should set `CompletedAt` when the run finishes or fails after a
|
||||
manifest has been initialized.
|
||||
|
||||
### Required Tests
|
||||
|
||||
- `contracts.OutputFile` JSON shape omits bytes and includes name/content type.
|
||||
- Runner rejects unsafe output file names returned by an encoder.
|
||||
- Runner preserves output warnings out-of-band.
|
||||
- Runner output contains logical files returned by the encoder.
|
||||
- Manifest includes run ID, started/completed timestamps when supplied or
|
||||
generated.
|
||||
- Manifest includes LLM profile metadata supplied in `RunInput`.
|
||||
- Manifest includes extractor-provided lane metadata when the extractor
|
||||
implements `ManifestMetadataProvider`.
|
||||
|
||||
### Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
gofmt -w internal/framework/contracts internal/core/artifacts internal/framework/pipeline
|
||||
go test ./internal/framework/contracts ./internal/core/artifacts ./internal/framework/pipeline
|
||||
go test ./...
|
||||
```
|
||||
|
||||
## Stage 3: Implement Production Default Stage Modules
|
||||
|
||||
### Goal
|
||||
|
||||
Make pipeline defaults real production modules instead of test-only fakes or
|
||||
framework-only helpers.
|
||||
|
||||
### Files To Add Or Update
|
||||
|
||||
Expected packages:
|
||||
|
||||
- `internal/modules/chunk/generic`
|
||||
- `internal/modules/merge/appendorder`
|
||||
- `internal/modules/normalize/noop`
|
||||
- `internal/modules/output/json`
|
||||
|
||||
Expected framework cleanup:
|
||||
|
||||
- `internal/framework/pipeline/generic_stages.go`
|
||||
- pipeline tests that currently instantiate framework `AppendOrderMerger` or
|
||||
`NoopNormalizer`.
|
||||
|
||||
### Required Design
|
||||
|
||||
#### `generic` chunker
|
||||
|
||||
Package: `internal/modules/chunk/generic`
|
||||
|
||||
Key: `generic`
|
||||
|
||||
Module spec:
|
||||
|
||||
- stage: `chunk`;
|
||||
- requires: `source.transcript` is **not** required;
|
||||
- provides: `chunks`.
|
||||
|
||||
Behavior:
|
||||
|
||||
- accepts any valid `SourceDocument`;
|
||||
- preserves source-unit order;
|
||||
- returns stable chunk IDs: `chunk-000001`, `chunk-000002`, and so on;
|
||||
- copies source units defensively;
|
||||
- adds chunk metadata:
|
||||
- `start_unit_id`;
|
||||
- `end_unit_id`;
|
||||
- `unit_count`.
|
||||
|
||||
Options:
|
||||
|
||||
- `max_units`: positive integer, default `50`;
|
||||
- `overlap_units`: non-negative integer, default `0`, must be less than
|
||||
`max_units`.
|
||||
|
||||
If the source has no units, return a clear error. If options have the wrong type
|
||||
or invalid values, return a clear module-specific error.
|
||||
|
||||
#### `appendorder` merger
|
||||
|
||||
Package: `internal/modules/merge/appendorder`
|
||||
|
||||
Key: `appendorder`
|
||||
|
||||
Module spec:
|
||||
|
||||
- stage: `merge`;
|
||||
- requires: no artifact-type-specific capability;
|
||||
- provides: `merged`.
|
||||
|
||||
Behavior:
|
||||
|
||||
- preserves chunk order as provided by the runner;
|
||||
- preserves candidate order within each chunk;
|
||||
- defensively copies candidates, payloads, source refs, and metadata;
|
||||
- does not merge, deduplicate, or rewrite source references.
|
||||
|
||||
#### `noop` normalizer
|
||||
|
||||
Package: `internal/modules/normalize/noop`
|
||||
|
||||
Key: `noop`
|
||||
|
||||
Module spec:
|
||||
|
||||
- stage: `normalize`;
|
||||
- requires: `merged`;
|
||||
- provides: `normalized`.
|
||||
|
||||
Behavior:
|
||||
|
||||
- defensively copies candidates;
|
||||
- does not deduplicate, rewrite, or validate domain content.
|
||||
|
||||
#### `json` output encoder
|
||||
|
||||
Package: `internal/modules/output/json`
|
||||
|
||||
Key: `json`
|
||||
|
||||
Module spec:
|
||||
|
||||
- stage: `output`;
|
||||
- requires: `normalized`;
|
||||
- provides: `encoded`.
|
||||
|
||||
Behavior:
|
||||
|
||||
- returns logical output files:
|
||||
- `index.json`;
|
||||
- `manifest.json`;
|
||||
- `artifacts/<artifact_type>.json` for each approved artifact type;
|
||||
- `rejected.json`;
|
||||
- `warnings.json`.
|
||||
- groups approved artifacts by `Artifact.ArtifactType`;
|
||||
- sorts artifact-type file names by artifact type;
|
||||
- preserves artifact order within each artifact type according to runner order;
|
||||
- pretty-prints JSON with two-space indentation and trailing newline;
|
||||
- uses content type `application/json`;
|
||||
- includes rejected artifacts and warnings even when the arrays are empty;
|
||||
- does not include output warnings inside artifact payloads.
|
||||
|
||||
File-name safety:
|
||||
|
||||
- artifact type may contain dots and hyphens;
|
||||
- replace any character outside `[A-Za-z0-9._-]` with `_` for artifact file
|
||||
names;
|
||||
- if sanitization produces an empty name, return an error.
|
||||
|
||||
### Required Tests
|
||||
|
||||
- Generic chunker tests cover defaults, exact chunk boundaries, overlap,
|
||||
invalid options, empty source, defensive copies, and stable IDs.
|
||||
- Append-order merge tests cover ordering and defensive copies.
|
||||
- Noop normalizer tests cover pass-through behavior and defensive copies.
|
||||
- JSON output tests cover all logical files, grouping, sorted filenames,
|
||||
rejected/warnings presence, pretty JSON, unsafe artifact type sanitization,
|
||||
and no mutation of inputs.
|
||||
- Pipeline config tests using defaults resolve when these module specs are
|
||||
registered.
|
||||
|
||||
### Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
gofmt -w internal/modules/chunk/generic internal/modules/merge/appendorder internal/modules/normalize/noop internal/modules/output/json internal/framework/pipeline
|
||||
go test ./internal/modules/chunk/generic ./internal/modules/merge/appendorder ./internal/modules/normalize/noop ./internal/modules/output/json
|
||||
go test ./internal/framework/pipeline
|
||||
go test ./...
|
||||
```
|
||||
|
||||
## Stage 4: Add Production CLI Catalog And Runtime Wiring
|
||||
|
||||
### Goal
|
||||
|
||||
Make implemented modules selectable by real CLI commands without test-injected
|
||||
catalogs.
|
||||
|
||||
### Files To Add Or Update
|
||||
|
||||
Expected files:
|
||||
|
||||
- `internal/cli/run.go`
|
||||
- new `internal/cli/catalog.go` or equivalent;
|
||||
- `internal/cli/run_test.go`;
|
||||
- module registry tests as needed.
|
||||
|
||||
### Required Design
|
||||
|
||||
Add production wiring in `internal/cli`:
|
||||
|
||||
```go
|
||||
func productionRegistries() (pipeline.Registries, error)
|
||||
func productionCatalog() (pipeline.ModuleCatalog, error)
|
||||
```
|
||||
|
||||
The production wiring must register:
|
||||
|
||||
- input: `seriatim`;
|
||||
- chunk: `generic`;
|
||||
- extract: `dnd/spells`;
|
||||
- merge: `appendorder`;
|
||||
- normalize: `noop`;
|
||||
- output: `json`.
|
||||
|
||||
Keep all business logic in module packages. `internal/cli` should only compose
|
||||
registries/catalogs and command behavior.
|
||||
|
||||
Update `cli.Options` so tests may inject registries/catalog/runtime without
|
||||
disabling production defaults unintentionally.
|
||||
|
||||
Recommended option fields:
|
||||
|
||||
```go
|
||||
type Options struct {
|
||||
Catalog pipeline.ModuleCatalog
|
||||
Registries pipeline.Registries
|
||||
LLMClientFactory LLMClientFactory
|
||||
LookupEnv func(string) (string, bool)
|
||||
Now func() time.Time
|
||||
}
|
||||
```
|
||||
|
||||
If `Catalog` or `Registries` is empty in normal `Run`, use production wiring.
|
||||
If tests provide either, use the provided value.
|
||||
|
||||
Define `LLMClientFactory` in `internal/cli` or a small local file:
|
||||
|
||||
```go
|
||||
type LLMClientFactory func(ctx context.Context, cfg config.Config, profileID string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error)
|
||||
```
|
||||
|
||||
The production factory should:
|
||||
|
||||
- read the selected LLM profile from effective config;
|
||||
- construct `llm.OpenAICompatibleClient`;
|
||||
- construct `llm.Scheduler` using the most specific configured concurrency:
|
||||
profile `max_concurrency` if set, otherwise global `concurrency.total_llm`,
|
||||
otherwise `1`;
|
||||
- wrap the client in a scheduled client so every completion acquires/releases a
|
||||
scheduler permit;
|
||||
- return manifest-safe LLM metadata with profile ID, provider, and model.
|
||||
|
||||
Add a scheduled client wrapper in `internal/framework/llm` if it does not
|
||||
already exist:
|
||||
|
||||
```go
|
||||
func NewScheduledClient(client contracts.StructuredLLMClient, scheduler *Scheduler) contracts.StructuredLLMClient
|
||||
```
|
||||
|
||||
### Required CLI Behavior
|
||||
|
||||
- `notarius config validate --config <file> --pipeline <id>` uses the
|
||||
production catalog by default.
|
||||
- `notarius pipelines list --config <file>` still lists configured pipeline
|
||||
IDs and validates config shape.
|
||||
- `notarius pipelines list --config <file> --json` remains stable.
|
||||
|
||||
Do not implement `notarius run` in this stage.
|
||||
|
||||
### Required Tests
|
||||
|
||||
- Production catalog includes the six MVP modules and their module specs.
|
||||
- `config validate --pipeline` succeeds for a real MVP config fixture using no
|
||||
injected catalog.
|
||||
- Unknown module keys still fail with stage/pipeline context.
|
||||
- Production LLM client factory rejects missing/invalid LLM profiles with clear
|
||||
errors.
|
||||
- Scheduled client wrapper enforces scheduler use and propagates errors.
|
||||
- Existing CLI tests using injected catalogs still pass.
|
||||
|
||||
### Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
gofmt -w internal/cli internal/framework/llm
|
||||
go test ./internal/cli ./internal/framework/llm
|
||||
go test ./...
|
||||
go vet ./...
|
||||
go build ./cmd/notarius
|
||||
```
|
||||
|
||||
## Stage 5: Implement `notarius run` Without Durable File Writing
|
||||
|
||||
### Goal
|
||||
|
||||
Add the user-facing run command and prove it can drive the configured pipeline
|
||||
with injected fake runtime pieces. This stage should return/run data in memory
|
||||
or through test buffers, but durable file writing may be completed in Stage 6.
|
||||
|
||||
### Files To Update
|
||||
|
||||
Expected files:
|
||||
|
||||
- `internal/cli/run.go`
|
||||
- `internal/cli/run_test.go`
|
||||
- `cmd/notarius/main.go`, only if command wiring requires it.
|
||||
|
||||
### Required Command Shape
|
||||
|
||||
Support:
|
||||
|
||||
```sh
|
||||
notarius run <pipeline-id> --input path/to/source.json
|
||||
notarius run <pipeline-id> --input path/to/source.json --only spells
|
||||
```
|
||||
|
||||
Supported flags:
|
||||
|
||||
- `--config path`;
|
||||
- `--input path`, required;
|
||||
- `--only lane-a,lane-b`;
|
||||
- `--output-dir path`, parsed and passed through metadata for Stage 6;
|
||||
- `--diagnostics-dir path`, overrides config diagnostics work dir for this run;
|
||||
- `--llm-profile profile-id`, operational override for MVP runs.
|
||||
|
||||
Do not add flags for structural module selection, such as `--extractor`,
|
||||
`--chunker`, `--merge`, or `--output`.
|
||||
|
||||
### Required Behavior
|
||||
|
||||
- Missing pipeline ID returns exit code `2`.
|
||||
- Missing `--input` returns exit code `2`.
|
||||
- Unknown flags return exit code `2`.
|
||||
- Config/load/resolve/runtime failures return exit code `1`.
|
||||
- Successful runs return exit code `0`.
|
||||
- `--only` uses existing lane selection behavior.
|
||||
- Extend `config.ResolveInput` with `LLMProfileOverride string` or an
|
||||
equivalent option. When `--llm-profile` is provided, apply it to every
|
||||
resolved module binding before the resolved pipeline digest is computed. This
|
||||
keeps the override operational rather than structural while still making the
|
||||
effective pipeline digest truthful.
|
||||
- After resolution, collect the distinct effective LLM profile IDs used by the
|
||||
selected pipeline. For the MVP, require exactly one distinct profile ID and
|
||||
fail clearly if more than one is present.
|
||||
- Input file is read as raw bytes and passed to the runner with path metadata.
|
||||
- Source ID defaults to the input path or basename only if the input adapter
|
||||
needs one; do not invent transcript-specific source IDs in the CLI.
|
||||
- The command resolves the selected pipeline with the production catalog.
|
||||
- The command constructs the LLM client through `LLMClientFactory`.
|
||||
- The command invokes `pipeline.New(registries).Run(...)`.
|
||||
- The command prints a concise success message that includes at least:
|
||||
- pipeline ID;
|
||||
- approved artifact count;
|
||||
- rejected artifact count.
|
||||
- If warnings exist, print a concise warning count to stderr.
|
||||
|
||||
### Required Tests
|
||||
|
||||
Use fake LLM/runtime injection; do not call external services.
|
||||
|
||||
- Missing pipeline ID.
|
||||
- Missing input flag.
|
||||
- Unknown pipeline.
|
||||
- Unknown `--only` lane.
|
||||
- Invalid input file path.
|
||||
- Successful run invokes runner path through real registries and fake LLM.
|
||||
- `--only spells` runs only the selected lane.
|
||||
- LLM factory failure is reported clearly.
|
||||
- Validation rejection produces a failed/non-zero or successful-with-rejections
|
||||
behavior according to current runner semantics. For MVP, keep runner
|
||||
semantics: a run with rejected artifacts completes successfully with
|
||||
`ValidationStatus` set to `rejected`, unless an error occurs.
|
||||
|
||||
### Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
gofmt -w internal/cli
|
||||
go test ./internal/cli
|
||||
go test ./...
|
||||
go vet ./...
|
||||
go build ./cmd/notarius
|
||||
```
|
||||
|
||||
## Stage 6: Write Durable Output And Diagnostics For `notarius run`
|
||||
|
||||
### Goal
|
||||
|
||||
Complete the MVP run workflow by writing output files and diagnostics.
|
||||
|
||||
### Files To Update
|
||||
|
||||
Expected files:
|
||||
|
||||
- `internal/cli/run.go`
|
||||
- `internal/cli/run_test.go`
|
||||
- `internal/core/diagnostics/*.go`, only if helper methods are needed.
|
||||
|
||||
### Required Design
|
||||
|
||||
Output directory behavior:
|
||||
|
||||
- default root: `./notarius-output`;
|
||||
- override: `--output-dir`;
|
||||
- each run writes to `<output-root>/<run-id>/`;
|
||||
- run ID comes from diagnostics run directory when available or from a
|
||||
generated UTC nanosecond timestamp using the same style as diagnostics;
|
||||
- create directories with `0755`;
|
||||
- write files with `0644`;
|
||||
- write each file atomically where practical.
|
||||
|
||||
Logical output files from `pipeline.RunOutput.OutputFiles` should be written
|
||||
under the run output directory. Reject unsafe logical file names before writing:
|
||||
|
||||
- empty;
|
||||
- absolute;
|
||||
- contains `..`;
|
||||
- contains backslash;
|
||||
- escapes the run output directory after path cleaning.
|
||||
|
||||
Diagnostics behavior:
|
||||
|
||||
- create a diagnostics run directory at command start unless retention is
|
||||
`never` and the implementation can still reliably capture failures; simplest
|
||||
MVP behavior is to create it and then apply retention at the end;
|
||||
- write invocation metadata;
|
||||
- write redacted effective config;
|
||||
- write resolved pipeline;
|
||||
- write run manifest;
|
||||
- write warnings;
|
||||
- write run report containing output path, counts, and validation status;
|
||||
- write error log on failure;
|
||||
- apply retention with existing diagnostics policy.
|
||||
|
||||
`--diagnostics-dir` should override `Config.Diagnostics.WorkDir` after file and
|
||||
environment config have been applied, without changing structural pipeline
|
||||
definition or pipeline digest.
|
||||
|
||||
Success output:
|
||||
|
||||
- stdout includes the durable output run directory path;
|
||||
- stderr includes warning count when warnings are present;
|
||||
- no raw prompt text, raw API keys, or large source payloads should be printed.
|
||||
|
||||
### Required Tests
|
||||
|
||||
- Successful `notarius run` writes output files under a temp output directory.
|
||||
- Output write rejects unsafe logical file names from a fake encoder.
|
||||
- Writes are atomic enough that no temporary files remain after success.
|
||||
- Diagnostics artifacts are written on success.
|
||||
- Error log is written on failure after diagnostics directory creation.
|
||||
- Retention `never` removes successful warning-free diagnostics directories.
|
||||
- Warnings are present in diagnostics and are reported to stderr.
|
||||
- `--diagnostics-dir` overrides config diagnostics directory.
|
||||
|
||||
### Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
gofmt -w internal/cli internal/core/diagnostics
|
||||
go test ./internal/cli ./internal/core/diagnostics
|
||||
go test ./...
|
||||
go vet ./...
|
||||
go build ./cmd/notarius
|
||||
```
|
||||
|
||||
## Stage 7: Add MVP Fixtures And End-To-End Acceptance Coverage
|
||||
|
||||
### Goal
|
||||
|
||||
Make the MVP path continuously testable without network access.
|
||||
|
||||
### Files To Add Or Update
|
||||
|
||||
Expected fixtures:
|
||||
|
||||
- `examples/seriatim-minimal-transcript.json`, if the example can be kept
|
||||
accurate before the deferred documentation pass;
|
||||
- `examples/dnd-spells.config.yml`, if config examples are tested in this
|
||||
stage;
|
||||
- or equivalent `internal/cli/testdata/...` fixtures if examples are deferred.
|
||||
|
||||
Expected tests:
|
||||
|
||||
- `internal/cli/run_test.go`
|
||||
- `internal/modules/extract/dnd/spells/runner_test.go`
|
||||
- config tests as needed.
|
||||
|
||||
### Required Design
|
||||
|
||||
Add a maintained MVP config fixture:
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
llm_profiles:
|
||||
default:
|
||||
provider: openai-compatible
|
||||
base_url: http://127.0.0.1:1
|
||||
model: fake-model
|
||||
pipelines:
|
||||
dnd-session:
|
||||
input: seriatim
|
||||
chunk:
|
||||
module: generic
|
||||
options:
|
||||
max_units: 50
|
||||
artifacts:
|
||||
spells:
|
||||
extract: dnd/spells
|
||||
```
|
||||
|
||||
The fixture may use a fake base URL because tests should inject a fake LLM
|
||||
client factory. Do not require a real network call.
|
||||
|
||||
Acceptance tests should execute the public CLI entry path with:
|
||||
|
||||
```sh
|
||||
notarius run dnd-session --config <fixture> --input <fixture> --output-dir <tmp>
|
||||
notarius run dnd-session --config <fixture> --input <fixture> --only spells --output-dir <tmp>
|
||||
notarius config validate --config <fixture> --pipeline dnd-session
|
||||
notarius pipelines list --config <fixture>
|
||||
```
|
||||
|
||||
The fake LLM should return deterministic D&D spell output with valid source
|
||||
references. The resulting output files should be parsed as JSON and checked for:
|
||||
|
||||
- manifest pipeline ID and digest;
|
||||
- spell artifact payload;
|
||||
- source references;
|
||||
- prompt/schema metadata in manifest or artifact metadata;
|
||||
- validation status;
|
||||
- warning behavior.
|
||||
|
||||
### Required Failure Coverage
|
||||
|
||||
Add fixture-driven tests for:
|
||||
|
||||
- missing config;
|
||||
- unknown pipeline;
|
||||
- invalid Seriatim input;
|
||||
- invalid `--only` lane;
|
||||
- fake LLM failure;
|
||||
- malformed LLM response;
|
||||
- invalid source reference rejection.
|
||||
|
||||
### Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
gofmt -w internal/cli internal/modules/extract/dnd/spells
|
||||
go test ./internal/cli ./internal/modules/extract/dnd/spells
|
||||
go test ./...
|
||||
go vet ./...
|
||||
go build ./cmd/notarius
|
||||
```
|
||||
|
||||
## Stage 8: MVP Final Review And Roadmap Cleanup
|
||||
|
||||
### Goal
|
||||
|
||||
Confirm the MVP is complete enough to trigger the deferred documentation pass.
|
||||
|
||||
### Required Review
|
||||
|
||||
Perform a code review against:
|
||||
|
||||
- [`mvp.md`](mvp.md);
|
||||
- [`../policy/architecture.md`](../policy/architecture.md);
|
||||
- [`../policy/documentation.md`](../policy/documentation.md).
|
||||
|
||||
Check specifically:
|
||||
|
||||
- no D&D prompt/schema assets or constants remain in framework packages;
|
||||
- production CLI commands use production wiring by default;
|
||||
- `config validate --pipeline` works with the MVP fixture;
|
||||
- `pipelines list` works with the MVP fixture;
|
||||
- `notarius run` writes durable output and diagnostics;
|
||||
- default modules resolve without test-only registration;
|
||||
- output warnings remain out-of-band from artifact payloads;
|
||||
- no private data or secrets appear in fixtures;
|
||||
- docs outside `docs/roadmap/` describe only implemented behavior.
|
||||
|
||||
### Required Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
go test ./...
|
||||
go vet ./...
|
||||
go build ./cmd/notarius
|
||||
```
|
||||
|
||||
### Required Roadmap Update
|
||||
|
||||
After the MVP is implemented and reviewed:
|
||||
|
||||
- update [`mvp.md`](mvp.md) to mark MVP functionality complete or reduce it to
|
||||
remaining release/documentation work;
|
||||
- keep the full documentation pass deferred until this review passes;
|
||||
- do not tag alpha `0.1.0` until the documentation pass is complete.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The plan above makes the required MVP implementation choices explicitly.
|
||||
@@ -1,632 +0,0 @@
|
||||
# Initial Architecture Roadmap
|
||||
|
||||
## Status
|
||||
|
||||
This document captures proposed architecture and implementation sequencing for
|
||||
Notarius. It describes planned work, not implemented behavior.
|
||||
|
||||
## Goal
|
||||
|
||||
Notarius should extract structured JSON artifacts from primary source inputs
|
||||
using modular, LLM-backed extractors.
|
||||
|
||||
The first MVP should target audio transcripts generated by Seriatim. That
|
||||
choice should be implemented as an input-stage module, not as a
|
||||
transcript-specific assumption in the application core. Later input sources,
|
||||
such as unstructured Markdown notes or Obsidian documents, should be addable
|
||||
through new input and extract modules without reshaping the framework.
|
||||
|
||||
The first extraction domain should be D&D session analysis, starting with spell
|
||||
casts. That domain should live in extract-stage modules and related schemas, not
|
||||
in core framework packages.
|
||||
|
||||
The application should follow the same broad architecture as Audita:
|
||||
|
||||
- deterministic core packages for source documents, artifacts, and configuration once needed;
|
||||
- input-stage modules that translate external source formats into a small internal source model;
|
||||
- reusable framework packages for contracts, orchestration, LLM runtime, structured output, and validation;
|
||||
- independent extract-stage modules that own domain-specific behavior;
|
||||
- independent validator packages;
|
||||
- embedded prompt and JSON schema assets;
|
||||
- CLI orchestration that wires the pieces together without owning domain logic.
|
||||
|
||||
The main domain difference from Audita is that Notarius emits extracted
|
||||
artifacts rather than proposing and applying transcript corrections.
|
||||
|
||||
## Architectural Principles
|
||||
|
||||
- Keep the core input model generic: ordered text units plus metadata.
|
||||
- Keep source-format details in hexagonal input modules.
|
||||
- Keep extraction-domain details in extract modules.
|
||||
- Treat evidence as source references, not transcript references.
|
||||
- Prefer narrow, useful abstractions over a universal document model.
|
||||
- Preserve enough provenance for validation, replay, and downstream inspection.
|
||||
|
||||
## Proposed Package Shape
|
||||
|
||||
```text
|
||||
cmd/notarius
|
||||
internal/cli
|
||||
|
||||
internal/core/source
|
||||
internal/core/artifacts
|
||||
|
||||
internal/framework/contracts
|
||||
internal/framework/pipeline
|
||||
internal/framework/validate
|
||||
internal/framework/llm
|
||||
internal/framework/prompt
|
||||
|
||||
internal/modules/input/seriatim
|
||||
internal/modules/input/markdown
|
||||
|
||||
internal/modules/chunk/generic
|
||||
internal/modules/chunk/dndtranscript
|
||||
|
||||
internal/modules/extract/dnd/spells
|
||||
internal/modules/extract/dnd/items
|
||||
internal/modules/extract/dnd/npcs
|
||||
internal/modules/extract/dnd/combat
|
||||
|
||||
internal/modules/merge/appendorder
|
||||
internal/modules/merge/dnd/spells
|
||||
|
||||
internal/modules/normalize/noop
|
||||
internal/modules/normalize/dnd/spells
|
||||
|
||||
internal/modules/output/json
|
||||
|
||||
internal/validators/source_refs
|
||||
internal/validators/schema_validity
|
||||
internal/validators/domain_consistency
|
||||
internal/validators/llm_review
|
||||
|
||||
examples
|
||||
docs/internal
|
||||
```
|
||||
|
||||
The `markdown` input module and D&D-specific chunk, merge, normalize, and
|
||||
output modules are listed as likely future packages. The MVP should implement
|
||||
only the stage modules needed by the checkpoint sequence.
|
||||
|
||||
`internal/core/config` should be added when production configuration exists.
|
||||
|
||||
The framework package list is intentionally consolidated. `pipeline` should own
|
||||
runner orchestration, stage registries, and small merge/normalize/output helpers
|
||||
until those boundaries prove they need separate packages. `llm` should own
|
||||
structured output and response-schema mechanics until those concerns become too
|
||||
large or import-heavy. `prompt` should own prompt assets and rendering helpers
|
||||
once prompt assets exist.
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### SourceDocument
|
||||
|
||||
Canonical internal representation of source material. This should be the object
|
||||
extractors receive, regardless of whether the original input was a transcript,
|
||||
Markdown file, note export, or another source type.
|
||||
|
||||
```go
|
||||
type SourceDocument struct {
|
||||
ID string `json:"id"`
|
||||
Kind string `json:"kind"`
|
||||
Format string `json:"format"`
|
||||
Digest string `json:"digest"`
|
||||
Units []SourceUnit `json:"units"`
|
||||
Metadata map[string]any `json:"metadata,omitempty"`
|
||||
}
|
||||
|
||||
type SourceUnit struct {
|
||||
ID string `json:"id"`
|
||||
Kind string `json:"kind"`
|
||||
Text string `json:"text"`
|
||||
Metadata map[string]any `json:"metadata,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Initial source-unit assumptions:
|
||||
|
||||
- units are ordered;
|
||||
- unit IDs are stable within a source document;
|
||||
- each unit has extractable text;
|
||||
- adapter-specific metadata may carry speaker, timestamps, heading paths, page
|
||||
numbers, or other source details.
|
||||
|
||||
Core source metadata should remain `map[string]any`. Notarius should not define
|
||||
a universal document model. Instead, the project should document well-known
|
||||
metadata keys, such as `speaker`, `start`, `end`, and `heading_path`, as
|
||||
conventions. Input modules may export typed accessor helpers for their own
|
||||
metadata, such as `seriatim.SpeakerOf(unit)`, without leaking those helpers into
|
||||
core framework contracts.
|
||||
|
||||
### Input Module / Adapter Contract
|
||||
|
||||
Hexagonal boundary for external source formats.
|
||||
|
||||
```go
|
||||
type InputAdapter interface {
|
||||
Key() string
|
||||
Parse(ctx context.Context, req ParseRequest) (*source.SourceDocument, error)
|
||||
}
|
||||
```
|
||||
|
||||
The MVP input module should target Seriatim minimal transcript JSON. Seriatim segment
|
||||
fields should map as follows:
|
||||
|
||||
- `id` becomes `SourceUnit.ID`;
|
||||
- `text` becomes `SourceUnit.Text`;
|
||||
- `speaker`, `start`, and `end` become unit metadata;
|
||||
- Seriatim output metadata becomes document metadata.
|
||||
|
||||
The core runner should not know that these units came from transcript segments.
|
||||
|
||||
### SourceRef
|
||||
|
||||
Grounding reference from an extracted fact back to source units.
|
||||
|
||||
```go
|
||||
type SourceRef struct {
|
||||
SourceID string `json:"source_id"`
|
||||
StartUnitID string `json:"start_unit_id"`
|
||||
EndUnitID string `json:"end_unit_id"`
|
||||
}
|
||||
```
|
||||
|
||||
Initial source-reference validation should require:
|
||||
|
||||
- source ID exists for the current run;
|
||||
- start and end unit IDs exist;
|
||||
- start is less than or equal to end in document order;
|
||||
- the referenced range is contiguous within the source document;
|
||||
- every extracted fact has at least one source reference unless its schema
|
||||
explicitly allows ungrounded metadata.
|
||||
|
||||
Transcript-oriented output can still present these as transcript segment ranges
|
||||
when the adapter metadata makes that interpretation available.
|
||||
|
||||
Source references should preserve the exact ranges produced by extractors and
|
||||
validators. Overlapping ranges should not be merged or rewritten by generic
|
||||
pipeline code. If a domain module wants a derived compact range later, that
|
||||
should be additional output, not a replacement for the original evidence.
|
||||
|
||||
### Extractor
|
||||
|
||||
Reusable module contract for producing one artifact type.
|
||||
|
||||
```go
|
||||
type Extractor interface {
|
||||
Key() string
|
||||
ArtifactType() string
|
||||
SchemaVersion() string
|
||||
Validators() []Validator
|
||||
Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error)
|
||||
}
|
||||
```
|
||||
|
||||
An extractor should receive either a whole source document or a source chunk,
|
||||
depending on processing mode. It should return typed artifact candidates plus
|
||||
warnings. It should not mutate the source document.
|
||||
|
||||
`ExtractionRequest` should be designed now to carry both the active chunk and
|
||||
optional ambient context, even if the MVP leaves that context empty. Useful
|
||||
ambient context may include a document synopsis, prior-chunk summaries, known
|
||||
entities, or other module-provided state. D&D spell extraction can likely work
|
||||
per chunk, but combat, NPC, and identity-oriented extraction will need broader
|
||||
context. Adding the field later would force churn across every extractor.
|
||||
|
||||
Extract modules own domain concepts. For example, D&D spell extraction should
|
||||
live under `internal/modules/extract/dnd/spells`; a future to-do extractor for
|
||||
notes should live under a different extract-module path and use the same
|
||||
framework contract.
|
||||
|
||||
### Chunker
|
||||
|
||||
Reusable stage contract for splitting a source document into ordered source
|
||||
chunks.
|
||||
|
||||
Chunking is a first-class pipeline concern because source documents may exceed a
|
||||
single LLM extraction pass. Chunkers should preserve source-unit order and
|
||||
produce stable chunk metadata suitable for diagnostics and replay.
|
||||
|
||||
### Merger
|
||||
|
||||
Reusable stage contract for combining per-chunk artifact candidates into one
|
||||
merged candidate collection.
|
||||
|
||||
Merge should combine outputs without doing semantic reconciliation. A generic
|
||||
append-in-chunk-order merger should be sufficient for many artifact streams,
|
||||
including the likely first D&D spell-cast extractor.
|
||||
|
||||
### Normalizer
|
||||
|
||||
Reusable stage contract for reconciling merged artifact candidates.
|
||||
|
||||
Normalize is distinct from merge. Normalizers may deduplicate repeated facts,
|
||||
resolve aliases, reconcile conflicting fields, check cross-chunk consistency,
|
||||
or attach normalization warnings.
|
||||
|
||||
### Validator
|
||||
|
||||
Reusable validation contract for artifact candidates.
|
||||
|
||||
Validators should cover:
|
||||
|
||||
- JSON/schema validity;
|
||||
- source-reference validity;
|
||||
- required-field and shape checks;
|
||||
- domain consistency;
|
||||
- optional LLM review for high-risk or ambiguous artifacts.
|
||||
|
||||
Validator output should follow Audita's decision-cardinality model: each
|
||||
candidate artifact receives exactly one decision per validator.
|
||||
|
||||
LLM-backed review should be modeled as part of a module's validator chain, not
|
||||
as a separate global review phase. Extract modules should be able to attach one
|
||||
or more deterministic or LLM-backed validators. Normalize-stage modules may also
|
||||
run validator chains, including LLM-backed validators, when semantic
|
||||
reconciliation needs review.
|
||||
|
||||
### Artifact
|
||||
|
||||
Final approved JSON output from one or more extractors.
|
||||
|
||||
Artifacts should preserve enough metadata to support downstream validation,
|
||||
debugging, and replay.
|
||||
|
||||
The pipeline should carry artifact candidates through a generic envelope with a
|
||||
`json.RawMessage` payload. Extract modules should own typed Go structs at their
|
||||
module boundary, then encode those typed records into the generic artifact
|
||||
candidate envelope before returning to framework code. This keeps stage
|
||||
contracts simple and avoids generic type plumbing across unrelated artifact
|
||||
families.
|
||||
|
||||
Final durable output should be one file per artifact type plus a run-level
|
||||
manifest/index file. This supports partial success and lets downstream consumers
|
||||
read only the artifact types they need. Each artifact file should include its
|
||||
artifact type, extractor key, extractor schema version, envelope format version,
|
||||
records, source references, and enough provenance to connect it to the run
|
||||
manifest.
|
||||
|
||||
Every artifact record should require source references unless that artifact
|
||||
schema explicitly opts into ungrounded fields. Artifact-level metadata, counts,
|
||||
run information, and other derived summary fields are exempt from the per-record
|
||||
grounding rule.
|
||||
|
||||
Schemas should be versioned per extractor, with a separate envelope/manifest
|
||||
format version. A single global schema version would couple unrelated extractor
|
||||
release cadence.
|
||||
|
||||
### RunManifest
|
||||
|
||||
Per-run provenance record.
|
||||
|
||||
```go
|
||||
type RunManifest struct {
|
||||
EnvelopeVersion string `json:"envelope_version"`
|
||||
PipelineID string `json:"pipeline_id"`
|
||||
PipelineDigest string `json:"pipeline_digest"`
|
||||
InputModule string `json:"input_module"`
|
||||
Chunker string `json:"chunker"`
|
||||
SourceDigests []string `json:"source_digests"`
|
||||
Extractors []string `json:"extractors"`
|
||||
Merger string `json:"merger"`
|
||||
Normalizer string `json:"normalizer"`
|
||||
OutputEncoder string `json:"output_encoder"`
|
||||
SchemaVersion string `json:"schema_version"`
|
||||
ValidationStatus string `json:"validation_status"`
|
||||
}
|
||||
```
|
||||
|
||||
The manifest should eventually include model names, prompt IDs, prompt hashes,
|
||||
response schema versions, config source, redacted resolved config digest,
|
||||
started/completed timestamps, and diagnostics paths.
|
||||
|
||||
## Initial Extractor Targets
|
||||
|
||||
### D&D Spells
|
||||
|
||||
Recommended first vertical slice because it is narrow but representative.
|
||||
|
||||
```go
|
||||
type SpellCast struct {
|
||||
Player string `json:"player"`
|
||||
Spell string `json:"spell"`
|
||||
Effect string `json:"effect"`
|
||||
NarrativeDescription string `json:"narrative_description"`
|
||||
SourceRefs []SourceRef `json:"source_refs"`
|
||||
}
|
||||
```
|
||||
|
||||
The spell extractor should be D&D-specific. The framework should not know what a
|
||||
spell is.
|
||||
|
||||
### D&D Items
|
||||
|
||||
Tracks items gained, lost, transferred, consumed, or transformed.
|
||||
|
||||
Open questions:
|
||||
|
||||
- Should currency be represented as items or as its own artifact type?
|
||||
- Should item ownership be a required field?
|
||||
- How should ambiguous ownership changes be represented?
|
||||
|
||||
### D&D NPCs
|
||||
|
||||
Tracks NPCs interacted with, newly introduced, renamed, described, or otherwise
|
||||
made relevant to campaign state.
|
||||
|
||||
Open questions:
|
||||
|
||||
- Should NPC identity resolution happen inside this extractor or in a later
|
||||
deduplication stage?
|
||||
- Should location/faction/relationship facts be separate artifact types?
|
||||
|
||||
### D&D Combat
|
||||
|
||||
Likely warrants a dedicated schema rather than a generic event list.
|
||||
|
||||
Proposed first shape:
|
||||
|
||||
```go
|
||||
type CombatTurn struct {
|
||||
Actor string `json:"actor"`
|
||||
Action string `json:"action"`
|
||||
Outcome string `json:"outcome"`
|
||||
NarrativeDescription string `json:"narrative_description"`
|
||||
SourceRefs []SourceRef `json:"source_refs"`
|
||||
}
|
||||
```
|
||||
|
||||
Open questions:
|
||||
|
||||
- Should combat be extracted as turns, rounds, encounters, or all three?
|
||||
- Should mechanical fields such as damage, conditions, saves, attacks, and spell
|
||||
slots be normalized immediately or added later?
|
||||
- How should uncertain initiative order be represented?
|
||||
|
||||
### Future Non-D&D Extractors
|
||||
|
||||
The architecture should support extractors outside the D&D domain. Examples:
|
||||
|
||||
- to-do items from Markdown or Obsidian notes;
|
||||
- decisions and action items from meeting transcripts;
|
||||
- named people, places, and dates from research notes.
|
||||
|
||||
These should be addable as extract modules without changing runner,
|
||||
validator, source-reference, or LLM framework contracts.
|
||||
|
||||
## Configuration Model
|
||||
|
||||
Notarius should use named pipeline profiles selected by ID at the CLI. A
|
||||
pipeline is a fixed-shape template for the known application workflow, not a
|
||||
free-form list of steps:
|
||||
|
||||
```text
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
|
||||
A pipeline profile should define one shared front end and one or more artifact
|
||||
lanes:
|
||||
|
||||
- shared input module;
|
||||
- shared chunk module by default;
|
||||
- artifact lanes containing extract, merge, normalize, and validator behavior;
|
||||
- shared output module.
|
||||
|
||||
The MVP should use one shared chunk module per pipeline. Per-lane chunk
|
||||
overrides can be added later if an artifact lane, such as combat, proves it
|
||||
needs a different chunking strategy.
|
||||
|
||||
Example shape:
|
||||
|
||||
```yaml
|
||||
llm_profiles:
|
||||
default:
|
||||
model: example-model
|
||||
max_concurrency: 4
|
||||
|
||||
pipelines:
|
||||
dnd-session:
|
||||
input: seriatim
|
||||
chunk: dnd/transcript
|
||||
artifacts:
|
||||
spells:
|
||||
extract: dnd/spells
|
||||
normalize: dnd/spells
|
||||
npcs:
|
||||
extract: dnd/npcs
|
||||
items:
|
||||
extract: dnd/items
|
||||
```
|
||||
|
||||
The CLI should run named pipelines:
|
||||
|
||||
```sh
|
||||
notarius run dnd-session --input session-014.json
|
||||
notarius run dnd-session --input session-014.json --only spells,npcs
|
||||
```
|
||||
|
||||
`--only` should select configured artifact lanes. It should not create an
|
||||
ad hoc pipeline. Structural module selection should come from config, while CLI
|
||||
flags may override operational knobs such as model, concurrency, output
|
||||
directory, and diagnostics directory.
|
||||
|
||||
Initial defaults:
|
||||
|
||||
- `chunk`: `generic`;
|
||||
- lane `merge`: `appendorder`;
|
||||
- lane `normalize`: `noop`;
|
||||
- `output`: `json`;
|
||||
- `llm_profile`: `default` where an LLM profile is needed.
|
||||
|
||||
Module bindings should support both string shorthand and object form:
|
||||
|
||||
```yaml
|
||||
extract: dnd/spells
|
||||
```
|
||||
|
||||
```yaml
|
||||
extract:
|
||||
module: dnd/spells
|
||||
llm_profile: fast
|
||||
prompt_version: v1
|
||||
```
|
||||
|
||||
Both forms should normalize into a single internal `ModuleBinding` shape before
|
||||
validation and manifest hashing.
|
||||
|
||||
Pipeline validation should use module metadata declared through registries.
|
||||
Modules should expose flat string capability metadata, such as `speaker` or
|
||||
`timestamps`, without requiring module construction. Config validation should
|
||||
fail fast for:
|
||||
|
||||
- unknown pipeline IDs;
|
||||
- unknown module keys;
|
||||
- missing required slots;
|
||||
- missing required capabilities;
|
||||
- unknown LLM profiles;
|
||||
- empty artifact-lane sets;
|
||||
- `--only` lane names that do not exist in the selected pipeline.
|
||||
|
||||
The MVP should keep pipelines config-file-only. Built-in pipeline profiles can
|
||||
be added later if the project needs embedded defaults, but that introduces
|
||||
merge/override semantics that the MVP does not need.
|
||||
|
||||
The resolved pipeline definition should be hashed after defaults and lane
|
||||
selection are applied. The run manifest should record both `pipeline_id` and
|
||||
`pipeline_digest`; a pipeline ID alone is not stable provenance.
|
||||
|
||||
## Proposed Pipeline Flow
|
||||
|
||||
The application workflow should be first-class:
|
||||
|
||||
```text
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
|
||||
Proposed runner flow:
|
||||
|
||||
1. Load effective config.
|
||||
2. Resolve the selected pipeline profile by ID.
|
||||
3. Apply defaults and `--only` lane selection.
|
||||
4. Validate module keys, lane definitions, LLM profiles, and capabilities.
|
||||
5. Hash the resolved pipeline definition.
|
||||
6. Create diagnostics run directory.
|
||||
7. Resolve the configured input module through the input adapter registry.
|
||||
8. Read source input.
|
||||
9. Parse source input into a `SourceDocument`.
|
||||
10. Validate source-document invariants.
|
||||
11. Resolve the configured chunker.
|
||||
12. Chunk source units into deterministic source chunks.
|
||||
13. Resolve configured artifact lanes through registries.
|
||||
14. Extract, merge, normalize, and validate each selected artifact lane.
|
||||
15. Retain approved artifacts and rejected-artifact diagnostics.
|
||||
16. Serialize output files and run-level manifest/index.
|
||||
17. Write diagnostics and optional report JSON.
|
||||
|
||||
The runner should operate on source documents and source chunks only. Any
|
||||
transcript-specific behavior should happen before the runner, inside the input
|
||||
adapter, or after the runner, inside output rendering that understands source
|
||||
metadata.
|
||||
|
||||
## Audita Patterns To Reuse
|
||||
|
||||
Reuse these architectural patterns:
|
||||
|
||||
- deterministic parsing and schema validation style;
|
||||
- deterministic chunking of ordered source units;
|
||||
- explicit extractor registry;
|
||||
- explicit pipeline stage contracts;
|
||||
- `contracts` package for transport-neutral interfaces;
|
||||
- OpenAI-compatible structured LLM client;
|
||||
- scheduler for bounded LLM concurrency;
|
||||
- embedded prompt registry with prompt metadata and hashes;
|
||||
- embedded response-schema registry with schema metadata and hashes;
|
||||
- diagnostics run directory with redacted effective config;
|
||||
- validator decision cardinality and deterministic validator ordering;
|
||||
- CLI tests and fixture-driven integration tests.
|
||||
|
||||
The fixture-driven integration-test pattern should remain part of the codebase:
|
||||
walking skeleton tests over fake modules and fake LLM clients should be
|
||||
preserved as real Seriatim, runtime, and D&D modules are added, so the
|
||||
end-to-end contract coverage is not lost.
|
||||
|
||||
Avoid copying these Audita concepts directly:
|
||||
|
||||
- transcript-specific core types;
|
||||
- correction proposals;
|
||||
- replacement policies;
|
||||
- deterministic transcript mutation;
|
||||
- correction ledger terminology.
|
||||
|
||||
Those concepts are specific to Audita's transcript-editing role and should be
|
||||
replaced with source-document, artifact-candidate, artifact-validation, and
|
||||
extraction-report concepts.
|
||||
|
||||
## Checkpoint Roadmap
|
||||
|
||||
The initial six checkpoint roadmap has been implemented and retired. The
|
||||
checkpoint files have been removed from `docs/roadmap/` so the active roadmap
|
||||
does not compete with completed implementation history.
|
||||
|
||||
The remaining work needed to reach the first functional MVP is tracked in
|
||||
[`mvp.md`](mvp.md). Future staged implementation plans should be written to
|
||||
[`implementation.md`](implementation.md) from that active MVP roadmap.
|
||||
|
||||
## Architecture Decisions
|
||||
|
||||
- Final durable output should use one artifact file per artifact type plus a
|
||||
run-level manifest/index file.
|
||||
- Framework artifact flow should use a generic envelope with `json.RawMessage`
|
||||
payloads. Extract modules should use typed Go structs at their own boundaries.
|
||||
- Schemas should be versioned per extractor, with a separate envelope/manifest
|
||||
format version.
|
||||
- Artifact records should require source references by default. Individual
|
||||
schemas may explicitly opt into ungrounded fields. Artifact-level metadata is
|
||||
exempt.
|
||||
- Source-reference ranges should be preserved exactly. Generic pipeline code
|
||||
should not merge or rewrite overlapping ranges.
|
||||
- `ExtractionRequest` should carry the active chunk plus optional ambient
|
||||
context for document synopsis, prior-chunk summaries, known entities, or
|
||||
similar module-provided state.
|
||||
- LLM-backed review should be part of module-owned validator chains. Extract
|
||||
modules and normalize modules may both use deterministic and LLM-backed
|
||||
validators.
|
||||
- The Seriatim MVP should support only the minimal Seriatim schema. Broader
|
||||
Seriatim schema support should be added later without changing core source
|
||||
contracts.
|
||||
- Core source metadata should remain `map[string]any`. Well-known metadata keys
|
||||
should be documented as conventions, and input modules may expose typed
|
||||
accessor helpers for their own metadata.
|
||||
- Configuration should use named pipeline profiles selected by ID at the CLI.
|
||||
- A pipeline profile should be a fixed template, not a free-form DAG: shared
|
||||
input and chunk stages, one or more artifact lanes, and shared output.
|
||||
- `--only` should select configured artifact lanes without creating ad hoc
|
||||
pipelines.
|
||||
- Module bindings should support string shorthand and inline object settings,
|
||||
normalized into one internal binding shape.
|
||||
- Registries should expose flat capability metadata so config can fail fast on
|
||||
invalid module combinations.
|
||||
- The run manifest should record both `pipeline_id` and a digest of the resolved
|
||||
pipeline definition after defaults and lane selection.
|
||||
|
||||
## Open Design Questions
|
||||
|
||||
- Which artifact types should use generic append-in-chunk-order merge, and which
|
||||
should use domain-specific merge?
|
||||
- Which artifact types need domain-specific normalization for deduplication,
|
||||
identity resolution, or consistency?
|
||||
- Which operational settings should be allowed as CLI/environment overrides
|
||||
without weakening pipeline provenance?
|
||||
|
||||
## Near-Term Documentation Tasks
|
||||
|
||||
Once behavior is implemented, move implemented contracts out of roadmap docs and
|
||||
into canonical docs:
|
||||
|
||||
- `README.md` for purpose and shortest useful command;
|
||||
- `docs/cli.md` for CLI behavior;
|
||||
- `docs/config.md` for config fields and precedence;
|
||||
- `docs/internal/` for implemented architecture and package boundaries;
|
||||
- `docs/integrations/` for source input and artifact file formats;
|
||||
- `examples/` for maintained source, config, and artifact examples.
|
||||
@@ -1,157 +1,29 @@
|
||||
# MVP Roadmap
|
||||
# Future Work
|
||||
|
||||
## Status
|
||||
Current Notarius behavior is documented in the canonical README, CLI,
|
||||
configuration, operations, internal, and integration docs. This roadmap records
|
||||
future work only.
|
||||
|
||||
The first functional Notarius MVP implementation is complete enough to start the
|
||||
deferred documentation pass.
|
||||
## Candidate Product Work
|
||||
|
||||
The previous numbered checkpoint roadmaps have been implemented and retired.
|
||||
This document now records the implemented MVP scope and the remaining
|
||||
release/documentation work before alpha `0.1.0`.
|
||||
- Additional input adapters, such as Markdown or note-export formats.
|
||||
- Additional D&D extractors beyond spell casts.
|
||||
- Cross-lane entity normalization.
|
||||
- Cross-chunk semantic deduplication.
|
||||
- Configurable validator chains with production validator modules.
|
||||
- Multiple effective LLM profiles in one run.
|
||||
- Parallel execution where it preserves deterministic manifests and diagnostics.
|
||||
- Additional output encoders.
|
||||
|
||||
Implementation staging belongs in [`implementation.md`](implementation.md).
|
||||
## Candidate Operational Work
|
||||
|
||||
## MVP Goal
|
||||
- Packaged release artifacts for alpha distribution.
|
||||
- A documented versioning and release process.
|
||||
- Optional generated example output fixtures with a regeneration procedure.
|
||||
- Additional diagnostics or reporting views if operator workflows need them.
|
||||
|
||||
The MVP should let a user run Notarius against a Seriatim minimal transcript
|
||||
JSON file, select a configured pipeline profile, extract D&D spell-cast
|
||||
artifacts with an LLM-backed extractor, validate those artifacts, and write
|
||||
durable JSON output plus diagnostics.
|
||||
## Non-Goals To Revisit Deliberately
|
||||
|
||||
The intended command shape is:
|
||||
|
||||
```sh
|
||||
notarius run dnd-session --input session-014.json
|
||||
notarius run dnd-session --input session-014.json --only spells
|
||||
```
|
||||
|
||||
The MVP remains intentionally narrow:
|
||||
|
||||
- one production input module: `seriatim`;
|
||||
- one production extract module: `dnd/spells`;
|
||||
- one generic chunk module sufficient for transcript-scale processing;
|
||||
- generic append-order merge;
|
||||
- generic noop normalization;
|
||||
- generic JSON output;
|
||||
- config-driven pipeline profiles;
|
||||
- OpenAI-compatible structured LLM execution through the existing LLM client.
|
||||
|
||||
## Implemented MVP Behavior
|
||||
|
||||
The MVP now includes:
|
||||
|
||||
- framework/domain asset boundaries: D&D spell prompt and response schema assets
|
||||
are owned by `internal/modules/extract/dnd/spells`, while framework prompt and
|
||||
LLM packages provide only generic primitives;
|
||||
- production CLI wiring that registers `seriatim`, `dnd/spells`, and the
|
||||
default `generic`, `appendorder`, `noop`, and `json` modules;
|
||||
- a config-driven `notarius run` command that reads Seriatim input, resolves a
|
||||
configured pipeline, invokes the runner, and writes durable output plus
|
||||
diagnostics;
|
||||
- fixture-driven CLI acceptance coverage using maintained example config and
|
||||
transcript fixtures with a fake LLM path, so `go test ./...` exercises the MVP
|
||||
without network access.
|
||||
|
||||
The production pipeline defaults are:
|
||||
|
||||
- `chunk: generic`;
|
||||
- `merge: appendorder`;
|
||||
- `normalize: noop`;
|
||||
- `output: json`.
|
||||
|
||||
The implemented command shape is:
|
||||
|
||||
```sh
|
||||
notarius run <pipeline-id> --input path/to/source.json
|
||||
notarius run <pipeline-id> --input path/to/source.json --only spells
|
||||
```
|
||||
|
||||
Current output and diagnostics behavior:
|
||||
|
||||
- output is written under `<output-root>/<run-id>/`, defaulting to
|
||||
`./notarius-output`;
|
||||
- JSON output includes a manifest, grouped approved artifacts, rejected
|
||||
artifacts, and warnings;
|
||||
- artifact records include generic source references;
|
||||
- output-stage warnings remain out-of-band from artifact payloads and are
|
||||
captured for CLI reporting and diagnostics;
|
||||
- run manifest data includes source digest, resolved pipeline digest, LLM
|
||||
profile/model metadata, prompt/schema identifiers, and validation status;
|
||||
- diagnostics redact secrets and include invocation metadata, redacted effective
|
||||
config, resolved pipeline, manifest, warnings, run report, and error logs.
|
||||
|
||||
Maintained MVP fixtures:
|
||||
|
||||
- `examples/dnd-spells.config.yml`;
|
||||
- `examples/seriatim-minimal-transcript.json`;
|
||||
- `internal/cli/testdata/invalid-seriatim-empty-segments.json`.
|
||||
|
||||
## Remaining Release And Documentation Work
|
||||
|
||||
The full documentation pass is intentionally deferred until MVP functionality
|
||||
exists. It should happen before tagging alpha `0.1.0`.
|
||||
|
||||
Remaining work before alpha `0.1.0`:
|
||||
|
||||
- move implemented CLI behavior into `docs/cli.md`;
|
||||
- move implemented config behavior into `docs/config.md`;
|
||||
- document run output and diagnostics behavior in canonical docs;
|
||||
- update integration docs for Seriatim input, D&D spell artifacts, and JSON
|
||||
output where needed;
|
||||
- update `README.md` with a shortest useful command based on the maintained
|
||||
examples;
|
||||
- keep examples validated by tests.
|
||||
|
||||
## Out Of Scope For MVP
|
||||
|
||||
- D&D item extraction;
|
||||
- NPC extraction;
|
||||
- combat extraction;
|
||||
- D&D rules validation beyond the spell extractor's deterministic checks;
|
||||
- Markdown or Obsidian input;
|
||||
- cross-lane entity normalization;
|
||||
- cross-chunk semantic deduplication beyond whatever a simple normalizer can
|
||||
safely support;
|
||||
- a general DAG or workflow engine;
|
||||
- ad hoc CLI flags for structural module selection;
|
||||
- release-quality documentation before the MVP behavior is implemented.
|
||||
|
||||
## MVP Done Criteria
|
||||
|
||||
- D&D prompt and response schema assets are owned by the D&D spells module, not
|
||||
by framework packages.
|
||||
- Production CLI commands use a real app catalog rather than test-injected
|
||||
module catalogs.
|
||||
- A config profile can bind `input: seriatim` and an artifact lane with
|
||||
`extract: dnd/spells`.
|
||||
- Default `generic`, `appendorder`, `noop`, and `json` modules resolve through
|
||||
production wiring.
|
||||
- `notarius config validate --config <file> --pipeline <id>` works with the
|
||||
MVP config.
|
||||
- `notarius pipelines list --config <file>` works with the MVP config.
|
||||
- `notarius run <pipeline-id> --input <file>` reads a Seriatim transcript,
|
||||
extracts D&D spell artifacts, validates them, and writes JSON output.
|
||||
- `notarius run <pipeline-id> --input <file> --only spells` runs only the
|
||||
selected artifact lane.
|
||||
- The run manifest records source digest, resolved pipeline digest, LLM profile
|
||||
and model, prompt/schema identifiers, and validation status.
|
||||
- Output warnings are available to CLI/diagnostics without becoming artifact
|
||||
payload fields.
|
||||
- MVP fixture tests cover the full path without network access.
|
||||
- `go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` pass.
|
||||
|
||||
## Deferred Documentation Pass
|
||||
|
||||
Before alpha `0.1.0`, complete a full documentation pass/rewrite. That pass
|
||||
should move implemented behavior out of roadmap documents and into canonical
|
||||
docs required by
|
||||
[`../policy/documentation.md`](../policy/documentation.md), including at least:
|
||||
|
||||
- `README.md`;
|
||||
- `docs/cli.md`;
|
||||
- `docs/config.md`;
|
||||
- `docs/operations.md`, if diagnostics/run recovery behavior warrants it;
|
||||
- `docs/internal/` architecture and package-boundary docs;
|
||||
- `docs/integrations/` updates for Seriatim input, D&D spell artifacts, and
|
||||
JSON output;
|
||||
- maintained `examples/` files.
|
||||
- A general workflow language.
|
||||
- Structural module selection through ad hoc run flags.
|
||||
- Storing secrets in config files, diagnostics, manifests, or examples.
|
||||
|
||||
Reference in New Issue
Block a user