narratio/docs/roadmap/implementation.md

Notarius v0.6 CLI References Implementation Plan

Purpose And Status

This is the executable implementation plan for the accepted target state in Notarius v0.6 CLI Reference Integration. It is written for a gpt-5.6-terra coding agent that will implement exactly one pending stage per prompt, in order.

The feature roadmap owns user intent, architectural boundaries, settled policy, and the target end state. This document owns delivery order, concrete changes, test allocation, and implementation status. Do not restate or change a roadmap decision here during implementation; if current Notarius v0.6.0 evidence contradicts the roadmap, stop and record the conflict instead of inventing a different contract.

Stage	Outcome	Status
1	Add the reference-selector and configuration vocabulary, including optional spell-catalog inputs.	Completed
2	Materialize and inventory the optional spell catalog through the prepare and operator lifecycle.	Completed
3	Centralize manifest-authoritative prepared-input resolution and migrate analyze to it.	Completed
4	Add deterministic Notarius v0.6 reference arguments at the subprocess adapter boundary.	Completed
5	Resolve references in extract and bind fingerprints, resume, and metadata to their identities.	Completed
6	Prove assembled extraction lifecycle and downstream invalidation behavior.	Pending
7	Update canonical documentation and maintained examples for the completed feature.	Pending
8	Perform compatibility, quality, and repository-wide closure validation.	Pending

Governing Decisions

The following requirements are settled and are not questions for the implementing agent:

1. pipeline.notarius.references is a map whose key is a Notarius v0.6 CLI reference selector and whose value is a prepared Narratio source ID. It is not a map of paths.
2. Every configured binding is required. There is no per-entry required field. An optional Notarius reference is omitted by omitting the map entry. An empty or omitted map remains valid for custom pipelines and backward compatibility. The map is limited by the centrally declared configuration constant MaxNotariusReferenceBindings = 256, which is far above the four-entry maintained D&D case while bounding argv and manifest growth.
3. The supported prepared reference sources are narratio.input.party, narratio.input.players, narratio.input.glossary, and narratio.input.spell_catalog. Arbitrary Notarius slot names and qualified selectors may bind those sources; direct paths and later-stage artifacts may not.
4. Campaign and session spell_catalog_file are optional. A session value overrides the campaign value; an empty session value inherits the campaign value. A narratio.input.spell_catalog reference binding requires an effective configured file.
5. Prepared sources are authoritative only when the current session manifest records the matching canonical input and checksum. Consumers do not fall back to campaign/session source paths or accept an incidental workspace file.
6. Narratio passes absolute prepared-file paths to Notarius. Fingerprints and metadata use the canonical workspace-relative path identity together with selector, source ID, SHA-256 checksum, and size so workspace relocation does not become the only identity signal.
7. External CLI references are sorted by normalized selector. They override matching external references in the Notarius configuration. Narratio never emits --without-reference, the deprecated roster alias, or CLI bindings for generated D&D artifact handoffs.
8. Notarius remains authoritative for whether a selected target declares a slot, accepted reference media types and sizes, generated-handoff collisions, pipeline topology, and D&D payload schemas. Narratio validates selector structure and its own source contract only.
9. Notarius v0.6.0 is the minimum supported CLI contract when references are configured. Do not add version-string parsing or an automatic per-session notarius config validate subprocess.
10. The existing receipt-v2, bundle-confinement, diagnostic, ten-lane selection, immutable promotion, and analysis-source behavior must remain intact.
11. The default test suite remains offline, deterministic, and independent of a sibling checkout or installed Notarius binary. A real v0.6.0 smoke run is useful supplementary evidence when locally available, not a default-suite dependency.
12. Add no external Go dependency for this feature. Use narrow owner-specific types and existing file, path, artifact, manifest, adapter, and stage facilities.

Instructions For Every Stage

For each implementation prompt, the coding agent must:

1. Read docs/development.md, all three files under docs/policy/, the feature roadmap, this plan, and the stage-specific documents and source named below. Inspect the current tree because earlier stages may have changed names or ownership boundaries.

2. Use the repository knowledge graph first for code discovery and call tracing; use text search for documentation, configuration, examples, string literals, and evidence the graph cannot supply.

3. Confirm the worktree state and preserve unrelated changes. Implement only the current stage. Do not begin a later stage merely because an adjacent file is open.

4. Keep production code, focused tests, fakes, and fixtures consistent within the stage. Remove superseded helpers when their final caller migrates. Do not retain two competing source maps, path resolvers, fingerprint paths, or subprocess argument builders.

5. Follow the testing policy's ownership rule. Parser/config tests own selector and configuration cases; artifact tests own prepared-file identity and integrity; adapter tests own exact arguments; stage tests own orchestration and resume; application tests own lifecycle invalidation. Do not repeat every lower-level case at higher levels.

6. Keep errors actionable and content-free. They may identify a selector, Narratio source ID, canonical path, or checksum mismatch, but must not include reference contents. Preserve ordinary group-workspace permissions and restrictive API-key handling.

7. Run gofmt on changed Go files and focused tests while iterating. Before marking any stage complete, run at minimum:

   go test ./...
   go test -race ./...
   go vet ./...
   go build ./...
   go test ./internal/doccheck
   go test ./internal/config -run '^TestExamplesLoadAndValidate$'
   

   Default tests must not contact live services or require credentials.

8. Compare the final diff against the stage goal and exit criteria. Update only the current stage's status row from Pending to Completed. Do not mark a stage complete while a required check fails or required behavior is absent. Intermediate commits are implementation-branch state and must not be released before Stage 7 has reconciled current-behavior documentation.

Stage 1 — Reference And Configuration Vocabulary

Read first: docs/config.md, docs/integrations/notarius.md, internal/config/config.go, internal/config/defaults.go, internal/config/load.go, internal/config/validate.go, internal/config/notarius_test.go, internal/config/campaign_config_test.go, and internal/artifactpolicy/policy.go and its tests. Read the tagged Notarius v0.6.0 docs/cli.md reference-selector section from ../notarius when that checkout is available; otherwise use the canonical link from the feature roadmap.

Depends on: None.

Goal: Establish one normalized reference-selector grammar and the strict configuration model needed by later stages, without adding path-valued Notarius configuration or making spell catalogs mandatory for every campaign.

Work:

• Add a small dependency-free internal/notariusref package as the contract owner for Notarius reference selector normalization. Its exported normalizer must trim the selector and each component, reject empty components and =, and accept only the v0.6 forms slot, chunk.slot, lane.slot, lane.extract.slot, lane.merge.slot, and lane.normalize.slot. A three-component selector accepts only extract, merge, or normalize in its middle component. Do not check the selector against a Notarius module or lane registry.
• Add References map[string]string with YAML key references to config.NotariusConfig. During enabled Notarius validation, sort raw keys, normalize each selector through the shared contract, trim each source value, reject empty values and normalized-selector collisions, require each value to be one of the four prepared reference sources, and replace the config map with its normalized form. Enforce the named, centrally discoverable config.MaxNotariusReferenceBindings limit of 256 entries with an error that identifies the field and limit. Keep a nil/empty map valid. Rely on strict YAML decoding to reject duplicate identical keys, but explicitly reject distinct raw keys that normalize to one selector.
• Add SpellCatalogFile string with YAML key spell_catalog_file to campaign inputs and session inputs, plus SpellCatalogFile ResolvedInputFile to resolved stable inputs. Merge it with the existing session-over-campaign helper. It is not part of the campaign-required input set. Reject a non-empty configured scalar that becomes empty after trimming.
• Add artifactpolicy.SourceInputSpellCatalog and make the artifact-policy owner describe all four prepared reference sources, including their canonical manifest kind and filename. Use that owner for stable-source recognition instead of adding a second switch in configuration validation. Preserve the existing three source IDs and their behavior.
• Extend cross-configuration validation so a normalized reference to narratio.input.spell_catalog requires a non-empty effective resolved spell_catalog_file. Existing campaign requirements already guarantee party, players, and glossary declarations. Do not check filesystem existence during configuration validation.
• If analyze's current private filename switch must change to keep the tree behaviorally coherent, make it delegate to the artifact-policy descriptor and recognize spell catalog; Stage 3 will replace the filesystem-only resolver.

Tests and exit criteria: At the contract/config owners, cover every accepted selector shape; zero, empty, excess, invalid-stage, and = forms; whitespace normalization; normalized collisions; unsupported and empty source IDs; nil and empty maps; exactly the configured binding limit and limit plus one; strict unknown fields; campaign inheritance and session override; optional omission; and the cross-config missing-spell-catalog failure. Prefer table-driven parser and validator tests over assertions against private helper structure. Existing configuration and example tests must still pass without adding spell catalogs to every campaign. The codebase has one selector grammar owner and one prepared-source descriptor owner.

Stage 2 — Spell Catalog Prepare And Operator Lifecycle

Read first: docs/internal/stage-prepare.md, docs/internal/workspace.md, docs/operations.md, internal/stage/prepare.go and its tests, internal/app/operator_inspection.go, internal/app/operator_findings.go and their focused tests, internal/manifest/manifest.go, and the relevant confined file-operation helpers.

Depends on: Stage 1.

Goal: Make the effective optional spell catalog a normal prepared session input with canonical storage, checksum/provenance, safe stale-file handling, and operator visibility.

Work:

• Resolve StableInputs.SpellCatalogFile with the same origin-preserving campaign/session behavior as the five existing stable inputs. When configured, require a regular readable source, copy it atomically to inputs/spell_catalog.json, preserve ordinary workspace permissions, and add one manifest input record with kind spell_catalog, canonical destination, checksum, and campaign_config or session_config source provenance.
• Treat the input as optional when no effective path is configured. Do not call the required-input path resolver with an empty value and do not create a manifest record. Remove an obsolete canonical inputs/spell_catalog.json without following it when a forced prepare transitions from configured to absent; refuse to recursively remove a directory or other ambiguous object at that exact file path.
• Include a configured spell catalog in operator inspection and validation findings. Omission is not an error unless Stage 1 cross-configuration policy says the enabled Notarius reference requires it. Reuse the common resolved stable-input enumeration where practical instead of extending parallel hand-written lists in several functions.
• Adjust input-slice capacity, deterministic ordering, test fixtures, and any manifest assumptions affected by the optional sixth stable file. Do not parse or schema-validate the JSON payload in Narratio; Notarius owns that contract.

Tests and exit criteria: Through prepare and operator package behavior, cover campaign and session source provenance, canonical destination bytes and checksum, optional omission, missing configured source, replacement after source change, safe removal when configuration is removed, rejection of an ambiguous destination object, deterministic manifest ordering, and operator reporting. Do not duplicate selector-validation cases from Stage 1. Existing sessions with no spell catalog remain valid and produce no stale manifest entry.

Stage 3 — Manifest-Authoritative Prepared Input Resolution

Read first: docs/internal/artifacts.md, docs/internal/manifest.md, docs/internal/stage-analyze.md, internal/artifacts/artifact_resolver.go, internal/artifacts/resolve.go, internal/artifacts/checksum.go, their tests, and the prepared stable-input resolution path in internal/stage/analyze.go and internal/stage/analyze_test.go.

Depends on: Stage 2.

Goal: Give analyze and extract one integrity-checked resolver for prepared stable sources so neither stage trusts incidental files or reconstructs its own source-to-filename table.

Work:

• Add an artifacts-owned PreparedInputIdentity contract containing source ID, manifest kind, absolute canonical path, slash-separated path relative to the session root, SHA-256 checksum, and byte size. Add one resolver that accepts session paths, the current session manifest, and a stable source ID. Provide a typed or sentinel absence classification so callers can distinguish no current manifest record from corrupt or unsafe recorded evidence.
• Derive kind and filename exclusively from the artifact-policy descriptor. The resolver must require exactly one current manifest input record with the expected kind and canonical path; resolve/rebase recorded local paths through existing session-local path safety helpers; require the result to equal the canonical file below inputs/; reject escapes, symlinks, non-regular files, empty files, missing checksums, duplicate records, and checksum mismatches; and calculate size without loading the complete file into memory. Do not fall back to the configured campaign/session path or accept canonical file presence without manifest evidence. Zero matching manifest records is the typed absent case; once a record exists, a missing or invalid file is an integrity error, not optional absence.
• Return owner-neutral errors from internal/artifacts. At stage boundaries, wrap unavailable or stale prepared inputs with the source ID and actionable narratio run-stage prepare <session_id> --force guidance. Do not include file contents.
• Replace analyze's private prepared-source filename switch and filesystem-only resolver with the shared artifact resolver. Preserve required-versus-optional Scriptorium input behavior: a typed absent optional source is omitted, an absent required source fails with prepare guidance, and invalid recorded evidence fails regardless of optionality. Make narratio.input.spell_catalog usable wherever another prepared Scriptorium source is accepted.

Tests and exit criteria: Artifact-package tests own valid resolution and the missing-record, duplicate-record, wrong-kind/path, traversal/rebase, symlink, non-regular, empty, missing-checksum, and checksum-mismatch boundaries. Analyze tests need only prove required/optional stage behavior and successful use of the shared source, including spell catalog; do not repeat the artifact resolver's full matrix. Remove the old filename/path resolver after its final caller moves.

Stage 4 — Notarius Adapter Reference Arguments

Read first: docs/internal/adapters.md, docs/integrations/notarius.md, internal/adapters/notarius/runner.go, fake.go, subprocess.go, and focused adapter tests. Re-read the Notarius v0.6.0 subprocess and CLI reference-selector contracts from the tagged sibling checkout when available.

Depends on: Stage 1.

Goal: Extend the transport-neutral Notarius request and production adapter to emit safe, exact, repeatable v0.6 --reference arguments without changing receipt or bundle ingestion.

Work:

• Add a transport-neutral reference binding containing normalized selector and absolute path, and add an ordered slice of those bindings to RunRequest. Keep source IDs and manifest identities out of the adapter contract; those are stage policy.
• Validate each adapter binding before process launch: normalize/validate the selector through the shared Stage 1 contract, require a non-empty absolute path, reject duplicate normalized selectors, and avoid mutating the caller's slice. Do not open or parse the reference file in the adapter.
• Build arguments as repeated pairs --reference, <normalized-selector>=<absolute-path> after --output-dir and before --json. Preserve one argument for the combined selector/path value so spaces, additional = characters within the path portion, and platform separators do not involve shell interpretation. The request order is authoritative; Stage 5 will supply sorted bindings.
• Preserve current executable, environment, timeout, cancellation, diagnostic, receipt-v2, bounded-read, confinement, and bundle-discovery behavior. Do not add --without-reference, generated reference arguments, version probing, or configuration preflight.
• Update the fake only as required to retain and expose the extended request.

Tests and exit criteria: Adapter tests own exact argv with zero and multiple references, position before --json, spaces and = in paths, selector normalization, duplicate/invalid selector rejection, relative/empty path rejection, and no subprocess start after request-validation failure. Existing receipt-v2 and bundle fixture tests must remain unchanged in meaning and pass. Do not assert stage-level source sorting here beyond preserving the request order.

Stage 5 — Extract Reference Identity, Invocation, And Resume

Read first: docs/internal/stage-extract.md, docs/integrations/notarius.md, docs/internal/manifest.md, internal/stage/extract.go, internal/stage/extract_resume.go, their focused tests, the Stage 3 prepared-input identity contract, and the Stage 4 Notarius request contract.

Depends on: Stages 3 and 4.

Goal: Make configured references part of the actual extraction invocation and durable reuse contract, using one resolution path for initial execution and resume validation.

Work:

• Add one extract-owned reference-resolution helper used by both Run and ValidateResume. Iterate normalized config bindings in lexical selector order, resolve each source through the Stage 3 manifest-authoritative resolver, and produce both adapter bindings and immutable reference identities. Resolve every reference before creating run-local receipt, log, output, or promotion directories and before invoking the adapter.
• Define the fingerprint/metadata identity as normalized selector, source ID, canonical session-relative slash path, SHA-256 checksum, and byte size. Do not include contents or original campaign/session absolute paths. Pass only selector and absolute prepared path to the adapter.
• Extend the extraction fingerprint document with the sorted reference identities. Keep all existing binary, config path, pipeline, timeout, working directory, trimmed-transcript identity, and required-output identities. The result must be independent of YAML map iteration order and must change for a selector, source, relative path, checksum, or size change.
• Persist reference_count and a bounded deterministic references metadata list on successful extraction. Each entry contains exactly selector, source_id, path, checksum, and size_bytes. Empty bindings produce count zero and an empty list. Do not duplicate Notarius reference payloads or downstream error messages.
• Make resume recompute current reference identities through the same helper before comparing the configuration fingerprint. A valid changed prepared input yields a fingerprint mismatch and a non-resumable result so extraction reruns. Missing, unsafe, or checksum-inconsistent current input is an error with prepare-force guidance because immediately rerunning extract cannot succeed. Do not silently reuse the old bundle.
• Preserve explicit disabled-stage skip without resolving references. Preserve required lane selection, immutable promotion, receipt identity, and bundle evidence behavior.

Tests and exit criteria: Stage tests own sorted request construction for all four D&D bindings, zero bindings, failure before adapter invocation for an unavailable source, content-free metadata, and fingerprint changes for each identity field while remaining stable across map order. Resume tests must prove reuse with unchanged references, non-reuse after a valid prepared-reference change, hard failure for missing or checksum-invalid current evidence, and no reference resolution when disabled. Use the fake adapter; do not duplicate exact subprocess argv cases from Stage 4.

Stage 6 — Assembled Lifecycle And Invalidation Coverage

Read first: docs/internal/overview.md, docs/internal/manifest.md, docs/internal/stage-extract.md, docs/internal/stage-prepare.md, internal/app/runner.go, internal/app/extract_lifecycle_test.go, and representative full pipeline and stage fixtures. Inspect existing downstream invalidation tests before adding new cases.

Depends on: Stage 5.

Goal: Prove at the application boundary that prepared campaign context reaches Notarius and that reference changes cannot leave extraction or later analysis falsely current.

Work:

• Extend the smallest existing assembled runner fixture to execute prepare and extract with party, players, glossary, and spell catalog bindings. Assert that the fake Notarius request receives the four canonical prepared absolute paths, not the original campaign/session source paths, and that the successful manifest records bounded reference identity.
• Add one lifecycle regression covering a valid reference-content change: rerun/force prepare so the manifest and prepared checksum change, then verify extract resume is rejected, Notarius runs again, and succeeded canonical downstream stages are invalidated according to the existing stage-order policy. Assert outcomes, not private runner call choreography.
• Add one representative session override case to prove the overridden prepared bytes/checksum reach extraction. Do not repeat all four configuration merge cases or artifact-integrity failures already owned by earlier stages.
• Confirm an empty reference map preserves the pre-v0.6 invocation behavior and that all ten configured D&D lanes remain registered as the same narratio.extraction.<key> sources available to analyze.
• Fix production integration defects exposed by these assembled tests without broadening the feature or adding a DAG, generic reference workflow, or direct Notarius payload parsing.

Tests and exit criteria: The application-level tests must be deterministic, offline, and fake only the external Notarius boundary. They must credibly fail if Narratio passes original paths, omits one configured reference, reuses stale extraction, or loses a configured lane, while remaining insensitive to private helper structure and exact non-contractual diagnostics. Earlier focused suites and the repository baseline remain green.

Stage 7 — Canonical Documentation And Maintained Examples

Read first: docs/policy/documentation.md, docs/config.md, docs/operations.md, docs/troubleshooting.md, docs/integrations/notarius.md, docs/internal/overview.md, docs/internal/adapters.md, docs/internal/artifacts.md, docs/internal/stage-prepare.md, docs/internal/stage-extract.md, docs/internal/stage-analyze.md, examples/README.md, and all maintained pipeline, campaign, and session examples affected by the new fields.

Depends on: Stage 6.

Goal: Move the completed behavior from roadmap-only future state into its canonical current-behavior owners and provide valid copyable D&D examples without duplicating volatile Notarius contracts.

Work:

• Update docs/config.md with pipeline.notarius.references, its selector-to- source shape, normalization/validation rules, required-by-presence behavior, supported stable source IDs, and campaign/session spell_catalog_file precedence and optionality. Keep complete copyable YAML in examples/.
• Update docs/integrations/notarius.md to the v0.6.0 baseline and exact repeatable-reference invocation boundary. Explain absolute CLI paths, precedence over configured external paths, the four maintained external D&D slots, the generated-handoff exclusion, and unchanged receipt-v2/ten-lane output compatibility. Link to Notarius's canonical v0.6 CLI and D&D consumer docs instead of copying its target/module matrix.
• Update operations and troubleshooting with prepared input location, fingerprint/rerun consequences, operator inspection, missing-reference diagnosis, and Notarius undeclared-slot/generated-collision failures. Update internal component documents only with implemented ownership and flow; do not duplicate configuration field definitions there.
• Add a valid, secret-free sample spell catalog following Notarius v0.6's published overlay schema, add spell_catalog_file to the sample campaign, and configure all four external reference bindings in the complete annotated D&D pipeline. Add the same bindings to other Notarius-enabled maintained examples only when their selected pipeline declares them; do not add a Notarius section to examples that intentionally omit extraction.
• Ensure the maintained command snippets place repeated --reference arguments before --json, use party rather than roster, and never show generated handoffs on the CLI. Remove stale v0.5 compatibility wording where it refers to the supported invocation baseline.

Tests and exit criteria: Run documentation-link checks and the example loader explicitly. Verify every changed example is accepted by strict config validation, contains no credentials or private infrastructure values, and has one canonical owner for each volatile fact. Search current-behavior docs and examples for stale v0.5 invocation wording, deprecated roster emission, and generated D&D CLI handoff examples. Do not mark the roadmap itself implemented; its status remains target-state context until the implementation sprint is reviewed and closed.

Stage 8 — Compatibility And Quality Closure

Read first: The feature roadmap, every completed stage diff, the final current-behavior docs, .woodpecker/verify.yml, .woodpecker/release.yml, and .woodpecker/shuffle.yml. Re-read the tagged Notarius v0.6.0 docs/consumers/dnd-pipeline.md, docs/cli.md, and linked spell-catalog overlay contract when the sibling checkout is available.

Depends on: Stage 7.

Goal: Verify the delivered code matches the accepted boundary, remains compatible with all ten default D&D artifacts, and is ready for review without dead compatibility paths or duplicated policy.

Work:

• Audit the final diff against every target-state and out-of-scope statement in the feature roadmap. Confirm only four external prepared sources are exposed, custom selectors remain possible, every configured binding is required, and no Notarius pipeline topology or generated-handoff logic moved into Narratio.

• Trace initial extract and resume paths to confirm both use the same prepared identity and reference resolution, the adapter is the sole argv builder, and artifact policy is the sole source-to-kind/filename vocabulary. Remove dead helpers, redundant switches, stale fixtures, and low-value duplicate tests found during this review.

• Confirm the complete D&D example still declares and validates the exact ten output lanes and that analyze can consume those sources after reference- enabled extraction. Confirm empty-reference custom pipelines remain supported.

• If a local Notarius v0.6.0 binary and its required offline/test configuration are already available, perform a non-credentialed smoke invocation with all four reference flags and record the result in the implementation handoff. Do not download tools, contact paid providers, add a default test dependency, or block completion solely because this supplementary environment is absent.

• Run the repository baseline plus the scheduled shuffled suite and release cross-build commands:

  go test ./...
  go test -race ./...
  go test -race -shuffle=on -count=3 ./...
  go vet ./...
  go build ./...
  go test ./internal/doccheck
  go test ./internal/config -run '^TestExamplesLoadAndValidate$'
  narratio_cross_dir="$(mktemp -d)"
  CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o "$narratio_cross_dir/narratio-linux-amd64" ./cmd/narratio
  CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -o "$narratio_cross_dir/narratio-darwin-amd64" ./cmd/narratio
  CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -o "$narratio_cross_dir/narratio-windows-amd64.exe" ./cmd/narratio
  

  Cross-builds are compilation evidence only; do not claim native macOS or Windows runtime validation.

Tests and exit criteria: Every required command passes, git diff --check is clean, the worktree contains no unintended generated test artifacts, and the implementation is traceably complete against the roadmap. Summarize any unavailable supplementary smoke evidence without treating it as a product question or silently weakening the default suite.

Open Questions

None. The feature roadmap and governing decisions above are sufficient to implement the plan without additional product or architecture choices.