notarius/docs/roadmap/implementation.md

Minimal Evidence Context Implementation Plan

Objective

Implement the target state defined in Minimal Evidence Context: retain the optional evidence-context.json output and its existing configuration while replacing its provenance-heavy payload with the ordered, deduplicated union of selected generic source units.

This plan intentionally changes the existing v1 contract in place. Notarius is pre-release, so do not add legacy decoding, schema-version branching, migration, dual publication, or other backward-compatibility machinery.

Global Implementation Rules

Apply these rules in every stage:

• Follow docs/policy/architecture.md, docs/policy/documentation.md, and docs/policy/testing.md.
• Treat the normalized lane artifacts as authoritative for citations and the source document as authoritative for source-unit content.
• Preserve the typed evidence-projection boundary. Generic code must not inspect domain JSON or depend on D&D artifact types.
• Preserve the logical filename evidence-context.json, artifact kind source/evidence-context, media type application/json, schema ID notarius.source.evidence_context, schema name notarius_source_evidence_context_v1, and schema version v1.
• Preserve the current configuration keys, validation, lane allowlist semantics, and default window_units value.
• Do not introduce a payload wrapper. The durable JSON value is a top-level array of generic source units; the empty value is [].
• Preserve source-unit fields and metadata exactly as represented in the validated generic source document. Add no evidence-context-specific fields.
• Do not add byte-count, token-count, or compression-ratio tests. Test the structural boundedness and behavioral contract instead.
• Use focused tests while iterating and leave the repository compiling and its relevant test suites passing at the end of every stage.

Stage 1: Replace The Core Evidence-Context Contract

Goal

Replace the context/provenance model with a strict top-level source-unit array and simplify construction to select the union of expanded source ranges.

Required Changes

1. Update internal/framework/evidencecontext/model.go:
   • represent Document as the durable collection of source.SourceUnit values that serializes directly as a JSON array;
   • retain the existing artifact identity constants;
   • reduce BuildRequest to Source, WindowUnits, and a flat collection of projected SourceRefs;
   • remove Context, EvidenceRef, LaneEvidence, and any other types whose only purpose was durable lane attribution or context grouping.
2. Update internal/framework/evidencecontext/build.go:
   • validate the window and complete source document, including its digest;
   • validate every projected source reference against the source index;
   • convert referenced endpoints to source positions, expand and clamp them, and merge their union;
   • append owned copies of the selected source units once in document-position order;
   • return an initialized empty document when there are no references;
   • remove sorting keys and helpers that exist only to order lane attribution or evidence records.
3. Update internal/framework/evidencecontext/codec.go and internal/framework/evidencecontext/assets/schemas/source_evidence_context.v1.json:
   • define the root schema as an array of strict source-unit objects;
   • preserve the existing source-unit contract: required id, kind, text, and ref, with optional open-ended source-owned metadata;
   • ensure decode rejects null, wrapper objects, malformed units, unknown fixed unit/reference fields, invalid self-references, and noncanonical documents;
   • ensure encode and decode own all slice and nested metadata values and serialize an empty document as [];
   • remove canonicalization logic used only by the deleted wrapper and context structures.
4. Replace the fixture in internal/framework/evidencecontext/testdata/source_evidence_context.v1.json with a compact top-level unit array.
5. Rewrite internal/framework/evidencecontext/evidencecontext_test.go around durable behavior:
   • cover expanded overlapping and adjacent ranges, boundary clamping, repeated references, and non-monotonic unit IDs;
   • prove that the output is source-ordered, contains each selected source unit once, and never invents a unit;
   • prove preservation and ownership of source-unit metadata;
   • cover empty-array encoding and strict invalid-input rejection;
   • retain artifact identity and deterministic round-trip coverage without testing deleted internal structures.
6. Adapt internal/framework/pipeline/evidence_output.go so it flattens the typed references from prepared selected lanes into the new build request. Preserve all existing checks for duplicate accepted lane outputs, source and artifact-kind compatibility, codec decoding, and evidence projection failures.
7. Adapt the pipeline debug summary and its use in runner.go as necessary:
   • calculate selected lanes, window size, and source digest from the prepared plan and source document rather than the payload;
   • report the emitted unit count directly;
   • remove context_count;
   • keep source text and unit metadata out of debug summaries.
8. Rewrite the focused tests in internal/framework/pipeline/evidence_output_test.go to assert the new payload while preserving coverage of selected accepted outputs, absent and rejected lanes, incompatible artifacts, failure propagation, ownership, checkpoint reconstruction, opt-in behavior, and redacted debug reporting.

Acceptance Criteria

• The evidence-context package has no durable or internal model for context groups, lane attribution, or published evidence references.
• Its codec accepts and emits only the new top-level array contract.
• Its builder returns the exact ordered union of source units selected by the expanded validated ranges.
• Pipeline construction still obtains references exclusively through prepared typed evidence projections.
• The payload remains absent when publication is not configured and is [] when enabled with no contributing evidence.
• Focused framework tests pass.

Validation

gofmt -w internal/framework/evidencecontext internal/framework/pipeline/evidence_output.go internal/framework/pipeline/evidence_output_test.go
go test ./internal/framework/evidencecontext ./internal/framework/pipeline

Prompt Scope

This is one coherent but substantial gpt-5.6-terra prompt. Keep the work limited to the core contract, its pipeline construction boundary, and their focused tests; do not update generic JSON output tests, production integration tests, or documentation in this stage.

Stage 2: Update Output And Production Integration Coverage

Goal

Carry the revised artifact through the generic JSON output and representative assembled D&D workflows without changing publication or discovery semantics.

Required Changes

1. Review internal/modules/generic/output/json/encoder.go and retain the existing option decoding, default window, lane validation, logical filename, artifact identity checks, pretty JSON publication, and index.json descriptor. Change production code only where it assumes the old payload.
2. Rewrite evidence-context fixtures and assertions in internal/modules/generic/output/json/encoder_test.go:
   • validate and publish a top-level unit array;
   • retain descriptor, omission, strict artifact validation, and content-nondisclosure failure tests;
   • remove expectations for wrapper fields or contexts.
3. Update evidence assertions in the representative integration and CLI contract tests, including:
   • internal/modules/integration/dnd_npc_grounded_test.go;
   • internal/modules/integration/dnd_location_registry_runner_test.go;
   • internal/cli/dnd_enemy_events_contract_test.go; and
   • any additional test located by searching for evidence-context, EvidenceContext, Contexts, context_count, or old fixture fields.
4. Ensure the integration assertions protect the behavior that matters:
   • only selected accepted lanes contribute references;
   • multiple lanes and overlapping windows produce one ordered unit union;
   • non-monotonic unit IDs do not change source-position order;
   • scene-description selection remains explicit;
   • the output descriptor remains pipeline-wide rather than a lane artifact.
5. Remove obsolete helpers and fixtures that only inspect old context groups or per-lane evidence records. Do not preserve compatibility helpers.

Acceptance Criteria

• The generic JSON encoder publishes the new payload under the existing filename and descriptor.
• Maintained integration and CLI workflows observe a deduplicated array of unchanged source units.
• No production code or test outside the archived roadmap refers to deleted payload fields or types.
• Focused output, integration, and CLI tests pass.

Validation

gofmt -w internal/modules/generic/output/json internal/modules/integration internal/cli
go test ./internal/modules/generic/output/json ./internal/modules/integration ./internal/cli
rg -n 'context_ref|evidence_refs|selected_lanes|source_digest|context_count|\.Contexts\b' internal --glob '*.go' --glob '*.json'

Review every remaining search result rather than requiring the search to be empty: source_digest and similar terms are valid in unrelated contracts and content-free debug state.

Prompt Scope

This stage is appropriately sized for one gpt-5.6-terra prompt. It owns output and assembled-workflow adaptation only; do not revise public documentation in this stage.

Stage 3: Rewrite The Canonical Documentation

Goal

Make all current documentation describe the implemented minimal evidence excerpt, with the durable contract defined in one canonical location.

Required Changes

1. Rewrite docs/integrations/evidence-context.md as the canonical durable contract:
   • retain identity, discovery, filename, and optionality;
   • define the top-level source-unit array and empty-array behavior;
   • define typed-evidence selection, window expansion, union, deduplication, source-position ordering, and unchanged-unit semantics;
   • state that the excerpt can include at most every generic source unit once but has no byte/token-size guarantee;
   • state that lane artifacts remain authoritative for citations and the excerpt contains no lane attribution;
   • preserve sensitivity and consumer validation guidance.
2. Update docs/config.md only where it describes the result of evidence_context. Preserve the existing keys, validation rules, and default window. Link to the integration contract for payload details.
3. Update docs/integrations/json-output.md to describe the logical file as the optional selected source-unit excerpt without duplicating its schema.
4. Update docs/consumers/subprocess.md so consumers decode the array and use lane artifacts—not the excerpt—for authoritative citations and lane provenance.
5. Update docs/operations.md, docs/internal/modules.md, and any other current documentation that describes context groups, provenance records, or the old wrapper. Keep operational sensitivity guidance and internal ownership in their canonical documents.
6. Review examples/dnd-complete.config.yml. Retain the current evidence_context configuration and lane list unless repository inspection reveals an independently stale module key. Do not add a new example solely for this unchanged configuration surface.
7. Do not modify ADRs or add a new ADR. Do not rewrite archived audit history to match current behavior.

Acceptance Criteria

• docs/integrations/evidence-context.md is the sole canonical definition of the payload.
• Other current documents accurately summarize their audience-specific aspect and link to the contract instead of duplicating it.
• No current documentation claims that evidence-context.json contains context groups, evidence-reference records, lane attribution, source-level wrapper metadata, or a guaranteed byte/token reduction.
• The maintained complete example remains valid and reflects implemented configuration.

Validation

rg -n 'context_ref|evidence_refs|selected_lanes|context_count' README.md docs examples --glob '!docs/roadmap/**' --glob '!docs/adr/**'
go test ./internal/config ./internal/modules/generic/output/json ./internal/cli

Inspect any search matches for legitimate historical or unrelated use. If the repository provides no automated Markdown link checker, manually verify every link changed in this stage.

Prompt Scope

This documentation migration is small enough for one gpt-5.6-terra prompt.

Stage 4: Final Verification And Cleanup

Goal

Verify the complete feature across package, integration, documentation, and repository boundaries, then remove only obsolete implementation debris found by that verification.

Required Changes

1. Review the diff against docs/roadmap/evidence-context.md and confirm every acceptance criterion is implemented.
2. Search the non-archived repository for deleted Go types, old JSON fields, obsolete fixture shapes, and statements that describe the former contract.
3. Confirm the schema, codec, builder, pipeline request, debug summary, output encoder, index descriptor, example configuration, and integration contract agree on the final shape and identities.
4. Run formatting and the complete offline validation suite.
5. Make only narrowly scoped corrections required by failed checks or direct roadmap divergence. Do not introduce unrelated refactors or new features.
6. Leave the active roadmap documents in place for post-implementation review; retirement occurs only when separately requested after the implementation is accepted.

Acceptance Criteria

• All roadmap acceptance criteria are satisfied with no legacy compatibility path.
• No current code, schema, fixture, example, or documentation depends on the former wrapper/context/provenance payload.
• The structural bound is protected: the artifact contains only distinct, unchanged units from the source document in source order.
• Repository formatting, tests, vet, and build all pass.
• The worktree contains only intentional feature changes.

Validation

gofmt -w internal/framework/evidencecontext internal/framework/pipeline internal/modules/generic/output/json internal/modules/integration internal/cli
go test ./...
go vet ./...
go build ./cmd/notarius
git diff --check
git status --short

Prompt Scope

This verification pass is small enough for one gpt-5.6-terra prompt. If it uncovers a material unrelated defect, report it separately instead of widening this feature.