notarius/docs/roadmap/implementation.md

# Minimal Evidence Context Implementation Plan

## Objective

Implement the target state defined in
[Minimal Evidence Context](evidence-context.md): retain the optional
`evidence-context.json` output and its existing configuration while replacing
its provenance-heavy payload with the ordered, deduplicated union of selected
generic source units.

This plan intentionally changes the existing v1 contract in place. Notarius is
pre-release, so do not add legacy decoding, schema-version branching,
migration, dual publication, or other backward-compatibility machinery.

## Global Implementation Rules

Apply these rules in every stage:

- Follow `docs/policy/architecture.md`, `docs/policy/documentation.md`, and
  `docs/policy/testing.md`.
- Treat the normalized lane artifacts as authoritative for citations and the
  source document as authoritative for source-unit content.
- Preserve the typed evidence-projection boundary. Generic code must not
  inspect domain JSON or depend on D&D artifact types.
- Preserve the logical filename `evidence-context.json`, artifact kind
  `source/evidence-context`, media type `application/json`, schema ID
  `notarius.source.evidence_context`, schema name
  `notarius_source_evidence_context_v1`, and schema version `v1`.
- Preserve the current configuration keys, validation, lane allowlist
  semantics, and default `window_units` value.
- Do not introduce a payload wrapper. The durable JSON value is a top-level
  array of generic source units; the empty value is `[]`.
- Preserve source-unit fields and metadata exactly as represented in the
  validated generic source document. Add no evidence-context-specific fields.
- Do not add byte-count, token-count, or compression-ratio tests. Test the
  structural boundedness and behavioral contract instead.
- Use focused tests while iterating and leave the repository compiling and its
  relevant test suites passing at the end of every stage.

## Stage 1: Replace The Core Evidence-Context Contract

### Goal

Replace the context/provenance model with a strict top-level source-unit array
and simplify construction to select the union of expanded source ranges.

### Required Changes

1. Update `internal/framework/evidencecontext/model.go`:
   - represent `Document` as the durable collection of `source.SourceUnit`
     values that serializes directly as a JSON array;
   - retain the existing artifact identity constants;
   - reduce `BuildRequest` to `Source`, `WindowUnits`, and a flat collection of
     projected `SourceRefs`;
   - remove `Context`, `EvidenceRef`, `LaneEvidence`, and any other types whose
     only purpose was durable lane attribution or context grouping.
2. Update `internal/framework/evidencecontext/build.go`:
   - validate the window and complete source document, including its digest;
   - validate every projected source reference against the source index;
   - convert referenced endpoints to source positions, expand and clamp them,
     and merge their union;
   - append owned copies of the selected source units once in document-position
     order;
   - return an initialized empty document when there are no references;
   - remove sorting keys and helpers that exist only to order lane attribution
     or evidence records.
3. Update `internal/framework/evidencecontext/codec.go` and
   `internal/framework/evidencecontext/assets/schemas/source_evidence_context.v1.json`:
   - define the root schema as an array of strict source-unit objects;
   - preserve the existing source-unit contract: required `id`, `kind`, `text`,
     and `ref`, with optional open-ended source-owned `metadata`;
   - ensure decode rejects `null`, wrapper objects, malformed units, unknown
     fixed unit/reference fields, invalid self-references, and noncanonical
     documents;
   - ensure encode and decode own all slice and nested metadata values and
     serialize an empty document as `[]`;
   - remove canonicalization logic used only by the deleted wrapper and context
     structures.
4. Replace the fixture in
   `internal/framework/evidencecontext/testdata/source_evidence_context.v1.json`
   with a compact top-level unit array.
5. Rewrite `internal/framework/evidencecontext/evidencecontext_test.go` around
   durable behavior:
   - cover expanded overlapping and adjacent ranges, boundary clamping,
     repeated references, and non-monotonic unit IDs;
   - prove that the output is source-ordered, contains each selected source
     unit once, and never invents a unit;
   - prove preservation and ownership of source-unit metadata;
   - cover empty-array encoding and strict invalid-input rejection;
   - retain artifact identity and deterministic round-trip coverage without
     testing deleted internal structures.
6. Adapt `internal/framework/pipeline/evidence_output.go` so it flattens the
   typed references from prepared selected lanes into the new build request.
   Preserve all existing checks for duplicate accepted lane outputs, source and
   artifact-kind compatibility, codec decoding, and evidence projection
   failures.
7. Adapt the pipeline debug summary and its use in `runner.go` as necessary:
   - calculate selected lanes, window size, and source digest from the prepared
     plan and source document rather than the payload;
   - report the emitted unit count directly;
   - remove `context_count`;
   - keep source text and unit metadata out of debug summaries.
8. Rewrite the focused tests in
   `internal/framework/pipeline/evidence_output_test.go` to assert the new
   payload while preserving coverage of selected accepted outputs, absent and
   rejected lanes, incompatible artifacts, failure propagation, ownership,
   checkpoint reconstruction, opt-in behavior, and redacted debug reporting.

### Acceptance Criteria

- The evidence-context package has no durable or internal model for context
  groups, lane attribution, or published evidence references.
- Its codec accepts and emits only the new top-level array contract.
- Its builder returns the exact ordered union of source units selected by the
  expanded validated ranges.
- Pipeline construction still obtains references exclusively through prepared
  typed evidence projections.
- The payload remains absent when publication is not configured and is `[]`
  when enabled with no contributing evidence.
- Focused framework tests pass.

### Validation

```sh
gofmt -w internal/framework/evidencecontext internal/framework/pipeline/evidence_output.go internal/framework/pipeline/evidence_output_test.go
go test ./internal/framework/evidencecontext ./internal/framework/pipeline
```

### Prompt Scope

This is one coherent but substantial gpt-5.6-terra prompt. Keep the work
limited to the core contract, its pipeline construction boundary, and their
focused tests; do not update generic JSON output tests, production integration
tests, or documentation in this stage.

## Stage 2: Update Output And Production Integration Coverage

### Goal

Carry the revised artifact through the generic JSON output and representative
assembled D&D workflows without changing publication or discovery semantics.

### Required Changes

1. Review `internal/modules/generic/output/json/encoder.go` and retain the
   existing option decoding, default window, lane validation, logical filename,
   artifact identity checks, pretty JSON publication, and `index.json`
   descriptor. Change production code only where it assumes the old payload.
2. Rewrite evidence-context fixtures and assertions in
   `internal/modules/generic/output/json/encoder_test.go`:
   - validate and publish a top-level unit array;
   - retain descriptor, omission, strict artifact validation, and
     content-nondisclosure failure tests;
   - remove expectations for wrapper fields or contexts.
3. Update evidence assertions in the representative integration and CLI
   contract tests, including:
   - `internal/modules/integration/dnd_npc_grounded_test.go`;
   - `internal/modules/integration/dnd_location_registry_runner_test.go`;
   - `internal/cli/dnd_enemy_events_contract_test.go`; and
   - any additional test located by searching for `evidence-context`,
     `EvidenceContext`, `Contexts`, `context_count`, or old fixture fields.
4. Ensure the integration assertions protect the behavior that matters:
   - only selected accepted lanes contribute references;
   - multiple lanes and overlapping windows produce one ordered unit union;
   - non-monotonic unit IDs do not change source-position order;
   - scene-description selection remains explicit;
   - the output descriptor remains pipeline-wide rather than a lane artifact.
5. Remove obsolete helpers and fixtures that only inspect old context groups or
   per-lane evidence records. Do not preserve compatibility helpers.

### Acceptance Criteria

- The generic JSON encoder publishes the new payload under the existing
  filename and descriptor.
- Maintained integration and CLI workflows observe a deduplicated array of
  unchanged source units.
- No production code or test outside the archived roadmap refers to deleted
  payload fields or types.
- Focused output, integration, and CLI tests pass.

### Validation

```sh
gofmt -w internal/modules/generic/output/json internal/modules/integration internal/cli
go test ./internal/modules/generic/output/json ./internal/modules/integration ./internal/cli
rg -n 'context_ref|evidence_refs|selected_lanes|source_digest|context_count|\.Contexts\b' internal --glob '*.go' --glob '*.json'
```

Review every remaining search result rather than requiring the search to be
empty: `source_digest` and similar terms are valid in unrelated contracts and
content-free debug state.

### Prompt Scope

This stage is appropriately sized for one gpt-5.6-terra prompt. It owns output
and assembled-workflow adaptation only; do not revise public documentation in
this stage.

## Stage 3: Rewrite The Canonical Documentation

### Goal

Make all current documentation describe the implemented minimal evidence
excerpt, with the durable contract defined in one canonical location.

### Required Changes

1. Rewrite `docs/integrations/evidence-context.md` as the canonical durable
   contract:
   - retain identity, discovery, filename, and optionality;
   - define the top-level source-unit array and empty-array behavior;
   - define typed-evidence selection, window expansion, union, deduplication,
     source-position ordering, and unchanged-unit semantics;
   - state that the excerpt can include at most every generic source unit once
     but has no byte/token-size guarantee;
   - state that lane artifacts remain authoritative for citations and the
     excerpt contains no lane attribution;
   - preserve sensitivity and consumer validation guidance.
2. Update `docs/config.md` only where it describes the result of
   `evidence_context`. Preserve the existing keys, validation rules, and default
   window. Link to the integration contract for payload details.
3. Update `docs/integrations/json-output.md` to describe the logical file as the
   optional selected source-unit excerpt without duplicating its schema.
4. Update `docs/consumers/subprocess.md` so consumers decode the array and use
   lane artifacts—not the excerpt—for authoritative citations and lane
   provenance.
5. Update `docs/operations.md`, `docs/internal/modules.md`, and any other current
   documentation that describes context groups, provenance records, or the old
   wrapper. Keep operational sensitivity guidance and internal ownership in
   their canonical documents.
6. Review `examples/dnd-complete.config.yml`. Retain the current
   `evidence_context` configuration and lane list unless repository inspection
   reveals an independently stale module key. Do not add a new example solely
   for this unchanged configuration surface.
7. Do not modify ADRs or add a new ADR. Do not rewrite archived audit history
   to match current behavior.

### Acceptance Criteria

- `docs/integrations/evidence-context.md` is the sole canonical definition of
  the payload.
- Other current documents accurately summarize their audience-specific aspect
  and link to the contract instead of duplicating it.
- No current documentation claims that `evidence-context.json` contains
  context groups, evidence-reference records, lane attribution, source-level
  wrapper metadata, or a guaranteed byte/token reduction.
- The maintained complete example remains valid and reflects implemented
  configuration.

### Validation

```sh
rg -n 'context_ref|evidence_refs|selected_lanes|context_count' README.md docs examples --glob '!docs/roadmap/**' --glob '!docs/adr/**'
go test ./internal/config ./internal/modules/generic/output/json ./internal/cli
```

Inspect any search matches for legitimate historical or unrelated use. If the
repository provides no automated Markdown link checker, manually verify every
link changed in this stage.

### Prompt Scope

This documentation migration is small enough for one gpt-5.6-terra prompt.

## Stage 4: Final Verification And Cleanup

### Goal

Verify the complete feature across package, integration, documentation, and
repository boundaries, then remove only obsolete implementation debris found
by that verification.

### Required Changes

1. Review the diff against `docs/roadmap/evidence-context.md` and confirm every
   acceptance criterion is implemented.
2. Search the non-archived repository for deleted Go types, old JSON fields,
   obsolete fixture shapes, and statements that describe the former contract.
3. Confirm the schema, codec, builder, pipeline request, debug summary, output
   encoder, index descriptor, example configuration, and integration contract
   agree on the final shape and identities.
4. Run formatting and the complete offline validation suite.
5. Make only narrowly scoped corrections required by failed checks or direct
   roadmap divergence. Do not introduce unrelated refactors or new features.
6. Leave the active roadmap documents in place for post-implementation review;
   retirement occurs only when separately requested after the implementation
   is accepted.

### Acceptance Criteria

- All roadmap acceptance criteria are satisfied with no legacy compatibility
  path.
- No current code, schema, fixture, example, or documentation depends on the
  former wrapper/context/provenance payload.
- The structural bound is protected: the artifact contains only distinct,
  unchanged units from the source document in source order.
- Repository formatting, tests, vet, and build all pass.
- The worktree contains only intentional feature changes.

### Validation

```sh
gofmt -w internal/framework/evidencecontext internal/framework/pipeline internal/modules/generic/output/json internal/modules/integration internal/cli
go test ./...
go vet ./...
go build ./cmd/notarius
git diff --check
git status --short
```

### Prompt Scope

This verification pass is small enough for one gpt-5.6-terra prompt. If it
uncovers a material unrelated defect, report it separately instead of widening
this feature.