eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Tighten evidence context source excerpts

Browse Source
This commit is contained in:
Eric Rakestraw
2026-08-09 20:04:46 +00:00
parent f6d037b613
commit bef8ca263b
6 changed files with 15 additions and 491 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/integrations/evidence-context.md
View File

@@ -63,7 +63,8 @@ Each source unit has required `id`, `kind`, `text`, and self `ref` fields.
endpoints identify that unit's `id`. A unit may also contain source-owned
`metadata`, an open-ended JSON object. Fixed unit and reference fields are
strict: consumers must reject unknown fixed fields, malformed units, invalid
self-references, and a payload that is not the array described here.
self-references, units whose `source_id` differs from other units in the same
excerpt, and a payload that is not the array described here.
The excerpt preserves each selected unit exactly as represented by the
validated generic source document. It does not add evidence-context-specific

174
docs/roadmap/evidence-context.md
View File

@@ -1,174 +0,0 @@
# Minimal Evidence Context
## Purpose
Redefine the optional `evidence-context.json` output as a compact reading
excerpt rather than a second provenance model. The artifact should contain the
smallest useful source projection: the ordered union of source units cited by
selected accepted lane artifacts, expanded by the configured surrounding
window.
The normalized lane artifacts remain authoritative for which lane cited which
source ranges. The original source remains authoritative for the complete
input. Evidence context is an optional convenience artifact for consumers that
need the relevant source material alongside the extracted results.
## Motivation
The current artifact repeats source units inside context objects and adds
context ranges, lane attribution, direct evidence references, source identity,
source digest, selected lanes, and window configuration. With several lanes
and a nonzero window, this representation can be substantially larger than the
complete input by both bytes and tokens. That defeats its intended value as a
convenient evidence excerpt.
The application already publishes authoritative source references in each
normalized artifact. Repeating their provenance in `evidence-context.json`
does not justify the additional size or contract complexity.
## Target Contract
`evidence-context.json` remains an optional pipeline-wide JSON artifact with
the existing logical filename, artifact kind, media type, schema identity, and
schema version. Because Notarius is pre-release, the v1 schema may be replaced
in place and no reader compatibility or migration path is required.
The payload is a top-level JSON array of generic source units. It has no
evidence-context-specific wrapper or metadata. Every element uses the existing
`source.SourceUnit` representation and therefore preserves its required
`id`, `kind`, `text`, and self-reference together with any metadata already
owned by the source unit. “No additional metadata” means that evidence-context
construction does not annotate, reshape, or enrich a source unit; it does not
mean stripping metadata supplied by the input adapter.
An enabled output with no accepted cited evidence emits `[]`, not `null` and
not an absent artifact. The artifact remains absent when evidence-context
publication is not enabled.
## Selection Semantics
Construction must:
1. obtain source references only through the typed evidence projections of
accepted normalized outputs from the configured lane allowlist;
2. validate every projected reference against the current source document;
3. expand each valid referenced range by `window_units` positions on both
sides, clamping at the source boundaries;
4. take the union of all expanded ranges;
5. emit every selected source unit exactly once and in source-document
position order; and
6. return owned copies so later mutation cannot alias the source document or
output artifact.
Overlapping and adjacent ranges may be coalesced internally, but range groups
are not represented in the payload. Repeated citations, citations from
multiple lanes, and overlapping windows never duplicate a source unit.
Rejected, failed, absent, inactive, and unselected lanes contribute nothing.
The selected set is therefore bounded by the source document: it can contain
at most every source unit once. With broad evidence coverage or a sufficiently
large window, it may legitimately equal the complete generic source document.
No byte- or token-size guarantee is made because generic source-unit
serialization can differ from the external input format and output formatting
has its own overhead. The application must not truncate a complete excerpt to
meet an arbitrary size limit.
## Configuration And Publication
Retain the current `output.options.evidence_context` configuration:
- `enabled` continues to control publication;
- `lanes` remains the non-empty allowlist of configured artifact lanes whose
accepted normalized evidence contributes to the excerpt; and
- `window_units` remains a non-negative optional value with the existing
default of three.
Retain `evidence-context.json` and the pipeline-wide `index.json`
`evidence_context` descriptor. The descriptor remains the canonical place for
artifact identity and discovery. It must not be copied into the payload.
## Internal Design
Keep evidence preparation at the current framework and output boundary. The
generic framework should continue to consume typed evidence projections rather
than inspect domain JSON or import domain artifact types.
Simplify `internal/framework/evidencecontext` around the new contract:
- represent the durable document as a collection of source units rather than
contexts and evidence contributions;
- reduce the build request to the source document, window size, and projected
source references needed to select units;
- remove durable and internal types used only for lane attribution, context
ranges, and evidence-reference publication;
- retain strict schema validation, deterministic serialization, source and
reference validation, ownership, and stable artifact identity; and
- preserve the existing pipeline preparation logic that validates configured
lanes and typed evidence projections.
The pipeline may continue to record a content-free debug summary containing
artifact identity, selected lane configuration, window size, source digest,
and emitted unit count. Remove the obsolete context count. Debug information
is operational state outside `evidence-context.json` and must not contain the
selected source text or source-unit metadata.
## Documentation Impact
When the implementation lands, rewrite the durable contract in
`docs/integrations/evidence-context.md`. Update the configuration, JSON-output,
subprocess-consumer, operations, and internal-module documentation wherever it
describes the old context or provenance structure. Preserve canonical
ownership: the integration contract defines the payload; other documents give
only the information appropriate to their audience and link to that contract.
The maintained complete D&D configuration should retain its existing evidence
publication settings. No new example is needed because the configuration
surface is unchanged.
## Testing Expectations
Tests should protect the revised behavior rather than the former internal
shape. At the narrowest stable boundaries, verify:
- window expansion and source-boundary clamping;
- union, deduplication, and source-position ordering, including non-monotonic
unit IDs;
- unchanged source-unit values and owned metadata;
- `[]` for enabled publication with no accepted cited evidence;
- rejection of invalid sources, references, and durable payloads;
- omission when publication is disabled;
- contribution only from selected accepted normalized outputs, including
checkpoint reconstruction; and
- correct JSON bundle publication and index discovery.
Do not add an exact byte-length, token-count, or ratio test. Such a test would
be format-sensitive and would not protect the durable contract. The meaningful
boundedness invariant is structural: every emitted element corresponds to one
distinct unit from the source document, and no source unit is emitted more
than once.
## Documentation Decision
No new ADR is required. This work simplifies a pre-release output contract but
does not change the architectural ownership of source evidence, typed
projections, pipeline stages, or output publication. The durable contract and
current implementation documentation should be updated when the behavior is
implemented; this roadmap owns the proposed behavior until then.
## Acceptance Criteria
- `evidence-context.json` is a top-level array containing only selected generic
source units.
- The payload contains no context groups, lane IDs, evidence references,
source-level wrapper fields, or evidence-context-specific annotations.
- Units are copied unchanged from the source document, appear once, and retain
source-document order.
- Existing lane selection and `window_units` behavior remain configurable and
retain their current validation and defaults.
- The artifact remains optional, keeps its filename and identity, and remains
discoverable through `index.json`.
- The implementation retains strict validation, deterministic output,
ownership, failure propagation, checkpoint behavior, and content-free debug
reporting.
- Current documentation and maintained examples accurately describe the new
contract, and the repository-wide tests, vet, and build pass.

315
docs/roadmap/implementation.md
View File

@@ -1,315 +0,0 @@
# Minimal Evidence Context Implementation Plan
## Objective
Implement the target state defined in
[Minimal Evidence Context](evidence-context.md): retain the optional
`evidence-context.json` output and its existing configuration while replacing
its provenance-heavy payload with the ordered, deduplicated union of selected
generic source units.
This plan intentionally changes the existing v1 contract in place. Notarius is
pre-release, so do not add legacy decoding, schema-version branching,
migration, dual publication, or other backward-compatibility machinery.
## Global Implementation Rules
Apply these rules in every stage:
- Follow `docs/policy/architecture.md`, `docs/policy/documentation.md`, and
`docs/policy/testing.md`.
- Treat the normalized lane artifacts as authoritative for citations and the
source document as authoritative for source-unit content.
- Preserve the typed evidence-projection boundary. Generic code must not
inspect domain JSON or depend on D&D artifact types.
- Preserve the logical filename `evidence-context.json`, artifact kind
`source/evidence-context`, media type `application/json`, schema ID
`notarius.source.evidence_context`, schema name
`notarius_source_evidence_context_v1`, and schema version `v1`.
- Preserve the current configuration keys, validation, lane allowlist
semantics, and default `window_units` value.
- Do not introduce a payload wrapper. The durable JSON value is a top-level
array of generic source units; the empty value is `[]`.
- Preserve source-unit fields and metadata exactly as represented in the
validated generic source document. Add no evidence-context-specific fields.
- Do not add byte-count, token-count, or compression-ratio tests. Test the
structural boundedness and behavioral contract instead.
- Use focused tests while iterating and leave the repository compiling and its
relevant test suites passing at the end of every stage.
## Stage 1: Replace The Core Evidence-Context Contract
### Goal
Replace the context/provenance model with a strict top-level source-unit array
and simplify construction to select the union of expanded source ranges.
### Required Changes
1. Update `internal/framework/evidencecontext/model.go`:
- represent `Document` as the durable collection of `source.SourceUnit`
values that serializes directly as a JSON array;
- retain the existing artifact identity constants;
- reduce `BuildRequest` to `Source`, `WindowUnits`, and a flat collection of
projected `SourceRefs`;
- remove `Context`, `EvidenceRef`, `LaneEvidence`, and any other types whose
only purpose was durable lane attribution or context grouping.
2. Update `internal/framework/evidencecontext/build.go`:
- validate the window and complete source document, including its digest;
- validate every projected source reference against the source index;
- convert referenced endpoints to source positions, expand and clamp them,
and merge their union;
- append owned copies of the selected source units once in document-position
order;
- return an initialized empty document when there are no references;
- remove sorting keys and helpers that exist only to order lane attribution
or evidence records.
3. Update `internal/framework/evidencecontext/codec.go` and
`internal/framework/evidencecontext/assets/schemas/source_evidence_context.v1.json`:
- define the root schema as an array of strict source-unit objects;
- preserve the existing source-unit contract: required `id`, `kind`, `text`,
and `ref`, with optional open-ended source-owned `metadata`;
- ensure decode rejects `null`, wrapper objects, malformed units, unknown
fixed unit/reference fields, invalid self-references, and noncanonical
documents;
- ensure encode and decode own all slice and nested metadata values and
serialize an empty document as `[]`;
- remove canonicalization logic used only by the deleted wrapper and context
structures.
4. Replace the fixture in
`internal/framework/evidencecontext/testdata/source_evidence_context.v1.json`
with a compact top-level unit array.
5. Rewrite `internal/framework/evidencecontext/evidencecontext_test.go` around
durable behavior:
- cover expanded overlapping and adjacent ranges, boundary clamping,
repeated references, and non-monotonic unit IDs;
- prove that the output is source-ordered, contains each selected source
unit once, and never invents a unit;
- prove preservation and ownership of source-unit metadata;
- cover empty-array encoding and strict invalid-input rejection;
- retain artifact identity and deterministic round-trip coverage without
testing deleted internal structures.
6. Adapt `internal/framework/pipeline/evidence_output.go` so it flattens the
typed references from prepared selected lanes into the new build request.
Preserve all existing checks for duplicate accepted lane outputs, source and
artifact-kind compatibility, codec decoding, and evidence projection
failures.
7. Adapt the pipeline debug summary and its use in `runner.go` as necessary:
- calculate selected lanes, window size, and source digest from the prepared
plan and source document rather than the payload;
- report the emitted unit count directly;
- remove `context_count`;
- keep source text and unit metadata out of debug summaries.
8. Rewrite the focused tests in
`internal/framework/pipeline/evidence_output_test.go` to assert the new
payload while preserving coverage of selected accepted outputs, absent and
rejected lanes, incompatible artifacts, failure propagation, ownership,
checkpoint reconstruction, opt-in behavior, and redacted debug reporting.
### Acceptance Criteria
- The evidence-context package has no durable or internal model for context
groups, lane attribution, or published evidence references.
- Its codec accepts and emits only the new top-level array contract.
- Its builder returns the exact ordered union of source units selected by the
expanded validated ranges.
- Pipeline construction still obtains references exclusively through prepared
typed evidence projections.
- The payload remains absent when publication is not configured and is `[]`
when enabled with no contributing evidence.
- Focused framework tests pass.
### Validation
```sh
gofmt -w internal/framework/evidencecontext internal/framework/pipeline/evidence_output.go internal/framework/pipeline/evidence_output_test.go
go test ./internal/framework/evidencecontext ./internal/framework/pipeline
```
### Prompt Scope
This is one coherent but substantial gpt-5.6-terra prompt. Keep the work
limited to the core contract, its pipeline construction boundary, and their
focused tests; do not update generic JSON output tests, production integration
tests, or documentation in this stage.
## Stage 2: Update Output And Production Integration Coverage
### Goal
Carry the revised artifact through the generic JSON output and representative
assembled D&D workflows without changing publication or discovery semantics.
### Required Changes
1. Review `internal/modules/generic/output/json/encoder.go` and retain the
existing option decoding, default window, lane validation, logical filename,
artifact identity checks, pretty JSON publication, and `index.json`
descriptor. Change production code only where it assumes the old payload.
2. Rewrite evidence-context fixtures and assertions in
`internal/modules/generic/output/json/encoder_test.go`:
- validate and publish a top-level unit array;
- retain descriptor, omission, strict artifact validation, and
content-nondisclosure failure tests;
- remove expectations for wrapper fields or contexts.
3. Update evidence assertions in the representative integration and CLI
contract tests, including:
- `internal/modules/integration/dnd_npc_grounded_test.go`;
- `internal/modules/integration/dnd_location_registry_runner_test.go`;
- `internal/cli/dnd_enemy_events_contract_test.go`; and
- any additional test located by searching for `evidence-context`,
`EvidenceContext`, `Contexts`, `context_count`, or old fixture fields.
4. Ensure the integration assertions protect the behavior that matters:
- only selected accepted lanes contribute references;
- multiple lanes and overlapping windows produce one ordered unit union;
- non-monotonic unit IDs do not change source-position order;
- scene-description selection remains explicit;
- the output descriptor remains pipeline-wide rather than a lane artifact.
5. Remove obsolete helpers and fixtures that only inspect old context groups or
per-lane evidence records. Do not preserve compatibility helpers.
### Acceptance Criteria
- The generic JSON encoder publishes the new payload under the existing
filename and descriptor.
- Maintained integration and CLI workflows observe a deduplicated array of
unchanged source units.
- No production code or test outside the archived roadmap refers to deleted
payload fields or types.
- Focused output, integration, and CLI tests pass.
### Validation
```sh
gofmt -w internal/modules/generic/output/json internal/modules/integration internal/cli
go test ./internal/modules/generic/output/json ./internal/modules/integration ./internal/cli
rg -n 'context_ref|evidence_refs|selected_lanes|source_digest|context_count|\.Contexts\b' internal --glob '*.go' --glob '*.json'
```
Review every remaining search result rather than requiring the search to be
empty: `source_digest` and similar terms are valid in unrelated contracts and
content-free debug state.
### Prompt Scope
This stage is appropriately sized for one gpt-5.6-terra prompt. It owns output
and assembled-workflow adaptation only; do not revise public documentation in
this stage.
## Stage 3: Rewrite The Canonical Documentation
### Goal
Make all current documentation describe the implemented minimal evidence
excerpt, with the durable contract defined in one canonical location.
### Required Changes
1. Rewrite `docs/integrations/evidence-context.md` as the canonical durable
contract:
- retain identity, discovery, filename, and optionality;
- define the top-level source-unit array and empty-array behavior;
- define typed-evidence selection, window expansion, union, deduplication,
source-position ordering, and unchanged-unit semantics;
- state that the excerpt can include at most every generic source unit once
but has no byte/token-size guarantee;
- state that lane artifacts remain authoritative for citations and the
excerpt contains no lane attribution;
- preserve sensitivity and consumer validation guidance.
2. Update `docs/config.md` only where it describes the result of
`evidence_context`. Preserve the existing keys, validation rules, and default
window. Link to the integration contract for payload details.
3. Update `docs/integrations/json-output.md` to describe the logical file as the
optional selected source-unit excerpt without duplicating its schema.
4. Update `docs/consumers/subprocess.md` so consumers decode the array and use
lane artifacts—not the excerpt—for authoritative citations and lane
provenance.
5. Update `docs/operations.md`, `docs/internal/modules.md`, and any other current
documentation that describes context groups, provenance records, or the old
wrapper. Keep operational sensitivity guidance and internal ownership in
their canonical documents.
6. Review `examples/dnd-complete.config.yml`. Retain the current
`evidence_context` configuration and lane list unless repository inspection
reveals an independently stale module key. Do not add a new example solely
for this unchanged configuration surface.
7. Do not modify ADRs or add a new ADR. Do not rewrite archived audit history
to match current behavior.
### Acceptance Criteria
- `docs/integrations/evidence-context.md` is the sole canonical definition of
the payload.
- Other current documents accurately summarize their audience-specific aspect
and link to the contract instead of duplicating it.
- No current documentation claims that `evidence-context.json` contains
context groups, evidence-reference records, lane attribution, source-level
wrapper metadata, or a guaranteed byte/token reduction.
- The maintained complete example remains valid and reflects implemented
configuration.
### Validation
```sh
rg -n 'context_ref|evidence_refs|selected_lanes|context_count' README.md docs examples --glob '!docs/roadmap/**' --glob '!docs/adr/**'
go test ./internal/config ./internal/modules/generic/output/json ./internal/cli
```
Inspect any search matches for legitimate historical or unrelated use. If the
repository provides no automated Markdown link checker, manually verify every
link changed in this stage.
### Prompt Scope
This documentation migration is small enough for one gpt-5.6-terra prompt.
## Stage 4: Final Verification And Cleanup
### Goal
Verify the complete feature across package, integration, documentation, and
repository boundaries, then remove only obsolete implementation debris found
by that verification.
### Required Changes
1. Review the diff against `docs/roadmap/evidence-context.md` and confirm every
acceptance criterion is implemented.
2. Search the non-archived repository for deleted Go types, old JSON fields,
obsolete fixture shapes, and statements that describe the former contract.
3. Confirm the schema, codec, builder, pipeline request, debug summary, output
encoder, index descriptor, example configuration, and integration contract
agree on the final shape and identities.
4. Run formatting and the complete offline validation suite.
5. Make only narrowly scoped corrections required by failed checks or direct
roadmap divergence. Do not introduce unrelated refactors or new features.
6. Leave the active roadmap documents in place for post-implementation review;
retirement occurs only when separately requested after the implementation
is accepted.
### Acceptance Criteria
- All roadmap acceptance criteria are satisfied with no legacy compatibility
path.
- No current code, schema, fixture, example, or documentation depends on the
former wrapper/context/provenance payload.
- The structural bound is protected: the artifact contains only distinct,
unchanged units from the source document in source order.
- Repository formatting, tests, vet, and build all pass.
- The worktree contains only intentional feature changes.
### Validation
```sh
gofmt -w internal/framework/evidencecontext internal/framework/pipeline internal/modules/generic/output/json internal/modules/integration internal/cli
go test ./...
go vet ./...
go build ./cmd/notarius
git diff --check
git status --short
```
### Prompt Scope
This verification pass is small enough for one gpt-5.6-terra prompt. If it
uncovers a material unrelated defect, report it separately instead of widening
this feature.

6
internal/framework/evidencecontext/build.go
View File

@@ -44,7 +44,11 @@ func Build(request BuildRequest) (Document, error) {
}
merged := mergeRanges(ranges)
document := make(Document, 0)
unitCount := 0
for _, value := range merged {
unitCount += value.endPos - value.startPos + 1
}
document := make(Document, 0, unitCount)
for _, value := range merged {
for position := value.startPos; position <= value.endPos; position++ {
unit, err := cloneSourceUnit(request.Source.Units[position])

6
internal/framework/evidencecontext/codec.go
View File

@@ -167,6 +167,7 @@ func canonicalizeOwned(value Document) (Document, error) {
return nil, fmt.Errorf("document must be a JSON array")
}
seenUnitIDs := make(map[int]struct{}, len(value))
sourceID := ""
for unitIndex := range value {
unit := value[unitIndex]
if unit.ID <= 0 || strings.TrimSpace(unit.Kind) == "" || strings.TrimSpace(unit.Text) == "" {
@@ -175,6 +176,11 @@ func canonicalizeOwned(value Document) (Document, error) {
if err := validateUnitRef(unit, unitIndex); err != nil {
return nil, err
}
if sourceID == "" {
sourceID = unit.Ref.SourceID
} else if unit.Ref.SourceID != sourceID {
return nil, fmt.Errorf("units[%d].ref.source_id must match units[0].ref.source_id", unitIndex)
}
if _, exists := seenUnitIDs[unit.ID]; exists {
return nil, fmt.Errorf("units contains duplicate unit id %d", unit.ID)
}

2
internal/framework/evidencecontext/evidencecontext_test.go
View File

@@ -19,6 +19,7 @@ func TestBuildSelectsExpandedSourceUnitUnion(t *testing.T) {
wantIDs []int
}{
{name: "zero window", refs: []source.SourceRef{ref(3, 3)}, wantIDs: []int{3}},
{name: "multi unit citation includes complete range", refs: []source.SourceRef{ref(3, 7)}, wantIDs: []int{3, 30, 7}},
{name: "non monotonic IDs use document positions", window: 1, refs: []source.SourceRef{ref(3, 3)}, wantIDs: []int{10, 3, 30}},
{name: "boundary clamping", window: 1, refs: []source.SourceRef{ref(10, 10), ref(50, 50)}, wantIDs: []int{10, 3, 7, 50}},
{name: "overlapping and adjacent windows merge", window: 1, refs: []source.SourceRef{ref(3, 3), ref(30, 30), ref(30, 30)}, wantIDs: []int{10, 3, 30, 7}},
@@ -143,6 +144,7 @@ func TestCodecRejectsInvalidDurablePayloads(t *testing.T) {
{name: "unknown unit field", content: `[{"id":1,"kind":"segment","text":"text","ref":{"source_id":"session","start_unit_id":1,"end_unit_id":1},"unknown":true}]`},
{name: "unknown reference field", content: `[{"id":1,"kind":"segment","text":"text","ref":{"source_id":"session","start_unit_id":1,"end_unit_id":1,"unknown":true}}]`},
{name: "invalid self reference", content: `[{"id":1,"kind":"segment","text":"text","ref":{"source_id":"session","start_unit_id":1,"end_unit_id":2}}]`},
{name: "mixed source documents", content: `[{"id":1,"kind":"segment","text":"one","ref":{"source_id":"session-one","start_unit_id":1,"end_unit_id":1}},{"id":2,"kind":"segment","text":"two","ref":{"source_id":"session-two","start_unit_id":2,"end_unit_id":2}}]`},
{name: "duplicate units", content: `[{"id":1,"kind":"segment","text":"one","ref":{"source_id":"session","start_unit_id":1,"end_unit_id":1}},{"id":1,"kind":"segment","text":"two","ref":{"source_id":"session","start_unit_id":1,"end_unit_id":1}}]`},
{name: "multiple JSON values", content: `[] []`},
} {