Minimal Evidence Context

Purpose

Redefine the optional evidence-context.json output as a compact reading excerpt rather than a second provenance model. The artifact should contain the smallest useful source projection: the ordered union of source units cited by selected accepted lane artifacts, expanded by the configured surrounding window.

The normalized lane artifacts remain authoritative for which lane cited which source ranges. The original source remains authoritative for the complete input. Evidence context is an optional convenience artifact for consumers that need the relevant source material alongside the extracted results.

Motivation

The current artifact repeats source units inside context objects and adds context ranges, lane attribution, direct evidence references, source identity, source digest, selected lanes, and window configuration. With several lanes and a nonzero window, this representation can be substantially larger than the complete input by both bytes and tokens. That defeats its intended value as a convenient evidence excerpt.

The application already publishes authoritative source references in each normalized artifact. Repeating their provenance in evidence-context.json does not justify the additional size or contract complexity.

Target Contract

evidence-context.json remains an optional pipeline-wide JSON artifact with the existing logical filename, artifact kind, media type, schema identity, and schema version. Because Notarius is pre-release, the v1 schema may be replaced in place and no reader compatibility or migration path is required.

The payload is a top-level JSON array of generic source units. It has no evidence-context-specific wrapper or metadata. Every element uses the existing source.SourceUnit representation and therefore preserves its required id, kind, text, and self-reference together with any metadata already owned by the source unit. “No additional metadata” means that evidence-context construction does not annotate, reshape, or enrich a source unit; it does not mean stripping metadata supplied by the input adapter.

An enabled output with no accepted cited evidence emits [], not null and not an absent artifact. The artifact remains absent when evidence-context publication is not enabled.

Selection Semantics

Construction must:

obtain source references only through the typed evidence projections of accepted normalized outputs from the configured lane allowlist;
validate every projected reference against the current source document;
expand each valid referenced range by window_units positions on both sides, clamping at the source boundaries;
take the union of all expanded ranges;
emit every selected source unit exactly once and in source-document position order; and
return owned copies so later mutation cannot alias the source document or output artifact.

Overlapping and adjacent ranges may be coalesced internally, but range groups are not represented in the payload. Repeated citations, citations from multiple lanes, and overlapping windows never duplicate a source unit. Rejected, failed, absent, inactive, and unselected lanes contribute nothing.

The selected set is therefore bounded by the source document: it can contain at most every source unit once. With broad evidence coverage or a sufficiently large window, it may legitimately equal the complete generic source document. No byte- or token-size guarantee is made because generic source-unit serialization can differ from the external input format and output formatting has its own overhead. The application must not truncate a complete excerpt to meet an arbitrary size limit.

Configuration And Publication

Retain the current output.options.evidence_context configuration:

enabled continues to control publication;
lanes remains the non-empty allowlist of configured artifact lanes whose accepted normalized evidence contributes to the excerpt; and
window_units remains a non-negative optional value with the existing default of three.

Retain evidence-context.json and the pipeline-wide index.json evidence_context descriptor. The descriptor remains the canonical place for artifact identity and discovery. It must not be copied into the payload.

Internal Design

Keep evidence preparation at the current framework and output boundary. The generic framework should continue to consume typed evidence projections rather than inspect domain JSON or import domain artifact types.

Simplify internal/framework/evidencecontext around the new contract:

represent the durable document as a collection of source units rather than contexts and evidence contributions;
reduce the build request to the source document, window size, and projected source references needed to select units;
remove durable and internal types used only for lane attribution, context ranges, and evidence-reference publication;
retain strict schema validation, deterministic serialization, source and reference validation, ownership, and stable artifact identity; and
preserve the existing pipeline preparation logic that validates configured lanes and typed evidence projections.

The pipeline may continue to record a content-free debug summary containing artifact identity, selected lane configuration, window size, source digest, and emitted unit count. Remove the obsolete context count. Debug information is operational state outside evidence-context.json and must not contain the selected source text or source-unit metadata.

Documentation Impact

When the implementation lands, rewrite the durable contract in docs/integrations/evidence-context.md. Update the configuration, JSON-output, subprocess-consumer, operations, and internal-module documentation wherever it describes the old context or provenance structure. Preserve canonical ownership: the integration contract defines the payload; other documents give only the information appropriate to their audience and link to that contract.

The maintained complete D&D configuration should retain its existing evidence publication settings. No new example is needed because the configuration surface is unchanged.

Testing Expectations

Tests should protect the revised behavior rather than the former internal shape. At the narrowest stable boundaries, verify:

window expansion and source-boundary clamping;
union, deduplication, and source-position ordering, including non-monotonic unit IDs;
unchanged source-unit values and owned metadata;
[] for enabled publication with no accepted cited evidence;
rejection of invalid sources, references, and durable payloads;
omission when publication is disabled;
contribution only from selected accepted normalized outputs, including checkpoint reconstruction; and
correct JSON bundle publication and index discovery.

Do not add an exact byte-length, token-count, or ratio test. Such a test would be format-sensitive and would not protect the durable contract. The meaningful boundedness invariant is structural: every emitted element corresponds to one distinct unit from the source document, and no source unit is emitted more than once.

Documentation Decision

No new ADR is required. This work simplifies a pre-release output contract but does not change the architectural ownership of source evidence, typed projections, pipeline stages, or output publication. The durable contract and current implementation documentation should be updated when the behavior is implemented; this roadmap owns the proposed behavior until then.

Acceptance Criteria

evidence-context.json is a top-level array containing only selected generic source units.
The payload contains no context groups, lane IDs, evidence references, source-level wrapper fields, or evidence-context-specific annotations.
Units are copied unchanged from the source document, appear once, and retain source-document order.
Existing lane selection and window_units behavior remain configurable and retain their current validation and defaults.
The artifact remains optional, keeps its filename and identity, and remains discoverable through index.json.
The implementation retains strict validation, deterministic output, ownership, failure propagation, checkpoint behavior, and content-free debug reporting.
Current documentation and maintained examples accurately describe the new contract, and the repository-wide tests, vet, and build pass.

8.4 KiB Raw Blame History