diff --git a/docs/config.md b/docs/config.md index fb2789c..7eb49b3 100644 --- a/docs/config.md +++ b/docs/config.md @@ -319,7 +319,8 @@ Unknown outer or nested option fields are rejected, as are incompatible YAML types. The allowlist remains valid when a run uses lane filtering: a configured lane that is not active for that invocation simply contributes no evidence. Evidence publication is opt-in because it can persist source text and metadata. -Its payload contract is [Published Evidence Context](integrations/evidence-context.md). +When enabled, it publishes the selected source-unit excerpt defined by the +[Published Evidence Context contract](integrations/evidence-context.md). ## References And Ordered Handoffs diff --git a/docs/consumers/subprocess.md b/docs/consumers/subprocess.md index fe67c5a..a53e523 100644 --- a/docs/consumers/subprocess.md +++ b/docs/consumers/subprocess.md @@ -55,10 +55,10 @@ contract. The JSON bundle contract links to the available lane contracts. If `index.json` has an `evidence_context` descriptor, treat it as a pipeline-wide artifact rather than a lane entry. Verify its six descriptor fields before decoding the linked file according to the [Published Evidence -Context contract](../integrations/evidence-context.md). Use each -`evidence_refs` entry as the citation to source material. Its surrounding -context range and included units explain the citation, but do not widen or -replace the cited source reference. +Context contract](../integrations/evidence-context.md). Decode its top-level +source-unit array as a reading excerpt. Obtain authoritative citations and lane +provenance from the normalized lane artifacts; the excerpt has neither and its +nearby units do not widen a lane artifact's cited source reference. A zero exit status may still report rejected outputs, warnings, or absent lanes. The caller decides which lane IDs are required for its own work and @@ -73,5 +73,5 @@ them. Treat the input, output bundle, cache, debug bundle, and captured process logs as potentially sensitive data. Apply the caller's access controls and retention policy, and avoid copying secrets into arguments, logs, or provenance records. An evidence-context artifact contains source-unit text and -metadata, and selected lanes can cover most of an input; preserve and share it -only when that source content is authorized for the recipient. +metadata and can cover most of an input; preserve and share it only when that +source content is authorized for the recipient. diff --git a/docs/integrations/evidence-context.md b/docs/integrations/evidence-context.md index 01ac4cf..886e9a1 100644 --- a/docs/integrations/evidence-context.md +++ b/docs/integrations/evidence-context.md @@ -1,9 +1,11 @@ # Published Evidence Context This contract defines the optional `source/evidence-context` artifact emitted -by the production JSON output. Its configuration is owned by -[Configuration](../config.md#module-bindings-and-validators); its logical-file -discovery is owned by [Published JSON Output](json-output.md). +by the production JSON output. It is a selected source-unit excerpt for +convenient reading alongside normalized lane artifacts; it is not a second +citation or provenance model. Its configuration is owned by +[Configuration](../config.md#module-bindings-and-validators), and its +logical-file discovery is owned by [Published JSON Output](json-output.md). ## Identity And Discovery @@ -26,91 +28,79 @@ its absence means evidence publication was not enabled for that bundle. ## Payload -The v1 payload is a JSON object with required `source_id`, `source_digest`, -`window_units`, `selected_lanes`, and `contexts` fields. `selected_lanes` and -`contexts` are always arrays; an enabled configuration with no accepted direct -evidence publishes `contexts: []`. +The v1 payload is a top-level JSON array of generic source units. There is no +wrapper, source-level metadata, context grouping, lane identifier, or evidence +reference in the payload. An enabled configuration with no contributing +accepted evidence publishes `[]`. ```json -{ - "source_id": "session-alpha", - "source_digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", - "window_units": 1, - "selected_lanes": ["npc_registry", "spells"], - "contexts": [ - { - "context_ref": { - "source_id": "session-alpha", - "start_unit_id": 10, - "end_unit_id": 20 - }, - "evidence_refs": [ - { - "lane_id": "spells", - "source_ref": { - "source_id": "session-alpha", - "start_unit_id": 10, - "end_unit_id": 10 - } - } - ], - "units": [ - { - "id": 10, - "kind": "transcript_segment", - "text": "Aria casts Cure Wounds.", - "ref": { - "source_id": "session-alpha", - "start_unit_id": 10, - "end_unit_id": 10 - } - }, - { - "id": 20, - "kind": "transcript_segment", - "text": "The party regroups.", - "ref": { - "source_id": "session-alpha", - "start_unit_id": 20, - "end_unit_id": 20 - } - } - ] +[ + { + "id": 10, + "kind": "transcript_segment", + "text": "Aria casts Cure Wounds.", + "ref": { + "source_id": "session-alpha", + "start_unit_id": 10, + "end_unit_id": 10 } - ] -} + }, + { + "id": 20, + "kind": "transcript_segment", + "text": "The party regroups.", + "ref": { + "source_id": "session-alpha", + "start_unit_id": 20, + "end_unit_id": 20 + } + } +] ``` -Each context requires a `context_ref` object and `evidence_refs` and `units` -arrays. `context_ref` identifies the first and last included unit. Each -evidence entry contains a selected `lane_id` and an original `source_ref`. A -unit uses the existing source-unit shape: required `id`, `kind`, `text`, and -self `ref`, plus optional JSON-object `metadata`. Fixed payload objects reject -unknown fields; unit metadata may contain application-defined JSON values. +Each source unit has required `id`, `kind`, `text`, and self `ref` fields. +`ref` contains `source_id`, `start_unit_id`, and `end_unit_id`, and both unit +endpoints identify that unit's `id`. A unit may also contain source-owned +`metadata`, an open-ended JSON object. Fixed unit and reference fields are +strict: consumers must reject unknown fixed fields, malformed units, invalid +self-references, and a payload that is not the array described here. -## Citations And Context +The excerpt preserves each selected unit exactly as represented by the +validated generic source document. It does not add evidence-context-specific +annotations or reshape source-owned metadata. -`evidence_refs` are the authoritative citations. They identify the direct -references emitted by accepted normalized artifacts. `context_ref` and the -units collection include those cited units plus nearby source units selected by -the configured window. They are explanatory context, not widened citations. +## Selection And Citations -Only accepted outputs from the configured lane allowlist contribute. Rejected, -failed, absent, and lane-filtered outputs do not contribute. The artifact never -contains raw input bytes, prompts, model responses, auxiliary reference -content, credentials, or filesystem paths. +The framework obtains direct source references only through typed evidence +projections of accepted normalized artifacts in the configured lane allowlist. +It validates each reference against the current source document, expands its +range by `window_units` source-unit positions on each side, clamps at document +boundaries, and takes the union of all expanded ranges. The output contains +each selected source unit once in source-document position order, regardless +of numeric unit IDs. Repeated references, overlapping windows, and citations +from multiple lanes do not duplicate a unit. Rejected, failed, absent, +inactive, and unselected lanes contribute nothing. -## Ordering And Compatibility +Normalized lane artifacts remain authoritative for citations and for which lane +cited a range. The excerpt has no lane attribution and must not be used to +reconstruct it. Its included nearby units provide reading context only; they +do not widen any citation in a lane artifact. -The selected lane allowlist is lexical. Contexts and units are in source -document position order, not numeric unit-ID order. Direct evidence entries -are deterministically ordered by lane and source reference. Overlapping or -contiguous windows merge, and each source unit appears at most once in the -resulting contexts. +The excerpt contains at most every generic source unit once. It can therefore +equal the complete generic source document when coverage is broad or the +window is large. No byte-, token-, or compression-size guarantee is made, and +the framework does not truncate the excerpt to meet an arbitrary size limit. + +## Consumer Responsibilities And Data Handling The artifact is additive to the JSON bundle and is not a lane payload, normalized-output count, checkpoint, or generated reference. Consumers that -do not need it must tolerate the absent optional descriptor. Consumers that do -use it should preserve the artifact and its schema identity with the run -provenance, and should treat its source text and metadata as sensitive durable -content. +do not need it must tolerate an absent descriptor. Consumers that do use it +should validate the descriptor and payload before use, retain the artifact with +its schema identity when needed for a run record, and read citations from the +corresponding normalized lane artifacts. + +The excerpt contains source-unit text and source-owned metadata and is durable +output. Treat it as sensitive source content, apply appropriate access controls +and retention, and do not assume its selected form is materially smaller or +less sensitive than the original input. diff --git a/docs/integrations/json-output.md b/docs/integrations/json-output.md index 7c31931..8e2abfd 100644 --- a/docs/integrations/json-output.md +++ b/docs/integrations/json-output.md @@ -24,7 +24,7 @@ root for the logical discovery described here. | `warnings.json` | Accepted-output and run warnings. | | `lanes/.json` | One normalized artifact payload for each lane. | | `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. | -| `evidence-context.json` | Optional source-context artifact, when evidence publication is enabled. | +| `evidence-context.json` | Optional selected source-unit excerpt, when evidence publication is enabled. | JSON files are pretty-printed with a trailing newline. Lane payloads are accepted only when their media type is `application/json`. diff --git a/docs/internal/modules.md b/docs/internal/modules.md index a01b6ed..815bf99 100644 --- a/docs/internal/modules.md +++ b/docs/internal/modules.md @@ -42,7 +42,8 @@ generic source references and must use the codec's exact Go type. It does not interpret surrounding context or publish files; the pipeline validates the capability during preparation and the output boundary owns publication. See the [Published Evidence Context contract](../integrations/evidence-context.md) -for the durable result. +for the durable source-unit excerpt. Lane artifacts retain citation and lane +provenance; the framework does not add either to that published excerpt. An artifact family is broader than a module: it owns the cohesive domain feature across its artifact type, codec, stage modules, validators, prompt diff --git a/docs/operations.md b/docs/operations.md index 2f21777..2e554f9 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -110,9 +110,9 @@ are defined in [Accepted Chunk Map](integrations/chunk-map.md). An optional [evidence context](integrations/evidence-context.md) contains source-unit text and metadata. It is not a cache or debug artifact: retain it with the output bundle only for as long as consumers need it, and apply source-content access -controls to the entire bundle. Selected lanes may collectively cite most of a -transcript, so a broad allowlist can make the evidence artifact nearly as -sensitive and large as the source itself. +controls to the entire bundle. Its selected source-unit excerpt may include +every source unit once when coverage is broad or its configured window is +large, so do not assume a byte or token reduction or reduced sensitivity. ## Chunk-Plan Cache