Document published evidence context

This commit is contained in:
2026-07-27 18:28:02 +00:00
parent 256cc98ddb
commit 893b03fccf
10 changed files with 207 additions and 18 deletions

View File

@@ -211,19 +211,41 @@ Validator bindings accept only **module**, **llm_profile**, and **options**.
They reject **references**, **retries**, and nested **validators**. Deterministic
validators reject an explicit **llm_profile**.
The **json** output module accepts one option:
The **json** output module accepts optional **include_chunk_map** and
**evidence_context** settings:
~~~yaml
output:
module: json
options:
include_chunk_map: true
evidence_context:
enabled: true
window_units: 3
lanes:
- npcs
- spells
~~~
**include_chunk_map** is a boolean and defaults to false. It adds the accepted
chunk map when one exists; its wire format is defined in the
[chunk-map contract](integrations/chunk-map.md).
Omitting **evidence_context** disables evidence publication. When present, it
is an object with these strict fields:
| Field | Type | Rules |
| --- | --- | --- |
| **enabled** | boolean | Required. `false` permits no other evidence fields. |
| **lanes** | array of strings | Required and non-empty when enabled. Each value is trimmed and must be unique; every value must name a configured pipeline lane. |
| **window_units** | non-negative integer | Optional when enabled; defaults to 3. Zero retains only directly cited units. |
Unknown outer or nested option fields are rejected, as are incompatible YAML
types. The allowlist remains valid when a run uses lane filtering: a configured
lane that is not active for that invocation simply contributes no evidence.
Evidence publication is opt-in because it can persist source text and metadata.
Its payload contract is [Published Evidence Context](integrations/evidence-context.md).
## References And Ordered Handoffs
Reference maps bind named slots that the selected target declares. A scalar is

View File

@@ -49,6 +49,14 @@ guessed filename. Before decoding a selected payload, verify its descriptor's
media type and schema identity against the relevant published artifact
contract. The JSON bundle contract links to the available lane contracts.
If `index.json` has an `evidence_context` descriptor, treat it as a
pipeline-wide artifact rather than a lane entry. Verify its six descriptor
fields before decoding the linked file according to the [Published Evidence
Context contract](../integrations/evidence-context.md). Use each
`evidence_refs` entry as the citation to source material. Its surrounding
context range and included units explain the citation, but do not widen or
replace the cited source reference.
A zero exit status may still report rejected outputs, warnings, or absent
lanes. The caller decides which lane IDs are required for its own work and
which are optional; it should make that decision explicitly rather than infer
@@ -61,4 +69,6 @@ Keep the receipt with the published `manifest.json`, and retain
them. Treat the input, output bundle, cache, debug bundle, and captured process
logs as potentially sensitive data. Apply the caller's access controls and
retention policy, and avoid copying secrets into arguments, logs, or
provenance records.
provenance records. An evidence-context artifact contains source-unit text and
metadata, and selected lanes can cover most of an input; preserve and share it
only when that source content is authorized for the recipient.

View File

@@ -0,0 +1,116 @@
# Published Evidence Context
This contract defines the optional `source/evidence-context` artifact emitted
by the production JSON output. Its configuration is owned by
[Configuration](../config.md#module-bindings-and-validators); its logical-file
discovery is owned by [Published JSON Output](json-output.md).
## Identity And Discovery
When enabled, the JSON bundle contains `evidence-context.json` and an
`index.json` `evidence_context` descriptor with the same six fields as other
pipeline-wide artifact descriptors.
| Property | Value |
| --- | --- |
| Artifact kind | `source/evidence-context` |
| Media type | `application/json` |
| Schema ID | `notarius.source.evidence_context` |
| Schema name | `notarius_source_evidence_context_v1` |
| Schema version | `v1` |
| Logical file | `evidence-context.json` |
Consumers must discover the file from the descriptor, verify all six descriptor
fields, and decode only a supported schema version. The descriptor is optional:
its absence means evidence publication was not enabled for that bundle.
## Payload
The v1 payload is a JSON object with required `source_id`, `source_digest`,
`window_units`, `selected_lanes`, and `contexts` fields. `selected_lanes` and
`contexts` are always arrays; an enabled configuration with no accepted direct
evidence publishes `contexts: []`.
```json
{
"source_id": "session-alpha",
"source_digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"window_units": 1,
"selected_lanes": ["npcs", "spells"],
"contexts": [
{
"context_ref": {
"source_id": "session-alpha",
"start_unit_id": 10,
"end_unit_id": 20
},
"evidence_refs": [
{
"lane_id": "spells",
"source_ref": {
"source_id": "session-alpha",
"start_unit_id": 10,
"end_unit_id": 10
}
}
],
"units": [
{
"id": 10,
"kind": "transcript_segment",
"text": "Aria casts Cure Wounds.",
"ref": {
"source_id": "session-alpha",
"start_unit_id": 10,
"end_unit_id": 10
}
},
{
"id": 20,
"kind": "transcript_segment",
"text": "The party regroups.",
"ref": {
"source_id": "session-alpha",
"start_unit_id": 20,
"end_unit_id": 20
}
}
]
}
]
}
```
Each context requires `context_ref`, `evidence_refs`, and `units` arrays.
`context_ref` identifies the first and last included unit. Each evidence entry
contains a selected `lane_id` and an original `source_ref`. A unit uses the
existing source-unit shape: required `id`, `kind`, `text`, and self `ref`, plus
optional JSON-object `metadata`. Fixed payload objects reject unknown fields;
unit metadata may contain application-defined JSON values.
## Citations And Context
`evidence_refs` are the authoritative citations. They identify the direct
references emitted by accepted normalized artifacts. `context_ref` and the
units collection include those cited units plus nearby source units selected by
the configured window. They are explanatory context, not widened citations.
Only accepted outputs from the configured lane allowlist contribute. Rejected,
failed, absent, and lane-filtered outputs do not contribute. The artifact never
contains raw input bytes, prompts, model responses, auxiliary reference
content, credentials, or filesystem paths.
## Ordering And Compatibility
The selected lane allowlist is lexical. Contexts and units are in source
document position order, not numeric unit-ID order. Direct evidence entries
are deterministically ordered by lane and source reference. Overlapping or
contiguous windows merge, and each source unit appears at most once in the
resulting contexts.
The artifact is additive to the JSON bundle and is not a lane payload,
normalized-output count, checkpoint, or generated reference. Consumers that
do not need it must tolerate the absent optional descriptor. Consumers that do
use it should preserve the artifact and its schema identity with the run
provenance, and should treat its source text and metadata as sensitive durable
content.

View File

@@ -3,14 +3,14 @@
This document defines the logical JSON bundle emitted by the production JSON
output encoder. The bundles physical destination, atomic publication, and
retention are operational concerns; see [Operations](../operations.md#output-bundles).
Output configuration, including chunk-map export, belongs in
Output configuration, including chunk-map and evidence-context publication, belongs in
[Configuration](../config.md#module-bindings-and-validators).
## Bundle Layout
All paths below are logical, relative, slash-separated bundle paths. The
encoder always emits the first four JSON files below and adds lane or chunk-map
files when their corresponding artifacts are available:
encoder always emits the first four JSON files below and adds lane or
pipeline-wide artifact files when their corresponding artifacts are available:
A subprocess caller first obtains the physical bundle root from the
[run-result receipt](run-result.md), then resolves `index.json` beneath that
@@ -24,6 +24,7 @@ root for the logical discovery described here.
| `warnings.json` | Accepted-output and run warnings. |
| `lanes/<safe-lane-id>.json` | One normalized artifact payload for each lane. |
| `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. |
| `evidence-context.json` | Optional source-context artifact, when evidence publication is enabled. |
JSON files are pretty-printed with a trailing newline. Lane payloads are
accepted only when their media type is `application/json`.
@@ -49,13 +50,15 @@ normalized lanes has this valid minimal index:
| `rejected_file` | Yes | Always `rejected.json`. |
| `warnings_file` | Yes | Always `warnings.json`. |
| `chunk_map` | No | Descriptor for the pipeline-wide `chunk-map.json`; never a lane descriptor. |
| `evidence_context` | No | Descriptor for the pipeline-wide `evidence-context.json`; never a lane descriptor. |
Each lane descriptor has required `lane_id` and `file`. It may also include
`media_type`, `module_key`, `schema_id`, `schema_name`, and `schema_version`
when supplied by the normalized artifact. A `chunk_map` descriptor contains
`artifact_kind`, `file`, `media_type`, `schema_id`, `schema_name`, and
`schema_version`; its payload is defined by the
[Accepted Chunk Map contract](chunk-map.md).
when supplied by the normalized artifact. Each pipeline-wide artifact
descriptor (`chunk_map` or `evidence_context`) contains `artifact_kind`,
`file`, `media_type`, `schema_id`, `schema_name`, and `schema_version`. Their
payloads are defined by the [Accepted Chunk Map contract](chunk-map.md) and
[Published Evidence Context](evidence-context.md), respectively.
The lane path is derived from its lane ID. Characters outside letters, digits,
periods, underscores, and hyphens become underscores; `..` sequences are

View File

@@ -30,6 +30,14 @@ they need, register each leaf implementation, and add any family-owned assets
or default validator chains. They return contextual errors so production
composition fails at startup rather than at the first run.
An artifact family can register an optional typed evidence projector alongside
its codec. The projector returns defensive copies of the artifact's direct
generic source references and must use the codec's exact Go type. It does not
interpret surrounding context or publish files; the pipeline validates the
capability during preparation and the output boundary owns publication. See
the [Published Evidence Context contract](../integrations/evidence-context.md)
for the durable result.
## Production Composition
Production composition is intentionally split by family:

View File

@@ -52,6 +52,13 @@ checkpoint fingerprints. Missing registrations, incompatible typed entries,
nil implementations, and constructor failures are reported before source
parsing or any stage operation begins.
An output encoder can opt into source-evidence publication through its output
policy. Preparation keeps the configured lane allowlist and active lanes
separate, then verifies an exact typed evidence projector and registered codec
for each active lane. The resulting private plan is immutable; lanes excluded
by invocation filtering remain configured but do not acquire a projector for
that run.
## Typed Lanes And References
Each resolved lane has one artifact kind, codec, and exact Go type. The
@@ -110,11 +117,16 @@ directives consume this same budget and validate any final safe fallback through
the normalizer chain.
After terminal lane work, the runner assembles manifest provenance, normalized
artifacts, rejections, warnings, and an optional accepted chunk map. The output
encoder returns logical files; it does not choose a physical directory. The CLI
publishes those files only after the runner returns without a framework error.
Logical file names and schemas are defined by the
[output integration contracts](../integrations/).
artifacts, rejections, warnings, and an optional accepted chunk map. When an
output policy selected evidence lanes, it decodes accepted serialized normalize
outputs through their registered codecs and invokes the prepared typed
projectors. Rejected or absent lanes contribute nothing. This reconstruction is
also used after normalized-checkpoint reuse, so no second typed output channel
is retained. The runner passes the resulting owned artifact to the output
encoder, which returns logical files and does not choose a physical directory.
The CLI publishes those files only after the runner returns without a framework
error. Logical file names and schemas are defined by the [output integration
contracts](../integrations/).
## Checkpoint And Debug Hooks

View File

@@ -36,7 +36,11 @@ On supported Unix systems, output directories and files are created with
requested modes **0755** and **0644**. Chunk-plan, checkpoint, and debug
directories and files use **0700** and **0600**. The operating system's umask
may impose stricter output modes. Cache and debug roots may contain sensitive
source-derived data, so provision them for one trusted account or service.
source-derived data, so provision them for one trusted account or service. An
output bundle can also contain source content when its JSON output enables
evidence publication. Apply an appropriate umask and output-root access policy
before enabling that option; the requested output modes alone may not be
suitable for transcript-bearing bundles.
## Run Lifecycle
@@ -66,7 +70,13 @@ run directory remains for inspection and is never removed automatically.
Treat an output bundle as durable user data. Do not use cache-cleanup policy to
remove it. An optional accepted chunk map is also durable output and can carry
source- or model-derived annotations; its content and compatibility contract
are defined in [Accepted Chunk Map](integrations/chunk-map.md).
are defined in [Accepted Chunk Map](integrations/chunk-map.md). An optional
[evidence context](integrations/evidence-context.md) contains source-unit text
and metadata. It is not a cache or debug artifact: retain it with the output
bundle only for as long as consumers need it, and apply source-content access
controls to the entire bundle. Selected lanes may collectively cite most of a
transcript, so a broad allowlist can make the evidence artifact nearly as
sensitive and large as the source itself.
## Chunk-Plan Cache

View File

@@ -79,6 +79,14 @@ Pipeline resolution requires a compatible codec and matching kind-specific
variants before a typed lane can be accepted. Framework-owned erasure remains
private and must report type incompatibility as an error rather than a panic.
An artifact kind may additionally provide a typed evidence projection that
copies its direct generic source references. Preparation proves that projection
matches the artifact codec's exact Go type before retaining it for an output
policy. The runner reconstructs evidence only from accepted serialized
normalized artifacts, and the output boundary owns any resulting publication.
Generic framework code never infers evidence by inspecting domain JSON or
depends on domain artifact types.
Auxiliary references provide context or disambiguation. They are not source
evidence and must not be converted into source references.

View File

@@ -2,7 +2,7 @@
## Status
Accepted for implementation.
Implemented.
## Purpose

View File

@@ -2,7 +2,7 @@
## Status
Ready for implementation.
Completed.
## Objective