Document published evidence context
This commit is contained in:
@@ -211,19 +211,41 @@ Validator bindings accept only **module**, **llm_profile**, and **options**.
|
|||||||
They reject **references**, **retries**, and nested **validators**. Deterministic
|
They reject **references**, **retries**, and nested **validators**. Deterministic
|
||||||
validators reject an explicit **llm_profile**.
|
validators reject an explicit **llm_profile**.
|
||||||
|
|
||||||
The **json** output module accepts one option:
|
The **json** output module accepts optional **include_chunk_map** and
|
||||||
|
**evidence_context** settings:
|
||||||
|
|
||||||
~~~yaml
|
~~~yaml
|
||||||
output:
|
output:
|
||||||
module: json
|
module: json
|
||||||
options:
|
options:
|
||||||
include_chunk_map: true
|
include_chunk_map: true
|
||||||
|
evidence_context:
|
||||||
|
enabled: true
|
||||||
|
window_units: 3
|
||||||
|
lanes:
|
||||||
|
- npcs
|
||||||
|
- spells
|
||||||
~~~
|
~~~
|
||||||
|
|
||||||
**include_chunk_map** is a boolean and defaults to false. It adds the accepted
|
**include_chunk_map** is a boolean and defaults to false. It adds the accepted
|
||||||
chunk map when one exists; its wire format is defined in the
|
chunk map when one exists; its wire format is defined in the
|
||||||
[chunk-map contract](integrations/chunk-map.md).
|
[chunk-map contract](integrations/chunk-map.md).
|
||||||
|
|
||||||
|
Omitting **evidence_context** disables evidence publication. When present, it
|
||||||
|
is an object with these strict fields:
|
||||||
|
|
||||||
|
| Field | Type | Rules |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| **enabled** | boolean | Required. `false` permits no other evidence fields. |
|
||||||
|
| **lanes** | array of strings | Required and non-empty when enabled. Each value is trimmed and must be unique; every value must name a configured pipeline lane. |
|
||||||
|
| **window_units** | non-negative integer | Optional when enabled; defaults to 3. Zero retains only directly cited units. |
|
||||||
|
|
||||||
|
Unknown outer or nested option fields are rejected, as are incompatible YAML
|
||||||
|
types. The allowlist remains valid when a run uses lane filtering: a configured
|
||||||
|
lane that is not active for that invocation simply contributes no evidence.
|
||||||
|
Evidence publication is opt-in because it can persist source text and metadata.
|
||||||
|
Its payload contract is [Published Evidence Context](integrations/evidence-context.md).
|
||||||
|
|
||||||
## References And Ordered Handoffs
|
## References And Ordered Handoffs
|
||||||
|
|
||||||
Reference maps bind named slots that the selected target declares. A scalar is
|
Reference maps bind named slots that the selected target declares. A scalar is
|
||||||
|
|||||||
@@ -49,6 +49,14 @@ guessed filename. Before decoding a selected payload, verify its descriptor's
|
|||||||
media type and schema identity against the relevant published artifact
|
media type and schema identity against the relevant published artifact
|
||||||
contract. The JSON bundle contract links to the available lane contracts.
|
contract. The JSON bundle contract links to the available lane contracts.
|
||||||
|
|
||||||
|
If `index.json` has an `evidence_context` descriptor, treat it as a
|
||||||
|
pipeline-wide artifact rather than a lane entry. Verify its six descriptor
|
||||||
|
fields before decoding the linked file according to the [Published Evidence
|
||||||
|
Context contract](../integrations/evidence-context.md). Use each
|
||||||
|
`evidence_refs` entry as the citation to source material. Its surrounding
|
||||||
|
context range and included units explain the citation, but do not widen or
|
||||||
|
replace the cited source reference.
|
||||||
|
|
||||||
A zero exit status may still report rejected outputs, warnings, or absent
|
A zero exit status may still report rejected outputs, warnings, or absent
|
||||||
lanes. The caller decides which lane IDs are required for its own work and
|
lanes. The caller decides which lane IDs are required for its own work and
|
||||||
which are optional; it should make that decision explicitly rather than infer
|
which are optional; it should make that decision explicitly rather than infer
|
||||||
@@ -61,4 +69,6 @@ Keep the receipt with the published `manifest.json`, and retain
|
|||||||
them. Treat the input, output bundle, cache, debug bundle, and captured process
|
them. Treat the input, output bundle, cache, debug bundle, and captured process
|
||||||
logs as potentially sensitive data. Apply the caller's access controls and
|
logs as potentially sensitive data. Apply the caller's access controls and
|
||||||
retention policy, and avoid copying secrets into arguments, logs, or
|
retention policy, and avoid copying secrets into arguments, logs, or
|
||||||
provenance records.
|
provenance records. An evidence-context artifact contains source-unit text and
|
||||||
|
metadata, and selected lanes can cover most of an input; preserve and share it
|
||||||
|
only when that source content is authorized for the recipient.
|
||||||
|
|||||||
116
docs/integrations/evidence-context.md
Normal file
116
docs/integrations/evidence-context.md
Normal file
@@ -0,0 +1,116 @@
|
|||||||
|
# Published Evidence Context
|
||||||
|
|
||||||
|
This contract defines the optional `source/evidence-context` artifact emitted
|
||||||
|
by the production JSON output. Its configuration is owned by
|
||||||
|
[Configuration](../config.md#module-bindings-and-validators); its logical-file
|
||||||
|
discovery is owned by [Published JSON Output](json-output.md).
|
||||||
|
|
||||||
|
## Identity And Discovery
|
||||||
|
|
||||||
|
When enabled, the JSON bundle contains `evidence-context.json` and an
|
||||||
|
`index.json` `evidence_context` descriptor with the same six fields as other
|
||||||
|
pipeline-wide artifact descriptors.
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
| --- | --- |
|
||||||
|
| Artifact kind | `source/evidence-context` |
|
||||||
|
| Media type | `application/json` |
|
||||||
|
| Schema ID | `notarius.source.evidence_context` |
|
||||||
|
| Schema name | `notarius_source_evidence_context_v1` |
|
||||||
|
| Schema version | `v1` |
|
||||||
|
| Logical file | `evidence-context.json` |
|
||||||
|
|
||||||
|
Consumers must discover the file from the descriptor, verify all six descriptor
|
||||||
|
fields, and decode only a supported schema version. The descriptor is optional:
|
||||||
|
its absence means evidence publication was not enabled for that bundle.
|
||||||
|
|
||||||
|
## Payload
|
||||||
|
|
||||||
|
The v1 payload is a JSON object with required `source_id`, `source_digest`,
|
||||||
|
`window_units`, `selected_lanes`, and `contexts` fields. `selected_lanes` and
|
||||||
|
`contexts` are always arrays; an enabled configuration with no accepted direct
|
||||||
|
evidence publishes `contexts: []`.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"source_id": "session-alpha",
|
||||||
|
"source_digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
|
||||||
|
"window_units": 1,
|
||||||
|
"selected_lanes": ["npcs", "spells"],
|
||||||
|
"contexts": [
|
||||||
|
{
|
||||||
|
"context_ref": {
|
||||||
|
"source_id": "session-alpha",
|
||||||
|
"start_unit_id": 10,
|
||||||
|
"end_unit_id": 20
|
||||||
|
},
|
||||||
|
"evidence_refs": [
|
||||||
|
{
|
||||||
|
"lane_id": "spells",
|
||||||
|
"source_ref": {
|
||||||
|
"source_id": "session-alpha",
|
||||||
|
"start_unit_id": 10,
|
||||||
|
"end_unit_id": 10
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"units": [
|
||||||
|
{
|
||||||
|
"id": 10,
|
||||||
|
"kind": "transcript_segment",
|
||||||
|
"text": "Aria casts Cure Wounds.",
|
||||||
|
"ref": {
|
||||||
|
"source_id": "session-alpha",
|
||||||
|
"start_unit_id": 10,
|
||||||
|
"end_unit_id": 10
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": 20,
|
||||||
|
"kind": "transcript_segment",
|
||||||
|
"text": "The party regroups.",
|
||||||
|
"ref": {
|
||||||
|
"source_id": "session-alpha",
|
||||||
|
"start_unit_id": 20,
|
||||||
|
"end_unit_id": 20
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Each context requires `context_ref`, `evidence_refs`, and `units` arrays.
|
||||||
|
`context_ref` identifies the first and last included unit. Each evidence entry
|
||||||
|
contains a selected `lane_id` and an original `source_ref`. A unit uses the
|
||||||
|
existing source-unit shape: required `id`, `kind`, `text`, and self `ref`, plus
|
||||||
|
optional JSON-object `metadata`. Fixed payload objects reject unknown fields;
|
||||||
|
unit metadata may contain application-defined JSON values.
|
||||||
|
|
||||||
|
## Citations And Context
|
||||||
|
|
||||||
|
`evidence_refs` are the authoritative citations. They identify the direct
|
||||||
|
references emitted by accepted normalized artifacts. `context_ref` and the
|
||||||
|
units collection include those cited units plus nearby source units selected by
|
||||||
|
the configured window. They are explanatory context, not widened citations.
|
||||||
|
|
||||||
|
Only accepted outputs from the configured lane allowlist contribute. Rejected,
|
||||||
|
failed, absent, and lane-filtered outputs do not contribute. The artifact never
|
||||||
|
contains raw input bytes, prompts, model responses, auxiliary reference
|
||||||
|
content, credentials, or filesystem paths.
|
||||||
|
|
||||||
|
## Ordering And Compatibility
|
||||||
|
|
||||||
|
The selected lane allowlist is lexical. Contexts and units are in source
|
||||||
|
document position order, not numeric unit-ID order. Direct evidence entries
|
||||||
|
are deterministically ordered by lane and source reference. Overlapping or
|
||||||
|
contiguous windows merge, and each source unit appears at most once in the
|
||||||
|
resulting contexts.
|
||||||
|
|
||||||
|
The artifact is additive to the JSON bundle and is not a lane payload,
|
||||||
|
normalized-output count, checkpoint, or generated reference. Consumers that
|
||||||
|
do not need it must tolerate the absent optional descriptor. Consumers that do
|
||||||
|
use it should preserve the artifact and its schema identity with the run
|
||||||
|
provenance, and should treat its source text and metadata as sensitive durable
|
||||||
|
content.
|
||||||
@@ -3,14 +3,14 @@
|
|||||||
This document defines the logical JSON bundle emitted by the production JSON
|
This document defines the logical JSON bundle emitted by the production JSON
|
||||||
output encoder. The bundle’s physical destination, atomic publication, and
|
output encoder. The bundle’s physical destination, atomic publication, and
|
||||||
retention are operational concerns; see [Operations](../operations.md#output-bundles).
|
retention are operational concerns; see [Operations](../operations.md#output-bundles).
|
||||||
Output configuration, including chunk-map export, belongs in
|
Output configuration, including chunk-map and evidence-context publication, belongs in
|
||||||
[Configuration](../config.md#module-bindings-and-validators).
|
[Configuration](../config.md#module-bindings-and-validators).
|
||||||
|
|
||||||
## Bundle Layout
|
## Bundle Layout
|
||||||
|
|
||||||
All paths below are logical, relative, slash-separated bundle paths. The
|
All paths below are logical, relative, slash-separated bundle paths. The
|
||||||
encoder always emits the first four JSON files below and adds lane or chunk-map
|
encoder always emits the first four JSON files below and adds lane or
|
||||||
files when their corresponding artifacts are available:
|
pipeline-wide artifact files when their corresponding artifacts are available:
|
||||||
|
|
||||||
A subprocess caller first obtains the physical bundle root from the
|
A subprocess caller first obtains the physical bundle root from the
|
||||||
[run-result receipt](run-result.md), then resolves `index.json` beneath that
|
[run-result receipt](run-result.md), then resolves `index.json` beneath that
|
||||||
@@ -24,6 +24,7 @@ root for the logical discovery described here.
|
|||||||
| `warnings.json` | Accepted-output and run warnings. |
|
| `warnings.json` | Accepted-output and run warnings. |
|
||||||
| `lanes/<safe-lane-id>.json` | One normalized artifact payload for each lane. |
|
| `lanes/<safe-lane-id>.json` | One normalized artifact payload for each lane. |
|
||||||
| `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. |
|
| `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. |
|
||||||
|
| `evidence-context.json` | Optional source-context artifact, when evidence publication is enabled. |
|
||||||
|
|
||||||
JSON files are pretty-printed with a trailing newline. Lane payloads are
|
JSON files are pretty-printed with a trailing newline. Lane payloads are
|
||||||
accepted only when their media type is `application/json`.
|
accepted only when their media type is `application/json`.
|
||||||
@@ -49,13 +50,15 @@ normalized lanes has this valid minimal index:
|
|||||||
| `rejected_file` | Yes | Always `rejected.json`. |
|
| `rejected_file` | Yes | Always `rejected.json`. |
|
||||||
| `warnings_file` | Yes | Always `warnings.json`. |
|
| `warnings_file` | Yes | Always `warnings.json`. |
|
||||||
| `chunk_map` | No | Descriptor for the pipeline-wide `chunk-map.json`; never a lane descriptor. |
|
| `chunk_map` | No | Descriptor for the pipeline-wide `chunk-map.json`; never a lane descriptor. |
|
||||||
|
| `evidence_context` | No | Descriptor for the pipeline-wide `evidence-context.json`; never a lane descriptor. |
|
||||||
|
|
||||||
Each lane descriptor has required `lane_id` and `file`. It may also include
|
Each lane descriptor has required `lane_id` and `file`. It may also include
|
||||||
`media_type`, `module_key`, `schema_id`, `schema_name`, and `schema_version`
|
`media_type`, `module_key`, `schema_id`, `schema_name`, and `schema_version`
|
||||||
when supplied by the normalized artifact. A `chunk_map` descriptor contains
|
when supplied by the normalized artifact. Each pipeline-wide artifact
|
||||||
`artifact_kind`, `file`, `media_type`, `schema_id`, `schema_name`, and
|
descriptor (`chunk_map` or `evidence_context`) contains `artifact_kind`,
|
||||||
`schema_version`; its payload is defined by the
|
`file`, `media_type`, `schema_id`, `schema_name`, and `schema_version`. Their
|
||||||
[Accepted Chunk Map contract](chunk-map.md).
|
payloads are defined by the [Accepted Chunk Map contract](chunk-map.md) and
|
||||||
|
[Published Evidence Context](evidence-context.md), respectively.
|
||||||
|
|
||||||
The lane path is derived from its lane ID. Characters outside letters, digits,
|
The lane path is derived from its lane ID. Characters outside letters, digits,
|
||||||
periods, underscores, and hyphens become underscores; `..` sequences are
|
periods, underscores, and hyphens become underscores; `..` sequences are
|
||||||
|
|||||||
@@ -30,6 +30,14 @@ they need, register each leaf implementation, and add any family-owned assets
|
|||||||
or default validator chains. They return contextual errors so production
|
or default validator chains. They return contextual errors so production
|
||||||
composition fails at startup rather than at the first run.
|
composition fails at startup rather than at the first run.
|
||||||
|
|
||||||
|
An artifact family can register an optional typed evidence projector alongside
|
||||||
|
its codec. The projector returns defensive copies of the artifact's direct
|
||||||
|
generic source references and must use the codec's exact Go type. It does not
|
||||||
|
interpret surrounding context or publish files; the pipeline validates the
|
||||||
|
capability during preparation and the output boundary owns publication. See
|
||||||
|
the [Published Evidence Context contract](../integrations/evidence-context.md)
|
||||||
|
for the durable result.
|
||||||
|
|
||||||
## Production Composition
|
## Production Composition
|
||||||
|
|
||||||
Production composition is intentionally split by family:
|
Production composition is intentionally split by family:
|
||||||
|
|||||||
@@ -52,6 +52,13 @@ checkpoint fingerprints. Missing registrations, incompatible typed entries,
|
|||||||
nil implementations, and constructor failures are reported before source
|
nil implementations, and constructor failures are reported before source
|
||||||
parsing or any stage operation begins.
|
parsing or any stage operation begins.
|
||||||
|
|
||||||
|
An output encoder can opt into source-evidence publication through its output
|
||||||
|
policy. Preparation keeps the configured lane allowlist and active lanes
|
||||||
|
separate, then verifies an exact typed evidence projector and registered codec
|
||||||
|
for each active lane. The resulting private plan is immutable; lanes excluded
|
||||||
|
by invocation filtering remain configured but do not acquire a projector for
|
||||||
|
that run.
|
||||||
|
|
||||||
## Typed Lanes And References
|
## Typed Lanes And References
|
||||||
|
|
||||||
Each resolved lane has one artifact kind, codec, and exact Go type. The
|
Each resolved lane has one artifact kind, codec, and exact Go type. The
|
||||||
@@ -110,11 +117,16 @@ directives consume this same budget and validate any final safe fallback through
|
|||||||
the normalizer chain.
|
the normalizer chain.
|
||||||
|
|
||||||
After terminal lane work, the runner assembles manifest provenance, normalized
|
After terminal lane work, the runner assembles manifest provenance, normalized
|
||||||
artifacts, rejections, warnings, and an optional accepted chunk map. The output
|
artifacts, rejections, warnings, and an optional accepted chunk map. When an
|
||||||
encoder returns logical files; it does not choose a physical directory. The CLI
|
output policy selected evidence lanes, it decodes accepted serialized normalize
|
||||||
publishes those files only after the runner returns without a framework error.
|
outputs through their registered codecs and invokes the prepared typed
|
||||||
Logical file names and schemas are defined by the
|
projectors. Rejected or absent lanes contribute nothing. This reconstruction is
|
||||||
[output integration contracts](../integrations/).
|
also used after normalized-checkpoint reuse, so no second typed output channel
|
||||||
|
is retained. The runner passes the resulting owned artifact to the output
|
||||||
|
encoder, which returns logical files and does not choose a physical directory.
|
||||||
|
The CLI publishes those files only after the runner returns without a framework
|
||||||
|
error. Logical file names and schemas are defined by the [output integration
|
||||||
|
contracts](../integrations/).
|
||||||
|
|
||||||
## Checkpoint And Debug Hooks
|
## Checkpoint And Debug Hooks
|
||||||
|
|
||||||
|
|||||||
@@ -36,7 +36,11 @@ On supported Unix systems, output directories and files are created with
|
|||||||
requested modes **0755** and **0644**. Chunk-plan, checkpoint, and debug
|
requested modes **0755** and **0644**. Chunk-plan, checkpoint, and debug
|
||||||
directories and files use **0700** and **0600**. The operating system's umask
|
directories and files use **0700** and **0600**. The operating system's umask
|
||||||
may impose stricter output modes. Cache and debug roots may contain sensitive
|
may impose stricter output modes. Cache and debug roots may contain sensitive
|
||||||
source-derived data, so provision them for one trusted account or service.
|
source-derived data, so provision them for one trusted account or service. An
|
||||||
|
output bundle can also contain source content when its JSON output enables
|
||||||
|
evidence publication. Apply an appropriate umask and output-root access policy
|
||||||
|
before enabling that option; the requested output modes alone may not be
|
||||||
|
suitable for transcript-bearing bundles.
|
||||||
|
|
||||||
## Run Lifecycle
|
## Run Lifecycle
|
||||||
|
|
||||||
@@ -66,7 +70,13 @@ run directory remains for inspection and is never removed automatically.
|
|||||||
Treat an output bundle as durable user data. Do not use cache-cleanup policy to
|
Treat an output bundle as durable user data. Do not use cache-cleanup policy to
|
||||||
remove it. An optional accepted chunk map is also durable output and can carry
|
remove it. An optional accepted chunk map is also durable output and can carry
|
||||||
source- or model-derived annotations; its content and compatibility contract
|
source- or model-derived annotations; its content and compatibility contract
|
||||||
are defined in [Accepted Chunk Map](integrations/chunk-map.md).
|
are defined in [Accepted Chunk Map](integrations/chunk-map.md). An optional
|
||||||
|
[evidence context](integrations/evidence-context.md) contains source-unit text
|
||||||
|
and metadata. It is not a cache or debug artifact: retain it with the output
|
||||||
|
bundle only for as long as consumers need it, and apply source-content access
|
||||||
|
controls to the entire bundle. Selected lanes may collectively cite most of a
|
||||||
|
transcript, so a broad allowlist can make the evidence artifact nearly as
|
||||||
|
sensitive and large as the source itself.
|
||||||
|
|
||||||
## Chunk-Plan Cache
|
## Chunk-Plan Cache
|
||||||
|
|
||||||
|
|||||||
@@ -79,6 +79,14 @@ Pipeline resolution requires a compatible codec and matching kind-specific
|
|||||||
variants before a typed lane can be accepted. Framework-owned erasure remains
|
variants before a typed lane can be accepted. Framework-owned erasure remains
|
||||||
private and must report type incompatibility as an error rather than a panic.
|
private and must report type incompatibility as an error rather than a panic.
|
||||||
|
|
||||||
|
An artifact kind may additionally provide a typed evidence projection that
|
||||||
|
copies its direct generic source references. Preparation proves that projection
|
||||||
|
matches the artifact codec's exact Go type before retaining it for an output
|
||||||
|
policy. The runner reconstructs evidence only from accepted serialized
|
||||||
|
normalized artifacts, and the output boundary owns any resulting publication.
|
||||||
|
Generic framework code never infers evidence by inspecting domain JSON or
|
||||||
|
depends on domain artifact types.
|
||||||
|
|
||||||
Auxiliary references provide context or disambiguation. They are not source
|
Auxiliary references provide context or disambiguation. They are not source
|
||||||
evidence and must not be converted into source references.
|
evidence and must not be converted into source references.
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
Accepted for implementation.
|
Implemented.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
Ready for implementation.
|
Completed.
|
||||||
|
|
||||||
## Objective
|
## Objective
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user