Add a feature roadmap and implementation plan for a context evidence artifact
This commit is contained in:
226
docs/roadmap/evidence.md
Normal file
226
docs/roadmap/evidence.md
Normal file
@@ -0,0 +1,226 @@
|
||||
# Published Evidence Context
|
||||
|
||||
## Status
|
||||
|
||||
Accepted for implementation.
|
||||
|
||||
## Purpose
|
||||
|
||||
Let downstream consumers build narrative reports from normalized artifacts
|
||||
without separately parsing the original transcript or resolving source-unit
|
||||
references themselves.
|
||||
|
||||
The production JSON output optionally publishes one deterministic, deduplicated
|
||||
evidence-context artifact containing the transcript units relevant to explicitly
|
||||
selected normalized lanes. Existing lane payloads remain the canonical semantic
|
||||
results and retain their precise source references.
|
||||
|
||||
## Desired End State
|
||||
|
||||
When evidence-context publication is enabled, a consumer can:
|
||||
|
||||
1. discover one versioned evidence-context document through `index.json`;
|
||||
2. obtain the union of source units needed to understand evidence cited by the
|
||||
selected normalized lanes;
|
||||
3. distinguish each artifact's direct evidence references from surrounding
|
||||
units included only for narrative context;
|
||||
4. retain speaker, timestamp, and other accepted source-unit metadata needed to
|
||||
interpret the transcript; and
|
||||
5. produce a narrative report without receiving duplicated transcript text in
|
||||
every lane payload.
|
||||
|
||||
This is deterministic output projection. It does not invoke an LLM, change
|
||||
normalization, or make surrounding context part of an artifact's evidence.
|
||||
|
||||
## Configuration Policy
|
||||
|
||||
Evidence publication is configured on the production JSON output module. The
|
||||
intended configuration shape is:
|
||||
|
||||
```yaml
|
||||
output:
|
||||
module: json
|
||||
options:
|
||||
evidence_context:
|
||||
enabled: true
|
||||
window_units: 3
|
||||
lanes:
|
||||
- combat-turns
|
||||
- item-events
|
||||
- npc-interactions
|
||||
- npcs
|
||||
- spells
|
||||
```
|
||||
|
||||
- Omitting `evidence_context` disables publication. When the object is present,
|
||||
`enabled` is required.
|
||||
- `enabled: false` accepts no `lanes` or `window_units` fields, preventing
|
||||
silently ignored configuration.
|
||||
- `lanes` is a required, non-empty allowlist of configured final lane IDs when
|
||||
evidence publication is enabled. Values are trimmed, unique, and normalized
|
||||
to lexical order.
|
||||
- `window_units` is a non-negative integer and defaults to `3`. Zero publishes
|
||||
only directly referenced units.
|
||||
- Unknown lanes, duplicate lane IDs, and selected lanes whose artifact kind
|
||||
cannot expose source evidence fail configuration resolution or pipeline
|
||||
preparation.
|
||||
- A selected lane that completes without a normalized output contributes no
|
||||
evidence and does not make an otherwise successful run fail.
|
||||
- Invocation-level lane filtering does not invalidate the configured allowlist.
|
||||
Allowlisted lanes excluded from the effective run contribute nothing, while
|
||||
the evidence document still records the configured allowlist.
|
||||
|
||||
The allowlist is intentional safety and stability policy. Scene descriptions
|
||||
and other broad-range lanes are excluded unless named expressly. Adding a new
|
||||
pipeline lane never silently increases output size or publishes more transcript
|
||||
content.
|
||||
|
||||
## Evidence Collection Boundary
|
||||
|
||||
Evidence collection applies to accepted final normalized artifacts from the
|
||||
selected lanes. It must not inspect arbitrary serialized JSON for fields named
|
||||
`source_ref` or `source_refs`, and the generic JSON output module must not
|
||||
depend on D&D artifact types.
|
||||
|
||||
Artifact-kind registrations expose their source references through an explicit
|
||||
typed projection contract. The framework uses that contract to assemble a
|
||||
domain-neutral evidence request containing:
|
||||
|
||||
- the accepted generic source document;
|
||||
- the selected lane and artifact identities; and
|
||||
- defensive copies of their direct source references.
|
||||
|
||||
The output stage owns publication of the resulting logical artifact. Generic
|
||||
framework code owns range validation, position-based expansion, and union
|
||||
logic. Domain-specific adapters own only the extraction of evidence references
|
||||
from their typed artifacts.
|
||||
|
||||
Both plural-reference artifacts and singular-reference artifacts, such as
|
||||
scene descriptions, can participate through the same projection contract.
|
||||
They do so only when their configured lane is allowlisted.
|
||||
|
||||
## Range Expansion And Deduplication
|
||||
|
||||
For every valid direct source reference:
|
||||
|
||||
1. resolve its endpoints through source-document positions, not numeric
|
||||
unit-ID arithmetic;
|
||||
2. expand the range by `window_units` positions on each side;
|
||||
3. clip the expanded range at document boundaries; and
|
||||
4. union overlapping or contiguous expanded ranges.
|
||||
|
||||
Published contexts and units remain in source-document order. Each source unit
|
||||
appears at most once in a merged context. Original direct references remain
|
||||
unchanged and are associated with their contributing lane IDs so consumers can
|
||||
tell why a context was included.
|
||||
|
||||
The projector must not silently omit or repair an invalid reference that
|
||||
reaches this boundary. Such a value violates the accepted normalized-artifact
|
||||
contract and causes output projection to fail with a content-safe error.
|
||||
|
||||
No implicit coverage limit truncates selected evidence. If the allowlisted
|
||||
lanes collectively cite most or all of a transcript, the evidence document may
|
||||
contain most or all of it. The explicit lane allowlist is the control that
|
||||
prevents a broad lane such as scene descriptions from doing so accidentally.
|
||||
|
||||
## Durable Evidence Artifact
|
||||
|
||||
The JSON bundle gains one optional, non-lane artifact with these durable
|
||||
identities:
|
||||
|
||||
| Property | Value |
|
||||
| --- | --- |
|
||||
| Logical file | `evidence-context.json` |
|
||||
| Index descriptor | `evidence_context` |
|
||||
| Artifact kind | `source/evidence-context` |
|
||||
| Media type | `application/json` |
|
||||
| Schema ID | `notarius.source.evidence_context` |
|
||||
| Schema name | `notarius_source_evidence_context_v1` |
|
||||
| Schema version | `v1` |
|
||||
|
||||
The descriptor in `index.json` carries the artifact and schema identities,
|
||||
analogous to the existing chunk-map descriptor. The artifact is present
|
||||
whenever evidence publication is enabled, including when its context collection
|
||||
is empty.
|
||||
|
||||
The document contains:
|
||||
|
||||
- the source document ID and semantic digest;
|
||||
- the effective window size;
|
||||
- the sorted configured lane allowlist;
|
||||
- an ordered context collection;
|
||||
- each context's expanded start and end unit IDs;
|
||||
- the original direct references and contributing lane IDs covered by that
|
||||
context; and
|
||||
- the ordered accepted source units, including unit ID, kind, text,
|
||||
self-reference, and metadata.
|
||||
|
||||
Expanded context bounds are navigation aids, not citations. The original
|
||||
references embedded in each context remain the authoritative direct evidence.
|
||||
The evidence artifact is discovered separately from lane payloads and does not
|
||||
increase the normalized-lane count reported by the runner or subprocess
|
||||
receipt.
|
||||
|
||||
## Failure And Publication Semantics
|
||||
|
||||
- Evidence projection occurs only after selected normalized outputs are known
|
||||
and before the output encoder returns its logical files.
|
||||
- Projection or encoding failure is an output-stage framework error; the CLI
|
||||
does not publish a partially assembled output bundle.
|
||||
- Rejected or absent lane outputs contribute nothing. Their attempted values
|
||||
and source references must not be published through this artifact.
|
||||
- Context generation is deterministic for the same source document, selected
|
||||
normalized outputs, lane allowlist, and window size.
|
||||
- Existing output, checkpoint, warning, rejection, debug, and subprocess
|
||||
success semantics remain unchanged.
|
||||
|
||||
## Sensitivity And Size
|
||||
|
||||
Unlike the current chunk map, the evidence artifact contains transcript text
|
||||
and source-unit metadata. Enabling it therefore creates additional durable
|
||||
sensitive data and may materially increase bundle size.
|
||||
|
||||
The implemented configuration, operations, integration, and consumer documents
|
||||
state that:
|
||||
|
||||
- evidence publication is opt-in;
|
||||
- output permissions and retention must be appropriate for source content;
|
||||
- selecting broad or numerous lanes can publish most of the transcript; and
|
||||
- the artifact must not contain raw input bytes, LLM prompts or responses,
|
||||
auxiliary reference content, credentials, debug-only data, or filesystem
|
||||
paths.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Evidence publication is disabled by default and leaves existing bundles
|
||||
unchanged.
|
||||
- Enabling it requires an explicit non-empty lane allowlist.
|
||||
- References from all selected successful lanes contribute to one deduplicated
|
||||
document.
|
||||
- Non-monotonic unit IDs are expanded and ordered correctly by document
|
||||
position.
|
||||
- Overlapping windows share one ordered copy of each included source unit.
|
||||
- Direct references remain distinguishable from added context.
|
||||
- Scene descriptions cannot contribute unless their lane is explicitly
|
||||
allowlisted.
|
||||
- Invalid selected lanes and unsupported artifact kinds fail before execution;
|
||||
invalid accepted references fail output projection rather than being ignored.
|
||||
- Empty selected-lane results produce a valid empty evidence artifact.
|
||||
- Existing D&D lane schemas, normalized-output counts, and source-reference
|
||||
semantics do not change.
|
||||
- Generic framework and output packages do not depend on D&D types or parse
|
||||
artifact JSON heuristically.
|
||||
- The published contract and operational documentation clearly describe source
|
||||
sensitivity, discovery, compatibility, and retention.
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
- Embedding transcript units directly into each D&D record or lane payload.
|
||||
- Replacing precise source references with expanded context ranges.
|
||||
- Automatically including every configured lane.
|
||||
- An explicit full-transcript publication mode.
|
||||
- LLM summarization, retrieval, ranking, or narrative generation.
|
||||
- Per-record window sizes or lane-specific window sizes.
|
||||
- CLI overrides for evidence configuration.
|
||||
- Reading rejected attempts, debug artifacts, auxiliary references, or prior
|
||||
output bundles as evidence sources.
|
||||
Reference in New Issue
Block a user