Add canonical source unit provenance

This commit is contained in:
2026-07-17 05:19:57 +00:00
parent 15c369c509
commit 40709e4ad8
26 changed files with 254 additions and 34 deletions

View File

@@ -52,9 +52,14 @@ Notarius identifies the parsed source in this order:
2. `metadata.source_id`, when it is a non-empty string after trimming;
3. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
The source digest recorded in output provenance is `sha256:<hex>` of the exact
raw input bytes. Segment IDs become the unit IDs used by artifact source
references.
The exact raw input SHA-256 remains the basis of the fallback source ID. The
source digest recorded in output provenance is instead the SHA-256 of the
canonical generic source document, excluding the digest field itself. It covers
the derived source identity, document kind and format, ordered units and their
self-references, and accepted metadata. Segment IDs become the unit IDs used by
artifact source references; each produced unit carries a self-reference whose
source ID is the derived document ID and whose start and end IDs both equal the
segment ID.
## Compatibility Limit

View File

@@ -40,9 +40,10 @@ evidence. The resolver and materializer behavior is described in
### `internal/modules/seriatim/input/transcript`
The adapter decodes the supported transcript JSON, selects the source identity,
computes the raw-input digest, validates segments, and maps each segment into a
generic source unit with speaker and timestamp metadata. Its spec advertises the
transcript capabilities consumed by D&D modules.
computes canonical source provenance, validates segments, and maps each segment
into a generic source unit with a self-reference plus speaker and timestamp
metadata. Its spec advertises the transcript capabilities consumed by D&D
modules.
Parsing is strict about required values and duplicate unit IDs but deliberately
ignores unrelated Seriatim fields. The external format and derived-identity

View File

@@ -74,6 +74,11 @@ selected input adapter. Later stage requests receive the generic source model;
extract requests receive chunk-scoped input material, while chunk, merge, and
normalize requests retain access to the original source material.
Source validation requires every unit to carry a canonical self-reference to
its containing document and its own unit ID. Explicit clone, checkpoint, and
debug boundaries retain that reference, and the canonical source digest covers
it deterministically.
`pipeline.RunOutput` carries the run manifest, accepted normalized results,
rejected results, warnings, checkpoint events, and logical files returned by the
output encoder. The CLI owns diagnostics and durable filesystem writes after the

View File

@@ -66,7 +66,8 @@ Framework stages operate on source documents, source units, and source
references rather than format-specific structures. A source reference identifies
an ordered range of generic source units. Framework code preserves those ranges
and does not merge or rewrite them unless a stage module explicitly owns that
behavior.
behavior. Every source unit carries a validated self-reference to its containing
document and its own unit ID.
Extract modules own artifact semantics, prompt use, response schemas, and
domain interpretation. Domain-specific concepts remain in the relevant module,