Add canonical source unit provenance
This commit is contained in:
@@ -52,9 +52,14 @@ Notarius identifies the parsed source in this order:
|
||||
2. `metadata.source_id`, when it is a non-empty string after trimming;
|
||||
3. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
|
||||
|
||||
The source digest recorded in output provenance is `sha256:<hex>` of the exact
|
||||
raw input bytes. Segment IDs become the unit IDs used by artifact source
|
||||
references.
|
||||
The exact raw input SHA-256 remains the basis of the fallback source ID. The
|
||||
source digest recorded in output provenance is instead the SHA-256 of the
|
||||
canonical generic source document, excluding the digest field itself. It covers
|
||||
the derived source identity, document kind and format, ordered units and their
|
||||
self-references, and accepted metadata. Segment IDs become the unit IDs used by
|
||||
artifact source references; each produced unit carries a self-reference whose
|
||||
source ID is the derived document ID and whose start and end IDs both equal the
|
||||
segment ID.
|
||||
|
||||
## Compatibility Limit
|
||||
|
||||
|
||||
@@ -40,9 +40,10 @@ evidence. The resolver and materializer behavior is described in
|
||||
### `internal/modules/seriatim/input/transcript`
|
||||
|
||||
The adapter decodes the supported transcript JSON, selects the source identity,
|
||||
computes the raw-input digest, validates segments, and maps each segment into a
|
||||
generic source unit with speaker and timestamp metadata. Its spec advertises the
|
||||
transcript capabilities consumed by D&D modules.
|
||||
computes canonical source provenance, validates segments, and maps each segment
|
||||
into a generic source unit with a self-reference plus speaker and timestamp
|
||||
metadata. Its spec advertises the transcript capabilities consumed by D&D
|
||||
modules.
|
||||
|
||||
Parsing is strict about required values and duplicate unit IDs but deliberately
|
||||
ignores unrelated Seriatim fields. The external format and derived-identity
|
||||
|
||||
@@ -74,6 +74,11 @@ selected input adapter. Later stage requests receive the generic source model;
|
||||
extract requests receive chunk-scoped input material, while chunk, merge, and
|
||||
normalize requests retain access to the original source material.
|
||||
|
||||
Source validation requires every unit to carry a canonical self-reference to
|
||||
its containing document and its own unit ID. Explicit clone, checkpoint, and
|
||||
debug boundaries retain that reference, and the canonical source digest covers
|
||||
it deterministically.
|
||||
|
||||
`pipeline.RunOutput` carries the run manifest, accepted normalized results,
|
||||
rejected results, warnings, checkpoint events, and logical files returned by the
|
||||
output encoder. The CLI owns diagnostics and durable filesystem writes after the
|
||||
|
||||
@@ -66,7 +66,8 @@ Framework stages operate on source documents, source units, and source
|
||||
references rather than format-specific structures. A source reference identifies
|
||||
an ordered range of generic source units. Framework code preserves those ranges
|
||||
and does not merge or rewrite them unless a stage module explicitly owns that
|
||||
behavior.
|
||||
behavior. Every source unit carries a validated self-reference to its containing
|
||||
document and its own unit ID.
|
||||
|
||||
Extract modules own artifact semantics, prompt use, response schemas, and
|
||||
domain interpretation. Domain-specific concepts remain in the relevant module,
|
||||
|
||||
Reference in New Issue
Block a user