Serialize typed artifacts at durable boundaries

This commit is contained in:
2026-07-17 07:33:49 +00:00
parent 66de1a5520
commit 814fcdc6ba
23 changed files with 624 additions and 159 deletions

View File

@@ -1,7 +1,7 @@
# D&D Spell Raw Output
# D&D Spell Artifact
This document is the durable raw output contract for the production D&D spell
extractor. Selectable extractor keys are cataloged in
This document is the durable serialized artifact contract for the production
D&D spell extractor. Selectable extractor keys are cataloged in
[Configuration](../config.md#implemented-production-modules).
## Identity

View File

@@ -15,7 +15,7 @@ The encoder writes:
- `index.json`
- `manifest.json`
- `lanes/<lane-id>.json`, one file per normalized raw lane output
- `lanes/<lane-id>.json`, one file per normalized serialized artifact
- `rejected.json`
- `warnings.json`
@@ -112,8 +112,8 @@ Reference `stage` is `chunk`, `extract`, `merge`, or `normalize`. `lane_id` is
omitted for chunk references and present for extract, merge, and normalize
references.
`validation_status` is `approved` when no raw outputs were rejected and
`rejected` when one or more raw outputs were rejected.
`validation_status` is `approved` when no outputs were rejected and `rejected`
when one or more outputs were rejected.
`validator_chains` records the resolved validator chain for each validation
point. Entries include stage, lane ID when applicable, module key, and validators
@@ -131,12 +131,13 @@ message, attempt count, and optional diagnostic artifact path.
## Output Payload Files
Each normalized raw output is written to `lanes/<sanitized-lane-id>.json`.
The JSON output encoder accepts only `application/json` normalized outputs. The
file contains the raw JSON payload pretty-printed.
Each normalized serialized artifact is written to
`lanes/<sanitized-lane-id>.json`. The JSON output encoder is domain-neutral and
accepts only artifacts whose codec media type is `application/json`. The file
contains the codec-owned JSON bytes pretty-printed.
The schema of each lane payload is owned by that artifact contract. For the
current D&D spell lane, see [D&D Spell Raw Output](dnd-spell-artifacts.md).
current D&D spell lane, see [D&D Spell Artifact](dnd-spell-artifacts.md).
## `rejected.json`
@@ -148,7 +149,7 @@ Shape:
}
```
When raw output validation rejects an output, each entry contains `stage` and
When output validation rejects an output, each entry contains `stage` and
`message`. It includes `lane_id`, `module_key`, `chunk_id`, `chunk_index`,
`validator_name`, `reason_code`, `attempt_count`, and
`diagnostic_artifact_path` when applicable.

View File

@@ -127,7 +127,8 @@ debug boundaries retain that reference, and the canonical source digest covers
it deterministically. Chunks use the same source model and carry one canonical
reference spanning the first selected unit through the last.
`pipeline.RunOutput` carries the run manifest, accepted normalized results,
`pipeline.RunOutput` carries the run manifest, accepted normalized serialized
artifacts with lane and normalizer provenance,
rejected results, warnings, checkpoint events, and logical files returned by the
output encoder. The CLI owns diagnostics and durable filesystem writes after the
runner returns.
@@ -199,15 +200,20 @@ The runner depends on recorder and loader interfaces, using no-op
implementations when collaborators are absent. Each checkpointed workflow
boundary records a running, succeeded, or failed transition. Reuse decisions
are consulted in workflow order and accepted payloads are cloned before
entering the normal handoff path. The typed D&D executor uses explicit
migration-only codec adapters to read and write the existing raw artifact
checkpoint envelopes. Dependency fingerprints connect later checkpoints to the
exact accepted results on which they depend.
entering the normal handoff path. Typed extract, merge, and normalize
checkpoints store codec bytes with artifact kind, schema ID and version, exact
schema digest, and media type. Reuse compares that identity with the prepared
codec and decodes through the codec; missing identity, mismatches, corrupt
bytes, and decode failures become explicit reuse misses and execute the step
normally. Dependency fingerprints and debug content digests use the same stable
codec bytes that cross those boundaries.
Debug instrumentation wraps run, stage, attempt, validator, and structured LLM
boundaries. Context scopes associate nested LLM calls with the module or
validator attempt that made them. Debug-write failures are framework errors;
debug data is never used as a checkpoint source.
debug data is never used as a checkpoint source. Typed artifact debug envelopes
are domain-neutral, redact sensitive metadata and bytes through the common
debug policy, and record codec identity plus schema and content digests.
Checkpoint identity, physical layout, reuse behavior, and debug artifact
handling are operator contracts in [Operations](../operations.md). Serialization
@@ -224,7 +230,7 @@ Module metadata providers may add non-secret singleton or lane-scoped metadata.
Execution errors include stage, module, lane, or validator context. Once a
manifest exists, a failing run returns it with failed status and completion
time. Successful status reflects whether any raw result was rejected. The
time. Successful status reflects whether any result was rejected. The
durable manifest and logical file schemas are defined in the
[JSON output contract](../integrations/json-output.md).

View File

@@ -91,6 +91,12 @@ digests match the current invocation. Changes to input bytes, the resolved
pipeline, selected lanes, the runtime LLM profile override, or bound reference
content invalidate reuse.
Typed artifact checkpoints additionally record codec-owned bytes, artifact
kind, schema ID and version, exact schema digest, and media type. A missing or
mismatched codec identity, or bytes the current codec cannot decode, is reported
as a checkpoint reuse miss. The affected operation executes normally and, when
checkpoint writing is enabled, replaces the incompatible checkpoint.
Current checkpoint manifests use workspace schema `notarius.workspace.v2`.
Manifests written with `notarius.workspace.v1` are incompatible because their
chunk provenance has an older shape. On the first explicit resume after an
@@ -126,7 +132,9 @@ the attempt `llm_calls` array. Prompt content is written inline in the prompt
artifact. The response metadata and body use the paired files described above;
the body is pretty-printed JSON when possible and raw text otherwise. Debug
artifacts may contain source material, reference material, prompt inputs, model
outputs, and other sensitive data. API keys are not written, and obvious
outputs, and other sensitive data. Typed artifact envelopes include
domain-neutral codec identity, redacted metadata and content, and digests of
the stable codec bytes. API keys are not written, and obvious
credential-shaped values and sensitive map keys are redacted, but debug
directories should still be protected as sensitive local state.