Serialize typed artifacts at durable boundaries
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
# D&D Spell Raw Output
|
||||
# D&D Spell Artifact
|
||||
|
||||
This document is the durable raw output contract for the production D&D spell
|
||||
extractor. Selectable extractor keys are cataloged in
|
||||
This document is the durable serialized artifact contract for the production
|
||||
D&D spell extractor. Selectable extractor keys are cataloged in
|
||||
[Configuration](../config.md#implemented-production-modules).
|
||||
|
||||
## Identity
|
||||
|
||||
@@ -15,7 +15,7 @@ The encoder writes:
|
||||
|
||||
- `index.json`
|
||||
- `manifest.json`
|
||||
- `lanes/<lane-id>.json`, one file per normalized raw lane output
|
||||
- `lanes/<lane-id>.json`, one file per normalized serialized artifact
|
||||
- `rejected.json`
|
||||
- `warnings.json`
|
||||
|
||||
@@ -112,8 +112,8 @@ Reference `stage` is `chunk`, `extract`, `merge`, or `normalize`. `lane_id` is
|
||||
omitted for chunk references and present for extract, merge, and normalize
|
||||
references.
|
||||
|
||||
`validation_status` is `approved` when no raw outputs were rejected and
|
||||
`rejected` when one or more raw outputs were rejected.
|
||||
`validation_status` is `approved` when no outputs were rejected and `rejected`
|
||||
when one or more outputs were rejected.
|
||||
|
||||
`validator_chains` records the resolved validator chain for each validation
|
||||
point. Entries include stage, lane ID when applicable, module key, and validators
|
||||
@@ -131,12 +131,13 @@ message, attempt count, and optional diagnostic artifact path.
|
||||
|
||||
## Output Payload Files
|
||||
|
||||
Each normalized raw output is written to `lanes/<sanitized-lane-id>.json`.
|
||||
The JSON output encoder accepts only `application/json` normalized outputs. The
|
||||
file contains the raw JSON payload pretty-printed.
|
||||
Each normalized serialized artifact is written to
|
||||
`lanes/<sanitized-lane-id>.json`. The JSON output encoder is domain-neutral and
|
||||
accepts only artifacts whose codec media type is `application/json`. The file
|
||||
contains the codec-owned JSON bytes pretty-printed.
|
||||
|
||||
The schema of each lane payload is owned by that artifact contract. For the
|
||||
current D&D spell lane, see [D&D Spell Raw Output](dnd-spell-artifacts.md).
|
||||
current D&D spell lane, see [D&D Spell Artifact](dnd-spell-artifacts.md).
|
||||
|
||||
## `rejected.json`
|
||||
|
||||
@@ -148,7 +149,7 @@ Shape:
|
||||
}
|
||||
```
|
||||
|
||||
When raw output validation rejects an output, each entry contains `stage` and
|
||||
When output validation rejects an output, each entry contains `stage` and
|
||||
`message`. It includes `lane_id`, `module_key`, `chunk_id`, `chunk_index`,
|
||||
`validator_name`, `reason_code`, `attempt_count`, and
|
||||
`diagnostic_artifact_path` when applicable.
|
||||
|
||||
@@ -127,7 +127,8 @@ debug boundaries retain that reference, and the canonical source digest covers
|
||||
it deterministically. Chunks use the same source model and carry one canonical
|
||||
reference spanning the first selected unit through the last.
|
||||
|
||||
`pipeline.RunOutput` carries the run manifest, accepted normalized results,
|
||||
`pipeline.RunOutput` carries the run manifest, accepted normalized serialized
|
||||
artifacts with lane and normalizer provenance,
|
||||
rejected results, warnings, checkpoint events, and logical files returned by the
|
||||
output encoder. The CLI owns diagnostics and durable filesystem writes after the
|
||||
runner returns.
|
||||
@@ -199,15 +200,20 @@ The runner depends on recorder and loader interfaces, using no-op
|
||||
implementations when collaborators are absent. Each checkpointed workflow
|
||||
boundary records a running, succeeded, or failed transition. Reuse decisions
|
||||
are consulted in workflow order and accepted payloads are cloned before
|
||||
entering the normal handoff path. The typed D&D executor uses explicit
|
||||
migration-only codec adapters to read and write the existing raw artifact
|
||||
checkpoint envelopes. Dependency fingerprints connect later checkpoints to the
|
||||
exact accepted results on which they depend.
|
||||
entering the normal handoff path. Typed extract, merge, and normalize
|
||||
checkpoints store codec bytes with artifact kind, schema ID and version, exact
|
||||
schema digest, and media type. Reuse compares that identity with the prepared
|
||||
codec and decodes through the codec; missing identity, mismatches, corrupt
|
||||
bytes, and decode failures become explicit reuse misses and execute the step
|
||||
normally. Dependency fingerprints and debug content digests use the same stable
|
||||
codec bytes that cross those boundaries.
|
||||
|
||||
Debug instrumentation wraps run, stage, attempt, validator, and structured LLM
|
||||
boundaries. Context scopes associate nested LLM calls with the module or
|
||||
validator attempt that made them. Debug-write failures are framework errors;
|
||||
debug data is never used as a checkpoint source.
|
||||
debug data is never used as a checkpoint source. Typed artifact debug envelopes
|
||||
are domain-neutral, redact sensitive metadata and bytes through the common
|
||||
debug policy, and record codec identity plus schema and content digests.
|
||||
|
||||
Checkpoint identity, physical layout, reuse behavior, and debug artifact
|
||||
handling are operator contracts in [Operations](../operations.md). Serialization
|
||||
@@ -224,7 +230,7 @@ Module metadata providers may add non-secret singleton or lane-scoped metadata.
|
||||
|
||||
Execution errors include stage, module, lane, or validator context. Once a
|
||||
manifest exists, a failing run returns it with failed status and completion
|
||||
time. Successful status reflects whether any raw result was rejected. The
|
||||
time. Successful status reflects whether any result was rejected. The
|
||||
durable manifest and logical file schemas are defined in the
|
||||
[JSON output contract](../integrations/json-output.md).
|
||||
|
||||
|
||||
@@ -91,6 +91,12 @@ digests match the current invocation. Changes to input bytes, the resolved
|
||||
pipeline, selected lanes, the runtime LLM profile override, or bound reference
|
||||
content invalidate reuse.
|
||||
|
||||
Typed artifact checkpoints additionally record codec-owned bytes, artifact
|
||||
kind, schema ID and version, exact schema digest, and media type. A missing or
|
||||
mismatched codec identity, or bytes the current codec cannot decode, is reported
|
||||
as a checkpoint reuse miss. The affected operation executes normally and, when
|
||||
checkpoint writing is enabled, replaces the incompatible checkpoint.
|
||||
|
||||
Current checkpoint manifests use workspace schema `notarius.workspace.v2`.
|
||||
Manifests written with `notarius.workspace.v1` are incompatible because their
|
||||
chunk provenance has an older shape. On the first explicit resume after an
|
||||
@@ -126,7 +132,9 @@ the attempt `llm_calls` array. Prompt content is written inline in the prompt
|
||||
artifact. The response metadata and body use the paired files described above;
|
||||
the body is pretty-printed JSON when possible and raw text otherwise. Debug
|
||||
artifacts may contain source material, reference material, prompt inputs, model
|
||||
outputs, and other sensitive data. API keys are not written, and obvious
|
||||
outputs, and other sensitive data. Typed artifact envelopes include
|
||||
domain-neutral codec identity, redacted metadata and content, and digests of
|
||||
the stable codec bytes. API keys are not written, and obvious
|
||||
credential-shaped values and sensitive map keys are redacted, but debug
|
||||
directories should still be protected as sensitive local state.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user