Implement raw module output contracts

This commit is contained in:
2026-07-07 18:58:23 +00:00
parent 9e3f8809b3
commit c05ecb58d8
33 changed files with 1166 additions and 1780 deletions

View File

@@ -144,11 +144,10 @@ text, or secrets.
Package: `internal/modules/extract/dnd/spells`
The `dnd/spells` extractor owns D&D spell-cast artifact semantics. It supplies
the embedded Scriptorium prompt ID, prompt version, transcript and reference
input materials, response schema, and session ID to the runtime; converts
spell-cast responses into artifact candidates; and supplies deterministic
validators.
The `dnd/spells` extractor owns D&D spell-cast extraction semantics. It
supplies the embedded Scriptorium prompt ID, prompt version, transcript and
reference input materials, response schema, and session ID to the runtime; then
returns the structured LLM `spell_casts` response as raw JSON.
Its LLM-facing source-reference schema uses integer `start_unit_id` and
`end_unit_id` values matching source-unit IDs.
@@ -166,13 +165,14 @@ Provides:
- `dnd.spell_casts`
Artifact type and schema version:
Response schema identity:
- artifact type: `dnd.spell_cast`
- schema ID: `notarius.dnd.spells`
- schema name: `notarius_dnd_spells_v1`
- schema version: `v1`
The extractor adds prompt and response-schema provenance to lane manifest
metadata under `artifact_lanes[].metadata.extractor`. Durable artifact payload
metadata under `artifact_lanes[].metadata.extractor`. Durable raw output
details belong in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
@@ -182,32 +182,13 @@ YAML, or JSON. They also accept `roster` as a deprecated compatibility alias for
`party`. Their prompts frame references as supporting disambiguation material
only; spell-cast artifacts must still be grounded in the source transcript.
## D&D Spell Validators
The spell extractor returns two built-in validators:
- `dnd/spells/shape`: rejects malformed payloads and missing required fields.
- `dnd/spells/source_refs`: rejects candidates without valid source references.
It also emits a warning when the extracted spell name is not found in the
cited source text.
Reason codes include:
- `invalid_payload`
- `missing_required_field`
- `missing_source_ref`
- `invalid_source_ref`
- `spell_not_near_source`
These validators are supplied by the extractor when no validators are configured
for the lane.
## `appendorder` Merger
Package: `internal/modules/merge/appendorder`
The `appendorder` merger clones and appends candidates in chunk order. It does
not deduplicate or reconcile candidates.
The `appendorder` merger preserves chunk order for raw extract outputs. A
single extract output is passed through as the merge output. Multiple extract
outputs are wrapped in one JSON payload under `outputs`.
Provides:
@@ -217,7 +198,7 @@ Provides:
Package: `internal/modules/normalize/noop`
The `noop` normalizer clones merged candidates and returns them unchanged.
The `noop` normalizer clones the raw merge output and returns it unchanged.
Requires:
@@ -231,9 +212,9 @@ Provides:
Package: `internal/modules/output/json`
The `json` output encoder converts approved artifacts, rejected artifacts,
warnings, and the run manifest into logical JSON output files. It groups
approved artifacts by artifact type and sanitizes artifact-type file names.
The `json` output encoder converts normalized raw outputs, rejected raw outputs,
warnings, and the run manifest into logical JSON output files. It writes one
payload file per lane under `outputs/` and sanitizes lane IDs for file names.
Requires:
@@ -261,7 +242,7 @@ When adding a module, keep source-format and extraction-domain boundaries clear:
- input modules may know external source formats;
- extract modules may know artifact semantics and prompt/schema assets;
- merge and normalize modules own candidate combination and reconciliation;
- merge and normalize modules own raw output combination and reconciliation;
- output modules own serialization, not diagnostics or CLI reporting.
Update [Development](../policy/development.md), [Configuration](../config.md),

View File

@@ -21,8 +21,8 @@ belongs in modules, not in command handlers.
## Core Packages
- `internal/core/artifacts`: artifact candidates, approved artifacts, rejected
artifacts, validation decisions, and run manifests.
- `internal/core/artifacts`: run manifests and legacy artifact serialization
shapes retained while pipeline handoff contracts use raw outputs.
- `internal/core/config`: defaults, YAML config parsing, environment overrides,
validation, redaction, and resolved pipeline config.
- `internal/core/diagnostics`: per-run diagnostics directory creation,

View File

@@ -102,8 +102,8 @@ Capability checks prevent incompatible pipeline composition before a run starts.
`pipeline.RunOutput` carries:
- run manifest;
- approved artifacts;
- rejected artifacts;
- normalized raw outputs;
- rejected raw outputs;
- warnings;
- logical output files returned by the output encoder.
@@ -159,33 +159,24 @@ Within an artifact lane, the runner:
1. builds the extractor, merger, and normalizer;
2. records module manifest metadata when modules provide it;
3. extracts candidates from each chunk;
4. normalizes candidate envelope fields such as index, extractor key, artifact
type, and schema version;
5. merges candidates;
6. normalizes merged candidates;
7. validates candidate envelope consistency;
8. runs validators;
9. converts approved candidates to artifacts.
3. extracts one raw `ExtractOutput` from each chunk;
4. fills runner-owned provenance on each extract output, including lane ID,
extractor key, source ID, chunk ID, and chunk index;
5. merges ordered extract outputs into one raw `MergeOutput`;
6. normalizes the merge output into one raw `NormalizeOutput`;
7. appends the normalized raw output to `RunOutput.NormalizeOutputs`.
## Validators
If a lane declares validators in config, the runner builds those validators from
the validator registry. Otherwise it uses validators returned by the extractor.
Each validator must return exactly one decision for each eligible candidate. The
runner enforces decision cardinality with `internal/framework/validate`.
Rejected candidates are removed before the next validator runs. Approved
candidates continue through the chain.
The production CLI currently registers no standalone validator modules. The
current D&D spell extractor supplies deterministic shape and source-reference
validators.
The current runner handoff is raw-output based. Extractors, mergers, and
normalizers do not advertise candidate validator chains through their module
interfaces. `RunOutput.Rejected` is reserved for rejected raw outputs when raw
validation is wired into the runner.
## Warnings And Failures
Warnings from chunking, extraction, merging, normalization, validation, and
output encoding are accumulated in `RunOutput.Warnings`.
Warnings from chunking, extraction, merging, normalization, and output encoding
are accumulated in `RunOutput.Warnings`.
Errors wrap the operation and module key or lane context. If execution fails
after a manifest exists, the returned manifest is marked `failed` and receives a
@@ -193,8 +184,8 @@ completion timestamp.
On successful execution, the manifest validation status is:
- `approved` when no candidates were rejected;
- `rejected` when at least one candidate was rejected.
- `approved` when no raw outputs were rejected;
- `rejected` when at least one raw output was rejected.
## Manifest Population