Document raw pipeline completion

This commit is contained in:
2026-07-07 19:32:15 +00:00
parent 7c95791e94
commit a9d8505cdb
7 changed files with 91 additions and 822 deletions

View File

@@ -15,7 +15,9 @@ CompleteStructured(ctx, request, out) (response, error)
The request contains prompt ID/version, profile ID, session ID, prompt input
materials, and variables. The caller supplies a pointer target for decoded
structured output.
structured output. The response also carries the raw structured output bytes
returned by the runtime so modules can preserve raw payloads in pipeline stage
outputs.
Modules that call the LLM own their prompts, schemas, prompt IDs, validators,
and domain-specific interpretation. Provider adapters should not contain
@@ -59,6 +61,7 @@ Notarius prompt requests into Scriptorium `RunRequest` values. It:
- lets Scriptorium render prompts, call the configured provider, and validate
structured output;
- unmarshals successful JSON into the caller-provided target;
- returns the validated raw structured output bytes to the caller;
- maps token usage and selected profile/model metadata into the Notarius
response and manifest profile recorder.

View File

@@ -178,7 +178,7 @@ Response schema identity:
The extractor adds prompt and response-schema provenance to lane manifest
metadata under `artifact_lanes[].metadata.extractor`. Durable raw output
details belong in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
[D&D spell raw output contract](../integrations/dnd-spell-artifacts.md).
The `dnd/scenes` chunker and `dnd/spells` extractor declare optional `players`,
`party`, and `glossary` reference slots accepting UTF-8 plain text, Markdown,

View File

@@ -121,6 +121,8 @@ The runner:
chunk validators;
6. runs each selected artifact lane in sorted resolved order;
7. builds the output encoder and validates logical output file names.
8. passes accepted normalized raw outputs, rejected output records, warnings,
and the manifest to the output encoder.
## Chunk Results
@@ -179,8 +181,8 @@ Within an artifact lane, the runner:
## Validators
The current runner handoff is raw-output based. Extractors, mergers, and
normalizers do not advertise candidate validator chains through their module
interfaces. Runner-side raw validation chains receive the raw module output plus
normalizers do not advertise validator chains through their module interfaces.
Runner-side raw validation chains receive the raw module output plus
stage, lane, module, source, and chunk provenance. Empty raw validation chains
approve output by default.
@@ -208,7 +210,12 @@ On successful execution, the manifest validation status is:
The manifest records run ID, pipeline ID, pipeline digest, module keys, top-level
module metadata, artifact lanes, LLM profile metadata, source digest,
reference provenance, validation status, and timing.
reference provenance, normalized raw output summaries, rejected output
summaries, validation status, and timing. Raw output summaries include lane ID,
normalizer module key, media type, source ID, and response-schema provenance
when present. Rejected output summaries include stage, lane, module, chunk,
validator or reason, message, attempt count, and optional diagnostic artifact
path. The manifest does not include raw output payload bytes.
Singleton pipeline modules may add non-secret metadata by implementing
`contracts.ManifestMetadataProvider`. The runner records that metadata under
@@ -218,3 +225,12 @@ Lane-owned modules may add non-secret metadata through
`artifact_lanes[].metadata`. The runner records extractor, merger, and
normalizer metadata there. The D&D spell extractor uses lane metadata for
prompt and response-schema provenance.
## JSON Output
The production JSON output encoder writes `manifest.json`, `index.json`,
`warnings.json`, `rejected.json`, and one pretty-printed JSON file per accepted
normalized lane output under `lanes/`. It accepts only normalized outputs with
valid `application/json` payloads. Unsupported media types, invalid JSON, unsafe
logical paths, and duplicate sanitized lane file names fail the run before
durable output files are written.