Document raw pipeline completion
This commit is contained in:
@@ -15,7 +15,9 @@ CompleteStructured(ctx, request, out) (response, error)
|
||||
|
||||
The request contains prompt ID/version, profile ID, session ID, prompt input
|
||||
materials, and variables. The caller supplies a pointer target for decoded
|
||||
structured output.
|
||||
structured output. The response also carries the raw structured output bytes
|
||||
returned by the runtime so modules can preserve raw payloads in pipeline stage
|
||||
outputs.
|
||||
|
||||
Modules that call the LLM own their prompts, schemas, prompt IDs, validators,
|
||||
and domain-specific interpretation. Provider adapters should not contain
|
||||
@@ -59,6 +61,7 @@ Notarius prompt requests into Scriptorium `RunRequest` values. It:
|
||||
- lets Scriptorium render prompts, call the configured provider, and validate
|
||||
structured output;
|
||||
- unmarshals successful JSON into the caller-provided target;
|
||||
- returns the validated raw structured output bytes to the caller;
|
||||
- maps token usage and selected profile/model metadata into the Notarius
|
||||
response and manifest profile recorder.
|
||||
|
||||
|
||||
@@ -178,7 +178,7 @@ Response schema identity:
|
||||
The extractor adds prompt and response-schema provenance to lane manifest
|
||||
metadata under `artifact_lanes[].metadata.extractor`. Durable raw output
|
||||
details belong in the
|
||||
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
|
||||
[D&D spell raw output contract](../integrations/dnd-spell-artifacts.md).
|
||||
|
||||
The `dnd/scenes` chunker and `dnd/spells` extractor declare optional `players`,
|
||||
`party`, and `glossary` reference slots accepting UTF-8 plain text, Markdown,
|
||||
|
||||
@@ -121,6 +121,8 @@ The runner:
|
||||
chunk validators;
|
||||
6. runs each selected artifact lane in sorted resolved order;
|
||||
7. builds the output encoder and validates logical output file names.
|
||||
8. passes accepted normalized raw outputs, rejected output records, warnings,
|
||||
and the manifest to the output encoder.
|
||||
|
||||
## Chunk Results
|
||||
|
||||
@@ -179,8 +181,8 @@ Within an artifact lane, the runner:
|
||||
## Validators
|
||||
|
||||
The current runner handoff is raw-output based. Extractors, mergers, and
|
||||
normalizers do not advertise candidate validator chains through their module
|
||||
interfaces. Runner-side raw validation chains receive the raw module output plus
|
||||
normalizers do not advertise validator chains through their module interfaces.
|
||||
Runner-side raw validation chains receive the raw module output plus
|
||||
stage, lane, module, source, and chunk provenance. Empty raw validation chains
|
||||
approve output by default.
|
||||
|
||||
@@ -208,7 +210,12 @@ On successful execution, the manifest validation status is:
|
||||
|
||||
The manifest records run ID, pipeline ID, pipeline digest, module keys, top-level
|
||||
module metadata, artifact lanes, LLM profile metadata, source digest,
|
||||
reference provenance, validation status, and timing.
|
||||
reference provenance, normalized raw output summaries, rejected output
|
||||
summaries, validation status, and timing. Raw output summaries include lane ID,
|
||||
normalizer module key, media type, source ID, and response-schema provenance
|
||||
when present. Rejected output summaries include stage, lane, module, chunk,
|
||||
validator or reason, message, attempt count, and optional diagnostic artifact
|
||||
path. The manifest does not include raw output payload bytes.
|
||||
|
||||
Singleton pipeline modules may add non-secret metadata by implementing
|
||||
`contracts.ManifestMetadataProvider`. The runner records that metadata under
|
||||
@@ -218,3 +225,12 @@ Lane-owned modules may add non-secret metadata through
|
||||
`artifact_lanes[].metadata`. The runner records extractor, merger, and
|
||||
normalizer metadata there. The D&D spell extractor uses lane metadata for
|
||||
prompt and response-schema provenance.
|
||||
|
||||
## JSON Output
|
||||
|
||||
The production JSON output encoder writes `manifest.json`, `index.json`,
|
||||
`warnings.json`, `rejected.json`, and one pretty-printed JSON file per accepted
|
||||
normalized lane output under `lanes/`. It accepts only normalized outputs with
|
||||
valid `application/json` payloads. Unsupported media types, invalid JSON, unsafe
|
||||
logical paths, and duplicate sanitized lane file names fail the run before
|
||||
durable output files are written.
|
||||
|
||||
Reference in New Issue
Block a user