Finish Scriptorium runtime documentation
This commit is contained in:
@@ -21,13 +21,18 @@ Modules that call the LLM own their prompts, schemas, prompt IDs, validators,
|
||||
and domain-specific interpretation. Provider adapters should not contain
|
||||
domain-specific prompt logic.
|
||||
|
||||
Prompt input materials carry source or reference bytes with optional origin
|
||||
metadata. The Scriptorium-backed runtime receives them as named artifacts rather
|
||||
than rendered prompt strings owned by Notarius modules.
|
||||
|
||||
## Production Client Construction
|
||||
|
||||
`internal/cli` builds the production LLM client from the effective config:
|
||||
|
||||
1. collect production Scriptorium prompt and schema assets from module packages;
|
||||
2. create a Scriptorium-backed structured client using effective Scriptorium
|
||||
profile source settings;
|
||||
profile source settings from `scriptorium.profile_dir` or
|
||||
`scriptorium.profile_file`;
|
||||
3. create a scheduler from global LLM concurrency;
|
||||
4. wrap the client with `NewScheduledClient`;
|
||||
5. let the runtime report non-secret profile manifest metadata after calls.
|
||||
@@ -42,8 +47,10 @@ profile ID before pipeline execution.
|
||||
Notarius prompt requests into Scriptorium `RunRequest` values. It:
|
||||
|
||||
- validates the caller output target and prompt ID;
|
||||
- converts `LLMInputMaterial` values into inline Scriptorium artifacts;
|
||||
- passes `session_id` through Scriptorium variables when present;
|
||||
- converts `LLMInputMaterial` values into inline Scriptorium artifacts, using a
|
||||
single space for empty material so optional blank references remain explicit;
|
||||
- passes `session_id` through Scriptorium variables and request metadata when
|
||||
present;
|
||||
- sends explicit profile IDs only when the request supplies one;
|
||||
- lets Scriptorium render prompts, call the configured provider, and validate
|
||||
structured output;
|
||||
@@ -53,7 +60,9 @@ Notarius prompt requests into Scriptorium `RunRequest` values. It:
|
||||
|
||||
Generated-output validation failures are returned as Notarius errors. Provider
|
||||
and runtime errors are wrapped with prompt context and bearer tokens are
|
||||
redacted from error strings.
|
||||
redacted from error strings. Prompt text, raw source input, reference content,
|
||||
schema JSON, API keys, and bearer tokens are not added to default diagnostics or
|
||||
run manifests.
|
||||
|
||||
## Scheduler
|
||||
|
||||
|
||||
@@ -40,12 +40,20 @@ uses references, pass them as prompt input materials through the structured LLM
|
||||
request. Prompt metadata hashes remain based on prompt asset source, not
|
||||
rendered reference bytes.
|
||||
|
||||
Chunk modules receive the structured LLM client through `contracts.ChunkRequest`
|
||||
when they need model-backed chunking. The pipeline runner validates generic
|
||||
chunk result invariants before extraction; module-owned policies may be stricter
|
||||
but must stay within the module package.
|
||||
LLM-backed modules own Scriptorium prompt definitions and response schemas in
|
||||
their embedded assets. Module contracts should expose prompt IDs, versions,
|
||||
input material names, and non-secret prompt/schema hashes through manifest
|
||||
metadata; they should not expose Scriptorium public types through chunk,
|
||||
extract, or normalize contracts.
|
||||
|
||||
Normalize modules receive the structured LLM client through
|
||||
Chunk modules receive the structured LLM client, configured Scriptorium profile
|
||||
ID, prompt session ID, and raw source input material through
|
||||
`contracts.ChunkRequest` when they need model-backed chunking. The pipeline
|
||||
runner validates generic chunk result invariants before extraction; module-owned
|
||||
policies may be stricter but must stay within the module package.
|
||||
|
||||
Normalize modules receive the structured LLM client, configured Scriptorium
|
||||
profile ID, prompt session ID, and reference material through
|
||||
`contracts.NormalizeRequest` when they need model-backed reconciliation.
|
||||
|
||||
## `seriatim` Input
|
||||
@@ -91,9 +99,10 @@ Provides:
|
||||
Package: `internal/modules/chunk/dnd/scenes`
|
||||
|
||||
The `dnd/scenes` chunker uses the structured LLM client to divide transcript
|
||||
source units into coherent D&D scenes. It renders embedded prompts, loads the
|
||||
embedded structured response schema, validates model-authored source-unit
|
||||
boundaries, and converts each scene into a deterministic source chunk.
|
||||
source units into coherent D&D scenes. It supplies the embedded Scriptorium
|
||||
prompt ID, prompt version, transcript input material, response schema, and
|
||||
session ID to the runtime; validates model-authored source-unit boundaries; and
|
||||
converts each scene into a deterministic source chunk.
|
||||
|
||||
Requires:
|
||||
|
||||
@@ -123,10 +132,11 @@ text, or secrets.
|
||||
|
||||
Package: `internal/modules/extract/dnd/spells`
|
||||
|
||||
The `dnd/spells` extractor owns D&D spell-cast artifact semantics. It renders
|
||||
embedded prompts, loads the embedded structured response schema, calls the
|
||||
structured LLM client, converts spell-cast responses into artifact candidates,
|
||||
and supplies deterministic validators.
|
||||
The `dnd/spells` extractor owns D&D spell-cast artifact semantics. It supplies
|
||||
the embedded Scriptorium prompt ID, prompt version, transcript and reference
|
||||
input materials, response schema, and session ID to the runtime; converts
|
||||
spell-cast responses into artifact candidates; and supplies deterministic
|
||||
validators.
|
||||
|
||||
Requires:
|
||||
|
||||
|
||||
@@ -23,7 +23,8 @@ before execution:
|
||||
- merge: `appendorder`
|
||||
- normalize: `noop`
|
||||
- output: `json`
|
||||
- LLM profile: `default`
|
||||
- LLM profile: empty, which lets Scriptorium prompt defaults choose a
|
||||
profile.
|
||||
4. The module catalog is checked for each bound module key.
|
||||
5. Module capabilities are checked in workflow order.
|
||||
6. A digest is calculated from the resolved pipeline without the digest field.
|
||||
@@ -54,13 +55,19 @@ empty bound files. Media-type acceptance is checked only when a slot declares
|
||||
manifests. The CLI writes provenance-only resolved reference diagnostics, and
|
||||
the run manifest records target-stage reference provenance separately from
|
||||
source digests. Runtime reference content is passed to the matching chunker,
|
||||
extractor, or normalizer request.
|
||||
extractor, or normalizer request. LLM-backed modules pass that material onward
|
||||
as named Scriptorium prompt inputs.
|
||||
|
||||
Prompt bundles can declare reference slots and use `reference` and
|
||||
`hasreference` template functions. Bundle loading validates string-literal slot
|
||||
names against the declaration. Rendering receives a target reference set from the
|
||||
caller; unbound optional slots render as empty strings, and `hasreference`
|
||||
returns true only when at least one bound item has content.
|
||||
The CLI carries raw input bytes into `pipeline.RunInput`. Input adapters parse
|
||||
those bytes into the source document, while LLM-backed modules that need the
|
||||
original transcript material can pass the same bytes as a prompt input with
|
||||
origin metadata. The raw input payload is not written to manifests or default
|
||||
diagnostics.
|
||||
|
||||
The CLI also carries an optional run `session_id`. The runner makes it available
|
||||
to chunk, extract, and normalize requests; LLM-backed modules forward it through
|
||||
their structured completion requests so Scriptorium can include it in prompt
|
||||
execution metadata.
|
||||
|
||||
## Registries And Module Specs
|
||||
|
||||
|
||||
Reference in New Issue
Block a user