Compare commits
3 Commits
6db2dc8d2a
...
3b07b64a0f
| Author | SHA1 | Date | |
|---|---|---|---|
| 3b07b64a0f | |||
| 6e6375521d | |||
| b1fe9dc5a7 |
@@ -14,9 +14,9 @@ go run ./cmd/notarius run dnd-session \
|
||||
--input examples/seriatim-minimal-transcript.json
|
||||
```
|
||||
|
||||
The maintained example uses Scriptorium's built-in `mistral-small-3` profile,
|
||||
which reads `OPENROUTER_API_KEY`. Outputs are written under
|
||||
`./notarius-output/<run-id>/` unless `--output-dir` is provided.
|
||||
This invocation uses the maintained example configuration and input. See the
|
||||
configuration and operations references for profile selection, credentials, and
|
||||
run artifacts.
|
||||
|
||||
Useful references:
|
||||
|
||||
|
||||
120
docs/cli.md
120
docs/cli.md
@@ -3,27 +3,15 @@
|
||||
This is the canonical reference for the implemented Notarius command-line
|
||||
interface.
|
||||
|
||||
## Quick Run
|
||||
|
||||
```sh
|
||||
OPENROUTER_API_KEY=... \
|
||||
go run ./cmd/notarius run dnd-session \
|
||||
--config examples/dnd-spells.config.yml \
|
||||
--input examples/seriatim-minimal-transcript.json
|
||||
```
|
||||
|
||||
The maintained example uses prompt defaults and Scriptorium's built-in
|
||||
`mistral-small-3` profile, which reads `OPENROUTER_API_KEY`. To use another
|
||||
endpoint or model, configure a Scriptorium profile source and select its profile
|
||||
ID in config or with `--llm-profile`.
|
||||
For the minimal end-to-end invocation, see the [README](../README.md).
|
||||
|
||||
## Commands
|
||||
|
||||
```text
|
||||
notarius help
|
||||
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--resume] [--session-id id] [--reference selector=path] [--without-reference selector]
|
||||
notarius config validate --config path/to/config.yml [--pipeline pipeline-id] [--only lane-a,lane-b]
|
||||
notarius pipelines list --config path/to/config.yml [--json]
|
||||
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--output-dir path] [--diagnostics-dir path] [--llm-profile id] [--resume] [--session-id id] [--reference selector=path] [--without-reference selector]
|
||||
notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b]
|
||||
notarius pipelines list [--config path/to/config.yml] [--json]
|
||||
```
|
||||
|
||||
Running `notarius` with no arguments, `notarius help`, `notarius --help`, or
|
||||
@@ -37,20 +25,18 @@ file.
|
||||
Flags:
|
||||
|
||||
- `--input path`: required source input file.
|
||||
- `--config path`: config file path. If omitted, Notarius checks
|
||||
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
|
||||
- `--config path`: config file path. If omitted, Notarius uses the discovery
|
||||
rules in [Configuration](config.md#discovery).
|
||||
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are
|
||||
comma-separated and must be non-empty.
|
||||
- `--resume`: reuse valid workspace checkpoints for this invocation. Requires
|
||||
an effective workspace directory and `workspace.resume.enabled: true`.
|
||||
- `--output-dir path`: output root. The run writes to `<path>/<run-id>/`.
|
||||
Defaults to `./notarius-output`.
|
||||
- `--resume`: request checkpoint reuse for this invocation. See
|
||||
[Operations](operations.md#checkpoints) for prerequisites and reuse behavior.
|
||||
- `--output-dir path`: output root. Defaults to `./notarius-output`.
|
||||
- `--diagnostics-dir path`: diagnostics work directory override for this
|
||||
invocation. It does not change the workspace directory.
|
||||
- `--llm-profile id`: override every effective LLM-capable pipeline module
|
||||
binding to use one Scriptorium profile ID. Validator-specific profiles are
|
||||
not overridden. Configured LLM-backed validators with explicit profiles are
|
||||
validated against the configured Scriptorium profile source.
|
||||
binding with one Scriptorium profile ID. Validator-specific profiles are not
|
||||
overridden.
|
||||
- `--session-id id`: pass a stable prompt session identifier through LLM-backed
|
||||
module calls.
|
||||
- `--reference selector=path`: bind a reference path to a chunk, extractor,
|
||||
@@ -63,23 +49,11 @@ On success, the command prints the completed pipeline ID, normalized output and
|
||||
rejected output counts, and the output directory. If the run completes with warnings,
|
||||
the warning count is printed to stderr.
|
||||
|
||||
Reference flags are resolved against selected chunk, extractor, merger, and normalizer
|
||||
targets before the run starts. Flat slot names are accepted only when exactly
|
||||
one selected target declares that slot. Bound reference files are read before
|
||||
pipeline work starts, validated as UTF-8 text, and recorded as provenance for
|
||||
the target that declares the slot. Runtime reference content is passed to the
|
||||
chunker, extractor, merger, or normalizer target that declares the slot. Notarius infers
|
||||
reference media types from file extensions for provenance and for optional slot
|
||||
checks. Reference content is not written to diagnostics, logs, errors, or
|
||||
manifests.
|
||||
|
||||
Reference binding precedence is:
|
||||
|
||||
1. pipeline-level config `references`;
|
||||
2. target-local config references, including legacy lane-level extractor
|
||||
`references`;
|
||||
3. `--reference` run flags;
|
||||
4. `--without-reference` run flags.
|
||||
Reference flags are resolved against selected chunk, extractor, merger, and
|
||||
normalizer targets before the run starts. Flat slot names are accepted only
|
||||
when exactly one selected target declares that slot. For configured reference
|
||||
bindings, precedence, path resolution, and validation, see
|
||||
[Configuration](config.md#pipelines).
|
||||
|
||||
`--reference` binds or replaces one slot for one selected target. Selectors are:
|
||||
|
||||
@@ -142,8 +116,7 @@ go run ./cmd/notarius run dnd-session \
|
||||
--session-id campaign-17-session-04
|
||||
```
|
||||
|
||||
Use `--resume` to reuse valid checkpoints from a previous compatible
|
||||
invocation:
|
||||
The resume flag can be added to an otherwise identical run invocation:
|
||||
|
||||
```sh
|
||||
go run ./cmd/notarius run dnd-session \
|
||||
@@ -152,13 +125,8 @@ go run ./cmd/notarius run dnd-session \
|
||||
--resume
|
||||
```
|
||||
|
||||
Plain `run` does not skip completed work. It executes the pipeline normally and
|
||||
refreshes checkpoints when checkpointing is enabled. `--resume` verifies each
|
||||
checkpoint before reuse and executes any missing, corrupt, or incompatible step
|
||||
normally.
|
||||
|
||||
For durable output, diagnostics, retention, and failure inspection, see
|
||||
[Operations](operations.md).
|
||||
For checkpoint behavior, durable output, diagnostics, retention, and failure
|
||||
inspection, see [Operations](operations.md).
|
||||
|
||||
## `config validate`
|
||||
|
||||
@@ -166,8 +134,8 @@ For durable output, diagnostics, retention, and failure inspection, see
|
||||
|
||||
Flags:
|
||||
|
||||
- `--config path`: config file path. If omitted, discovery uses
|
||||
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
|
||||
- `--config path`: config file path. If omitted, Notarius uses the discovery
|
||||
rules in [Configuration](config.md#discovery).
|
||||
- `--pipeline pipeline-id`: additionally resolve one configured pipeline against
|
||||
the production module catalog.
|
||||
- `--only lane-a,lane-b`: validate resolution for selected artifact lanes. This
|
||||
@@ -191,8 +159,8 @@ go run ./cmd/notarius config validate \
|
||||
|
||||
Flags:
|
||||
|
||||
- `--config path`: config file path. If omitted, discovery uses
|
||||
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
|
||||
- `--config path`: config file path. If omitted, Notarius uses the discovery
|
||||
rules in [Configuration](config.md#discovery).
|
||||
- `--json`: print `{"pipelines":[...]}` instead of one ID per line.
|
||||
|
||||
Examples:
|
||||
@@ -215,42 +183,6 @@ go run ./cmd/notarius pipelines list \
|
||||
- `2`: command syntax was invalid, a command was unknown, a required argument
|
||||
was missing, or a flag value was malformed.
|
||||
|
||||
## Implemented Production Pipeline Modules
|
||||
|
||||
The production CLI currently registers these module keys:
|
||||
|
||||
- input: `seriatim`
|
||||
- chunk: `generic`, `dnd/scenes`
|
||||
- extract: `dnd/spells`
|
||||
- merge: `appendorder`
|
||||
- normalize: `noop`
|
||||
- output: `json`
|
||||
|
||||
## Implemented Production Validators
|
||||
|
||||
The production CLI currently registers these validator keys:
|
||||
|
||||
- `generic/always_accept`
|
||||
- `generic/always_reject`
|
||||
- `generic/valid_json`
|
||||
- `generic/valid_json_schema`
|
||||
- `extract/dnd/spells/shape`
|
||||
- `extract/dnd/spells/source_refs`
|
||||
- `extract/dnd/spells/source_relatedness`
|
||||
|
||||
The production default chain for the `dnd/spells` extractor is:
|
||||
|
||||
1. `generic/valid_json`
|
||||
2. `generic/valid_json_schema`
|
||||
3. `extract/dnd/spells/shape`
|
||||
4. `extract/dnd/spells/source_refs`
|
||||
5. `extract/dnd/spells/source_relatedness`
|
||||
|
||||
Validator chain overrides are configured on `chunk`, lane `extract`, lane
|
||||
`merge`, and lane `normalize` bindings. Omitted overrides use production
|
||||
defaults, `validators: []` disables validation for that binding, and non-empty
|
||||
lists replace the default chain in configured order. Validator keys are resolved
|
||||
against the registered validator catalog.
|
||||
|
||||
For YAML structure, Scriptorium profile sources, environment overrides, and
|
||||
module binding syntax, see [Configuration](config.md).
|
||||
For YAML structure, defaults, Scriptorium profile sources, environment
|
||||
overrides, and selectable module and validator keys, see
|
||||
[Configuration](config.md).
|
||||
|
||||
161
docs/config.md
161
docs/config.md
@@ -7,34 +7,22 @@ built-in defaults, then environment overrides are applied.
|
||||
|
||||
## Discovery
|
||||
|
||||
Commands that accept `--config` load configuration in this order:
|
||||
Commands that load configuration use this order:
|
||||
|
||||
1. the `--config` path, when provided;
|
||||
1. an explicit path supplied through the CLI, when provided;
|
||||
2. `NOTARIUS_CONFIG`, when set to a non-empty path;
|
||||
3. `/usr/local/etc/notarius/config.yml`.
|
||||
|
||||
If none is available, the command fails with a config file not found error.
|
||||
The explicit-path option is defined in the [CLI reference](cli.md).
|
||||
|
||||
## Minimal Example
|
||||
## Maintained Examples
|
||||
|
||||
```yaml
|
||||
version: 2
|
||||
pipelines:
|
||||
dnd-session:
|
||||
input: seriatim
|
||||
references:
|
||||
party: ./dnd-spells-roster.txt
|
||||
glossary: ./dnd-spells-glossary.txt
|
||||
chunk:
|
||||
module: generic
|
||||
options:
|
||||
max_units: 50
|
||||
artifacts:
|
||||
spells:
|
||||
extract: dnd/spells
|
||||
```
|
||||
- [Minimal D&D spell configuration](../examples/dnd-spells.config.yml)
|
||||
- [Production-oriented D&D spell configuration](../examples/dnd-spells-production.config.yml)
|
||||
|
||||
The maintained fixture is [examples/dnd-spells.config.yml](../examples/dnd-spells.config.yml).
|
||||
Both complete files are validated by the CLI test suite. The fragments below
|
||||
illustrate individual fields and are not alternate complete configurations.
|
||||
|
||||
## Top-Level Fields
|
||||
|
||||
@@ -52,31 +40,21 @@ rejected; execution profiles now come from Scriptorium.
|
||||
|
||||
Built-in defaults:
|
||||
|
||||
```yaml
|
||||
concurrency:
|
||||
total_llm: 1
|
||||
diagnostics:
|
||||
work_dir: /tmp/notarius
|
||||
retention: auto
|
||||
workspace:
|
||||
diagnostics:
|
||||
enabled: true
|
||||
resume:
|
||||
enabled: false
|
||||
debug:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
`workspace.directory` is unset by default. Without a workspace directory,
|
||||
diagnostics continue to use `/tmp/notarius`, and checkpoint and debug workspace
|
||||
features have no storage root.
|
||||
- `concurrency.total_llm`: `1`
|
||||
- `diagnostics.work_dir`: `/tmp/notarius`
|
||||
- `diagnostics.retention`: `auto`
|
||||
- `workspace.directory`: unset
|
||||
- `workspace.diagnostics.enabled`: `true`
|
||||
- `workspace.resume.enabled`: `false`
|
||||
- `workspace.debug.enabled`: `false`
|
||||
|
||||
No pipelines are built in. A run requires a configured pipeline.
|
||||
|
||||
If `scriptorium` is omitted, Notarius uses Scriptorium's built-in profile
|
||||
catalog. Prompt definitions may also name default profile IDs. The current D&D
|
||||
scene and spell prompts use Scriptorium prompt defaults when a module binding
|
||||
does not set `llm_profile`.
|
||||
scene and spell prompts default to the built-in `mistral-small-3` profile when a
|
||||
module binding does not set `llm_profile`. That built-in profile reads its
|
||||
credential from `OPENROUTER_API_KEY`.
|
||||
|
||||
## Scriptorium Profiles
|
||||
|
||||
@@ -131,11 +109,8 @@ profiles.
|
||||
|
||||
## Pipelines
|
||||
|
||||
A pipeline defines the fixed Notarius workflow:
|
||||
|
||||
```text
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
A pipeline selects implementations for the fixed workflow defined by
|
||||
[Architecture](policy/architecture.md#system-shape).
|
||||
|
||||
Pipeline fields:
|
||||
|
||||
@@ -159,23 +134,20 @@ Artifact lane fields:
|
||||
- `references`: optional compatibility alias for extractor reference bindings.
|
||||
Lane bindings override pipeline-level bindings for the same slot.
|
||||
|
||||
`notarius run` and `notarius config validate --pipeline` resolve the pipeline
|
||||
against the production module catalog and fail fast for unknown or incompatible
|
||||
module keys.
|
||||
Commands that resolve a pipeline fail for unknown or incompatible module keys.
|
||||
See [CLI Reference](cli.md) for command syntax.
|
||||
|
||||
Reference bindings are validated against reference slots declared by eligible
|
||||
chunk, extract, merge, and normalize targets during pipeline resolution. Required slots
|
||||
must be bound after config defaults, target-local references, lane-level
|
||||
compatibility bindings, and run-time `--reference` or `--without-reference`
|
||||
overrides are applied. Config-relative paths are resolved relative to the config
|
||||
file; CLI reference paths are resolved relative to the current working
|
||||
directory. Materialized bound files must be UTF-8 text. Materialized reference
|
||||
provenance is recorded for chunk, extractor, merger, and normalizer targets, and runtime
|
||||
reference content is passed to the target that declares the slot. Reference
|
||||
media types are inferred from file extensions, recorded as canonical base media
|
||||
types, and checked only when a module declares `AcceptedMediaTypes`; unknown
|
||||
extensions are recorded as `application/octet-stream`. Reference content is not
|
||||
written to diagnostics, logs, errors, or manifests.
|
||||
compatibility bindings, and command-line reference overrides are applied.
|
||||
Config-relative paths are resolved relative to the config file; command-line
|
||||
reference paths are resolved relative to the current working directory. Bound
|
||||
files must be UTF-8 text. Reference media types are inferred from file
|
||||
extensions and checked when a module restricts accepted types; unknown
|
||||
extensions use `application/octet-stream`. See [CLI Reference](cli.md#run) for
|
||||
command-line selectors and [Operations](operations.md) for recorded provenance
|
||||
and sensitive-data handling.
|
||||
|
||||
Pipeline-level `references` are defaults. They are valid when at least one
|
||||
eligible target in the full configured pipeline declares the slot, including
|
||||
@@ -252,7 +224,7 @@ Binding fields:
|
||||
- `llm_profile`: optional Scriptorium profile ID. Empty or omitted lets the
|
||||
Scriptorium prompt default select the profile.
|
||||
- `retries`: non-negative retry count for extra runtime attempts after the
|
||||
first attempt. The runner applies retries to `chunk`, `extract`, `merge`, and
|
||||
first attempt. Default: `0`. Supported on `chunk`, `extract`, `merge`, and
|
||||
`normalize` bindings.
|
||||
- `options`: optional module-specific settings.
|
||||
- `references`: optional reference bindings. Supported only for `chunk`,
|
||||
@@ -275,11 +247,6 @@ Validator bindings reject `references`, `retries`, and nested `validators`.
|
||||
During resolution, deterministic validators reject explicit `llm_profile`
|
||||
values.
|
||||
|
||||
The `--llm-profile` run flag overrides every effective LLM-capable module
|
||||
binding to use one Scriptorium profile ID: chunk, every selected lane extract,
|
||||
merge, and normalize binding. It does not override validator-specific
|
||||
`llm_profile` values.
|
||||
|
||||
Configured LLM-backed validators with explicit `llm_profile` values are
|
||||
validated against the configured Scriptorium profile source. Deterministic
|
||||
production validators do not call the LLM and must not set `llm_profile`.
|
||||
@@ -350,80 +317,42 @@ casts still must be present in the source transcript.
|
||||
`workspace` fields:
|
||||
|
||||
- `directory`: optional workspace root for Notarius-owned local state.
|
||||
- `diagnostics.enabled`: set to `false` to skip diagnostics run directories and
|
||||
diagnostics artifact writes. Default: `true`.
|
||||
- `diagnostics.retention`: `auto`, `always`, or `never`.
|
||||
- `resume.enabled`: boolean resume checkpointing setting. Default: `false`.
|
||||
- `debug.enabled`: boolean debug artifact setting. Default: `false`.
|
||||
|
||||
Use `/var/lib/notarius` as the standard production workspace directory. For
|
||||
local development, prefer a project-local ignored path such as
|
||||
`./.notarius/workspace`.
|
||||
|
||||
```yaml
|
||||
workspace:
|
||||
directory: /var/lib/notarius
|
||||
diagnostics:
|
||||
enabled: true
|
||||
retention: auto
|
||||
resume:
|
||||
enabled: false
|
||||
debug:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
When `workspace.directory` is set, diagnostics are written under
|
||||
`<workspace.directory>/diagnostics/`.
|
||||
|
||||
When both `workspace.directory` and `workspace.resume.enabled` are set, runs
|
||||
write stage-owned checkpoint artifacts under
|
||||
`<workspace.directory>/checkpoints/`. `notarius run --resume` can reuse valid
|
||||
checkpoints from a compatible invocation. Checkpoints may contain source text,
|
||||
intermediate raw outputs, rejected outputs, metadata, and warnings. Protect the
|
||||
workspace as sensitive local state.
|
||||
|
||||
When both `workspace.directory` and `workspace.debug.enabled` are set, runs
|
||||
write per-invocation debug artifacts under
|
||||
`<workspace.directory>/debug/<run-id>/`. Debug artifacts may contain source
|
||||
material, reference material, prompt inputs, model outputs, validation payloads,
|
||||
and other sensitive content. Debug is disabled by default.
|
||||
- `resume.enabled`: boolean resume checkpointing setting.
|
||||
- `debug.enabled`: boolean debug artifact setting.
|
||||
- `diagnostics`: optional diagnostics settings defined below.
|
||||
|
||||
`workspace.resume.enabled` and `workspace.debug.enabled` are independent.
|
||||
Enabling one does not enable the other.
|
||||
Enabling one does not enable the other. For directory layout, state lifecycle,
|
||||
permissions, and sensitive content, see [Operations](operations.md).
|
||||
|
||||
## Diagnostics
|
||||
|
||||
Preferred workspace diagnostics fields:
|
||||
|
||||
- `workspace.directory`: workspace root for Notarius-owned local state.
|
||||
- `workspace.diagnostics.enabled`: set to `false` to skip creating diagnostics
|
||||
run directories and diagnostics artifacts. Default: `true`.
|
||||
run directories and diagnostics artifacts.
|
||||
- `workspace.diagnostics.retention`: `auto`, `always`, or `never`.
|
||||
|
||||
When `workspace.directory` is set, diagnostics use
|
||||
`<workspace.directory>/diagnostics` as their work directory.
|
||||
Defaults for workspace and diagnostics fields are listed in
|
||||
[Defaults](#defaults).
|
||||
|
||||
`workspace.diagnostics.retention` overrides legacy diagnostics retention when
|
||||
set.
|
||||
|
||||
`diagnostics` fields:
|
||||
|
||||
- `work_dir`: deprecated compatibility directory for per-run diagnostics.
|
||||
Default: `/tmp/notarius`.
|
||||
- `retention`: deprecated compatibility retention mode. `auto`, `always`, or
|
||||
`never`. Empty uses `auto`.
|
||||
`never`.
|
||||
|
||||
Existing `diagnostics.work_dir`, `diagnostics.retention`, `NOTARIUS_WORK_DIR`,
|
||||
and `NOTARIUS_DIAGNOSTICS_RETENTION` inputs remain supported for compatibility.
|
||||
New configuration should use `workspace.directory` and
|
||||
`workspace.diagnostics.retention` instead.
|
||||
|
||||
`auto` retains diagnostics for failed runs and successful runs with warnings.
|
||||
`always` retains diagnostics for every run. `never` removes diagnostics for
|
||||
successful runs without regard to warnings; failed runs are retained.
|
||||
|
||||
The `--diagnostics-dir` run flag overrides the effective diagnostics work
|
||||
directory for that invocation. It affects diagnostics only and does not change
|
||||
the workspace directory.
|
||||
For retention behavior and the physical diagnostics layout, see
|
||||
[Operations](operations.md#retention). For the invocation-specific diagnostics
|
||||
override, see [CLI Reference](cli.md#run).
|
||||
|
||||
## Validation
|
||||
|
||||
@@ -440,7 +369,7 @@ Pipeline resolution additionally checks:
|
||||
|
||||
- the pipeline ID exists;
|
||||
- at least one artifact lane is declared and selected;
|
||||
- selected lanes exist when `--only` is used;
|
||||
- lanes selected through the CLI exist in the resolved pipeline;
|
||||
- required module keys are present;
|
||||
- module keys are registered for the expected slot;
|
||||
- module capability requirements are satisfied;
|
||||
|
||||
@@ -4,41 +4,16 @@ This is the first-read landing page for people and LLM coding agents working on
|
||||
Notarius. It provides a concise repository orientation and routes each kind of
|
||||
change to its canonical documentation.
|
||||
|
||||
## Orientation
|
||||
|
||||
Notarius is a small Go application for extracting structured data from source
|
||||
material. It is a general extraction platform with an initial Seriatim and D&D
|
||||
implementation. Configured modules run through a fixed workflow:
|
||||
|
||||
```text
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
|
||||
The CLI is the application boundary. Core packages own deterministic models and
|
||||
policy, framework packages own reusable contracts and orchestration, and module
|
||||
and validator packages own concrete behavior.
|
||||
|
||||
## Repository Map
|
||||
|
||||
- `cmd/notarius`: executable entry point.
|
||||
- `internal/cli`: CLI behavior and production composition.
|
||||
- `internal/core`: deterministic source, config, artifact, diagnostics, and
|
||||
workspace packages.
|
||||
- `internal/framework`: contracts, pipeline orchestration, validation helpers,
|
||||
checkpoints, debug recording, and LLM runtime plumbing.
|
||||
- `internal/modules`: concrete implementations of the six pipeline stages.
|
||||
- `internal/validators`: concrete output validators.
|
||||
- `docs`: canonical policy, reference, integration, internal, ADR, and roadmap
|
||||
documentation.
|
||||
- `examples`: maintained, secret-free example inputs and configuration.
|
||||
|
||||
See [Internal Overview](internal/overview.md) for the implemented component
|
||||
map and links to focused internal documentation.
|
||||
Notarius is a Go CLI for configured structured extraction workflows. Start with
|
||||
the [README](../README.md) for product context, [Architecture](policy/architecture.md)
|
||||
for system boundaries, and [Internal Overview](internal/overview.md) for the
|
||||
implemented component map.
|
||||
|
||||
## What To Read
|
||||
|
||||
| When working on | Read | Why |
|
||||
| --- | --- | --- |
|
||||
| Finding the package or component that owns current behavior | [Internal Overview](internal/overview.md) | It is the implemented component inventory and routes to focused internals. |
|
||||
| Application shape, package boundaries, contracts, dependency direction, runtime guarantees, or safety properties | [Architecture](policy/architecture.md) and relevant [ADRs](adr/) | Architecture defines the intended system and its invariants; ADRs preserve significant decision rationale. |
|
||||
| Any documentation addition or revision | [Documentation Policy](policy/documentation.md) | It defines canonical homes, audiences, current-behavior rules, and maintenance requirements. |
|
||||
| Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. |
|
||||
@@ -63,13 +38,3 @@ go test ./...
|
||||
go vet ./...
|
||||
go build ./cmd/notarius
|
||||
```
|
||||
|
||||
## Universal Reminders
|
||||
|
||||
- Keep non-roadmap documentation limited to implemented behavior.
|
||||
- Update affected documentation and maintained examples in the same change as
|
||||
behavior.
|
||||
- Use fakes, fixtures, or local test servers instead of real external services
|
||||
in tests.
|
||||
- Do not expose secrets in code, errors, logs, diagnostics, manifests,
|
||||
documentation, or examples.
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
# D&D Spell Raw Output
|
||||
|
||||
This document is the durable raw output contract for the implemented
|
||||
`dnd/spells` extractor.
|
||||
This document is the durable raw output contract for the production D&D spell
|
||||
extractor. Selectable extractor keys are cataloged in
|
||||
[Configuration](../config.md#implemented-production-modules).
|
||||
|
||||
## Identity
|
||||
|
||||
- Extractor key: `dnd/spells`
|
||||
- Prompt ID: `dnd.spells`
|
||||
- Response schema key: `dnd_spells`
|
||||
- Response schema ID: `notarius.dnd.spells`
|
||||
@@ -13,100 +13,47 @@ This document is the durable raw output contract for the implemented
|
||||
- Response schema version: `v1`
|
||||
- Media type: `application/json`
|
||||
|
||||
The extractor requires source chunks and transcript source capability. It
|
||||
returns canonical spell-cast JSON derived from the structured LLM response. The
|
||||
extractor assigns source IDs deterministically and keeps source-unit ranges as
|
||||
model-authored evidence locations. The default `appendorder` merger passes a
|
||||
single chunk output through and concatenates multiple
|
||||
`spell_casts` arrays in chunk order. The default `noop` normalizer passes the
|
||||
merge output through unchanged.
|
||||
The output contains canonical spell casts derived from transcript evidence.
|
||||
Source IDs are assigned from the input identity; source-unit ranges identify
|
||||
the evidence location.
|
||||
|
||||
## Output Shape
|
||||
|
||||
For a single chunk, `lanes/spells.json` has this shape:
|
||||
`lanes/spells.json` is a JSON object with one required top-level array. Its
|
||||
structure is:
|
||||
|
||||
```json
|
||||
{
|
||||
"spell_casts": [
|
||||
{
|
||||
"caster": "Aria",
|
||||
"spell": "Cure Wounds",
|
||||
"effect": "heals an injured ally",
|
||||
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds.",
|
||||
"source_refs": [
|
||||
{
|
||||
"source_id": "session-alpha",
|
||||
"start_unit_id": 1,
|
||||
"end_unit_id": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```text
|
||||
{"spell_casts": [<spell-cast object>, ...]}
|
||||
```
|
||||
|
||||
`spell_casts` must be present. It may be empty when no spell casts are found.
|
||||
|
||||
For multiple chunks with the default merger, the lane output keeps the same
|
||||
top-level shape and concatenates `spell_casts` in chunk order:
|
||||
|
||||
```json
|
||||
{
|
||||
"spell_casts": [
|
||||
{
|
||||
"caster": "Aria",
|
||||
"spell": "Cure Wounds",
|
||||
"effect": "heals an injured ally",
|
||||
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds.",
|
||||
"source_refs": [
|
||||
{
|
||||
"source_id": "session-alpha",
|
||||
"start_unit_id": 1,
|
||||
"end_unit_id": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
When multiple chunk results are combined, spell casts remain in chunk order.
|
||||
|
||||
## Spell-Cast Fields
|
||||
|
||||
Each spell cast contains:
|
||||
Each spell cast contains exactly these required fields:
|
||||
|
||||
- `caster`: in-world character or creature casting the spell;
|
||||
- `spell`: spell name;
|
||||
- `effect`: concise spell effect in the scene;
|
||||
- `narrative_description`: short description of the spell cast in context;
|
||||
- `source_refs`: transcript source references with extractor-assigned source
|
||||
IDs and model-supplied unit ranges.
|
||||
IDs and evidence unit ranges. It must contain at least one entry.
|
||||
|
||||
`caster` is the in-world caster, not the transcript speaker.
|
||||
All four string fields must be non-empty. `caster` is the in-world caster, not
|
||||
the transcript speaker. Unknown fields are rejected.
|
||||
|
||||
## Source References
|
||||
|
||||
Each source reference uses the generic source-reference shape:
|
||||
Each source reference contains exactly three required fields: `source_id`,
|
||||
`start_unit_id`, and `end_unit_id`. The source ID must match the input identity.
|
||||
The unit IDs must be positive integers present in the input, and the start unit
|
||||
must not appear after the end unit. Unknown fields are rejected.
|
||||
|
||||
- `source_id`
|
||||
- `start_unit_id`
|
||||
- `end_unit_id`
|
||||
|
||||
The LLM-facing prompt schema asks only for integer `start_unit_id` and
|
||||
`end_unit_id` values matching source-unit IDs. `source_id` is assigned by the
|
||||
extractor from the source document ID before validation and output, and is
|
||||
required in this durable output contract.
|
||||
|
||||
## References
|
||||
|
||||
The extractor accepts optional UTF-8 text references:
|
||||
|
||||
- `players`
|
||||
- `party`
|
||||
- `glossary`
|
||||
- `roster`, a deprecated compatibility alias for `party`
|
||||
|
||||
References are supporting disambiguation material only. They are not source
|
||||
evidence and are not addressable through `source_refs`.
|
||||
Reference slot keys and accepted file types are defined in
|
||||
[Configuration](../config.md#implemented-production-modules). References are
|
||||
supporting disambiguation material, not source evidence, and are not
|
||||
addressable through `source_refs`.
|
||||
|
||||
## Manifest Metadata
|
||||
|
||||
|
||||
@@ -1,22 +1,17 @@
|
||||
# JSON Output
|
||||
|
||||
This document is the durable JSON output file-format contract produced by the
|
||||
implemented `json` output module and written by the CLI.
|
||||
production JSON encoder and written by the CLI. Selectable output-encoder keys
|
||||
are cataloged in
|
||||
[Configuration](../config.md#implemented-production-modules).
|
||||
|
||||
## Output Directory
|
||||
|
||||
The CLI writes logical output files under:
|
||||
|
||||
```text
|
||||
<output-root>/<run-id>/
|
||||
```
|
||||
|
||||
The default output root is `./notarius-output`. Operational behavior is covered
|
||||
in [Operations](../operations.md).
|
||||
The output module produces the logical bundle described here. The CLI's
|
||||
physical placement and lifecycle for that bundle are defined in
|
||||
[Operations](../operations.md#output-directory).
|
||||
|
||||
## Files
|
||||
|
||||
The `json` output module writes:
|
||||
The encoder writes:
|
||||
|
||||
- `index.json`
|
||||
- `manifest.json`
|
||||
@@ -25,6 +20,7 @@ The `json` output module writes:
|
||||
- `warnings.json`
|
||||
|
||||
Files are pretty-printed JSON with a trailing newline when the payload is JSON.
|
||||
Logical file paths are relative, slash-separated, and may not contain `..`.
|
||||
|
||||
## `index.json`
|
||||
|
||||
@@ -58,22 +54,21 @@ sanitizing the lane ID:
|
||||
- empty sanitized names are rejected;
|
||||
- two lanes that sanitize to the same output file are rejected.
|
||||
|
||||
`manifest_file`, `rejected_file`, and `warnings_file` contain the fixed paths
|
||||
shown above. Each `output_files` entry requires `lane_id` and `file`. It also
|
||||
contains the normalized payload `media_type`, normalizer `module_key`, and
|
||||
response `schema_id`, `schema_name`, and `schema_version` when those values are
|
||||
available.
|
||||
|
||||
## `manifest.json`
|
||||
|
||||
`manifest.json` contains a run manifest:
|
||||
`manifest.json` contains a run manifest. This abridged example shows its core
|
||||
structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"run_id": "run-123",
|
||||
"pipeline_id": "dnd-session",
|
||||
"pipeline_digest": "sha256:...",
|
||||
"input_module": "seriatim",
|
||||
"chunker": "dnd/scenes",
|
||||
"source_digests": ["sha256:..."],
|
||||
"extractors": ["dnd/spells"],
|
||||
"merger": "appendorder",
|
||||
"normalizer": "noop",
|
||||
"output_encoder": "json",
|
||||
"artifact_lanes": [
|
||||
{
|
||||
"id": "spells",
|
||||
@@ -82,35 +77,6 @@ sanitizing the lane ID:
|
||||
"normalizer": "noop"
|
||||
}
|
||||
],
|
||||
"validator_chains": [
|
||||
{
|
||||
"stage": "extract",
|
||||
"lane_id": "spells",
|
||||
"module_key": "dnd/spells",
|
||||
"validators": [
|
||||
{
|
||||
"key": "generic/valid_json",
|
||||
"execution_class": "deterministic"
|
||||
},
|
||||
{
|
||||
"key": "generic/valid_json_schema",
|
||||
"execution_class": "deterministic"
|
||||
},
|
||||
{
|
||||
"key": "extract/dnd/spells/shape",
|
||||
"execution_class": "deterministic"
|
||||
},
|
||||
{
|
||||
"key": "extract/dnd/spells/source_refs",
|
||||
"execution_class": "deterministic"
|
||||
},
|
||||
{
|
||||
"key": "extract/dnd/spells/source_relatedness",
|
||||
"execution_class": "deterministic"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"validation_status": "approved",
|
||||
"started_at": "2026-01-01T00:00:00Z",
|
||||
"completed_at": "2026-01-01T00:00:01Z"
|
||||
@@ -119,6 +85,23 @@ sanitizing the lane ID:
|
||||
|
||||
Fields with empty values may be omitted by JSON encoding.
|
||||
|
||||
The manifest fields are:
|
||||
|
||||
- `run_id`, `pipeline_id`, and `pipeline_digest`: run and resolved-pipeline
|
||||
identity;
|
||||
- `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, and
|
||||
`output_encoder`: resolved module keys;
|
||||
- `module_metadata` and `artifact_lanes`: module and per-lane provenance,
|
||||
including prompt and response-schema provenance when provided;
|
||||
- `validator_chains`: resolved validation points and validators;
|
||||
- `source_digests` and `references`: source and reference provenance;
|
||||
- `normalized_outputs` and `rejected_outputs`: payload-free result summaries;
|
||||
- `llm_profiles`: selected profile IDs and provider or model names when
|
||||
available;
|
||||
- `metadata`: the effective prompt `session_id`;
|
||||
- `validation_status`: `approved` or `rejected`;
|
||||
- `started_at` and `completed_at`: UTC run timestamps.
|
||||
|
||||
`source_digests` contains source document digests only. Bound references are
|
||||
recorded separately under `references`, which contains provenance only: target
|
||||
stage, lane ID when present, slot name, origin type and URI, digest, media
|
||||
@@ -152,27 +135,8 @@ Each normalized raw output is written to `lanes/<sanitized-lane-id>.json`.
|
||||
The JSON output encoder accepts only `application/json` normalized outputs. The
|
||||
file contains the raw JSON payload pretty-printed.
|
||||
|
||||
For the current D&D spell extractor, `lanes/spells.json` has this shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"spell_casts": [
|
||||
{
|
||||
"caster": "Aria",
|
||||
"spell": "Cure Wounds",
|
||||
"effect": "heals an injured ally",
|
||||
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds.",
|
||||
"source_refs": [
|
||||
{
|
||||
"source_id": "session-alpha",
|
||||
"start_unit_id": 1,
|
||||
"end_unit_id": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
The schema of each lane payload is owned by that artifact contract. For the
|
||||
current D&D spell lane, see [D&D Spell Raw Output](dnd-spell-artifacts.md).
|
||||
|
||||
## `rejected.json`
|
||||
|
||||
@@ -184,10 +148,10 @@ Shape:
|
||||
}
|
||||
```
|
||||
|
||||
When raw output validation rejects an output, entries use the
|
||||
`contracts.RejectedOutput` shape, including stage, lane ID, module key,
|
||||
validator name, reason code, message, attempt count, and optional diagnostic
|
||||
artifact path.
|
||||
When raw output validation rejects an output, each entry contains `stage` and
|
||||
`message`. It includes `lane_id`, `module_key`, `chunk_id`, `chunk_index`,
|
||||
`validator_name`, `reason_code`, `attempt_count`, and
|
||||
`diagnostic_artifact_path` when applicable.
|
||||
|
||||
## `warnings.json`
|
||||
|
||||
@@ -206,3 +170,5 @@ Shape:
|
||||
```
|
||||
|
||||
`warnings` is an empty array when no warnings are reported.
|
||||
Each warning requires `reason_code` and `message`; `scope` is omitted when it is
|
||||
empty.
|
||||
|
||||
@@ -1,114 +1,60 @@
|
||||
# Seriatim Transcript JSON
|
||||
|
||||
This document is the external input contract for the implemented `seriatim`
|
||||
input adapter.
|
||||
This document is the external input contract consumed by the production
|
||||
Seriatim input adapter. Selectable input-adapter keys are cataloged in
|
||||
[Configuration](../config.md#implemented-production-modules).
|
||||
|
||||
## Adapter
|
||||
|
||||
- Module key: `seriatim`
|
||||
- Document kind: `transcript`
|
||||
- Unit kind: `transcript_segment`
|
||||
- Source format: `application/vnd.seriatim+json`
|
||||
|
||||
The adapter parses raw Seriatim JSON into a generic source document. It owns
|
||||
transcript-specific JSON parsing and metadata mapping; core source and pipeline
|
||||
code stay source-format agnostic.
|
||||
|
||||
## Accepted Shape
|
||||
|
||||
The input must be one JSON object with top-level `metadata` and `segments`
|
||||
fields. This covers the maintained minimal fixture and Seriatim intermediate
|
||||
output that provides the same required segment fields.
|
||||
|
||||
```json
|
||||
{
|
||||
"metadata": {
|
||||
"id": "session-alpha",
|
||||
"title": "Synthetic D&D spell session"
|
||||
},
|
||||
"segments": [
|
||||
{
|
||||
"id": 1,
|
||||
"start": 0,
|
||||
"end": 4,
|
||||
"speaker": "Aria",
|
||||
"text": "Aria raises her holy symbol and casts Cure Wounds."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The maintained example is
|
||||
[examples/seriatim-minimal-transcript.json](../../examples/seriatim-minimal-transcript.json).
|
||||
|
||||
Top-level metadata entries are preserved. Other segment fields, such as
|
||||
`categories`, are ignored.
|
||||
Required top-level fields:
|
||||
|
||||
- `metadata`: an object. Its entries are accepted as source metadata.
|
||||
- `segments`: a non-empty array of segment objects.
|
||||
|
||||
Required segment fields:
|
||||
|
||||
- `id`: a positive integer JSON number or canonical decimal string without
|
||||
leading zeros or surrounding whitespace;
|
||||
- `start`: a finite, non-negative JSON number or numeric string;
|
||||
- `end`: a finite, non-negative JSON number or numeric string that is not less
|
||||
than `start`;
|
||||
- `speaker`: a non-empty string;
|
||||
- `text`: a non-empty string.
|
||||
|
||||
Other top-level and segment fields, such as `categories`, are ignored.
|
||||
|
||||
Multiple top-level JSON values are rejected.
|
||||
|
||||
## Validation
|
||||
|
||||
The adapter rejects:
|
||||
|
||||
- empty raw input;
|
||||
- malformed JSON;
|
||||
- top-level JSON that is not an object;
|
||||
- missing, null, or non-object `metadata`;
|
||||
- missing, null, non-array, or empty `segments`;
|
||||
- segment values that are not objects;
|
||||
- segment `id` values that are not positive integer JSON numbers or numeric
|
||||
strings;
|
||||
- non-string `speaker` or `text`;
|
||||
- duplicate segment IDs;
|
||||
- missing or empty `speaker`;
|
||||
- missing, empty, invalid, non-finite, or negative `start`;
|
||||
- missing, empty, invalid, non-finite, or negative `end`;
|
||||
- `end` values before `start`;
|
||||
- missing or empty `text`.
|
||||
The adapter rejects empty input, malformed JSON, multiple top-level JSON values,
|
||||
non-object segment values, duplicate segment IDs, and any violation of the
|
||||
shape or field constraints above.
|
||||
|
||||
Segment text is preserved as provided, but it must not be empty after trimming.
|
||||
|
||||
## Source Mapping
|
||||
## Derived Identity
|
||||
|
||||
The adapter maps input to `SourceDocument`:
|
||||
Notarius identifies the parsed source in this order:
|
||||
|
||||
- `metadata` becomes `SourceDocument.Metadata`;
|
||||
- `SourceDocument.Kind` is `transcript`;
|
||||
- `SourceDocument.Format` is `application/vnd.seriatim+json`;
|
||||
- `SourceDocument.Digest` is `sha256:<hex>` of the exact raw input bytes.
|
||||
1. `metadata.id`, when it is a non-empty string after trimming;
|
||||
2. `metadata.source_id`, when it is a non-empty string after trimming;
|
||||
3. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
|
||||
|
||||
`SourceDocument.ID` is selected in this order:
|
||||
|
||||
1. the parse request source ID, after trimming;
|
||||
2. `metadata.id`, when it is a non-empty string after trimming;
|
||||
3. `metadata.source_id`, when it is a non-empty string after trimming;
|
||||
4. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
|
||||
|
||||
Each segment becomes one `SourceUnit`:
|
||||
|
||||
- `segment.id` becomes integer `SourceUnit.ID`;
|
||||
- `segment.text` becomes `SourceUnit.Text`;
|
||||
- `SourceUnit.Kind` is `transcript_segment`;
|
||||
- `speaker`, `start`, and `end` are stored in source-unit metadata.
|
||||
|
||||
## Metadata Keys
|
||||
|
||||
Seriatim unit metadata uses these keys:
|
||||
|
||||
- `speaker`: string speaker label;
|
||||
- `start`: `json.Number` start value;
|
||||
- `end`: `json.Number` end value.
|
||||
|
||||
The `internal/modules/input/seriatim` package exposes typed accessors for these
|
||||
values.
|
||||
|
||||
## Capabilities
|
||||
|
||||
The module declares these provided capabilities:
|
||||
|
||||
- `source.transcript`
|
||||
- `transcript.speaker`
|
||||
- `transcript.timestamps`
|
||||
The source digest recorded in output provenance is `sha256:<hex>` of the exact
|
||||
raw input bytes. Segment IDs become the unit IDs used by artifact source
|
||||
references.
|
||||
|
||||
## Compatibility Limit
|
||||
|
||||
|
||||
@@ -1,102 +1,93 @@
|
||||
# Diagnostics Internals
|
||||
|
||||
Diagnostics internals live in `internal/core/diagnostics`. Operator-facing run
|
||||
behavior is documented in [Operations](../operations.md).
|
||||
|
||||
## Purpose
|
||||
|
||||
Diagnostics provide local inspection artifacts for a run without becoming the
|
||||
durable output contract. Durable user output is produced by output modules and
|
||||
written by the CLI.
|
||||
|
||||
Diagnostics must not expose secrets.
|
||||
`internal/core/diagnostics` provides the scoped writer and retention decision
|
||||
used by `internal/cli`. The physical layout, artifact inventory, retention
|
||||
semantics, failure inspection, and cleanup procedures are canonical in
|
||||
[Operations](../operations.md#diagnostics-directory). Configuration fields and
|
||||
defaults are canonical in [Configuration](../config.md#diagnostics).
|
||||
|
||||
## Run Directory
|
||||
|
||||
`NewRunDirectory(workDir, retention)` creates:
|
||||
`NewRunDirectory` normalizes empty constructor inputs, creates the effective
|
||||
diagnostics root when needed, and allocates a unique timestamp-based child
|
||||
directory. It retries a bounded number of collisions before failing. The
|
||||
resulting `RunDirectory` retains its creation time and retention mode for later
|
||||
metadata and cleanup decisions.
|
||||
|
||||
```text
|
||||
<workDir>/run-<unix-nanoseconds>/
|
||||
```
|
||||
The package does not resolve workspace configuration. `internal/cli` derives
|
||||
effective workspace settings first and passes the diagnostics root into the
|
||||
constructor.
|
||||
|
||||
If `workDir` is empty, it defaults to `/tmp/notarius`. Empty retention defaults
|
||||
to `auto`.
|
||||
## Scoped Writers
|
||||
|
||||
The CLI passes the effective diagnostics root from workspace configuration.
|
||||
When `workspace.directory` is set and diagnostics are enabled, that root is
|
||||
`<workspace.directory>/diagnostics`. The legacy diagnostics work directory and
|
||||
`--diagnostics-dir` still pass a diagnostics-only root to this constructor.
|
||||
Typed methods on `RunDirectory` write invocation metadata, redacted effective
|
||||
configuration, resolved pipeline/reference data, checkpoint events, source data
|
||||
when explicitly requested, manifests, reports, warnings, and error text. The
|
||||
current filenames and their operator-facing contents are listed in
|
||||
[Operations](../operations.md#diagnostics-directory).
|
||||
|
||||
The writer makes the work directory if needed, then attempts to create a unique
|
||||
run directory. It retries run ID creation a bounded number of times if a
|
||||
collision occurs.
|
||||
JSON methods indent their payload and append a newline. All artifact writes use
|
||||
a temporary file in the target directory, apply the requested permissions, and
|
||||
rename it into place. Artifact resolution accepts only a single relative base
|
||||
name; absolute paths, separators, and paths escaping the run directory fail
|
||||
before writing.
|
||||
|
||||
## Artifact Writers
|
||||
## Redacted Configuration
|
||||
|
||||
Implemented artifact names:
|
||||
`WriteRedactedEffectiveConfig` accepts a `RedactedDiagnosticsPayload` provider
|
||||
rather than a raw config value. `internal/core/config` implements that contract
|
||||
by cloning effective config data and removing secret-shaped values before JSON
|
||||
encoding. The diagnostics package therefore never needs configuration-specific
|
||||
field knowledge.
|
||||
|
||||
- `invocation.json`
|
||||
- `effective-config.json`
|
||||
- `resolved-pipeline.json`
|
||||
- `resolved-references.json`
|
||||
- `checkpoint-events.json`
|
||||
- `source-document.json`
|
||||
- `run-manifest.json`
|
||||
- `run-report.json`
|
||||
- `warnings.json`
|
||||
- `error.log`
|
||||
## Retention Coordination
|
||||
|
||||
JSON artifacts are encoded with indentation and a trailing newline. Writes are
|
||||
atomic through a temporary file in the target directory followed by rename.
|
||||
`ShouldRetainRunDirectory` is a pure decision over the effective retention mode,
|
||||
run success, and warning presence. `ApplyRetention` uses that result to remove
|
||||
only its own run directory. Unsupported modes retain data as a fail-safe, though
|
||||
normal CLI execution rejects them during config validation.
|
||||
|
||||
Artifact names must be single relative file names. Absolute paths, path
|
||||
separators, and names resolving outside the run directory are rejected.
|
||||
The meaning of each supported mode belongs in
|
||||
[Operations](../operations.md#retention); this package implements that contract
|
||||
without loading config or inspecting run artifacts.
|
||||
|
||||
## Redacted Effective Config
|
||||
## CLI State Flow
|
||||
|
||||
Diagnostics writers accept payloads that implement
|
||||
`RedactedDiagnosticsPayload`. `internal/core/config` uses this to redact API
|
||||
keys in effective config diagnostics while preserving resolved pipeline context.
|
||||
When diagnostics are enabled, `internal/cli` creates the run directory after
|
||||
configuration loading and before pipeline resolution. It then writes artifacts
|
||||
as state becomes available: invocation data, effective resolution data,
|
||||
pipeline results, and the final report. This ordering permits later failures to
|
||||
retain the context already established.
|
||||
|
||||
The redaction path clones config data before replacing secret values.
|
||||
Failures before construction have no `RunDirectory`. Later failures write an
|
||||
error log, preserve any available partial manifest, and apply a failed-run
|
||||
retention decision. A diagnostics write failure is itself a command failure so
|
||||
the CLI does not report success after losing requested inspection data.
|
||||
|
||||
## Retention
|
||||
When diagnostics are disabled, the CLI carries a nil run directory and the
|
||||
shared `writeDiagnostics` helper turns writes into no-ops. User-facing errors
|
||||
still go to stderr; that invocation behavior is documented in
|
||||
[Operations](../operations.md#failures).
|
||||
|
||||
Retention is decided by `ShouldRetainRunDirectory`.
|
||||
## Package Guarantees
|
||||
|
||||
- Failed runs are always retained.
|
||||
- `always` retains successful runs.
|
||||
- `never` removes successful runs.
|
||||
- `auto` retains successful runs only when warnings exist.
|
||||
- Unknown retention values are treated as retain by the retention decision, but
|
||||
config validation rejects unsupported values before normal runs.
|
||||
- A `RunDirectory` writes and removes only within its allocated directory.
|
||||
- JSON and error artifacts use atomic replacement.
|
||||
- Nil receivers and invalid typed payloads return errors rather than panicking.
|
||||
- Retention never removes a failed run and never targets the diagnostics root.
|
||||
- Diagnostics models contain inspection metadata, not the durable output
|
||||
contract.
|
||||
- Checkpoint and debug serializers remain separate framework components.
|
||||
- Secret-handling follows the invariant in
|
||||
[Architecture](../policy/architecture.md#state-output-and-safety).
|
||||
|
||||
`ApplyRetention` removes only the specific run directory.
|
||||
## Tests To Inspect
|
||||
|
||||
## CLI Failure Behavior
|
||||
|
||||
When diagnostics are enabled, the CLI creates the diagnostics run directory
|
||||
after config loading and before pipeline resolution. Failures before that point
|
||||
do not have diagnostics.
|
||||
|
||||
When workspace diagnostics are explicitly disabled, the CLI does not create a
|
||||
diagnostics run directory and skips diagnostics artifact writes. Failures are
|
||||
still printed to stderr.
|
||||
|
||||
After diagnostics creation, run failures call `WriteErrorLog` and apply
|
||||
retention with `RunSucceeded: false`, so the run directory remains available.
|
||||
|
||||
When the pipeline returns a partial manifest on failure, the CLI writes that
|
||||
manifest before logging the failure.
|
||||
|
||||
## Invariants
|
||||
|
||||
- Diagnostics paths must be narrow and run-directory scoped.
|
||||
- Writes should be atomic where practical.
|
||||
- Secrets must be redacted.
|
||||
- Diagnostics write failures are command failures because they can hide the
|
||||
information needed for recovery.
|
||||
- Durable output file contracts belong to output modules and integration docs,
|
||||
not to diagnostics.
|
||||
- Checkpoint and debug workspace files are separate framework-owned artifacts,
|
||||
not diagnostics artifacts.
|
||||
- `internal/core/diagnostics/run_dir_test.go`: allocation, artifact confinement,
|
||||
atomic writes, retention, and failure behavior.
|
||||
- `internal/core/diagnostics/artifacts_test.go`: stable artifact identifiers.
|
||||
- `internal/core/config/redaction_test.go`: clone-and-redact payload behavior.
|
||||
- `internal/core/workspace/settings_test.go`: effective diagnostics-root and
|
||||
enablement handoff.
|
||||
- `internal/cli/run_test.go`: creation timing, artifact sequencing, disabled
|
||||
diagnostics, overrides, failures, and retention integration.
|
||||
|
||||
@@ -1,114 +1,132 @@
|
||||
# LLM Runtime
|
||||
# LLM Runtime Internals
|
||||
|
||||
The implemented LLM runtime lives in `internal/framework/llm`. It provides
|
||||
transport-neutral structured completion contracts, a Scriptorium-backed
|
||||
production client, concurrency scheduling, prompt/schema asset registration,
|
||||
schema registry helpers, and secret redaction.
|
||||
`internal/framework/llm` implements Notarius's transport boundary for structured
|
||||
completion. It contains the Scriptorium adapter, concurrency scheduler,
|
||||
prompt/schema registries, selected-profile recording, and provider-error
|
||||
redaction.
|
||||
|
||||
## Contract
|
||||
Provider-neutral ownership rules are defined in
|
||||
[Architecture](../policy/architecture.md#llm-boundary). Profile sources,
|
||||
credentials, and concurrency settings are defined in
|
||||
[Configuration](../config.md).
|
||||
|
||||
Modules depend on `contracts.StructuredLLMClient`:
|
||||
## Structured Contract
|
||||
|
||||
```go
|
||||
CompleteStructured(ctx, request, out) (response, error)
|
||||
```
|
||||
Modules and LLM-backed validators depend on
|
||||
`contracts.StructuredLLMClient.CompleteStructured`. A request identifies a
|
||||
prompt and optional profile/session, supplies named input materials and
|
||||
variables, and provides a caller-owned decoding target. A successful response
|
||||
contains the validated raw structured bytes plus non-secret provider, model,
|
||||
profile, and token metadata.
|
||||
|
||||
The request contains prompt ID/version, profile ID, session ID, prompt input
|
||||
materials, and variables. The caller supplies a pointer target for decoded
|
||||
structured output. The response also carries the raw structured output bytes
|
||||
returned by the runtime so modules can preserve raw payloads in pipeline stage
|
||||
outputs.
|
||||
The caller owns prompt selection, response-schema selection, and interpretation
|
||||
of the decoded result. `LLMInputMaterial` keeps source and reference bytes with
|
||||
their origin metadata so the adapter can pass named artifacts to Scriptorium
|
||||
without exposing Scriptorium types through stage contracts.
|
||||
|
||||
Modules that call the LLM own their prompts, schemas, prompt IDs, and
|
||||
domain-specific interpretation. Validator packages own approve/reject policy,
|
||||
and central catalog mappings decide which validators run by default. Provider
|
||||
adapters should not contain domain-specific prompt logic.
|
||||
## Production Construction
|
||||
|
||||
Prompt input materials carry source or reference bytes with optional origin
|
||||
metadata. The Scriptorium-backed runtime receives them as named artifacts rather
|
||||
than rendered prompt strings owned by Notarius modules.
|
||||
`internal/cli` constructs the production runtime by:
|
||||
|
||||
## Production Client Construction
|
||||
1. collecting embedded prompt and response-schema assets from production module
|
||||
packages;
|
||||
2. creating a `ScriptoriumClient` from the effective profile source;
|
||||
3. attaching an `LLMProfileRecorder`;
|
||||
4. creating a scheduler from the effective concurrency limit;
|
||||
5. returning a `ScheduledClient` wrapper.
|
||||
|
||||
`internal/cli` builds the production LLM client from the effective config:
|
||||
|
||||
1. collect production Scriptorium prompt and schema assets from module packages;
|
||||
2. create a Scriptorium-backed structured client using effective Scriptorium
|
||||
profile source settings from `scriptorium.profile_dir` or
|
||||
`scriptorium.profile_file`;
|
||||
3. create a scheduler from global LLM concurrency;
|
||||
4. wrap the client with `NewScheduledClient`;
|
||||
5. let the runtime report non-secret profile manifest metadata after calls.
|
||||
|
||||
The runtime records the actual selected Scriptorium profile, provider, and model
|
||||
used during execution. Manifest population does not rely on a precomputed
|
||||
profile ID before pipeline execution.
|
||||
|
||||
Explicit profile validation applies to LLM-capable pipeline stages: chunk,
|
||||
extract, merge, normalize, and LLM-backed validators with explicit
|
||||
`llm_profile` values. Input, output, and deterministic validators do not call
|
||||
the LLM. The `--llm-profile` run flag overrides effective chunk, extract, merge,
|
||||
and normalize bindings; it does not override validator-specific profiles.
|
||||
The CLI separately gathers explicit profile IDs from resolved LLM-capable stage
|
||||
and validator bindings. It prepares a small internal check prompt for each ID so
|
||||
missing or invalid profiles fail before pipeline execution. The runtime profile
|
||||
override syntax and scope are defined in the
|
||||
[CLI reference](../cli.md#run); binding rules are defined in
|
||||
[Configuration](../config.md#module-bindings).
|
||||
|
||||
## Scriptorium Adapter
|
||||
|
||||
`ScriptoriumClient` implements `contracts.StructuredLLMClient` by converting
|
||||
Notarius prompt requests into Scriptorium `RunRequest` values. It:
|
||||
`ScriptoriumClient` converts a Notarius request into a Scriptorium `RunRequest`.
|
||||
It validates the decoding target and prompt identity, maps named input materials
|
||||
to inline artifacts, forwards explicit profile and session context, delegates
|
||||
rendering/provider execution/structured validation, and unmarshals successful
|
||||
JSON into the caller target.
|
||||
|
||||
- validates the caller output target and prompt ID;
|
||||
- converts `LLMInputMaterial` values into inline Scriptorium artifacts, using a
|
||||
single space for empty material so optional blank references remain explicit;
|
||||
- passes `session_id` through Scriptorium variables and request metadata when
|
||||
present;
|
||||
- sends explicit profile IDs only when the request supplies one;
|
||||
- lets Scriptorium render prompts, call the configured provider, and validate
|
||||
structured output;
|
||||
- unmarshals successful JSON into the caller-provided target;
|
||||
- returns the validated raw structured output bytes to the caller;
|
||||
- maps token usage and selected profile/model metadata into the Notarius
|
||||
response and manifest profile recorder.
|
||||
Empty optional input material is represented by a single space so Scriptorium
|
||||
retains the named input. The client returns Scriptorium's validated structured
|
||||
bytes rather than re-encoding the caller target, allowing modules to preserve
|
||||
the runtime result exactly.
|
||||
|
||||
Generated-output validation failures are returned as Notarius errors. Provider
|
||||
and runtime errors are wrapped with prompt context and bearer tokens are
|
||||
redacted from error strings. Prompt text, raw source input, reference content,
|
||||
schema JSON, API keys, and bearer tokens are not added to default diagnostics or
|
||||
run manifests.
|
||||
Selected profile, provider, model, and token metadata are mapped into the
|
||||
Notarius response. The recorder deduplicates profiles by identity and supplies
|
||||
manifest-safe profile summaries after actual calls; manifest population does
|
||||
not guess the selected prompt default in advance.
|
||||
|
||||
## Scheduler
|
||||
Generated-output validation failures and provider failures are wrapped with
|
||||
prompt context. Error strings pass through bearer-token redaction before they
|
||||
cross the runtime boundary.
|
||||
|
||||
`Scheduler` bounds concurrent provider calls. It tracks in-flight calls and a
|
||||
FIFO queue of waiters. Cancellation removes queued waiters or releases granted
|
||||
permits.
|
||||
## Scheduling
|
||||
|
||||
`NewScheduledClient` wraps any structured LLM client and runs each completion
|
||||
inside the scheduler.
|
||||
`Scheduler` uses a bounded permit count and a FIFO waiter queue. Immediate
|
||||
acquisition increments the in-flight count; queued acquisition waits for a
|
||||
permit or context cancellation. Cancellation removes a queued waiter, while a
|
||||
cancelled waiter that has already received a permit releases it.
|
||||
|
||||
Effective concurrency is:
|
||||
`ScheduledClient` acquires a permit around each structured completion and
|
||||
defers release on every result path. The effective limit and default are
|
||||
configuration facts in [Configuration](../config.md#defaults).
|
||||
|
||||
1. `concurrency.total_llm`, when greater than zero;
|
||||
2. `1`.
|
||||
## Prompt And Schema Assets
|
||||
|
||||
## Schema Registry
|
||||
`AssetRegistry` combines caller-owned prompt filesystems under stable prefixes
|
||||
and rejects invalid or conflicting registrations. Production module packages
|
||||
register their own prompt and schema assets; generic framework code contains no
|
||||
D&D prompt content.
|
||||
|
||||
The framework schema registry embeds generic test schemas. It also exposes
|
||||
helpers for caller-owned schemas:
|
||||
Schema helpers load embedded JSON Schema with identity and digest metadata,
|
||||
return defensive copies, and expose a diagnostics map that omits schema bytes.
|
||||
The small framework registry contains only generic test schemas; production
|
||||
schemas remain package-owned.
|
||||
|
||||
- `LoadResponseSchema`
|
||||
- `LookupResponseSchema`
|
||||
- `MustLookupResponseSchema`
|
||||
- `ResponseSchema.DiagnosticsMap`
|
||||
## Debug And Redaction Boundaries
|
||||
|
||||
`DiagnosticsMap` omits raw schema content and includes metadata such as key,
|
||||
ID, version, name, and SHA-256.
|
||||
The pipeline may wrap the client with a debug recorder that captures prepared
|
||||
prompt/response material for an explicitly enabled debug run. Default
|
||||
diagnostics and manifests receive identities, hashes, usage, and selected
|
||||
profile summaries rather than prompt, source, reference, schema, or response
|
||||
content.
|
||||
|
||||
Production modules own and register their Scriptorium prompt and schema assets.
|
||||
Framework packages may collect those files but must not contain D&D-specific
|
||||
prompt content.
|
||||
The Scriptorium error wrapper removes bearer credential values from surfaced
|
||||
provider errors; `RedactSecrets` and `ErrorWithSecretsRedacted` support known
|
||||
secret values elsewhere in the runtime. Config diagnostics use a separate
|
||||
clone-and-redact path in `internal/core/config`. These mechanisms implement the
|
||||
security invariant in
|
||||
[Architecture](../policy/architecture.md#state-output-and-safety); operator
|
||||
handling of debug data is defined in [Operations](../operations.md#debug).
|
||||
|
||||
## Secret Redaction
|
||||
## Failure Behavior
|
||||
|
||||
Provider errors are redacted before surfacing through the Scriptorium-backed
|
||||
client. Config diagnostics use redacted effective config payloads.
|
||||
- Invalid targets, missing prompt IDs, malformed structured output, and
|
||||
Scriptorium failures return contextual errors to the calling module.
|
||||
- Scheduler construction rejects non-positive limits; acquisition respects
|
||||
context cancellation.
|
||||
- Asset registration rejects invalid roots, missing content, and path conflicts.
|
||||
- Schema loading distinguishes missing assets, invalid JSON, and invalid
|
||||
metadata.
|
||||
- Profile validation errors occur during CLI preparation when an explicit
|
||||
selected ID cannot be prepared.
|
||||
|
||||
Do not add raw provider request bodies, response bodies, API keys, or prompt
|
||||
payloads to diagnostics by default.
|
||||
## Tests To Inspect
|
||||
|
||||
- `internal/framework/llm/scriptorium_client_test.go` and
|
||||
`scriptorium_api_test.go`: adapter mapping and local HTTP integration.
|
||||
- `internal/framework/llm/scheduler_test.go` and
|
||||
`scheduled_client_test.go`: permits, FIFO behavior, cancellation, and wrapper
|
||||
release.
|
||||
- `internal/framework/llm/asset_registry_test.go` and
|
||||
`schema_registry_test.go`: asset composition, validation, and defensive
|
||||
copies.
|
||||
- `internal/framework/llm/secrets_test.go`: provider-error redaction.
|
||||
- `internal/cli/run_test.go`: profile validation, production client wiring,
|
||||
manifest recording, and debug integration.
|
||||
- Module-local `scriptorium_assets_test.go` files: prompt inputs and package
|
||||
asset registration.
|
||||
|
||||
@@ -1,268 +1,194 @@
|
||||
# Modules
|
||||
|
||||
Production modules live under `internal/modules`. Each module implements one
|
||||
contract from `internal/framework/contracts`, exposes a `ModuleSpec`, and
|
||||
registers itself with the matching pipeline registry.
|
||||
# Module And Validator Internals
|
||||
|
||||
The CLI production catalog currently registers only the modules listed here.
|
||||
Validator implementations live under `internal/validators` and are registered
|
||||
separately from modules. Production default validator chains are central CLI
|
||||
catalog policy; module packages do not own their default validation chains.
|
||||
Production stage implementations live under `internal/modules`; production
|
||||
validators live under `internal/validators`. The selectable keys, configuration
|
||||
options, reference slots, and default validator chain are canonical in the
|
||||
[module](../config.md#implemented-production-modules) and
|
||||
[validator](../config.md#implemented-production-validators) catalogs in
|
||||
Configuration.
|
||||
|
||||
## Contract Pattern
|
||||
## Extension Pattern
|
||||
|
||||
A production module package should provide:
|
||||
A stage module package provides a stable key, constructor, contract
|
||||
implementation, `ModuleSpec`, `Register`, and focused behavior and registration
|
||||
tests. A validator package follows the same pattern with `ValidatorSpec` and the
|
||||
validator registry.
|
||||
|
||||
- a stable module key;
|
||||
- a constructor such as `New`;
|
||||
- the relevant contract implementation;
|
||||
- `ModuleSpec`;
|
||||
- `Register`;
|
||||
- focused tests for registration, options, contract behavior, and errors.
|
||||
Specs expose capability and execution metadata without constructing an
|
||||
implementation. Chunk, extract, merge, and normalize modules that accept
|
||||
auxiliary material declare identical reference slots from both
|
||||
`ReferenceSlots()` and `ModuleSpec().ReferenceSlots`; registration tests enforce
|
||||
that agreement. Runtime delivery uses the corresponding stage request's
|
||||
`References` field.
|
||||
|
||||
Module specs should describe capabilities accurately. Resolution uses specs to
|
||||
reject incompatible pipelines before execution.
|
||||
LLM-backed extensions own their prompt definitions and response schemas under
|
||||
package-local embedded assets. Shared filesystem composition belongs in
|
||||
`internal/modules/sharedassets`; reusable D&D prompt fragments, reference
|
||||
declarations, prompt-input assembly, and source-unit helpers belong in
|
||||
`internal/modules/sharedassets/dnd`. Stage contracts expose only Notarius
|
||||
structured-completion types, not Scriptorium public types.
|
||||
|
||||
Chunk, extract, merge, and normalize modules that accept auxiliary reference material
|
||||
must declare slots through both `ReferenceSlots()` and
|
||||
`ModuleSpec().ReferenceSlots`. The runtime slot list and registry metadata
|
||||
should match so config validation can inspect slots without constructing module
|
||||
instances. A slot declaration names the slot, whether it is required, accepted
|
||||
media types, whether multiple items are allowed, and any byte limit. Empty
|
||||
`AcceptedMediaTypes` means any inferred media type is accepted, though the file
|
||||
must still be UTF-8 text. When a slot declares accepted media types, Notarius
|
||||
compares the canonical base media type inferred from the file extension,
|
||||
case-insensitively and without parameters.
|
||||
Reference material may inform a module or prompt but must not become source
|
||||
evidence. The resolver and materializer behavior is described in
|
||||
[Pipeline Internals](pipeline.md#reference-materialization).
|
||||
|
||||
The resolver materializes reference content for chunk, extractor, merger, and
|
||||
normalizer targets. Runtime delivery uses `contracts.ChunkRequest.References`,
|
||||
`contracts.ExtractionRequest.References`, `contracts.MergeRequest.References`,
|
||||
and `contracts.NormalizeRequest.References`. Reference material is not source
|
||||
evidence and must not be converted into `SourceRef` values. If a module prompt
|
||||
uses references, pass them as prompt input materials through the structured LLM
|
||||
request. Prompt metadata hashes remain based on prompt asset source, not
|
||||
rendered reference bytes.
|
||||
## Input Adapter
|
||||
|
||||
LLM-backed modules own Scriptorium prompt definitions and response schemas in
|
||||
their embedded assets. Module-owned prompts live under each module's shallow
|
||||
`assets/prompts` tree and schemas live under `assets/schemas`. Generic shared
|
||||
prompt filesystem composition lives under `internal/modules/sharedassets`.
|
||||
Common D&D prompt fragments, reference slot helpers, prompt input assembly, and
|
||||
reference rendering live under `internal/modules/sharedassets/dnd`. Module
|
||||
contracts should expose prompt IDs, versions, input material names, and
|
||||
non-secret prompt/schema hashes through manifest metadata; they should not
|
||||
expose Scriptorium public types through chunk, extract, merge, or normalize contracts.
|
||||
### `internal/modules/input/seriatim`
|
||||
|
||||
Chunk modules receive the structured LLM client, configured Scriptorium profile
|
||||
ID, prompt session ID, and raw source input material through
|
||||
`contracts.ChunkRequest` when they need model-backed chunking. The pipeline
|
||||
runner validates generic chunk result invariants before extraction; module-owned
|
||||
policies may be stricter but must stay within the module package.
|
||||
The adapter decodes the supported transcript JSON, selects the source identity,
|
||||
computes the raw-input digest, validates segments, and maps each segment into a
|
||||
generic source unit with speaker and timestamp metadata. Its spec advertises the
|
||||
transcript capabilities consumed by D&D modules.
|
||||
|
||||
Normalize modules receive the structured LLM client, configured Scriptorium
|
||||
profile ID, prompt session ID, and reference material through
|
||||
`contracts.NormalizeRequest` when they need model-backed reconciliation.
|
||||
Parsing is strict about required values and duplicate unit IDs but deliberately
|
||||
ignores unrelated Seriatim fields. The external format and derived-identity
|
||||
rules are defined in the
|
||||
[Seriatim contract](../integrations/seriatim.md).
|
||||
|
||||
Merge modules receive the structured LLM client, configured Scriptorium profile
|
||||
ID, prompt session ID, raw source input material, and reference material through
|
||||
`contracts.MergeRequest` when they need model-backed merge behavior.
|
||||
## Chunkers
|
||||
|
||||
## `seriatim` Input
|
||||
### `internal/modules/chunk/generic`
|
||||
|
||||
Package: `internal/modules/input/seriatim`
|
||||
The generic chunker validates the source document, walks units in configured
|
||||
windows, clones each selected unit, and emits deterministic ordered chunk IDs.
|
||||
Overlap changes the next window start but never reorders units. It records the
|
||||
first and last unit and unit count in chunk metadata.
|
||||
|
||||
The `seriatim` adapter parses Seriatim transcript JSON into a generic source
|
||||
document. It owns transcript JSON details, source ID selection, source digest
|
||||
creation, transcript segment validation, and segment metadata mapping.
|
||||
The accepted options and defaults are defined in
|
||||
[Configuration](../config.md#implemented-production-modules). Generic
|
||||
framework validation canonicalizes the returned unit slices before extraction.
|
||||
|
||||
Provides:
|
||||
### `internal/modules/chunk/dnd/scenes`
|
||||
|
||||
- `source.transcript`
|
||||
- `transcript.speaker`
|
||||
- `transcript.timestamps`
|
||||
The scene chunker prepares a structured Scriptorium request from the full
|
||||
transcript, session, and optional D&D reference inputs. It validates the model's
|
||||
scene boundaries against source-unit IDs and converts them into deterministic
|
||||
chunks.
|
||||
|
||||
External JSON shape belongs in the Seriatim integration doc.
|
||||
Scene validation requires sequential, contiguous, non-overlapping coverage from
|
||||
the first source unit through the last. Each chunk contains JSON scene content
|
||||
and module-owned metadata for the scene description, boundaries, confidence,
|
||||
participants, and unit count. Boundary caveats become warnings. Malformed
|
||||
structured output is returned as an error; there is no fallback chunker.
|
||||
|
||||
## `generic` Chunker
|
||||
The package embeds its prompt and response schema and reports their non-secret
|
||||
identity and hashes through singleton module metadata. Shared D&D assets supply
|
||||
reference declarations and prompt inputs; their user-facing keys and accepted
|
||||
file types remain canonical in [Configuration](../config.md).
|
||||
|
||||
Package: `internal/modules/chunk/generic`
|
||||
## Extractor
|
||||
|
||||
The `generic` chunker splits source units into ordered chunks. It validates the
|
||||
source document, clones source units, assigns chunk IDs such as `chunk-000001`,
|
||||
and records chunk metadata for start unit, end unit, and unit count.
|
||||
### `internal/modules/extract/dnd/spells`
|
||||
|
||||
The pipeline runner canonicalizes chunk units from the source document by
|
||||
integer ID before extractors and mergers run. Chunkers also populate chunk
|
||||
start and end unit IDs, content bytes, and media type. Chunker-owned context
|
||||
should stay in `SourceChunk.Metadata`.
|
||||
The spell extractor prepares a structured request from one chunk, the
|
||||
chunk-scoped source input, the session, and optional D&D reference inputs. It
|
||||
decodes the model response, assigns the generic source identity to every source
|
||||
reference, canonicalizes duplicate references, orders spell casts by their
|
||||
earliest cited unit, and returns raw JSON plus response-schema provenance.
|
||||
|
||||
Options:
|
||||
The package owns its embedded prompt, response schemas, and prompt/schema
|
||||
manifest metadata. Shared D&D helpers keep prompt input names and source-unit
|
||||
reference conversion consistent with the scene chunker. The extractor produces
|
||||
raw output; production validators own approval policy.
|
||||
|
||||
- `max_units`: positive integer, default `50`;
|
||||
- `overlap_units`: non-negative integer, default `0`, and less than
|
||||
`max_units`.
|
||||
The durable payload and manifest metadata shapes are defined in the
|
||||
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
|
||||
|
||||
Provides:
|
||||
## Merger And Normalizer
|
||||
|
||||
- `chunks`
|
||||
### `internal/modules/merge/appendorder`
|
||||
|
||||
## `dnd/scenes` Chunker
|
||||
The merger preserves extract-result order. It passes through one JSON result,
|
||||
concatenates a common top-level array field across multiple JSON objects, and
|
||||
otherwise emits an array of the decoded values. It rejects invalid JSON and
|
||||
non-JSON media types, and it preserves compatible schema provenance.
|
||||
|
||||
Package: `internal/modules/chunk/dnd/scenes`
|
||||
### `internal/modules/normalize/noop`
|
||||
|
||||
The `dnd/scenes` chunker uses the structured LLM client to divide transcript
|
||||
source units into coherent D&D scenes. It supplies the embedded Scriptorium
|
||||
prompt ID, prompt version, transcript input material, response schema, and
|
||||
session ID to the runtime; validates model-authored source-unit boundaries; and
|
||||
converts each scene into a deterministic source chunk.
|
||||
The normalizer defensively clones the accepted merge result, including payload
|
||||
bytes, metadata, warnings, and schema provenance, without changing its logical
|
||||
content.
|
||||
|
||||
Its prompt definition lives under `assets/prompts` and its schema under
|
||||
`assets/schemas`. Shared reusable D&D prompt fragments are provided by
|
||||
`internal/modules/sharedassets/dnd` and referenced from prompt definitions under
|
||||
`./sharedassets/`.
|
||||
## Output Encoder
|
||||
|
||||
Requires:
|
||||
### `internal/modules/output/json`
|
||||
|
||||
- `source.transcript`
|
||||
The JSON encoder sorts normalized results by lane, derives collision-checked
|
||||
safe logical names, pretty-prints JSON payloads, and assembles the logical index,
|
||||
manifest, rejected-result, warning, and lane files. Invalid JSON, unsupported
|
||||
media types, unsafe names, and sanitized-name collisions are errors.
|
||||
|
||||
Provides:
|
||||
The encoder returns logical files only. The CLI places them on disk, and the
|
||||
[JSON output contract](../integrations/json-output.md) defines their external
|
||||
paths and schemas.
|
||||
|
||||
- `chunks`
|
||||
- `chunks.scenes`
|
||||
## Generic Validators
|
||||
|
||||
Options: none. Non-empty options are rejected.
|
||||
The unconditional accept and reject validators provide deterministic production
|
||||
registrations used primarily for controlled composition and tests.
|
||||
|
||||
The chunker enforces full source-unit coverage from the first source unit to the
|
||||
last, sequential contiguous scenes, and no overlap. Its LLM-facing schema uses
|
||||
integer `start_unit_id` and `end_unit_id` values matching source-unit IDs. It
|
||||
assigns chunk IDs such as `scene-000001`, emits JSON chunk content, and stores
|
||||
scene metadata including title, primary mode, participants, summary, boundary
|
||||
note, confidence, boundary unit IDs, and unit count. Boundary caveats become
|
||||
warnings with reason code
|
||||
`scene_boundary_caveat`. Whitespace-only caveats are treated as malformed
|
||||
structured output rather than silently dropped.
|
||||
The JSON syntax validator uses `encoding/json` to reject malformed payloads. The
|
||||
JSON Schema validator requires schema bytes on the validation request, parses
|
||||
the instance and schema with `jsonschema`, and distinguishes payload rejection
|
||||
from schema loading or compilation errors. Neither validator calls the LLM.
|
||||
|
||||
Malformed model output fails explicitly rather than falling back to another
|
||||
chunker. The chunker exposes prompt and response-schema provenance through
|
||||
top-level `module_metadata.chunker` without raw prompts, raw schemas, source
|
||||
text, or secrets.
|
||||
## D&D Spell Validators
|
||||
|
||||
## `dnd/spells` Extractor
|
||||
`internal/validators/extract/dnd/spells/spellpayload` provides strict decoding,
|
||||
shape checks, source-reference candidates, and cited-text lookup shared by the
|
||||
three validators.
|
||||
|
||||
Package: `internal/modules/extract/dnd/spells`
|
||||
The shape validator rejects malformed JSON, unknown fields, missing or empty
|
||||
spell fields, and empty reference lists. The source-reference validator applies
|
||||
generic source-reference validation to every cited range. The relatedness
|
||||
validator approves structurally valid payloads but warns when a case-insensitive
|
||||
spell name is absent from all cited source text. It leaves malformed payloads to
|
||||
the earlier validators in the configured chain.
|
||||
|
||||
The `dnd/spells` extractor owns D&D spell-cast extraction semantics. It
|
||||
supplies the embedded Scriptorium prompt ID, prompt version, chunk-scoped
|
||||
transcript input material, reference input materials, response schema, and
|
||||
session ID to the runtime; then returns the structured LLM `spell_casts`
|
||||
response as raw JSON.
|
||||
Its LLM-facing source-reference schema uses integer `start_unit_id` and
|
||||
`end_unit_id` values matching source-unit IDs.
|
||||
|
||||
Its prompt definition lives under `assets/prompts` and its schema under
|
||||
`assets/schemas`. Shared reusable D&D prompt fragments are provided by
|
||||
`internal/modules/sharedassets/dnd` and referenced from prompt definitions under
|
||||
`./sharedassets/`.
|
||||
|
||||
Requires:
|
||||
|
||||
- `chunks`
|
||||
- `source.transcript`
|
||||
|
||||
Provides:
|
||||
|
||||
- `dnd.spell_casts`
|
||||
|
||||
Response schema identity:
|
||||
|
||||
- schema ID: `notarius.dnd.spells`
|
||||
- schema name: `notarius_dnd_spells_v1`
|
||||
- schema version: `v1`
|
||||
|
||||
The extractor adds prompt and response-schema provenance to lane manifest
|
||||
metadata under `artifact_lanes[].metadata.extractor`. Durable raw output
|
||||
details belong in the
|
||||
[D&D spell raw output contract](../integrations/dnd-spell-artifacts.md).
|
||||
|
||||
The production catalog validates `dnd/spells` raw extract output with generic
|
||||
JSON validators followed by D&D spell validators under
|
||||
`internal/validators/extract/dnd/spells`. The extractor itself remains
|
||||
responsible for prompt, schema, and raw output production rather than
|
||||
approve/reject policy.
|
||||
|
||||
The `dnd/scenes` chunker and `dnd/spells` extractor declare optional `players`,
|
||||
`party`, and `glossary` reference slots accepting UTF-8 plain text, Markdown,
|
||||
YAML, or JSON. They also accept `roster` as a deprecated compatibility alias for
|
||||
`party`. Their prompts frame references as supporting disambiguation material
|
||||
only; spell-cast artifacts must still be grounded in the source transcript.
|
||||
|
||||
## `appendorder` Merger
|
||||
|
||||
Package: `internal/modules/merge/appendorder`
|
||||
|
||||
The `appendorder` merger preserves chunk order for raw extract outputs. A
|
||||
single JSON extract output is passed through as the merge output. Multiple JSON
|
||||
object outputs with one common top-level array field are merged by concatenating
|
||||
that array field in chunk order. Other valid JSON shapes are merged as a JSON
|
||||
array of decoded values in chunk order. Non-JSON media types and invalid JSON
|
||||
are rejected.
|
||||
|
||||
Provides:
|
||||
|
||||
- `merged`
|
||||
|
||||
## `noop` Normalizer
|
||||
|
||||
Package: `internal/modules/normalize/noop`
|
||||
|
||||
The `noop` normalizer clones the raw merge output and returns it unchanged.
|
||||
|
||||
Requires:
|
||||
|
||||
- `merged`
|
||||
|
||||
Provides:
|
||||
|
||||
- `normalized`
|
||||
|
||||
## `json` Output
|
||||
|
||||
Package: `internal/modules/output/json`
|
||||
|
||||
The `json` output encoder converts normalized raw outputs, rejected raw outputs,
|
||||
warnings, and the run manifest into logical JSON output files. It writes one
|
||||
payload file per lane under `lanes/` and sanitizes lane IDs for file names.
|
||||
Normalized output payloads must be valid `application/json`.
|
||||
|
||||
Requires:
|
||||
|
||||
- `normalized`
|
||||
|
||||
Provides:
|
||||
|
||||
- `encoded`
|
||||
|
||||
Durable output file shapes belong in the
|
||||
[JSON output contract](../integrations/json-output.md). Operator behavior
|
||||
belongs in [Operations](../operations.md).
|
||||
These validators are deterministic. Their selectable keys and production order
|
||||
are defined in
|
||||
[Configuration](../config.md#implemented-production-validators); their durable
|
||||
payload rules are defined in the
|
||||
[artifact contract](../integrations/dnd-spell-artifacts.md).
|
||||
|
||||
## Production Registration
|
||||
|
||||
Production registration is centralized in `internal/cli/catalog.go`.
|
||||
`internal/cli/catalog.go` builds the production registries, registers module and
|
||||
validator constructors, installs default validator-chain mappings, and exposes
|
||||
the matching catalog for resolution. It also collects prompt assets from
|
||||
LLM-backed packages before constructing the production client.
|
||||
|
||||
Do not make framework code import production modules. The CLI wires production
|
||||
modules at the application boundary; tests may provide fake registries or fake
|
||||
catalogs directly.
|
||||
Framework packages must not import production extensions. Tests may compose
|
||||
registries and catalogs directly with fakes.
|
||||
|
||||
## Adding A Module
|
||||
## Adding An Extension
|
||||
|
||||
When adding a module, keep source-format and extraction-domain boundaries clear:
|
||||
When adding a production module or validator:
|
||||
|
||||
- input modules may know external source formats;
|
||||
- extract modules may know artifact semantics and prompt/schema assets;
|
||||
- merge and normalize modules own raw output combination and reconciliation;
|
||||
- output modules own serialization, not diagnostics or CLI reporting.
|
||||
1. implement the stage or validator contract and package-local key;
|
||||
2. expose and test its spec, constructor, and registration function;
|
||||
3. keep format or domain parsing inside the concrete package;
|
||||
4. add package-owned prompt/schema assets when the extension is LLM-backed;
|
||||
5. register it in `internal/cli/catalog.go` and add a default chain only when
|
||||
production policy requires one;
|
||||
6. add resolution and composition coverage for capabilities, options,
|
||||
references, and validation behavior;
|
||||
7. update the selectable-key catalog in [Configuration](../config.md), the
|
||||
relevant external contract, this inventory, and maintained examples when
|
||||
user-visible behavior changes.
|
||||
|
||||
Update [Development](../development.md), [Configuration](../config.md),
|
||||
internal docs, integration docs, and examples when the new module becomes
|
||||
implemented production behavior.
|
||||
Do not add the extension to `docs/development.md`; that file routes by task and
|
||||
does not inventory implementations.
|
||||
|
||||
## Tests To Inspect
|
||||
|
||||
- Package-local `*_test.go` files under the module or validator being changed.
|
||||
- `internal/framework/pipeline/registry_integration_test.go`: registry and spec
|
||||
composition.
|
||||
- `internal/framework/pipeline/default_modules_test.go`: framework binding
|
||||
defaults.
|
||||
- `internal/cli/run_test.go`: production catalog, config resolution, and
|
||||
end-to-end CLI composition.
|
||||
- `internal/modules/sharedassets/**/*_test.go`: shared prompt and reference
|
||||
assembly.
|
||||
|
||||
@@ -1,127 +1,105 @@
|
||||
# Internal Overview
|
||||
|
||||
This document maps the implemented Notarius components and their ownership. It
|
||||
complements the durable invariants in [Architecture](../policy/architecture.md)
|
||||
and links to focused internal documentation for deeper behavior.
|
||||
This document inventories the implemented Notarius components. Normative
|
||||
boundaries and dependency direction belong in
|
||||
[Architecture](../policy/architecture.md); external behavior belongs in the
|
||||
[CLI](../cli.md), [Configuration](../config.md),
|
||||
[Operations](../operations.md), and [integration contracts](../integrations/).
|
||||
|
||||
## Execution Path
|
||||
|
||||
The executable delegates to the CLI, which resolves configuration and wires the
|
||||
production application around the framework runner:
|
||||
`cmd/notarius` delegates to `internal/cli`, the production composition root.
|
||||
The CLI loads configuration, builds the production catalogs and runtime
|
||||
collaborators, invokes `internal/framework/pipeline`, and places the logical
|
||||
output files returned by the runner. Diagnostics, checkpoints, and debug
|
||||
recorders are optional side-channel collaborators supplied at this boundary.
|
||||
|
||||
```text
|
||||
cmd/notarius
|
||||
-> internal/cli
|
||||
-> config resolution + production registries + LLM client
|
||||
-> input -> chunk -> extract -> merge -> normalize -> output
|
||||
-> durable output writes
|
||||
|
||||
Pipeline side channels:
|
||||
diagnostics checkpoints debug artifacts
|
||||
```
|
||||
|
||||
Pipeline execution is serial. Configuration selects modules for the fixed stage
|
||||
shape; registries construct them after profile, capability, validator, and
|
||||
reference resolution.
|
||||
Pipeline execution is serial. Resolution produces a fixed ordered workflow and
|
||||
a sorted set of artifact lanes before the runner constructs any stage module.
|
||||
|
||||
## Application Boundary
|
||||
|
||||
`cmd/notarius` contains the executable entry point and delegates process exit
|
||||
behavior to `internal/cli`.
|
||||
|
||||
`internal/cli` owns command parsing, configuration discovery, production module
|
||||
and validator registration, prompt asset collection, production LLM client
|
||||
construction, reference preparation, workspace recorder setup, durable output
|
||||
writes, and user-facing stdout, stderr, and exit codes. It is the composition
|
||||
root for concrete production packages.
|
||||
| Package | Implemented responsibility |
|
||||
| --- | --- |
|
||||
| `cmd/notarius` | Executable entry point and process exit delegation. |
|
||||
| `internal/cli` | Command parsing, config discovery, production registration, prompt asset collection, LLM client construction, reference materialization, workspace collaborator setup, durable writes, and user-facing results. |
|
||||
|
||||
## Core Packages
|
||||
|
||||
| Package | Implemented responsibility |
|
||||
| --- | --- |
|
||||
| `internal/core/artifacts` | Run manifests and artifact serialization shapes. |
|
||||
| `internal/core/config` | Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline configuration. |
|
||||
| `internal/core/diagnostics` | Diagnostics run directories, artifact writers, atomic writes, and retention decisions. |
|
||||
| `internal/core/source` | Generic source documents, units, references, and validation. |
|
||||
| `internal/core/workspace` | Workspace settings, safe paths and writes, checkpoint identities, and checkpoint manifest types. |
|
||||
|
||||
These packages provide concrete, deterministic models and policy. Production
|
||||
module registration occurs at the CLI boundary rather than in core packages.
|
||||
| `internal/core/artifacts` | Run-manifest and provenance models. |
|
||||
| `internal/core/config` | Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline resolution. |
|
||||
| `internal/core/diagnostics` | Scoped run directories, diagnostics writers, atomic writes, and retention decisions. |
|
||||
| `internal/core/source` | Generic source documents, units, references, lookup, and validation. |
|
||||
| `internal/core/workspace` | Effective workspace settings, confined paths and writes, checkpoint identity, and checkpoint manifest models. |
|
||||
|
||||
## Framework Packages
|
||||
|
||||
| Package | Implemented responsibility |
|
||||
| --- | --- |
|
||||
| `internal/framework/contracts` | Stage, validator, reference, output, and structured LLM interfaces and request/result types. |
|
||||
| `internal/framework/pipeline` | Module registries, profile resolution, capability checks, reference materialization, validation chains, retries, orchestration, warnings, and manifest population. |
|
||||
| `internal/framework/contracts` | Stage, validator, reference, output, and structured-completion interfaces and data types. |
|
||||
| `internal/framework/pipeline` | Registries, profile resolution, capability checks, reference materialization, validator-chain resolution, retries, orchestration, warnings, and manifest population. |
|
||||
| `internal/framework/validate` | Shared validator decision and cardinality helpers. |
|
||||
| `internal/framework/llm` | Scriptorium-backed structured completions, prompt/schema asset registration, scheduling, profile recording, and secret redaction. |
|
||||
| `internal/framework/checkpoint` | Workspace-backed checkpoint loading, recording, and payload envelopes. |
|
||||
| `internal/framework/debug` | Workspace-backed framework and LLM debug artifacts. |
|
||||
| `internal/framework/llm` | Scriptorium-backed structured completions, prompt/schema registration, scheduling, profile recording, and secret redaction. |
|
||||
| `internal/framework/checkpoint` | Workspace-backed checkpoint loading, recording, and payload serialization. |
|
||||
| `internal/framework/debug` | Workspace-backed framework and LLM debug recording. |
|
||||
|
||||
Framework contracts carry raw stage outputs between modules. The runner owns
|
||||
provenance, validation sequencing, rejection handling, checkpoint boundaries,
|
||||
debug boundaries, and final manifest assembly.
|
||||
Framework contracts carry raw stage results between implementations. The
|
||||
runner owns handoff provenance, validation sequencing, rejection handling,
|
||||
checkpoint and debug boundaries, and final manifest assembly.
|
||||
|
||||
## Production Modules
|
||||
## Production Extensions
|
||||
|
||||
Production implementations live under `internal/modules` and register through
|
||||
the CLI catalog.
|
||||
The canonical catalogs of user-selectable
|
||||
[module](../config.md#implemented-production-modules) and
|
||||
[validator](../config.md#implemented-production-validators) keys are in
|
||||
Configuration. The implemented module packages are:
|
||||
|
||||
| Stage | Module key | Package | Role |
|
||||
| --- | --- | --- | --- |
|
||||
| Input | `seriatim` | `internal/modules/input/seriatim` | Converts Seriatim transcript JSON into the generic source model. |
|
||||
| Chunk | `generic` | `internal/modules/chunk/generic` | Splits ordered source units by configured unit counts and overlap. |
|
||||
| Chunk | `dnd/scenes` | `internal/modules/chunk/dnd/scenes` | Uses structured LLM output to create contiguous D&D scene chunks. |
|
||||
| Extract | `dnd/spells` | `internal/modules/extract/dnd/spells` | Extracts source-grounded D&D spell-cast artifacts. |
|
||||
| Merge | `appendorder` | `internal/modules/merge/appendorder` | Combines accepted extract outputs in chunk order. |
|
||||
| Normalize | `noop` | `internal/modules/normalize/noop` | Preserves accepted merged output unchanged. |
|
||||
| Output | `json` | `internal/modules/output/json` | Encodes manifests, indexes, warnings, rejections, and accepted lane payloads as logical JSON files. |
|
||||
| Package | Implemented responsibility |
|
||||
| --- | --- |
|
||||
| `internal/modules/input/seriatim` | Parses the supported Seriatim transcript format into the generic source model. |
|
||||
| `internal/modules/chunk/generic` | Splits ordered source units by unit count and overlap. |
|
||||
| `internal/modules/chunk/dnd/scenes` | Produces contiguous D&D scene chunks from structured model output. |
|
||||
| `internal/modules/extract/dnd/spells` | Produces source-grounded D&D spell-cast raw output. |
|
||||
| `internal/modules/merge/appendorder` | Combines accepted extraction results in chunk order. |
|
||||
| `internal/modules/normalize/noop` | Preserves accepted merged output. |
|
||||
| `internal/modules/output/json` | Encodes manifests, lane payloads, warnings, and rejections as logical JSON files. |
|
||||
|
||||
`internal/modules/sharedassets` composes shared prompt filesystems.
|
||||
`internal/modules/sharedassets/dnd` owns shared D&D prompt fragments, reference
|
||||
slots, prompt input assembly, and source-unit reference helpers.
|
||||
`internal/modules/sharedassets/dnd` owns reusable D&D prompt fragments,
|
||||
reference declarations, prompt input assembly, and source-unit reference
|
||||
helpers.
|
||||
|
||||
## Validators
|
||||
Concrete validators live under `internal/validators`. Generic packages provide
|
||||
unconditional test decisions, JSON syntax validation, and JSON Schema
|
||||
validation. D&D spell packages provide shape, source-reference, and
|
||||
source-relatedness decisions, with `spellpayload` holding their shared parser
|
||||
and lookup helpers. Production chain composition is owned by `internal/cli`.
|
||||
|
||||
Concrete validators live under `internal/validators` and register separately
|
||||
from stage modules. Generic validators cover unconditional test decisions, JSON
|
||||
syntax, and JSON Schema. D&D spell validators cover artifact shape, source
|
||||
reference validity, and source relatedness.
|
||||
Implementation details for all production extensions are in
|
||||
[Module Internals](modules.md).
|
||||
|
||||
The production default chain for `dnd/spells` extract output is registered
|
||||
centrally in `internal/cli`; module packages produce raw output but do not own
|
||||
the production approve/reject policy.
|
||||
## Run-State Components
|
||||
|
||||
## Files And Run State
|
||||
|
||||
Notarius keeps distinct output and inspection surfaces:
|
||||
|
||||
| Surface | Owner | Purpose |
|
||||
| Surface | Implemented owners | Internal purpose |
|
||||
| --- | --- | --- |
|
||||
| Durable output | Output module and CLI writer | User-consumable run files. |
|
||||
| Diagnostics | `internal/core/diagnostics` and CLI | Redacted run inspection, reports, warnings, and failures. |
|
||||
| Checkpoints | `internal/framework/checkpoint` | Validated stage reuse for explicit resume. |
|
||||
| Debug artifacts | `internal/framework/debug` and pipeline instrumentation | Sensitive framework-boundary and LLM call inspection. |
|
||||
| Durable output | Output module, pipeline runner, and CLI writer | Return logical consumer files and place them for a run. |
|
||||
| Diagnostics | `internal/core/diagnostics` and `internal/cli` | Record redacted invocation, resolution, result, and failure inspection data. |
|
||||
| Checkpoints | `internal/framework/checkpoint` and `internal/core/workspace` | Validate and serialize reusable stage outcomes. |
|
||||
| Debug artifacts | `internal/framework/debug` and pipeline instrumentation | Capture sensitive framework-boundary and LLM-call material. |
|
||||
|
||||
Workspace settings determine whether and where diagnostics, checkpoints, and
|
||||
debug artifacts are written. Concrete stage modules do not receive workspace
|
||||
paths.
|
||||
Physical layout, retention, recovery, and sensitive-data handling are defined
|
||||
in [Operations](../operations.md). Concrete stage modules receive recorder
|
||||
interfaces and request data, not workspace paths.
|
||||
|
||||
## Focused Internal Documentation
|
||||
## Focused Documentation
|
||||
|
||||
- [Pipeline Internals](pipeline.md): resolution, execution, validators,
|
||||
references, retries, checkpoints, outputs, and manifests.
|
||||
- [Module Internals](modules.md): production module contracts, capabilities,
|
||||
options, prompts, schemas, and registration.
|
||||
- [Pipeline Internals](pipeline.md): resolution, execution, validation, retries,
|
||||
checkpoint/debug hooks, and result assembly.
|
||||
- [Module Internals](modules.md): production modules, validators, assets,
|
||||
registration, and the contributor recipe for adding an extension.
|
||||
- [LLM Runtime](llm.md): structured completion contracts, Scriptorium adapter,
|
||||
assets, scheduling, profile recording, and redaction.
|
||||
- [Diagnostics Internals](diagnostics.md): diagnostics files, retention,
|
||||
failure behavior, and path safety.
|
||||
|
||||
## Test Surfaces
|
||||
|
||||
The repository uses focused package tests, registry and pipeline composition
|
||||
tests, a fake-backed walking skeleton, fixture-driven CLI coverage, and local
|
||||
test servers for LLM integration behavior. Tests do not require real provider
|
||||
calls.
|
||||
- [Diagnostics Internals](diagnostics.md): scoped writers, retention
|
||||
coordination, CLI failure flow, and path safety.
|
||||
|
||||
@@ -1,295 +1,185 @@
|
||||
# Pipeline Internals
|
||||
|
||||
The implemented pipeline runner lives in `internal/framework/pipeline`. It
|
||||
executes the fixed workflow defined by the architecture policy:
|
||||
The implemented resolver and runner live in `internal/framework/pipeline`.
|
||||
Their fixed workflow and ownership boundaries are defined by
|
||||
[Architecture](../policy/architecture.md#system-shape). Configuration fields,
|
||||
defaults, and selectable keys are defined in
|
||||
[Configuration](../config.md#pipelines).
|
||||
|
||||
```text
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
Pipeline execution is serial. Resolution fixes the selected lanes and all
|
||||
stage bindings before the runner constructs stage implementations.
|
||||
|
||||
Pipeline execution is serial. The runner executes the resolved lanes one after
|
||||
another in the fixed workflow order.
|
||||
## Resolution
|
||||
|
||||
## Profile Resolution
|
||||
`internal/core/config.Config.Resolve` validates the loaded configuration,
|
||||
selects the named profile, applies the runtime inputs supplied by the CLI, and
|
||||
calls `pipeline.ResolvePipeline`.
|
||||
|
||||
Config loading produces `pipeline.PipelineProfile` values. Resolution happens
|
||||
before execution:
|
||||
`ResolvePipeline`:
|
||||
|
||||
1. `internal/core/config.Config.Resolve` validates config and finds the named
|
||||
pipeline.
|
||||
2. The optional lane selection is passed to `pipeline.ResolvePipeline`.
|
||||
3. Module bindings are defaulted:
|
||||
- chunk: `generic`
|
||||
- merge: `appendorder`
|
||||
- normalize: `noop`
|
||||
- output: `json`
|
||||
- LLM profile: empty, which lets Scriptorium prompt defaults choose a
|
||||
profile.
|
||||
4. The module catalog is checked for each bound module key.
|
||||
5. Module capabilities are checked in workflow order.
|
||||
6. A digest is calculated from the resolved pipeline without the digest field.
|
||||
1. selects and sorts artifact lanes;
|
||||
2. completes omitted bindings using the documented configuration defaults;
|
||||
3. looks up each module and validator spec without constructing it;
|
||||
4. checks required and provided capabilities in workflow order;
|
||||
5. resolves target-aware reference bindings and validator chains;
|
||||
6. calculates a digest over the resolved structure.
|
||||
|
||||
The CLI writes the resolved pipeline and digest to diagnostics.
|
||||
Resolution returns a `ResolvedPipeline` containing ordered lanes, concrete
|
||||
bindings, validator chains, reference targets, and the digest. It does not read
|
||||
reference bytes or construct runtime modules. CLI lane and reference selector
|
||||
syntax is defined in the [CLI reference](../cli.md#run).
|
||||
|
||||
Pipeline profiles and artifact lanes may include reference binding maps keyed by
|
||||
reference slot name. During resolution, pipeline-level bindings act as defaults
|
||||
for selected chunk, extractor, merger, and normalizer targets that declare the
|
||||
slot; target-local bindings override or add bindings for that target. Runtime
|
||||
`--reference` requests override target config bindings, and runtime unbinds
|
||||
remove optional target bindings. Flat runtime slot names are resolved only when
|
||||
exactly one selected target declares the slot; otherwise the CLI requires a more
|
||||
specific selector such as `chunk.slot`, `lane.extract.slot`,
|
||||
`lane.merge.slot`, or `lane.normalize.slot`. Resolution validates bindings
|
||||
against the declaring target specs and stores the bindings in target-aware
|
||||
resolved reference holders. It does not read reference files or include
|
||||
reference bytes in source digests.
|
||||
## Reference Materialization
|
||||
|
||||
During run preparation, resolved file references for chunk, extractor, merger,
|
||||
and normalizer targets are materialized before any LLM-backed pipeline work. Config
|
||||
bindings resolve relative to the config file, and CLI bindings resolve relative
|
||||
to the current working directory. Materialization accepts UTF-8 text files,
|
||||
computes `sha256:` content digests, records file origins, infers canonical base
|
||||
media types from file extensions, enforces declared byte limits, and warns for
|
||||
empty bound files. Media-type acceptance is checked only when a slot declares
|
||||
`AcceptedMediaTypes`; unknown extensions are recorded as
|
||||
`application/octet-stream`. Reference content is omitted from diagnostics and
|
||||
manifests. The CLI writes provenance-only resolved reference diagnostics, and
|
||||
the run manifest records target-stage reference provenance separately from
|
||||
source digests. Runtime reference content is passed to the matching chunker,
|
||||
extractor, merger, or normalizer request. LLM-backed modules pass that material
|
||||
onward as named Scriptorium prompt inputs.
|
||||
The CLI calls `MaterializeReferences` after resolution and before constructing
|
||||
the LLM client or running the pipeline. The materializer checks each binding
|
||||
against its resolved target declaration, reads and validates the file, and
|
||||
builds both a `contracts.ReferenceSet` and provenance-only metadata on the
|
||||
corresponding `ResolvedReferenceTarget`.
|
||||
|
||||
The CLI carries raw input bytes into `pipeline.RunInput`. Input adapters parse
|
||||
those bytes into the source document. Chunk, merge, and normalize requests
|
||||
receive the original source material as `SourceInput`; extraction requests
|
||||
receive chunk-scoped source material built from the current `SourceChunk`
|
||||
content, media type, and origin metadata. The raw input payload is not written
|
||||
to manifests or default diagnostics.
|
||||
The runner clones the resulting set into the chunk, extract, merge, or normalize
|
||||
request that owns the target. LLM-backed extensions may convert those items into
|
||||
named prompt inputs. Reference content remains separate from source evidence and
|
||||
source digests.
|
||||
|
||||
The CLI also carries an optional run `session_id`. The runner makes it available
|
||||
to chunk, extract, merge, and normalize requests; LLM-backed modules forward it
|
||||
through their structured completion requests so Scriptorium can include it in
|
||||
prompt execution metadata.
|
||||
Binding precedence, path resolution, accepted content, and media-type behavior
|
||||
are configuration contracts; see [Configuration](../config.md#pipelines).
|
||||
Durable provenance is defined in the
|
||||
[JSON output contract](../integrations/json-output.md#manifestjson), while
|
||||
runtime sensitive-data handling belongs in [Operations](../operations.md).
|
||||
|
||||
When workspace resume checkpointing is enabled, the CLI constructs a checkpoint
|
||||
recorder after pipeline resolution and reference materialization and passes it
|
||||
through `pipeline.RunInput`. The runner records source, chunk, extract, merge,
|
||||
and normalize outcomes through that interface. Concrete modules do not receive
|
||||
workspace paths and do not write checkpoint files directly.
|
||||
## Registries And Specs
|
||||
|
||||
For `run --resume`, the CLI also passes a checkpoint loader. The runner consults
|
||||
the loader in workflow order and reuses only checkpoints whose manifest schema,
|
||||
status, identity digest, dependency fingerprints, payload files, and payload
|
||||
digests validate for the current invocation. The identity includes the resolved
|
||||
pipeline, selected lanes, source/input digest, runtime overrides that affect
|
||||
execution, and materialized reference digests. Missing or invalid checkpoints
|
||||
fall back to normal execution and are refreshed by the recorder.
|
||||
`pipeline.Registries` holds constructors used during execution.
|
||||
`pipeline.ModuleCatalog` exposes their specs during configuration validation and
|
||||
resolution. Separate registries exist for every stage and for validators;
|
||||
`ValidatorChainRegistry` stores production default-chain mappings.
|
||||
|
||||
When workspace debug output is enabled, the CLI passes a debug recorder for the
|
||||
current run ID. The runner writes framework-boundary inputs, outputs,
|
||||
structured LLM calls, validator calls, timing, and retry attempt metadata
|
||||
through that interface. Each retry or validator attempt records any LLM calls
|
||||
made within that attempt in an `llm_calls` array and writes paired
|
||||
`prompt-000N.json` and `response-000N.json` metadata files under the attempt
|
||||
directory. LLM response bodies are written as sibling `response-content-000N.*`
|
||||
files, using pretty-printed JSON when the content is valid JSON and raw text
|
||||
otherwise. Debug output is not used for resume and can contain sensitive source,
|
||||
reference, prompt, and model-output material. Concrete modules still do not
|
||||
receive workspace paths.
|
||||
A `ModuleSpec` declares its stage plus required and provided capabilities.
|
||||
Chunk, extract, merge, and normalize specs may also declare reference slots.
|
||||
Registry implementations defensively copy spec metadata, reject duplicate keys,
|
||||
and verify that a constructed implementation reports the registered key.
|
||||
|
||||
## Registries And Module Specs
|
||||
A `ValidatorSpec` declares a validator key and execution class. Resolution uses
|
||||
the execution class to reject incompatible profile bindings before execution.
|
||||
The current production catalog and default chain are listed only in
|
||||
[Configuration](../config.md#implemented-production-validators).
|
||||
|
||||
`pipeline.Registries` holds concrete constructors for execution. A
|
||||
`pipeline.ModuleCatalog` exposes module specs for config validation and
|
||||
resolution. The catalog also exposes validator specs and central default
|
||||
validator-chain mappings without constructing modules or validators.
|
||||
## Runner Boundary
|
||||
|
||||
Every production module registers a `ModuleSpec` with:
|
||||
`pipeline.RunInput` carries the resolved pipeline, raw source input, structured
|
||||
LLM client, run identity and timing, optional session and profile metadata, and
|
||||
checkpoint/debug collaborators. The runner parses source bytes through the
|
||||
selected input adapter. Later stage requests receive the generic source model;
|
||||
extract requests receive chunk-scoped input material, while chunk, merge, and
|
||||
normalize requests retain access to the original source material.
|
||||
|
||||
- `Key`: module key used in config;
|
||||
- `Stage`: module kind such as input, chunk, extract, merge, normalize,
|
||||
validate, or output;
|
||||
- `Provides`: capabilities added after that module runs;
|
||||
- `Requires`: capabilities that must already be available.
|
||||
`pipeline.RunOutput` carries the run manifest, accepted normalized results,
|
||||
rejected results, warnings, checkpoint events, and logical files returned by the
|
||||
output encoder. The CLI owns diagnostics and durable filesystem writes after the
|
||||
runner returns.
|
||||
|
||||
Chunk, extract, merge, and normalize specs may also declare reference slots. Slot
|
||||
declarations are available from registry metadata without constructing module
|
||||
instances. Input, validate, and output specs must not declare reference slots.
|
||||
|
||||
Capability checks prevent incompatible pipeline composition before a run starts.
|
||||
|
||||
Every production validator registers a `ValidatorSpec` with:
|
||||
|
||||
- `Key`: validator key used in config and manifests;
|
||||
- `ExecutionClass`: `deterministic` or `llm_backed`.
|
||||
|
||||
Default validator chains are keyed by workflow stage and module key. Production
|
||||
currently registers a default chain for `extract` module `dnd/spells` only.
|
||||
|
||||
## Runner Input And Output
|
||||
|
||||
`pipeline.RunInput` carries:
|
||||
|
||||
- a `ResolvedPipeline`;
|
||||
- optional source ID, input path, and raw input bytes;
|
||||
- a structured LLM client;
|
||||
- run ID, start time, LLM profile manifest metadata, and CLI metadata.
|
||||
|
||||
`pipeline.RunOutput` carries:
|
||||
|
||||
- run manifest;
|
||||
- normalized raw outputs;
|
||||
- rejected raw outputs;
|
||||
- warnings;
|
||||
- logical output files returned by the output encoder.
|
||||
|
||||
The CLI owns durable file writes and diagnostics writes after the runner returns.
|
||||
|
||||
## Execution
|
||||
## Execution Flow
|
||||
|
||||
The runner:
|
||||
|
||||
1. validates run input and registries;
|
||||
2. builds the input adapter and parses the raw input into a source document;
|
||||
3. validates the source document;
|
||||
4. builds the chunker and produces source chunks, retrying when configured;
|
||||
5. validates source chunks against framework invariants and the resolved chunk
|
||||
validator chain;
|
||||
6. runs each selected artifact lane in sorted resolved order;
|
||||
7. builds the output encoder and validates logical output file names.
|
||||
8. passes accepted normalized raw outputs, rejected output records, warnings,
|
||||
and the manifest to the output encoder.
|
||||
1. validates its input and registries;
|
||||
2. builds the input adapter, parses the raw input, and validates the generic
|
||||
source document;
|
||||
3. obtains or executes the chunk result;
|
||||
4. validates and canonicalizes chunks;
|
||||
5. executes each resolved artifact lane in order;
|
||||
6. builds the output encoder and validates its logical file results;
|
||||
7. returns the assembled manifest, outcomes, warnings, and files.
|
||||
|
||||
## Chunk Results
|
||||
Within each artifact lane, it builds the extractor, merger, and normalizer,
|
||||
then performs these transitions:
|
||||
|
||||
Chunkers implement `contracts.Chunker` and receive a `contracts.ChunkRequest`
|
||||
with the validated source document, reference set, structured LLM client, the
|
||||
configured LLM profile, module options, and run metadata. Deterministic and
|
||||
LLM-backed chunkers use the same contract; provider construction stays outside
|
||||
chunk modules.
|
||||
1. extract once per accepted chunk and add runner-owned lane, source, and chunk
|
||||
provenance;
|
||||
2. validate each raw extract result and omit rejected results from merge input;
|
||||
3. skip the rest of the lane when no extract result is accepted;
|
||||
4. merge accepted extract results in their existing order;
|
||||
5. validate the merge result and skip normalization on rejection;
|
||||
6. normalize the accepted merge result;
|
||||
7. validate and append the accepted normalized result.
|
||||
|
||||
When chunking succeeds, the runner validates generic chunk invariants before
|
||||
running extractors:
|
||||
Module-provided warnings and payload warnings are promoted only from attempts
|
||||
whose results are accepted and used.
|
||||
|
||||
- chunk IDs must be non-empty and unique in the chunk result;
|
||||
- each chunk `SourceID` must match the source document ID;
|
||||
- each chunk `Index` must match its zero-based returned order;
|
||||
- each chunk start and end unit ID must exist in the source document, with the
|
||||
start unit at or before the end unit;
|
||||
- each chunk must include non-empty extraction content and media type;
|
||||
- each chunk must contain at least one source unit;
|
||||
- a chunk must not repeat a source unit;
|
||||
- every chunk source unit must exist in the source document;
|
||||
- source units inside each chunk must appear in source-document order.
|
||||
## Chunk Canonicalization
|
||||
|
||||
After validation, the runner rebuilds each chunk from source-document units by
|
||||
integer ID, preserving chunk boundaries, content bytes, media type, and cloned
|
||||
chunk metadata. Extractors and downstream stages therefore see canonical source
|
||||
units, while `SourceChunk.Metadata` remains the supported place for
|
||||
chunker-owned context.
|
||||
Before lane execution, generic validation requires unique chunk IDs, matching
|
||||
source identity, indexes matching returned order, valid ordered boundaries,
|
||||
non-empty content and media type, and at least one valid source unit per chunk.
|
||||
Units may not repeat inside a chunk and must preserve source-document order.
|
||||
|
||||
If chunk validation rejects the chunk result after configured retries, the runner
|
||||
records a rejected raw output and skips downstream lane execution. Framework-level
|
||||
chunking or validation errors that remain after configured retries fail the run.
|
||||
The runner then rebuilds each chunk's unit slice from the source document by
|
||||
unit ID. It preserves the module-owned boundaries, content, media type, and
|
||||
cloned metadata. The framework permits gaps and overlap between separate
|
||||
chunks; stricter coverage policy belongs to the chunk implementation.
|
||||
|
||||
The framework does not require complete source-unit coverage and does not reject
|
||||
overlap between different chunks. Stricter policies, such as full coverage or
|
||||
non-overlap, belong to individual chunk modules when they are part of that
|
||||
module's contract.
|
||||
## Validation And Retries
|
||||
|
||||
Within an artifact lane, the runner:
|
||||
Chunk, extract, merge, and normalize results pass through the resolved validator
|
||||
chain for their stage and module. Each validator receives the raw payload plus
|
||||
the relevant source, chunk, prior-stage, schema, reference, session, LLM, option,
|
||||
and run context. Validators execute in resolved order and stop at the first
|
||||
error or rejection. An empty chain approves the result.
|
||||
|
||||
1. builds the extractor, merger, and normalizer;
|
||||
2. records module manifest metadata when modules provide it;
|
||||
3. extracts one raw `ExtractOutput` from each accepted chunk, retrying when
|
||||
configured;
|
||||
4. fills runner-owned provenance on each extract output, including lane ID,
|
||||
extractor key, source ID, chunk ID, and chunk index;
|
||||
5. validates raw extract outputs and omits rejected outputs from merge input;
|
||||
6. merges ordered accepted extract outputs into one raw `MergeOutput`, retrying
|
||||
when configured;
|
||||
7. validates raw merge output and skips normalization for rejected merge output;
|
||||
8. normalizes the accepted merge output into one raw `NormalizeOutput`,
|
||||
retrying when configured;
|
||||
9. validates raw normalize output and appends accepted normalized raw output to
|
||||
`RunOutput.NormalizeOutputs`.
|
||||
`runWithRetry` applies the effective retry policy around module execution and
|
||||
its complete validation chain. A module or validator error becomes a framework
|
||||
error when attempts are exhausted. A rejection becomes a recorded
|
||||
`RejectedOutput` when attempts are exhausted. Cancellation stops retry
|
||||
processing immediately.
|
||||
|
||||
## Validators
|
||||
Rejected output is a non-fatal pipeline outcome and does not advance. Warnings
|
||||
from discarded attempts are not promoted. Configuration owns retry counts and
|
||||
validator overrides; see [Module Bindings](../config.md#module-bindings).
|
||||
|
||||
The runner handoff is raw-output based. Chunkers, extractors, mergers, and
|
||||
normalizers do not advertise validator chains through their module interfaces.
|
||||
Resolved validation chains receive the raw module output plus stage, lane,
|
||||
module, source, chunk, schema, session, reference, LLM client/profile, binding
|
||||
option, and run metadata context. Chunk validators receive the chunk result
|
||||
collection, merge validators receive the ordered extract outputs used by the
|
||||
merge, and normalize validators receive the accepted merge output. Empty chains
|
||||
approve output by default. Response-schema provenance may include in-memory JSON
|
||||
schema bytes for validators. Those bytes are omitted from manifests,
|
||||
diagnostics, and encoded output files.
|
||||
## Checkpoint And Debug Hooks
|
||||
|
||||
Resolved validator chains come from central default mappings unless a
|
||||
stage-local config override is set on `chunk`, lane `extract`, lane `merge`, or
|
||||
lane `normalize`. Explicit empty overrides are valid and are recorded as empty
|
||||
chains in manifests. Explicit non-empty overrides replace the default chain and
|
||||
preserve configured order.
|
||||
The runner depends on recorder and loader interfaces, using no-op
|
||||
implementations when collaborators are absent. Each checkpointed workflow
|
||||
boundary records a running, succeeded, or failed transition. Reuse decisions
|
||||
are consulted in workflow order and accepted payloads are cloned before
|
||||
entering the normal handoff path. Dependency fingerprints connect later
|
||||
checkpoints to the exact accepted results on which they depend.
|
||||
|
||||
The production default chain for `extract` module `dnd/spells` is:
|
||||
Debug instrumentation wraps run, stage, attempt, validator, and structured LLM
|
||||
boundaries. Context scopes associate nested LLM calls with the module or
|
||||
validator attempt that made them. Debug-write failures are framework errors;
|
||||
debug data is never used as a checkpoint source.
|
||||
|
||||
1. `generic/valid_json`
|
||||
2. `generic/valid_json_schema`
|
||||
3. `extract/dnd/spells/shape`
|
||||
4. `extract/dnd/spells/source_refs`
|
||||
5. `extract/dnd/spells/source_relatedness`
|
||||
Checkpoint identity, physical layout, reuse behavior, and debug artifact
|
||||
handling are operator contracts in [Operations](../operations.md). Serialization
|
||||
and recorder implementation are inventoried in
|
||||
[Internal Overview](overview.md#run-state-components).
|
||||
|
||||
No other production module currently has a default validator chain.
|
||||
## Results And Failures
|
||||
|
||||
Validator rejection is a non-fatal run outcome: the rejected output is recorded
|
||||
in `RunOutput.Rejected` and does not pass to the next stage. Validator execution
|
||||
errors are framework-level errors and retry according to the relevant binding.
|
||||
Warning-only validators return approved results with warnings; those warnings
|
||||
are promoted only from successful attempts whose outputs are used.
|
||||
The runner owns manifest assembly and handoff summaries but not the durable JSON
|
||||
schema. It records resolved module and lane provenance, validator chains,
|
||||
source/reference identities, selected LLM profiles, normalized and rejected
|
||||
summaries, status, and timing. Raw payload bytes remain outside the manifest.
|
||||
Module metadata providers may add non-secret singleton or lane-scoped metadata.
|
||||
|
||||
## Warnings And Failures
|
||||
Execution errors include stage, module, lane, or validator context. Once a
|
||||
manifest exists, a failing run returns it with failed status and completion
|
||||
time. Successful status reflects whether any raw result was rejected. The
|
||||
durable manifest and logical file schemas are defined in the
|
||||
[JSON output contract](../integrations/json-output.md).
|
||||
|
||||
Warnings from the successful chunking, extraction, merging, and normalization
|
||||
attempts whose outputs are used are accumulated in `RunOutput.Warnings`, along
|
||||
with output encoder warnings. Warnings from discarded retry attempts are not
|
||||
promoted to final warnings.
|
||||
## Tests To Inspect
|
||||
|
||||
Errors wrap the operation and module key or lane context. If execution fails
|
||||
after a manifest exists, the returned manifest is marked `failed` and receives a
|
||||
completion timestamp.
|
||||
|
||||
On successful execution, the manifest validation status is:
|
||||
|
||||
- `approved` when no raw outputs were rejected;
|
||||
- `rejected` when at least one raw output was rejected.
|
||||
|
||||
## Manifest Population
|
||||
|
||||
The manifest records run ID, pipeline ID, pipeline digest, module keys, top-level
|
||||
module metadata, artifact lanes, LLM profile metadata, source digest,
|
||||
reference provenance, normalized raw output summaries, rejected output
|
||||
summaries, validation status, and timing. Raw output summaries include lane ID,
|
||||
normalizer module key, media type, source ID, and response-schema provenance
|
||||
when present. Rejected output summaries include stage, lane, module, chunk,
|
||||
validator or reason, message, attempt count, and optional diagnostic artifact
|
||||
path. The manifest does not include raw output payload bytes.
|
||||
|
||||
Singleton pipeline modules may add non-secret metadata by implementing
|
||||
`contracts.ManifestMetadataProvider`. The runner records that metadata under
|
||||
`module_metadata` with stable keys for `input`, `chunker`, and `output`.
|
||||
|
||||
Lane-owned modules may add non-secret metadata through
|
||||
`artifact_lanes[].metadata`. The runner records extractor, merger, and
|
||||
normalizer metadata there. The D&D spell extractor uses lane metadata for
|
||||
prompt and response-schema provenance.
|
||||
|
||||
## JSON Output
|
||||
|
||||
The production JSON output encoder writes `manifest.json`, `index.json`,
|
||||
`warnings.json`, `rejected.json`, and one pretty-printed JSON file per accepted
|
||||
normalized lane output under `lanes/`. It accepts only normalized outputs with
|
||||
valid `application/json` payloads. Unsupported media types, invalid JSON, unsafe
|
||||
logical paths, and duplicate sanitized lane file names fail the run before
|
||||
durable output files are written.
|
||||
- `internal/core/config/effective_config_test.go`: config-to-resolution boundary.
|
||||
- `internal/framework/pipeline/profile_test.go`: selection, defaults,
|
||||
capabilities, validator chains, and digest behavior.
|
||||
- `internal/framework/pipeline/references_test.go`: target resolution and
|
||||
materialization.
|
||||
- `internal/framework/pipeline/runner_test.go`: stage transitions, retries,
|
||||
rejections, warnings, checkpoints, debug hooks, and manifests.
|
||||
- `internal/framework/pipeline/walking_skeleton_test.go`: fake-backed complete
|
||||
workflow composition.
|
||||
- `internal/framework/checkpoint/*_test.go`: checkpoint serialization and reuse
|
||||
collaborators.
|
||||
|
||||
@@ -4,24 +4,16 @@ This is the canonical reference for operating implemented Notarius runs.
|
||||
|
||||
## Normal Run
|
||||
|
||||
A run reads one source file, resolves one configured pipeline, calls the
|
||||
configured Scriptorium-backed LLM runtime, writes durable JSON output, and
|
||||
writes diagnostics for inspection.
|
||||
A run reads one source file, resolves one configured pipeline, executes its
|
||||
modules, writes durable output, and writes diagnostics when enabled. Start with
|
||||
the [README quickstart](../README.md), then use the [CLI reference](cli.md) for
|
||||
invocation options.
|
||||
|
||||
```sh
|
||||
go run ./cmd/notarius run dnd-session \
|
||||
--config examples/dnd-spells.config.yml \
|
||||
--input examples/seriatim-minimal-transcript.json \
|
||||
--output-dir ./notarius-output \
|
||||
--diagnostics-dir /tmp/notarius
|
||||
```
|
||||
|
||||
The command prints a success line with the pipeline ID, normalized output count,
|
||||
rejected output count, and the output path.
|
||||
|
||||
For production, configure a workspace such as `/var/lib/notarius` and ensure the
|
||||
Notarius process can create files below it. For local development, prefer an
|
||||
ignored project-local workspace such as `./.notarius/workspace`.
|
||||
For production, configure an application-owned workspace such as
|
||||
`/var/lib/notarius` and ensure the Notarius process can create files below it.
|
||||
For local development, prefer an ignored project-local workspace such as
|
||||
`./.notarius/workspace`. See [Configuration](config.md#workspace) for workspace
|
||||
fields.
|
||||
|
||||
## Output Directory
|
||||
|
||||
@@ -31,23 +23,10 @@ Durable output is written to:
|
||||
<output-root>/<run-id>/
|
||||
```
|
||||
|
||||
The default output root is `./notarius-output`. Use `--output-dir` to choose a
|
||||
different root.
|
||||
|
||||
The `json` output module writes these files:
|
||||
|
||||
- `index.json`: file index with paths to the manifest, lane output files,
|
||||
rejected outputs, and warnings.
|
||||
- `manifest.json`: run manifest with resolved pipeline provenance, top-level
|
||||
module metadata, module keys, reference provenance, validation status, and
|
||||
timing.
|
||||
- `lanes/<lane-id>.json`: normalized raw JSON output payloads, one file per
|
||||
lane. For the current D&D spell extractor, this includes `lanes/spells.json`.
|
||||
- `rejected.json`: rejected raw output records.
|
||||
- `warnings.json`: warnings reported by pipeline modules or the output encoder.
|
||||
|
||||
Output writes are atomic per file. Logical output file names must be clean,
|
||||
relative, slash-separated paths and must not contain `..`.
|
||||
The output root and its invocation-specific override are defined in the
|
||||
[CLI reference](cli.md#run). Output writes are atomic per file. The
|
||||
[JSON output contract](integrations/json-output.md) defines the logical files,
|
||||
paths, schemas, and media types inside each run directory.
|
||||
|
||||
## Diagnostics Directory
|
||||
|
||||
@@ -57,17 +36,15 @@ Diagnostics are written under:
|
||||
<diagnostics-work-dir>/<run-id>/
|
||||
```
|
||||
|
||||
The default diagnostics work directory is `/tmp/notarius`. It can be set with
|
||||
`workspace.directory`, `NOTARIUS_WORKSPACE_DIR`, legacy
|
||||
`diagnostics.work_dir`, legacy `NOTARIUS_WORK_DIR`, or `--diagnostics-dir`.
|
||||
When a workspace directory is set, diagnostics are written under
|
||||
`<workspace.directory>/diagnostics/<run-id>/` unless `--diagnostics-dir`
|
||||
overrides the diagnostics work directory for that invocation.
|
||||
When a workspace directory is configured, diagnostics are written under
|
||||
`<workspace.directory>/diagnostics/<run-id>/`. An invocation-specific override
|
||||
changes only the diagnostics root, not the workspace root. Configuration and
|
||||
environment controls are defined in [Configuration](config.md); the override
|
||||
flag is defined in the [CLI reference](cli.md#run).
|
||||
|
||||
Set `workspace.diagnostics.enabled: false` or
|
||||
`NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED=false` to skip diagnostics directory
|
||||
creation and diagnostics artifact writes. Concise failures are still printed to
|
||||
stderr.
|
||||
Diagnostics can be disabled through configuration. When disabled, Notarius
|
||||
does not create a diagnostics run directory or write diagnostics artifacts;
|
||||
concise failures are still printed to stderr.
|
||||
|
||||
Implemented diagnostics artifacts:
|
||||
|
||||
@@ -87,45 +64,41 @@ Implemented diagnostics artifacts:
|
||||
- `error.log`: failure message, written after diagnostics directory creation
|
||||
when a run fails.
|
||||
|
||||
`source-document.json` is supported by the diagnostics writer but is not written
|
||||
by the current CLI run workflow.
|
||||
|
||||
## Checkpoints
|
||||
|
||||
When `workspace.resume.enabled: true` and `workspace.directory` is set, runs
|
||||
write checkpoints under:
|
||||
When checkpoint writing is enabled for a configured workspace, runs write
|
||||
checkpoints under:
|
||||
|
||||
```text
|
||||
<workspace.directory>/checkpoints/<pipeline-id>/<input-key>-<source-or-input-digest>/<pipeline-digest>/<identity-digest>/
|
||||
```
|
||||
|
||||
Each workflow step owns its own manifest and payload files. There is no
|
||||
root-level checkpoint summary. Ordinary `notarius run` invocations execute the
|
||||
pipeline normally and refresh checkpoints. `notarius run --resume` reuses valid
|
||||
root-level checkpoint summary. Ordinary invocations execute the pipeline
|
||||
normally and refresh checkpoints. An explicit resume invocation reuses valid
|
||||
checkpoints and executes any missing, invalid, or incompatible step normally.
|
||||
Configuration controls checkpoint writing, while the explicit resume option is
|
||||
defined in the [Configuration](config.md#workspace) and
|
||||
[CLI](cli.md#run) references.
|
||||
|
||||
Checkpoint payloads preserve byte content with base64 envelopes, media type,
|
||||
metadata, warnings, and content digests where applicable. Checkpoints do not
|
||||
include raw prompts, raw reference contents, raw LLM request payloads, or debug
|
||||
traces. They can still contain source text, intermediate extracted content,
|
||||
rejected outputs, metadata, and warnings. Treat checkpoint directories as
|
||||
sensitive local state.
|
||||
Checkpoints do not include raw prompts, raw reference contents, raw LLM request
|
||||
payloads, or debug traces. They can still contain source text, intermediate
|
||||
extracted content, rejected outputs, metadata, warnings, and content digests.
|
||||
Treat checkpoint directories as sensitive local state.
|
||||
|
||||
A checkpoint is reused only when its workspace schema version, checkpoint
|
||||
identity digest, step status, dependency fingerprints, payload files, and
|
||||
payload digests match the current invocation. Changes to input bytes, resolved
|
||||
pipeline digest, selected lanes, runtime LLM profile override, materialized
|
||||
reference digests, or other identity material invalidate reuse and use a
|
||||
separate checkpoint directory.
|
||||
A checkpoint is reused only when its stored status, dependencies, payloads, and
|
||||
digests match the current invocation. Changes to input bytes, the resolved
|
||||
pipeline, selected lanes, the runtime LLM profile override, or bound reference
|
||||
content invalidate reuse.
|
||||
|
||||
Plain `notarius run` does not reuse checkpoints. It executes the workflow and
|
||||
refreshes checkpoint files when checkpointing is enabled. `notarius run
|
||||
--resume` is the explicit reuse path.
|
||||
Runs do not reuse checkpoints unless explicitly requested. Without reuse, the
|
||||
workflow executes normally and refreshes checkpoint files when checkpointing is
|
||||
enabled.
|
||||
|
||||
## Debug
|
||||
|
||||
When `workspace.debug.enabled: true` and `workspace.directory` is set, runs
|
||||
write debug artifacts under:
|
||||
When debug recording is enabled for a configured workspace, runs write debug
|
||||
artifacts under:
|
||||
|
||||
```text
|
||||
<workspace.directory>/debug/<run-id>/
|
||||
@@ -135,34 +108,31 @@ Debug output is per invocation. It is independent of checkpointing and is not
|
||||
used for resume. Enabling debug does not write checkpoints, and enabling resume
|
||||
checkpointing does not write debug output.
|
||||
|
||||
Debug artifacts include framework-boundary inputs and outputs for source,
|
||||
chunk, extract, merge, normalize, and output work, structured LLM request and
|
||||
response data from Notarius contracts, validator requests and results, timing,
|
||||
and retry attempt metadata. LLM calls made inside a retry or validator attempt
|
||||
Debug artifacts include inputs and outputs for source, chunk, extract, merge,
|
||||
normalize, and output work, structured LLM request and response data, validator
|
||||
requests and results, timing, and retry attempt metadata. LLM calls made inside
|
||||
a retry or validator attempt
|
||||
write `prompt-000N.json`, `response-000N.json`, and
|
||||
`response-content-000N.*` files under that attempt directory and are linked from
|
||||
the attempt `llm_calls` array. Prompt content is written inline in the prompt
|
||||
artifact. Response metadata is written to `response-000N.json`, while the
|
||||
response body is written separately as pretty-printed JSON when possible or as
|
||||
raw text otherwise. Debug artifacts may contain source material, reference
|
||||
material, prompt inputs, model outputs, and other sensitive data. API keys are
|
||||
not written, and obvious credential-shaped values and sensitive map keys are
|
||||
redacted in framework envelopes, but debug directories should still be protected
|
||||
as sensitive local state.
|
||||
artifact. The response metadata and body use the paired files described above;
|
||||
the body is pretty-printed JSON when possible and raw text otherwise. Debug
|
||||
artifacts may contain source material, reference material, prompt inputs, model
|
||||
outputs, and other sensitive data. API keys are not written, and obvious
|
||||
credential-shaped values and sensitive map keys are redacted, but debug
|
||||
directories should still be protected as sensitive local state.
|
||||
|
||||
## Retention
|
||||
|
||||
Diagnostics retention is configured with `workspace.diagnostics.retention`,
|
||||
`NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION`, legacy `diagnostics.retention`,
|
||||
legacy `NOTARIUS_DIAGNOSTICS_RETENTION`, or the default `auto`.
|
||||
Diagnostics retention uses the effective mode selected through configuration;
|
||||
see [Configuration](config.md#diagnostics) for the fields, environment
|
||||
overrides, precedence, and default.
|
||||
|
||||
- `auto`: keep failed runs and successful runs with warnings; remove successful
|
||||
warning-free runs.
|
||||
- `always`: keep every diagnostics run directory.
|
||||
- `never`: remove successful run directories; failed runs are still retained.
|
||||
|
||||
Unknown retention values are rejected during config validation.
|
||||
|
||||
## Failures
|
||||
|
||||
Failures before diagnostics directory creation, such as a missing config file or
|
||||
@@ -170,24 +140,19 @@ an unusable diagnostics work directory, are printed to stderr and may not have a
|
||||
diagnostics run directory.
|
||||
|
||||
Failures after diagnostics directory creation are printed to stderr and written
|
||||
to `error.log`. Depending on where the failure occurred, diagnostics may also
|
||||
include invocation metadata, redacted effective config, resolved pipeline data,
|
||||
the run manifest, warnings, and a run report.
|
||||
to `error.log`. Depending on where the failure occurred, the directory may also
|
||||
contain artifacts written before the failure.
|
||||
|
||||
If durable output writing fails after the pipeline completes, diagnostics are
|
||||
retained for inspection and may include `run-manifest.json`, `warnings.json`,
|
||||
`run-report.json`, and `error.log`.
|
||||
retained for inspection.
|
||||
|
||||
## Warnings
|
||||
|
||||
A successful run with warnings exits with code `0`, prints a warning count to
|
||||
stderr, and writes warnings to durable output and diagnostics when retained.
|
||||
|
||||
The run manifest `validation_status` indicates whether raw outputs were
|
||||
approved or rejected after validation.
|
||||
|
||||
Reference-related warnings include empty bound reference files. Empty references
|
||||
are still passed to extractors so optional slots can be intentionally blank.
|
||||
The [JSON output contract](integrations/json-output.md) defines durable warning
|
||||
and validation-status fields.
|
||||
|
||||
## Cleanup
|
||||
|
||||
@@ -212,13 +177,10 @@ directories unless they are part of your own operational policy.
|
||||
|
||||
## Operational Limits
|
||||
|
||||
If `--resume` cannot reuse a checkpoint, Notarius executes that step and writes
|
||||
a fresh checkpoint when checkpointing is enabled.
|
||||
|
||||
Provider retries and timeouts are handled by Scriptorium according to the
|
||||
selected execution profile. Pipeline module retries are controlled by module
|
||||
binding `retries` values in config for chunk, extract, merge, and normalize.
|
||||
There is no separate CLI retry command.
|
||||
selected execution profile. Pipeline module retry settings are defined in
|
||||
[Configuration](config.md#module-bindings). There is no separate CLI retry
|
||||
command.
|
||||
|
||||
Notarius writes local files only. Remote storage and archive management are not
|
||||
part of the implemented CLI.
|
||||
|
||||
@@ -32,21 +32,17 @@ Notarius is contract-first without being abstraction-heavy. Interfaces and
|
||||
extension points should protect demonstrated boundaries. New abstraction is not
|
||||
itself an architectural goal.
|
||||
|
||||
## Package Layout And Dependency Direction
|
||||
## Layers And Dependency Direction
|
||||
|
||||
| Area | Ownership |
|
||||
| --- | --- |
|
||||
| `cmd/notarius` | Executable entry point; delegates to the CLI. |
|
||||
| `internal/cli` | Application boundary, production composition, runtime setup, durable writes, and user-facing results. |
|
||||
| `internal/core` | Generic deterministic models and policy for source material, configuration, manifests, diagnostics, and workspace identity. |
|
||||
| `internal/framework` | Reusable contracts, registries, pipeline orchestration, validation mechanics, checkpoints, debug boundaries, and LLM runtime plumbing. |
|
||||
| `internal/modules` | Concrete pipeline stage behavior. |
|
||||
| `internal/validators` | Concrete approve/reject policies. |
|
||||
The application boundary is the composition root and may depend on concrete
|
||||
implementations. Domain-neutral model and framework layers provide reusable
|
||||
policy, contracts, and orchestration. Concrete input, pipeline, output, and
|
||||
validation extensions depend inward on those generic layers.
|
||||
|
||||
The CLI is the composition root and may import concrete implementations. Core
|
||||
and framework packages cooperate as generic application layers; neither may
|
||||
depend on production modules or validators. Concrete implementations may depend
|
||||
on core models and framework contracts.
|
||||
Generic layers must not depend on production extensions. Concrete extensions
|
||||
must not compose the application or take ownership of process behavior. The
|
||||
current packages implementing these layers are inventoried in
|
||||
[Internal Overview](../internal/overview.md).
|
||||
|
||||
The following dependency boundaries are mandatory:
|
||||
|
||||
|
||||
@@ -1,130 +0,0 @@
|
||||
# Documentation Policy Migration
|
||||
|
||||
The documentation ownership policy has been revised to assign each topic to one
|
||||
canonical document. This roadmap organizes the likely migration work into three
|
||||
ordered passes.
|
||||
|
||||
This is a planning inventory, not a review of the current documents. Each item
|
||||
must be verified before content is moved, removed, or rewritten. Complete the
|
||||
passes in order so canonical content is established before non-owning copies are
|
||||
removed.
|
||||
|
||||
## Pass 1: User And Operator Contracts
|
||||
|
||||
Establish the canonical externally observable facts first. These documents will
|
||||
become the targets linked from developer and internal documentation in the next
|
||||
pass.
|
||||
|
||||
### Product And CLI
|
||||
|
||||
- Audit `README.md` so it owns product orientation and the single minimal
|
||||
end-to-end quickstart without becoming a command or configuration reference.
|
||||
- Audit `docs/cli.md` for material owned by the README, configuration reference,
|
||||
or operations guide.
|
||||
- Keep commands, arguments, flags, invocation semantics, and exit codes
|
||||
canonical in `docs/cli.md`.
|
||||
|
||||
### Configuration And Examples
|
||||
|
||||
- Audit `docs/config.md` for complete example files, CLI syntax, runtime state
|
||||
lifecycle, or implementation detail.
|
||||
- Keep fields, defaults, precedence, environment overrides, validation, and
|
||||
user-selectable module and validator keys canonical in `docs/config.md`.
|
||||
- Audit minimal and production-oriented configuration examples, moving complete
|
||||
copyable files under `examples/` and leaving only small illustrative fragments
|
||||
in reference documentation.
|
||||
- Inventory complete configuration, input, command, and output examples embedded
|
||||
in prose documents.
|
||||
- Select one maintained copy for each complete artifact under `examples/` where
|
||||
practical, and verify important examples through automated tests.
|
||||
|
||||
### Operations And Integrations
|
||||
|
||||
- Audit `docs/operations.md` for duplicated CLI syntax, configuration field
|
||||
definitions, logical output schemas, or implementation mechanics.
|
||||
- Keep runtime workflows, physical state, retention, recovery, permissions, and
|
||||
operational limits canonical in `docs/operations.md`.
|
||||
- Audit `docs/integrations/` so external formats, protocols, logical output
|
||||
paths, schemas, media types, and compatibility rules have one canonical home.
|
||||
- Separate logical output bundle contracts from physical runtime placement and
|
||||
lifecycle.
|
||||
|
||||
### Pass 1 Completion
|
||||
|
||||
- Confirm that README, CLI, configuration, operations, integrations, and
|
||||
examples have non-overlapping ownership.
|
||||
- Validate user-facing commands, fields, defaults, keys, schemas, paths, and
|
||||
maintained examples against implemented behavior.
|
||||
- Validate links among the user and operator documents.
|
||||
|
||||
## Pass 2: Developer And Internal Documentation
|
||||
|
||||
Use the canonical contracts established in Pass 1 to remove duplicated facts
|
||||
from contributor and implementation documentation.
|
||||
|
||||
### Orientation And Architecture
|
||||
|
||||
- Audit `docs/development.md` so it routes contributors without maintaining a
|
||||
parallel package inventory or architectural description.
|
||||
- Audit `docs/internal/overview.md` so it owns the implemented component map
|
||||
without restating normative architecture.
|
||||
- Audit `docs/policy/architecture.md` so it contains current normative
|
||||
architecture without implementation inventory, decision history, or future
|
||||
behavior.
|
||||
|
||||
### Internal Components
|
||||
|
||||
- Audit internal component documents for repeated configuration definitions,
|
||||
external input or output schemas, operator procedures, and global
|
||||
architectural invariants.
|
||||
- Replace duplicated external schemas or field definitions with links to the
|
||||
canonical configuration or integration contracts from Pass 1.
|
||||
- Keep implementation flow, internal collaborators, state transitions,
|
||||
package-local guarantees, failures, and relevant tests in focused internal
|
||||
documents.
|
||||
- Identify task-specific contributor recipes that need a focused internal home
|
||||
rather than the developer landing page.
|
||||
- Retain production module and validator implementation details in module
|
||||
internals while linking user-selectable keys to `docs/config.md`.
|
||||
|
||||
### Pass 2 Completion
|
||||
|
||||
- Confirm that development routes, architecture governs, internal overview
|
||||
inventories, and focused internal docs explain implementation.
|
||||
- Confirm that developer documents link to configuration, operations, and
|
||||
integration contracts rather than redefining them.
|
||||
- Validate developer-facing links and relevant focused tests.
|
||||
|
||||
## Pass 3: Lifecycle And Final Deduplication
|
||||
|
||||
Finish the migration by reconciling historical and future documentation, then
|
||||
perform a repository-wide ownership audit.
|
||||
|
||||
### ADR And Roadmap Lifecycle
|
||||
|
||||
- Audit roadmap files for implemented behavior or completed status summaries
|
||||
that should be removed or replaced with links to current canonical docs.
|
||||
- Ensure ADRs own architectural rationale and supersession history without
|
||||
becoming current behavior references or implementation trackers.
|
||||
- Ensure accepted but unimplemented ADR decisions link to roadmap-owned
|
||||
implementation status where appropriate.
|
||||
- Distinguish rejected architectural alternatives in ADRs from rejected product
|
||||
ideas in roadmap files.
|
||||
|
||||
### Repository-Wide Audit
|
||||
|
||||
- Inventory repeated commands, flags, defaults, module keys, validator keys,
|
||||
file names, paths, schemas, retry semantics, and runtime guarantees.
|
||||
- Assign each repeated fact to the canonical owner defined by the policy.
|
||||
- Replace non-owning copies with short summaries and links where navigation is
|
||||
useful.
|
||||
- Retain only minimal illustrative snippets in prose and link them to maintained
|
||||
examples.
|
||||
- Validate local links and remove references to deleted or relocated material.
|
||||
|
||||
### Pass 3 Completion
|
||||
|
||||
- Confirm that every contractual or volatile fact has one canonical owner.
|
||||
- Confirm that roadmap files contain future work and implementation status, ADRs
|
||||
contain decision rationale, and current docs contain implemented behavior.
|
||||
- Run final link, example, and documentation consistency checks.
|
||||
@@ -25,7 +25,7 @@ future work only.
|
||||
- Production LLM-backed validators when there is a concrete review policy that
|
||||
benefits from model judgment.
|
||||
- Validator diagnostics and timing summaries if operators need more detail than
|
||||
`manifest.json`, `rejected.json`, and `warnings.json` provide.
|
||||
the current [durable output bundle](../integrations/json-output.md) provides.
|
||||
- Media-type validators for non-JSON module outputs when such modules are
|
||||
introduced.
|
||||
- Validator compatibility metadata if real deployments need config-time
|
||||
@@ -42,8 +42,7 @@ future work only.
|
||||
- Optional generated example output fixtures with a regeneration procedure.
|
||||
- Additional diagnostics or reporting views if operator workflows need them.
|
||||
|
||||
## Non-Goals To Revisit Deliberately
|
||||
## Candidate Workspace Work
|
||||
|
||||
- A general workflow language.
|
||||
- Structural module selection through ad hoc run flags.
|
||||
- Storing secrets in config files, diagnostics, manifests, or examples.
|
||||
Workspace storage, cleanup, archival, and reuse candidates are tracked in the
|
||||
[Workspace Future Work](workspace.md) roadmap.
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
# Workspace Implementation Status
|
||||
|
||||
The workspace implementation described by this roadmap has landed. Current
|
||||
behavior is documented in the canonical current-behavior docs:
|
||||
|
||||
- [Configuration](../config.md)
|
||||
- [CLI Reference](../cli.md)
|
||||
- [Operations](../operations.md)
|
||||
- [Diagnostics Internals](../internal/diagnostics.md)
|
||||
- [Pipeline Internals](../internal/pipeline.md)
|
||||
|
||||
Implemented behavior includes:
|
||||
|
||||
- `workspace.directory` as the root for Notarius-owned local state;
|
||||
- workspace diagnostics under `<workspace.directory>/diagnostics/<run-id>/`;
|
||||
- compatibility for legacy `diagnostics.work_dir`, `diagnostics.retention`,
|
||||
`NOTARIUS_WORK_DIR`, and `NOTARIUS_DIAGNOSTICS_RETENTION`;
|
||||
- checkpoint writes under `<workspace.directory>/checkpoints/` when resume
|
||||
checkpointing is enabled;
|
||||
- explicit checkpoint reuse through `notarius run --resume`;
|
||||
- debug artifacts under `<workspace.directory>/debug/<run-id>/` when debug
|
||||
output is enabled;
|
||||
- independent resume and debug settings.
|
||||
|
||||
Deferred workspace ideas remain in [Workspace Roadmap](workspace.md).
|
||||
@@ -1,26 +1,14 @@
|
||||
# Workspace Roadmap Status
|
||||
# Workspace Future Work
|
||||
|
||||
The local workspace feature has been implemented. Current behavior is documented
|
||||
in [Configuration](../config.md), [CLI Reference](../cli.md),
|
||||
[Operations](../operations.md), and the relevant internal docs.
|
||||
Current workspace settings and operating behavior are documented in
|
||||
[Configuration](../config.md#workspace) and
|
||||
[Operations](../operations.md). This roadmap contains only candidate additions
|
||||
to that behavior.
|
||||
|
||||
The implemented workspace provides one configurable root for Notarius-owned
|
||||
local state:
|
||||
## Candidate Work
|
||||
|
||||
```text
|
||||
<workspace.directory>/
|
||||
diagnostics/
|
||||
checkpoints/
|
||||
debug/
|
||||
```
|
||||
|
||||
Implemented behavior includes workspace-backed diagnostics, checkpoint writing,
|
||||
explicit checkpoint reuse through `notarius run --resume`, workspace debug
|
||||
artifacts, safe workspace-relative writes, and compatibility for legacy
|
||||
diagnostics configuration.
|
||||
|
||||
## Deferred Work
|
||||
|
||||
Default-idempotent `run` behavior with a force override, remote workspace
|
||||
storage, workspace garbage collection, archival policy, and cross-machine resume
|
||||
remain deferred.
|
||||
- Default-idempotent run behavior with an explicit force override.
|
||||
- Remote workspace storage.
|
||||
- Workspace garbage collection.
|
||||
- Workspace archival policy.
|
||||
- Cross-machine checkpoint reuse.
|
||||
|
||||
25
examples/dnd-spells-production.config.yml
Normal file
25
examples/dnd-spells-production.config.yml
Normal file
@@ -0,0 +1,25 @@
|
||||
version: 2
|
||||
concurrency:
|
||||
total_llm: 1
|
||||
workspace:
|
||||
directory: /var/lib/notarius
|
||||
diagnostics:
|
||||
enabled: true
|
||||
retention: auto
|
||||
resume:
|
||||
enabled: false
|
||||
debug:
|
||||
enabled: false
|
||||
pipelines:
|
||||
dnd-session:
|
||||
input: seriatim
|
||||
references:
|
||||
party: ./dnd-spells-roster.txt
|
||||
glossary: ./dnd-spells-glossary.txt
|
||||
chunk:
|
||||
module: generic
|
||||
options:
|
||||
max_units: 50
|
||||
artifacts:
|
||||
spells:
|
||||
extract: dnd/spells
|
||||
@@ -1,29 +1,7 @@
|
||||
version: 2
|
||||
# For production runs, use a writable application-owned workspace such as:
|
||||
#
|
||||
# workspace:
|
||||
# directory: /var/lib/notarius
|
||||
# diagnostics:
|
||||
# retention: auto
|
||||
# resume:
|
||||
# enabled: false
|
||||
# debug:
|
||||
# enabled: false
|
||||
#
|
||||
# For local development, use a project-local ignored path such as:
|
||||
#
|
||||
# workspace:
|
||||
# directory: ./.notarius/workspace
|
||||
pipelines:
|
||||
dnd-session:
|
||||
input: seriatim
|
||||
references:
|
||||
party: ./dnd-spells-roster.txt
|
||||
glossary: ./dnd-spells-glossary.txt
|
||||
chunk:
|
||||
module: generic
|
||||
options:
|
||||
max_units: 50
|
||||
artifacts:
|
||||
spells:
|
||||
extract: dnd/spells
|
||||
|
||||
@@ -2647,23 +2647,29 @@ func TestRunPipelineDiagnosticsDirFlagOverridesWorkspaceDiagnosticsOnly(t *testi
|
||||
}
|
||||
|
||||
func TestExampleFixtureConfigValidateAndPipelinesList(t *testing.T) {
|
||||
configPath := fixturePath(t, "examples/dnd-spells.config.yml")
|
||||
for _, path := range []string{
|
||||
"examples/dnd-spells.config.yml",
|
||||
"examples/dnd-spells-production.config.yml",
|
||||
} {
|
||||
path := path
|
||||
t.Run("validate "+filepath.Base(path), func(t *testing.T) {
|
||||
configPath := fixturePath(t, path)
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
|
||||
t.Run("validate configured pipeline", func(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := RunWithOptions([]string{"config", "validate", "--config", configPath, "--pipeline", "dnd-session"}, &stdout, &stderr, Options{})
|
||||
|
||||
code := RunWithOptions([]string{"config", "validate", "--config", configPath, "--pipeline", "dnd-session"}, &stdout, &stderr, Options{})
|
||||
|
||||
if code != 0 {
|
||||
t.Fatalf("RunWithOptions() code = %d, stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "dnd-session") {
|
||||
t.Fatalf("stdout = %q, want pipeline ID", stdout.String())
|
||||
}
|
||||
})
|
||||
if code != 0 {
|
||||
t.Fatalf("RunWithOptions() code = %d, stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "dnd-session") {
|
||||
t.Fatalf("stdout = %q, want pipeline ID", stdout.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("list configured pipelines", func(t *testing.T) {
|
||||
configPath := fixturePath(t, "examples/dnd-spells.config.yml")
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
|
||||
|
||||
Reference in New Issue
Block a user