Compare commits

...

3 Commits

21 changed files with 838 additions and 1551 deletions

View File

@@ -14,9 +14,9 @@ go run ./cmd/notarius run dnd-session \
--input examples/seriatim-minimal-transcript.json
```
The maintained example uses Scriptorium's built-in `mistral-small-3` profile,
which reads `OPENROUTER_API_KEY`. Outputs are written under
`./notarius-output/<run-id>/` unless `--output-dir` is provided.
This invocation uses the maintained example configuration and input. See the
configuration and operations references for profile selection, credentials, and
run artifacts.
Useful references:

View File

@@ -3,27 +3,15 @@
This is the canonical reference for the implemented Notarius command-line
interface.
## Quick Run
```sh
OPENROUTER_API_KEY=... \
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json
```
The maintained example uses prompt defaults and Scriptorium's built-in
`mistral-small-3` profile, which reads `OPENROUTER_API_KEY`. To use another
endpoint or model, configure a Scriptorium profile source and select its profile
ID in config or with `--llm-profile`.
For the minimal end-to-end invocation, see the [README](../README.md).
## Commands
```text
notarius help
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--resume] [--session-id id] [--reference selector=path] [--without-reference selector]
notarius config validate --config path/to/config.yml [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list --config path/to/config.yml [--json]
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--output-dir path] [--diagnostics-dir path] [--llm-profile id] [--resume] [--session-id id] [--reference selector=path] [--without-reference selector]
notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list [--config path/to/config.yml] [--json]
```
Running `notarius` with no arguments, `notarius help`, `notarius --help`, or
@@ -37,20 +25,18 @@ file.
Flags:
- `--input path`: required source input file.
- `--config path`: config file path. If omitted, Notarius checks
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
- `--config path`: config file path. If omitted, Notarius uses the discovery
rules in [Configuration](config.md#discovery).
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are
comma-separated and must be non-empty.
- `--resume`: reuse valid workspace checkpoints for this invocation. Requires
an effective workspace directory and `workspace.resume.enabled: true`.
- `--output-dir path`: output root. The run writes to `<path>/<run-id>/`.
Defaults to `./notarius-output`.
- `--resume`: request checkpoint reuse for this invocation. See
[Operations](operations.md#checkpoints) for prerequisites and reuse behavior.
- `--output-dir path`: output root. Defaults to `./notarius-output`.
- `--diagnostics-dir path`: diagnostics work directory override for this
invocation. It does not change the workspace directory.
- `--llm-profile id`: override every effective LLM-capable pipeline module
binding to use one Scriptorium profile ID. Validator-specific profiles are
not overridden. Configured LLM-backed validators with explicit profiles are
validated against the configured Scriptorium profile source.
binding with one Scriptorium profile ID. Validator-specific profiles are not
overridden.
- `--session-id id`: pass a stable prompt session identifier through LLM-backed
module calls.
- `--reference selector=path`: bind a reference path to a chunk, extractor,
@@ -63,23 +49,11 @@ On success, the command prints the completed pipeline ID, normalized output and
rejected output counts, and the output directory. If the run completes with warnings,
the warning count is printed to stderr.
Reference flags are resolved against selected chunk, extractor, merger, and normalizer
targets before the run starts. Flat slot names are accepted only when exactly
one selected target declares that slot. Bound reference files are read before
pipeline work starts, validated as UTF-8 text, and recorded as provenance for
the target that declares the slot. Runtime reference content is passed to the
chunker, extractor, merger, or normalizer target that declares the slot. Notarius infers
reference media types from file extensions for provenance and for optional slot
checks. Reference content is not written to diagnostics, logs, errors, or
manifests.
Reference binding precedence is:
1. pipeline-level config `references`;
2. target-local config references, including legacy lane-level extractor
`references`;
3. `--reference` run flags;
4. `--without-reference` run flags.
Reference flags are resolved against selected chunk, extractor, merger, and
normalizer targets before the run starts. Flat slot names are accepted only
when exactly one selected target declares that slot. For configured reference
bindings, precedence, path resolution, and validation, see
[Configuration](config.md#pipelines).
`--reference` binds or replaces one slot for one selected target. Selectors are:
@@ -142,8 +116,7 @@ go run ./cmd/notarius run dnd-session \
--session-id campaign-17-session-04
```
Use `--resume` to reuse valid checkpoints from a previous compatible
invocation:
The resume flag can be added to an otherwise identical run invocation:
```sh
go run ./cmd/notarius run dnd-session \
@@ -152,13 +125,8 @@ go run ./cmd/notarius run dnd-session \
--resume
```
Plain `run` does not skip completed work. It executes the pipeline normally and
refreshes checkpoints when checkpointing is enabled. `--resume` verifies each
checkpoint before reuse and executes any missing, corrupt, or incompatible step
normally.
For durable output, diagnostics, retention, and failure inspection, see
[Operations](operations.md).
For checkpoint behavior, durable output, diagnostics, retention, and failure
inspection, see [Operations](operations.md).
## `config validate`
@@ -166,8 +134,8 @@ For durable output, diagnostics, retention, and failure inspection, see
Flags:
- `--config path`: config file path. If omitted, discovery uses
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
- `--config path`: config file path. If omitted, Notarius uses the discovery
rules in [Configuration](config.md#discovery).
- `--pipeline pipeline-id`: additionally resolve one configured pipeline against
the production module catalog.
- `--only lane-a,lane-b`: validate resolution for selected artifact lanes. This
@@ -191,8 +159,8 @@ go run ./cmd/notarius config validate \
Flags:
- `--config path`: config file path. If omitted, discovery uses
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
- `--config path`: config file path. If omitted, Notarius uses the discovery
rules in [Configuration](config.md#discovery).
- `--json`: print `{"pipelines":[...]}` instead of one ID per line.
Examples:
@@ -215,42 +183,6 @@ go run ./cmd/notarius pipelines list \
- `2`: command syntax was invalid, a command was unknown, a required argument
was missing, or a flag value was malformed.
## Implemented Production Pipeline Modules
The production CLI currently registers these module keys:
- input: `seriatim`
- chunk: `generic`, `dnd/scenes`
- extract: `dnd/spells`
- merge: `appendorder`
- normalize: `noop`
- output: `json`
## Implemented Production Validators
The production CLI currently registers these validator keys:
- `generic/always_accept`
- `generic/always_reject`
- `generic/valid_json`
- `generic/valid_json_schema`
- `extract/dnd/spells/shape`
- `extract/dnd/spells/source_refs`
- `extract/dnd/spells/source_relatedness`
The production default chain for the `dnd/spells` extractor is:
1. `generic/valid_json`
2. `generic/valid_json_schema`
3. `extract/dnd/spells/shape`
4. `extract/dnd/spells/source_refs`
5. `extract/dnd/spells/source_relatedness`
Validator chain overrides are configured on `chunk`, lane `extract`, lane
`merge`, and lane `normalize` bindings. Omitted overrides use production
defaults, `validators: []` disables validation for that binding, and non-empty
lists replace the default chain in configured order. Validator keys are resolved
against the registered validator catalog.
For YAML structure, Scriptorium profile sources, environment overrides, and
module binding syntax, see [Configuration](config.md).
For YAML structure, defaults, Scriptorium profile sources, environment
overrides, and selectable module and validator keys, see
[Configuration](config.md).

View File

@@ -7,34 +7,22 @@ built-in defaults, then environment overrides are applied.
## Discovery
Commands that accept `--config` load configuration in this order:
Commands that load configuration use this order:
1. the `--config` path, when provided;
1. an explicit path supplied through the CLI, when provided;
2. `NOTARIUS_CONFIG`, when set to a non-empty path;
3. `/usr/local/etc/notarius/config.yml`.
If none is available, the command fails with a config file not found error.
The explicit-path option is defined in the [CLI reference](cli.md).
## Minimal Example
## Maintained Examples
```yaml
version: 2
pipelines:
dnd-session:
input: seriatim
references:
party: ./dnd-spells-roster.txt
glossary: ./dnd-spells-glossary.txt
chunk:
module: generic
options:
max_units: 50
artifacts:
spells:
extract: dnd/spells
```
- [Minimal D&D spell configuration](../examples/dnd-spells.config.yml)
- [Production-oriented D&D spell configuration](../examples/dnd-spells-production.config.yml)
The maintained fixture is [examples/dnd-spells.config.yml](../examples/dnd-spells.config.yml).
Both complete files are validated by the CLI test suite. The fragments below
illustrate individual fields and are not alternate complete configurations.
## Top-Level Fields
@@ -52,31 +40,21 @@ rejected; execution profiles now come from Scriptorium.
Built-in defaults:
```yaml
concurrency:
total_llm: 1
diagnostics:
work_dir: /tmp/notarius
retention: auto
workspace:
diagnostics:
enabled: true
resume:
enabled: false
debug:
enabled: false
```
`workspace.directory` is unset by default. Without a workspace directory,
diagnostics continue to use `/tmp/notarius`, and checkpoint and debug workspace
features have no storage root.
- `concurrency.total_llm`: `1`
- `diagnostics.work_dir`: `/tmp/notarius`
- `diagnostics.retention`: `auto`
- `workspace.directory`: unset
- `workspace.diagnostics.enabled`: `true`
- `workspace.resume.enabled`: `false`
- `workspace.debug.enabled`: `false`
No pipelines are built in. A run requires a configured pipeline.
If `scriptorium` is omitted, Notarius uses Scriptorium's built-in profile
catalog. Prompt definitions may also name default profile IDs. The current D&D
scene and spell prompts use Scriptorium prompt defaults when a module binding
does not set `llm_profile`.
scene and spell prompts default to the built-in `mistral-small-3` profile when a
module binding does not set `llm_profile`. That built-in profile reads its
credential from `OPENROUTER_API_KEY`.
## Scriptorium Profiles
@@ -131,11 +109,8 @@ profiles.
## Pipelines
A pipeline defines the fixed Notarius workflow:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
A pipeline selects implementations for the fixed workflow defined by
[Architecture](policy/architecture.md#system-shape).
Pipeline fields:
@@ -159,23 +134,20 @@ Artifact lane fields:
- `references`: optional compatibility alias for extractor reference bindings.
Lane bindings override pipeline-level bindings for the same slot.
`notarius run` and `notarius config validate --pipeline` resolve the pipeline
against the production module catalog and fail fast for unknown or incompatible
module keys.
Commands that resolve a pipeline fail for unknown or incompatible module keys.
See [CLI Reference](cli.md) for command syntax.
Reference bindings are validated against reference slots declared by eligible
chunk, extract, merge, and normalize targets during pipeline resolution. Required slots
must be bound after config defaults, target-local references, lane-level
compatibility bindings, and run-time `--reference` or `--without-reference`
overrides are applied. Config-relative paths are resolved relative to the config
file; CLI reference paths are resolved relative to the current working
directory. Materialized bound files must be UTF-8 text. Materialized reference
provenance is recorded for chunk, extractor, merger, and normalizer targets, and runtime
reference content is passed to the target that declares the slot. Reference
media types are inferred from file extensions, recorded as canonical base media
types, and checked only when a module declares `AcceptedMediaTypes`; unknown
extensions are recorded as `application/octet-stream`. Reference content is not
written to diagnostics, logs, errors, or manifests.
compatibility bindings, and command-line reference overrides are applied.
Config-relative paths are resolved relative to the config file; command-line
reference paths are resolved relative to the current working directory. Bound
files must be UTF-8 text. Reference media types are inferred from file
extensions and checked when a module restricts accepted types; unknown
extensions use `application/octet-stream`. See [CLI Reference](cli.md#run) for
command-line selectors and [Operations](operations.md) for recorded provenance
and sensitive-data handling.
Pipeline-level `references` are defaults. They are valid when at least one
eligible target in the full configured pipeline declares the slot, including
@@ -252,7 +224,7 @@ Binding fields:
- `llm_profile`: optional Scriptorium profile ID. Empty or omitted lets the
Scriptorium prompt default select the profile.
- `retries`: non-negative retry count for extra runtime attempts after the
first attempt. The runner applies retries to `chunk`, `extract`, `merge`, and
first attempt. Default: `0`. Supported on `chunk`, `extract`, `merge`, and
`normalize` bindings.
- `options`: optional module-specific settings.
- `references`: optional reference bindings. Supported only for `chunk`,
@@ -275,11 +247,6 @@ Validator bindings reject `references`, `retries`, and nested `validators`.
During resolution, deterministic validators reject explicit `llm_profile`
values.
The `--llm-profile` run flag overrides every effective LLM-capable module
binding to use one Scriptorium profile ID: chunk, every selected lane extract,
merge, and normalize binding. It does not override validator-specific
`llm_profile` values.
Configured LLM-backed validators with explicit `llm_profile` values are
validated against the configured Scriptorium profile source. Deterministic
production validators do not call the LLM and must not set `llm_profile`.
@@ -350,80 +317,42 @@ casts still must be present in the source transcript.
`workspace` fields:
- `directory`: optional workspace root for Notarius-owned local state.
- `diagnostics.enabled`: set to `false` to skip diagnostics run directories and
diagnostics artifact writes. Default: `true`.
- `diagnostics.retention`: `auto`, `always`, or `never`.
- `resume.enabled`: boolean resume checkpointing setting. Default: `false`.
- `debug.enabled`: boolean debug artifact setting. Default: `false`.
Use `/var/lib/notarius` as the standard production workspace directory. For
local development, prefer a project-local ignored path such as
`./.notarius/workspace`.
```yaml
workspace:
directory: /var/lib/notarius
diagnostics:
enabled: true
retention: auto
resume:
enabled: false
debug:
enabled: false
```
When `workspace.directory` is set, diagnostics are written under
`<workspace.directory>/diagnostics/`.
When both `workspace.directory` and `workspace.resume.enabled` are set, runs
write stage-owned checkpoint artifacts under
`<workspace.directory>/checkpoints/`. `notarius run --resume` can reuse valid
checkpoints from a compatible invocation. Checkpoints may contain source text,
intermediate raw outputs, rejected outputs, metadata, and warnings. Protect the
workspace as sensitive local state.
When both `workspace.directory` and `workspace.debug.enabled` are set, runs
write per-invocation debug artifacts under
`<workspace.directory>/debug/<run-id>/`. Debug artifacts may contain source
material, reference material, prompt inputs, model outputs, validation payloads,
and other sensitive content. Debug is disabled by default.
- `resume.enabled`: boolean resume checkpointing setting.
- `debug.enabled`: boolean debug artifact setting.
- `diagnostics`: optional diagnostics settings defined below.
`workspace.resume.enabled` and `workspace.debug.enabled` are independent.
Enabling one does not enable the other.
Enabling one does not enable the other. For directory layout, state lifecycle,
permissions, and sensitive content, see [Operations](operations.md).
## Diagnostics
Preferred workspace diagnostics fields:
- `workspace.directory`: workspace root for Notarius-owned local state.
- `workspace.diagnostics.enabled`: set to `false` to skip creating diagnostics
run directories and diagnostics artifacts. Default: `true`.
run directories and diagnostics artifacts.
- `workspace.diagnostics.retention`: `auto`, `always`, or `never`.
When `workspace.directory` is set, diagnostics use
`<workspace.directory>/diagnostics` as their work directory.
Defaults for workspace and diagnostics fields are listed in
[Defaults](#defaults).
`workspace.diagnostics.retention` overrides legacy diagnostics retention when
set.
`diagnostics` fields:
- `work_dir`: deprecated compatibility directory for per-run diagnostics.
Default: `/tmp/notarius`.
- `retention`: deprecated compatibility retention mode. `auto`, `always`, or
`never`. Empty uses `auto`.
`never`.
Existing `diagnostics.work_dir`, `diagnostics.retention`, `NOTARIUS_WORK_DIR`,
and `NOTARIUS_DIAGNOSTICS_RETENTION` inputs remain supported for compatibility.
New configuration should use `workspace.directory` and
`workspace.diagnostics.retention` instead.
`auto` retains diagnostics for failed runs and successful runs with warnings.
`always` retains diagnostics for every run. `never` removes diagnostics for
successful runs without regard to warnings; failed runs are retained.
The `--diagnostics-dir` run flag overrides the effective diagnostics work
directory for that invocation. It affects diagnostics only and does not change
the workspace directory.
For retention behavior and the physical diagnostics layout, see
[Operations](operations.md#retention). For the invocation-specific diagnostics
override, see [CLI Reference](cli.md#run).
## Validation
@@ -440,7 +369,7 @@ Pipeline resolution additionally checks:
- the pipeline ID exists;
- at least one artifact lane is declared and selected;
- selected lanes exist when `--only` is used;
- lanes selected through the CLI exist in the resolved pipeline;
- required module keys are present;
- module keys are registered for the expected slot;
- module capability requirements are satisfied;

View File

@@ -4,41 +4,16 @@ This is the first-read landing page for people and LLM coding agents working on
Notarius. It provides a concise repository orientation and routes each kind of
change to its canonical documentation.
## Orientation
Notarius is a small Go application for extracting structured data from source
material. It is a general extraction platform with an initial Seriatim and D&D
implementation. Configured modules run through a fixed workflow:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
The CLI is the application boundary. Core packages own deterministic models and
policy, framework packages own reusable contracts and orchestration, and module
and validator packages own concrete behavior.
## Repository Map
- `cmd/notarius`: executable entry point.
- `internal/cli`: CLI behavior and production composition.
- `internal/core`: deterministic source, config, artifact, diagnostics, and
workspace packages.
- `internal/framework`: contracts, pipeline orchestration, validation helpers,
checkpoints, debug recording, and LLM runtime plumbing.
- `internal/modules`: concrete implementations of the six pipeline stages.
- `internal/validators`: concrete output validators.
- `docs`: canonical policy, reference, integration, internal, ADR, and roadmap
documentation.
- `examples`: maintained, secret-free example inputs and configuration.
See [Internal Overview](internal/overview.md) for the implemented component
map and links to focused internal documentation.
Notarius is a Go CLI for configured structured extraction workflows. Start with
the [README](../README.md) for product context, [Architecture](policy/architecture.md)
for system boundaries, and [Internal Overview](internal/overview.md) for the
implemented component map.
## What To Read
| When working on | Read | Why |
| --- | --- | --- |
| Finding the package or component that owns current behavior | [Internal Overview](internal/overview.md) | It is the implemented component inventory and routes to focused internals. |
| Application shape, package boundaries, contracts, dependency direction, runtime guarantees, or safety properties | [Architecture](policy/architecture.md) and relevant [ADRs](adr/) | Architecture defines the intended system and its invariants; ADRs preserve significant decision rationale. |
| Any documentation addition or revision | [Documentation Policy](policy/documentation.md) | It defines canonical homes, audiences, current-behavior rules, and maintenance requirements. |
| Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. |
@@ -63,13 +38,3 @@ go test ./...
go vet ./...
go build ./cmd/notarius
```
## Universal Reminders
- Keep non-roadmap documentation limited to implemented behavior.
- Update affected documentation and maintained examples in the same change as
behavior.
- Use fakes, fixtures, or local test servers instead of real external services
in tests.
- Do not expose secrets in code, errors, logs, diagnostics, manifests,
documentation, or examples.

View File

@@ -1,11 +1,11 @@
# D&D Spell Raw Output
This document is the durable raw output contract for the implemented
`dnd/spells` extractor.
This document is the durable raw output contract for the production D&D spell
extractor. Selectable extractor keys are cataloged in
[Configuration](../config.md#implemented-production-modules).
## Identity
- Extractor key: `dnd/spells`
- Prompt ID: `dnd.spells`
- Response schema key: `dnd_spells`
- Response schema ID: `notarius.dnd.spells`
@@ -13,100 +13,47 @@ This document is the durable raw output contract for the implemented
- Response schema version: `v1`
- Media type: `application/json`
The extractor requires source chunks and transcript source capability. It
returns canonical spell-cast JSON derived from the structured LLM response. The
extractor assigns source IDs deterministically and keeps source-unit ranges as
model-authored evidence locations. The default `appendorder` merger passes a
single chunk output through and concatenates multiple
`spell_casts` arrays in chunk order. The default `noop` normalizer passes the
merge output through unchanged.
The output contains canonical spell casts derived from transcript evidence.
Source IDs are assigned from the input identity; source-unit ranges identify
the evidence location.
## Output Shape
For a single chunk, `lanes/spells.json` has this shape:
`lanes/spells.json` is a JSON object with one required top-level array. Its
structure is:
```json
{
"spell_casts": [
{
"caster": "Aria",
"spell": "Cure Wounds",
"effect": "heals an injured ally",
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds.",
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": 1,
"end_unit_id": 1
}
]
}
]
}
```text
{"spell_casts": [<spell-cast object>, ...]}
```
`spell_casts` must be present. It may be empty when no spell casts are found.
For multiple chunks with the default merger, the lane output keeps the same
top-level shape and concatenates `spell_casts` in chunk order:
```json
{
"spell_casts": [
{
"caster": "Aria",
"spell": "Cure Wounds",
"effect": "heals an injured ally",
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds.",
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": 1,
"end_unit_id": 1
}
]
}
]
}
```
When multiple chunk results are combined, spell casts remain in chunk order.
## Spell-Cast Fields
Each spell cast contains:
Each spell cast contains exactly these required fields:
- `caster`: in-world character or creature casting the spell;
- `spell`: spell name;
- `effect`: concise spell effect in the scene;
- `narrative_description`: short description of the spell cast in context;
- `source_refs`: transcript source references with extractor-assigned source
IDs and model-supplied unit ranges.
IDs and evidence unit ranges. It must contain at least one entry.
`caster` is the in-world caster, not the transcript speaker.
All four string fields must be non-empty. `caster` is the in-world caster, not
the transcript speaker. Unknown fields are rejected.
## Source References
Each source reference uses the generic source-reference shape:
Each source reference contains exactly three required fields: `source_id`,
`start_unit_id`, and `end_unit_id`. The source ID must match the input identity.
The unit IDs must be positive integers present in the input, and the start unit
must not appear after the end unit. Unknown fields are rejected.
- `source_id`
- `start_unit_id`
- `end_unit_id`
The LLM-facing prompt schema asks only for integer `start_unit_id` and
`end_unit_id` values matching source-unit IDs. `source_id` is assigned by the
extractor from the source document ID before validation and output, and is
required in this durable output contract.
## References
The extractor accepts optional UTF-8 text references:
- `players`
- `party`
- `glossary`
- `roster`, a deprecated compatibility alias for `party`
References are supporting disambiguation material only. They are not source
evidence and are not addressable through `source_refs`.
Reference slot keys and accepted file types are defined in
[Configuration](../config.md#implemented-production-modules). References are
supporting disambiguation material, not source evidence, and are not
addressable through `source_refs`.
## Manifest Metadata

View File

@@ -1,22 +1,17 @@
# JSON Output
This document is the durable JSON output file-format contract produced by the
implemented `json` output module and written by the CLI.
production JSON encoder and written by the CLI. Selectable output-encoder keys
are cataloged in
[Configuration](../config.md#implemented-production-modules).
## Output Directory
The CLI writes logical output files under:
```text
<output-root>/<run-id>/
```
The default output root is `./notarius-output`. Operational behavior is covered
in [Operations](../operations.md).
The output module produces the logical bundle described here. The CLI's
physical placement and lifecycle for that bundle are defined in
[Operations](../operations.md#output-directory).
## Files
The `json` output module writes:
The encoder writes:
- `index.json`
- `manifest.json`
@@ -25,6 +20,7 @@ The `json` output module writes:
- `warnings.json`
Files are pretty-printed JSON with a trailing newline when the payload is JSON.
Logical file paths are relative, slash-separated, and may not contain `..`.
## `index.json`
@@ -58,22 +54,21 @@ sanitizing the lane ID:
- empty sanitized names are rejected;
- two lanes that sanitize to the same output file are rejected.
`manifest_file`, `rejected_file`, and `warnings_file` contain the fixed paths
shown above. Each `output_files` entry requires `lane_id` and `file`. It also
contains the normalized payload `media_type`, normalizer `module_key`, and
response `schema_id`, `schema_name`, and `schema_version` when those values are
available.
## `manifest.json`
`manifest.json` contains a run manifest:
`manifest.json` contains a run manifest. This abridged example shows its core
structure:
```json
{
"run_id": "run-123",
"pipeline_id": "dnd-session",
"pipeline_digest": "sha256:...",
"input_module": "seriatim",
"chunker": "dnd/scenes",
"source_digests": ["sha256:..."],
"extractors": ["dnd/spells"],
"merger": "appendorder",
"normalizer": "noop",
"output_encoder": "json",
"artifact_lanes": [
{
"id": "spells",
@@ -82,35 +77,6 @@ sanitizing the lane ID:
"normalizer": "noop"
}
],
"validator_chains": [
{
"stage": "extract",
"lane_id": "spells",
"module_key": "dnd/spells",
"validators": [
{
"key": "generic/valid_json",
"execution_class": "deterministic"
},
{
"key": "generic/valid_json_schema",
"execution_class": "deterministic"
},
{
"key": "extract/dnd/spells/shape",
"execution_class": "deterministic"
},
{
"key": "extract/dnd/spells/source_refs",
"execution_class": "deterministic"
},
{
"key": "extract/dnd/spells/source_relatedness",
"execution_class": "deterministic"
}
]
}
],
"validation_status": "approved",
"started_at": "2026-01-01T00:00:00Z",
"completed_at": "2026-01-01T00:00:01Z"
@@ -119,6 +85,23 @@ sanitizing the lane ID:
Fields with empty values may be omitted by JSON encoding.
The manifest fields are:
- `run_id`, `pipeline_id`, and `pipeline_digest`: run and resolved-pipeline
identity;
- `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, and
`output_encoder`: resolved module keys;
- `module_metadata` and `artifact_lanes`: module and per-lane provenance,
including prompt and response-schema provenance when provided;
- `validator_chains`: resolved validation points and validators;
- `source_digests` and `references`: source and reference provenance;
- `normalized_outputs` and `rejected_outputs`: payload-free result summaries;
- `llm_profiles`: selected profile IDs and provider or model names when
available;
- `metadata`: the effective prompt `session_id`;
- `validation_status`: `approved` or `rejected`;
- `started_at` and `completed_at`: UTC run timestamps.
`source_digests` contains source document digests only. Bound references are
recorded separately under `references`, which contains provenance only: target
stage, lane ID when present, slot name, origin type and URI, digest, media
@@ -152,27 +135,8 @@ Each normalized raw output is written to `lanes/<sanitized-lane-id>.json`.
The JSON output encoder accepts only `application/json` normalized outputs. The
file contains the raw JSON payload pretty-printed.
For the current D&D spell extractor, `lanes/spells.json` has this shape:
```json
{
"spell_casts": [
{
"caster": "Aria",
"spell": "Cure Wounds",
"effect": "heals an injured ally",
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds.",
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": 1,
"end_unit_id": 1
}
]
}
]
}
```
The schema of each lane payload is owned by that artifact contract. For the
current D&D spell lane, see [D&D Spell Raw Output](dnd-spell-artifacts.md).
## `rejected.json`
@@ -184,10 +148,10 @@ Shape:
}
```
When raw output validation rejects an output, entries use the
`contracts.RejectedOutput` shape, including stage, lane ID, module key,
validator name, reason code, message, attempt count, and optional diagnostic
artifact path.
When raw output validation rejects an output, each entry contains `stage` and
`message`. It includes `lane_id`, `module_key`, `chunk_id`, `chunk_index`,
`validator_name`, `reason_code`, `attempt_count`, and
`diagnostic_artifact_path` when applicable.
## `warnings.json`
@@ -206,3 +170,5 @@ Shape:
```
`warnings` is an empty array when no warnings are reported.
Each warning requires `reason_code` and `message`; `scope` is omitted when it is
empty.

View File

@@ -1,114 +1,60 @@
# Seriatim Transcript JSON
This document is the external input contract for the implemented `seriatim`
input adapter.
This document is the external input contract consumed by the production
Seriatim input adapter. Selectable input-adapter keys are cataloged in
[Configuration](../config.md#implemented-production-modules).
## Adapter
- Module key: `seriatim`
- Document kind: `transcript`
- Unit kind: `transcript_segment`
- Source format: `application/vnd.seriatim+json`
The adapter parses raw Seriatim JSON into a generic source document. It owns
transcript-specific JSON parsing and metadata mapping; core source and pipeline
code stay source-format agnostic.
## Accepted Shape
The input must be one JSON object with top-level `metadata` and `segments`
fields. This covers the maintained minimal fixture and Seriatim intermediate
output that provides the same required segment fields.
```json
{
"metadata": {
"id": "session-alpha",
"title": "Synthetic D&D spell session"
},
"segments": [
{
"id": 1,
"start": 0,
"end": 4,
"speaker": "Aria",
"text": "Aria raises her holy symbol and casts Cure Wounds."
}
]
}
```
The maintained example is
[examples/seriatim-minimal-transcript.json](../../examples/seriatim-minimal-transcript.json).
Top-level metadata entries are preserved. Other segment fields, such as
`categories`, are ignored.
Required top-level fields:
- `metadata`: an object. Its entries are accepted as source metadata.
- `segments`: a non-empty array of segment objects.
Required segment fields:
- `id`: a positive integer JSON number or canonical decimal string without
leading zeros or surrounding whitespace;
- `start`: a finite, non-negative JSON number or numeric string;
- `end`: a finite, non-negative JSON number or numeric string that is not less
than `start`;
- `speaker`: a non-empty string;
- `text`: a non-empty string.
Other top-level and segment fields, such as `categories`, are ignored.
Multiple top-level JSON values are rejected.
## Validation
The adapter rejects:
- empty raw input;
- malformed JSON;
- top-level JSON that is not an object;
- missing, null, or non-object `metadata`;
- missing, null, non-array, or empty `segments`;
- segment values that are not objects;
- segment `id` values that are not positive integer JSON numbers or numeric
strings;
- non-string `speaker` or `text`;
- duplicate segment IDs;
- missing or empty `speaker`;
- missing, empty, invalid, non-finite, or negative `start`;
- missing, empty, invalid, non-finite, or negative `end`;
- `end` values before `start`;
- missing or empty `text`.
The adapter rejects empty input, malformed JSON, multiple top-level JSON values,
non-object segment values, duplicate segment IDs, and any violation of the
shape or field constraints above.
Segment text is preserved as provided, but it must not be empty after trimming.
## Source Mapping
## Derived Identity
The adapter maps input to `SourceDocument`:
Notarius identifies the parsed source in this order:
- `metadata` becomes `SourceDocument.Metadata`;
- `SourceDocument.Kind` is `transcript`;
- `SourceDocument.Format` is `application/vnd.seriatim+json`;
- `SourceDocument.Digest` is `sha256:<hex>` of the exact raw input bytes.
1. `metadata.id`, when it is a non-empty string after trimming;
2. `metadata.source_id`, when it is a non-empty string after trimming;
3. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
`SourceDocument.ID` is selected in this order:
1. the parse request source ID, after trimming;
2. `metadata.id`, when it is a non-empty string after trimming;
3. `metadata.source_id`, when it is a non-empty string after trimming;
4. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
Each segment becomes one `SourceUnit`:
- `segment.id` becomes integer `SourceUnit.ID`;
- `segment.text` becomes `SourceUnit.Text`;
- `SourceUnit.Kind` is `transcript_segment`;
- `speaker`, `start`, and `end` are stored in source-unit metadata.
## Metadata Keys
Seriatim unit metadata uses these keys:
- `speaker`: string speaker label;
- `start`: `json.Number` start value;
- `end`: `json.Number` end value.
The `internal/modules/input/seriatim` package exposes typed accessors for these
values.
## Capabilities
The module declares these provided capabilities:
- `source.transcript`
- `transcript.speaker`
- `transcript.timestamps`
The source digest recorded in output provenance is `sha256:<hex>` of the exact
raw input bytes. Segment IDs become the unit IDs used by artifact source
references.
## Compatibility Limit

View File

@@ -1,102 +1,93 @@
# Diagnostics Internals
Diagnostics internals live in `internal/core/diagnostics`. Operator-facing run
behavior is documented in [Operations](../operations.md).
## Purpose
Diagnostics provide local inspection artifacts for a run without becoming the
durable output contract. Durable user output is produced by output modules and
written by the CLI.
Diagnostics must not expose secrets.
`internal/core/diagnostics` provides the scoped writer and retention decision
used by `internal/cli`. The physical layout, artifact inventory, retention
semantics, failure inspection, and cleanup procedures are canonical in
[Operations](../operations.md#diagnostics-directory). Configuration fields and
defaults are canonical in [Configuration](../config.md#diagnostics).
## Run Directory
`NewRunDirectory(workDir, retention)` creates:
`NewRunDirectory` normalizes empty constructor inputs, creates the effective
diagnostics root when needed, and allocates a unique timestamp-based child
directory. It retries a bounded number of collisions before failing. The
resulting `RunDirectory` retains its creation time and retention mode for later
metadata and cleanup decisions.
```text
<workDir>/run-<unix-nanoseconds>/
```
The package does not resolve workspace configuration. `internal/cli` derives
effective workspace settings first and passes the diagnostics root into the
constructor.
If `workDir` is empty, it defaults to `/tmp/notarius`. Empty retention defaults
to `auto`.
## Scoped Writers
The CLI passes the effective diagnostics root from workspace configuration.
When `workspace.directory` is set and diagnostics are enabled, that root is
`<workspace.directory>/diagnostics`. The legacy diagnostics work directory and
`--diagnostics-dir` still pass a diagnostics-only root to this constructor.
Typed methods on `RunDirectory` write invocation metadata, redacted effective
configuration, resolved pipeline/reference data, checkpoint events, source data
when explicitly requested, manifests, reports, warnings, and error text. The
current filenames and their operator-facing contents are listed in
[Operations](../operations.md#diagnostics-directory).
The writer makes the work directory if needed, then attempts to create a unique
run directory. It retries run ID creation a bounded number of times if a
collision occurs.
JSON methods indent their payload and append a newline. All artifact writes use
a temporary file in the target directory, apply the requested permissions, and
rename it into place. Artifact resolution accepts only a single relative base
name; absolute paths, separators, and paths escaping the run directory fail
before writing.
## Artifact Writers
## Redacted Configuration
Implemented artifact names:
`WriteRedactedEffectiveConfig` accepts a `RedactedDiagnosticsPayload` provider
rather than a raw config value. `internal/core/config` implements that contract
by cloning effective config data and removing secret-shaped values before JSON
encoding. The diagnostics package therefore never needs configuration-specific
field knowledge.
- `invocation.json`
- `effective-config.json`
- `resolved-pipeline.json`
- `resolved-references.json`
- `checkpoint-events.json`
- `source-document.json`
- `run-manifest.json`
- `run-report.json`
- `warnings.json`
- `error.log`
## Retention Coordination
JSON artifacts are encoded with indentation and a trailing newline. Writes are
atomic through a temporary file in the target directory followed by rename.
`ShouldRetainRunDirectory` is a pure decision over the effective retention mode,
run success, and warning presence. `ApplyRetention` uses that result to remove
only its own run directory. Unsupported modes retain data as a fail-safe, though
normal CLI execution rejects them during config validation.
Artifact names must be single relative file names. Absolute paths, path
separators, and names resolving outside the run directory are rejected.
The meaning of each supported mode belongs in
[Operations](../operations.md#retention); this package implements that contract
without loading config or inspecting run artifacts.
## Redacted Effective Config
## CLI State Flow
Diagnostics writers accept payloads that implement
`RedactedDiagnosticsPayload`. `internal/core/config` uses this to redact API
keys in effective config diagnostics while preserving resolved pipeline context.
When diagnostics are enabled, `internal/cli` creates the run directory after
configuration loading and before pipeline resolution. It then writes artifacts
as state becomes available: invocation data, effective resolution data,
pipeline results, and the final report. This ordering permits later failures to
retain the context already established.
The redaction path clones config data before replacing secret values.
Failures before construction have no `RunDirectory`. Later failures write an
error log, preserve any available partial manifest, and apply a failed-run
retention decision. A diagnostics write failure is itself a command failure so
the CLI does not report success after losing requested inspection data.
## Retention
When diagnostics are disabled, the CLI carries a nil run directory and the
shared `writeDiagnostics` helper turns writes into no-ops. User-facing errors
still go to stderr; that invocation behavior is documented in
[Operations](../operations.md#failures).
Retention is decided by `ShouldRetainRunDirectory`.
## Package Guarantees
- Failed runs are always retained.
- `always` retains successful runs.
- `never` removes successful runs.
- `auto` retains successful runs only when warnings exist.
- Unknown retention values are treated as retain by the retention decision, but
config validation rejects unsupported values before normal runs.
- A `RunDirectory` writes and removes only within its allocated directory.
- JSON and error artifacts use atomic replacement.
- Nil receivers and invalid typed payloads return errors rather than panicking.
- Retention never removes a failed run and never targets the diagnostics root.
- Diagnostics models contain inspection metadata, not the durable output
contract.
- Checkpoint and debug serializers remain separate framework components.
- Secret-handling follows the invariant in
[Architecture](../policy/architecture.md#state-output-and-safety).
`ApplyRetention` removes only the specific run directory.
## Tests To Inspect
## CLI Failure Behavior
When diagnostics are enabled, the CLI creates the diagnostics run directory
after config loading and before pipeline resolution. Failures before that point
do not have diagnostics.
When workspace diagnostics are explicitly disabled, the CLI does not create a
diagnostics run directory and skips diagnostics artifact writes. Failures are
still printed to stderr.
After diagnostics creation, run failures call `WriteErrorLog` and apply
retention with `RunSucceeded: false`, so the run directory remains available.
When the pipeline returns a partial manifest on failure, the CLI writes that
manifest before logging the failure.
## Invariants
- Diagnostics paths must be narrow and run-directory scoped.
- Writes should be atomic where practical.
- Secrets must be redacted.
- Diagnostics write failures are command failures because they can hide the
information needed for recovery.
- Durable output file contracts belong to output modules and integration docs,
not to diagnostics.
- Checkpoint and debug workspace files are separate framework-owned artifacts,
not diagnostics artifacts.
- `internal/core/diagnostics/run_dir_test.go`: allocation, artifact confinement,
atomic writes, retention, and failure behavior.
- `internal/core/diagnostics/artifacts_test.go`: stable artifact identifiers.
- `internal/core/config/redaction_test.go`: clone-and-redact payload behavior.
- `internal/core/workspace/settings_test.go`: effective diagnostics-root and
enablement handoff.
- `internal/cli/run_test.go`: creation timing, artifact sequencing, disabled
diagnostics, overrides, failures, and retention integration.

View File

@@ -1,114 +1,132 @@
# LLM Runtime
# LLM Runtime Internals
The implemented LLM runtime lives in `internal/framework/llm`. It provides
transport-neutral structured completion contracts, a Scriptorium-backed
production client, concurrency scheduling, prompt/schema asset registration,
schema registry helpers, and secret redaction.
`internal/framework/llm` implements Notarius's transport boundary for structured
completion. It contains the Scriptorium adapter, concurrency scheduler,
prompt/schema registries, selected-profile recording, and provider-error
redaction.
## Contract
Provider-neutral ownership rules are defined in
[Architecture](../policy/architecture.md#llm-boundary). Profile sources,
credentials, and concurrency settings are defined in
[Configuration](../config.md).
Modules depend on `contracts.StructuredLLMClient`:
## Structured Contract
```go
CompleteStructured(ctx, request, out) (response, error)
```
Modules and LLM-backed validators depend on
`contracts.StructuredLLMClient.CompleteStructured`. A request identifies a
prompt and optional profile/session, supplies named input materials and
variables, and provides a caller-owned decoding target. A successful response
contains the validated raw structured bytes plus non-secret provider, model,
profile, and token metadata.
The request contains prompt ID/version, profile ID, session ID, prompt input
materials, and variables. The caller supplies a pointer target for decoded
structured output. The response also carries the raw structured output bytes
returned by the runtime so modules can preserve raw payloads in pipeline stage
outputs.
The caller owns prompt selection, response-schema selection, and interpretation
of the decoded result. `LLMInputMaterial` keeps source and reference bytes with
their origin metadata so the adapter can pass named artifacts to Scriptorium
without exposing Scriptorium types through stage contracts.
Modules that call the LLM own their prompts, schemas, prompt IDs, and
domain-specific interpretation. Validator packages own approve/reject policy,
and central catalog mappings decide which validators run by default. Provider
adapters should not contain domain-specific prompt logic.
## Production Construction
Prompt input materials carry source or reference bytes with optional origin
metadata. The Scriptorium-backed runtime receives them as named artifacts rather
than rendered prompt strings owned by Notarius modules.
`internal/cli` constructs the production runtime by:
## Production Client Construction
1. collecting embedded prompt and response-schema assets from production module
packages;
2. creating a `ScriptoriumClient` from the effective profile source;
3. attaching an `LLMProfileRecorder`;
4. creating a scheduler from the effective concurrency limit;
5. returning a `ScheduledClient` wrapper.
`internal/cli` builds the production LLM client from the effective config:
1. collect production Scriptorium prompt and schema assets from module packages;
2. create a Scriptorium-backed structured client using effective Scriptorium
profile source settings from `scriptorium.profile_dir` or
`scriptorium.profile_file`;
3. create a scheduler from global LLM concurrency;
4. wrap the client with `NewScheduledClient`;
5. let the runtime report non-secret profile manifest metadata after calls.
The runtime records the actual selected Scriptorium profile, provider, and model
used during execution. Manifest population does not rely on a precomputed
profile ID before pipeline execution.
Explicit profile validation applies to LLM-capable pipeline stages: chunk,
extract, merge, normalize, and LLM-backed validators with explicit
`llm_profile` values. Input, output, and deterministic validators do not call
the LLM. The `--llm-profile` run flag overrides effective chunk, extract, merge,
and normalize bindings; it does not override validator-specific profiles.
The CLI separately gathers explicit profile IDs from resolved LLM-capable stage
and validator bindings. It prepares a small internal check prompt for each ID so
missing or invalid profiles fail before pipeline execution. The runtime profile
override syntax and scope are defined in the
[CLI reference](../cli.md#run); binding rules are defined in
[Configuration](../config.md#module-bindings).
## Scriptorium Adapter
`ScriptoriumClient` implements `contracts.StructuredLLMClient` by converting
Notarius prompt requests into Scriptorium `RunRequest` values. It:
`ScriptoriumClient` converts a Notarius request into a Scriptorium `RunRequest`.
It validates the decoding target and prompt identity, maps named input materials
to inline artifacts, forwards explicit profile and session context, delegates
rendering/provider execution/structured validation, and unmarshals successful
JSON into the caller target.
- validates the caller output target and prompt ID;
- converts `LLMInputMaterial` values into inline Scriptorium artifacts, using a
single space for empty material so optional blank references remain explicit;
- passes `session_id` through Scriptorium variables and request metadata when
present;
- sends explicit profile IDs only when the request supplies one;
- lets Scriptorium render prompts, call the configured provider, and validate
structured output;
- unmarshals successful JSON into the caller-provided target;
- returns the validated raw structured output bytes to the caller;
- maps token usage and selected profile/model metadata into the Notarius
response and manifest profile recorder.
Empty optional input material is represented by a single space so Scriptorium
retains the named input. The client returns Scriptorium's validated structured
bytes rather than re-encoding the caller target, allowing modules to preserve
the runtime result exactly.
Generated-output validation failures are returned as Notarius errors. Provider
and runtime errors are wrapped with prompt context and bearer tokens are
redacted from error strings. Prompt text, raw source input, reference content,
schema JSON, API keys, and bearer tokens are not added to default diagnostics or
run manifests.
Selected profile, provider, model, and token metadata are mapped into the
Notarius response. The recorder deduplicates profiles by identity and supplies
manifest-safe profile summaries after actual calls; manifest population does
not guess the selected prompt default in advance.
## Scheduler
Generated-output validation failures and provider failures are wrapped with
prompt context. Error strings pass through bearer-token redaction before they
cross the runtime boundary.
`Scheduler` bounds concurrent provider calls. It tracks in-flight calls and a
FIFO queue of waiters. Cancellation removes queued waiters or releases granted
permits.
## Scheduling
`NewScheduledClient` wraps any structured LLM client and runs each completion
inside the scheduler.
`Scheduler` uses a bounded permit count and a FIFO waiter queue. Immediate
acquisition increments the in-flight count; queued acquisition waits for a
permit or context cancellation. Cancellation removes a queued waiter, while a
cancelled waiter that has already received a permit releases it.
Effective concurrency is:
`ScheduledClient` acquires a permit around each structured completion and
defers release on every result path. The effective limit and default are
configuration facts in [Configuration](../config.md#defaults).
1. `concurrency.total_llm`, when greater than zero;
2. `1`.
## Prompt And Schema Assets
## Schema Registry
`AssetRegistry` combines caller-owned prompt filesystems under stable prefixes
and rejects invalid or conflicting registrations. Production module packages
register their own prompt and schema assets; generic framework code contains no
D&D prompt content.
The framework schema registry embeds generic test schemas. It also exposes
helpers for caller-owned schemas:
Schema helpers load embedded JSON Schema with identity and digest metadata,
return defensive copies, and expose a diagnostics map that omits schema bytes.
The small framework registry contains only generic test schemas; production
schemas remain package-owned.
- `LoadResponseSchema`
- `LookupResponseSchema`
- `MustLookupResponseSchema`
- `ResponseSchema.DiagnosticsMap`
## Debug And Redaction Boundaries
`DiagnosticsMap` omits raw schema content and includes metadata such as key,
ID, version, name, and SHA-256.
The pipeline may wrap the client with a debug recorder that captures prepared
prompt/response material for an explicitly enabled debug run. Default
diagnostics and manifests receive identities, hashes, usage, and selected
profile summaries rather than prompt, source, reference, schema, or response
content.
Production modules own and register their Scriptorium prompt and schema assets.
Framework packages may collect those files but must not contain D&D-specific
prompt content.
The Scriptorium error wrapper removes bearer credential values from surfaced
provider errors; `RedactSecrets` and `ErrorWithSecretsRedacted` support known
secret values elsewhere in the runtime. Config diagnostics use a separate
clone-and-redact path in `internal/core/config`. These mechanisms implement the
security invariant in
[Architecture](../policy/architecture.md#state-output-and-safety); operator
handling of debug data is defined in [Operations](../operations.md#debug).
## Secret Redaction
## Failure Behavior
Provider errors are redacted before surfacing through the Scriptorium-backed
client. Config diagnostics use redacted effective config payloads.
- Invalid targets, missing prompt IDs, malformed structured output, and
Scriptorium failures return contextual errors to the calling module.
- Scheduler construction rejects non-positive limits; acquisition respects
context cancellation.
- Asset registration rejects invalid roots, missing content, and path conflicts.
- Schema loading distinguishes missing assets, invalid JSON, and invalid
metadata.
- Profile validation errors occur during CLI preparation when an explicit
selected ID cannot be prepared.
Do not add raw provider request bodies, response bodies, API keys, or prompt
payloads to diagnostics by default.
## Tests To Inspect
- `internal/framework/llm/scriptorium_client_test.go` and
`scriptorium_api_test.go`: adapter mapping and local HTTP integration.
- `internal/framework/llm/scheduler_test.go` and
`scheduled_client_test.go`: permits, FIFO behavior, cancellation, and wrapper
release.
- `internal/framework/llm/asset_registry_test.go` and
`schema_registry_test.go`: asset composition, validation, and defensive
copies.
- `internal/framework/llm/secrets_test.go`: provider-error redaction.
- `internal/cli/run_test.go`: profile validation, production client wiring,
manifest recording, and debug integration.
- Module-local `scriptorium_assets_test.go` files: prompt inputs and package
asset registration.

View File

@@ -1,268 +1,194 @@
# Modules
Production modules live under `internal/modules`. Each module implements one
contract from `internal/framework/contracts`, exposes a `ModuleSpec`, and
registers itself with the matching pipeline registry.
# Module And Validator Internals
The CLI production catalog currently registers only the modules listed here.
Validator implementations live under `internal/validators` and are registered
separately from modules. Production default validator chains are central CLI
catalog policy; module packages do not own their default validation chains.
Production stage implementations live under `internal/modules`; production
validators live under `internal/validators`. The selectable keys, configuration
options, reference slots, and default validator chain are canonical in the
[module](../config.md#implemented-production-modules) and
[validator](../config.md#implemented-production-validators) catalogs in
Configuration.
## Contract Pattern
## Extension Pattern
A production module package should provide:
A stage module package provides a stable key, constructor, contract
implementation, `ModuleSpec`, `Register`, and focused behavior and registration
tests. A validator package follows the same pattern with `ValidatorSpec` and the
validator registry.
- a stable module key;
- a constructor such as `New`;
- the relevant contract implementation;
- `ModuleSpec`;
- `Register`;
- focused tests for registration, options, contract behavior, and errors.
Specs expose capability and execution metadata without constructing an
implementation. Chunk, extract, merge, and normalize modules that accept
auxiliary material declare identical reference slots from both
`ReferenceSlots()` and `ModuleSpec().ReferenceSlots`; registration tests enforce
that agreement. Runtime delivery uses the corresponding stage request's
`References` field.
Module specs should describe capabilities accurately. Resolution uses specs to
reject incompatible pipelines before execution.
LLM-backed extensions own their prompt definitions and response schemas under
package-local embedded assets. Shared filesystem composition belongs in
`internal/modules/sharedassets`; reusable D&D prompt fragments, reference
declarations, prompt-input assembly, and source-unit helpers belong in
`internal/modules/sharedassets/dnd`. Stage contracts expose only Notarius
structured-completion types, not Scriptorium public types.
Chunk, extract, merge, and normalize modules that accept auxiliary reference material
must declare slots through both `ReferenceSlots()` and
`ModuleSpec().ReferenceSlots`. The runtime slot list and registry metadata
should match so config validation can inspect slots without constructing module
instances. A slot declaration names the slot, whether it is required, accepted
media types, whether multiple items are allowed, and any byte limit. Empty
`AcceptedMediaTypes` means any inferred media type is accepted, though the file
must still be UTF-8 text. When a slot declares accepted media types, Notarius
compares the canonical base media type inferred from the file extension,
case-insensitively and without parameters.
Reference material may inform a module or prompt but must not become source
evidence. The resolver and materializer behavior is described in
[Pipeline Internals](pipeline.md#reference-materialization).
The resolver materializes reference content for chunk, extractor, merger, and
normalizer targets. Runtime delivery uses `contracts.ChunkRequest.References`,
`contracts.ExtractionRequest.References`, `contracts.MergeRequest.References`,
and `contracts.NormalizeRequest.References`. Reference material is not source
evidence and must not be converted into `SourceRef` values. If a module prompt
uses references, pass them as prompt input materials through the structured LLM
request. Prompt metadata hashes remain based on prompt asset source, not
rendered reference bytes.
## Input Adapter
LLM-backed modules own Scriptorium prompt definitions and response schemas in
their embedded assets. Module-owned prompts live under each module's shallow
`assets/prompts` tree and schemas live under `assets/schemas`. Generic shared
prompt filesystem composition lives under `internal/modules/sharedassets`.
Common D&D prompt fragments, reference slot helpers, prompt input assembly, and
reference rendering live under `internal/modules/sharedassets/dnd`. Module
contracts should expose prompt IDs, versions, input material names, and
non-secret prompt/schema hashes through manifest metadata; they should not
expose Scriptorium public types through chunk, extract, merge, or normalize contracts.
### `internal/modules/input/seriatim`
Chunk modules receive the structured LLM client, configured Scriptorium profile
ID, prompt session ID, and raw source input material through
`contracts.ChunkRequest` when they need model-backed chunking. The pipeline
runner validates generic chunk result invariants before extraction; module-owned
policies may be stricter but must stay within the module package.
The adapter decodes the supported transcript JSON, selects the source identity,
computes the raw-input digest, validates segments, and maps each segment into a
generic source unit with speaker and timestamp metadata. Its spec advertises the
transcript capabilities consumed by D&D modules.
Normalize modules receive the structured LLM client, configured Scriptorium
profile ID, prompt session ID, and reference material through
`contracts.NormalizeRequest` when they need model-backed reconciliation.
Parsing is strict about required values and duplicate unit IDs but deliberately
ignores unrelated Seriatim fields. The external format and derived-identity
rules are defined in the
[Seriatim contract](../integrations/seriatim.md).
Merge modules receive the structured LLM client, configured Scriptorium profile
ID, prompt session ID, raw source input material, and reference material through
`contracts.MergeRequest` when they need model-backed merge behavior.
## Chunkers
## `seriatim` Input
### `internal/modules/chunk/generic`
Package: `internal/modules/input/seriatim`
The generic chunker validates the source document, walks units in configured
windows, clones each selected unit, and emits deterministic ordered chunk IDs.
Overlap changes the next window start but never reorders units. It records the
first and last unit and unit count in chunk metadata.
The `seriatim` adapter parses Seriatim transcript JSON into a generic source
document. It owns transcript JSON details, source ID selection, source digest
creation, transcript segment validation, and segment metadata mapping.
The accepted options and defaults are defined in
[Configuration](../config.md#implemented-production-modules). Generic
framework validation canonicalizes the returned unit slices before extraction.
Provides:
### `internal/modules/chunk/dnd/scenes`
- `source.transcript`
- `transcript.speaker`
- `transcript.timestamps`
The scene chunker prepares a structured Scriptorium request from the full
transcript, session, and optional D&D reference inputs. It validates the model's
scene boundaries against source-unit IDs and converts them into deterministic
chunks.
External JSON shape belongs in the Seriatim integration doc.
Scene validation requires sequential, contiguous, non-overlapping coverage from
the first source unit through the last. Each chunk contains JSON scene content
and module-owned metadata for the scene description, boundaries, confidence,
participants, and unit count. Boundary caveats become warnings. Malformed
structured output is returned as an error; there is no fallback chunker.
## `generic` Chunker
The package embeds its prompt and response schema and reports their non-secret
identity and hashes through singleton module metadata. Shared D&D assets supply
reference declarations and prompt inputs; their user-facing keys and accepted
file types remain canonical in [Configuration](../config.md).
Package: `internal/modules/chunk/generic`
## Extractor
The `generic` chunker splits source units into ordered chunks. It validates the
source document, clones source units, assigns chunk IDs such as `chunk-000001`,
and records chunk metadata for start unit, end unit, and unit count.
### `internal/modules/extract/dnd/spells`
The pipeline runner canonicalizes chunk units from the source document by
integer ID before extractors and mergers run. Chunkers also populate chunk
start and end unit IDs, content bytes, and media type. Chunker-owned context
should stay in `SourceChunk.Metadata`.
The spell extractor prepares a structured request from one chunk, the
chunk-scoped source input, the session, and optional D&D reference inputs. It
decodes the model response, assigns the generic source identity to every source
reference, canonicalizes duplicate references, orders spell casts by their
earliest cited unit, and returns raw JSON plus response-schema provenance.
Options:
The package owns its embedded prompt, response schemas, and prompt/schema
manifest metadata. Shared D&D helpers keep prompt input names and source-unit
reference conversion consistent with the scene chunker. The extractor produces
raw output; production validators own approval policy.
- `max_units`: positive integer, default `50`;
- `overlap_units`: non-negative integer, default `0`, and less than
`max_units`.
The durable payload and manifest metadata shapes are defined in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
Provides:
## Merger And Normalizer
- `chunks`
### `internal/modules/merge/appendorder`
## `dnd/scenes` Chunker
The merger preserves extract-result order. It passes through one JSON result,
concatenates a common top-level array field across multiple JSON objects, and
otherwise emits an array of the decoded values. It rejects invalid JSON and
non-JSON media types, and it preserves compatible schema provenance.
Package: `internal/modules/chunk/dnd/scenes`
### `internal/modules/normalize/noop`
The `dnd/scenes` chunker uses the structured LLM client to divide transcript
source units into coherent D&D scenes. It supplies the embedded Scriptorium
prompt ID, prompt version, transcript input material, response schema, and
session ID to the runtime; validates model-authored source-unit boundaries; and
converts each scene into a deterministic source chunk.
The normalizer defensively clones the accepted merge result, including payload
bytes, metadata, warnings, and schema provenance, without changing its logical
content.
Its prompt definition lives under `assets/prompts` and its schema under
`assets/schemas`. Shared reusable D&D prompt fragments are provided by
`internal/modules/sharedassets/dnd` and referenced from prompt definitions under
`./sharedassets/`.
## Output Encoder
Requires:
### `internal/modules/output/json`
- `source.transcript`
The JSON encoder sorts normalized results by lane, derives collision-checked
safe logical names, pretty-prints JSON payloads, and assembles the logical index,
manifest, rejected-result, warning, and lane files. Invalid JSON, unsupported
media types, unsafe names, and sanitized-name collisions are errors.
Provides:
The encoder returns logical files only. The CLI places them on disk, and the
[JSON output contract](../integrations/json-output.md) defines their external
paths and schemas.
- `chunks`
- `chunks.scenes`
## Generic Validators
Options: none. Non-empty options are rejected.
The unconditional accept and reject validators provide deterministic production
registrations used primarily for controlled composition and tests.
The chunker enforces full source-unit coverage from the first source unit to the
last, sequential contiguous scenes, and no overlap. Its LLM-facing schema uses
integer `start_unit_id` and `end_unit_id` values matching source-unit IDs. It
assigns chunk IDs such as `scene-000001`, emits JSON chunk content, and stores
scene metadata including title, primary mode, participants, summary, boundary
note, confidence, boundary unit IDs, and unit count. Boundary caveats become
warnings with reason code
`scene_boundary_caveat`. Whitespace-only caveats are treated as malformed
structured output rather than silently dropped.
The JSON syntax validator uses `encoding/json` to reject malformed payloads. The
JSON Schema validator requires schema bytes on the validation request, parses
the instance and schema with `jsonschema`, and distinguishes payload rejection
from schema loading or compilation errors. Neither validator calls the LLM.
Malformed model output fails explicitly rather than falling back to another
chunker. The chunker exposes prompt and response-schema provenance through
top-level `module_metadata.chunker` without raw prompts, raw schemas, source
text, or secrets.
## D&D Spell Validators
## `dnd/spells` Extractor
`internal/validators/extract/dnd/spells/spellpayload` provides strict decoding,
shape checks, source-reference candidates, and cited-text lookup shared by the
three validators.
Package: `internal/modules/extract/dnd/spells`
The shape validator rejects malformed JSON, unknown fields, missing or empty
spell fields, and empty reference lists. The source-reference validator applies
generic source-reference validation to every cited range. The relatedness
validator approves structurally valid payloads but warns when a case-insensitive
spell name is absent from all cited source text. It leaves malformed payloads to
the earlier validators in the configured chain.
The `dnd/spells` extractor owns D&D spell-cast extraction semantics. It
supplies the embedded Scriptorium prompt ID, prompt version, chunk-scoped
transcript input material, reference input materials, response schema, and
session ID to the runtime; then returns the structured LLM `spell_casts`
response as raw JSON.
Its LLM-facing source-reference schema uses integer `start_unit_id` and
`end_unit_id` values matching source-unit IDs.
Its prompt definition lives under `assets/prompts` and its schema under
`assets/schemas`. Shared reusable D&D prompt fragments are provided by
`internal/modules/sharedassets/dnd` and referenced from prompt definitions under
`./sharedassets/`.
Requires:
- `chunks`
- `source.transcript`
Provides:
- `dnd.spell_casts`
Response schema identity:
- schema ID: `notarius.dnd.spells`
- schema name: `notarius_dnd_spells_v1`
- schema version: `v1`
The extractor adds prompt and response-schema provenance to lane manifest
metadata under `artifact_lanes[].metadata.extractor`. Durable raw output
details belong in the
[D&D spell raw output contract](../integrations/dnd-spell-artifacts.md).
The production catalog validates `dnd/spells` raw extract output with generic
JSON validators followed by D&D spell validators under
`internal/validators/extract/dnd/spells`. The extractor itself remains
responsible for prompt, schema, and raw output production rather than
approve/reject policy.
The `dnd/scenes` chunker and `dnd/spells` extractor declare optional `players`,
`party`, and `glossary` reference slots accepting UTF-8 plain text, Markdown,
YAML, or JSON. They also accept `roster` as a deprecated compatibility alias for
`party`. Their prompts frame references as supporting disambiguation material
only; spell-cast artifacts must still be grounded in the source transcript.
## `appendorder` Merger
Package: `internal/modules/merge/appendorder`
The `appendorder` merger preserves chunk order for raw extract outputs. A
single JSON extract output is passed through as the merge output. Multiple JSON
object outputs with one common top-level array field are merged by concatenating
that array field in chunk order. Other valid JSON shapes are merged as a JSON
array of decoded values in chunk order. Non-JSON media types and invalid JSON
are rejected.
Provides:
- `merged`
## `noop` Normalizer
Package: `internal/modules/normalize/noop`
The `noop` normalizer clones the raw merge output and returns it unchanged.
Requires:
- `merged`
Provides:
- `normalized`
## `json` Output
Package: `internal/modules/output/json`
The `json` output encoder converts normalized raw outputs, rejected raw outputs,
warnings, and the run manifest into logical JSON output files. It writes one
payload file per lane under `lanes/` and sanitizes lane IDs for file names.
Normalized output payloads must be valid `application/json`.
Requires:
- `normalized`
Provides:
- `encoded`
Durable output file shapes belong in the
[JSON output contract](../integrations/json-output.md). Operator behavior
belongs in [Operations](../operations.md).
These validators are deterministic. Their selectable keys and production order
are defined in
[Configuration](../config.md#implemented-production-validators); their durable
payload rules are defined in the
[artifact contract](../integrations/dnd-spell-artifacts.md).
## Production Registration
Production registration is centralized in `internal/cli/catalog.go`.
`internal/cli/catalog.go` builds the production registries, registers module and
validator constructors, installs default validator-chain mappings, and exposes
the matching catalog for resolution. It also collects prompt assets from
LLM-backed packages before constructing the production client.
Do not make framework code import production modules. The CLI wires production
modules at the application boundary; tests may provide fake registries or fake
catalogs directly.
Framework packages must not import production extensions. Tests may compose
registries and catalogs directly with fakes.
## Adding A Module
## Adding An Extension
When adding a module, keep source-format and extraction-domain boundaries clear:
When adding a production module or validator:
- input modules may know external source formats;
- extract modules may know artifact semantics and prompt/schema assets;
- merge and normalize modules own raw output combination and reconciliation;
- output modules own serialization, not diagnostics or CLI reporting.
1. implement the stage or validator contract and package-local key;
2. expose and test its spec, constructor, and registration function;
3. keep format or domain parsing inside the concrete package;
4. add package-owned prompt/schema assets when the extension is LLM-backed;
5. register it in `internal/cli/catalog.go` and add a default chain only when
production policy requires one;
6. add resolution and composition coverage for capabilities, options,
references, and validation behavior;
7. update the selectable-key catalog in [Configuration](../config.md), the
relevant external contract, this inventory, and maintained examples when
user-visible behavior changes.
Update [Development](../development.md), [Configuration](../config.md),
internal docs, integration docs, and examples when the new module becomes
implemented production behavior.
Do not add the extension to `docs/development.md`; that file routes by task and
does not inventory implementations.
## Tests To Inspect
- Package-local `*_test.go` files under the module or validator being changed.
- `internal/framework/pipeline/registry_integration_test.go`: registry and spec
composition.
- `internal/framework/pipeline/default_modules_test.go`: framework binding
defaults.
- `internal/cli/run_test.go`: production catalog, config resolution, and
end-to-end CLI composition.
- `internal/modules/sharedassets/**/*_test.go`: shared prompt and reference
assembly.

View File

@@ -1,127 +1,105 @@
# Internal Overview
This document maps the implemented Notarius components and their ownership. It
complements the durable invariants in [Architecture](../policy/architecture.md)
and links to focused internal documentation for deeper behavior.
This document inventories the implemented Notarius components. Normative
boundaries and dependency direction belong in
[Architecture](../policy/architecture.md); external behavior belongs in the
[CLI](../cli.md), [Configuration](../config.md),
[Operations](../operations.md), and [integration contracts](../integrations/).
## Execution Path
The executable delegates to the CLI, which resolves configuration and wires the
production application around the framework runner:
`cmd/notarius` delegates to `internal/cli`, the production composition root.
The CLI loads configuration, builds the production catalogs and runtime
collaborators, invokes `internal/framework/pipeline`, and places the logical
output files returned by the runner. Diagnostics, checkpoints, and debug
recorders are optional side-channel collaborators supplied at this boundary.
```text
cmd/notarius
-> internal/cli
-> config resolution + production registries + LLM client
-> input -> chunk -> extract -> merge -> normalize -> output
-> durable output writes
Pipeline side channels:
diagnostics checkpoints debug artifacts
```
Pipeline execution is serial. Configuration selects modules for the fixed stage
shape; registries construct them after profile, capability, validator, and
reference resolution.
Pipeline execution is serial. Resolution produces a fixed ordered workflow and
a sorted set of artifact lanes before the runner constructs any stage module.
## Application Boundary
`cmd/notarius` contains the executable entry point and delegates process exit
behavior to `internal/cli`.
`internal/cli` owns command parsing, configuration discovery, production module
and validator registration, prompt asset collection, production LLM client
construction, reference preparation, workspace recorder setup, durable output
writes, and user-facing stdout, stderr, and exit codes. It is the composition
root for concrete production packages.
| Package | Implemented responsibility |
| --- | --- |
| `cmd/notarius` | Executable entry point and process exit delegation. |
| `internal/cli` | Command parsing, config discovery, production registration, prompt asset collection, LLM client construction, reference materialization, workspace collaborator setup, durable writes, and user-facing results. |
## Core Packages
| Package | Implemented responsibility |
| --- | --- |
| `internal/core/artifacts` | Run manifests and artifact serialization shapes. |
| `internal/core/config` | Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline configuration. |
| `internal/core/diagnostics` | Diagnostics run directories, artifact writers, atomic writes, and retention decisions. |
| `internal/core/source` | Generic source documents, units, references, and validation. |
| `internal/core/workspace` | Workspace settings, safe paths and writes, checkpoint identities, and checkpoint manifest types. |
These packages provide concrete, deterministic models and policy. Production
module registration occurs at the CLI boundary rather than in core packages.
| `internal/core/artifacts` | Run-manifest and provenance models. |
| `internal/core/config` | Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline resolution. |
| `internal/core/diagnostics` | Scoped run directories, diagnostics writers, atomic writes, and retention decisions. |
| `internal/core/source` | Generic source documents, units, references, lookup, and validation. |
| `internal/core/workspace` | Effective workspace settings, confined paths and writes, checkpoint identity, and checkpoint manifest models. |
## Framework Packages
| Package | Implemented responsibility |
| --- | --- |
| `internal/framework/contracts` | Stage, validator, reference, output, and structured LLM interfaces and request/result types. |
| `internal/framework/pipeline` | Module registries, profile resolution, capability checks, reference materialization, validation chains, retries, orchestration, warnings, and manifest population. |
| `internal/framework/contracts` | Stage, validator, reference, output, and structured-completion interfaces and data types. |
| `internal/framework/pipeline` | Registries, profile resolution, capability checks, reference materialization, validator-chain resolution, retries, orchestration, warnings, and manifest population. |
| `internal/framework/validate` | Shared validator decision and cardinality helpers. |
| `internal/framework/llm` | Scriptorium-backed structured completions, prompt/schema asset registration, scheduling, profile recording, and secret redaction. |
| `internal/framework/checkpoint` | Workspace-backed checkpoint loading, recording, and payload envelopes. |
| `internal/framework/debug` | Workspace-backed framework and LLM debug artifacts. |
| `internal/framework/llm` | Scriptorium-backed structured completions, prompt/schema registration, scheduling, profile recording, and secret redaction. |
| `internal/framework/checkpoint` | Workspace-backed checkpoint loading, recording, and payload serialization. |
| `internal/framework/debug` | Workspace-backed framework and LLM debug recording. |
Framework contracts carry raw stage outputs between modules. The runner owns
provenance, validation sequencing, rejection handling, checkpoint boundaries,
debug boundaries, and final manifest assembly.
Framework contracts carry raw stage results between implementations. The
runner owns handoff provenance, validation sequencing, rejection handling,
checkpoint and debug boundaries, and final manifest assembly.
## Production Modules
## Production Extensions
Production implementations live under `internal/modules` and register through
the CLI catalog.
The canonical catalogs of user-selectable
[module](../config.md#implemented-production-modules) and
[validator](../config.md#implemented-production-validators) keys are in
Configuration. The implemented module packages are:
| Stage | Module key | Package | Role |
| --- | --- | --- | --- |
| Input | `seriatim` | `internal/modules/input/seriatim` | Converts Seriatim transcript JSON into the generic source model. |
| Chunk | `generic` | `internal/modules/chunk/generic` | Splits ordered source units by configured unit counts and overlap. |
| Chunk | `dnd/scenes` | `internal/modules/chunk/dnd/scenes` | Uses structured LLM output to create contiguous D&D scene chunks. |
| Extract | `dnd/spells` | `internal/modules/extract/dnd/spells` | Extracts source-grounded D&D spell-cast artifacts. |
| Merge | `appendorder` | `internal/modules/merge/appendorder` | Combines accepted extract outputs in chunk order. |
| Normalize | `noop` | `internal/modules/normalize/noop` | Preserves accepted merged output unchanged. |
| Output | `json` | `internal/modules/output/json` | Encodes manifests, indexes, warnings, rejections, and accepted lane payloads as logical JSON files. |
| Package | Implemented responsibility |
| --- | --- |
| `internal/modules/input/seriatim` | Parses the supported Seriatim transcript format into the generic source model. |
| `internal/modules/chunk/generic` | Splits ordered source units by unit count and overlap. |
| `internal/modules/chunk/dnd/scenes` | Produces contiguous D&D scene chunks from structured model output. |
| `internal/modules/extract/dnd/spells` | Produces source-grounded D&D spell-cast raw output. |
| `internal/modules/merge/appendorder` | Combines accepted extraction results in chunk order. |
| `internal/modules/normalize/noop` | Preserves accepted merged output. |
| `internal/modules/output/json` | Encodes manifests, lane payloads, warnings, and rejections as logical JSON files. |
`internal/modules/sharedassets` composes shared prompt filesystems.
`internal/modules/sharedassets/dnd` owns shared D&D prompt fragments, reference
slots, prompt input assembly, and source-unit reference helpers.
`internal/modules/sharedassets/dnd` owns reusable D&D prompt fragments,
reference declarations, prompt input assembly, and source-unit reference
helpers.
## Validators
Concrete validators live under `internal/validators`. Generic packages provide
unconditional test decisions, JSON syntax validation, and JSON Schema
validation. D&D spell packages provide shape, source-reference, and
source-relatedness decisions, with `spellpayload` holding their shared parser
and lookup helpers. Production chain composition is owned by `internal/cli`.
Concrete validators live under `internal/validators` and register separately
from stage modules. Generic validators cover unconditional test decisions, JSON
syntax, and JSON Schema. D&D spell validators cover artifact shape, source
reference validity, and source relatedness.
Implementation details for all production extensions are in
[Module Internals](modules.md).
The production default chain for `dnd/spells` extract output is registered
centrally in `internal/cli`; module packages produce raw output but do not own
the production approve/reject policy.
## Run-State Components
## Files And Run State
Notarius keeps distinct output and inspection surfaces:
| Surface | Owner | Purpose |
| Surface | Implemented owners | Internal purpose |
| --- | --- | --- |
| Durable output | Output module and CLI writer | User-consumable run files. |
| Diagnostics | `internal/core/diagnostics` and CLI | Redacted run inspection, reports, warnings, and failures. |
| Checkpoints | `internal/framework/checkpoint` | Validated stage reuse for explicit resume. |
| Debug artifacts | `internal/framework/debug` and pipeline instrumentation | Sensitive framework-boundary and LLM call inspection. |
| Durable output | Output module, pipeline runner, and CLI writer | Return logical consumer files and place them for a run. |
| Diagnostics | `internal/core/diagnostics` and `internal/cli` | Record redacted invocation, resolution, result, and failure inspection data. |
| Checkpoints | `internal/framework/checkpoint` and `internal/core/workspace` | Validate and serialize reusable stage outcomes. |
| Debug artifacts | `internal/framework/debug` and pipeline instrumentation | Capture sensitive framework-boundary and LLM-call material. |
Workspace settings determine whether and where diagnostics, checkpoints, and
debug artifacts are written. Concrete stage modules do not receive workspace
paths.
Physical layout, retention, recovery, and sensitive-data handling are defined
in [Operations](../operations.md). Concrete stage modules receive recorder
interfaces and request data, not workspace paths.
## Focused Internal Documentation
## Focused Documentation
- [Pipeline Internals](pipeline.md): resolution, execution, validators,
references, retries, checkpoints, outputs, and manifests.
- [Module Internals](modules.md): production module contracts, capabilities,
options, prompts, schemas, and registration.
- [Pipeline Internals](pipeline.md): resolution, execution, validation, retries,
checkpoint/debug hooks, and result assembly.
- [Module Internals](modules.md): production modules, validators, assets,
registration, and the contributor recipe for adding an extension.
- [LLM Runtime](llm.md): structured completion contracts, Scriptorium adapter,
assets, scheduling, profile recording, and redaction.
- [Diagnostics Internals](diagnostics.md): diagnostics files, retention,
failure behavior, and path safety.
## Test Surfaces
The repository uses focused package tests, registry and pipeline composition
tests, a fake-backed walking skeleton, fixture-driven CLI coverage, and local
test servers for LLM integration behavior. Tests do not require real provider
calls.
- [Diagnostics Internals](diagnostics.md): scoped writers, retention
coordination, CLI failure flow, and path safety.

View File

@@ -1,295 +1,185 @@
# Pipeline Internals
The implemented pipeline runner lives in `internal/framework/pipeline`. It
executes the fixed workflow defined by the architecture policy:
The implemented resolver and runner live in `internal/framework/pipeline`.
Their fixed workflow and ownership boundaries are defined by
[Architecture](../policy/architecture.md#system-shape). Configuration fields,
defaults, and selectable keys are defined in
[Configuration](../config.md#pipelines).
```text
input -> chunk -> extract -> merge -> normalize -> output
```
Pipeline execution is serial. Resolution fixes the selected lanes and all
stage bindings before the runner constructs stage implementations.
Pipeline execution is serial. The runner executes the resolved lanes one after
another in the fixed workflow order.
## Resolution
## Profile Resolution
`internal/core/config.Config.Resolve` validates the loaded configuration,
selects the named profile, applies the runtime inputs supplied by the CLI, and
calls `pipeline.ResolvePipeline`.
Config loading produces `pipeline.PipelineProfile` values. Resolution happens
before execution:
`ResolvePipeline`:
1. `internal/core/config.Config.Resolve` validates config and finds the named
pipeline.
2. The optional lane selection is passed to `pipeline.ResolvePipeline`.
3. Module bindings are defaulted:
- chunk: `generic`
- merge: `appendorder`
- normalize: `noop`
- output: `json`
- LLM profile: empty, which lets Scriptorium prompt defaults choose a
profile.
4. The module catalog is checked for each bound module key.
5. Module capabilities are checked in workflow order.
6. A digest is calculated from the resolved pipeline without the digest field.
1. selects and sorts artifact lanes;
2. completes omitted bindings using the documented configuration defaults;
3. looks up each module and validator spec without constructing it;
4. checks required and provided capabilities in workflow order;
5. resolves target-aware reference bindings and validator chains;
6. calculates a digest over the resolved structure.
The CLI writes the resolved pipeline and digest to diagnostics.
Resolution returns a `ResolvedPipeline` containing ordered lanes, concrete
bindings, validator chains, reference targets, and the digest. It does not read
reference bytes or construct runtime modules. CLI lane and reference selector
syntax is defined in the [CLI reference](../cli.md#run).
Pipeline profiles and artifact lanes may include reference binding maps keyed by
reference slot name. During resolution, pipeline-level bindings act as defaults
for selected chunk, extractor, merger, and normalizer targets that declare the
slot; target-local bindings override or add bindings for that target. Runtime
`--reference` requests override target config bindings, and runtime unbinds
remove optional target bindings. Flat runtime slot names are resolved only when
exactly one selected target declares the slot; otherwise the CLI requires a more
specific selector such as `chunk.slot`, `lane.extract.slot`,
`lane.merge.slot`, or `lane.normalize.slot`. Resolution validates bindings
against the declaring target specs and stores the bindings in target-aware
resolved reference holders. It does not read reference files or include
reference bytes in source digests.
## Reference Materialization
During run preparation, resolved file references for chunk, extractor, merger,
and normalizer targets are materialized before any LLM-backed pipeline work. Config
bindings resolve relative to the config file, and CLI bindings resolve relative
to the current working directory. Materialization accepts UTF-8 text files,
computes `sha256:` content digests, records file origins, infers canonical base
media types from file extensions, enforces declared byte limits, and warns for
empty bound files. Media-type acceptance is checked only when a slot declares
`AcceptedMediaTypes`; unknown extensions are recorded as
`application/octet-stream`. Reference content is omitted from diagnostics and
manifests. The CLI writes provenance-only resolved reference diagnostics, and
the run manifest records target-stage reference provenance separately from
source digests. Runtime reference content is passed to the matching chunker,
extractor, merger, or normalizer request. LLM-backed modules pass that material
onward as named Scriptorium prompt inputs.
The CLI calls `MaterializeReferences` after resolution and before constructing
the LLM client or running the pipeline. The materializer checks each binding
against its resolved target declaration, reads and validates the file, and
builds both a `contracts.ReferenceSet` and provenance-only metadata on the
corresponding `ResolvedReferenceTarget`.
The CLI carries raw input bytes into `pipeline.RunInput`. Input adapters parse
those bytes into the source document. Chunk, merge, and normalize requests
receive the original source material as `SourceInput`; extraction requests
receive chunk-scoped source material built from the current `SourceChunk`
content, media type, and origin metadata. The raw input payload is not written
to manifests or default diagnostics.
The runner clones the resulting set into the chunk, extract, merge, or normalize
request that owns the target. LLM-backed extensions may convert those items into
named prompt inputs. Reference content remains separate from source evidence and
source digests.
The CLI also carries an optional run `session_id`. The runner makes it available
to chunk, extract, merge, and normalize requests; LLM-backed modules forward it
through their structured completion requests so Scriptorium can include it in
prompt execution metadata.
Binding precedence, path resolution, accepted content, and media-type behavior
are configuration contracts; see [Configuration](../config.md#pipelines).
Durable provenance is defined in the
[JSON output contract](../integrations/json-output.md#manifestjson), while
runtime sensitive-data handling belongs in [Operations](../operations.md).
When workspace resume checkpointing is enabled, the CLI constructs a checkpoint
recorder after pipeline resolution and reference materialization and passes it
through `pipeline.RunInput`. The runner records source, chunk, extract, merge,
and normalize outcomes through that interface. Concrete modules do not receive
workspace paths and do not write checkpoint files directly.
## Registries And Specs
For `run --resume`, the CLI also passes a checkpoint loader. The runner consults
the loader in workflow order and reuses only checkpoints whose manifest schema,
status, identity digest, dependency fingerprints, payload files, and payload
digests validate for the current invocation. The identity includes the resolved
pipeline, selected lanes, source/input digest, runtime overrides that affect
execution, and materialized reference digests. Missing or invalid checkpoints
fall back to normal execution and are refreshed by the recorder.
`pipeline.Registries` holds constructors used during execution.
`pipeline.ModuleCatalog` exposes their specs during configuration validation and
resolution. Separate registries exist for every stage and for validators;
`ValidatorChainRegistry` stores production default-chain mappings.
When workspace debug output is enabled, the CLI passes a debug recorder for the
current run ID. The runner writes framework-boundary inputs, outputs,
structured LLM calls, validator calls, timing, and retry attempt metadata
through that interface. Each retry or validator attempt records any LLM calls
made within that attempt in an `llm_calls` array and writes paired
`prompt-000N.json` and `response-000N.json` metadata files under the attempt
directory. LLM response bodies are written as sibling `response-content-000N.*`
files, using pretty-printed JSON when the content is valid JSON and raw text
otherwise. Debug output is not used for resume and can contain sensitive source,
reference, prompt, and model-output material. Concrete modules still do not
receive workspace paths.
A `ModuleSpec` declares its stage plus required and provided capabilities.
Chunk, extract, merge, and normalize specs may also declare reference slots.
Registry implementations defensively copy spec metadata, reject duplicate keys,
and verify that a constructed implementation reports the registered key.
## Registries And Module Specs
A `ValidatorSpec` declares a validator key and execution class. Resolution uses
the execution class to reject incompatible profile bindings before execution.
The current production catalog and default chain are listed only in
[Configuration](../config.md#implemented-production-validators).
`pipeline.Registries` holds concrete constructors for execution. A
`pipeline.ModuleCatalog` exposes module specs for config validation and
resolution. The catalog also exposes validator specs and central default
validator-chain mappings without constructing modules or validators.
## Runner Boundary
Every production module registers a `ModuleSpec` with:
`pipeline.RunInput` carries the resolved pipeline, raw source input, structured
LLM client, run identity and timing, optional session and profile metadata, and
checkpoint/debug collaborators. The runner parses source bytes through the
selected input adapter. Later stage requests receive the generic source model;
extract requests receive chunk-scoped input material, while chunk, merge, and
normalize requests retain access to the original source material.
- `Key`: module key used in config;
- `Stage`: module kind such as input, chunk, extract, merge, normalize,
validate, or output;
- `Provides`: capabilities added after that module runs;
- `Requires`: capabilities that must already be available.
`pipeline.RunOutput` carries the run manifest, accepted normalized results,
rejected results, warnings, checkpoint events, and logical files returned by the
output encoder. The CLI owns diagnostics and durable filesystem writes after the
runner returns.
Chunk, extract, merge, and normalize specs may also declare reference slots. Slot
declarations are available from registry metadata without constructing module
instances. Input, validate, and output specs must not declare reference slots.
Capability checks prevent incompatible pipeline composition before a run starts.
Every production validator registers a `ValidatorSpec` with:
- `Key`: validator key used in config and manifests;
- `ExecutionClass`: `deterministic` or `llm_backed`.
Default validator chains are keyed by workflow stage and module key. Production
currently registers a default chain for `extract` module `dnd/spells` only.
## Runner Input And Output
`pipeline.RunInput` carries:
- a `ResolvedPipeline`;
- optional source ID, input path, and raw input bytes;
- a structured LLM client;
- run ID, start time, LLM profile manifest metadata, and CLI metadata.
`pipeline.RunOutput` carries:
- run manifest;
- normalized raw outputs;
- rejected raw outputs;
- warnings;
- logical output files returned by the output encoder.
The CLI owns durable file writes and diagnostics writes after the runner returns.
## Execution
## Execution Flow
The runner:
1. validates run input and registries;
2. builds the input adapter and parses the raw input into a source document;
3. validates the source document;
4. builds the chunker and produces source chunks, retrying when configured;
5. validates source chunks against framework invariants and the resolved chunk
validator chain;
6. runs each selected artifact lane in sorted resolved order;
7. builds the output encoder and validates logical output file names.
8. passes accepted normalized raw outputs, rejected output records, warnings,
and the manifest to the output encoder.
1. validates its input and registries;
2. builds the input adapter, parses the raw input, and validates the generic
source document;
3. obtains or executes the chunk result;
4. validates and canonicalizes chunks;
5. executes each resolved artifact lane in order;
6. builds the output encoder and validates its logical file results;
7. returns the assembled manifest, outcomes, warnings, and files.
## Chunk Results
Within each artifact lane, it builds the extractor, merger, and normalizer,
then performs these transitions:
Chunkers implement `contracts.Chunker` and receive a `contracts.ChunkRequest`
with the validated source document, reference set, structured LLM client, the
configured LLM profile, module options, and run metadata. Deterministic and
LLM-backed chunkers use the same contract; provider construction stays outside
chunk modules.
1. extract once per accepted chunk and add runner-owned lane, source, and chunk
provenance;
2. validate each raw extract result and omit rejected results from merge input;
3. skip the rest of the lane when no extract result is accepted;
4. merge accepted extract results in their existing order;
5. validate the merge result and skip normalization on rejection;
6. normalize the accepted merge result;
7. validate and append the accepted normalized result.
When chunking succeeds, the runner validates generic chunk invariants before
running extractors:
Module-provided warnings and payload warnings are promoted only from attempts
whose results are accepted and used.
- chunk IDs must be non-empty and unique in the chunk result;
- each chunk `SourceID` must match the source document ID;
- each chunk `Index` must match its zero-based returned order;
- each chunk start and end unit ID must exist in the source document, with the
start unit at or before the end unit;
- each chunk must include non-empty extraction content and media type;
- each chunk must contain at least one source unit;
- a chunk must not repeat a source unit;
- every chunk source unit must exist in the source document;
- source units inside each chunk must appear in source-document order.
## Chunk Canonicalization
After validation, the runner rebuilds each chunk from source-document units by
integer ID, preserving chunk boundaries, content bytes, media type, and cloned
chunk metadata. Extractors and downstream stages therefore see canonical source
units, while `SourceChunk.Metadata` remains the supported place for
chunker-owned context.
Before lane execution, generic validation requires unique chunk IDs, matching
source identity, indexes matching returned order, valid ordered boundaries,
non-empty content and media type, and at least one valid source unit per chunk.
Units may not repeat inside a chunk and must preserve source-document order.
If chunk validation rejects the chunk result after configured retries, the runner
records a rejected raw output and skips downstream lane execution. Framework-level
chunking or validation errors that remain after configured retries fail the run.
The runner then rebuilds each chunk's unit slice from the source document by
unit ID. It preserves the module-owned boundaries, content, media type, and
cloned metadata. The framework permits gaps and overlap between separate
chunks; stricter coverage policy belongs to the chunk implementation.
The framework does not require complete source-unit coverage and does not reject
overlap between different chunks. Stricter policies, such as full coverage or
non-overlap, belong to individual chunk modules when they are part of that
module's contract.
## Validation And Retries
Within an artifact lane, the runner:
Chunk, extract, merge, and normalize results pass through the resolved validator
chain for their stage and module. Each validator receives the raw payload plus
the relevant source, chunk, prior-stage, schema, reference, session, LLM, option,
and run context. Validators execute in resolved order and stop at the first
error or rejection. An empty chain approves the result.
1. builds the extractor, merger, and normalizer;
2. records module manifest metadata when modules provide it;
3. extracts one raw `ExtractOutput` from each accepted chunk, retrying when
configured;
4. fills runner-owned provenance on each extract output, including lane ID,
extractor key, source ID, chunk ID, and chunk index;
5. validates raw extract outputs and omits rejected outputs from merge input;
6. merges ordered accepted extract outputs into one raw `MergeOutput`, retrying
when configured;
7. validates raw merge output and skips normalization for rejected merge output;
8. normalizes the accepted merge output into one raw `NormalizeOutput`,
retrying when configured;
9. validates raw normalize output and appends accepted normalized raw output to
`RunOutput.NormalizeOutputs`.
`runWithRetry` applies the effective retry policy around module execution and
its complete validation chain. A module or validator error becomes a framework
error when attempts are exhausted. A rejection becomes a recorded
`RejectedOutput` when attempts are exhausted. Cancellation stops retry
processing immediately.
## Validators
Rejected output is a non-fatal pipeline outcome and does not advance. Warnings
from discarded attempts are not promoted. Configuration owns retry counts and
validator overrides; see [Module Bindings](../config.md#module-bindings).
The runner handoff is raw-output based. Chunkers, extractors, mergers, and
normalizers do not advertise validator chains through their module interfaces.
Resolved validation chains receive the raw module output plus stage, lane,
module, source, chunk, schema, session, reference, LLM client/profile, binding
option, and run metadata context. Chunk validators receive the chunk result
collection, merge validators receive the ordered extract outputs used by the
merge, and normalize validators receive the accepted merge output. Empty chains
approve output by default. Response-schema provenance may include in-memory JSON
schema bytes for validators. Those bytes are omitted from manifests,
diagnostics, and encoded output files.
## Checkpoint And Debug Hooks
Resolved validator chains come from central default mappings unless a
stage-local config override is set on `chunk`, lane `extract`, lane `merge`, or
lane `normalize`. Explicit empty overrides are valid and are recorded as empty
chains in manifests. Explicit non-empty overrides replace the default chain and
preserve configured order.
The runner depends on recorder and loader interfaces, using no-op
implementations when collaborators are absent. Each checkpointed workflow
boundary records a running, succeeded, or failed transition. Reuse decisions
are consulted in workflow order and accepted payloads are cloned before
entering the normal handoff path. Dependency fingerprints connect later
checkpoints to the exact accepted results on which they depend.
The production default chain for `extract` module `dnd/spells` is:
Debug instrumentation wraps run, stage, attempt, validator, and structured LLM
boundaries. Context scopes associate nested LLM calls with the module or
validator attempt that made them. Debug-write failures are framework errors;
debug data is never used as a checkpoint source.
1. `generic/valid_json`
2. `generic/valid_json_schema`
3. `extract/dnd/spells/shape`
4. `extract/dnd/spells/source_refs`
5. `extract/dnd/spells/source_relatedness`
Checkpoint identity, physical layout, reuse behavior, and debug artifact
handling are operator contracts in [Operations](../operations.md). Serialization
and recorder implementation are inventoried in
[Internal Overview](overview.md#run-state-components).
No other production module currently has a default validator chain.
## Results And Failures
Validator rejection is a non-fatal run outcome: the rejected output is recorded
in `RunOutput.Rejected` and does not pass to the next stage. Validator execution
errors are framework-level errors and retry according to the relevant binding.
Warning-only validators return approved results with warnings; those warnings
are promoted only from successful attempts whose outputs are used.
The runner owns manifest assembly and handoff summaries but not the durable JSON
schema. It records resolved module and lane provenance, validator chains,
source/reference identities, selected LLM profiles, normalized and rejected
summaries, status, and timing. Raw payload bytes remain outside the manifest.
Module metadata providers may add non-secret singleton or lane-scoped metadata.
## Warnings And Failures
Execution errors include stage, module, lane, or validator context. Once a
manifest exists, a failing run returns it with failed status and completion
time. Successful status reflects whether any raw result was rejected. The
durable manifest and logical file schemas are defined in the
[JSON output contract](../integrations/json-output.md).
Warnings from the successful chunking, extraction, merging, and normalization
attempts whose outputs are used are accumulated in `RunOutput.Warnings`, along
with output encoder warnings. Warnings from discarded retry attempts are not
promoted to final warnings.
## Tests To Inspect
Errors wrap the operation and module key or lane context. If execution fails
after a manifest exists, the returned manifest is marked `failed` and receives a
completion timestamp.
On successful execution, the manifest validation status is:
- `approved` when no raw outputs were rejected;
- `rejected` when at least one raw output was rejected.
## Manifest Population
The manifest records run ID, pipeline ID, pipeline digest, module keys, top-level
module metadata, artifact lanes, LLM profile metadata, source digest,
reference provenance, normalized raw output summaries, rejected output
summaries, validation status, and timing. Raw output summaries include lane ID,
normalizer module key, media type, source ID, and response-schema provenance
when present. Rejected output summaries include stage, lane, module, chunk,
validator or reason, message, attempt count, and optional diagnostic artifact
path. The manifest does not include raw output payload bytes.
Singleton pipeline modules may add non-secret metadata by implementing
`contracts.ManifestMetadataProvider`. The runner records that metadata under
`module_metadata` with stable keys for `input`, `chunker`, and `output`.
Lane-owned modules may add non-secret metadata through
`artifact_lanes[].metadata`. The runner records extractor, merger, and
normalizer metadata there. The D&D spell extractor uses lane metadata for
prompt and response-schema provenance.
## JSON Output
The production JSON output encoder writes `manifest.json`, `index.json`,
`warnings.json`, `rejected.json`, and one pretty-printed JSON file per accepted
normalized lane output under `lanes/`. It accepts only normalized outputs with
valid `application/json` payloads. Unsupported media types, invalid JSON, unsafe
logical paths, and duplicate sanitized lane file names fail the run before
durable output files are written.
- `internal/core/config/effective_config_test.go`: config-to-resolution boundary.
- `internal/framework/pipeline/profile_test.go`: selection, defaults,
capabilities, validator chains, and digest behavior.
- `internal/framework/pipeline/references_test.go`: target resolution and
materialization.
- `internal/framework/pipeline/runner_test.go`: stage transitions, retries,
rejections, warnings, checkpoints, debug hooks, and manifests.
- `internal/framework/pipeline/walking_skeleton_test.go`: fake-backed complete
workflow composition.
- `internal/framework/checkpoint/*_test.go`: checkpoint serialization and reuse
collaborators.

View File

@@ -4,24 +4,16 @@ This is the canonical reference for operating implemented Notarius runs.
## Normal Run
A run reads one source file, resolves one configured pipeline, calls the
configured Scriptorium-backed LLM runtime, writes durable JSON output, and
writes diagnostics for inspection.
A run reads one source file, resolves one configured pipeline, executes its
modules, writes durable output, and writes diagnostics when enabled. Start with
the [README quickstart](../README.md), then use the [CLI reference](cli.md) for
invocation options.
```sh
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json \
--output-dir ./notarius-output \
--diagnostics-dir /tmp/notarius
```
The command prints a success line with the pipeline ID, normalized output count,
rejected output count, and the output path.
For production, configure a workspace such as `/var/lib/notarius` and ensure the
Notarius process can create files below it. For local development, prefer an
ignored project-local workspace such as `./.notarius/workspace`.
For production, configure an application-owned workspace such as
`/var/lib/notarius` and ensure the Notarius process can create files below it.
For local development, prefer an ignored project-local workspace such as
`./.notarius/workspace`. See [Configuration](config.md#workspace) for workspace
fields.
## Output Directory
@@ -31,23 +23,10 @@ Durable output is written to:
<output-root>/<run-id>/
```
The default output root is `./notarius-output`. Use `--output-dir` to choose a
different root.
The `json` output module writes these files:
- `index.json`: file index with paths to the manifest, lane output files,
rejected outputs, and warnings.
- `manifest.json`: run manifest with resolved pipeline provenance, top-level
module metadata, module keys, reference provenance, validation status, and
timing.
- `lanes/<lane-id>.json`: normalized raw JSON output payloads, one file per
lane. For the current D&D spell extractor, this includes `lanes/spells.json`.
- `rejected.json`: rejected raw output records.
- `warnings.json`: warnings reported by pipeline modules or the output encoder.
Output writes are atomic per file. Logical output file names must be clean,
relative, slash-separated paths and must not contain `..`.
The output root and its invocation-specific override are defined in the
[CLI reference](cli.md#run). Output writes are atomic per file. The
[JSON output contract](integrations/json-output.md) defines the logical files,
paths, schemas, and media types inside each run directory.
## Diagnostics Directory
@@ -57,17 +36,15 @@ Diagnostics are written under:
<diagnostics-work-dir>/<run-id>/
```
The default diagnostics work directory is `/tmp/notarius`. It can be set with
`workspace.directory`, `NOTARIUS_WORKSPACE_DIR`, legacy
`diagnostics.work_dir`, legacy `NOTARIUS_WORK_DIR`, or `--diagnostics-dir`.
When a workspace directory is set, diagnostics are written under
`<workspace.directory>/diagnostics/<run-id>/` unless `--diagnostics-dir`
overrides the diagnostics work directory for that invocation.
When a workspace directory is configured, diagnostics are written under
`<workspace.directory>/diagnostics/<run-id>/`. An invocation-specific override
changes only the diagnostics root, not the workspace root. Configuration and
environment controls are defined in [Configuration](config.md); the override
flag is defined in the [CLI reference](cli.md#run).
Set `workspace.diagnostics.enabled: false` or
`NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED=false` to skip diagnostics directory
creation and diagnostics artifact writes. Concise failures are still printed to
stderr.
Diagnostics can be disabled through configuration. When disabled, Notarius
does not create a diagnostics run directory or write diagnostics artifacts;
concise failures are still printed to stderr.
Implemented diagnostics artifacts:
@@ -87,45 +64,41 @@ Implemented diagnostics artifacts:
- `error.log`: failure message, written after diagnostics directory creation
when a run fails.
`source-document.json` is supported by the diagnostics writer but is not written
by the current CLI run workflow.
## Checkpoints
When `workspace.resume.enabled: true` and `workspace.directory` is set, runs
write checkpoints under:
When checkpoint writing is enabled for a configured workspace, runs write
checkpoints under:
```text
<workspace.directory>/checkpoints/<pipeline-id>/<input-key>-<source-or-input-digest>/<pipeline-digest>/<identity-digest>/
```
Each workflow step owns its own manifest and payload files. There is no
root-level checkpoint summary. Ordinary `notarius run` invocations execute the
pipeline normally and refresh checkpoints. `notarius run --resume` reuses valid
root-level checkpoint summary. Ordinary invocations execute the pipeline
normally and refresh checkpoints. An explicit resume invocation reuses valid
checkpoints and executes any missing, invalid, or incompatible step normally.
Configuration controls checkpoint writing, while the explicit resume option is
defined in the [Configuration](config.md#workspace) and
[CLI](cli.md#run) references.
Checkpoint payloads preserve byte content with base64 envelopes, media type,
metadata, warnings, and content digests where applicable. Checkpoints do not
include raw prompts, raw reference contents, raw LLM request payloads, or debug
traces. They can still contain source text, intermediate extracted content,
rejected outputs, metadata, and warnings. Treat checkpoint directories as
sensitive local state.
Checkpoints do not include raw prompts, raw reference contents, raw LLM request
payloads, or debug traces. They can still contain source text, intermediate
extracted content, rejected outputs, metadata, warnings, and content digests.
Treat checkpoint directories as sensitive local state.
A checkpoint is reused only when its workspace schema version, checkpoint
identity digest, step status, dependency fingerprints, payload files, and
payload digests match the current invocation. Changes to input bytes, resolved
pipeline digest, selected lanes, runtime LLM profile override, materialized
reference digests, or other identity material invalidate reuse and use a
separate checkpoint directory.
A checkpoint is reused only when its stored status, dependencies, payloads, and
digests match the current invocation. Changes to input bytes, the resolved
pipeline, selected lanes, the runtime LLM profile override, or bound reference
content invalidate reuse.
Plain `notarius run` does not reuse checkpoints. It executes the workflow and
refreshes checkpoint files when checkpointing is enabled. `notarius run
--resume` is the explicit reuse path.
Runs do not reuse checkpoints unless explicitly requested. Without reuse, the
workflow executes normally and refreshes checkpoint files when checkpointing is
enabled.
## Debug
When `workspace.debug.enabled: true` and `workspace.directory` is set, runs
write debug artifacts under:
When debug recording is enabled for a configured workspace, runs write debug
artifacts under:
```text
<workspace.directory>/debug/<run-id>/
@@ -135,34 +108,31 @@ Debug output is per invocation. It is independent of checkpointing and is not
used for resume. Enabling debug does not write checkpoints, and enabling resume
checkpointing does not write debug output.
Debug artifacts include framework-boundary inputs and outputs for source,
chunk, extract, merge, normalize, and output work, structured LLM request and
response data from Notarius contracts, validator requests and results, timing,
and retry attempt metadata. LLM calls made inside a retry or validator attempt
Debug artifacts include inputs and outputs for source, chunk, extract, merge,
normalize, and output work, structured LLM request and response data, validator
requests and results, timing, and retry attempt metadata. LLM calls made inside
a retry or validator attempt
write `prompt-000N.json`, `response-000N.json`, and
`response-content-000N.*` files under that attempt directory and are linked from
the attempt `llm_calls` array. Prompt content is written inline in the prompt
artifact. Response metadata is written to `response-000N.json`, while the
response body is written separately as pretty-printed JSON when possible or as
raw text otherwise. Debug artifacts may contain source material, reference
material, prompt inputs, model outputs, and other sensitive data. API keys are
not written, and obvious credential-shaped values and sensitive map keys are
redacted in framework envelopes, but debug directories should still be protected
as sensitive local state.
artifact. The response metadata and body use the paired files described above;
the body is pretty-printed JSON when possible and raw text otherwise. Debug
artifacts may contain source material, reference material, prompt inputs, model
outputs, and other sensitive data. API keys are not written, and obvious
credential-shaped values and sensitive map keys are redacted, but debug
directories should still be protected as sensitive local state.
## Retention
Diagnostics retention is configured with `workspace.diagnostics.retention`,
`NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION`, legacy `diagnostics.retention`,
legacy `NOTARIUS_DIAGNOSTICS_RETENTION`, or the default `auto`.
Diagnostics retention uses the effective mode selected through configuration;
see [Configuration](config.md#diagnostics) for the fields, environment
overrides, precedence, and default.
- `auto`: keep failed runs and successful runs with warnings; remove successful
warning-free runs.
- `always`: keep every diagnostics run directory.
- `never`: remove successful run directories; failed runs are still retained.
Unknown retention values are rejected during config validation.
## Failures
Failures before diagnostics directory creation, such as a missing config file or
@@ -170,24 +140,19 @@ an unusable diagnostics work directory, are printed to stderr and may not have a
diagnostics run directory.
Failures after diagnostics directory creation are printed to stderr and written
to `error.log`. Depending on where the failure occurred, diagnostics may also
include invocation metadata, redacted effective config, resolved pipeline data,
the run manifest, warnings, and a run report.
to `error.log`. Depending on where the failure occurred, the directory may also
contain artifacts written before the failure.
If durable output writing fails after the pipeline completes, diagnostics are
retained for inspection and may include `run-manifest.json`, `warnings.json`,
`run-report.json`, and `error.log`.
retained for inspection.
## Warnings
A successful run with warnings exits with code `0`, prints a warning count to
stderr, and writes warnings to durable output and diagnostics when retained.
The run manifest `validation_status` indicates whether raw outputs were
approved or rejected after validation.
Reference-related warnings include empty bound reference files. Empty references
are still passed to extractors so optional slots can be intentionally blank.
The [JSON output contract](integrations/json-output.md) defines durable warning
and validation-status fields.
## Cleanup
@@ -212,13 +177,10 @@ directories unless they are part of your own operational policy.
## Operational Limits
If `--resume` cannot reuse a checkpoint, Notarius executes that step and writes
a fresh checkpoint when checkpointing is enabled.
Provider retries and timeouts are handled by Scriptorium according to the
selected execution profile. Pipeline module retries are controlled by module
binding `retries` values in config for chunk, extract, merge, and normalize.
There is no separate CLI retry command.
selected execution profile. Pipeline module retry settings are defined in
[Configuration](config.md#module-bindings). There is no separate CLI retry
command.
Notarius writes local files only. Remote storage and archive management are not
part of the implemented CLI.

View File

@@ -32,21 +32,17 @@ Notarius is contract-first without being abstraction-heavy. Interfaces and
extension points should protect demonstrated boundaries. New abstraction is not
itself an architectural goal.
## Package Layout And Dependency Direction
## Layers And Dependency Direction
| Area | Ownership |
| --- | --- |
| `cmd/notarius` | Executable entry point; delegates to the CLI. |
| `internal/cli` | Application boundary, production composition, runtime setup, durable writes, and user-facing results. |
| `internal/core` | Generic deterministic models and policy for source material, configuration, manifests, diagnostics, and workspace identity. |
| `internal/framework` | Reusable contracts, registries, pipeline orchestration, validation mechanics, checkpoints, debug boundaries, and LLM runtime plumbing. |
| `internal/modules` | Concrete pipeline stage behavior. |
| `internal/validators` | Concrete approve/reject policies. |
The application boundary is the composition root and may depend on concrete
implementations. Domain-neutral model and framework layers provide reusable
policy, contracts, and orchestration. Concrete input, pipeline, output, and
validation extensions depend inward on those generic layers.
The CLI is the composition root and may import concrete implementations. Core
and framework packages cooperate as generic application layers; neither may
depend on production modules or validators. Concrete implementations may depend
on core models and framework contracts.
Generic layers must not depend on production extensions. Concrete extensions
must not compose the application or take ownership of process behavior. The
current packages implementing these layers are inventoried in
[Internal Overview](../internal/overview.md).
The following dependency boundaries are mandatory:

View File

@@ -1,130 +0,0 @@
# Documentation Policy Migration
The documentation ownership policy has been revised to assign each topic to one
canonical document. This roadmap organizes the likely migration work into three
ordered passes.
This is a planning inventory, not a review of the current documents. Each item
must be verified before content is moved, removed, or rewritten. Complete the
passes in order so canonical content is established before non-owning copies are
removed.
## Pass 1: User And Operator Contracts
Establish the canonical externally observable facts first. These documents will
become the targets linked from developer and internal documentation in the next
pass.
### Product And CLI
- Audit `README.md` so it owns product orientation and the single minimal
end-to-end quickstart without becoming a command or configuration reference.
- Audit `docs/cli.md` for material owned by the README, configuration reference,
or operations guide.
- Keep commands, arguments, flags, invocation semantics, and exit codes
canonical in `docs/cli.md`.
### Configuration And Examples
- Audit `docs/config.md` for complete example files, CLI syntax, runtime state
lifecycle, or implementation detail.
- Keep fields, defaults, precedence, environment overrides, validation, and
user-selectable module and validator keys canonical in `docs/config.md`.
- Audit minimal and production-oriented configuration examples, moving complete
copyable files under `examples/` and leaving only small illustrative fragments
in reference documentation.
- Inventory complete configuration, input, command, and output examples embedded
in prose documents.
- Select one maintained copy for each complete artifact under `examples/` where
practical, and verify important examples through automated tests.
### Operations And Integrations
- Audit `docs/operations.md` for duplicated CLI syntax, configuration field
definitions, logical output schemas, or implementation mechanics.
- Keep runtime workflows, physical state, retention, recovery, permissions, and
operational limits canonical in `docs/operations.md`.
- Audit `docs/integrations/` so external formats, protocols, logical output
paths, schemas, media types, and compatibility rules have one canonical home.
- Separate logical output bundle contracts from physical runtime placement and
lifecycle.
### Pass 1 Completion
- Confirm that README, CLI, configuration, operations, integrations, and
examples have non-overlapping ownership.
- Validate user-facing commands, fields, defaults, keys, schemas, paths, and
maintained examples against implemented behavior.
- Validate links among the user and operator documents.
## Pass 2: Developer And Internal Documentation
Use the canonical contracts established in Pass 1 to remove duplicated facts
from contributor and implementation documentation.
### Orientation And Architecture
- Audit `docs/development.md` so it routes contributors without maintaining a
parallel package inventory or architectural description.
- Audit `docs/internal/overview.md` so it owns the implemented component map
without restating normative architecture.
- Audit `docs/policy/architecture.md` so it contains current normative
architecture without implementation inventory, decision history, or future
behavior.
### Internal Components
- Audit internal component documents for repeated configuration definitions,
external input or output schemas, operator procedures, and global
architectural invariants.
- Replace duplicated external schemas or field definitions with links to the
canonical configuration or integration contracts from Pass 1.
- Keep implementation flow, internal collaborators, state transitions,
package-local guarantees, failures, and relevant tests in focused internal
documents.
- Identify task-specific contributor recipes that need a focused internal home
rather than the developer landing page.
- Retain production module and validator implementation details in module
internals while linking user-selectable keys to `docs/config.md`.
### Pass 2 Completion
- Confirm that development routes, architecture governs, internal overview
inventories, and focused internal docs explain implementation.
- Confirm that developer documents link to configuration, operations, and
integration contracts rather than redefining them.
- Validate developer-facing links and relevant focused tests.
## Pass 3: Lifecycle And Final Deduplication
Finish the migration by reconciling historical and future documentation, then
perform a repository-wide ownership audit.
### ADR And Roadmap Lifecycle
- Audit roadmap files for implemented behavior or completed status summaries
that should be removed or replaced with links to current canonical docs.
- Ensure ADRs own architectural rationale and supersession history without
becoming current behavior references or implementation trackers.
- Ensure accepted but unimplemented ADR decisions link to roadmap-owned
implementation status where appropriate.
- Distinguish rejected architectural alternatives in ADRs from rejected product
ideas in roadmap files.
### Repository-Wide Audit
- Inventory repeated commands, flags, defaults, module keys, validator keys,
file names, paths, schemas, retry semantics, and runtime guarantees.
- Assign each repeated fact to the canonical owner defined by the policy.
- Replace non-owning copies with short summaries and links where navigation is
useful.
- Retain only minimal illustrative snippets in prose and link them to maintained
examples.
- Validate local links and remove references to deleted or relocated material.
### Pass 3 Completion
- Confirm that every contractual or volatile fact has one canonical owner.
- Confirm that roadmap files contain future work and implementation status, ADRs
contain decision rationale, and current docs contain implemented behavior.
- Run final link, example, and documentation consistency checks.

View File

@@ -25,7 +25,7 @@ future work only.
- Production LLM-backed validators when there is a concrete review policy that
benefits from model judgment.
- Validator diagnostics and timing summaries if operators need more detail than
`manifest.json`, `rejected.json`, and `warnings.json` provide.
the current [durable output bundle](../integrations/json-output.md) provides.
- Media-type validators for non-JSON module outputs when such modules are
introduced.
- Validator compatibility metadata if real deployments need config-time
@@ -42,8 +42,7 @@ future work only.
- Optional generated example output fixtures with a regeneration procedure.
- Additional diagnostics or reporting views if operator workflows need them.
## Non-Goals To Revisit Deliberately
## Candidate Workspace Work
- A general workflow language.
- Structural module selection through ad hoc run flags.
- Storing secrets in config files, diagnostics, manifests, or examples.
Workspace storage, cleanup, archival, and reuse candidates are tracked in the
[Workspace Future Work](workspace.md) roadmap.

View File

@@ -1,25 +0,0 @@
# Workspace Implementation Status
The workspace implementation described by this roadmap has landed. Current
behavior is documented in the canonical current-behavior docs:
- [Configuration](../config.md)
- [CLI Reference](../cli.md)
- [Operations](../operations.md)
- [Diagnostics Internals](../internal/diagnostics.md)
- [Pipeline Internals](../internal/pipeline.md)
Implemented behavior includes:
- `workspace.directory` as the root for Notarius-owned local state;
- workspace diagnostics under `<workspace.directory>/diagnostics/<run-id>/`;
- compatibility for legacy `diagnostics.work_dir`, `diagnostics.retention`,
`NOTARIUS_WORK_DIR`, and `NOTARIUS_DIAGNOSTICS_RETENTION`;
- checkpoint writes under `<workspace.directory>/checkpoints/` when resume
checkpointing is enabled;
- explicit checkpoint reuse through `notarius run --resume`;
- debug artifacts under `<workspace.directory>/debug/<run-id>/` when debug
output is enabled;
- independent resume and debug settings.
Deferred workspace ideas remain in [Workspace Roadmap](workspace.md).

View File

@@ -1,26 +1,14 @@
# Workspace Roadmap Status
# Workspace Future Work
The local workspace feature has been implemented. Current behavior is documented
in [Configuration](../config.md), [CLI Reference](../cli.md),
[Operations](../operations.md), and the relevant internal docs.
Current workspace settings and operating behavior are documented in
[Configuration](../config.md#workspace) and
[Operations](../operations.md). This roadmap contains only candidate additions
to that behavior.
The implemented workspace provides one configurable root for Notarius-owned
local state:
## Candidate Work
```text
<workspace.directory>/
diagnostics/
checkpoints/
debug/
```
Implemented behavior includes workspace-backed diagnostics, checkpoint writing,
explicit checkpoint reuse through `notarius run --resume`, workspace debug
artifacts, safe workspace-relative writes, and compatibility for legacy
diagnostics configuration.
## Deferred Work
Default-idempotent `run` behavior with a force override, remote workspace
storage, workspace garbage collection, archival policy, and cross-machine resume
remain deferred.
- Default-idempotent run behavior with an explicit force override.
- Remote workspace storage.
- Workspace garbage collection.
- Workspace archival policy.
- Cross-machine checkpoint reuse.

View File

@@ -0,0 +1,25 @@
version: 2
concurrency:
total_llm: 1
workspace:
directory: /var/lib/notarius
diagnostics:
enabled: true
retention: auto
resume:
enabled: false
debug:
enabled: false
pipelines:
dnd-session:
input: seriatim
references:
party: ./dnd-spells-roster.txt
glossary: ./dnd-spells-glossary.txt
chunk:
module: generic
options:
max_units: 50
artifacts:
spells:
extract: dnd/spells

View File

@@ -1,29 +1,7 @@
version: 2
# For production runs, use a writable application-owned workspace such as:
#
# workspace:
# directory: /var/lib/notarius
# diagnostics:
# retention: auto
# resume:
# enabled: false
# debug:
# enabled: false
#
# For local development, use a project-local ignored path such as:
#
# workspace:
# directory: ./.notarius/workspace
pipelines:
dnd-session:
input: seriatim
references:
party: ./dnd-spells-roster.txt
glossary: ./dnd-spells-glossary.txt
chunk:
module: generic
options:
max_units: 50
artifacts:
spells:
extract: dnd/spells

View File

@@ -2647,23 +2647,29 @@ func TestRunPipelineDiagnosticsDirFlagOverridesWorkspaceDiagnosticsOnly(t *testi
}
func TestExampleFixtureConfigValidateAndPipelinesList(t *testing.T) {
configPath := fixturePath(t, "examples/dnd-spells.config.yml")
for _, path := range []string{
"examples/dnd-spells.config.yml",
"examples/dnd-spells-production.config.yml",
} {
path := path
t.Run("validate "+filepath.Base(path), func(t *testing.T) {
configPath := fixturePath(t, path)
var stdout bytes.Buffer
var stderr bytes.Buffer
t.Run("validate configured pipeline", func(t *testing.T) {
var stdout bytes.Buffer
var stderr bytes.Buffer
code := RunWithOptions([]string{"config", "validate", "--config", configPath, "--pipeline", "dnd-session"}, &stdout, &stderr, Options{})
code := RunWithOptions([]string{"config", "validate", "--config", configPath, "--pipeline", "dnd-session"}, &stdout, &stderr, Options{})
if code != 0 {
t.Fatalf("RunWithOptions() code = %d, stderr=%q", code, stderr.String())
}
if !strings.Contains(stdout.String(), "dnd-session") {
t.Fatalf("stdout = %q, want pipeline ID", stdout.String())
}
})
if code != 0 {
t.Fatalf("RunWithOptions() code = %d, stderr=%q", code, stderr.String())
}
if !strings.Contains(stdout.String(), "dnd-session") {
t.Fatalf("stdout = %q, want pipeline ID", stdout.String())
}
})
}
t.Run("list configured pipelines", func(t *testing.T) {
configPath := fixturePath(t, "examples/dnd-spells.config.yml")
var stdout bytes.Buffer
var stderr bytes.Buffer