Compare commits

7 Commits

23 changed files with 2044 additions and 2017 deletions

View File

@@ -1,2 +1,36 @@
# go-application-template
# Notarius
Notarius is a Go CLI for extracting structured artifacts from source material
with explicit, configurable pipeline modules.
The current implementation reads Seriatim transcript JSON, chunks the source
units, extracts D&D spell-cast artifacts with an OpenAI-compatible LLM, and
writes JSON output plus diagnostics for each run.
```sh
NOTARIUS_LLM_DEFAULT_BASE_URL=http://127.0.0.1:8080/v1 \
NOTARIUS_LLM_DEFAULT_MODEL=your-model \
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json
```
If the provider requires authentication, set
`NOTARIUS_LLM_DEFAULT_API_KEY` in the environment before running the command.
Outputs are written under `./notarius-output/<run-id>/` unless `--output-dir`
is provided.
Useful references:
- [CLI reference](docs/cli.md)
- [Configuration reference](docs/config.md)
- [Operations](docs/operations.md)
- [Troubleshooting](docs/troubleshooting.md)
- [Seriatim input contract](docs/integrations/seriatim.md)
- [OpenAI-compatible provider contract](docs/integrations/openai-compatible.md)
- [JSON output contract](docs/integrations/json-output.md)
- [D&D spell artifact contract](docs/integrations/dnd-spell-artifacts.md)
- [Developer workflow](docs/policy/development.md)
- [Internal architecture docs](docs/internal/overview.md)
- [Maintained example config](examples/dnd-spells.config.yml)
- [Maintained example input](examples/seriatim-minimal-transcript.json)

129
docs/cli.md Normal file
View File

@@ -0,0 +1,129 @@
# CLI Reference
This is the canonical reference for the implemented Notarius command-line
interface.
## Quick Run
```sh
NOTARIUS_LLM_DEFAULT_BASE_URL=http://127.0.0.1:8080/v1 \
NOTARIUS_LLM_DEFAULT_MODEL=your-model \
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json
```
Set `NOTARIUS_LLM_DEFAULT_API_KEY` if the OpenAI-compatible provider requires
a bearer token.
## Commands
```text
notarius help
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b]
notarius config validate --config path/to/config.yml [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list --config path/to/config.yml [--json]
```
Running `notarius` with no arguments, `notarius help`, `notarius --help`, or
`notarius -h` prints usage and exits successfully.
## `run`
`notarius run <pipeline-id>` executes a configured pipeline against one input
file.
Flags:
- `--input path`: required source input file.
- `--config path`: config file path. If omitted, Notarius checks
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are
comma-separated and must be non-empty.
- `--output-dir path`: output root. The run writes to `<path>/<run-id>/`.
Defaults to `./notarius-output`.
- `--diagnostics-dir path`: diagnostics work directory override for this
invocation.
- `--llm-profile id`: override every effective module binding to use one LLM
profile.
On success, the command prints the completed pipeline ID, approved and rejected
artifact counts, and the output directory. If the run completes with warnings,
the warning count is printed to stderr.
For durable output, diagnostics, retention, and failure inspection, see
[Operations](operations.md).
The current `run` command requires the resolved pipeline to use exactly one
distinct LLM profile after defaults and overrides are applied.
## `config validate`
`notarius config validate` loads and validates configuration.
Flags:
- `--config path`: config file path. If omitted, discovery uses
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
- `--pipeline pipeline-id`: additionally resolve one configured pipeline against
the production module catalog.
- `--only lane-a,lane-b`: validate resolution for selected artifact lanes. This
flag requires `--pipeline`.
Examples:
```sh
go run ./cmd/notarius config validate \
--config examples/dnd-spells.config.yml
go run ./cmd/notarius config validate \
--config examples/dnd-spells.config.yml \
--pipeline dnd-session \
--only spells
```
## `pipelines list`
`notarius pipelines list` prints configured pipeline IDs in sorted order.
Flags:
- `--config path`: config file path. If omitted, discovery uses
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
- `--json`: print `{"pipelines":[...]}` instead of one ID per line.
Examples:
```sh
go run ./cmd/notarius pipelines list \
--config examples/dnd-spells.config.yml
go run ./cmd/notarius pipelines list \
--config examples/dnd-spells.config.yml \
--json
```
## Exit Codes
- `0`: command succeeded.
- `1`: command syntax was valid, but loading config, resolving modules, running
the pipeline, calling the provider, writing output, or writing diagnostics
failed.
- `2`: command syntax was invalid, a command was unknown, a required argument
was missing, or a flag value was malformed.
## Implemented Production Pipeline Modules
The production CLI currently registers these module keys:
- input: `seriatim`
- chunk: `generic`
- extract: `dnd/spells`
- merge: `appendorder`
- normalize: `noop`
- output: `json`
The production CLI does not currently register validator modules.
For YAML structure, defaults, environment overrides, and module binding syntax,
see [Configuration](config.md).

213
docs/config.md Normal file
View File

@@ -0,0 +1,213 @@
# Configuration
This is the canonical reference for implemented Notarius configuration.
Notarius reads YAML config files with `version: 1`. File config is applied over
built-in defaults, then environment overrides are applied.
## Discovery
Commands that accept `--config` load configuration in this order:
1. the `--config` path, when provided;
2. `NOTARIUS_CONFIG`, when set to a non-empty path;
3. `/usr/local/etc/notarius/config.yml`.
If none is available, the command fails with a config file not found error.
## Minimal Example
```yaml
version: 1
llm_profiles:
default:
provider: openai-compatible
base_url: http://127.0.0.1:8080/v1
model: your-model
pipelines:
dnd-session:
input: seriatim
chunk:
module: generic
options:
max_units: 50
artifacts:
spells:
extract: dnd/spells
```
The maintained fixture is [examples/dnd-spells.config.yml](../examples/dnd-spells.config.yml).
## Top-Level Fields
- `version`: required. The only supported value is `1`.
- `llm_profiles`: optional map of LLM profile IDs to profile settings.
- `pipelines`: optional map of pipeline IDs to pipeline definitions.
- `concurrency`: optional global concurrency settings.
- `diagnostics`: optional diagnostics settings.
Unknown YAML fields are rejected.
## Defaults
Built-in defaults:
```yaml
llm_profiles:
default:
provider: openai-compatible
timeout: 600
max_retries: 3
max_concurrency: 1
concurrency:
total_llm: 1
diagnostics:
work_dir: /tmp/notarius
retention: auto
```
No pipelines are built in. A run requires a configured pipeline.
## LLM Profiles
Each `llm_profiles` entry may contain:
- `provider`: optional provider key. Empty means `openai-compatible`; any other
non-empty value must be `openai-compatible`.
- `base_url`: provider base URL. Required for actual LLM calls.
- `model`: provider model name. Required for actual LLM calls.
- `api_key_env`: environment variable name to read for the API key.
- `timeout`: request timeout as whole seconds or a Go-style duration string such
as `10m`.
- `max_retries`: retry count for provider calls. Must be zero or greater.
- `max_concurrency`: per-profile LLM concurrency. Must be zero or greater; when
zero, Notarius uses `concurrency.total_llm`.
Raw API keys are not accepted as file config fields. Use `api_key_env` or an
environment override.
## Environment Overrides
These environment variables are applied after the config file:
- `NOTARIUS_CONFIG`: config discovery path.
- `NOTARIUS_LLM_DEFAULT_API_KEY`: API key for the `default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_BASE_URL`: base URL for the `default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_MODEL`: model for the `default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_TIMEOUT_SECONDS`: integer timeout seconds for the
`default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_MAX_RETRIES`: integer retry count for the `default` LLM
profile.
- `NOTARIUS_LLM_DEFAULT_MAX_CONCURRENCY`: integer max concurrency for the
`default` LLM profile.
- `NOTARIUS_TOTAL_LLM_CONCURRENCY`: integer global LLM concurrency.
- `NOTARIUS_WORK_DIR`: diagnostics work directory.
- `NOTARIUS_DIAGNOSTICS_RETENTION`: diagnostics retention mode.
Integer environment values must parse as base-10 integers.
## Pipelines
A pipeline defines the fixed Notarius workflow:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
Pipeline fields:
- `input`: required module binding.
- `chunk`: optional module binding. Default module is `generic`.
- `artifacts`: required for pipeline resolution. It maps artifact lane IDs to
lane definitions.
- `output`: optional module binding. Default module is `json`.
Artifact lane fields:
- `extract`: required module binding.
- `merge`: optional module binding. Default module is `appendorder`.
- `normalize`: optional module binding. Default module is `noop`.
- `validators`: optional list of module bindings. The production CLI currently
does not register validator modules.
`notarius run` and `notarius config validate --pipeline` resolve the pipeline
against the production module catalog and fail fast for unknown or incompatible
module keys.
## Module Bindings
Every module binding may use shorthand:
```yaml
input: seriatim
```
or object form:
```yaml
chunk:
module: generic
llm_profile: default
options:
max_units: 50
```
Binding fields:
- `module`: module key.
- `llm_profile`: optional LLM profile ID. Empty means `default`.
- `options`: optional module-specific settings.
The `--llm-profile` run flag overrides every effective module binding to use
one configured profile.
## Implemented Production Modules
| Slot | Key | Notes |
| --- | --- | --- |
| input | `seriatim` | Reads Seriatim transcript JSON. |
| chunk | `generic` | Splits source units into ordered chunks. |
| extract | `dnd/spells` | Extracts `dnd.spell_cast` artifacts. |
| merge | `appendorder` | Keeps candidates in append order. |
| normalize | `noop` | Passes merged artifacts through unchanged. |
| output | `json` | Produces JSON output files. |
The `generic` chunker accepts:
- `max_units`: positive integer, default `50`;
- `overlap_units`: non-negative integer, default `0`, and must be less than
`max_units`.
## Diagnostics
`diagnostics` fields:
- `work_dir`: directory for per-run diagnostics. Default: `/tmp/notarius`.
- `retention`: `auto`, `always`, or `never`. Empty uses `auto`.
`auto` retains diagnostics for failed runs and successful runs with warnings.
`always` retains diagnostics for every run. `never` removes diagnostics for
successful runs without regard to warnings; failed runs are retained.
The `--diagnostics-dir` run flag overrides `diagnostics.work_dir` for that
invocation.
## Validation
Configuration validation checks:
- supported config version and known YAML fields;
- non-empty, non-duplicated IDs after trimming;
- supported LLM provider and non-negative profile limits;
- positive global LLM concurrency;
- supported diagnostics retention and non-empty work directory;
- module binding LLM profiles refer to configured profiles.
Pipeline resolution additionally checks:
- the pipeline ID exists;
- at least one artifact lane is declared and selected;
- selected lanes exist when `--only` is used;
- required module keys are present;
- module keys are registered for the expected slot;
- module capability requirements are satisfied.

View File

@@ -0,0 +1,152 @@
# D&D Spell-Cast Artifacts
This document is the durable artifact contract for approved
`dnd.spell_cast` artifacts produced by the implemented `dnd/spells` extractor.
## Artifact Identity
- Extractor key: `dnd/spells`
- Artifact type: `dnd.spell_cast`
- Schema version: `v1`
- Prompt ID: `dnd.spells`
- Response schema key: `dnd_spells`
- Response schema ID: `notarius.dnd.spells`
- Response schema name: `notarius_dnd_spells_v1`
The extractor requires source chunks and transcript source capability. It
returns generic artifact candidates that are serialized by the JSON output
module.
## Artifact Envelope
Approved artifacts use the generic artifact envelope documented in
[JSON Output](json-output.md#artifact-files):
```json
{
"extractor_key": "dnd/spells",
"artifact_type": "dnd.spell_cast",
"schema_version": "v1",
"payload": {
"caster": "Aria",
"spell": "Cure Wounds",
"effect": "heals an injured ally",
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds."
},
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": "seg-001",
"end_unit_id": "seg-001"
}
]
}
```
## Payload Fields
The `payload` object contains:
- `caster`: in-world character or creature casting the spell;
- `spell`: spell name;
- `effect`: concise spell effect in the scene;
- `narrative_description`: short description of the spell cast in context.
All payload fields are strings and must be non-empty after trimming.
`caster` is the in-world caster, not the transcript speaker.
## Source References
Source references live on the artifact envelope as `source_refs`; they are not
duplicated inside the `payload`.
Each source reference uses the generic source-reference shape:
- `source_id`
- `start_unit_id`
- `end_unit_id`
Validation requires:
- at least one source reference;
- non-empty source ID and unit IDs;
- source ID matching the source document ID;
- start and end unit IDs existing in the source document;
- start unit appearing before or at the same position as end unit.
## Structured LLM Response Shape
The extractor asks the LLM for this top-level response shape:
```json
{
"spell_casts": [
{
"caster": "Aria",
"spell": "Cure Wounds",
"effect": "heals an injured ally",
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds.",
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": "seg-001",
"end_unit_id": "seg-001"
}
]
}
]
}
```
`spell_casts` must be present. It may be empty when no spell casts are found.
The response schema asset is embedded at
`internal/modules/extract/dnd/spells/assets/schemas/dnd_spells.v1.json`.
## Validators
The extractor supplies two deterministic validators by default:
- `dnd/spells/shape`
- `dnd/spells/source_refs`
Rejection reason codes:
- `invalid_payload`: payload JSON cannot be decoded as a spell-cast payload.
- `missing_required_field`: `caster`, `spell`, `effect`, or
`narrative_description` is blank.
- `missing_source_ref`: candidate has no source references.
- `invalid_source_ref`: at least one source reference fails generic source
reference validation.
Rejected candidates are written to `rejected.json` by the JSON output module.
## Manifest Metadata
The extractor adds prompt and response-schema provenance under the artifact lane
manifest metadata:
```json
{
"metadata": {
"extractor": {
"prompt_id": "dnd.spells",
"prompt_version": "v1",
"prompt_sha256": "sha256:...",
"response_schema_key": "dnd_spells",
"response_schema_id": "notarius.dnd.spells",
"response_schema_name": "notarius_dnd_spells_v1",
"response_schema_version": "v1",
"response_schema_sha256": "sha256:..."
}
}
}
```
Raw prompt and schema content are not included in manifest metadata.
## Compatibility Limit
This contract covers only `dnd.spell_cast` artifacts produced by the
implemented spell-cast extractor.

View File

@@ -1,116 +0,0 @@
# D&D Spell Cast Extraction
This document describes the D&D spell-cast extractor currently implemented in
Notarius.
## Module
- Module key: `dnd/spells`
- Artifact type: `dnd.spell_cast`
- Schema version: `v1`
- Prompt ID: `dnd.spells`
- Response schema key: `dnd_spells`
- Response schema ID: `notarius.dnd.spells`
- Response schema name: `notarius_dnd_spells_v1`
The module is an extract module. It reads a generic source chunk, renders the
`dnd.spells` prompt, calls the configured structured LLM client, and returns
spell-cast artifact candidates.
## Source Expectations
The extractor expects a generic `SourceDocument` and active source chunk. It
does not depend on concrete Seriatim package types.
Pipeline resolution must provide these capabilities before the extractor runs:
- `chunks`
- `source.transcript`
Source units may include transcript metadata such as speaker and timestamps.
That metadata is optional prompt context. It is not part of the durable spell
payload.
## Artifact Payload
Each approved artifact payload is a JSON object with these fields:
- `caster`: in-world character or creature casting the spell;
- `spell`: spell name;
- `effect`: concise spell effect in the scene;
- `narrative_description`: short description of the spell cast in context.
`caster` is not the table speaker. NPCs, monsters, and other DM-voiced
characters can be casters.
## Source References
The structured LLM response must include `source_refs` for each spell cast.
Each source reference uses the generic source-reference shape:
- `source_id`
- `start_unit_id`
- `end_unit_id`
The extractor copies those references into the generic artifact envelope
`source_refs` field. The durable `dnd.spell_cast` payload does not duplicate
source references.
Source-reference IDs must match the source document and source-unit IDs
exactly. The validator chain rejects unknown source IDs, unknown unit IDs, and
reversed unit ranges.
## Validators
The extractor provides these deterministic validators by default, in order:
- `dnd/spells/shape`
- `dnd/spells/source_refs`
`dnd/spells/shape` rejects:
- malformed JSON payloads with reason code `invalid_payload`;
- blank `caster`, `spell`, `effect`, or `narrative_description` fields with
reason code `missing_required_field`.
`dnd/spells/source_refs` rejects:
- candidates with no source references using reason code `missing_source_ref`;
- invalid source references using reason code `invalid_source_ref`.
The source-reference validator uses the core `source.ValidateRef` behavior, so
its rejection message includes the underlying source-reference validation
error.
## Capabilities
The module declares these required capabilities:
- `chunks`
- `source.transcript`
The module declares this provided capability:
- `dnd.spell_casts`
A pipeline artifact lane can reference the extractor with:
```yaml
artifacts:
spells:
extract: dnd/spells
merge: appendorder
normalize: noop
```
## Limits
The current implementation covers only D&D spell-cast extraction. It does not
yet implement:
- item extraction;
- NPC extraction;
- combat extraction;
- encounter extraction;
- broad D&D rules validation;
- a CLI `run` workflow.

View File

@@ -0,0 +1,192 @@
# JSON Output
This document is the durable JSON output file-format contract produced by the
implemented `json` output module and written by the CLI.
## Output Directory
The CLI writes logical output files under:
```text
<output-root>/<run-id>/
```
The default output root is `./notarius-output`. Operational behavior is covered
in [Operations](../operations.md).
## Files
The `json` output module writes:
- `index.json`
- `manifest.json`
- `artifacts/<artifact-type>.json`, one file per approved artifact type
- `rejected.json`
- `warnings.json`
Files are pretty-printed JSON with a trailing newline.
## `index.json`
Shape:
```json
{
"manifest_file": "manifest.json",
"artifact_files": [
{
"artifact_type": "dnd.spell_cast",
"file": "artifacts/dnd.spell_cast.json"
}
],
"rejected_file": "rejected.json",
"warnings_file": "warnings.json"
}
```
`artifact_files` is sorted by artifact type. It is empty when no artifacts are
approved.
## `manifest.json`
`manifest.json` contains a run manifest:
```json
{
"run_id": "run-123",
"pipeline_id": "dnd-session",
"pipeline_digest": "sha256:...",
"input_module": "seriatim",
"chunker": "generic",
"source_digests": ["sha256:..."],
"extractors": ["dnd/spells"],
"merger": "appendorder",
"normalizer": "noop",
"output_encoder": "json",
"artifact_lanes": [
{
"id": "spells",
"extractor": "dnd/spells",
"merger": "appendorder",
"normalizer": "noop"
}
],
"llm_profiles": [
{
"id": "default",
"provider": "openai-compatible",
"model": "configured-model"
}
],
"validation_status": "approved",
"started_at": "2026-01-01T00:00:00Z",
"completed_at": "2026-01-01T00:00:01Z"
}
```
Fields with empty values may be omitted by JSON encoding.
`validation_status` is `approved` when no candidates were rejected and
`rejected` when one or more candidates were rejected.
## Artifact Files
Each artifact file has this shape:
```json
{
"artifact_type": "dnd.spell_cast",
"artifacts": [
{
"extractor_key": "dnd/spells",
"artifact_type": "dnd.spell_cast",
"schema_version": "v1",
"payload": {},
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": "seg-001",
"end_unit_id": "seg-001"
}
]
}
]
}
```
Artifact envelope fields:
- `extractor_key`: extractor module key.
- `artifact_type`: artifact type.
- `schema_version`: artifact schema version.
- `payload`: artifact-type-specific JSON payload.
- `source_refs`: optional generic source references.
- `metadata`: optional artifact metadata.
Artifact file names are produced by sanitizing the artifact type:
- characters outside `A-Z`, `a-z`, `0-9`, `.`, `_`, and `-` become `_`;
- repeated `..` sequences are replaced;
- leading and trailing `.`, `_`, and `-` are trimmed;
- empty sanitized names are rejected.
For current D&D spell-cast artifacts, the file is
`artifacts/dnd.spell_cast.json`.
## `rejected.json`
Shape:
```json
{
"rejected": [
{
"candidate": {
"index": 0,
"extractor_key": "dnd/spells",
"artifact_type": "dnd.spell_cast",
"schema_version": "v1",
"payload": {},
"source_refs": []
},
"validator_name": "dnd/spells/source_refs",
"reason_code": "missing_source_ref",
"message": "spell cast candidate must include at least one source ref"
}
]
}
```
`rejected` is an empty array when no candidates are rejected.
## `warnings.json`
Shape:
```json
{
"warnings": [
{
"scope": "output",
"reason_code": "example_warning",
"message": "warning message"
}
]
}
```
`warnings` is an empty array when no warnings are reported.
## Path Safety
The output module returns slash-separated logical paths. The CLI also validates
logical output names before writing:
- names must be non-empty;
- names must be relative;
- names must be clean;
- names must use `/`, not `\`;
- names must not contain `..`;
- resolved paths must stay under the run output directory.
Durable writes are atomic per file.

View File

@@ -0,0 +1,128 @@
# OpenAI-Compatible Structured Output
This document describes the external LLM provider contract implemented by the
production Notarius LLM client.
## Provider
- Provider key: `openai-compatible`
- HTTP method: `POST`
- Endpoint: `<base_url>/chat/completions`
- Request body: JSON
- Response mode: chat completions with structured JSON schema output
`base_url` is trimmed of trailing slashes before `/chat/completions` is
appended. Configure provider settings in [Configuration](../config.md).
## Request
The client sends a JSON object with:
```json
{
"model": "configured-model",
"messages": [
{
"role": "system",
"content": "..."
},
{
"role": "user",
"content": "..."
}
],
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "schema_name",
"strict": true,
"schema": {}
}
}
}
```
Implemented request behavior:
- `model` comes from the structured completion request when set, otherwise from
the configured LLM profile.
- `messages` must be non-empty; each role and content must be non-empty after
trimming.
- `response_format.type` is always `json_schema`.
- `response_format.json_schema.strict` is always `true`.
- `response_format.json_schema.name` and `schema` come from the extractor or
validator making the call.
If an API key is configured, the client sends:
```text
Authorization: Bearer <api-key>
```
The client always sends `Content-Type: application/json`.
## Response
The client expects a JSON response with at least one choice:
```json
{
"model": "provider-model",
"choices": [
{
"message": {
"content": "{\"field\":\"value\"}"
}
}
],
"usage": {
"prompt_tokens": 10,
"completion_tokens": 5,
"total_tokens": 15
}
}
```
`choices[0].message.content` may be either:
- a JSON string whose contents are valid JSON; or
- raw JSON.
The decoded content is unmarshaled into the caller-provided structured output
target. If `usage` is present, prompt, completion, and total token counts are
copied into the completion response.
## Errors And Retries
The client validates base URL, model, response schema name, response schema
JSON, messages, and output target before or during the call.
Retryable failures:
- HTTP request failure;
- response body read failure;
- HTTP `429`;
- HTTP `5xx`;
- malformed provider response envelope;
- missing choices;
- missing, empty, or invalid assistant JSON content;
- structured-output decode failure.
Non-retryable provider status codes include non-`429` `4xx` responses.
Provider error bodies are parsed for `error.message` or `message` when present.
Configured API key values and bearer-token values are redacted from returned
provider errors.
## Timeouts And Concurrency
The configured profile timeout is applied per provider request when greater
than zero. Context cancellation is respected.
The production CLI wraps the provider client with the LLM scheduler. Effective
concurrency is described in [LLM runtime internals](../internal/llm.md).
## Limits
This contract documents only the fields the implemented client sends and reads.
Provider-specific extensions are ignored unless they affect those fields.

View File

@@ -1,42 +1,50 @@
# Seriatim Minimal Transcript JSON
# Seriatim Transcript JSON
This document describes the Seriatim input format currently accepted by the
`seriatim` input adapter.
This document is the external input contract for the implemented `seriatim`
input adapter.
## Adapter
- Module key: `seriatim`
- Document kind: `transcript`
- Unit kind: `transcript_segment`
- Source format: `application/vnd.seriatim.minimal+json`
- Source format: `application/vnd.seriatim+json`
The adapter parses raw Seriatim JSON into a generic `SourceDocument`. It does
not add transcript-specific fields to core source or runner contracts.
The adapter parses raw Seriatim JSON into a generic source document. It owns
transcript-specific JSON parsing and metadata mapping; core source and pipeline
code stay source-format agnostic.
## Accepted Shape
The input must be a JSON object with top-level `metadata` and `segments` fields:
The input must be one JSON object with top-level `metadata` and `segments`
fields. This covers the maintained minimal fixture and Seriatim intermediate
output that provides the same required segment fields.
```json
{
"metadata": {
"id": "session-alpha",
"title": "Synthetic session transcript"
"title": "Synthetic D&D spell session"
},
"segments": [
{
"id": "seg-001",
"start": 0,
"end": 4.5,
"speaker": "Narrator",
"text": "The stone door opens."
"end": 4,
"speaker": "Aria",
"text": "Aria raises her holy symbol and casts Cure Wounds."
}
]
}
```
Extra compatible fields are ignored. Multiple top-level JSON values are
rejected.
The maintained example is
[examples/seriatim-minimal-transcript.json](../../examples/seriatim-minimal-transcript.json).
Top-level metadata entries are preserved. Other segment fields, such as
`categories`, are ignored.
Multiple top-level JSON values are rejected.
## Validation
@@ -44,8 +52,12 @@ The adapter rejects:
- empty raw input;
- malformed JSON;
- top-level JSON that is not an object;
- missing, null, or non-object `metadata`;
- missing, null, non-array, or empty `segments`;
- segment values that are not objects;
- segment `id` values that are neither strings nor numbers;
- non-string `speaker` or `text`;
- empty segment IDs;
- segment IDs with leading or trailing whitespace;
- duplicate segment IDs;
@@ -55,19 +67,16 @@ The adapter rejects:
- `end` values before `start`;
- missing or empty `text`.
Segment text may keep leading or trailing whitespace, but it must not be empty
after trimming.
Segment text is preserved as provided, but it must not be empty after trimming.
## Source Mapping
The adapter maps Seriatim input into the source model as follows:
The adapter maps input to `SourceDocument`:
- top-level `metadata` becomes `SourceDocument.Metadata`;
- `SourceDocument.Digest` is `sha256:<hex>` of the exact raw input bytes;
- `segment.id` becomes `SourceUnit.ID`;
- `segment.text` becomes `SourceUnit.Text`;
- each source unit has kind `transcript_segment`;
- segment `speaker`, `start`, and `end` are stored in source-unit metadata.
- `metadata` becomes `SourceDocument.Metadata`;
- `SourceDocument.Kind` is `transcript`;
- `SourceDocument.Format` is `application/vnd.seriatim+json`;
- `SourceDocument.Digest` is `sha256:<hex>` of the exact raw input bytes.
`SourceDocument.ID` is selected in this order:
@@ -76,6 +85,14 @@ The adapter maps Seriatim input into the source model as follows:
3. `metadata.source_id`, when it is a non-empty string after trimming;
4. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
Each segment becomes one `SourceUnit`:
- `segment.id` becomes `SourceUnit.ID`; numeric IDs are converted to their JSON
number text, so `1` becomes `"1"`;
- `segment.text` becomes `SourceUnit.Text`;
- `SourceUnit.Kind` is `transcript_segment`;
- `speaker`, `start`, and `end` are stored in source-unit metadata.
## Metadata Keys
Seriatim unit metadata uses these keys:
@@ -84,18 +101,20 @@ Seriatim unit metadata uses these keys:
- `start`: `json.Number` start value;
- `end`: `json.Number` end value.
The `internal/modules/input/seriatim` package provides typed accessors for
these metadata values.
The `internal/modules/input/seriatim` package exposes typed accessors for these
values.
## Capabilities
The module declares these provided capabilities for pipeline validation:
The module declares these provided capabilities:
- `source.transcript`
- `transcript.speaker`
- `transcript.timestamps`
## Limits
## Compatibility Limit
Only the Seriatim minimal transcript shape described here is supported. Broader
Seriatim schema variants are not currently accepted as a compatibility contract.
This contract covers only Seriatim transcript JSON with the top-level
`metadata` object and `segments` array described here. Broader Seriatim output
schemas are compatible only when they provide these required fields with the
accepted types.

View File

@@ -0,0 +1,88 @@
# Diagnostics Internals
Diagnostics internals live in `internal/core/diagnostics`. Operator-facing run
behavior is documented in [Operations](../operations.md).
## Purpose
Diagnostics provide local inspection artifacts for a run without becoming the
durable output contract. Durable user output is produced by output modules and
written by the CLI.
Diagnostics must not expose secrets.
## Run Directory
`NewRunDirectory(workDir, retention)` creates:
```text
<workDir>/run-<unix-nanoseconds>/
```
If `workDir` is empty, it defaults to `/tmp/notarius`. Empty retention defaults
to `auto`.
The writer makes the work directory if needed, then attempts to create a unique
run directory. It retries run ID creation a bounded number of times if a
collision occurs.
## Artifact Writers
Implemented artifact names:
- `invocation.json`
- `effective-config.json`
- `resolved-pipeline.json`
- `source-document.json`
- `run-manifest.json`
- `run-report.json`
- `warnings.json`
- `error.log`
JSON artifacts are encoded with indentation and a trailing newline. Writes are
atomic through a temporary file in the target directory followed by rename.
Artifact names must be single relative file names. Absolute paths, path
separators, and names resolving outside the run directory are rejected.
## Redacted Effective Config
Diagnostics writers accept payloads that implement
`RedactedDiagnosticsPayload`. `internal/core/config` uses this to redact API
keys in effective config diagnostics while preserving resolved pipeline context.
The redaction path clones config data before replacing secret values.
## Retention
Retention is decided by `ShouldRetainRunDirectory`.
- Failed runs are always retained.
- `always` retains successful runs.
- `never` removes successful runs.
- `auto` retains successful runs only when warnings exist.
- Unknown retention values are treated as retain by the retention decision, but
config validation rejects unsupported values before normal runs.
`ApplyRetention` removes only the specific run directory.
## CLI Failure Behavior
The CLI creates the diagnostics run directory after config loading and before
pipeline resolution. Failures before that point do not have diagnostics.
After diagnostics creation, run failures call `WriteErrorLog` and apply
retention with `RunSucceeded: false`, so the run directory remains available.
When the pipeline returns a partial manifest on failure, the CLI writes that
manifest before logging the failure.
## Invariants
- Diagnostics paths must be narrow and run-directory scoped.
- Writes should be atomic where practical.
- Secrets must be redacted.
- Diagnostics write failures are command failures because they can hide the
information needed for recovery.
- Durable output file contracts belong to output modules and integration docs,
not to diagnostics.

116
docs/internal/llm.md Normal file
View File

@@ -0,0 +1,116 @@
# LLM Runtime
The implemented LLM runtime lives in `internal/framework/llm`. It provides
transport-neutral structured completion contracts, an OpenAI-compatible HTTP
adapter, concurrency scheduling, schema registry helpers, retry behavior, and
secret redaction.
## Contract
Modules depend on `contracts.StructuredLLMClient`:
```go
CompleteStructured(ctx, request, out) (response, error)
```
The request contains messages, optional model override, response schema name,
and response schema JSON. The caller supplies a pointer target for decoded
structured output.
Extractors own prompts and schemas. Provider adapters should not contain
domain-specific prompt logic.
## Production Client Construction
`internal/cli` builds the production LLM client from the effective config:
1. find the effective LLM profile;
2. build `OpenAICompatibleClientConfig`;
3. create an OpenAI-compatible client;
4. create a scheduler from profile or global concurrency;
5. wrap the client with `NewScheduledClient`;
6. return non-secret LLM profile manifest metadata.
The current run command requires exactly one distinct effective LLM profile for
the resolved pipeline.
## OpenAI-Compatible Adapter
`OpenAICompatibleClient` posts JSON to:
```text
<base_url>/chat/completions
```
It sends:
- `model`
- `messages`
- `response_format.type = "json_schema"`
- `response_format.json_schema.name`
- `response_format.json_schema.strict = true`
- `response_format.json_schema.schema`
If an API key is configured, the adapter sends an `Authorization: Bearer ...`
header.
The adapter accepts assistant content either as a JSON string containing JSON or
as raw JSON content. It then unmarshals that content into the caller-provided
target.
External wire-contract details belong in the
[OpenAI-compatible integration doc](../integrations/openai-compatible.md).
## Retries And Timeouts
The adapter retries:
- provider request failures;
- response read failures;
- HTTP `429`;
- HTTP `5xx`;
- malformed provider envelopes;
- malformed assistant JSON;
- structured-output decode failures.
Non-retryable `4xx` responses are returned without retry. Request timeout comes
from the effective LLM profile. Context cancellation is respected.
## Scheduler
`Scheduler` bounds concurrent provider calls. It tracks in-flight calls and a
FIFO queue of waiters. Cancellation removes queued waiters or releases granted
permits.
`NewScheduledClient` wraps any structured LLM client and runs each completion
inside the scheduler.
Effective concurrency is:
1. `llm_profiles.<id>.max_concurrency`, when greater than zero;
2. `concurrency.total_llm`, when greater than zero;
3. `1`.
## Schema Registry
The framework schema registry embeds generic test schemas. It also exposes
helpers for caller-owned schemas:
- `LoadResponseSchema`
- `LookupResponseSchema`
- `MustLookupResponseSchema`
- `ResponseSchema.DiagnosticsMap`
`DiagnosticsMap` omits raw schema content and includes metadata such as key,
ID, version, name, and SHA-256.
The D&D spell extractor owns and loads its own embedded response schema.
## Secret Redaction
Provider errors are passed through `ErrorWithSecretsRedacted` with the API key
and bearer-token value. Config diagnostics use redacted effective config
payloads.
Do not add raw provider request bodies, response bodies, API keys, or prompt
payloads to diagnostics by default.

165
docs/internal/modules.md Normal file
View File

@@ -0,0 +1,165 @@
# Modules
Production modules live under `internal/modules`. Each module implements one
contract from `internal/framework/contracts`, exposes a `ModuleSpec`, and
registers itself with the matching pipeline registry.
The CLI production catalog currently registers only the modules listed here.
## Contract Pattern
A production module package should provide:
- a stable module key;
- a constructor such as `New`;
- the relevant contract implementation;
- `ModuleSpec`;
- `Register`;
- focused tests for registration, options, contract behavior, and errors.
Module specs should describe capabilities accurately. Resolution uses specs to
reject incompatible pipelines before execution.
## `seriatim` Input
Package: `internal/modules/input/seriatim`
The `seriatim` adapter parses Seriatim transcript JSON into a generic source
document. It owns transcript JSON details, source ID selection, source digest
creation, transcript segment validation, and segment metadata mapping.
Provides:
- `source.transcript`
- `transcript.speaker`
- `transcript.timestamps`
External JSON shape belongs in the Seriatim integration doc.
## `generic` Chunker
Package: `internal/modules/chunk/generic`
The `generic` chunker splits source units into ordered chunks. It validates the
source document, clones source units, assigns chunk IDs such as `chunk-000001`,
and records chunk metadata for start unit, end unit, and unit count.
Options:
- `max_units`: positive integer, default `50`;
- `overlap_units`: non-negative integer, default `0`, and less than
`max_units`.
Provides:
- `chunks`
## `dnd/spells` Extractor
Package: `internal/modules/extract/dnd/spells`
The `dnd/spells` extractor owns D&D spell-cast artifact semantics. It renders
embedded prompts, loads the embedded structured response schema, calls the
structured LLM client, converts spell-cast responses into artifact candidates,
and supplies deterministic validators.
Requires:
- `chunks`
- `source.transcript`
Provides:
- `dnd.spell_casts`
Artifact type and schema version:
- artifact type: `dnd.spell_cast`
- schema version: `v1`
The extractor adds prompt and response-schema provenance to lane manifest
metadata. Durable artifact payload details belong in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
## D&D Spell Validators
The spell extractor returns two built-in validators:
- `dnd/spells/shape`: rejects malformed payloads and missing required fields.
- `dnd/spells/source_refs`: rejects candidates without valid source references.
Reason codes include:
- `invalid_payload`
- `missing_required_field`
- `missing_source_ref`
- `invalid_source_ref`
These validators are supplied by the extractor when no validators are configured
for the lane.
## `appendorder` Merger
Package: `internal/modules/merge/appendorder`
The `appendorder` merger clones and appends candidates in chunk order. It does
not deduplicate or reconcile candidates.
Provides:
- `merged`
## `noop` Normalizer
Package: `internal/modules/normalize/noop`
The `noop` normalizer clones merged candidates and returns them unchanged.
Requires:
- `merged`
Provides:
- `normalized`
## `json` Output
Package: `internal/modules/output/json`
The `json` output encoder converts approved artifacts, rejected artifacts,
warnings, and the run manifest into logical JSON output files. It groups
approved artifacts by artifact type and sanitizes artifact-type file names.
Requires:
- `normalized`
Provides:
- `encoded`
Durable output file shapes belong in the
[JSON output contract](../integrations/json-output.md). Operator behavior
belongs in [Operations](../operations.md).
## Production Registration
Production registration is centralized in `internal/cli/catalog.go`.
Do not make framework code import production modules. The CLI wires production
modules at the application boundary; tests may provide fake registries or fake
catalogs directly.
## Adding A Module
When adding a module, keep source-format and extraction-domain boundaries clear:
- input modules may know external source formats;
- extract modules may know artifact semantics and prompt/schema assets;
- merge and normalize modules own candidate combination and reconciliation;
- output modules own serialization, not diagnostics or CLI reporting.
Update [Development](../policy/development.md), [Configuration](../config.md),
internal docs, integration docs, and examples when the new module becomes
implemented production behavior.

86
docs/internal/overview.md Normal file
View File

@@ -0,0 +1,86 @@
# Internal Overview
This directory documents implemented Notarius internals for developers and LLM
coding agents. It complements [Architecture](../policy/architecture.md), which
is the durable policy for boundaries and invariants.
## Executable And CLI
`cmd/notarius` calls the CLI package. `internal/cli` owns:
- command parsing and usage;
- config discovery and loading;
- production module catalog and registry wiring;
- production LLM client construction;
- run directory creation;
- durable output writes;
- user-facing stdout, stderr, and exit codes.
The CLI should stay thin around framework contracts. Domain extraction behavior
belongs in modules, not in command handlers.
## Core Packages
- `internal/core/artifacts`: artifact candidates, approved artifacts, rejected
artifacts, validation decisions, and run manifests.
- `internal/core/config`: defaults, YAML config parsing, environment overrides,
validation, redaction, and resolved pipeline config.
- `internal/core/diagnostics`: per-run diagnostics directory creation,
diagnostics artifact writers, atomic writes, and retention decisions.
- `internal/core/source`: source documents, source units, source references, and
validation.
Core packages should remain deterministic and concrete. They should not import
production modules.
## Framework Packages
- `internal/framework/contracts`: interfaces and request/result structs for
input adapters, chunkers, extractors, mergers, normalizers, validators, output
encoders, and structured LLM clients.
- `internal/framework/pipeline`: module registries, module specs, profile
resolution, capability checks, run orchestration, warnings, validation, and
manifest population.
- `internal/framework/llm`: OpenAI-compatible structured-output client,
scheduler, schema registry, retries, and secret redaction.
- `internal/framework/prompt`: embedded prompt registry and template rendering.
- `internal/framework/validate`: validator decision helpers and cardinality
enforcement.
Framework code should stay source-agnostic and domain-agnostic.
## Module Packages
Production module packages live under `internal/modules`:
- `input/seriatim`
- `chunk/generic`
- `extract/dnd/spells`
- `merge/appendorder`
- `normalize/noop`
- `output/json`
Each module package owns its contract implementation, module spec,
registration, options, focused tests, and module-specific errors.
## Fixtures And Tests
The repository uses focused package tests plus a fixture-driven CLI workflow.
- CLI acceptance tests cover maintained examples under `examples/`.
- Pipeline tests cover registry composition and end-to-end framework behavior
with fakes.
- Module tests cover implemented module contracts without requiring real
provider calls.
- LLM tests use local test servers and fakes.
Do not use real external services in tests. Use fakes, fixtures, or local test
servers.
## Boundary Reminders
- Source-format details stay in input modules and integration docs.
- Extraction-domain details stay in extract modules and artifact docs.
- Provider wire details stay in the LLM runtime and provider integration docs.
- Durable output contracts belong in integration docs.
- Operator procedures belong in `docs/operations.md`, not internal docs.

127
docs/internal/pipeline.md Normal file
View File

@@ -0,0 +1,127 @@
# Pipeline Internals
The implemented pipeline runner lives in `internal/framework/pipeline`. It
executes the fixed workflow defined by the architecture policy:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
Pipeline execution is serial. The runner executes the resolved lanes one after
another in the fixed workflow order.
## Profile Resolution
Config loading produces `pipeline.PipelineProfile` values. Resolution happens
before execution:
1. `internal/core/config.Config.Resolve` validates config and finds the named
pipeline.
2. The optional lane selection is passed to `pipeline.ResolvePipeline`.
3. Module bindings are defaulted:
- chunk: `generic`
- merge: `appendorder`
- normalize: `noop`
- output: `json`
- LLM profile: `default`
4. The module catalog is checked for each bound module key.
5. Module capabilities are checked in workflow order.
6. A digest is calculated from the resolved pipeline without the digest field.
The CLI writes the resolved pipeline and digest to diagnostics.
## Registries And Module Specs
`pipeline.Registries` holds concrete constructors for execution. A
`pipeline.ModuleCatalog` exposes module specs for config validation and
resolution.
Every production module registers a `ModuleSpec` with:
- `Key`: module key used in config;
- `Stage`: module kind such as input, chunk, extract, merge, normalize,
validate, or output;
- `Provides`: capabilities added after that module runs;
- `Requires`: capabilities that must already be available.
Capability checks prevent incompatible pipeline composition before a run starts.
## Runner Input And Output
`pipeline.RunInput` carries:
- a `ResolvedPipeline`;
- optional source ID, input path, and raw input bytes;
- a structured LLM client;
- run ID, start time, LLM profile manifest metadata, and CLI metadata.
`pipeline.RunOutput` carries:
- run manifest;
- approved artifacts;
- rejected artifacts;
- warnings;
- logical output files returned by the output encoder.
The CLI owns durable file writes and diagnostics writes after the runner returns.
## Execution
The runner:
1. validates run input and registries;
2. builds the input adapter and parses the raw input into a source document;
3. validates the source document;
4. builds the chunker and produces source chunks;
5. runs each selected artifact lane in sorted resolved order;
6. builds the output encoder and validates logical output file names.
Within an artifact lane, the runner:
1. builds the extractor, merger, and normalizer;
2. records module manifest metadata when modules provide it;
3. extracts candidates from each chunk;
4. normalizes candidate envelope fields such as index, extractor key, artifact
type, and schema version;
5. merges candidates;
6. normalizes merged candidates;
7. validates candidate envelope consistency;
8. runs validators;
9. converts approved candidates to artifacts.
## Validators
If a lane declares validators in config, the runner builds those validators from
the validator registry. Otherwise it uses validators returned by the extractor.
Each validator must return exactly one decision for each eligible candidate. The
runner enforces decision cardinality with `internal/framework/validate`.
Rejected candidates are removed before the next validator runs. Approved
candidates continue through the chain.
The production CLI currently registers no standalone validator modules. The
current D&D spell extractor supplies deterministic shape and source-reference
validators.
## Warnings And Failures
Warnings from chunking, extraction, merging, normalization, validation, and
output encoding are accumulated in `RunOutput.Warnings`.
Errors wrap the operation and module key or lane context. If execution fails
after a manifest exists, the returned manifest is marked `failed` and receives a
completion timestamp.
On successful execution, the manifest validation status is:
- `approved` when no candidates were rejected;
- `rejected` when at least one candidate was rejected.
## Manifest Population
The manifest records run ID, pipeline ID, pipeline digest, module keys, artifact
lanes, LLM profile metadata, source digest, validation status, and timing.
Modules can add non-secret manifest metadata by implementing
`contracts.ManifestMetadataProvider`. The D&D spell extractor uses this for
prompt and response-schema provenance.

132
docs/operations.md Normal file
View File

@@ -0,0 +1,132 @@
# Operations
This is the canonical reference for operating implemented Notarius runs.
## Normal Run
A run reads one source file, resolves one configured pipeline, calls the
configured OpenAI-compatible LLM profile, writes durable JSON output, and writes
diagnostics for inspection.
```sh
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json \
--output-dir ./notarius-output \
--diagnostics-dir /tmp/notarius
```
The command prints a success line with the pipeline ID, approved and rejected
artifact counts, and the output path.
## Output Directory
Durable output is written to:
```text
<output-root>/<run-id>/
```
The default output root is `./notarius-output`. Use `--output-dir` to choose a
different root.
The `json` output module writes these files:
- `index.json`: file index with paths to the manifest, artifact files,
rejected artifacts, and warnings.
- `manifest.json`: run manifest with resolved pipeline provenance, module keys,
validation status, and timing.
- `artifacts/<artifact-type>.json`: approved artifacts grouped by artifact
type. For the current D&D spell extractor, this includes
`artifacts/dnd.spell_cast.json` when spell-cast artifacts are approved.
- `rejected.json`: rejected candidates and validator decisions.
- `warnings.json`: warnings reported by pipeline modules or the output encoder.
Output writes are atomic per file. Logical output file names must be clean,
relative, slash-separated paths and must not contain `..`.
## Diagnostics Directory
Diagnostics are written under:
```text
<diagnostics-work-dir>/<run-id>/
```
The default diagnostics work directory is `/tmp/notarius`. It can be set with
`diagnostics.work_dir`, `NOTARIUS_WORK_DIR`, or `--diagnostics-dir`.
Implemented diagnostics artifacts:
- `invocation.json`: command metadata such as operation, config path, input
path, selected lanes, run ID, and pipeline digest when available.
- `effective-config.json`: resolved config with API keys redacted.
- `resolved-pipeline.json`: resolved module bindings and pipeline digest.
- `run-manifest.json`: the same run manifest written to durable output when it
is available.
- `warnings.json`: warning list.
- `run-report.json`: counts, status, output path, diagnostics path, and run ID.
- `error.log`: failure message, written after diagnostics directory creation
when a run fails.
`source-document.json` is supported by the diagnostics writer but is not written
by the current CLI run workflow.
## Retention
Diagnostics retention is configured with `diagnostics.retention`,
`NOTARIUS_DIAGNOSTICS_RETENTION`, or the default `auto`.
- `auto`: keep failed runs and successful runs with warnings; remove successful
warning-free runs.
- `always`: keep every diagnostics run directory.
- `never`: remove successful run directories; failed runs are still retained.
Unknown retention values are rejected during config validation.
## Failures
Failures before diagnostics directory creation, such as a missing config file or
an unusable diagnostics work directory, are printed to stderr and may not have a
diagnostics run directory.
Failures after diagnostics directory creation are printed to stderr and written
to `error.log`. Depending on where the failure occurred, diagnostics may also
include invocation metadata, redacted effective config, resolved pipeline data,
the run manifest, warnings, and a run report.
If durable output writing fails after the pipeline completes, diagnostics are
retained for inspection and may include `run-manifest.json`, `warnings.json`,
`run-report.json`, and `error.log`.
## Warnings
A successful run with warnings exits with code `0`, prints a warning count to
stderr, and writes warnings to durable output and diagnostics when retained.
The run manifest `validation_status` indicates whether final artifacts were
approved or rejected after validation.
## Cleanup
It is safe to remove specific old run directories after their output and
diagnostics are no longer needed:
```sh
rm -rf /tmp/notarius/run-1234567890
rm -rf ./notarius-output/run-1234567890
```
Use exact run-directory paths. Avoid broad cleanup commands against parent
directories unless they are part of your own operational policy.
## Operational Limits
There is no command to resume a failed run. Re-run `notarius run` after fixing
the cause.
Provider retries are limited to the OpenAI-compatible client retry behavior
configured by the effective LLM profile. There is no separate CLI retry command.
Notarius writes local files only. Remote storage and archive management are not
part of the implemented CLI.

139
docs/policy/development.md Normal file
View File

@@ -0,0 +1,139 @@
# Development
This document defines contributor workflow for Notarius. For architectural
invariants and package boundaries, read [Architecture](architecture.md) first.
## Required Reading
Before changing the repository, review:
- [Architecture](architecture.md)
- [Documentation Policy](documentation.md)
Keep current-behavior documentation limited to implemented behavior. Put planned
or deferred behavior under `docs/roadmap/`.
## Repository Layout
- `cmd/notarius`: executable entry point.
- `internal/cli`: CLI parsing, production catalog wiring, config loading, run
command orchestration, output writes, and user-facing errors.
- `internal/core`: deterministic models and policy for artifacts, source
documents, config, and diagnostics.
- `internal/framework`: reusable contracts, pipeline orchestration, prompt
helpers, validation helpers, and LLM runtime plumbing.
- `internal/modules`: concrete input, chunk, extract, merge, normalize, and
output modules.
- `docs`: policy, user/operator docs, internal docs, integration docs, and
roadmap files.
- `examples`: maintained, secret-free examples covered by tests where practical.
## Validation Commands
Run focused tests for the area changed, then run the broader checks when the
change affects shared contracts, CLI behavior, or documentation examples.
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
```
Useful focused checks:
```sh
go test ./internal/cli
go test ./internal/core/config
go test ./internal/framework/pipeline
go test ./internal/framework/llm
go test ./internal/modules/input/seriatim
go test ./internal/modules/extract/dnd/spells
go test ./internal/modules/output/json
```
## Go Conventions
- Prefer the standard library unless a dependency is justified by correctness,
security, interoperability, or substantial complexity reduction.
- Keep package names short, lowercase, and idiomatic.
- Preserve import direction: framework and core code must not depend on concrete
production modules.
- Use `context.Context` for long-running operations and external calls.
- Return contextual errors that name the operation and relevant module, path, or
resource.
- Do not include secrets in errors, logs, diagnostics, manifests, or docs.
## Adding Config Fields
Config behavior is centralized under `internal/core/config`.
When adding a file config field:
1. Update file config structs and YAML parsing in `file_config.go`.
2. Apply the field over defaults in config application code.
3. Add validation in `validation.go` when the field has constraints.
4. Add environment override support in `env.go` only for operational overrides.
5. Update redaction if the field can contain secrets.
6. Add focused config tests.
7. Update [Configuration](../config.md) and maintained examples when behavior
changes.
Pipeline composition should remain config-driven. Do not add command flags that
silently replace structural pipeline definitions.
## Adding CLI Flags Or Commands
CLI behavior lives in `internal/cli`.
When adding CLI surface:
1. Keep syntax explicit and update usage text.
2. Validate arguments before running expensive work.
3. Convert internal errors into concise user-facing messages.
4. Add CLI tests for success, syntax errors, and failure modes.
5. Update [CLI Reference](../cli.md), and update
[Operations](../operations.md) or [Troubleshooting](../troubleshooting.md)
if run behavior changes.
## Adding Modules Or Adapters
Concrete modules live under `internal/modules/<kind>/...` and implement the
interfaces in `internal/framework/contracts`.
For a new production module:
1. Implement the relevant contract.
2. Expose a `ModuleSpec` with the correct module key, module kind, provided
capabilities, and required capabilities.
3. Expose a `Register` function that registers the module with its registry.
4. Add focused module tests for contract behavior, registration, options,
validation, and errors.
5. Register the module in `internal/cli/catalog.go` only when it is production
ready.
6. Update internal docs and user-facing docs only for implemented behavior.
Source-format behavior belongs in input modules and integration docs.
Extraction-domain behavior belongs in extract modules and artifact docs.
## Updating Examples
Examples must be valid, secret-free, and small.
- Prefer environment-based secret configuration.
- Keep `examples/dnd-spells.config.yml` loadable by CLI tests.
- Keep `examples/seriatim-minimal-transcript.json` compatible with the Seriatim
adapter.
- Do not add expected-output fixtures unless they are validated or have a clear
regeneration procedure.
## Documentation Updates
Update docs in the same change when behavior changes.
- CLI syntax: `docs/cli.md`
- Config fields and defaults: `docs/config.md`
- Output, diagnostics, retention, or recovery: `docs/operations.md`
- Common user-facing failures: `docs/troubleshooting.md`
- Internal architecture and contracts: `docs/internal/`
- External file formats and durable integration contracts: `docs/integrations/`
- Future or planned work only: `docs/roadmap/`

View File

@@ -1,199 +0,0 @@
# Documentation Roadmap
## Status
This document captures planned documentation decisions for Notarius. It records
policy choices while the application architecture is still being shaped. It does
not describe implemented behavior.
## Documentation Goals
Notarius documentation should make three boundaries obvious:
- source-format support belongs to input-stage modules;
- extraction-domain behavior belongs to extract-stage modules;
- core framework behavior is source-agnostic and domain-agnostic.
Documentation should avoid making the MVP look more transcript-specific or
D&D-specific than the architecture intends.
## Current Policy Decisions
### Planned Work Stays In Roadmap Docs
Until code exists, planned behavior belongs under `docs/roadmap/`.
Implemented behavior should later move into canonical docs. Roadmap files may
then link to those docs or be reduced to remaining future work.
### Core Docs Should Use Generic Terms
Core architecture docs should prefer:
- source document;
- source unit;
- source reference;
- input adapter;
- extractor;
- chunker;
- merger;
- normalizer;
- output encoder;
- artifact;
- validator;
- run manifest.
Core docs should avoid transcript-specific terms such as segment, speaker,
timestamp, and transcript range unless discussing an input adapter or an example.
Core docs should avoid D&D-specific terms such as spell, NPC, item, combat, and
encounter unless discussing extract modules, artifact docs, or examples.
### Input Module Docs Own Source Formats
Each implemented input-stage module should have a canonical integration
document.
Likely future files:
```text
docs/integrations/seriatim-transcript.md
docs/integrations/markdown-source.md
```
Input module docs should cover:
- accepted external schema or file shape;
- mapping into `SourceDocument` and `SourceUnit`;
- metadata preserved by the module;
- validation rules and failure behavior;
- examples.
The Seriatim input module doc should reference the Seriatim schema it supports and
explain how transcript segment IDs become source-unit IDs.
### Stage Module Docs Own Business Logic
Each implemented stage-module family should have canonical internal or
integration docs.
Likely future files:
```text
docs/internal/stage-modules.md
docs/integrations/artifacts-dnd.md
```
Stage module docs should cover:
- module key;
- stage;
- artifact type;
- schema version;
- required source-reference behavior;
- validator chain;
- prompt and response-schema ownership;
- examples.
D&D concepts should be documented in D&D extract-module or artifact docs, not in
generic runner or framework docs.
### CLI Docs Should Reflect Extensibility
The CLI reference should present named pipeline profiles as the primary
user-facing abstraction. Individual stage modules should be visible through
pipeline configuration and discovery commands, not through ad hoc structural
run flags.
Provisional command shape:
```sh
notarius run dnd-session --input ./source.json
notarius run dnd-session --input ./source.json --only spells,npcs
notarius config validate
notarius pipelines list
```
Once implemented, `docs/cli.md` should document:
- pipeline ID selection;
- required input path flags;
- `--only` artifact-lane selection;
- config path behavior;
- operational overrides such as output path, model, concurrency, and diagnostics
directory;
- output path behavior;
- diagnostics and report behavior;
- exit codes.
### Config Docs Should Separate Framework And Plugin-Like Options
`docs/config.md` should describe named pipeline profiles and the resolved
pipeline model.
It should cover:
- config file locations and precedence;
- `llm_profiles`;
- `pipelines.<pipeline_id>.input`;
- `pipelines.<pipeline_id>.chunk`;
- `pipelines.<pipeline_id>.artifacts.<lane>.extract`;
- lane `merge`, `normalize`, and validator settings;
- output module selection;
- string shorthand versus inline module-binding object form;
- defaults for omitted slots;
- capability validation;
- pipeline digest and manifest provenance.
Module-specific config should stay inline with the pipeline slot that owns it.
Top-level named module instances should not be introduced until repeated inline
settings create real drift. `llm_profiles` are the cross-cutting exception.
### Examples Should Stay Real
Examples should be added only when the matching behavior exists and should be
covered by tests where practical.
Likely future examples:
```text
examples/seriatim-minimal-transcript.json
examples/minimal-config.yml
examples/dnd-spells.artifacts.json
examples/dnd-session.config.yml
```
Examples should be secret-free and should use the same command shapes documented
in `docs/cli.md`.
## Canonical Documentation Targets
When the first vertical slice is implemented, add or update:
- `README.md`: concise purpose, shortest useful command, links.
- `docs/cli.md`: implemented command behavior.
- `docs/config.md`: implemented config behavior.
- `docs/operations.md`: diagnostics, retention, failure inspection.
- `docs/troubleshooting.md`: common failures.
- `docs/internal/overview.md`: implemented package map.
- `docs/internal/pipeline.md`: implemented extraction flow.
- `docs/internal/stage-modules.md`: stage contracts and implemented modules.
- `docs/internal/input-modules.md`: input adapter contract and implemented input modules.
- `docs/internal/validators.md`: validator contract and built-ins.
- `docs/integrations/seriatim-transcript.md`: Seriatim input contract.
- `docs/integrations/artifacts.md`: output artifact envelope.
## Review Checklist For Future Documentation Changes
Before merging docs, check:
- Does the document describe implemented behavior outside `docs/roadmap/`?
- Are source-format details isolated to input module or integration docs?
- Are D&D details isolated to extract module or artifact docs?
- Is there one canonical home for the topic?
- Do command examples match implemented CLI syntax?
- Do config examples use named pipeline profiles rather than ad hoc module
flags?
- Are examples valid, maintained, and free of secrets?
- Did any architecture, config, CLI, stage module, validator, or artifact
contract change require a docs update?

View File

@@ -1,888 +0,0 @@
# Implementation Plan: MVP
## Status
This is the staged implementation plan for the active MVP roadmap:
[`mvp.md`](mvp.md).
The target audience is an LLM coding agent. Implement the stages in order.
Each stage should leave the repository compiling and tested. Do not skip ahead
to later stages unless the current stage's done criteria are satisfied.
## Policy Context
Follow:
- [`../policy/architecture.md`](../policy/architecture.md)
- [`../policy/documentation.md`](../policy/documentation.md)
- [`mvp.md`](mvp.md)
- [`initial-architecture.md`](initial-architecture.md)
Required boundaries:
- framework packages must remain source-agnostic and domain-agnostic;
- source-format behavior belongs in input modules;
- D&D spell behavior, prompt assets, response schema assets, and stable
prompt/schema identifiers belong in `internal/modules/extract/dnd/spells`;
- stage business logic belongs under `internal/modules/<stage>/...` unless it
is genuinely tiny shared framework plumbing;
- structural pipeline selection must remain config-driven;
- `--only` may select artifact lanes but must not alter pipeline structure;
- output-stage warnings are out-of-band from artifact payloads and must be
available to CLI/diagnostics;
- keep planned documentation in `docs/roadmap/` until MVP behavior exists.
## Global Implementation Decisions
- Add no new third-party dependencies.
- Keep YAML config version `1` unless a user-visible config syntax change is
unavoidable. New module options can use existing binding `options`.
- Use the existing six-stage workflow:
`input -> chunk -> extract -> merge -> normalize -> output`.
- Use production CLI wiring in `internal/cli` for the MVP instead of adding a
new app package. The CLI may compose modules, but it must not own module
business logic.
- Keep `notarius run` serial over chunks for the MVP. The contracts and LLM
scheduler should still permit later parallel execution.
- Use one effective LLM profile per MVP run. The current runner accepts one
`StructuredLLMClient`, so a selected pipeline with multiple distinct effective
LLM profile IDs should fail clearly until multi-client runtime support is
intentionally added.
- Use the existing OpenAI-compatible client for real runs.
- Add a scheduled LLM client wrapper so every structured completion passes
through the configured scheduler.
- Use the existing `seriatim` input module and `dnd/spells` extractor module.
- Implement production default modules with these keys:
- `generic` chunker;
- `appendorder` merger;
- `noop` normalizer;
- `json` output encoder.
- Put production default modules under:
- `internal/modules/chunk/generic`;
- `internal/modules/merge/appendorder`;
- `internal/modules/normalize/noop`;
- `internal/modules/output/json`.
- The output encoder should return logical output files; the CLI/application
layer should write those files to disk. Encoders should not own filesystem
side effects.
- Use an output directory per run. The MVP default output root should be
`./notarius-output`, overrideable by `--output-dir`.
- File writes for durable output should be atomic where practical: write to a
temporary file in the target directory, then rename.
- Use synthetic fixtures only. Do not add private campaign transcript content,
real API keys, or private infrastructure values.
## Stage 1: Move D&D Prompt And Schema Assets Into The Spells Module
### Goal
Restore the intended framework/domain boundary before building additional MVP
functionality.
`internal/framework/llm` and `internal/framework/prompt` should provide generic
asset loading, metadata, and rendering primitives. They must not define
D&D-specific prompt IDs, response schema keys, asset paths, or tests.
### Files To Update Or Move
Expected files:
- `internal/framework/llm/schema_registry.go`
- `internal/framework/llm/schema_registry_test.go`
- `internal/framework/llm/assets/schemas/dnd_spells.v1.json`
- `internal/framework/prompt/registry.go`
- `internal/framework/prompt/render.go`
- `internal/framework/prompt/render_test.go`
- `internal/framework/prompt/assets/dnd/spells/system.md`
- `internal/framework/prompt/assets/dnd/spells/user.md`
- `internal/modules/extract/dnd/spells/extractor.go`
- `internal/modules/extract/dnd/spells/prompt.go`
- `internal/modules/extract/dnd/spells/prompt_test.go`
- `internal/modules/extract/dnd/spells/schema_test.go`
- `docs/integrations/dnd-spells.md`, only if prompt/schema ownership text needs
to be corrected.
### Required Design
Refactor `internal/framework/llm` so it can load schemas from caller-owned
embedded files.
Add or expose a generic constructor similar to:
```go
func LoadResponseSchema(fsys fs.FS, def ResponseSchemaDefinition) (ResponseSchema, error)
```
where `ResponseSchemaDefinition` carries:
- key;
- ID;
- version;
- name;
- asset path.
The existing framework registry may keep test schemas, but it must not include
`DNDSpellsSchemaKey` or `dnd_spells.v1.json`.
Refactor `internal/framework/prompt` so it can compile/render prompt pairs from
caller-owned embedded files.
Add or expose a generic constructor/rendering type similar to:
```go
type Bundle struct { ... }
func LoadBundle(fsys fs.FS, def Definition) (*Bundle, error)
func (b *Bundle) RenderUserSystem(data any) (system string, user string, metadata Metadata, err error)
```
The framework prompt package may continue to own shared hardening text if that
is useful, but it must not include `DNDSpellsPromptID` or D&D prompt paths.
Move D&D prompt and schema assets under `internal/modules/extract/dnd/spells`.
Recommended paths:
```text
internal/modules/extract/dnd/spells/assets/prompts/system.md
internal/modules/extract/dnd/spells/assets/prompts/user.md
internal/modules/extract/dnd/spells/assets/schemas/dnd_spells.v1.json
```
The spells package should define its own stable identifiers:
```go
const PromptID = "dnd.spells"
const ResponseSchemaKey = "dnd_spells"
const ResponseSchemaID = "notarius.dnd.spells"
const ResponseSchemaName = "notarius_dnd_spells_v1"
```
The spells extractor must call module-owned prompt/schema helpers and pass only
generic framework values into the LLM client.
### Required Tests
- Framework LLM schema tests prove test schemas still load, sort, clone, and
omit raw schema content from diagnostics.
- Framework LLM schema tests prove looking up `dnd_spells` in the framework
registry fails.
- Framework prompt tests prove test prompts still render and missing template
data still errors.
- Framework prompt tests contain no D&D prompt assertions.
- Spells package schema tests load the module-owned D&D schema and verify:
- key;
- ID;
- version;
- response schema name;
- valid JSON;
- clone/mutation safety;
- diagnostics omit raw schema content.
- Spells package prompt tests render the module-owned prompt and verify
hardening text and prompt metadata.
- Existing spells extractor tests still pass without importing framework-owned
D&D constants.
### Validation
Run:
```sh
gofmt -w internal/framework/llm internal/framework/prompt internal/modules/extract/dnd/spells
go test ./internal/framework/llm ./internal/framework/prompt ./internal/modules/extract/dnd/spells
go test ./...
```
## Stage 2: Add MVP Manifest And Logical Output File Contracts
### Goal
Make output and manifest contracts capable of representing the MVP's durable
run output before implementing the production JSON encoder or CLI writing.
### Files To Update
Expected files:
- `internal/framework/contracts/contracts.go`
- `internal/core/artifacts/*.go`
- `internal/framework/pipeline/runner.go`
- `internal/framework/pipeline/runner_test.go`
- `internal/framework/contracts/contracts_test.go`
- `docs/policy/architecture.md`, only if the implemented output contract
requires clarifying policy text.
### Required Design
Extend the output contract to support logical files:
```go
type OutputFile struct {
Name string `json:"name"`
ContentType string `json:"content_type,omitempty"`
Bytes []byte `json:"-"`
}
type OutputResult struct {
Files []OutputFile `json:"files,omitempty"`
Warnings []Warning `json:"warnings,omitempty"`
}
```
Remove or stop using the single `OutputResult.Bytes` / `ContentType` path once
all tests are updated. If keeping those fields temporarily reduces churn, mark
them as legacy in comments and make the runner prefer `Files`.
Add matching fields to `pipeline.RunOutput`:
```go
OutputFiles []contracts.OutputFile `json:"-"`
```
The runner should collect output-stage warnings exactly as it does now, after
calling the output encoder.
Define safe logical file names:
- names are slash-separated relative paths;
- names must not be empty, absolute, contain `..`, or contain `\`;
- names are validated before the runner returns them;
- file names are sorted deterministically by the encoder that creates them.
Extend manifest data enough for MVP provenance:
- add `RunManifest.LLMProfiles []LLMProfileManifest`;
- add `ArtifactLaneManifest.Metadata map[string]any`;
- add `RunManifest.StartedAt`, `CompletedAt`, and `RunID` population support
in the runner input/output path.
Recommended structs:
```go
type LLMProfileManifest struct {
ID string `json:"id"`
Provider string `json:"provider,omitempty"`
Model string `json:"model,omitempty"`
}
```
Add an optional metadata interface for modules:
```go
type ManifestMetadataProvider interface {
ManifestMetadata() map[string]any
}
```
When a stage module implements the interface, the runner should include that
metadata in the appropriate manifest area. For the MVP, the D&D spells extractor
will use this to report prompt and response schema identifiers/hashes on its
artifact lane.
Update `pipeline.RunInput` to accept:
- `RunID string`;
- `StartedAt time.Time`;
- `LLMProfiles []artifacts.LLMProfileManifest`.
The runner should set `CompletedAt` when the run finishes or fails after a
manifest has been initialized.
### Required Tests
- `contracts.OutputFile` JSON shape omits bytes and includes name/content type.
- Runner rejects unsafe output file names returned by an encoder.
- Runner preserves output warnings out-of-band.
- Runner output contains logical files returned by the encoder.
- Manifest includes run ID, started/completed timestamps when supplied or
generated.
- Manifest includes LLM profile metadata supplied in `RunInput`.
- Manifest includes extractor-provided lane metadata when the extractor
implements `ManifestMetadataProvider`.
### Validation
Run:
```sh
gofmt -w internal/framework/contracts internal/core/artifacts internal/framework/pipeline
go test ./internal/framework/contracts ./internal/core/artifacts ./internal/framework/pipeline
go test ./...
```
## Stage 3: Implement Production Default Stage Modules
### Goal
Make pipeline defaults real production modules instead of test-only fakes or
framework-only helpers.
### Files To Add Or Update
Expected packages:
- `internal/modules/chunk/generic`
- `internal/modules/merge/appendorder`
- `internal/modules/normalize/noop`
- `internal/modules/output/json`
Expected framework cleanup:
- `internal/framework/pipeline/generic_stages.go`
- pipeline tests that currently instantiate framework `AppendOrderMerger` or
`NoopNormalizer`.
### Required Design
#### `generic` chunker
Package: `internal/modules/chunk/generic`
Key: `generic`
Module spec:
- stage: `chunk`;
- requires: `source.transcript` is **not** required;
- provides: `chunks`.
Behavior:
- accepts any valid `SourceDocument`;
- preserves source-unit order;
- returns stable chunk IDs: `chunk-000001`, `chunk-000002`, and so on;
- copies source units defensively;
- adds chunk metadata:
- `start_unit_id`;
- `end_unit_id`;
- `unit_count`.
Options:
- `max_units`: positive integer, default `50`;
- `overlap_units`: non-negative integer, default `0`, must be less than
`max_units`.
If the source has no units, return a clear error. If options have the wrong type
or invalid values, return a clear module-specific error.
#### `appendorder` merger
Package: `internal/modules/merge/appendorder`
Key: `appendorder`
Module spec:
- stage: `merge`;
- requires: no artifact-type-specific capability;
- provides: `merged`.
Behavior:
- preserves chunk order as provided by the runner;
- preserves candidate order within each chunk;
- defensively copies candidates, payloads, source refs, and metadata;
- does not merge, deduplicate, or rewrite source references.
#### `noop` normalizer
Package: `internal/modules/normalize/noop`
Key: `noop`
Module spec:
- stage: `normalize`;
- requires: `merged`;
- provides: `normalized`.
Behavior:
- defensively copies candidates;
- does not deduplicate, rewrite, or validate domain content.
#### `json` output encoder
Package: `internal/modules/output/json`
Key: `json`
Module spec:
- stage: `output`;
- requires: `normalized`;
- provides: `encoded`.
Behavior:
- returns logical output files:
- `index.json`;
- `manifest.json`;
- `artifacts/<artifact_type>.json` for each approved artifact type;
- `rejected.json`;
- `warnings.json`.
- groups approved artifacts by `Artifact.ArtifactType`;
- sorts artifact-type file names by artifact type;
- preserves artifact order within each artifact type according to runner order;
- pretty-prints JSON with two-space indentation and trailing newline;
- uses content type `application/json`;
- includes rejected artifacts and warnings even when the arrays are empty;
- does not include output warnings inside artifact payloads.
File-name safety:
- artifact type may contain dots and hyphens;
- replace any character outside `[A-Za-z0-9._-]` with `_` for artifact file
names;
- if sanitization produces an empty name, return an error.
### Required Tests
- Generic chunker tests cover defaults, exact chunk boundaries, overlap,
invalid options, empty source, defensive copies, and stable IDs.
- Append-order merge tests cover ordering and defensive copies.
- Noop normalizer tests cover pass-through behavior and defensive copies.
- JSON output tests cover all logical files, grouping, sorted filenames,
rejected/warnings presence, pretty JSON, unsafe artifact type sanitization,
and no mutation of inputs.
- Pipeline config tests using defaults resolve when these module specs are
registered.
### Validation
Run:
```sh
gofmt -w internal/modules/chunk/generic internal/modules/merge/appendorder internal/modules/normalize/noop internal/modules/output/json internal/framework/pipeline
go test ./internal/modules/chunk/generic ./internal/modules/merge/appendorder ./internal/modules/normalize/noop ./internal/modules/output/json
go test ./internal/framework/pipeline
go test ./...
```
## Stage 4: Add Production CLI Catalog And Runtime Wiring
### Goal
Make implemented modules selectable by real CLI commands without test-injected
catalogs.
### Files To Add Or Update
Expected files:
- `internal/cli/run.go`
- new `internal/cli/catalog.go` or equivalent;
- `internal/cli/run_test.go`;
- module registry tests as needed.
### Required Design
Add production wiring in `internal/cli`:
```go
func productionRegistries() (pipeline.Registries, error)
func productionCatalog() (pipeline.ModuleCatalog, error)
```
The production wiring must register:
- input: `seriatim`;
- chunk: `generic`;
- extract: `dnd/spells`;
- merge: `appendorder`;
- normalize: `noop`;
- output: `json`.
Keep all business logic in module packages. `internal/cli` should only compose
registries/catalogs and command behavior.
Update `cli.Options` so tests may inject registries/catalog/runtime without
disabling production defaults unintentionally.
Recommended option fields:
```go
type Options struct {
Catalog pipeline.ModuleCatalog
Registries pipeline.Registries
LLMClientFactory LLMClientFactory
LookupEnv func(string) (string, bool)
Now func() time.Time
}
```
If `Catalog` or `Registries` is empty in normal `Run`, use production wiring.
If tests provide either, use the provided value.
Define `LLMClientFactory` in `internal/cli` or a small local file:
```go
type LLMClientFactory func(ctx context.Context, cfg config.Config, profileID string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error)
```
The production factory should:
- read the selected LLM profile from effective config;
- construct `llm.OpenAICompatibleClient`;
- construct `llm.Scheduler` using the most specific configured concurrency:
profile `max_concurrency` if set, otherwise global `concurrency.total_llm`,
otherwise `1`;
- wrap the client in a scheduled client so every completion acquires/releases a
scheduler permit;
- return manifest-safe LLM metadata with profile ID, provider, and model.
Add a scheduled client wrapper in `internal/framework/llm` if it does not
already exist:
```go
func NewScheduledClient(client contracts.StructuredLLMClient, scheduler *Scheduler) contracts.StructuredLLMClient
```
### Required CLI Behavior
- `notarius config validate --config <file> --pipeline <id>` uses the
production catalog by default.
- `notarius pipelines list --config <file>` still lists configured pipeline
IDs and validates config shape.
- `notarius pipelines list --config <file> --json` remains stable.
Do not implement `notarius run` in this stage.
### Required Tests
- Production catalog includes the six MVP modules and their module specs.
- `config validate --pipeline` succeeds for a real MVP config fixture using no
injected catalog.
- Unknown module keys still fail with stage/pipeline context.
- Production LLM client factory rejects missing/invalid LLM profiles with clear
errors.
- Scheduled client wrapper enforces scheduler use and propagates errors.
- Existing CLI tests using injected catalogs still pass.
### Validation
Run:
```sh
gofmt -w internal/cli internal/framework/llm
go test ./internal/cli ./internal/framework/llm
go test ./...
go vet ./...
go build ./cmd/notarius
```
## Stage 5: Implement `notarius run` Without Durable File Writing
### Goal
Add the user-facing run command and prove it can drive the configured pipeline
with injected fake runtime pieces. This stage should return/run data in memory
or through test buffers, but durable file writing may be completed in Stage 6.
### Files To Update
Expected files:
- `internal/cli/run.go`
- `internal/cli/run_test.go`
- `cmd/notarius/main.go`, only if command wiring requires it.
### Required Command Shape
Support:
```sh
notarius run <pipeline-id> --input path/to/source.json
notarius run <pipeline-id> --input path/to/source.json --only spells
```
Supported flags:
- `--config path`;
- `--input path`, required;
- `--only lane-a,lane-b`;
- `--output-dir path`, parsed and passed through metadata for Stage 6;
- `--diagnostics-dir path`, overrides config diagnostics work dir for this run;
- `--llm-profile profile-id`, operational override for MVP runs.
Do not add flags for structural module selection, such as `--extractor`,
`--chunker`, `--merge`, or `--output`.
### Required Behavior
- Missing pipeline ID returns exit code `2`.
- Missing `--input` returns exit code `2`.
- Unknown flags return exit code `2`.
- Config/load/resolve/runtime failures return exit code `1`.
- Successful runs return exit code `0`.
- `--only` uses existing lane selection behavior.
- Extend `config.ResolveInput` with `LLMProfileOverride string` or an
equivalent option. When `--llm-profile` is provided, apply it to every
resolved module binding before the resolved pipeline digest is computed. This
keeps the override operational rather than structural while still making the
effective pipeline digest truthful.
- After resolution, collect the distinct effective LLM profile IDs used by the
selected pipeline. For the MVP, require exactly one distinct profile ID and
fail clearly if more than one is present.
- Input file is read as raw bytes and passed to the runner with path metadata.
- Source ID defaults to the input path or basename only if the input adapter
needs one; do not invent transcript-specific source IDs in the CLI.
- The command resolves the selected pipeline with the production catalog.
- The command constructs the LLM client through `LLMClientFactory`.
- The command invokes `pipeline.New(registries).Run(...)`.
- The command prints a concise success message that includes at least:
- pipeline ID;
- approved artifact count;
- rejected artifact count.
- If warnings exist, print a concise warning count to stderr.
### Required Tests
Use fake LLM/runtime injection; do not call external services.
- Missing pipeline ID.
- Missing input flag.
- Unknown pipeline.
- Unknown `--only` lane.
- Invalid input file path.
- Successful run invokes runner path through real registries and fake LLM.
- `--only spells` runs only the selected lane.
- LLM factory failure is reported clearly.
- Validation rejection produces a failed/non-zero or successful-with-rejections
behavior according to current runner semantics. For MVP, keep runner
semantics: a run with rejected artifacts completes successfully with
`ValidationStatus` set to `rejected`, unless an error occurs.
### Validation
Run:
```sh
gofmt -w internal/cli
go test ./internal/cli
go test ./...
go vet ./...
go build ./cmd/notarius
```
## Stage 6: Write Durable Output And Diagnostics For `notarius run`
### Goal
Complete the MVP run workflow by writing output files and diagnostics.
### Files To Update
Expected files:
- `internal/cli/run.go`
- `internal/cli/run_test.go`
- `internal/core/diagnostics/*.go`, only if helper methods are needed.
### Required Design
Output directory behavior:
- default root: `./notarius-output`;
- override: `--output-dir`;
- each run writes to `<output-root>/<run-id>/`;
- run ID comes from diagnostics run directory when available or from a
generated UTC nanosecond timestamp using the same style as diagnostics;
- create directories with `0755`;
- write files with `0644`;
- write each file atomically where practical.
Logical output files from `pipeline.RunOutput.OutputFiles` should be written
under the run output directory. Reject unsafe logical file names before writing:
- empty;
- absolute;
- contains `..`;
- contains backslash;
- escapes the run output directory after path cleaning.
Diagnostics behavior:
- create a diagnostics run directory at command start unless retention is
`never` and the implementation can still reliably capture failures; simplest
MVP behavior is to create it and then apply retention at the end;
- write invocation metadata;
- write redacted effective config;
- write resolved pipeline;
- write run manifest;
- write warnings;
- write run report containing output path, counts, and validation status;
- write error log on failure;
- apply retention with existing diagnostics policy.
`--diagnostics-dir` should override `Config.Diagnostics.WorkDir` after file and
environment config have been applied, without changing structural pipeline
definition or pipeline digest.
Success output:
- stdout includes the durable output run directory path;
- stderr includes warning count when warnings are present;
- no raw prompt text, raw API keys, or large source payloads should be printed.
### Required Tests
- Successful `notarius run` writes output files under a temp output directory.
- Output write rejects unsafe logical file names from a fake encoder.
- Writes are atomic enough that no temporary files remain after success.
- Diagnostics artifacts are written on success.
- Error log is written on failure after diagnostics directory creation.
- Retention `never` removes successful warning-free diagnostics directories.
- Warnings are present in diagnostics and are reported to stderr.
- `--diagnostics-dir` overrides config diagnostics directory.
### Validation
Run:
```sh
gofmt -w internal/cli internal/core/diagnostics
go test ./internal/cli ./internal/core/diagnostics
go test ./...
go vet ./...
go build ./cmd/notarius
```
## Stage 7: Add MVP Fixtures And End-To-End Acceptance Coverage
### Goal
Make the MVP path continuously testable without network access.
### Files To Add Or Update
Expected fixtures:
- `examples/seriatim-minimal-transcript.json`, if the example can be kept
accurate before the deferred documentation pass;
- `examples/dnd-spells.config.yml`, if config examples are tested in this
stage;
- or equivalent `internal/cli/testdata/...` fixtures if examples are deferred.
Expected tests:
- `internal/cli/run_test.go`
- `internal/modules/extract/dnd/spells/runner_test.go`
- config tests as needed.
### Required Design
Add a maintained MVP config fixture:
```yaml
version: 1
llm_profiles:
default:
provider: openai-compatible
base_url: http://127.0.0.1:1
model: fake-model
pipelines:
dnd-session:
input: seriatim
chunk:
module: generic
options:
max_units: 50
artifacts:
spells:
extract: dnd/spells
```
The fixture may use a fake base URL because tests should inject a fake LLM
client factory. Do not require a real network call.
Acceptance tests should execute the public CLI entry path with:
```sh
notarius run dnd-session --config <fixture> --input <fixture> --output-dir <tmp>
notarius run dnd-session --config <fixture> --input <fixture> --only spells --output-dir <tmp>
notarius config validate --config <fixture> --pipeline dnd-session
notarius pipelines list --config <fixture>
```
The fake LLM should return deterministic D&D spell output with valid source
references. The resulting output files should be parsed as JSON and checked for:
- manifest pipeline ID and digest;
- spell artifact payload;
- source references;
- prompt/schema metadata in manifest or artifact metadata;
- validation status;
- warning behavior.
### Required Failure Coverage
Add fixture-driven tests for:
- missing config;
- unknown pipeline;
- invalid Seriatim input;
- invalid `--only` lane;
- fake LLM failure;
- malformed LLM response;
- invalid source reference rejection.
### Validation
Run:
```sh
gofmt -w internal/cli internal/modules/extract/dnd/spells
go test ./internal/cli ./internal/modules/extract/dnd/spells
go test ./...
go vet ./...
go build ./cmd/notarius
```
## Stage 8: MVP Final Review And Roadmap Cleanup
### Goal
Confirm the MVP is complete enough to trigger the deferred documentation pass.
### Required Review
Perform a code review against:
- [`mvp.md`](mvp.md);
- [`../policy/architecture.md`](../policy/architecture.md);
- [`../policy/documentation.md`](../policy/documentation.md).
Check specifically:
- no D&D prompt/schema assets or constants remain in framework packages;
- production CLI commands use production wiring by default;
- `config validate --pipeline` works with the MVP fixture;
- `pipelines list` works with the MVP fixture;
- `notarius run` writes durable output and diagnostics;
- default modules resolve without test-only registration;
- output warnings remain out-of-band from artifact payloads;
- no private data or secrets appear in fixtures;
- docs outside `docs/roadmap/` describe only implemented behavior.
### Required Validation
Run:
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
```
### Required Roadmap Update
After the MVP is implemented and reviewed:
- update [`mvp.md`](mvp.md) to mark MVP functionality complete or reduce it to
remaining release/documentation work;
- keep the full documentation pass deferred until this review passes;
- do not tag alpha `0.1.0` until the documentation pass is complete.
## Open Questions
None. The plan above makes the required MVP implementation choices explicitly.

View File

@@ -1,632 +0,0 @@
# Initial Architecture Roadmap
## Status
This document captures proposed architecture and implementation sequencing for
Notarius. It describes planned work, not implemented behavior.
## Goal
Notarius should extract structured JSON artifacts from primary source inputs
using modular, LLM-backed extractors.
The first MVP should target audio transcripts generated by Seriatim. That
choice should be implemented as an input-stage module, not as a
transcript-specific assumption in the application core. Later input sources,
such as unstructured Markdown notes or Obsidian documents, should be addable
through new input and extract modules without reshaping the framework.
The first extraction domain should be D&D session analysis, starting with spell
casts. That domain should live in extract-stage modules and related schemas, not
in core framework packages.
The application should follow the same broad architecture as Audita:
- deterministic core packages for source documents, artifacts, and configuration once needed;
- input-stage modules that translate external source formats into a small internal source model;
- reusable framework packages for contracts, orchestration, LLM runtime, structured output, and validation;
- independent extract-stage modules that own domain-specific behavior;
- independent validator packages;
- embedded prompt and JSON schema assets;
- CLI orchestration that wires the pieces together without owning domain logic.
The main domain difference from Audita is that Notarius emits extracted
artifacts rather than proposing and applying transcript corrections.
## Architectural Principles
- Keep the core input model generic: ordered text units plus metadata.
- Keep source-format details in hexagonal input modules.
- Keep extraction-domain details in extract modules.
- Treat evidence as source references, not transcript references.
- Prefer narrow, useful abstractions over a universal document model.
- Preserve enough provenance for validation, replay, and downstream inspection.
## Proposed Package Shape
```text
cmd/notarius
internal/cli
internal/core/source
internal/core/artifacts
internal/framework/contracts
internal/framework/pipeline
internal/framework/validate
internal/framework/llm
internal/framework/prompt
internal/modules/input/seriatim
internal/modules/input/markdown
internal/modules/chunk/generic
internal/modules/chunk/dndtranscript
internal/modules/extract/dnd/spells
internal/modules/extract/dnd/items
internal/modules/extract/dnd/npcs
internal/modules/extract/dnd/combat
internal/modules/merge/appendorder
internal/modules/merge/dnd/spells
internal/modules/normalize/noop
internal/modules/normalize/dnd/spells
internal/modules/output/json
internal/validators/source_refs
internal/validators/schema_validity
internal/validators/domain_consistency
internal/validators/llm_review
examples
docs/internal
```
The `markdown` input module and D&D-specific chunk, merge, normalize, and
output modules are listed as likely future packages. The MVP should implement
only the stage modules needed by the checkpoint sequence.
`internal/core/config` should be added when production configuration exists.
The framework package list is intentionally consolidated. `pipeline` should own
runner orchestration, stage registries, and small merge/normalize/output helpers
until those boundaries prove they need separate packages. `llm` should own
structured output and response-schema mechanics until those concerns become too
large or import-heavy. `prompt` should own prompt assets and rendering helpers
once prompt assets exist.
## Core Concepts
### SourceDocument
Canonical internal representation of source material. This should be the object
extractors receive, regardless of whether the original input was a transcript,
Markdown file, note export, or another source type.
```go
type SourceDocument struct {
ID string `json:"id"`
Kind string `json:"kind"`
Format string `json:"format"`
Digest string `json:"digest"`
Units []SourceUnit `json:"units"`
Metadata map[string]any `json:"metadata,omitempty"`
}
type SourceUnit struct {
ID string `json:"id"`
Kind string `json:"kind"`
Text string `json:"text"`
Metadata map[string]any `json:"metadata,omitempty"`
}
```
Initial source-unit assumptions:
- units are ordered;
- unit IDs are stable within a source document;
- each unit has extractable text;
- adapter-specific metadata may carry speaker, timestamps, heading paths, page
numbers, or other source details.
Core source metadata should remain `map[string]any`. Notarius should not define
a universal document model. Instead, the project should document well-known
metadata keys, such as `speaker`, `start`, `end`, and `heading_path`, as
conventions. Input modules may export typed accessor helpers for their own
metadata, such as `seriatim.SpeakerOf(unit)`, without leaking those helpers into
core framework contracts.
### Input Module / Adapter Contract
Hexagonal boundary for external source formats.
```go
type InputAdapter interface {
Key() string
Parse(ctx context.Context, req ParseRequest) (*source.SourceDocument, error)
}
```
The MVP input module should target Seriatim minimal transcript JSON. Seriatim segment
fields should map as follows:
- `id` becomes `SourceUnit.ID`;
- `text` becomes `SourceUnit.Text`;
- `speaker`, `start`, and `end` become unit metadata;
- Seriatim output metadata becomes document metadata.
The core runner should not know that these units came from transcript segments.
### SourceRef
Grounding reference from an extracted fact back to source units.
```go
type SourceRef struct {
SourceID string `json:"source_id"`
StartUnitID string `json:"start_unit_id"`
EndUnitID string `json:"end_unit_id"`
}
```
Initial source-reference validation should require:
- source ID exists for the current run;
- start and end unit IDs exist;
- start is less than or equal to end in document order;
- the referenced range is contiguous within the source document;
- every extracted fact has at least one source reference unless its schema
explicitly allows ungrounded metadata.
Transcript-oriented output can still present these as transcript segment ranges
when the adapter metadata makes that interpretation available.
Source references should preserve the exact ranges produced by extractors and
validators. Overlapping ranges should not be merged or rewritten by generic
pipeline code. If a domain module wants a derived compact range later, that
should be additional output, not a replacement for the original evidence.
### Extractor
Reusable module contract for producing one artifact type.
```go
type Extractor interface {
Key() string
ArtifactType() string
SchemaVersion() string
Validators() []Validator
Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error)
}
```
An extractor should receive either a whole source document or a source chunk,
depending on processing mode. It should return typed artifact candidates plus
warnings. It should not mutate the source document.
`ExtractionRequest` should be designed now to carry both the active chunk and
optional ambient context, even if the MVP leaves that context empty. Useful
ambient context may include a document synopsis, prior-chunk summaries, known
entities, or other module-provided state. D&D spell extraction can likely work
per chunk, but combat, NPC, and identity-oriented extraction will need broader
context. Adding the field later would force churn across every extractor.
Extract modules own domain concepts. For example, D&D spell extraction should
live under `internal/modules/extract/dnd/spells`; a future to-do extractor for
notes should live under a different extract-module path and use the same
framework contract.
### Chunker
Reusable stage contract for splitting a source document into ordered source
chunks.
Chunking is a first-class pipeline concern because source documents may exceed a
single LLM extraction pass. Chunkers should preserve source-unit order and
produce stable chunk metadata suitable for diagnostics and replay.
### Merger
Reusable stage contract for combining per-chunk artifact candidates into one
merged candidate collection.
Merge should combine outputs without doing semantic reconciliation. A generic
append-in-chunk-order merger should be sufficient for many artifact streams,
including the likely first D&D spell-cast extractor.
### Normalizer
Reusable stage contract for reconciling merged artifact candidates.
Normalize is distinct from merge. Normalizers may deduplicate repeated facts,
resolve aliases, reconcile conflicting fields, check cross-chunk consistency,
or attach normalization warnings.
### Validator
Reusable validation contract for artifact candidates.
Validators should cover:
- JSON/schema validity;
- source-reference validity;
- required-field and shape checks;
- domain consistency;
- optional LLM review for high-risk or ambiguous artifacts.
Validator output should follow Audita's decision-cardinality model: each
candidate artifact receives exactly one decision per validator.
LLM-backed review should be modeled as part of a module's validator chain, not
as a separate global review phase. Extract modules should be able to attach one
or more deterministic or LLM-backed validators. Normalize-stage modules may also
run validator chains, including LLM-backed validators, when semantic
reconciliation needs review.
### Artifact
Final approved JSON output from one or more extractors.
Artifacts should preserve enough metadata to support downstream validation,
debugging, and replay.
The pipeline should carry artifact candidates through a generic envelope with a
`json.RawMessage` payload. Extract modules should own typed Go structs at their
module boundary, then encode those typed records into the generic artifact
candidate envelope before returning to framework code. This keeps stage
contracts simple and avoids generic type plumbing across unrelated artifact
families.
Final durable output should be one file per artifact type plus a run-level
manifest/index file. This supports partial success and lets downstream consumers
read only the artifact types they need. Each artifact file should include its
artifact type, extractor key, extractor schema version, envelope format version,
records, source references, and enough provenance to connect it to the run
manifest.
Every artifact record should require source references unless that artifact
schema explicitly opts into ungrounded fields. Artifact-level metadata, counts,
run information, and other derived summary fields are exempt from the per-record
grounding rule.
Schemas should be versioned per extractor, with a separate envelope/manifest
format version. A single global schema version would couple unrelated extractor
release cadence.
### RunManifest
Per-run provenance record.
```go
type RunManifest struct {
EnvelopeVersion string `json:"envelope_version"`
PipelineID string `json:"pipeline_id"`
PipelineDigest string `json:"pipeline_digest"`
InputModule string `json:"input_module"`
Chunker string `json:"chunker"`
SourceDigests []string `json:"source_digests"`
Extractors []string `json:"extractors"`
Merger string `json:"merger"`
Normalizer string `json:"normalizer"`
OutputEncoder string `json:"output_encoder"`
SchemaVersion string `json:"schema_version"`
ValidationStatus string `json:"validation_status"`
}
```
The manifest should eventually include model names, prompt IDs, prompt hashes,
response schema versions, config source, redacted resolved config digest,
started/completed timestamps, and diagnostics paths.
## Initial Extractor Targets
### D&D Spells
Recommended first vertical slice because it is narrow but representative.
```go
type SpellCast struct {
Player string `json:"player"`
Spell string `json:"spell"`
Effect string `json:"effect"`
NarrativeDescription string `json:"narrative_description"`
SourceRefs []SourceRef `json:"source_refs"`
}
```
The spell extractor should be D&D-specific. The framework should not know what a
spell is.
### D&D Items
Tracks items gained, lost, transferred, consumed, or transformed.
Open questions:
- Should currency be represented as items or as its own artifact type?
- Should item ownership be a required field?
- How should ambiguous ownership changes be represented?
### D&D NPCs
Tracks NPCs interacted with, newly introduced, renamed, described, or otherwise
made relevant to campaign state.
Open questions:
- Should NPC identity resolution happen inside this extractor or in a later
deduplication stage?
- Should location/faction/relationship facts be separate artifact types?
### D&D Combat
Likely warrants a dedicated schema rather than a generic event list.
Proposed first shape:
```go
type CombatTurn struct {
Actor string `json:"actor"`
Action string `json:"action"`
Outcome string `json:"outcome"`
NarrativeDescription string `json:"narrative_description"`
SourceRefs []SourceRef `json:"source_refs"`
}
```
Open questions:
- Should combat be extracted as turns, rounds, encounters, or all three?
- Should mechanical fields such as damage, conditions, saves, attacks, and spell
slots be normalized immediately or added later?
- How should uncertain initiative order be represented?
### Future Non-D&D Extractors
The architecture should support extractors outside the D&D domain. Examples:
- to-do items from Markdown or Obsidian notes;
- decisions and action items from meeting transcripts;
- named people, places, and dates from research notes.
These should be addable as extract modules without changing runner,
validator, source-reference, or LLM framework contracts.
## Configuration Model
Notarius should use named pipeline profiles selected by ID at the CLI. A
pipeline is a fixed-shape template for the known application workflow, not a
free-form list of steps:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
A pipeline profile should define one shared front end and one or more artifact
lanes:
- shared input module;
- shared chunk module by default;
- artifact lanes containing extract, merge, normalize, and validator behavior;
- shared output module.
The MVP should use one shared chunk module per pipeline. Per-lane chunk
overrides can be added later if an artifact lane, such as combat, proves it
needs a different chunking strategy.
Example shape:
```yaml
llm_profiles:
default:
model: example-model
max_concurrency: 4
pipelines:
dnd-session:
input: seriatim
chunk: dnd/transcript
artifacts:
spells:
extract: dnd/spells
normalize: dnd/spells
npcs:
extract: dnd/npcs
items:
extract: dnd/items
```
The CLI should run named pipelines:
```sh
notarius run dnd-session --input session-014.json
notarius run dnd-session --input session-014.json --only spells,npcs
```
`--only` should select configured artifact lanes. It should not create an
ad hoc pipeline. Structural module selection should come from config, while CLI
flags may override operational knobs such as model, concurrency, output
directory, and diagnostics directory.
Initial defaults:
- `chunk`: `generic`;
- lane `merge`: `appendorder`;
- lane `normalize`: `noop`;
- `output`: `json`;
- `llm_profile`: `default` where an LLM profile is needed.
Module bindings should support both string shorthand and object form:
```yaml
extract: dnd/spells
```
```yaml
extract:
module: dnd/spells
llm_profile: fast
prompt_version: v1
```
Both forms should normalize into a single internal `ModuleBinding` shape before
validation and manifest hashing.
Pipeline validation should use module metadata declared through registries.
Modules should expose flat string capability metadata, such as `speaker` or
`timestamps`, without requiring module construction. Config validation should
fail fast for:
- unknown pipeline IDs;
- unknown module keys;
- missing required slots;
- missing required capabilities;
- unknown LLM profiles;
- empty artifact-lane sets;
- `--only` lane names that do not exist in the selected pipeline.
The MVP should keep pipelines config-file-only. Built-in pipeline profiles can
be added later if the project needs embedded defaults, but that introduces
merge/override semantics that the MVP does not need.
The resolved pipeline definition should be hashed after defaults and lane
selection are applied. The run manifest should record both `pipeline_id` and
`pipeline_digest`; a pipeline ID alone is not stable provenance.
## Proposed Pipeline Flow
The application workflow should be first-class:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
Proposed runner flow:
1. Load effective config.
2. Resolve the selected pipeline profile by ID.
3. Apply defaults and `--only` lane selection.
4. Validate module keys, lane definitions, LLM profiles, and capabilities.
5. Hash the resolved pipeline definition.
6. Create diagnostics run directory.
7. Resolve the configured input module through the input adapter registry.
8. Read source input.
9. Parse source input into a `SourceDocument`.
10. Validate source-document invariants.
11. Resolve the configured chunker.
12. Chunk source units into deterministic source chunks.
13. Resolve configured artifact lanes through registries.
14. Extract, merge, normalize, and validate each selected artifact lane.
15. Retain approved artifacts and rejected-artifact diagnostics.
16. Serialize output files and run-level manifest/index.
17. Write diagnostics and optional report JSON.
The runner should operate on source documents and source chunks only. Any
transcript-specific behavior should happen before the runner, inside the input
adapter, or after the runner, inside output rendering that understands source
metadata.
## Audita Patterns To Reuse
Reuse these architectural patterns:
- deterministic parsing and schema validation style;
- deterministic chunking of ordered source units;
- explicit extractor registry;
- explicit pipeline stage contracts;
- `contracts` package for transport-neutral interfaces;
- OpenAI-compatible structured LLM client;
- scheduler for bounded LLM concurrency;
- embedded prompt registry with prompt metadata and hashes;
- embedded response-schema registry with schema metadata and hashes;
- diagnostics run directory with redacted effective config;
- validator decision cardinality and deterministic validator ordering;
- CLI tests and fixture-driven integration tests.
The fixture-driven integration-test pattern should remain part of the codebase:
walking skeleton tests over fake modules and fake LLM clients should be
preserved as real Seriatim, runtime, and D&D modules are added, so the
end-to-end contract coverage is not lost.
Avoid copying these Audita concepts directly:
- transcript-specific core types;
- correction proposals;
- replacement policies;
- deterministic transcript mutation;
- correction ledger terminology.
Those concepts are specific to Audita's transcript-editing role and should be
replaced with source-document, artifact-candidate, artifact-validation, and
extraction-report concepts.
## Checkpoint Roadmap
The initial six checkpoint roadmap has been implemented and retired. The
checkpoint files have been removed from `docs/roadmap/` so the active roadmap
does not compete with completed implementation history.
The remaining work needed to reach the first functional MVP is tracked in
[`mvp.md`](mvp.md). Future staged implementation plans should be written to
[`implementation.md`](implementation.md) from that active MVP roadmap.
## Architecture Decisions
- Final durable output should use one artifact file per artifact type plus a
run-level manifest/index file.
- Framework artifact flow should use a generic envelope with `json.RawMessage`
payloads. Extract modules should use typed Go structs at their own boundaries.
- Schemas should be versioned per extractor, with a separate envelope/manifest
format version.
- Artifact records should require source references by default. Individual
schemas may explicitly opt into ungrounded fields. Artifact-level metadata is
exempt.
- Source-reference ranges should be preserved exactly. Generic pipeline code
should not merge or rewrite overlapping ranges.
- `ExtractionRequest` should carry the active chunk plus optional ambient
context for document synopsis, prior-chunk summaries, known entities, or
similar module-provided state.
- LLM-backed review should be part of module-owned validator chains. Extract
modules and normalize modules may both use deterministic and LLM-backed
validators.
- The Seriatim MVP should support only the minimal Seriatim schema. Broader
Seriatim schema support should be added later without changing core source
contracts.
- Core source metadata should remain `map[string]any`. Well-known metadata keys
should be documented as conventions, and input modules may expose typed
accessor helpers for their own metadata.
- Configuration should use named pipeline profiles selected by ID at the CLI.
- A pipeline profile should be a fixed template, not a free-form DAG: shared
input and chunk stages, one or more artifact lanes, and shared output.
- `--only` should select configured artifact lanes without creating ad hoc
pipelines.
- Module bindings should support string shorthand and inline object settings,
normalized into one internal binding shape.
- Registries should expose flat capability metadata so config can fail fast on
invalid module combinations.
- The run manifest should record both `pipeline_id` and a digest of the resolved
pipeline definition after defaults and lane selection.
## Open Design Questions
- Which artifact types should use generic append-in-chunk-order merge, and which
should use domain-specific merge?
- Which artifact types need domain-specific normalization for deduplication,
identity resolution, or consistency?
- Which operational settings should be allowed as CLI/environment overrides
without weakening pipeline provenance?
## Near-Term Documentation Tasks
Once behavior is implemented, move implemented contracts out of roadmap docs and
into canonical docs:
- `README.md` for purpose and shortest useful command;
- `docs/cli.md` for CLI behavior;
- `docs/config.md` for config fields and precedence;
- `docs/internal/` for implemented architecture and package boundaries;
- `docs/integrations/` for source input and artifact file formats;
- `examples/` for maintained source, config, and artifact examples.

View File

@@ -1,157 +1,29 @@
# MVP Roadmap
# Future Work
## Status
Current Notarius behavior is documented in the canonical README, CLI,
configuration, operations, internal, and integration docs. This roadmap records
future work only.
The first functional Notarius MVP implementation is complete enough to start the
deferred documentation pass.
## Candidate Product Work
The previous numbered checkpoint roadmaps have been implemented and retired.
This document now records the implemented MVP scope and the remaining
release/documentation work before alpha `0.1.0`.
- Additional input adapters, such as Markdown or note-export formats.
- Additional D&D extractors beyond spell casts.
- Cross-lane entity normalization.
- Cross-chunk semantic deduplication.
- Configurable validator chains with production validator modules.
- Multiple effective LLM profiles in one run.
- Parallel execution where it preserves deterministic manifests and diagnostics.
- Additional output encoders.
Implementation staging belongs in [`implementation.md`](implementation.md).
## Candidate Operational Work
## MVP Goal
- Packaged release artifacts for alpha distribution.
- A documented versioning and release process.
- Optional generated example output fixtures with a regeneration procedure.
- Additional diagnostics or reporting views if operator workflows need them.
The MVP should let a user run Notarius against a Seriatim minimal transcript
JSON file, select a configured pipeline profile, extract D&D spell-cast
artifacts with an LLM-backed extractor, validate those artifacts, and write
durable JSON output plus diagnostics.
## Non-Goals To Revisit Deliberately
The intended command shape is:
```sh
notarius run dnd-session --input session-014.json
notarius run dnd-session --input session-014.json --only spells
```
The MVP remains intentionally narrow:
- one production input module: `seriatim`;
- one production extract module: `dnd/spells`;
- one generic chunk module sufficient for transcript-scale processing;
- generic append-order merge;
- generic noop normalization;
- generic JSON output;
- config-driven pipeline profiles;
- OpenAI-compatible structured LLM execution through the existing LLM client.
## Implemented MVP Behavior
The MVP now includes:
- framework/domain asset boundaries: D&D spell prompt and response schema assets
are owned by `internal/modules/extract/dnd/spells`, while framework prompt and
LLM packages provide only generic primitives;
- production CLI wiring that registers `seriatim`, `dnd/spells`, and the
default `generic`, `appendorder`, `noop`, and `json` modules;
- a config-driven `notarius run` command that reads Seriatim input, resolves a
configured pipeline, invokes the runner, and writes durable output plus
diagnostics;
- fixture-driven CLI acceptance coverage using maintained example config and
transcript fixtures with a fake LLM path, so `go test ./...` exercises the MVP
without network access.
The production pipeline defaults are:
- `chunk: generic`;
- `merge: appendorder`;
- `normalize: noop`;
- `output: json`.
The implemented command shape is:
```sh
notarius run <pipeline-id> --input path/to/source.json
notarius run <pipeline-id> --input path/to/source.json --only spells
```
Current output and diagnostics behavior:
- output is written under `<output-root>/<run-id>/`, defaulting to
`./notarius-output`;
- JSON output includes a manifest, grouped approved artifacts, rejected
artifacts, and warnings;
- artifact records include generic source references;
- output-stage warnings remain out-of-band from artifact payloads and are
captured for CLI reporting and diagnostics;
- run manifest data includes source digest, resolved pipeline digest, LLM
profile/model metadata, prompt/schema identifiers, and validation status;
- diagnostics redact secrets and include invocation metadata, redacted effective
config, resolved pipeline, manifest, warnings, run report, and error logs.
Maintained MVP fixtures:
- `examples/dnd-spells.config.yml`;
- `examples/seriatim-minimal-transcript.json`;
- `internal/cli/testdata/invalid-seriatim-empty-segments.json`.
## Remaining Release And Documentation Work
The full documentation pass is intentionally deferred until MVP functionality
exists. It should happen before tagging alpha `0.1.0`.
Remaining work before alpha `0.1.0`:
- move implemented CLI behavior into `docs/cli.md`;
- move implemented config behavior into `docs/config.md`;
- document run output and diagnostics behavior in canonical docs;
- update integration docs for Seriatim input, D&D spell artifacts, and JSON
output where needed;
- update `README.md` with a shortest useful command based on the maintained
examples;
- keep examples validated by tests.
## Out Of Scope For MVP
- D&D item extraction;
- NPC extraction;
- combat extraction;
- D&D rules validation beyond the spell extractor's deterministic checks;
- Markdown or Obsidian input;
- cross-lane entity normalization;
- cross-chunk semantic deduplication beyond whatever a simple normalizer can
safely support;
- a general DAG or workflow engine;
- ad hoc CLI flags for structural module selection;
- release-quality documentation before the MVP behavior is implemented.
## MVP Done Criteria
- D&D prompt and response schema assets are owned by the D&D spells module, not
by framework packages.
- Production CLI commands use a real app catalog rather than test-injected
module catalogs.
- A config profile can bind `input: seriatim` and an artifact lane with
`extract: dnd/spells`.
- Default `generic`, `appendorder`, `noop`, and `json` modules resolve through
production wiring.
- `notarius config validate --config <file> --pipeline <id>` works with the
MVP config.
- `notarius pipelines list --config <file>` works with the MVP config.
- `notarius run <pipeline-id> --input <file>` reads a Seriatim transcript,
extracts D&D spell artifacts, validates them, and writes JSON output.
- `notarius run <pipeline-id> --input <file> --only spells` runs only the
selected artifact lane.
- The run manifest records source digest, resolved pipeline digest, LLM profile
and model, prompt/schema identifiers, and validation status.
- Output warnings are available to CLI/diagnostics without becoming artifact
payload fields.
- MVP fixture tests cover the full path without network access.
- `go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` pass.
## Deferred Documentation Pass
Before alpha `0.1.0`, complete a full documentation pass/rewrite. That pass
should move implemented behavior out of roadmap documents and into canonical
docs required by
[`../policy/documentation.md`](../policy/documentation.md), including at least:
- `README.md`;
- `docs/cli.md`;
- `docs/config.md`;
- `docs/operations.md`, if diagnostics/run recovery behavior warrants it;
- `docs/internal/` architecture and package-boundary docs;
- `docs/integrations/` updates for Seriatim input, D&D spell artifacts, and
JSON output;
- maintained `examples/` files.
- A general workflow language.
- Structural module selection through ad hoc run flags.
- Storing secrets in config files, diagnostics, manifests, or examples.

215
docs/troubleshooting.md Normal file
View File

@@ -0,0 +1,215 @@
# Troubleshooting
This guide maps common implemented failure modes to inspection steps and fixes.
For command syntax, see [CLI Reference](cli.md). For YAML fields and
environment overrides, see [Configuration](config.md). For output and
diagnostics layout, see [Operations](operations.md).
## Config File Not Found
Symptom:
```text
notarius: config file not found; pass --config or set NOTARIUS_CONFIG
```
Fix:
- Pass `--config path/to/config.yml`.
- Or set `NOTARIUS_CONFIG` to a readable file.
- Or install a config at `/usr/local/etc/notarius/config.yml`.
If the message says the config path is a directory or is not available, correct
the path or file permissions.
## Unsupported Or Invalid Config
Symptoms include:
- `unsupported config version`
- `config version is required`
- `field <name> not found`
- `total LLM concurrency must be greater than zero`
- `diagnostics retention "<value>" is not supported`
Fix:
- Use `version: 1`.
- Remove unknown YAML fields.
- Validate with:
```sh
go run ./cmd/notarius config validate --config path/to/config.yml
```
## Unknown Pipeline
Symptom:
```text
notarius: pipeline "..." is not configured
```
Fix:
- List configured pipeline IDs:
```sh
go run ./cmd/notarius pipelines list --config path/to/config.yml
```
- Use one of those IDs in `notarius run <pipeline-id>`.
- Check indentation under the top-level `pipelines` map.
## Unknown Or Incompatible Module
Symptoms mention a module key, pipeline slot, lane, capability, or `not
registered`.
Fix:
- Validate the pipeline against the production module catalog:
```sh
go run ./cmd/notarius config validate \
--config path/to/config.yml \
--pipeline dnd-session
```
- Use only implemented production module keys listed in
[Configuration](config.md#implemented-production-modules).
- Check that artifact lanes include an `extract` binding.
## Invalid `--only`
Symptoms include:
- `--only must contain comma-separated non-empty artifact lane IDs`
- `--only requires --pipeline`
- `selected artifact lane`
Fix:
- Use comma-separated lane IDs with no empty entries:
```sh
go run ./cmd/notarius run dnd-session \
--config path/to/config.yml \
--input path/to/input.json \
--only spells
```
- For `config validate`, include `--pipeline` when using `--only`.
- Confirm the lane ID exists under `pipelines.<id>.artifacts`.
## Seriatim Input Validation Failure
Symptoms include `seriatim input`, `parse JSON`, `segments must not be empty`,
or validation errors naming a segment field.
Fix:
- Compare the input to
[examples/seriatim-minimal-transcript.json](../examples/seriatim-minimal-transcript.json).
- Ensure the JSON has a `metadata` object and a non-empty `segments` array.
- Each segment needs a non-empty `id`, non-empty `speaker`, non-empty `text`,
non-negative numeric `start`, and non-negative numeric `end`.
- Segment IDs must be unique and must not contain leading or trailing
whitespace.
- `end` must be greater than or equal to `start`.
## Missing LLM Base URL Or Model
Symptoms include:
- `LLM profile "default" base URL must not be empty`
- `LLM profile "default" model must not be empty`
- `base URL must be valid`
Fix:
- Set `base_url` and `model` in `llm_profiles.default`.
- Or set `NOTARIUS_LLM_DEFAULT_BASE_URL` and
`NOTARIUS_LLM_DEFAULT_MODEL`.
- If a profile needs authentication, set `api_key_env` in YAML or set
`NOTARIUS_LLM_DEFAULT_API_KEY`.
## LLM Profile Override Failure
Symptom:
```text
notarius: LLM profile override "..." is not configured
```
Fix:
- Add the profile under `llm_profiles`.
- Or use an existing profile ID with `--llm-profile`.
Current runs require exactly one distinct effective LLM profile. If a pipeline
uses several profiles, run with `--llm-profile <id>` or align the bindings in
configuration.
## Provider HTTP Or Response Failure
Symptoms include:
- `provider request failed`
- `provider returned status 400`
- `provider returned status 403`
- `provider response missing choices`
- `provider response assistant message content is not valid JSON`
- `decode structured output`
Fix:
- Confirm the `base_url` points to an OpenAI-compatible endpoint root. Notarius
posts to `<base_url>/chat/completions`.
- Check `model` and provider credentials.
- Inspect the retained diagnostics `error.log`.
- For 400 and 403 responses, fix the request configuration or credentials.
- For 429 and 5xx responses, the client retries according to `max_retries`; if
the failure persists, inspect the provider response and adjust capacity,
credentials, or model settings.
- The assistant message content must decode as JSON matching the extractor's
structured response schema.
Provider error messages are redacted for configured API key values.
## Output Write Failure
Symptoms include:
- `create output directory`
- `write output file`
- `output file name must`
Fix:
- Ensure `--output-dir` points to a directory path or a path that can be
created.
- Check filesystem permissions and available disk space.
- If diagnostics were retained, inspect `run-report.json`, `run-manifest.json`,
and `error.log`.
The CLI rejects unsafe logical output paths before writing files.
## Diagnostics Directory Surprise
Symptom: the diagnostics directory is missing after a successful run.
Fix:
- Check `diagnostics.retention`.
- With `auto`, successful runs without warnings are removed.
- Use `diagnostics.retention: always` when every diagnostics run directory
should be kept.
- Use `--diagnostics-dir` to override the configured work directory for a run.
Symptom: diagnostics exist even with `retention: never`.
Explanation:
- Failed runs are retained so that `error.log` and available context can be
inspected.

View File

@@ -20,7 +20,7 @@ const Key = "seriatim"
const (
DocumentKind = "transcript"
UnitKind = "transcript_segment"
Format = "application/vnd.seriatim.minimal+json"
Format = "application/vnd.seriatim+json"
)
var providedCapabilities = []string{

View File

@@ -70,6 +70,35 @@ func TestParseValidMinimalTranscript(t *testing.T) {
}
}
func TestParseAcceptsNumericSegmentIDs(t *testing.T) {
raw := []byte(`{"metadata":{"application":"seriatim","version":"v1.5.0","output_schema":"seriatim-intermediate"},"segments":[{"id":1,"start":451.821,"end":469.685,"speaker":"Narrator","text":"The stone door opens.","categories":["scene"]},{"id":2,"start":470,"end":471,"speaker":"Player","text":"I step inside."}]}`)
doc, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw})
if err != nil {
t.Fatalf("Parse() error = %v, want nil", err)
}
if got, want := doc.Metadata["output_schema"], "seriatim-intermediate"; got != want {
t.Fatalf("doc.Metadata[output_schema] = %#v, want %q", got, want)
}
if doc.Format != Format {
t.Fatalf("doc.Format = %q, want %q", doc.Format, Format)
}
if len(doc.Units) != 2 {
t.Fatalf("len(doc.Units) = %d, want 2", len(doc.Units))
}
if doc.Units[0].ID != "1" || doc.Units[1].ID != "2" {
t.Fatalf("unit IDs = %#v, want numeric IDs normalized to strings", []string{doc.Units[0].ID, doc.Units[1].ID})
}
ref := source.SourceRef{
SourceID: doc.ID,
StartUnitID: "1",
EndUnitID: "2",
}
if err := source.ValidateRef(doc, ref); err != nil {
t.Fatalf("ValidateRef() error = %v, want nil", err)
}
}
func TestParseRequestSourceIDOverridesMetadataIDs(t *testing.T) {
raw := readFixture(t, "testdata/valid_minimal.json")
@@ -176,6 +205,11 @@ func TestParseRejectsInvalidInput(t *testing.T) {
raw: validJSONWithSegment(`"start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"id", "empty"},
},
{
name: "invalid segment id type",
raw: validJSONWithSegment(`"id":{},"start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"id", "string or number"},
},
{
name: "whitespace segment id",
raw: validJSONWithSegment(`"id":" s1 ","start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."`),

View File

@@ -77,8 +77,8 @@ func decodeSegment(raw []byte, index int) (segment, error) {
}
var decoded segment
if err := decodeOptionalString(fields, "id", &decoded.ID); err != nil {
return segment{}, fmt.Errorf("segment[%d] id must be a string: %w", index, err)
if err := decodeOptionalSegmentID(fields, "id", &decoded.ID); err != nil {
return segment{}, fmt.Errorf("segment[%d] id must be a string or number: %w", index, err)
}
if err := decodeOptionalNumber(fields, "start", &decoded.Start); err != nil {
return segment{}, fmt.Errorf("segment[%d] start must be a number: %w", index, err)
@@ -103,6 +103,27 @@ func decodeOptionalString(fields map[string]json.RawMessage, key string, out *st
return decodeJSON(raw, out)
}
func decodeOptionalSegmentID(fields map[string]json.RawMessage, key string, out *string) error {
raw, ok := fields[key]
if !ok {
return nil
}
var text string
if err := decodeJSON(raw, &text); err == nil {
*out = text
return nil
}
var number json.Number
if err := decodeJSON(raw, &number); err == nil {
*out = number.String()
return nil
}
return fmt.Errorf("must be a string or number")
}
func decodeOptionalNumber(fields map[string]json.RawMessage, key string, out *json.Number) error {
raw, ok := fields[key]
if !ok {