Compare commits
5 Commits
5fab9936d0
...
4d0b2c69e6
| Author | SHA1 | Date | |
|---|---|---|---|
| 4d0b2c69e6 | |||
| 30e98a4d99 | |||
| 6ceabf40bb | |||
| 2d75f6ad13 | |||
| 4dd00347e0 |
36
README.md
36
README.md
@@ -1,2 +1,36 @@
|
|||||||
# go-application-template
|
# Notarius
|
||||||
|
|
||||||
|
Notarius is a Go CLI for extracting structured artifacts from source material
|
||||||
|
with explicit, configurable pipeline modules.
|
||||||
|
|
||||||
|
The current implementation reads Seriatim minimal transcript JSON, chunks the
|
||||||
|
source units, extracts D&D spell-cast artifacts with an OpenAI-compatible LLM,
|
||||||
|
and writes JSON output plus diagnostics for each run.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
NOTARIUS_LLM_DEFAULT_BASE_URL=http://127.0.0.1:8080/v1 \
|
||||||
|
NOTARIUS_LLM_DEFAULT_MODEL=your-model \
|
||||||
|
go run ./cmd/notarius run dnd-session \
|
||||||
|
--config examples/dnd-spells.config.yml \
|
||||||
|
--input examples/seriatim-minimal-transcript.json
|
||||||
|
```
|
||||||
|
|
||||||
|
If the provider requires authentication, set
|
||||||
|
`NOTARIUS_LLM_DEFAULT_API_KEY` in the environment before running the command.
|
||||||
|
Outputs are written under `./notarius-output/<run-id>/` unless `--output-dir`
|
||||||
|
is provided.
|
||||||
|
|
||||||
|
Useful references:
|
||||||
|
|
||||||
|
- [CLI reference](docs/cli.md)
|
||||||
|
- [Configuration reference](docs/config.md)
|
||||||
|
- [Operations](docs/operations.md)
|
||||||
|
- [Troubleshooting](docs/troubleshooting.md)
|
||||||
|
- [Seriatim input contract](docs/integrations/seriatim.md)
|
||||||
|
- [OpenAI-compatible provider contract](docs/integrations/openai-compatible.md)
|
||||||
|
- [JSON output contract](docs/integrations/json-output.md)
|
||||||
|
- [D&D spell artifact contract](docs/integrations/dnd-spell-artifacts.md)
|
||||||
|
- [Developer workflow](docs/policy/development.md)
|
||||||
|
- [Internal architecture docs](docs/internal/overview.md)
|
||||||
|
- [Maintained example config](examples/dnd-spells.config.yml)
|
||||||
|
- [Maintained example input](examples/seriatim-minimal-transcript.json)
|
||||||
|
|||||||
129
docs/cli.md
Normal file
129
docs/cli.md
Normal file
@@ -0,0 +1,129 @@
|
|||||||
|
# CLI Reference
|
||||||
|
|
||||||
|
This is the canonical reference for the implemented Notarius command-line
|
||||||
|
interface.
|
||||||
|
|
||||||
|
## Quick Run
|
||||||
|
|
||||||
|
```sh
|
||||||
|
NOTARIUS_LLM_DEFAULT_BASE_URL=http://127.0.0.1:8080/v1 \
|
||||||
|
NOTARIUS_LLM_DEFAULT_MODEL=your-model \
|
||||||
|
go run ./cmd/notarius run dnd-session \
|
||||||
|
--config examples/dnd-spells.config.yml \
|
||||||
|
--input examples/seriatim-minimal-transcript.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Set `NOTARIUS_LLM_DEFAULT_API_KEY` if the OpenAI-compatible provider requires
|
||||||
|
a bearer token.
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
```text
|
||||||
|
notarius help
|
||||||
|
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b]
|
||||||
|
notarius config validate --config path/to/config.yml [--pipeline pipeline-id] [--only lane-a,lane-b]
|
||||||
|
notarius pipelines list --config path/to/config.yml [--json]
|
||||||
|
```
|
||||||
|
|
||||||
|
Running `notarius` with no arguments, `notarius help`, `notarius --help`, or
|
||||||
|
`notarius -h` prints usage and exits successfully.
|
||||||
|
|
||||||
|
## `run`
|
||||||
|
|
||||||
|
`notarius run <pipeline-id>` executes a configured pipeline against one input
|
||||||
|
file.
|
||||||
|
|
||||||
|
Flags:
|
||||||
|
|
||||||
|
- `--input path`: required source input file.
|
||||||
|
- `--config path`: config file path. If omitted, Notarius checks
|
||||||
|
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
|
||||||
|
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are
|
||||||
|
comma-separated and must be non-empty.
|
||||||
|
- `--output-dir path`: output root. The run writes to `<path>/<run-id>/`.
|
||||||
|
Defaults to `./notarius-output`.
|
||||||
|
- `--diagnostics-dir path`: diagnostics work directory override for this
|
||||||
|
invocation.
|
||||||
|
- `--llm-profile id`: override every effective module binding to use one LLM
|
||||||
|
profile.
|
||||||
|
|
||||||
|
On success, the command prints the completed pipeline ID, approved and rejected
|
||||||
|
artifact counts, and the output directory. If the run completes with warnings,
|
||||||
|
the warning count is printed to stderr.
|
||||||
|
|
||||||
|
For durable output, diagnostics, retention, and failure inspection, see
|
||||||
|
[Operations](operations.md).
|
||||||
|
|
||||||
|
The current `run` command requires the resolved pipeline to use exactly one
|
||||||
|
distinct LLM profile after defaults and overrides are applied.
|
||||||
|
|
||||||
|
## `config validate`
|
||||||
|
|
||||||
|
`notarius config validate` loads and validates configuration.
|
||||||
|
|
||||||
|
Flags:
|
||||||
|
|
||||||
|
- `--config path`: config file path. If omitted, discovery uses
|
||||||
|
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
|
||||||
|
- `--pipeline pipeline-id`: additionally resolve one configured pipeline against
|
||||||
|
the production module catalog.
|
||||||
|
- `--only lane-a,lane-b`: validate resolution for selected artifact lanes. This
|
||||||
|
flag requires `--pipeline`.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/notarius config validate \
|
||||||
|
--config examples/dnd-spells.config.yml
|
||||||
|
|
||||||
|
go run ./cmd/notarius config validate \
|
||||||
|
--config examples/dnd-spells.config.yml \
|
||||||
|
--pipeline dnd-session \
|
||||||
|
--only spells
|
||||||
|
```
|
||||||
|
|
||||||
|
## `pipelines list`
|
||||||
|
|
||||||
|
`notarius pipelines list` prints configured pipeline IDs in sorted order.
|
||||||
|
|
||||||
|
Flags:
|
||||||
|
|
||||||
|
- `--config path`: config file path. If omitted, discovery uses
|
||||||
|
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
|
||||||
|
- `--json`: print `{"pipelines":[...]}` instead of one ID per line.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/notarius pipelines list \
|
||||||
|
--config examples/dnd-spells.config.yml
|
||||||
|
|
||||||
|
go run ./cmd/notarius pipelines list \
|
||||||
|
--config examples/dnd-spells.config.yml \
|
||||||
|
--json
|
||||||
|
```
|
||||||
|
|
||||||
|
## Exit Codes
|
||||||
|
|
||||||
|
- `0`: command succeeded.
|
||||||
|
- `1`: command syntax was valid, but loading config, resolving modules, running
|
||||||
|
the pipeline, calling the provider, writing output, or writing diagnostics
|
||||||
|
failed.
|
||||||
|
- `2`: command syntax was invalid, a command was unknown, a required argument
|
||||||
|
was missing, or a flag value was malformed.
|
||||||
|
|
||||||
|
## Implemented Production Pipeline Modules
|
||||||
|
|
||||||
|
The production CLI currently registers these module keys:
|
||||||
|
|
||||||
|
- input: `seriatim`
|
||||||
|
- chunk: `generic`
|
||||||
|
- extract: `dnd/spells`
|
||||||
|
- merge: `appendorder`
|
||||||
|
- normalize: `noop`
|
||||||
|
- output: `json`
|
||||||
|
|
||||||
|
The production CLI does not currently register validator modules.
|
||||||
|
|
||||||
|
For YAML structure, defaults, environment overrides, and module binding syntax,
|
||||||
|
see [Configuration](config.md).
|
||||||
213
docs/config.md
Normal file
213
docs/config.md
Normal file
@@ -0,0 +1,213 @@
|
|||||||
|
# Configuration
|
||||||
|
|
||||||
|
This is the canonical reference for implemented Notarius configuration.
|
||||||
|
|
||||||
|
Notarius reads YAML config files with `version: 1`. File config is applied over
|
||||||
|
built-in defaults, then environment overrides are applied.
|
||||||
|
|
||||||
|
## Discovery
|
||||||
|
|
||||||
|
Commands that accept `--config` load configuration in this order:
|
||||||
|
|
||||||
|
1. the `--config` path, when provided;
|
||||||
|
2. `NOTARIUS_CONFIG`, when set to a non-empty path;
|
||||||
|
3. `/usr/local/etc/notarius/config.yml`.
|
||||||
|
|
||||||
|
If none is available, the command fails with a config file not found error.
|
||||||
|
|
||||||
|
## Minimal Example
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
version: 1
|
||||||
|
llm_profiles:
|
||||||
|
default:
|
||||||
|
provider: openai-compatible
|
||||||
|
base_url: http://127.0.0.1:8080/v1
|
||||||
|
model: your-model
|
||||||
|
pipelines:
|
||||||
|
dnd-session:
|
||||||
|
input: seriatim
|
||||||
|
chunk:
|
||||||
|
module: generic
|
||||||
|
options:
|
||||||
|
max_units: 50
|
||||||
|
artifacts:
|
||||||
|
spells:
|
||||||
|
extract: dnd/spells
|
||||||
|
```
|
||||||
|
|
||||||
|
The maintained fixture is [examples/dnd-spells.config.yml](../examples/dnd-spells.config.yml).
|
||||||
|
|
||||||
|
## Top-Level Fields
|
||||||
|
|
||||||
|
- `version`: required. The only supported value is `1`.
|
||||||
|
- `llm_profiles`: optional map of LLM profile IDs to profile settings.
|
||||||
|
- `pipelines`: optional map of pipeline IDs to pipeline definitions.
|
||||||
|
- `concurrency`: optional global concurrency settings.
|
||||||
|
- `diagnostics`: optional diagnostics settings.
|
||||||
|
|
||||||
|
Unknown YAML fields are rejected.
|
||||||
|
|
||||||
|
## Defaults
|
||||||
|
|
||||||
|
Built-in defaults:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
llm_profiles:
|
||||||
|
default:
|
||||||
|
provider: openai-compatible
|
||||||
|
timeout: 600
|
||||||
|
max_retries: 3
|
||||||
|
max_concurrency: 1
|
||||||
|
concurrency:
|
||||||
|
total_llm: 1
|
||||||
|
diagnostics:
|
||||||
|
work_dir: /tmp/notarius
|
||||||
|
retention: auto
|
||||||
|
```
|
||||||
|
|
||||||
|
No pipelines are built in. A run requires a configured pipeline.
|
||||||
|
|
||||||
|
## LLM Profiles
|
||||||
|
|
||||||
|
Each `llm_profiles` entry may contain:
|
||||||
|
|
||||||
|
- `provider`: optional provider key. Empty means `openai-compatible`; any other
|
||||||
|
non-empty value must be `openai-compatible`.
|
||||||
|
- `base_url`: provider base URL. Required for actual LLM calls.
|
||||||
|
- `model`: provider model name. Required for actual LLM calls.
|
||||||
|
- `api_key_env`: environment variable name to read for the API key.
|
||||||
|
- `timeout`: request timeout as whole seconds or a Go-style duration string such
|
||||||
|
as `10m`.
|
||||||
|
- `max_retries`: retry count for provider calls. Must be zero or greater.
|
||||||
|
- `max_concurrency`: per-profile LLM concurrency. Must be zero or greater; when
|
||||||
|
zero, Notarius uses `concurrency.total_llm`.
|
||||||
|
|
||||||
|
Raw API keys are not accepted as file config fields. Use `api_key_env` or an
|
||||||
|
environment override.
|
||||||
|
|
||||||
|
## Environment Overrides
|
||||||
|
|
||||||
|
These environment variables are applied after the config file:
|
||||||
|
|
||||||
|
- `NOTARIUS_CONFIG`: config discovery path.
|
||||||
|
- `NOTARIUS_LLM_DEFAULT_API_KEY`: API key for the `default` LLM profile.
|
||||||
|
- `NOTARIUS_LLM_DEFAULT_BASE_URL`: base URL for the `default` LLM profile.
|
||||||
|
- `NOTARIUS_LLM_DEFAULT_MODEL`: model for the `default` LLM profile.
|
||||||
|
- `NOTARIUS_LLM_DEFAULT_TIMEOUT_SECONDS`: integer timeout seconds for the
|
||||||
|
`default` LLM profile.
|
||||||
|
- `NOTARIUS_LLM_DEFAULT_MAX_RETRIES`: integer retry count for the `default` LLM
|
||||||
|
profile.
|
||||||
|
- `NOTARIUS_LLM_DEFAULT_MAX_CONCURRENCY`: integer max concurrency for the
|
||||||
|
`default` LLM profile.
|
||||||
|
- `NOTARIUS_TOTAL_LLM_CONCURRENCY`: integer global LLM concurrency.
|
||||||
|
- `NOTARIUS_WORK_DIR`: diagnostics work directory.
|
||||||
|
- `NOTARIUS_DIAGNOSTICS_RETENTION`: diagnostics retention mode.
|
||||||
|
|
||||||
|
Integer environment values must parse as base-10 integers.
|
||||||
|
|
||||||
|
## Pipelines
|
||||||
|
|
||||||
|
A pipeline defines the fixed Notarius workflow:
|
||||||
|
|
||||||
|
```text
|
||||||
|
input -> chunk -> extract -> merge -> normalize -> output
|
||||||
|
```
|
||||||
|
|
||||||
|
Pipeline fields:
|
||||||
|
|
||||||
|
- `input`: required module binding.
|
||||||
|
- `chunk`: optional module binding. Default module is `generic`.
|
||||||
|
- `artifacts`: required for pipeline resolution. It maps artifact lane IDs to
|
||||||
|
lane definitions.
|
||||||
|
- `output`: optional module binding. Default module is `json`.
|
||||||
|
|
||||||
|
Artifact lane fields:
|
||||||
|
|
||||||
|
- `extract`: required module binding.
|
||||||
|
- `merge`: optional module binding. Default module is `appendorder`.
|
||||||
|
- `normalize`: optional module binding. Default module is `noop`.
|
||||||
|
- `validators`: optional list of module bindings. The production CLI currently
|
||||||
|
does not register validator modules.
|
||||||
|
|
||||||
|
`notarius run` and `notarius config validate --pipeline` resolve the pipeline
|
||||||
|
against the production module catalog and fail fast for unknown or incompatible
|
||||||
|
module keys.
|
||||||
|
|
||||||
|
## Module Bindings
|
||||||
|
|
||||||
|
Every module binding may use shorthand:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
input: seriatim
|
||||||
|
```
|
||||||
|
|
||||||
|
or object form:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
chunk:
|
||||||
|
module: generic
|
||||||
|
llm_profile: default
|
||||||
|
options:
|
||||||
|
max_units: 50
|
||||||
|
```
|
||||||
|
|
||||||
|
Binding fields:
|
||||||
|
|
||||||
|
- `module`: module key.
|
||||||
|
- `llm_profile`: optional LLM profile ID. Empty means `default`.
|
||||||
|
- `options`: optional module-specific settings.
|
||||||
|
|
||||||
|
The `--llm-profile` run flag overrides every effective module binding to use
|
||||||
|
one configured profile.
|
||||||
|
|
||||||
|
## Implemented Production Modules
|
||||||
|
|
||||||
|
| Slot | Key | Notes |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| input | `seriatim` | Reads Seriatim minimal transcript JSON. |
|
||||||
|
| chunk | `generic` | Splits source units into ordered chunks. |
|
||||||
|
| extract | `dnd/spells` | Extracts `dnd.spell_cast` artifacts. |
|
||||||
|
| merge | `appendorder` | Keeps candidates in append order. |
|
||||||
|
| normalize | `noop` | Passes merged artifacts through unchanged. |
|
||||||
|
| output | `json` | Produces JSON output files. |
|
||||||
|
|
||||||
|
The `generic` chunker accepts:
|
||||||
|
|
||||||
|
- `max_units`: positive integer, default `50`;
|
||||||
|
- `overlap_units`: non-negative integer, default `0`, and must be less than
|
||||||
|
`max_units`.
|
||||||
|
|
||||||
|
## Diagnostics
|
||||||
|
|
||||||
|
`diagnostics` fields:
|
||||||
|
|
||||||
|
- `work_dir`: directory for per-run diagnostics. Default: `/tmp/notarius`.
|
||||||
|
- `retention`: `auto`, `always`, or `never`. Empty uses `auto`.
|
||||||
|
|
||||||
|
`auto` retains diagnostics for failed runs and successful runs with warnings.
|
||||||
|
`always` retains diagnostics for every run. `never` removes diagnostics for
|
||||||
|
successful runs without regard to warnings; failed runs are retained.
|
||||||
|
|
||||||
|
The `--diagnostics-dir` run flag overrides `diagnostics.work_dir` for that
|
||||||
|
invocation.
|
||||||
|
|
||||||
|
## Validation
|
||||||
|
|
||||||
|
Configuration validation checks:
|
||||||
|
|
||||||
|
- supported config version and known YAML fields;
|
||||||
|
- non-empty, non-duplicated IDs after trimming;
|
||||||
|
- supported LLM provider and non-negative profile limits;
|
||||||
|
- positive global LLM concurrency;
|
||||||
|
- supported diagnostics retention and non-empty work directory;
|
||||||
|
- module binding LLM profiles refer to configured profiles.
|
||||||
|
|
||||||
|
Pipeline resolution additionally checks:
|
||||||
|
|
||||||
|
- the pipeline ID exists;
|
||||||
|
- at least one artifact lane is declared and selected;
|
||||||
|
- selected lanes exist when `--only` is used;
|
||||||
|
- required module keys are present;
|
||||||
|
- module keys are registered for the expected slot;
|
||||||
|
- module capability requirements are satisfied.
|
||||||
152
docs/integrations/dnd-spell-artifacts.md
Normal file
152
docs/integrations/dnd-spell-artifacts.md
Normal file
@@ -0,0 +1,152 @@
|
|||||||
|
# D&D Spell-Cast Artifacts
|
||||||
|
|
||||||
|
This document is the durable artifact contract for approved
|
||||||
|
`dnd.spell_cast` artifacts produced by the implemented `dnd/spells` extractor.
|
||||||
|
|
||||||
|
## Artifact Identity
|
||||||
|
|
||||||
|
- Extractor key: `dnd/spells`
|
||||||
|
- Artifact type: `dnd.spell_cast`
|
||||||
|
- Schema version: `v1`
|
||||||
|
- Prompt ID: `dnd.spells`
|
||||||
|
- Response schema key: `dnd_spells`
|
||||||
|
- Response schema ID: `notarius.dnd.spells`
|
||||||
|
- Response schema name: `notarius_dnd_spells_v1`
|
||||||
|
|
||||||
|
The extractor requires source chunks and transcript source capability. It
|
||||||
|
returns generic artifact candidates that are serialized by the JSON output
|
||||||
|
module.
|
||||||
|
|
||||||
|
## Artifact Envelope
|
||||||
|
|
||||||
|
Approved artifacts use the generic artifact envelope documented in
|
||||||
|
[JSON Output](json-output.md#artifact-files):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"extractor_key": "dnd/spells",
|
||||||
|
"artifact_type": "dnd.spell_cast",
|
||||||
|
"schema_version": "v1",
|
||||||
|
"payload": {
|
||||||
|
"caster": "Aria",
|
||||||
|
"spell": "Cure Wounds",
|
||||||
|
"effect": "heals an injured ally",
|
||||||
|
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds."
|
||||||
|
},
|
||||||
|
"source_refs": [
|
||||||
|
{
|
||||||
|
"source_id": "session-alpha",
|
||||||
|
"start_unit_id": "seg-001",
|
||||||
|
"end_unit_id": "seg-001"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Payload Fields
|
||||||
|
|
||||||
|
The `payload` object contains:
|
||||||
|
|
||||||
|
- `caster`: in-world character or creature casting the spell;
|
||||||
|
- `spell`: spell name;
|
||||||
|
- `effect`: concise spell effect in the scene;
|
||||||
|
- `narrative_description`: short description of the spell cast in context.
|
||||||
|
|
||||||
|
All payload fields are strings and must be non-empty after trimming.
|
||||||
|
|
||||||
|
`caster` is the in-world caster, not the transcript speaker.
|
||||||
|
|
||||||
|
## Source References
|
||||||
|
|
||||||
|
Source references live on the artifact envelope as `source_refs`; they are not
|
||||||
|
duplicated inside the `payload`.
|
||||||
|
|
||||||
|
Each source reference uses the generic source-reference shape:
|
||||||
|
|
||||||
|
- `source_id`
|
||||||
|
- `start_unit_id`
|
||||||
|
- `end_unit_id`
|
||||||
|
|
||||||
|
Validation requires:
|
||||||
|
|
||||||
|
- at least one source reference;
|
||||||
|
- non-empty source ID and unit IDs;
|
||||||
|
- source ID matching the source document ID;
|
||||||
|
- start and end unit IDs existing in the source document;
|
||||||
|
- start unit appearing before or at the same position as end unit.
|
||||||
|
|
||||||
|
## Structured LLM Response Shape
|
||||||
|
|
||||||
|
The extractor asks the LLM for this top-level response shape:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"spell_casts": [
|
||||||
|
{
|
||||||
|
"caster": "Aria",
|
||||||
|
"spell": "Cure Wounds",
|
||||||
|
"effect": "heals an injured ally",
|
||||||
|
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds.",
|
||||||
|
"source_refs": [
|
||||||
|
{
|
||||||
|
"source_id": "session-alpha",
|
||||||
|
"start_unit_id": "seg-001",
|
||||||
|
"end_unit_id": "seg-001"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`spell_casts` must be present. It may be empty when no spell casts are found.
|
||||||
|
|
||||||
|
The response schema asset is embedded at
|
||||||
|
`internal/modules/extract/dnd/spells/assets/schemas/dnd_spells.v1.json`.
|
||||||
|
|
||||||
|
## Validators
|
||||||
|
|
||||||
|
The extractor supplies two deterministic validators by default:
|
||||||
|
|
||||||
|
- `dnd/spells/shape`
|
||||||
|
- `dnd/spells/source_refs`
|
||||||
|
|
||||||
|
Rejection reason codes:
|
||||||
|
|
||||||
|
- `invalid_payload`: payload JSON cannot be decoded as a spell-cast payload.
|
||||||
|
- `missing_required_field`: `caster`, `spell`, `effect`, or
|
||||||
|
`narrative_description` is blank.
|
||||||
|
- `missing_source_ref`: candidate has no source references.
|
||||||
|
- `invalid_source_ref`: at least one source reference fails generic source
|
||||||
|
reference validation.
|
||||||
|
|
||||||
|
Rejected candidates are written to `rejected.json` by the JSON output module.
|
||||||
|
|
||||||
|
## Manifest Metadata
|
||||||
|
|
||||||
|
The extractor adds prompt and response-schema provenance under the artifact lane
|
||||||
|
manifest metadata:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"metadata": {
|
||||||
|
"extractor": {
|
||||||
|
"prompt_id": "dnd.spells",
|
||||||
|
"prompt_version": "v1",
|
||||||
|
"prompt_sha256": "sha256:...",
|
||||||
|
"response_schema_key": "dnd_spells",
|
||||||
|
"response_schema_id": "notarius.dnd.spells",
|
||||||
|
"response_schema_name": "notarius_dnd_spells_v1",
|
||||||
|
"response_schema_version": "v1",
|
||||||
|
"response_schema_sha256": "sha256:..."
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Raw prompt and schema content are not included in manifest metadata.
|
||||||
|
|
||||||
|
## Compatibility Limit
|
||||||
|
|
||||||
|
This contract covers only `dnd.spell_cast` artifacts produced by the
|
||||||
|
implemented spell-cast extractor.
|
||||||
@@ -1,116 +0,0 @@
|
|||||||
# D&D Spell Cast Extraction
|
|
||||||
|
|
||||||
This document describes the D&D spell-cast extractor currently implemented in
|
|
||||||
Notarius.
|
|
||||||
|
|
||||||
## Module
|
|
||||||
|
|
||||||
- Module key: `dnd/spells`
|
|
||||||
- Artifact type: `dnd.spell_cast`
|
|
||||||
- Schema version: `v1`
|
|
||||||
- Prompt ID: `dnd.spells`
|
|
||||||
- Response schema key: `dnd_spells`
|
|
||||||
- Response schema ID: `notarius.dnd.spells`
|
|
||||||
- Response schema name: `notarius_dnd_spells_v1`
|
|
||||||
|
|
||||||
The module is an extract module. It reads a generic source chunk, renders the
|
|
||||||
`dnd.spells` prompt, calls the configured structured LLM client, and returns
|
|
||||||
spell-cast artifact candidates.
|
|
||||||
|
|
||||||
## Source Expectations
|
|
||||||
|
|
||||||
The extractor expects a generic `SourceDocument` and active source chunk. It
|
|
||||||
does not depend on concrete Seriatim package types.
|
|
||||||
|
|
||||||
Pipeline resolution must provide these capabilities before the extractor runs:
|
|
||||||
|
|
||||||
- `chunks`
|
|
||||||
- `source.transcript`
|
|
||||||
|
|
||||||
Source units may include transcript metadata such as speaker and timestamps.
|
|
||||||
That metadata is optional prompt context. It is not part of the durable spell
|
|
||||||
payload.
|
|
||||||
|
|
||||||
## Artifact Payload
|
|
||||||
|
|
||||||
Each approved artifact payload is a JSON object with these fields:
|
|
||||||
|
|
||||||
- `caster`: in-world character or creature casting the spell;
|
|
||||||
- `spell`: spell name;
|
|
||||||
- `effect`: concise spell effect in the scene;
|
|
||||||
- `narrative_description`: short description of the spell cast in context.
|
|
||||||
|
|
||||||
`caster` is not the table speaker. NPCs, monsters, and other DM-voiced
|
|
||||||
characters can be casters.
|
|
||||||
|
|
||||||
## Source References
|
|
||||||
|
|
||||||
The structured LLM response must include `source_refs` for each spell cast.
|
|
||||||
Each source reference uses the generic source-reference shape:
|
|
||||||
|
|
||||||
- `source_id`
|
|
||||||
- `start_unit_id`
|
|
||||||
- `end_unit_id`
|
|
||||||
|
|
||||||
The extractor copies those references into the generic artifact envelope
|
|
||||||
`source_refs` field. The durable `dnd.spell_cast` payload does not duplicate
|
|
||||||
source references.
|
|
||||||
|
|
||||||
Source-reference IDs must match the source document and source-unit IDs
|
|
||||||
exactly. The validator chain rejects unknown source IDs, unknown unit IDs, and
|
|
||||||
reversed unit ranges.
|
|
||||||
|
|
||||||
## Validators
|
|
||||||
|
|
||||||
The extractor provides these deterministic validators by default, in order:
|
|
||||||
|
|
||||||
- `dnd/spells/shape`
|
|
||||||
- `dnd/spells/source_refs`
|
|
||||||
|
|
||||||
`dnd/spells/shape` rejects:
|
|
||||||
|
|
||||||
- malformed JSON payloads with reason code `invalid_payload`;
|
|
||||||
- blank `caster`, `spell`, `effect`, or `narrative_description` fields with
|
|
||||||
reason code `missing_required_field`.
|
|
||||||
|
|
||||||
`dnd/spells/source_refs` rejects:
|
|
||||||
|
|
||||||
- candidates with no source references using reason code `missing_source_ref`;
|
|
||||||
- invalid source references using reason code `invalid_source_ref`.
|
|
||||||
|
|
||||||
The source-reference validator uses the core `source.ValidateRef` behavior, so
|
|
||||||
its rejection message includes the underlying source-reference validation
|
|
||||||
error.
|
|
||||||
|
|
||||||
## Capabilities
|
|
||||||
|
|
||||||
The module declares these required capabilities:
|
|
||||||
|
|
||||||
- `chunks`
|
|
||||||
- `source.transcript`
|
|
||||||
|
|
||||||
The module declares this provided capability:
|
|
||||||
|
|
||||||
- `dnd.spell_casts`
|
|
||||||
|
|
||||||
A pipeline artifact lane can reference the extractor with:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
artifacts:
|
|
||||||
spells:
|
|
||||||
extract: dnd/spells
|
|
||||||
merge: appendorder
|
|
||||||
normalize: noop
|
|
||||||
```
|
|
||||||
|
|
||||||
## Limits
|
|
||||||
|
|
||||||
The current implementation covers only D&D spell-cast extraction. It does not
|
|
||||||
yet implement:
|
|
||||||
|
|
||||||
- item extraction;
|
|
||||||
- NPC extraction;
|
|
||||||
- combat extraction;
|
|
||||||
- encounter extraction;
|
|
||||||
- broad D&D rules validation;
|
|
||||||
- a CLI `run` workflow.
|
|
||||||
192
docs/integrations/json-output.md
Normal file
192
docs/integrations/json-output.md
Normal file
@@ -0,0 +1,192 @@
|
|||||||
|
# JSON Output
|
||||||
|
|
||||||
|
This document is the durable JSON output file-format contract produced by the
|
||||||
|
implemented `json` output module and written by the CLI.
|
||||||
|
|
||||||
|
## Output Directory
|
||||||
|
|
||||||
|
The CLI writes logical output files under:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<output-root>/<run-id>/
|
||||||
|
```
|
||||||
|
|
||||||
|
The default output root is `./notarius-output`. Operational behavior is covered
|
||||||
|
in [Operations](../operations.md).
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
The `json` output module writes:
|
||||||
|
|
||||||
|
- `index.json`
|
||||||
|
- `manifest.json`
|
||||||
|
- `artifacts/<artifact-type>.json`, one file per approved artifact type
|
||||||
|
- `rejected.json`
|
||||||
|
- `warnings.json`
|
||||||
|
|
||||||
|
Files are pretty-printed JSON with a trailing newline.
|
||||||
|
|
||||||
|
## `index.json`
|
||||||
|
|
||||||
|
Shape:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"manifest_file": "manifest.json",
|
||||||
|
"artifact_files": [
|
||||||
|
{
|
||||||
|
"artifact_type": "dnd.spell_cast",
|
||||||
|
"file": "artifacts/dnd.spell_cast.json"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"rejected_file": "rejected.json",
|
||||||
|
"warnings_file": "warnings.json"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`artifact_files` is sorted by artifact type. It is empty when no artifacts are
|
||||||
|
approved.
|
||||||
|
|
||||||
|
## `manifest.json`
|
||||||
|
|
||||||
|
`manifest.json` contains a run manifest:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"run_id": "run-123",
|
||||||
|
"pipeline_id": "dnd-session",
|
||||||
|
"pipeline_digest": "sha256:...",
|
||||||
|
"input_module": "seriatim",
|
||||||
|
"chunker": "generic",
|
||||||
|
"source_digests": ["sha256:..."],
|
||||||
|
"extractors": ["dnd/spells"],
|
||||||
|
"merger": "appendorder",
|
||||||
|
"normalizer": "noop",
|
||||||
|
"output_encoder": "json",
|
||||||
|
"artifact_lanes": [
|
||||||
|
{
|
||||||
|
"id": "spells",
|
||||||
|
"extractor": "dnd/spells",
|
||||||
|
"merger": "appendorder",
|
||||||
|
"normalizer": "noop"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"llm_profiles": [
|
||||||
|
{
|
||||||
|
"id": "default",
|
||||||
|
"provider": "openai-compatible",
|
||||||
|
"model": "configured-model"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"validation_status": "approved",
|
||||||
|
"started_at": "2026-01-01T00:00:00Z",
|
||||||
|
"completed_at": "2026-01-01T00:00:01Z"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Fields with empty values may be omitted by JSON encoding.
|
||||||
|
|
||||||
|
`validation_status` is `approved` when no candidates were rejected and
|
||||||
|
`rejected` when one or more candidates were rejected.
|
||||||
|
|
||||||
|
## Artifact Files
|
||||||
|
|
||||||
|
Each artifact file has this shape:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"artifact_type": "dnd.spell_cast",
|
||||||
|
"artifacts": [
|
||||||
|
{
|
||||||
|
"extractor_key": "dnd/spells",
|
||||||
|
"artifact_type": "dnd.spell_cast",
|
||||||
|
"schema_version": "v1",
|
||||||
|
"payload": {},
|
||||||
|
"source_refs": [
|
||||||
|
{
|
||||||
|
"source_id": "session-alpha",
|
||||||
|
"start_unit_id": "seg-001",
|
||||||
|
"end_unit_id": "seg-001"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Artifact envelope fields:
|
||||||
|
|
||||||
|
- `extractor_key`: extractor module key.
|
||||||
|
- `artifact_type`: artifact type.
|
||||||
|
- `schema_version`: artifact schema version.
|
||||||
|
- `payload`: artifact-type-specific JSON payload.
|
||||||
|
- `source_refs`: optional generic source references.
|
||||||
|
- `metadata`: optional artifact metadata.
|
||||||
|
|
||||||
|
Artifact file names are produced by sanitizing the artifact type:
|
||||||
|
|
||||||
|
- characters outside `A-Z`, `a-z`, `0-9`, `.`, `_`, and `-` become `_`;
|
||||||
|
- repeated `..` sequences are replaced;
|
||||||
|
- leading and trailing `.`, `_`, and `-` are trimmed;
|
||||||
|
- empty sanitized names are rejected.
|
||||||
|
|
||||||
|
For current D&D spell-cast artifacts, the file is
|
||||||
|
`artifacts/dnd.spell_cast.json`.
|
||||||
|
|
||||||
|
## `rejected.json`
|
||||||
|
|
||||||
|
Shape:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"rejected": [
|
||||||
|
{
|
||||||
|
"candidate": {
|
||||||
|
"index": 0,
|
||||||
|
"extractor_key": "dnd/spells",
|
||||||
|
"artifact_type": "dnd.spell_cast",
|
||||||
|
"schema_version": "v1",
|
||||||
|
"payload": {},
|
||||||
|
"source_refs": []
|
||||||
|
},
|
||||||
|
"validator_name": "dnd/spells/source_refs",
|
||||||
|
"reason_code": "missing_source_ref",
|
||||||
|
"message": "spell cast candidate must include at least one source ref"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`rejected` is an empty array when no candidates are rejected.
|
||||||
|
|
||||||
|
## `warnings.json`
|
||||||
|
|
||||||
|
Shape:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"warnings": [
|
||||||
|
{
|
||||||
|
"scope": "output",
|
||||||
|
"reason_code": "example_warning",
|
||||||
|
"message": "warning message"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`warnings` is an empty array when no warnings are reported.
|
||||||
|
|
||||||
|
## Path Safety
|
||||||
|
|
||||||
|
The output module returns slash-separated logical paths. The CLI also validates
|
||||||
|
logical output names before writing:
|
||||||
|
|
||||||
|
- names must be non-empty;
|
||||||
|
- names must be relative;
|
||||||
|
- names must be clean;
|
||||||
|
- names must use `/`, not `\`;
|
||||||
|
- names must not contain `..`;
|
||||||
|
- resolved paths must stay under the run output directory.
|
||||||
|
|
||||||
|
Durable writes are atomic per file.
|
||||||
128
docs/integrations/openai-compatible.md
Normal file
128
docs/integrations/openai-compatible.md
Normal file
@@ -0,0 +1,128 @@
|
|||||||
|
# OpenAI-Compatible Structured Output
|
||||||
|
|
||||||
|
This document describes the external LLM provider contract implemented by the
|
||||||
|
production Notarius LLM client.
|
||||||
|
|
||||||
|
## Provider
|
||||||
|
|
||||||
|
- Provider key: `openai-compatible`
|
||||||
|
- HTTP method: `POST`
|
||||||
|
- Endpoint: `<base_url>/chat/completions`
|
||||||
|
- Request body: JSON
|
||||||
|
- Response mode: chat completions with structured JSON schema output
|
||||||
|
|
||||||
|
`base_url` is trimmed of trailing slashes before `/chat/completions` is
|
||||||
|
appended. Configure provider settings in [Configuration](../config.md).
|
||||||
|
|
||||||
|
## Request
|
||||||
|
|
||||||
|
The client sends a JSON object with:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"model": "configured-model",
|
||||||
|
"messages": [
|
||||||
|
{
|
||||||
|
"role": "system",
|
||||||
|
"content": "..."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "..."
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"response_format": {
|
||||||
|
"type": "json_schema",
|
||||||
|
"json_schema": {
|
||||||
|
"name": "schema_name",
|
||||||
|
"strict": true,
|
||||||
|
"schema": {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Implemented request behavior:
|
||||||
|
|
||||||
|
- `model` comes from the structured completion request when set, otherwise from
|
||||||
|
the configured LLM profile.
|
||||||
|
- `messages` must be non-empty; each role and content must be non-empty after
|
||||||
|
trimming.
|
||||||
|
- `response_format.type` is always `json_schema`.
|
||||||
|
- `response_format.json_schema.strict` is always `true`.
|
||||||
|
- `response_format.json_schema.name` and `schema` come from the extractor or
|
||||||
|
validator making the call.
|
||||||
|
|
||||||
|
If an API key is configured, the client sends:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Authorization: Bearer <api-key>
|
||||||
|
```
|
||||||
|
|
||||||
|
The client always sends `Content-Type: application/json`.
|
||||||
|
|
||||||
|
## Response
|
||||||
|
|
||||||
|
The client expects a JSON response with at least one choice:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"model": "provider-model",
|
||||||
|
"choices": [
|
||||||
|
{
|
||||||
|
"message": {
|
||||||
|
"content": "{\"field\":\"value\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"usage": {
|
||||||
|
"prompt_tokens": 10,
|
||||||
|
"completion_tokens": 5,
|
||||||
|
"total_tokens": 15
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`choices[0].message.content` may be either:
|
||||||
|
|
||||||
|
- a JSON string whose contents are valid JSON; or
|
||||||
|
- raw JSON.
|
||||||
|
|
||||||
|
The decoded content is unmarshaled into the caller-provided structured output
|
||||||
|
target. If `usage` is present, prompt, completion, and total token counts are
|
||||||
|
copied into the completion response.
|
||||||
|
|
||||||
|
## Errors And Retries
|
||||||
|
|
||||||
|
The client validates base URL, model, response schema name, response schema
|
||||||
|
JSON, messages, and output target before or during the call.
|
||||||
|
|
||||||
|
Retryable failures:
|
||||||
|
|
||||||
|
- HTTP request failure;
|
||||||
|
- response body read failure;
|
||||||
|
- HTTP `429`;
|
||||||
|
- HTTP `5xx`;
|
||||||
|
- malformed provider response envelope;
|
||||||
|
- missing choices;
|
||||||
|
- missing, empty, or invalid assistant JSON content;
|
||||||
|
- structured-output decode failure.
|
||||||
|
|
||||||
|
Non-retryable provider status codes include non-`429` `4xx` responses.
|
||||||
|
|
||||||
|
Provider error bodies are parsed for `error.message` or `message` when present.
|
||||||
|
Configured API key values and bearer-token values are redacted from returned
|
||||||
|
provider errors.
|
||||||
|
|
||||||
|
## Timeouts And Concurrency
|
||||||
|
|
||||||
|
The configured profile timeout is applied per provider request when greater
|
||||||
|
than zero. Context cancellation is respected.
|
||||||
|
|
||||||
|
The production CLI wraps the provider client with the LLM scheduler. Effective
|
||||||
|
concurrency is described in [LLM runtime internals](../internal/llm.md).
|
||||||
|
|
||||||
|
## Limits
|
||||||
|
|
||||||
|
This contract documents only the fields the implemented client sends and reads.
|
||||||
|
Provider-specific extensions are ignored unless they affect those fields.
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
# Seriatim Minimal Transcript JSON
|
# Seriatim Minimal Transcript JSON
|
||||||
|
|
||||||
This document describes the Seriatim input format currently accepted by the
|
This document is the external input contract for the implemented `seriatim`
|
||||||
`seriatim` input adapter.
|
input adapter.
|
||||||
|
|
||||||
## Adapter
|
## Adapter
|
||||||
|
|
||||||
@@ -10,33 +10,39 @@ This document describes the Seriatim input format currently accepted by the
|
|||||||
- Unit kind: `transcript_segment`
|
- Unit kind: `transcript_segment`
|
||||||
- Source format: `application/vnd.seriatim.minimal+json`
|
- Source format: `application/vnd.seriatim.minimal+json`
|
||||||
|
|
||||||
The adapter parses raw Seriatim JSON into a generic `SourceDocument`. It does
|
The adapter parses raw Seriatim JSON into a generic source document. It owns
|
||||||
not add transcript-specific fields to core source or runner contracts.
|
transcript-specific JSON parsing and metadata mapping; core source and pipeline
|
||||||
|
code stay source-format agnostic.
|
||||||
|
|
||||||
## Accepted Shape
|
## Accepted Shape
|
||||||
|
|
||||||
The input must be a JSON object with top-level `metadata` and `segments` fields:
|
The input must be one JSON object with top-level `metadata` and `segments`
|
||||||
|
fields:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"metadata": {
|
"metadata": {
|
||||||
"id": "session-alpha",
|
"id": "session-alpha",
|
||||||
"title": "Synthetic session transcript"
|
"title": "Synthetic D&D spell session"
|
||||||
},
|
},
|
||||||
"segments": [
|
"segments": [
|
||||||
{
|
{
|
||||||
"id": "seg-001",
|
"id": "seg-001",
|
||||||
"start": 0,
|
"start": 0,
|
||||||
"end": 4.5,
|
"end": 4,
|
||||||
"speaker": "Narrator",
|
"speaker": "Aria",
|
||||||
"text": "The stone door opens."
|
"text": "Aria raises her holy symbol and casts Cure Wounds."
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
Extra compatible fields are ignored. Multiple top-level JSON values are
|
The maintained example is
|
||||||
rejected.
|
[examples/seriatim-minimal-transcript.json](../../examples/seriatim-minimal-transcript.json).
|
||||||
|
|
||||||
|
Top-level metadata entries are preserved. Other segment fields are ignored.
|
||||||
|
|
||||||
|
Multiple top-level JSON values are rejected.
|
||||||
|
|
||||||
## Validation
|
## Validation
|
||||||
|
|
||||||
@@ -44,8 +50,11 @@ The adapter rejects:
|
|||||||
|
|
||||||
- empty raw input;
|
- empty raw input;
|
||||||
- malformed JSON;
|
- malformed JSON;
|
||||||
|
- top-level JSON that is not an object;
|
||||||
- missing, null, or non-object `metadata`;
|
- missing, null, or non-object `metadata`;
|
||||||
- missing, null, non-array, or empty `segments`;
|
- missing, null, non-array, or empty `segments`;
|
||||||
|
- segment values that are not objects;
|
||||||
|
- non-string `id`, `speaker`, or `text`;
|
||||||
- empty segment IDs;
|
- empty segment IDs;
|
||||||
- segment IDs with leading or trailing whitespace;
|
- segment IDs with leading or trailing whitespace;
|
||||||
- duplicate segment IDs;
|
- duplicate segment IDs;
|
||||||
@@ -55,19 +64,16 @@ The adapter rejects:
|
|||||||
- `end` values before `start`;
|
- `end` values before `start`;
|
||||||
- missing or empty `text`.
|
- missing or empty `text`.
|
||||||
|
|
||||||
Segment text may keep leading or trailing whitespace, but it must not be empty
|
Segment text is preserved as provided, but it must not be empty after trimming.
|
||||||
after trimming.
|
|
||||||
|
|
||||||
## Source Mapping
|
## Source Mapping
|
||||||
|
|
||||||
The adapter maps Seriatim input into the source model as follows:
|
The adapter maps input to `SourceDocument`:
|
||||||
|
|
||||||
- top-level `metadata` becomes `SourceDocument.Metadata`;
|
- `metadata` becomes `SourceDocument.Metadata`;
|
||||||
- `SourceDocument.Digest` is `sha256:<hex>` of the exact raw input bytes;
|
- `SourceDocument.Kind` is `transcript`;
|
||||||
- `segment.id` becomes `SourceUnit.ID`;
|
- `SourceDocument.Format` is `application/vnd.seriatim.minimal+json`;
|
||||||
- `segment.text` becomes `SourceUnit.Text`;
|
- `SourceDocument.Digest` is `sha256:<hex>` of the exact raw input bytes.
|
||||||
- each source unit has kind `transcript_segment`;
|
|
||||||
- segment `speaker`, `start`, and `end` are stored in source-unit metadata.
|
|
||||||
|
|
||||||
`SourceDocument.ID` is selected in this order:
|
`SourceDocument.ID` is selected in this order:
|
||||||
|
|
||||||
@@ -76,6 +82,13 @@ The adapter maps Seriatim input into the source model as follows:
|
|||||||
3. `metadata.source_id`, when it is a non-empty string after trimming;
|
3. `metadata.source_id`, when it is a non-empty string after trimming;
|
||||||
4. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
|
4. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
|
||||||
|
|
||||||
|
Each segment becomes one `SourceUnit`:
|
||||||
|
|
||||||
|
- `segment.id` becomes `SourceUnit.ID`;
|
||||||
|
- `segment.text` becomes `SourceUnit.Text`;
|
||||||
|
- `SourceUnit.Kind` is `transcript_segment`;
|
||||||
|
- `speaker`, `start`, and `end` are stored in source-unit metadata.
|
||||||
|
|
||||||
## Metadata Keys
|
## Metadata Keys
|
||||||
|
|
||||||
Seriatim unit metadata uses these keys:
|
Seriatim unit metadata uses these keys:
|
||||||
@@ -84,18 +97,17 @@ Seriatim unit metadata uses these keys:
|
|||||||
- `start`: `json.Number` start value;
|
- `start`: `json.Number` start value;
|
||||||
- `end`: `json.Number` end value.
|
- `end`: `json.Number` end value.
|
||||||
|
|
||||||
The `internal/modules/input/seriatim` package provides typed accessors for
|
The `internal/modules/input/seriatim` package exposes typed accessors for these
|
||||||
these metadata values.
|
values.
|
||||||
|
|
||||||
## Capabilities
|
## Capabilities
|
||||||
|
|
||||||
The module declares these provided capabilities for pipeline validation:
|
The module declares these provided capabilities:
|
||||||
|
|
||||||
- `source.transcript`
|
- `source.transcript`
|
||||||
- `transcript.speaker`
|
- `transcript.speaker`
|
||||||
- `transcript.timestamps`
|
- `transcript.timestamps`
|
||||||
|
|
||||||
## Limits
|
## Compatibility Limit
|
||||||
|
|
||||||
Only the Seriatim minimal transcript shape described here is supported. Broader
|
This contract covers only the minimal transcript JSON shape described here.
|
||||||
Seriatim schema variants are not currently accepted as a compatibility contract.
|
|
||||||
|
|||||||
88
docs/internal/diagnostics.md
Normal file
88
docs/internal/diagnostics.md
Normal file
@@ -0,0 +1,88 @@
|
|||||||
|
# Diagnostics Internals
|
||||||
|
|
||||||
|
Diagnostics internals live in `internal/core/diagnostics`. Operator-facing run
|
||||||
|
behavior is documented in [Operations](../operations.md).
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Diagnostics provide local inspection artifacts for a run without becoming the
|
||||||
|
durable output contract. Durable user output is produced by output modules and
|
||||||
|
written by the CLI.
|
||||||
|
|
||||||
|
Diagnostics must not expose secrets.
|
||||||
|
|
||||||
|
## Run Directory
|
||||||
|
|
||||||
|
`NewRunDirectory(workDir, retention)` creates:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<workDir>/run-<unix-nanoseconds>/
|
||||||
|
```
|
||||||
|
|
||||||
|
If `workDir` is empty, it defaults to `/tmp/notarius`. Empty retention defaults
|
||||||
|
to `auto`.
|
||||||
|
|
||||||
|
The writer makes the work directory if needed, then attempts to create a unique
|
||||||
|
run directory. It retries run ID creation a bounded number of times if a
|
||||||
|
collision occurs.
|
||||||
|
|
||||||
|
## Artifact Writers
|
||||||
|
|
||||||
|
Implemented artifact names:
|
||||||
|
|
||||||
|
- `invocation.json`
|
||||||
|
- `effective-config.json`
|
||||||
|
- `resolved-pipeline.json`
|
||||||
|
- `source-document.json`
|
||||||
|
- `run-manifest.json`
|
||||||
|
- `run-report.json`
|
||||||
|
- `warnings.json`
|
||||||
|
- `error.log`
|
||||||
|
|
||||||
|
JSON artifacts are encoded with indentation and a trailing newline. Writes are
|
||||||
|
atomic through a temporary file in the target directory followed by rename.
|
||||||
|
|
||||||
|
Artifact names must be single relative file names. Absolute paths, path
|
||||||
|
separators, and names resolving outside the run directory are rejected.
|
||||||
|
|
||||||
|
## Redacted Effective Config
|
||||||
|
|
||||||
|
Diagnostics writers accept payloads that implement
|
||||||
|
`RedactedDiagnosticsPayload`. `internal/core/config` uses this to redact API
|
||||||
|
keys in effective config diagnostics while preserving resolved pipeline context.
|
||||||
|
|
||||||
|
The redaction path clones config data before replacing secret values.
|
||||||
|
|
||||||
|
## Retention
|
||||||
|
|
||||||
|
Retention is decided by `ShouldRetainRunDirectory`.
|
||||||
|
|
||||||
|
- Failed runs are always retained.
|
||||||
|
- `always` retains successful runs.
|
||||||
|
- `never` removes successful runs.
|
||||||
|
- `auto` retains successful runs only when warnings exist.
|
||||||
|
- Unknown retention values are treated as retain by the retention decision, but
|
||||||
|
config validation rejects unsupported values before normal runs.
|
||||||
|
|
||||||
|
`ApplyRetention` removes only the specific run directory.
|
||||||
|
|
||||||
|
## CLI Failure Behavior
|
||||||
|
|
||||||
|
The CLI creates the diagnostics run directory after config loading and before
|
||||||
|
pipeline resolution. Failures before that point do not have diagnostics.
|
||||||
|
|
||||||
|
After diagnostics creation, run failures call `WriteErrorLog` and apply
|
||||||
|
retention with `RunSucceeded: false`, so the run directory remains available.
|
||||||
|
|
||||||
|
When the pipeline returns a partial manifest on failure, the CLI writes that
|
||||||
|
manifest before logging the failure.
|
||||||
|
|
||||||
|
## Invariants
|
||||||
|
|
||||||
|
- Diagnostics paths must be narrow and run-directory scoped.
|
||||||
|
- Writes should be atomic where practical.
|
||||||
|
- Secrets must be redacted.
|
||||||
|
- Diagnostics write failures are command failures because they can hide the
|
||||||
|
information needed for recovery.
|
||||||
|
- Durable output file contracts belong to output modules and integration docs,
|
||||||
|
not to diagnostics.
|
||||||
116
docs/internal/llm.md
Normal file
116
docs/internal/llm.md
Normal file
@@ -0,0 +1,116 @@
|
|||||||
|
# LLM Runtime
|
||||||
|
|
||||||
|
The implemented LLM runtime lives in `internal/framework/llm`. It provides
|
||||||
|
transport-neutral structured completion contracts, an OpenAI-compatible HTTP
|
||||||
|
adapter, concurrency scheduling, schema registry helpers, retry behavior, and
|
||||||
|
secret redaction.
|
||||||
|
|
||||||
|
## Contract
|
||||||
|
|
||||||
|
Modules depend on `contracts.StructuredLLMClient`:
|
||||||
|
|
||||||
|
```go
|
||||||
|
CompleteStructured(ctx, request, out) (response, error)
|
||||||
|
```
|
||||||
|
|
||||||
|
The request contains messages, optional model override, response schema name,
|
||||||
|
and response schema JSON. The caller supplies a pointer target for decoded
|
||||||
|
structured output.
|
||||||
|
|
||||||
|
Extractors own prompts and schemas. Provider adapters should not contain
|
||||||
|
domain-specific prompt logic.
|
||||||
|
|
||||||
|
## Production Client Construction
|
||||||
|
|
||||||
|
`internal/cli` builds the production LLM client from the effective config:
|
||||||
|
|
||||||
|
1. find the effective LLM profile;
|
||||||
|
2. build `OpenAICompatibleClientConfig`;
|
||||||
|
3. create an OpenAI-compatible client;
|
||||||
|
4. create a scheduler from profile or global concurrency;
|
||||||
|
5. wrap the client with `NewScheduledClient`;
|
||||||
|
6. return non-secret LLM profile manifest metadata.
|
||||||
|
|
||||||
|
The current run command requires exactly one distinct effective LLM profile for
|
||||||
|
the resolved pipeline.
|
||||||
|
|
||||||
|
## OpenAI-Compatible Adapter
|
||||||
|
|
||||||
|
`OpenAICompatibleClient` posts JSON to:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<base_url>/chat/completions
|
||||||
|
```
|
||||||
|
|
||||||
|
It sends:
|
||||||
|
|
||||||
|
- `model`
|
||||||
|
- `messages`
|
||||||
|
- `response_format.type = "json_schema"`
|
||||||
|
- `response_format.json_schema.name`
|
||||||
|
- `response_format.json_schema.strict = true`
|
||||||
|
- `response_format.json_schema.schema`
|
||||||
|
|
||||||
|
If an API key is configured, the adapter sends an `Authorization: Bearer ...`
|
||||||
|
header.
|
||||||
|
|
||||||
|
The adapter accepts assistant content either as a JSON string containing JSON or
|
||||||
|
as raw JSON content. It then unmarshals that content into the caller-provided
|
||||||
|
target.
|
||||||
|
|
||||||
|
External wire-contract details belong in the
|
||||||
|
[OpenAI-compatible integration doc](../integrations/openai-compatible.md).
|
||||||
|
|
||||||
|
## Retries And Timeouts
|
||||||
|
|
||||||
|
The adapter retries:
|
||||||
|
|
||||||
|
- provider request failures;
|
||||||
|
- response read failures;
|
||||||
|
- HTTP `429`;
|
||||||
|
- HTTP `5xx`;
|
||||||
|
- malformed provider envelopes;
|
||||||
|
- malformed assistant JSON;
|
||||||
|
- structured-output decode failures.
|
||||||
|
|
||||||
|
Non-retryable `4xx` responses are returned without retry. Request timeout comes
|
||||||
|
from the effective LLM profile. Context cancellation is respected.
|
||||||
|
|
||||||
|
## Scheduler
|
||||||
|
|
||||||
|
`Scheduler` bounds concurrent provider calls. It tracks in-flight calls and a
|
||||||
|
FIFO queue of waiters. Cancellation removes queued waiters or releases granted
|
||||||
|
permits.
|
||||||
|
|
||||||
|
`NewScheduledClient` wraps any structured LLM client and runs each completion
|
||||||
|
inside the scheduler.
|
||||||
|
|
||||||
|
Effective concurrency is:
|
||||||
|
|
||||||
|
1. `llm_profiles.<id>.max_concurrency`, when greater than zero;
|
||||||
|
2. `concurrency.total_llm`, when greater than zero;
|
||||||
|
3. `1`.
|
||||||
|
|
||||||
|
## Schema Registry
|
||||||
|
|
||||||
|
The framework schema registry embeds generic test schemas. It also exposes
|
||||||
|
helpers for caller-owned schemas:
|
||||||
|
|
||||||
|
- `LoadResponseSchema`
|
||||||
|
- `LookupResponseSchema`
|
||||||
|
- `MustLookupResponseSchema`
|
||||||
|
- `ResponseSchema.DiagnosticsMap`
|
||||||
|
|
||||||
|
`DiagnosticsMap` omits raw schema content and includes metadata such as key,
|
||||||
|
ID, version, name, and SHA-256.
|
||||||
|
|
||||||
|
The D&D spell extractor owns and loads its own embedded response schema.
|
||||||
|
|
||||||
|
## Secret Redaction
|
||||||
|
|
||||||
|
Provider errors are passed through `ErrorWithSecretsRedacted` with the API key
|
||||||
|
and bearer-token value. Config diagnostics use redacted effective config
|
||||||
|
payloads.
|
||||||
|
|
||||||
|
Do not add raw provider request bodies, response bodies, API keys, or prompt
|
||||||
|
payloads to diagnostics by default.
|
||||||
165
docs/internal/modules.md
Normal file
165
docs/internal/modules.md
Normal file
@@ -0,0 +1,165 @@
|
|||||||
|
# Modules
|
||||||
|
|
||||||
|
Production modules live under `internal/modules`. Each module implements one
|
||||||
|
contract from `internal/framework/contracts`, exposes a `ModuleSpec`, and
|
||||||
|
registers itself with the matching pipeline registry.
|
||||||
|
|
||||||
|
The CLI production catalog currently registers only the modules listed here.
|
||||||
|
|
||||||
|
## Contract Pattern
|
||||||
|
|
||||||
|
A production module package should provide:
|
||||||
|
|
||||||
|
- a stable module key;
|
||||||
|
- a constructor such as `New`;
|
||||||
|
- the relevant contract implementation;
|
||||||
|
- `ModuleSpec`;
|
||||||
|
- `Register`;
|
||||||
|
- focused tests for registration, options, contract behavior, and errors.
|
||||||
|
|
||||||
|
Module specs should describe capabilities accurately. Resolution uses specs to
|
||||||
|
reject incompatible pipelines before execution.
|
||||||
|
|
||||||
|
## `seriatim` Input
|
||||||
|
|
||||||
|
Package: `internal/modules/input/seriatim`
|
||||||
|
|
||||||
|
The `seriatim` adapter parses Seriatim minimal transcript JSON into a generic
|
||||||
|
source document. It owns transcript JSON details, source ID selection, source
|
||||||
|
digest creation, transcript segment validation, and segment metadata mapping.
|
||||||
|
|
||||||
|
Provides:
|
||||||
|
|
||||||
|
- `source.transcript`
|
||||||
|
- `transcript.speaker`
|
||||||
|
- `transcript.timestamps`
|
||||||
|
|
||||||
|
External JSON shape belongs in the Seriatim integration doc.
|
||||||
|
|
||||||
|
## `generic` Chunker
|
||||||
|
|
||||||
|
Package: `internal/modules/chunk/generic`
|
||||||
|
|
||||||
|
The `generic` chunker splits source units into ordered chunks. It validates the
|
||||||
|
source document, clones source units, assigns chunk IDs such as `chunk-000001`,
|
||||||
|
and records chunk metadata for start unit, end unit, and unit count.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
|
||||||
|
- `max_units`: positive integer, default `50`;
|
||||||
|
- `overlap_units`: non-negative integer, default `0`, and less than
|
||||||
|
`max_units`.
|
||||||
|
|
||||||
|
Provides:
|
||||||
|
|
||||||
|
- `chunks`
|
||||||
|
|
||||||
|
## `dnd/spells` Extractor
|
||||||
|
|
||||||
|
Package: `internal/modules/extract/dnd/spells`
|
||||||
|
|
||||||
|
The `dnd/spells` extractor owns D&D spell-cast artifact semantics. It renders
|
||||||
|
embedded prompts, loads the embedded structured response schema, calls the
|
||||||
|
structured LLM client, converts spell-cast responses into artifact candidates,
|
||||||
|
and supplies deterministic validators.
|
||||||
|
|
||||||
|
Requires:
|
||||||
|
|
||||||
|
- `chunks`
|
||||||
|
- `source.transcript`
|
||||||
|
|
||||||
|
Provides:
|
||||||
|
|
||||||
|
- `dnd.spell_casts`
|
||||||
|
|
||||||
|
Artifact type and schema version:
|
||||||
|
|
||||||
|
- artifact type: `dnd.spell_cast`
|
||||||
|
- schema version: `v1`
|
||||||
|
|
||||||
|
The extractor adds prompt and response-schema provenance to lane manifest
|
||||||
|
metadata. Durable artifact payload details belong in the
|
||||||
|
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
|
||||||
|
|
||||||
|
## D&D Spell Validators
|
||||||
|
|
||||||
|
The spell extractor returns two built-in validators:
|
||||||
|
|
||||||
|
- `dnd/spells/shape`: rejects malformed payloads and missing required fields.
|
||||||
|
- `dnd/spells/source_refs`: rejects candidates without valid source references.
|
||||||
|
|
||||||
|
Reason codes include:
|
||||||
|
|
||||||
|
- `invalid_payload`
|
||||||
|
- `missing_required_field`
|
||||||
|
- `missing_source_ref`
|
||||||
|
- `invalid_source_ref`
|
||||||
|
|
||||||
|
These validators are supplied by the extractor when no validators are configured
|
||||||
|
for the lane.
|
||||||
|
|
||||||
|
## `appendorder` Merger
|
||||||
|
|
||||||
|
Package: `internal/modules/merge/appendorder`
|
||||||
|
|
||||||
|
The `appendorder` merger clones and appends candidates in chunk order. It does
|
||||||
|
not deduplicate or reconcile candidates.
|
||||||
|
|
||||||
|
Provides:
|
||||||
|
|
||||||
|
- `merged`
|
||||||
|
|
||||||
|
## `noop` Normalizer
|
||||||
|
|
||||||
|
Package: `internal/modules/normalize/noop`
|
||||||
|
|
||||||
|
The `noop` normalizer clones merged candidates and returns them unchanged.
|
||||||
|
|
||||||
|
Requires:
|
||||||
|
|
||||||
|
- `merged`
|
||||||
|
|
||||||
|
Provides:
|
||||||
|
|
||||||
|
- `normalized`
|
||||||
|
|
||||||
|
## `json` Output
|
||||||
|
|
||||||
|
Package: `internal/modules/output/json`
|
||||||
|
|
||||||
|
The `json` output encoder converts approved artifacts, rejected artifacts,
|
||||||
|
warnings, and the run manifest into logical JSON output files. It groups
|
||||||
|
approved artifacts by artifact type and sanitizes artifact-type file names.
|
||||||
|
|
||||||
|
Requires:
|
||||||
|
|
||||||
|
- `normalized`
|
||||||
|
|
||||||
|
Provides:
|
||||||
|
|
||||||
|
- `encoded`
|
||||||
|
|
||||||
|
Durable output file shapes belong in the
|
||||||
|
[JSON output contract](../integrations/json-output.md). Operator behavior
|
||||||
|
belongs in [Operations](../operations.md).
|
||||||
|
|
||||||
|
## Production Registration
|
||||||
|
|
||||||
|
Production registration is centralized in `internal/cli/catalog.go`.
|
||||||
|
|
||||||
|
Do not make framework code import production modules. The CLI wires production
|
||||||
|
modules at the application boundary; tests may provide fake registries or fake
|
||||||
|
catalogs directly.
|
||||||
|
|
||||||
|
## Adding A Module
|
||||||
|
|
||||||
|
When adding a module, keep source-format and extraction-domain boundaries clear:
|
||||||
|
|
||||||
|
- input modules may know external source formats;
|
||||||
|
- extract modules may know artifact semantics and prompt/schema assets;
|
||||||
|
- merge and normalize modules own candidate combination and reconciliation;
|
||||||
|
- output modules own serialization, not diagnostics or CLI reporting.
|
||||||
|
|
||||||
|
Update [Development](../policy/development.md), [Configuration](../config.md),
|
||||||
|
internal docs, integration docs, and examples when the new module becomes
|
||||||
|
implemented production behavior.
|
||||||
86
docs/internal/overview.md
Normal file
86
docs/internal/overview.md
Normal file
@@ -0,0 +1,86 @@
|
|||||||
|
# Internal Overview
|
||||||
|
|
||||||
|
This directory documents implemented Notarius internals for developers and LLM
|
||||||
|
coding agents. It complements [Architecture](../policy/architecture.md), which
|
||||||
|
is the durable policy for boundaries and invariants.
|
||||||
|
|
||||||
|
## Executable And CLI
|
||||||
|
|
||||||
|
`cmd/notarius` calls the CLI package. `internal/cli` owns:
|
||||||
|
|
||||||
|
- command parsing and usage;
|
||||||
|
- config discovery and loading;
|
||||||
|
- production module catalog and registry wiring;
|
||||||
|
- production LLM client construction;
|
||||||
|
- run directory creation;
|
||||||
|
- durable output writes;
|
||||||
|
- user-facing stdout, stderr, and exit codes.
|
||||||
|
|
||||||
|
The CLI should stay thin around framework contracts. Domain extraction behavior
|
||||||
|
belongs in modules, not in command handlers.
|
||||||
|
|
||||||
|
## Core Packages
|
||||||
|
|
||||||
|
- `internal/core/artifacts`: artifact candidates, approved artifacts, rejected
|
||||||
|
artifacts, validation decisions, and run manifests.
|
||||||
|
- `internal/core/config`: defaults, YAML config parsing, environment overrides,
|
||||||
|
validation, redaction, and resolved pipeline config.
|
||||||
|
- `internal/core/diagnostics`: per-run diagnostics directory creation,
|
||||||
|
diagnostics artifact writers, atomic writes, and retention decisions.
|
||||||
|
- `internal/core/source`: source documents, source units, source references, and
|
||||||
|
validation.
|
||||||
|
|
||||||
|
Core packages should remain deterministic and concrete. They should not import
|
||||||
|
production modules.
|
||||||
|
|
||||||
|
## Framework Packages
|
||||||
|
|
||||||
|
- `internal/framework/contracts`: interfaces and request/result structs for
|
||||||
|
input adapters, chunkers, extractors, mergers, normalizers, validators, output
|
||||||
|
encoders, and structured LLM clients.
|
||||||
|
- `internal/framework/pipeline`: module registries, module specs, profile
|
||||||
|
resolution, capability checks, run orchestration, warnings, validation, and
|
||||||
|
manifest population.
|
||||||
|
- `internal/framework/llm`: OpenAI-compatible structured-output client,
|
||||||
|
scheduler, schema registry, retries, and secret redaction.
|
||||||
|
- `internal/framework/prompt`: embedded prompt registry and template rendering.
|
||||||
|
- `internal/framework/validate`: validator decision helpers and cardinality
|
||||||
|
enforcement.
|
||||||
|
|
||||||
|
Framework code should stay source-agnostic and domain-agnostic.
|
||||||
|
|
||||||
|
## Module Packages
|
||||||
|
|
||||||
|
Production module packages live under `internal/modules`:
|
||||||
|
|
||||||
|
- `input/seriatim`
|
||||||
|
- `chunk/generic`
|
||||||
|
- `extract/dnd/spells`
|
||||||
|
- `merge/appendorder`
|
||||||
|
- `normalize/noop`
|
||||||
|
- `output/json`
|
||||||
|
|
||||||
|
Each module package owns its contract implementation, module spec,
|
||||||
|
registration, options, focused tests, and module-specific errors.
|
||||||
|
|
||||||
|
## Fixtures And Tests
|
||||||
|
|
||||||
|
The repository uses focused package tests plus a fixture-driven CLI workflow.
|
||||||
|
|
||||||
|
- CLI acceptance tests cover maintained examples under `examples/`.
|
||||||
|
- Pipeline tests cover registry composition and end-to-end framework behavior
|
||||||
|
with fakes.
|
||||||
|
- Module tests cover implemented module contracts without requiring real
|
||||||
|
provider calls.
|
||||||
|
- LLM tests use local test servers and fakes.
|
||||||
|
|
||||||
|
Do not use real external services in tests. Use fakes, fixtures, or local test
|
||||||
|
servers.
|
||||||
|
|
||||||
|
## Boundary Reminders
|
||||||
|
|
||||||
|
- Source-format details stay in input modules and integration docs.
|
||||||
|
- Extraction-domain details stay in extract modules and artifact docs.
|
||||||
|
- Provider wire details stay in the LLM runtime and provider integration docs.
|
||||||
|
- Durable output contracts belong in integration docs.
|
||||||
|
- Operator procedures belong in `docs/operations.md`, not internal docs.
|
||||||
127
docs/internal/pipeline.md
Normal file
127
docs/internal/pipeline.md
Normal file
@@ -0,0 +1,127 @@
|
|||||||
|
# Pipeline Internals
|
||||||
|
|
||||||
|
The implemented pipeline runner lives in `internal/framework/pipeline`. It
|
||||||
|
executes the fixed workflow defined by the architecture policy:
|
||||||
|
|
||||||
|
```text
|
||||||
|
input -> chunk -> extract -> merge -> normalize -> output
|
||||||
|
```
|
||||||
|
|
||||||
|
Pipeline execution is serial. The runner executes the resolved lanes one after
|
||||||
|
another in the fixed workflow order.
|
||||||
|
|
||||||
|
## Profile Resolution
|
||||||
|
|
||||||
|
Config loading produces `pipeline.PipelineProfile` values. Resolution happens
|
||||||
|
before execution:
|
||||||
|
|
||||||
|
1. `internal/core/config.Config.Resolve` validates config and finds the named
|
||||||
|
pipeline.
|
||||||
|
2. The optional lane selection is passed to `pipeline.ResolvePipeline`.
|
||||||
|
3. Module bindings are defaulted:
|
||||||
|
- chunk: `generic`
|
||||||
|
- merge: `appendorder`
|
||||||
|
- normalize: `noop`
|
||||||
|
- output: `json`
|
||||||
|
- LLM profile: `default`
|
||||||
|
4. The module catalog is checked for each bound module key.
|
||||||
|
5. Module capabilities are checked in workflow order.
|
||||||
|
6. A digest is calculated from the resolved pipeline without the digest field.
|
||||||
|
|
||||||
|
The CLI writes the resolved pipeline and digest to diagnostics.
|
||||||
|
|
||||||
|
## Registries And Module Specs
|
||||||
|
|
||||||
|
`pipeline.Registries` holds concrete constructors for execution. A
|
||||||
|
`pipeline.ModuleCatalog` exposes module specs for config validation and
|
||||||
|
resolution.
|
||||||
|
|
||||||
|
Every production module registers a `ModuleSpec` with:
|
||||||
|
|
||||||
|
- `Key`: module key used in config;
|
||||||
|
- `Stage`: module kind such as input, chunk, extract, merge, normalize,
|
||||||
|
validate, or output;
|
||||||
|
- `Provides`: capabilities added after that module runs;
|
||||||
|
- `Requires`: capabilities that must already be available.
|
||||||
|
|
||||||
|
Capability checks prevent incompatible pipeline composition before a run starts.
|
||||||
|
|
||||||
|
## Runner Input And Output
|
||||||
|
|
||||||
|
`pipeline.RunInput` carries:
|
||||||
|
|
||||||
|
- a `ResolvedPipeline`;
|
||||||
|
- optional source ID, input path, and raw input bytes;
|
||||||
|
- a structured LLM client;
|
||||||
|
- run ID, start time, LLM profile manifest metadata, and CLI metadata.
|
||||||
|
|
||||||
|
`pipeline.RunOutput` carries:
|
||||||
|
|
||||||
|
- run manifest;
|
||||||
|
- approved artifacts;
|
||||||
|
- rejected artifacts;
|
||||||
|
- warnings;
|
||||||
|
- logical output files returned by the output encoder.
|
||||||
|
|
||||||
|
The CLI owns durable file writes and diagnostics writes after the runner returns.
|
||||||
|
|
||||||
|
## Execution
|
||||||
|
|
||||||
|
The runner:
|
||||||
|
|
||||||
|
1. validates run input and registries;
|
||||||
|
2. builds the input adapter and parses the raw input into a source document;
|
||||||
|
3. validates the source document;
|
||||||
|
4. builds the chunker and produces source chunks;
|
||||||
|
5. runs each selected artifact lane in sorted resolved order;
|
||||||
|
6. builds the output encoder and validates logical output file names.
|
||||||
|
|
||||||
|
Within an artifact lane, the runner:
|
||||||
|
|
||||||
|
1. builds the extractor, merger, and normalizer;
|
||||||
|
2. records module manifest metadata when modules provide it;
|
||||||
|
3. extracts candidates from each chunk;
|
||||||
|
4. normalizes candidate envelope fields such as index, extractor key, artifact
|
||||||
|
type, and schema version;
|
||||||
|
5. merges candidates;
|
||||||
|
6. normalizes merged candidates;
|
||||||
|
7. validates candidate envelope consistency;
|
||||||
|
8. runs validators;
|
||||||
|
9. converts approved candidates to artifacts.
|
||||||
|
|
||||||
|
## Validators
|
||||||
|
|
||||||
|
If a lane declares validators in config, the runner builds those validators from
|
||||||
|
the validator registry. Otherwise it uses validators returned by the extractor.
|
||||||
|
|
||||||
|
Each validator must return exactly one decision for each eligible candidate. The
|
||||||
|
runner enforces decision cardinality with `internal/framework/validate`.
|
||||||
|
Rejected candidates are removed before the next validator runs. Approved
|
||||||
|
candidates continue through the chain.
|
||||||
|
|
||||||
|
The production CLI currently registers no standalone validator modules. The
|
||||||
|
current D&D spell extractor supplies deterministic shape and source-reference
|
||||||
|
validators.
|
||||||
|
|
||||||
|
## Warnings And Failures
|
||||||
|
|
||||||
|
Warnings from chunking, extraction, merging, normalization, validation, and
|
||||||
|
output encoding are accumulated in `RunOutput.Warnings`.
|
||||||
|
|
||||||
|
Errors wrap the operation and module key or lane context. If execution fails
|
||||||
|
after a manifest exists, the returned manifest is marked `failed` and receives a
|
||||||
|
completion timestamp.
|
||||||
|
|
||||||
|
On successful execution, the manifest validation status is:
|
||||||
|
|
||||||
|
- `approved` when no candidates were rejected;
|
||||||
|
- `rejected` when at least one candidate was rejected.
|
||||||
|
|
||||||
|
## Manifest Population
|
||||||
|
|
||||||
|
The manifest records run ID, pipeline ID, pipeline digest, module keys, artifact
|
||||||
|
lanes, LLM profile metadata, source digest, validation status, and timing.
|
||||||
|
|
||||||
|
Modules can add non-secret manifest metadata by implementing
|
||||||
|
`contracts.ManifestMetadataProvider`. The D&D spell extractor uses this for
|
||||||
|
prompt and response-schema provenance.
|
||||||
132
docs/operations.md
Normal file
132
docs/operations.md
Normal file
@@ -0,0 +1,132 @@
|
|||||||
|
# Operations
|
||||||
|
|
||||||
|
This is the canonical reference for operating implemented Notarius runs.
|
||||||
|
|
||||||
|
## Normal Run
|
||||||
|
|
||||||
|
A run reads one source file, resolves one configured pipeline, calls the
|
||||||
|
configured OpenAI-compatible LLM profile, writes durable JSON output, and writes
|
||||||
|
diagnostics for inspection.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/notarius run dnd-session \
|
||||||
|
--config examples/dnd-spells.config.yml \
|
||||||
|
--input examples/seriatim-minimal-transcript.json \
|
||||||
|
--output-dir ./notarius-output \
|
||||||
|
--diagnostics-dir /tmp/notarius
|
||||||
|
```
|
||||||
|
|
||||||
|
The command prints a success line with the pipeline ID, approved and rejected
|
||||||
|
artifact counts, and the output path.
|
||||||
|
|
||||||
|
## Output Directory
|
||||||
|
|
||||||
|
Durable output is written to:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<output-root>/<run-id>/
|
||||||
|
```
|
||||||
|
|
||||||
|
The default output root is `./notarius-output`. Use `--output-dir` to choose a
|
||||||
|
different root.
|
||||||
|
|
||||||
|
The `json` output module writes these files:
|
||||||
|
|
||||||
|
- `index.json`: file index with paths to the manifest, artifact files,
|
||||||
|
rejected artifacts, and warnings.
|
||||||
|
- `manifest.json`: run manifest with resolved pipeline provenance, module keys,
|
||||||
|
validation status, and timing.
|
||||||
|
- `artifacts/<artifact-type>.json`: approved artifacts grouped by artifact
|
||||||
|
type. For the current D&D spell extractor, this includes
|
||||||
|
`artifacts/dnd.spell_cast.json` when spell-cast artifacts are approved.
|
||||||
|
- `rejected.json`: rejected candidates and validator decisions.
|
||||||
|
- `warnings.json`: warnings reported by pipeline modules or the output encoder.
|
||||||
|
|
||||||
|
Output writes are atomic per file. Logical output file names must be clean,
|
||||||
|
relative, slash-separated paths and must not contain `..`.
|
||||||
|
|
||||||
|
## Diagnostics Directory
|
||||||
|
|
||||||
|
Diagnostics are written under:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<diagnostics-work-dir>/<run-id>/
|
||||||
|
```
|
||||||
|
|
||||||
|
The default diagnostics work directory is `/tmp/notarius`. It can be set with
|
||||||
|
`diagnostics.work_dir`, `NOTARIUS_WORK_DIR`, or `--diagnostics-dir`.
|
||||||
|
|
||||||
|
Implemented diagnostics artifacts:
|
||||||
|
|
||||||
|
- `invocation.json`: command metadata such as operation, config path, input
|
||||||
|
path, selected lanes, run ID, and pipeline digest when available.
|
||||||
|
- `effective-config.json`: resolved config with API keys redacted.
|
||||||
|
- `resolved-pipeline.json`: resolved module bindings and pipeline digest.
|
||||||
|
- `run-manifest.json`: the same run manifest written to durable output when it
|
||||||
|
is available.
|
||||||
|
- `warnings.json`: warning list.
|
||||||
|
- `run-report.json`: counts, status, output path, diagnostics path, and run ID.
|
||||||
|
- `error.log`: failure message, written after diagnostics directory creation
|
||||||
|
when a run fails.
|
||||||
|
|
||||||
|
`source-document.json` is supported by the diagnostics writer but is not written
|
||||||
|
by the current CLI run workflow.
|
||||||
|
|
||||||
|
## Retention
|
||||||
|
|
||||||
|
Diagnostics retention is configured with `diagnostics.retention`,
|
||||||
|
`NOTARIUS_DIAGNOSTICS_RETENTION`, or the default `auto`.
|
||||||
|
|
||||||
|
- `auto`: keep failed runs and successful runs with warnings; remove successful
|
||||||
|
warning-free runs.
|
||||||
|
- `always`: keep every diagnostics run directory.
|
||||||
|
- `never`: remove successful run directories; failed runs are still retained.
|
||||||
|
|
||||||
|
Unknown retention values are rejected during config validation.
|
||||||
|
|
||||||
|
## Failures
|
||||||
|
|
||||||
|
Failures before diagnostics directory creation, such as a missing config file or
|
||||||
|
an unusable diagnostics work directory, are printed to stderr and may not have a
|
||||||
|
diagnostics run directory.
|
||||||
|
|
||||||
|
Failures after diagnostics directory creation are printed to stderr and written
|
||||||
|
to `error.log`. Depending on where the failure occurred, diagnostics may also
|
||||||
|
include invocation metadata, redacted effective config, resolved pipeline data,
|
||||||
|
the run manifest, warnings, and a run report.
|
||||||
|
|
||||||
|
If durable output writing fails after the pipeline completes, diagnostics are
|
||||||
|
retained for inspection and may include `run-manifest.json`, `warnings.json`,
|
||||||
|
`run-report.json`, and `error.log`.
|
||||||
|
|
||||||
|
## Warnings
|
||||||
|
|
||||||
|
A successful run with warnings exits with code `0`, prints a warning count to
|
||||||
|
stderr, and writes warnings to durable output and diagnostics when retained.
|
||||||
|
|
||||||
|
The run manifest `validation_status` indicates whether final artifacts were
|
||||||
|
approved or rejected after validation.
|
||||||
|
|
||||||
|
## Cleanup
|
||||||
|
|
||||||
|
It is safe to remove specific old run directories after their output and
|
||||||
|
diagnostics are no longer needed:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
rm -rf /tmp/notarius/run-1234567890
|
||||||
|
rm -rf ./notarius-output/run-1234567890
|
||||||
|
```
|
||||||
|
|
||||||
|
Use exact run-directory paths. Avoid broad cleanup commands against parent
|
||||||
|
directories unless they are part of your own operational policy.
|
||||||
|
|
||||||
|
## Operational Limits
|
||||||
|
|
||||||
|
There is no command to resume a failed run. Re-run `notarius run` after fixing
|
||||||
|
the cause.
|
||||||
|
|
||||||
|
Provider retries are limited to the OpenAI-compatible client retry behavior
|
||||||
|
configured by the effective LLM profile. There is no separate CLI retry command.
|
||||||
|
|
||||||
|
Notarius writes local files only. Remote storage and archive management are not
|
||||||
|
part of the implemented CLI.
|
||||||
139
docs/policy/development.md
Normal file
139
docs/policy/development.md
Normal file
@@ -0,0 +1,139 @@
|
|||||||
|
# Development
|
||||||
|
|
||||||
|
This document defines contributor workflow for Notarius. For architectural
|
||||||
|
invariants and package boundaries, read [Architecture](architecture.md) first.
|
||||||
|
|
||||||
|
## Required Reading
|
||||||
|
|
||||||
|
Before changing the repository, review:
|
||||||
|
|
||||||
|
- [Architecture](architecture.md)
|
||||||
|
- [Documentation Policy](documentation.md)
|
||||||
|
|
||||||
|
Keep current-behavior documentation limited to implemented behavior. Put planned
|
||||||
|
or deferred behavior under `docs/roadmap/`.
|
||||||
|
|
||||||
|
## Repository Layout
|
||||||
|
|
||||||
|
- `cmd/notarius`: executable entry point.
|
||||||
|
- `internal/cli`: CLI parsing, production catalog wiring, config loading, run
|
||||||
|
command orchestration, output writes, and user-facing errors.
|
||||||
|
- `internal/core`: deterministic models and policy for artifacts, source
|
||||||
|
documents, config, and diagnostics.
|
||||||
|
- `internal/framework`: reusable contracts, pipeline orchestration, prompt
|
||||||
|
helpers, validation helpers, and LLM runtime plumbing.
|
||||||
|
- `internal/modules`: concrete input, chunk, extract, merge, normalize, and
|
||||||
|
output modules.
|
||||||
|
- `docs`: policy, user/operator docs, internal docs, integration docs, and
|
||||||
|
roadmap files.
|
||||||
|
- `examples`: maintained, secret-free examples covered by tests where practical.
|
||||||
|
|
||||||
|
## Validation Commands
|
||||||
|
|
||||||
|
Run focused tests for the area changed, then run the broader checks when the
|
||||||
|
change affects shared contracts, CLI behavior, or documentation examples.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test ./...
|
||||||
|
go vet ./...
|
||||||
|
go build ./cmd/notarius
|
||||||
|
```
|
||||||
|
|
||||||
|
Useful focused checks:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test ./internal/cli
|
||||||
|
go test ./internal/core/config
|
||||||
|
go test ./internal/framework/pipeline
|
||||||
|
go test ./internal/framework/llm
|
||||||
|
go test ./internal/modules/input/seriatim
|
||||||
|
go test ./internal/modules/extract/dnd/spells
|
||||||
|
go test ./internal/modules/output/json
|
||||||
|
```
|
||||||
|
|
||||||
|
## Go Conventions
|
||||||
|
|
||||||
|
- Prefer the standard library unless a dependency is justified by correctness,
|
||||||
|
security, interoperability, or substantial complexity reduction.
|
||||||
|
- Keep package names short, lowercase, and idiomatic.
|
||||||
|
- Preserve import direction: framework and core code must not depend on concrete
|
||||||
|
production modules.
|
||||||
|
- Use `context.Context` for long-running operations and external calls.
|
||||||
|
- Return contextual errors that name the operation and relevant module, path, or
|
||||||
|
resource.
|
||||||
|
- Do not include secrets in errors, logs, diagnostics, manifests, or docs.
|
||||||
|
|
||||||
|
## Adding Config Fields
|
||||||
|
|
||||||
|
Config behavior is centralized under `internal/core/config`.
|
||||||
|
|
||||||
|
When adding a file config field:
|
||||||
|
|
||||||
|
1. Update file config structs and YAML parsing in `file_config.go`.
|
||||||
|
2. Apply the field over defaults in config application code.
|
||||||
|
3. Add validation in `validation.go` when the field has constraints.
|
||||||
|
4. Add environment override support in `env.go` only for operational overrides.
|
||||||
|
5. Update redaction if the field can contain secrets.
|
||||||
|
6. Add focused config tests.
|
||||||
|
7. Update [Configuration](../config.md) and maintained examples when behavior
|
||||||
|
changes.
|
||||||
|
|
||||||
|
Pipeline composition should remain config-driven. Do not add command flags that
|
||||||
|
silently replace structural pipeline definitions.
|
||||||
|
|
||||||
|
## Adding CLI Flags Or Commands
|
||||||
|
|
||||||
|
CLI behavior lives in `internal/cli`.
|
||||||
|
|
||||||
|
When adding CLI surface:
|
||||||
|
|
||||||
|
1. Keep syntax explicit and update usage text.
|
||||||
|
2. Validate arguments before running expensive work.
|
||||||
|
3. Convert internal errors into concise user-facing messages.
|
||||||
|
4. Add CLI tests for success, syntax errors, and failure modes.
|
||||||
|
5. Update [CLI Reference](../cli.md), and update
|
||||||
|
[Operations](../operations.md) or [Troubleshooting](../troubleshooting.md)
|
||||||
|
if run behavior changes.
|
||||||
|
|
||||||
|
## Adding Modules Or Adapters
|
||||||
|
|
||||||
|
Concrete modules live under `internal/modules/<kind>/...` and implement the
|
||||||
|
interfaces in `internal/framework/contracts`.
|
||||||
|
|
||||||
|
For a new production module:
|
||||||
|
|
||||||
|
1. Implement the relevant contract.
|
||||||
|
2. Expose a `ModuleSpec` with the correct module key, module kind, provided
|
||||||
|
capabilities, and required capabilities.
|
||||||
|
3. Expose a `Register` function that registers the module with its registry.
|
||||||
|
4. Add focused module tests for contract behavior, registration, options,
|
||||||
|
validation, and errors.
|
||||||
|
5. Register the module in `internal/cli/catalog.go` only when it is production
|
||||||
|
ready.
|
||||||
|
6. Update internal docs and user-facing docs only for implemented behavior.
|
||||||
|
|
||||||
|
Source-format behavior belongs in input modules and integration docs.
|
||||||
|
Extraction-domain behavior belongs in extract modules and artifact docs.
|
||||||
|
|
||||||
|
## Updating Examples
|
||||||
|
|
||||||
|
Examples must be valid, secret-free, and small.
|
||||||
|
|
||||||
|
- Prefer environment-based secret configuration.
|
||||||
|
- Keep `examples/dnd-spells.config.yml` loadable by CLI tests.
|
||||||
|
- Keep `examples/seriatim-minimal-transcript.json` compatible with the Seriatim
|
||||||
|
adapter.
|
||||||
|
- Do not add expected-output fixtures unless they are validated or have a clear
|
||||||
|
regeneration procedure.
|
||||||
|
|
||||||
|
## Documentation Updates
|
||||||
|
|
||||||
|
Update docs in the same change when behavior changes.
|
||||||
|
|
||||||
|
- CLI syntax: `docs/cli.md`
|
||||||
|
- Config fields and defaults: `docs/config.md`
|
||||||
|
- Output, diagnostics, retention, or recovery: `docs/operations.md`
|
||||||
|
- Common user-facing failures: `docs/troubleshooting.md`
|
||||||
|
- Internal architecture and contracts: `docs/internal/`
|
||||||
|
- External file formats and durable integration contracts: `docs/integrations/`
|
||||||
|
- Future or planned work only: `docs/roadmap/`
|
||||||
@@ -1,759 +0,0 @@
|
|||||||
# Documentation Roadmap
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
This roadmap defines the work required to bring Notarius documentation into
|
|
||||||
compliance with [`docs/policy/documentation.md`](../policy/documentation.md) and
|
|
||||||
the current implementation. It is a planning document only. Future implementation
|
|
||||||
agents should use it to rewrite documentation in stages without documenting
|
|
||||||
unimplemented behavior outside `docs/roadmap/`.
|
|
||||||
|
|
||||||
## Repository Documentation Inventory
|
|
||||||
|
|
||||||
- `README.md`: keep and rewrite. It still contains template text
|
|
||||||
(`go-application-template`) and does not describe the implemented Notarius CLI.
|
|
||||||
- `AGENTS.md`: keep and lightly update if needed. It correctly directs agents to
|
|
||||||
read `docs/policy`, and should stay short.
|
|
||||||
- `docs/policy/documentation.md`: keep and do not update. It is the controlling documentation policy.
|
|
||||||
- `docs/policy/architecture.md`: keep and do not update. It is the controlling architecture policy.
|
|
||||||
- `docs/integrations/seriatim.md`: keep and lightly update. It is close to the
|
|
||||||
current Seriatim adapter behavior and belongs in `docs/integrations/` because
|
|
||||||
Seriatim minimal transcript JSON is an external file-format contract.
|
|
||||||
- `docs/integrations/dnd-spells.md`: split. It mixes implemented extractor
|
|
||||||
internals, artifact payload details, and stale roadmap text claiming the CLI
|
|
||||||
`run` workflow does not exist. Move module-internal behavior to
|
|
||||||
`docs/internal/modules.md` or a module-specific internal doc; move durable
|
|
||||||
spell artifact output shape to an integration/artifact contract doc.
|
|
||||||
- `docs/roadmap/documentation.md`: keep during the documentation migration. After
|
|
||||||
the migration is complete, either delete it or reduce it to any remaining
|
|
||||||
documentation work only.
|
|
||||||
- `docs/roadmap/implementation.md`: delete after migration. It is a completed MVP
|
|
||||||
implementation plan and is no longer an active roadmap.
|
|
||||||
- `docs/roadmap/initial-architecture.md`: delete after migration. The implemented
|
|
||||||
architecture now belongs in policy and internal docs; historical planning
|
|
||||||
detail should not remain as active roadmap content.
|
|
||||||
- `docs/roadmap/mvp.md`: delete or rewrite after migration. It currently records
|
|
||||||
implemented MVP behavior and deferred documentation work; once current-behavior
|
|
||||||
docs exist, keep only genuine future work under `docs/roadmap/`.
|
|
||||||
- `examples/dnd-spells.config.yml`: keep and lightly update. It is a maintained
|
|
||||||
example covered by CLI tests, but should be checked for user-facing clarity and
|
|
||||||
secret-free LLM configuration.
|
|
||||||
- `examples/seriatim-minimal-transcript.json`: keep and lightly update. It is a
|
|
||||||
maintained Seriatim input fixture covered by CLI tests.
|
|
||||||
|
|
||||||
## Policy Compliance Assessment
|
|
||||||
|
|
||||||
Required documents missing under the documentation policy:
|
|
||||||
|
|
||||||
- `docs/cli.md`: required because Notarius is a CLI application.
|
|
||||||
- `docs/config.md`: required because Notarius is config-driven.
|
|
||||||
- `docs/operations.md`: required because Notarius runs multi-step workflows,
|
|
||||||
writes durable artifacts, and maintains diagnostics directories.
|
|
||||||
- `docs/internal/`: required because Notarius is modular and orchestration-based.
|
|
||||||
- `docs/policy/development.md`: required because the project is maintained by
|
|
||||||
humans and LLM coding agents.
|
|
||||||
|
|
||||||
Recommended documents that should be added:
|
|
||||||
|
|
||||||
- `docs/troubleshooting.md`: recommended because implemented commands have common
|
|
||||||
failure modes: missing config, unknown pipeline, invalid lane selection,
|
|
||||||
invalid Seriatim input, LLM configuration failures, provider failures, output
|
|
||||||
write failures, and diagnostics retention behavior.
|
|
||||||
- Maintained examples under `examples/`: already present and should remain
|
|
||||||
linked from README, CLI, config, and integration docs.
|
|
||||||
|
|
||||||
Documents that exist but are stale or in the wrong canonical home:
|
|
||||||
|
|
||||||
- `README.md` is stale template content.
|
|
||||||
- `docs/integrations/dnd-spells.md` includes stale limits and belongs partly in
|
|
||||||
internal/module docs and partly in artifact integration docs.
|
|
||||||
- Completed roadmap documents describe implemented behavior and should not be the
|
|
||||||
only source of current-behavior documentation after the migration.
|
|
||||||
|
|
||||||
Content describing deprecated, historical, planned, or unimplemented behavior
|
|
||||||
outside `docs/roadmap/`:
|
|
||||||
|
|
||||||
- `docs/integrations/dnd-spells.md` says the CLI `run` workflow does not exist;
|
|
||||||
that is now false.
|
|
||||||
- No other non-roadmap stale claims were found during this pass, but future
|
|
||||||
implementation agents should grep before rewriting.
|
|
||||||
|
|
||||||
Examples:
|
|
||||||
|
|
||||||
- `examples/dnd-spells.config.yml` and
|
|
||||||
`examples/seriatim-minimal-transcript.json` are covered by
|
|
||||||
`internal/cli/run_test.go`.
|
|
||||||
- There is no maintained expected-output example under `examples/`. Do not add
|
|
||||||
one unless it can be validated or regenerated from implemented behavior.
|
|
||||||
|
|
||||||
Links likely needing verification:
|
|
||||||
|
|
||||||
- New README links to `docs/cli.md`, `docs/config.md`,
|
|
||||||
`docs/operations.md`, `docs/troubleshooting.md`, `docs/internal/`, and
|
|
||||||
`docs/integrations/` will need verification after those files are created.
|
|
||||||
- Existing roadmap links to old implementation documents should be removed or
|
|
||||||
deleted with the roadmap cleanup stage.
|
|
||||||
|
|
||||||
Documents not applicable:
|
|
||||||
|
|
||||||
- `docs/api.md` is not needed because Notarius does not implement a public HTTP
|
|
||||||
API.
|
|
||||||
- `docs/consumers/` is not needed for this pass because the repository exposes a
|
|
||||||
CLI, not a supported public package, SDK, plugin API, or HTTP consumer API.
|
|
||||||
|
|
||||||
## Target Documentation Set
|
|
||||||
|
|
||||||
Desired final tree after the migration:
|
|
||||||
|
|
||||||
```text
|
|
||||||
AGENTS.md
|
|
||||||
README.md
|
|
||||||
docs/cli.md
|
|
||||||
docs/config.md
|
|
||||||
docs/operations.md
|
|
||||||
docs/troubleshooting.md
|
|
||||||
docs/policy/architecture.md
|
|
||||||
docs/policy/development.md
|
|
||||||
docs/policy/documentation.md
|
|
||||||
docs/internal/overview.md
|
|
||||||
docs/internal/pipeline.md
|
|
||||||
docs/internal/modules.md
|
|
||||||
docs/internal/llm.md
|
|
||||||
docs/internal/diagnostics.md
|
|
||||||
docs/integrations/seriatim.md
|
|
||||||
docs/integrations/openai-compatible.md
|
|
||||||
docs/integrations/json-output.md
|
|
||||||
docs/integrations/dnd-spell-artifacts.md
|
|
||||||
docs/roadmap/mvp.md
|
|
||||||
examples/dnd-spells.config.yml
|
|
||||||
examples/seriatim-minimal-transcript.json
|
|
||||||
```
|
|
||||||
|
|
||||||
`docs/roadmap/mvp.md` should exist only if it contains genuine remaining future
|
|
||||||
work after the migration. `docs/roadmap/documentation.md` exists to drive this
|
|
||||||
migration and should be deleted or reduced when the migration is complete.
|
|
||||||
|
|
||||||
### `AGENTS.md`
|
|
||||||
|
|
||||||
- Audience: LLM coding agents.
|
|
||||||
- Purpose: shortest repository-specific agent instruction.
|
|
||||||
- Canonical scope: policy files that agents must read before changes.
|
|
||||||
- Recommended outline: one or two sentences linking to policy docs.
|
|
||||||
- Source-of-truth areas: `docs/policy/architecture.md`,
|
|
||||||
`docs/policy/documentation.md`.
|
|
||||||
- Acceptance criteria: stays short; no duplicated architecture or documentation
|
|
||||||
policy; links remain accurate.
|
|
||||||
|
|
||||||
### `README.md`
|
|
||||||
|
|
||||||
- Audience: users, administrators, operators.
|
|
||||||
- Purpose: concise outward-facing orientation.
|
|
||||||
- Canonical scope: project purpose, shortest useful command, and links to
|
|
||||||
targeted docs.
|
|
||||||
- Recommended outline: description; elevator pitch; shortest useful command
|
|
||||||
using maintained examples; where output appears; links.
|
|
||||||
- Source-of-truth areas: `cmd/notarius/main.go`, `internal/cli/run.go`,
|
|
||||||
`examples/dnd-spells.config.yml`,
|
|
||||||
`examples/seriatim-minimal-transcript.json`,
|
|
||||||
`internal/cli/run_test.go`.
|
|
||||||
- Acceptance criteria: no template text; no long manual content; commands match
|
|
||||||
real CLI syntax; links resolve; no unimplemented features.
|
|
||||||
|
|
||||||
### `docs/policy/architecture.md`
|
|
||||||
|
|
||||||
- Audience: developers, LLM coding agents.
|
|
||||||
- Purpose: durable development architecture policy.
|
|
||||||
- Canonical scope: stable architecture invariants and boundaries.
|
|
||||||
- Recommended outline: keep the current concise policy shape.
|
|
||||||
- Source-of-truth areas: current architecture policy and actual package
|
|
||||||
boundaries.
|
|
||||||
- Acceptance criteria: does not grow into CLI/config/reference documentation;
|
|
||||||
no planned behavior outside roadmap language.
|
|
||||||
|
|
||||||
### `docs/policy/documentation.md`
|
|
||||||
|
|
||||||
- Audience: developers, LLM coding agents.
|
|
||||||
- Purpose: controlling documentation policy.
|
|
||||||
- Canonical scope: documentation layout, audience, maintenance rules, and
|
|
||||||
canonical homes.
|
|
||||||
- Recommended outline: keep current policy unless project-wide documentation
|
|
||||||
rules change.
|
|
||||||
- Source-of-truth areas: current documentation policy and this migration.
|
|
||||||
- Acceptance criteria: remains policy-oriented; no project-specific
|
|
||||||
current-behavior reference material that belongs elsewhere.
|
|
||||||
|
|
||||||
### `docs/cli.md`
|
|
||||||
|
|
||||||
- Audience: users, administrators, operators.
|
|
||||||
- Purpose: canonical CLI reference.
|
|
||||||
- Canonical scope: implemented commands, flags, exit behavior, common workflows,
|
|
||||||
and command output.
|
|
||||||
- Recommended outline: shortest useful command; command overview; `help`;
|
|
||||||
`run`; `config validate`; `pipelines list`; exit codes; common workflows;
|
|
||||||
diagnostics/output pointers.
|
|
||||||
- Source-of-truth areas: `internal/cli/run.go`, `internal/cli/run_test.go`,
|
|
||||||
`internal/cli/catalog.go`.
|
|
||||||
- Acceptance criteria: documents only implemented commands and flags; explains
|
|
||||||
`--config`, `--input`, `--only`, `--output-dir`, `--diagnostics-dir`,
|
|
||||||
`--llm-profile`, `--pipeline`, and `--json`; does not duplicate full config
|
|
||||||
reference; examples are covered by tests or are direct variants of tested
|
|
||||||
commands.
|
|
||||||
|
|
||||||
### `docs/config.md`
|
|
||||||
|
|
||||||
- Audience: administrators, operators, advanced users.
|
|
||||||
- Purpose: canonical configuration reference.
|
|
||||||
- Canonical scope: config discovery, YAML version 1 structure, defaults,
|
|
||||||
environment overrides, pipeline profiles, module bindings, LLM profiles,
|
|
||||||
diagnostics settings, validation behavior, and secrets handling.
|
|
||||||
- Recommended outline: config discovery and precedence; minimal working config;
|
|
||||||
production-oriented config; `version`; `llm_profiles`; `pipelines`;
|
|
||||||
module-binding shorthand and object form; `concurrency`; `diagnostics`;
|
|
||||||
environment overrides; validation and capability checks; examples.
|
|
||||||
- Source-of-truth areas: `internal/core/config/*.go`,
|
|
||||||
`internal/framework/pipeline/profile.go`, `internal/framework/pipeline/module.go`,
|
|
||||||
`internal/cli/run.go`, `examples/dnd-spells.config.yml`, config tests.
|
|
||||||
- Acceptance criteria: full config reference is here and not duplicated
|
|
||||||
elsewhere; lists implemented defaults and supported env vars; explains that
|
|
||||||
raw secrets should not be committed; examples load successfully.
|
|
||||||
|
|
||||||
### `docs/operations.md`
|
|
||||||
|
|
||||||
- Audience: administrators, operators.
|
|
||||||
- Purpose: canonical operational behavior and recovery reference.
|
|
||||||
- Canonical scope: run output directory layout, diagnostics directory layout,
|
|
||||||
retention behavior, manifests, warnings, failure inspection, safe cleanup, and
|
|
||||||
operational caveats.
|
|
||||||
- Recommended outline: normal run workflow; output files; diagnostics files;
|
|
||||||
retention modes; failure behavior; warnings; atomic writes and path safety;
|
|
||||||
safe cleanup; what is not resumable yet.
|
|
||||||
- Source-of-truth areas: `internal/cli/run.go`,
|
|
||||||
`internal/core/diagnostics/*.go`, `internal/modules/output/json/encoder.go`,
|
|
||||||
output and diagnostics tests.
|
|
||||||
- Acceptance criteria: accurately names implemented durable files; states that
|
|
||||||
resume/retry is not implemented as a command; explains when diagnostics are
|
|
||||||
retained; does not describe unimplemented remote storage or archives.
|
|
||||||
|
|
||||||
### `docs/troubleshooting.md`
|
|
||||||
|
|
||||||
- Audience: users, administrators, operators.
|
|
||||||
- Purpose: task-oriented fixes for recurring failure modes.
|
|
||||||
- Canonical scope: symptoms, likely causes, inspection steps, and safe fixes.
|
|
||||||
- Recommended outline: config not found; unsupported config version; unknown
|
|
||||||
pipeline; invalid `--only`; Seriatim validation failure; missing LLM base URL
|
|
||||||
or model; provider HTTP failures; output write failure; diagnostics retention
|
|
||||||
surprise.
|
|
||||||
- Source-of-truth areas: `internal/cli/run.go`, `internal/cli/run_test.go`,
|
|
||||||
`internal/core/config/*_test.go`,
|
|
||||||
`internal/modules/input/seriatim/*_test.go`,
|
|
||||||
`internal/framework/llm/*_test.go`.
|
|
||||||
- Acceptance criteria: every entry maps to implemented behavior; no speculative
|
|
||||||
troubleshooting for future modules; links to CLI/config/operations docs
|
|
||||||
instead of duplicating reference material.
|
|
||||||
|
|
||||||
### `docs/policy/development.md`
|
|
||||||
|
|
||||||
- Audience: developers, LLM coding agents.
|
|
||||||
- Purpose: contributor workflow and change discipline.
|
|
||||||
- Canonical scope: repository layout, build/test commands, coding conventions,
|
|
||||||
dependency policy, how to add config fields, CLI flags, modules, adapters, and
|
|
||||||
examples.
|
|
||||||
- Recommended outline: repository layout; required policy reads; build/test
|
|
||||||
commands; Go conventions; dependency policy; adding CLI flags; adding config;
|
|
||||||
adding stage modules; adding examples; docs update rules.
|
|
||||||
- Source-of-truth areas: `AGENTS.md`, `go.mod`, `internal/cli`,
|
|
||||||
`internal/core/config`, `internal/framework/pipeline`, `internal/modules`,
|
|
||||||
`docs/policy`.
|
|
||||||
- Acceptance criteria: procedural and concise; links to `architecture.md` for
|
|
||||||
invariants; does not restate full CLI/config reference.
|
|
||||||
|
|
||||||
### `docs/internal/overview.md`
|
|
||||||
|
|
||||||
- Audience: developers, LLM coding agents.
|
|
||||||
- Purpose: high-level implemented internals map.
|
|
||||||
- Canonical scope: package roles and import/boundary expectations.
|
|
||||||
- Recommended outline: executable and CLI; core packages; framework packages;
|
|
||||||
module packages; tests and fixtures; where not to put domain/source-specific
|
|
||||||
logic.
|
|
||||||
- Source-of-truth areas: repository tree, `docs/policy/architecture.md`,
|
|
||||||
`internal/cli/catalog.go`.
|
|
||||||
- Acceptance criteria: concise package map; reflects current code; does not
|
|
||||||
become a generated file listing every source file.
|
|
||||||
|
|
||||||
### `docs/internal/pipeline.md`
|
|
||||||
|
|
||||||
- Audience: developers, LLM coding agents.
|
|
||||||
- Purpose: implemented pipeline orchestration reference.
|
|
||||||
- Canonical scope: runner inputs/outputs, stage ordering, registries, resolved
|
|
||||||
profiles, artifact lanes, validation, warnings, and manifest population.
|
|
||||||
- Recommended outline: stage flow; profile resolution; registries and module
|
|
||||||
specs; runner execution; validator behavior; output encoder handoff; failure
|
|
||||||
behavior; tests.
|
|
||||||
- Source-of-truth areas: `internal/framework/contracts/contracts.go`,
|
|
||||||
`internal/framework/pipeline/*.go`, pipeline tests.
|
|
||||||
- Acceptance criteria: explains implemented serial execution; does not promise
|
|
||||||
parallel execution or resume; links to config docs for profile syntax.
|
|
||||||
|
|
||||||
### `docs/internal/modules.md`
|
|
||||||
|
|
||||||
- Audience: developers, LLM coding agents.
|
|
||||||
- Purpose: implemented stage modules and how to add new ones.
|
|
||||||
- Canonical scope: current production modules and their boundaries.
|
|
||||||
- Recommended outline: module contract pattern; `seriatim`; `generic`;
|
|
||||||
`dnd/spells`; `appendorder`; `noop`; `json`; module registration; tests to
|
|
||||||
add for new modules.
|
|
||||||
- Source-of-truth areas: `internal/modules/**`, `internal/cli/catalog.go`,
|
|
||||||
module tests.
|
|
||||||
- Acceptance criteria: documents only implemented modules; moves relevant
|
|
||||||
non-external content from `docs/integrations/dnd-spells.md`; keeps detailed
|
|
||||||
external file-format contracts in `docs/integrations/`.
|
|
||||||
|
|
||||||
### `docs/internal/llm.md`
|
|
||||||
|
|
||||||
- Audience: developers, LLM coding agents.
|
|
||||||
- Purpose: implemented LLM runtime internals.
|
|
||||||
- Canonical scope: structured LLM client contract, OpenAI-compatible adapter,
|
|
||||||
scheduler, schema registry, retry/redaction behavior, and prompt boundary.
|
|
||||||
- Recommended outline: contract; provider request shape at a high level;
|
|
||||||
scheduler; retries and timeouts; structured output decoding; secret redaction;
|
|
||||||
tests.
|
|
||||||
- Source-of-truth areas: `internal/framework/contracts/contracts.go`,
|
|
||||||
`internal/framework/llm/*.go`, `internal/framework/prompt/*.go`,
|
|
||||||
`internal/modules/extract/dnd/spells/assets`.
|
|
||||||
- Acceptance criteria: no provider marketing claims; no API keys; links to
|
|
||||||
OpenAI-compatible integration doc for the external wire contract.
|
|
||||||
|
|
||||||
### `docs/internal/diagnostics.md`
|
|
||||||
|
|
||||||
- Audience: developers, LLM coding agents.
|
|
||||||
- Purpose: implemented diagnostics internals.
|
|
||||||
- Canonical scope: run directory creation, artifact writers, retention decision
|
|
||||||
mechanics, and redacted effective config payloads.
|
|
||||||
- Recommended outline: purpose; inputs and outputs; files written; retention;
|
|
||||||
failure behavior; tests; invariants.
|
|
||||||
- Source-of-truth areas: `internal/core/diagnostics/*.go`,
|
|
||||||
`internal/core/config/redaction.go`, CLI diagnostics tests.
|
|
||||||
- Acceptance criteria: complements `docs/operations.md`; does not duplicate
|
|
||||||
operator procedures.
|
|
||||||
|
|
||||||
### `docs/integrations/seriatim.md`
|
|
||||||
|
|
||||||
- Audience: developers, LLM coding agents.
|
|
||||||
- Purpose: Seriatim minimal transcript input contract.
|
|
||||||
- Canonical scope: accepted JSON shape, validation, source mapping, metadata
|
|
||||||
keys, capabilities, and compatibility limits.
|
|
||||||
- Recommended outline: adapter; accepted shape; validation; source mapping;
|
|
||||||
metadata keys; capabilities; example; limits.
|
|
||||||
- Source-of-truth areas: `internal/modules/input/seriatim/*.go`,
|
|
||||||
`internal/modules/input/seriatim/*_test.go`,
|
|
||||||
`examples/seriatim-minimal-transcript.json`.
|
|
||||||
- Acceptance criteria: matches implemented validation exactly; clearly states
|
|
||||||
only the minimal Seriatim shape is supported.
|
|
||||||
|
|
||||||
### `docs/integrations/openai-compatible.md`
|
|
||||||
|
|
||||||
- Audience: administrators, developers, LLM coding agents.
|
|
||||||
- Purpose: external LLM provider wire contract currently used by Notarius.
|
|
||||||
- Canonical scope: OpenAI-compatible chat completions endpoint assumptions,
|
|
||||||
structured output response format, request/response fields used, retry
|
|
||||||
behavior, and secret handling.
|
|
||||||
- Recommended outline: provider type; base URL handling; endpoint path;
|
|
||||||
request shape summary; response assumptions; retryable statuses; auth header;
|
|
||||||
configuration links; limits.
|
|
||||||
- Source-of-truth areas: `internal/framework/llm/openai_compatible_client.go`,
|
|
||||||
`internal/framework/llm/openai_compatible_client_test.go`,
|
|
||||||
`internal/core/config/effective_config.go`.
|
|
||||||
- Acceptance criteria: documents only what the client implements; avoids
|
|
||||||
copying external API docs wholesale; links to config docs for settings.
|
|
||||||
|
|
||||||
### `docs/integrations/json-output.md`
|
|
||||||
|
|
||||||
- Audience: users, administrators, downstream developers, LLM coding agents.
|
|
||||||
- Purpose: durable JSON output file-format contract.
|
|
||||||
- Canonical scope: output directory contents and JSON file shapes produced by
|
|
||||||
the `json` output module.
|
|
||||||
- Recommended outline: output encoder; file list; `index.json`;
|
|
||||||
`manifest.json`; `artifacts/<type>.json`; `rejected.json`; `warnings.json`;
|
|
||||||
artifact envelope; filename sanitization; limits.
|
|
||||||
- Source-of-truth areas: `internal/modules/output/json/encoder.go`,
|
|
||||||
`internal/core/artifacts/artifacts.go`, output tests,
|
|
||||||
`internal/cli/run_test.go`.
|
|
||||||
- Acceptance criteria: exactly matches implemented file names and shapes; does
|
|
||||||
not document future output formats.
|
|
||||||
|
|
||||||
### `docs/integrations/dnd-spell-artifacts.md`
|
|
||||||
|
|
||||||
- Audience: downstream developers, LLM coding agents.
|
|
||||||
- Purpose: D&D spell-cast artifact payload contract.
|
|
||||||
- Canonical scope: durable `dnd.spell_cast` payload fields, schema version,
|
|
||||||
source-reference expectations, prompt/schema provenance in manifests, and
|
|
||||||
validator rejection reasons.
|
|
||||||
- Recommended outline: artifact type; payload fields; source references;
|
|
||||||
validators; manifest metadata; example payload; limits.
|
|
||||||
- Source-of-truth areas: `internal/modules/extract/dnd/spells/*.go`,
|
|
||||||
`internal/modules/extract/dnd/spells/assets/schemas/dnd_spells.v1.json`,
|
|
||||||
spell extractor tests.
|
|
||||||
- Acceptance criteria: replaces the external-contract parts of
|
|
||||||
`docs/integrations/dnd-spells.md`; no stale statement that CLI run is missing;
|
|
||||||
no item/NPC/combat claims outside limitations.
|
|
||||||
|
|
||||||
### `docs/roadmap/mvp.md`
|
|
||||||
|
|
||||||
- Audience: maintainers, developers, LLM coding agents.
|
|
||||||
- Purpose: remaining roadmap work only, if any remains after the documentation
|
|
||||||
migration.
|
|
||||||
- Canonical scope: future, planned, deferred, or unimplemented work.
|
|
||||||
- Recommended outline: remaining future work; deferred ideas; release checklist,
|
|
||||||
if still useful.
|
|
||||||
- Source-of-truth areas: current code and this documentation roadmap.
|
|
||||||
- Acceptance criteria: contains no current-behavior reference material that
|
|
||||||
belongs in README, CLI, config, operations, internal, or integration docs.
|
|
||||||
|
|
||||||
### `examples/dnd-spells.config.yml`
|
|
||||||
|
|
||||||
- Audience: users, administrators, operators, LLM coding agents.
|
|
||||||
- Purpose: maintained minimal config example for the implemented spell
|
|
||||||
extraction workflow.
|
|
||||||
- Canonical scope: one valid, secret-free config example.
|
|
||||||
- Recommended outline: YAML fixture only, with comments only if they do not
|
|
||||||
interfere with tests.
|
|
||||||
- Source-of-truth areas: config parser/tests and CLI example-fixture tests.
|
|
||||||
- Acceptance criteria: remains loadable by tests; contains no secrets; is linked
|
|
||||||
from README, CLI, and config docs.
|
|
||||||
|
|
||||||
### `examples/seriatim-minimal-transcript.json`
|
|
||||||
|
|
||||||
- Audience: users, developers, LLM coding agents.
|
|
||||||
- Purpose: maintained minimal input example for the Seriatim adapter.
|
|
||||||
- Canonical scope: one valid minimal transcript JSON fixture.
|
|
||||||
- Recommended outline: JSON fixture only.
|
|
||||||
- Source-of-truth areas: Seriatim adapter/tests and CLI example-fixture tests.
|
|
||||||
- Acceptance criteria: remains valid; contains no private transcript data; is
|
|
||||||
linked from README, CLI, and Seriatim integration docs.
|
|
||||||
|
|
||||||
## File-by-File Rewrite Guidance
|
|
||||||
|
|
||||||
- `README.md`: cover what Notarius does, the shortest useful command, where
|
|
||||||
outputs go, and links. Avoid package diagrams, full config reference, and
|
|
||||||
roadmap content. Inspect `internal/cli/run.go`, `examples/`, and CLI tests.
|
|
||||||
Do not carry forward `go-application-template`.
|
|
||||||
- `docs/cli.md`: cover implemented commands and flags. Avoid full YAML field
|
|
||||||
reference and internal package details. Link to `docs/config.md` for config
|
|
||||||
fields and `docs/operations.md` for output/diagnostics.
|
|
||||||
- `docs/config.md`: cover config schema, defaults, discovery, env overrides,
|
|
||||||
validation, and secrets. Avoid command walkthroughs except minimal examples.
|
|
||||||
Link to `docs/cli.md` for command syntax and integration docs for module
|
|
||||||
contracts.
|
|
||||||
- `docs/operations.md`: cover output and diagnostics behavior, retention, and
|
|
||||||
failure inspection. Avoid developer implementation details that belong in
|
|
||||||
`docs/internal/diagnostics.md`.
|
|
||||||
- `docs/troubleshooting.md`: cover recurring implemented failures. Avoid generic
|
|
||||||
LLM advice not grounded in Notarius errors or tests.
|
|
||||||
- `docs/policy/development.md`: cover contributor workflow. Avoid copying
|
|
||||||
`docs/policy/architecture.md`; link to it for invariants.
|
|
||||||
- `docs/internal/overview.md`: cover package roles and boundaries. Avoid
|
|
||||||
user-facing command details.
|
|
||||||
- `docs/internal/pipeline.md`: cover runner and registry mechanics. Avoid config
|
|
||||||
syntax beyond linking to the canonical config doc.
|
|
||||||
- `docs/internal/modules.md`: cover implemented modules and how they fit stage
|
|
||||||
contracts. Avoid external Seriatim schema detail and durable JSON output
|
|
||||||
schemas beyond links to integration docs.
|
|
||||||
- `docs/internal/llm.md`: cover runtime internals. Avoid claiming support for
|
|
||||||
non-implemented providers.
|
|
||||||
- `docs/internal/diagnostics.md`: cover diagnostics implementation. Avoid
|
|
||||||
operator runbooks; link to operations.
|
|
||||||
- `docs/integrations/seriatim.md`: keep focused on Seriatim minimal transcript
|
|
||||||
JSON. Avoid D&D extractor behavior.
|
|
||||||
- `docs/integrations/openai-compatible.md`: document the implemented external
|
|
||||||
LLM contract. Avoid extensive copied provider docs.
|
|
||||||
- `docs/integrations/json-output.md`: document durable output files. Avoid
|
|
||||||
diagnostics run-directory details except a link to operations.
|
|
||||||
- `docs/integrations/dnd-spell-artifacts.md`: document the spell artifact
|
|
||||||
contract. Avoid extractor implementation details better covered internally.
|
|
||||||
- `docs/integrations/dnd-spells.md`: replace by split docs above, then delete or
|
|
||||||
leave a temporary redirect-style note only if that is useful during migration.
|
|
||||||
- `docs/roadmap/*.md`: remove completed implementation history or rewrite it as
|
|
||||||
future work. Do not leave implemented behavior solely in roadmap docs.
|
|
||||||
|
|
||||||
## Examples Plan
|
|
||||||
|
|
||||||
`examples/` exists and should remain because Notarius has non-trivial
|
|
||||||
configuration and workflows.
|
|
||||||
|
|
||||||
- `examples/dnd-spells.config.yml`
|
|
||||||
- Purpose: maintained minimal D&D spell extraction pipeline config.
|
|
||||||
- Expected validity check: `go test ./internal/cli` currently validates this
|
|
||||||
fixture with `config validate`, `pipelines list`, and fake-LLM `run`.
|
|
||||||
- Documentation links: README, `docs/cli.md`, `docs/config.md`,
|
|
||||||
`docs/integrations/seriatim.md`, `docs/integrations/dnd-spell-artifacts.md`.
|
|
||||||
- Update guidance: keep secret-free; prefer environment-based API key
|
|
||||||
behavior; preserve test coverage.
|
|
||||||
|
|
||||||
- `examples/seriatim-minimal-transcript.json`
|
|
||||||
- Purpose: maintained minimal Seriatim transcript input.
|
|
||||||
- Expected validity check: `go test ./internal/cli` uses it in the fake-LLM run
|
|
||||||
acceptance tests; Seriatim adapter tests cover compatible shapes separately.
|
|
||||||
- Documentation links: README, `docs/cli.md`,
|
|
||||||
`docs/integrations/seriatim.md`.
|
|
||||||
|
|
||||||
Do not add an expected-output example unless it can be kept in sync by tests or
|
|
||||||
a small documented regeneration procedure. Do not invent examples for NPC, item,
|
|
||||||
combat, Markdown, or Obsidian extraction until those features exist.
|
|
||||||
|
|
||||||
## Internal Documentation Plan
|
|
||||||
|
|
||||||
### Pipeline Orchestration
|
|
||||||
|
|
||||||
- Path: `docs/internal/pipeline.md`
|
|
||||||
- Purpose: explain implemented runner orchestration.
|
|
||||||
- Inputs and outputs: `pipeline.RunInput`, `pipeline.RunOutput`,
|
|
||||||
`contracts.*Request`, `contracts.*Result`, `artifacts.RunManifest`.
|
|
||||||
- Boundaries: orchestration and registries only; no source-format or D&D logic.
|
|
||||||
- Config fields used: resolved pipeline bindings and lane selection from
|
|
||||||
`internal/core/config`.
|
|
||||||
- Adapters used: input, chunk, extract, merge, normalize, validator, output
|
|
||||||
registries.
|
|
||||||
- Failure behavior: stage errors wrap module keys/lane/chunk context; partial
|
|
||||||
manifest may be returned on failure.
|
|
||||||
- Tests to inspect before changing: `internal/framework/pipeline/*_test.go`,
|
|
||||||
`internal/framework/pipeline/walking_skeleton_test.go`,
|
|
||||||
`internal/cli/run_test.go`.
|
|
||||||
- Architectural invariants: fixed six-stage flow; generic source references;
|
|
||||||
module selection via registries.
|
|
||||||
|
|
||||||
### Stage Modules
|
|
||||||
|
|
||||||
- Path: `docs/internal/modules.md`
|
|
||||||
- Purpose: summarize implemented production modules and extension pattern.
|
|
||||||
- Inputs and outputs: module-specific contract methods under
|
|
||||||
`internal/framework/contracts`.
|
|
||||||
- Boundaries: module logic under `internal/modules/<stage>/...`; CLI only wires
|
|
||||||
production modules.
|
|
||||||
- Config fields used: module bindings, `llm_profile`, and `options`.
|
|
||||||
- Adapters used: Seriatim input, generic chunker, D&D spells extractor,
|
|
||||||
append-order merger, noop normalizer, JSON output encoder.
|
|
||||||
- Failure behavior: module constructors and stage methods return contextual
|
|
||||||
errors; validators reject candidates rather than panicking.
|
|
||||||
- Tests to inspect before changing: module package tests and
|
|
||||||
`internal/cli/catalog.go` coverage in `internal/cli/run_test.go`.
|
|
||||||
- Architectural invariants: source/domain concerns stay in modules; framework
|
|
||||||
remains source- and domain-agnostic.
|
|
||||||
|
|
||||||
### LLM Runtime
|
|
||||||
|
|
||||||
- Path: `docs/internal/llm.md`
|
|
||||||
- Purpose: explain structured LLM execution internals.
|
|
||||||
- Inputs and outputs: `StructuredCompletionRequest`,
|
|
||||||
`StructuredCompletionResponse`, schema registry entries, scheduler calls.
|
|
||||||
- Boundaries: provider wire details stay in `internal/framework/llm`; prompts
|
|
||||||
and schemas are owned by modules or generic prompt helpers.
|
|
||||||
- Config fields used: `llm_profiles`, timeout, retries, concurrency, API key
|
|
||||||
environment handling.
|
|
||||||
- Adapters used: OpenAI-compatible HTTP client and scheduler.
|
|
||||||
- Failure behavior: validation errors before calls; retryable provider errors;
|
|
||||||
redacted error messages.
|
|
||||||
- Tests to inspect before changing: `internal/framework/llm/*_test.go`,
|
|
||||||
`internal/framework/prompt/*_test.go`, spell prompt/schema tests.
|
|
||||||
- Architectural invariants: no secrets in errors/diagnostics; no domain prompt
|
|
||||||
logic in provider adapters.
|
|
||||||
|
|
||||||
### Diagnostics
|
|
||||||
|
|
||||||
- Path: `docs/internal/diagnostics.md`
|
|
||||||
- Purpose: explain diagnostics implementation.
|
|
||||||
- Inputs and outputs: invocation metadata, redacted effective config, resolved
|
|
||||||
pipeline, run manifest, warnings, run report, error log.
|
|
||||||
- Boundaries: diagnostics writes inspection artifacts; durable user output is
|
|
||||||
written by CLI/output modules.
|
|
||||||
- Config fields used: diagnostics work dir and retention mode.
|
|
||||||
- Adapters used: none external.
|
|
||||||
- Failure behavior: diagnostics write failures fail the CLI command with
|
|
||||||
context; retention removes or preserves the run directory based on result and
|
|
||||||
warnings.
|
|
||||||
- Tests to inspect before changing: `internal/core/diagnostics/*_test.go`,
|
|
||||||
CLI diagnostics tests.
|
|
||||||
- Architectural invariants: diagnostics must not expose secrets; cleanup must
|
|
||||||
use narrow paths.
|
|
||||||
|
|
||||||
## Integration Documentation Plan
|
|
||||||
|
|
||||||
### Seriatim Minimal Transcript JSON
|
|
||||||
|
|
||||||
- Path: `docs/integrations/seriatim.md`
|
|
||||||
- External system or contract: Seriatim minimal transcript JSON.
|
|
||||||
- Current usage: parsed by the `seriatim` input adapter into a generic source
|
|
||||||
document.
|
|
||||||
- Version or compatibility notes: repository currently supports only the minimal
|
|
||||||
shape implemented by `internal/modules/input/seriatim`.
|
|
||||||
- What should be documented: accepted fields, validation, source mapping,
|
|
||||||
metadata keys, capabilities, example.
|
|
||||||
- What should not be documented: future Seriatim variants or D&D extraction
|
|
||||||
behavior.
|
|
||||||
|
|
||||||
### OpenAI-Compatible Structured Output
|
|
||||||
|
|
||||||
- Path: `docs/integrations/openai-compatible.md`
|
|
||||||
- External system or contract: OpenAI-compatible chat completions API with
|
|
||||||
`response_format.type=json_schema`.
|
|
||||||
- Current usage: production LLM client posts to `<base_url>/chat/completions`
|
|
||||||
and decodes assistant JSON content.
|
|
||||||
- Version or compatibility notes: compatibility is defined by the implemented
|
|
||||||
request and response fields in `internal/framework/llm`.
|
|
||||||
- What should be documented: endpoint construction, auth header behavior,
|
|
||||||
request shape summary, response assumptions, retry behavior, config links.
|
|
||||||
- What should not be documented: comprehensive provider API reference or
|
|
||||||
unsupported providers.
|
|
||||||
|
|
||||||
### Durable JSON Output
|
|
||||||
|
|
||||||
- Path: `docs/integrations/json-output.md`
|
|
||||||
- External system or contract: file-format contract for Notarius JSON output.
|
|
||||||
- Current usage: produced by the `json` output encoder and written by the CLI.
|
|
||||||
- Version or compatibility notes: manifest includes schema version and resolved
|
|
||||||
pipeline provenance; exact envelope versioning should match implemented
|
|
||||||
fields.
|
|
||||||
- What should be documented: file names, JSON shapes, artifact grouping, source
|
|
||||||
references, rejected artifacts, warnings, filename sanitization.
|
|
||||||
- What should not be documented: future output formats or remote storage.
|
|
||||||
|
|
||||||
### D&D Spell-Cast Artifacts
|
|
||||||
|
|
||||||
- Path: `docs/integrations/dnd-spell-artifacts.md`
|
|
||||||
- External system or contract: durable `dnd.spell_cast` artifact payload and
|
|
||||||
validation contract.
|
|
||||||
- Current usage: produced by the `dnd/spells` extractor and serialized by JSON
|
|
||||||
output.
|
|
||||||
- Version or compatibility notes: artifact type `dnd.spell_cast`, schema version
|
|
||||||
`v1`; response schema asset `dnd_spells.v1.json` is internal to extraction.
|
|
||||||
- What should be documented: payload fields, source references, rejection reason
|
|
||||||
codes, prompt/schema manifest metadata, limits.
|
|
||||||
- What should not be documented: unimplemented D&D item, NPC, combat, or rules
|
|
||||||
extraction.
|
|
||||||
|
|
||||||
## Recommended Implementation Sequence
|
|
||||||
|
|
||||||
### Stage 1: Root Orientation And Core References
|
|
||||||
|
|
||||||
- Goal: make the project immediately understandable and add required public
|
|
||||||
references.
|
|
||||||
- Files to create/update/delete/move: rewrite `README.md`; create
|
|
||||||
`docs/cli.md`; create `docs/config.md`; update examples only if needed for
|
|
||||||
clarity and tests.
|
|
||||||
- Repository areas to inspect: `internal/cli/run.go`, `internal/cli/run_test.go`,
|
|
||||||
`internal/core/config`, `examples/`.
|
|
||||||
- Acceptance criteria: README is concise; CLI and config docs are canonical and
|
|
||||||
describe only implemented behavior; maintained example commands match tests.
|
|
||||||
- Suggested validation commands: `go test ./internal/cli ./internal/core/config`;
|
|
||||||
`go test ./...` if examples changed.
|
|
||||||
- One prompt? Yes, if kept focused on README, CLI, and config only.
|
|
||||||
|
|
||||||
### Stage 2: Operations And Troubleshooting
|
|
||||||
|
|
||||||
- Goal: document durable output, diagnostics, retention, and common recovery
|
|
||||||
paths.
|
|
||||||
- Files to create/update/delete/move: create `docs/operations.md`; create
|
|
||||||
`docs/troubleshooting.md`.
|
|
||||||
- Repository areas to inspect: `internal/cli/run.go`,
|
|
||||||
`internal/core/diagnostics`, `internal/modules/output/json`,
|
|
||||||
`internal/framework/llm`, relevant tests.
|
|
||||||
- Acceptance criteria: operations doc names implemented output and diagnostics
|
|
||||||
files; troubleshooting entries map to implemented errors; no remote storage,
|
|
||||||
resume command, or archive behavior is documented.
|
|
||||||
- Suggested validation commands: `go test ./internal/cli ./internal/core/diagnostics ./internal/modules/output/json ./internal/framework/llm`.
|
|
||||||
- One prompt? Yes.
|
|
||||||
|
|
||||||
### Stage 3: Developer Policy And Internal Docs
|
|
||||||
|
|
||||||
- Goal: add required developer workflow and internal component documentation.
|
|
||||||
- Files to create/update/delete/move: create `docs/policy/development.md`;
|
|
||||||
create `docs/internal/overview.md`; create `docs/internal/pipeline.md`;
|
|
||||||
create `docs/internal/modules.md`; create `docs/internal/llm.md`; create
|
|
||||||
`docs/internal/diagnostics.md`.
|
|
||||||
- Repository areas to inspect: `AGENTS.md`, `go.mod`, `docs/policy`,
|
|
||||||
`internal/framework`, `internal/core`, `internal/modules`,
|
|
||||||
`internal/cli/catalog.go`.
|
|
||||||
- Acceptance criteria: docs preserve architecture-policy boundaries; internal
|
|
||||||
docs cover implemented components only; development doc gives actionable
|
|
||||||
commands and contribution patterns without duplicating reference docs.
|
|
||||||
- Suggested validation commands: `go test ./...`; `go vet ./...`;
|
|
||||||
`go build ./cmd/notarius`.
|
|
||||||
- One prompt? Possibly too large. If needed, split into Stage 3A
|
|
||||||
(`development.md` and `overview.md`) and Stage 3B (component docs).
|
|
||||||
|
|
||||||
### Stage 4: Integration And Artifact Contracts
|
|
||||||
|
|
||||||
- Goal: place external and durable file-format contracts in canonical
|
|
||||||
integration docs.
|
|
||||||
- Files to create/update/delete/move: update `docs/integrations/seriatim.md`;
|
|
||||||
create `docs/integrations/openai-compatible.md`; create
|
|
||||||
`docs/integrations/json-output.md`; create
|
|
||||||
`docs/integrations/dnd-spell-artifacts.md`; delete or replace
|
|
||||||
`docs/integrations/dnd-spells.md` after its content is split.
|
|
||||||
- Repository areas to inspect: `internal/modules/input/seriatim`,
|
|
||||||
`internal/framework/llm`, `internal/modules/output/json`,
|
|
||||||
`internal/core/artifacts`, `internal/modules/extract/dnd/spells`.
|
|
||||||
- Acceptance criteria: integration docs document actual external/file-format
|
|
||||||
contracts; no stale claim that CLI run is unimplemented; D&D details are kept
|
|
||||||
out of generic docs.
|
|
||||||
- Suggested validation commands: `go test ./internal/modules/input/seriatim ./internal/framework/llm ./internal/modules/output/json ./internal/modules/extract/dnd/spells`.
|
|
||||||
- One prompt? Yes.
|
|
||||||
|
|
||||||
### Stage 5: Roadmap Cleanup And Link Verification
|
|
||||||
|
|
||||||
- Goal: remove completed planning clutter and verify the final documentation
|
|
||||||
tree.
|
|
||||||
- Files to create/update/delete/move: delete completed roadmap files or reduce
|
|
||||||
them to genuine future work; update links across README and docs; keep
|
|
||||||
`docs/roadmap/documentation.md` only if remaining documentation work exists.
|
|
||||||
- Repository areas to inspect: all docs, examples, roadmap files.
|
|
||||||
- Acceptance criteria: future/planned behavior appears only under
|
|
||||||
`docs/roadmap/`; completed MVP implementation details are documented in
|
|
||||||
canonical current-behavior docs; links resolve; no stale template text remains.
|
|
||||||
- Suggested validation commands: `go test ./...`; `go vet ./...`;
|
|
||||||
`go build ./cmd/notarius`; grep checks from the validation plan.
|
|
||||||
- One prompt? Yes.
|
|
||||||
|
|
||||||
## Validation Plan
|
|
||||||
|
|
||||||
Run these checks during or after implementation:
|
|
||||||
|
|
||||||
- `go test ./...`
|
|
||||||
- `go vet ./...`
|
|
||||||
- `go build ./cmd/notarius`
|
|
||||||
- `go test ./internal/cli` after CLI docs or examples change.
|
|
||||||
- `go test ./internal/core/config` after config docs or examples change.
|
|
||||||
- `go test ./internal/modules/input/seriatim` after Seriatim docs/examples
|
|
||||||
change.
|
|
||||||
- `go test ./internal/modules/extract/dnd/spells` after spell artifact docs
|
|
||||||
change.
|
|
||||||
- `go test ./internal/modules/output/json` after JSON output docs change.
|
|
||||||
- `go test ./internal/framework/llm` after OpenAI-compatible docs change.
|
|
||||||
|
|
||||||
No automated documentation link checker currently exists in the repository.
|
|
||||||
Future implementation agents should manually verify Markdown links or add a
|
|
||||||
tool only if the project adopts one deliberately.
|
|
||||||
|
|
||||||
Recommended grep checks:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
rg -n 'go-application-template|does not yet|not yet implemented|provisional|Likely future|CLI `run` workflow' README.md docs examples --glob '!docs/policy/**' --glob '!docs/roadmap/**'
|
|
||||||
rg -n 'NPC|item|combat|Obsidian|Markdown' README.md docs examples --glob '!docs/roadmap/**'
|
|
||||||
rg -n 'TODO|FIXME' README.md docs examples
|
|
||||||
```
|
|
||||||
|
|
||||||
Manual review items:
|
|
||||||
|
|
||||||
- Confirm README remains short and orientation-focused.
|
|
||||||
- Confirm `docs/cli.md` contains every implemented command and flag from
|
|
||||||
`internal/cli/run.go`.
|
|
||||||
- Confirm `docs/config.md` contains every implemented file config field and env
|
|
||||||
override from `internal/core/config`.
|
|
||||||
- Confirm examples are secret-free and covered by tests where practical.
|
|
||||||
- Confirm roadmap docs do not masquerade as current-behavior docs.
|
|
||||||
- Confirm no docs mention unimplemented NPC, item, combat, Markdown, Obsidian,
|
|
||||||
remote storage, resume, or DAG behavior except under roadmap limitations or
|
|
||||||
explicit out-of-scope notes.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None. The repository evidence is sufficient to plan the documentation migration.
|
|
||||||
@@ -1,888 +0,0 @@
|
|||||||
# Implementation Plan: MVP
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
This is the staged implementation plan for the active MVP roadmap:
|
|
||||||
[`mvp.md`](mvp.md).
|
|
||||||
|
|
||||||
The target audience is an LLM coding agent. Implement the stages in order.
|
|
||||||
Each stage should leave the repository compiling and tested. Do not skip ahead
|
|
||||||
to later stages unless the current stage's done criteria are satisfied.
|
|
||||||
|
|
||||||
## Policy Context
|
|
||||||
|
|
||||||
Follow:
|
|
||||||
|
|
||||||
- [`../policy/architecture.md`](../policy/architecture.md)
|
|
||||||
- [`../policy/documentation.md`](../policy/documentation.md)
|
|
||||||
- [`mvp.md`](mvp.md)
|
|
||||||
- [`initial-architecture.md`](initial-architecture.md)
|
|
||||||
|
|
||||||
Required boundaries:
|
|
||||||
|
|
||||||
- framework packages must remain source-agnostic and domain-agnostic;
|
|
||||||
- source-format behavior belongs in input modules;
|
|
||||||
- D&D spell behavior, prompt assets, response schema assets, and stable
|
|
||||||
prompt/schema identifiers belong in `internal/modules/extract/dnd/spells`;
|
|
||||||
- stage business logic belongs under `internal/modules/<stage>/...` unless it
|
|
||||||
is genuinely tiny shared framework plumbing;
|
|
||||||
- structural pipeline selection must remain config-driven;
|
|
||||||
- `--only` may select artifact lanes but must not alter pipeline structure;
|
|
||||||
- output-stage warnings are out-of-band from artifact payloads and must be
|
|
||||||
available to CLI/diagnostics;
|
|
||||||
- keep planned documentation in `docs/roadmap/` until MVP behavior exists.
|
|
||||||
|
|
||||||
## Global Implementation Decisions
|
|
||||||
|
|
||||||
- Add no new third-party dependencies.
|
|
||||||
- Keep YAML config version `1` unless a user-visible config syntax change is
|
|
||||||
unavoidable. New module options can use existing binding `options`.
|
|
||||||
- Use the existing six-stage workflow:
|
|
||||||
`input -> chunk -> extract -> merge -> normalize -> output`.
|
|
||||||
- Use production CLI wiring in `internal/cli` for the MVP instead of adding a
|
|
||||||
new app package. The CLI may compose modules, but it must not own module
|
|
||||||
business logic.
|
|
||||||
- Keep `notarius run` serial over chunks for the MVP. The contracts and LLM
|
|
||||||
scheduler should still permit later parallel execution.
|
|
||||||
- Use one effective LLM profile per MVP run. The current runner accepts one
|
|
||||||
`StructuredLLMClient`, so a selected pipeline with multiple distinct effective
|
|
||||||
LLM profile IDs should fail clearly until multi-client runtime support is
|
|
||||||
intentionally added.
|
|
||||||
- Use the existing OpenAI-compatible client for real runs.
|
|
||||||
- Add a scheduled LLM client wrapper so every structured completion passes
|
|
||||||
through the configured scheduler.
|
|
||||||
- Use the existing `seriatim` input module and `dnd/spells` extractor module.
|
|
||||||
- Implement production default modules with these keys:
|
|
||||||
- `generic` chunker;
|
|
||||||
- `appendorder` merger;
|
|
||||||
- `noop` normalizer;
|
|
||||||
- `json` output encoder.
|
|
||||||
- Put production default modules under:
|
|
||||||
- `internal/modules/chunk/generic`;
|
|
||||||
- `internal/modules/merge/appendorder`;
|
|
||||||
- `internal/modules/normalize/noop`;
|
|
||||||
- `internal/modules/output/json`.
|
|
||||||
- The output encoder should return logical output files; the CLI/application
|
|
||||||
layer should write those files to disk. Encoders should not own filesystem
|
|
||||||
side effects.
|
|
||||||
- Use an output directory per run. The MVP default output root should be
|
|
||||||
`./notarius-output`, overrideable by `--output-dir`.
|
|
||||||
- File writes for durable output should be atomic where practical: write to a
|
|
||||||
temporary file in the target directory, then rename.
|
|
||||||
- Use synthetic fixtures only. Do not add private campaign transcript content,
|
|
||||||
real API keys, or private infrastructure values.
|
|
||||||
|
|
||||||
## Stage 1: Move D&D Prompt And Schema Assets Into The Spells Module
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Restore the intended framework/domain boundary before building additional MVP
|
|
||||||
functionality.
|
|
||||||
|
|
||||||
`internal/framework/llm` and `internal/framework/prompt` should provide generic
|
|
||||||
asset loading, metadata, and rendering primitives. They must not define
|
|
||||||
D&D-specific prompt IDs, response schema keys, asset paths, or tests.
|
|
||||||
|
|
||||||
### Files To Update Or Move
|
|
||||||
|
|
||||||
Expected files:
|
|
||||||
|
|
||||||
- `internal/framework/llm/schema_registry.go`
|
|
||||||
- `internal/framework/llm/schema_registry_test.go`
|
|
||||||
- `internal/framework/llm/assets/schemas/dnd_spells.v1.json`
|
|
||||||
- `internal/framework/prompt/registry.go`
|
|
||||||
- `internal/framework/prompt/render.go`
|
|
||||||
- `internal/framework/prompt/render_test.go`
|
|
||||||
- `internal/framework/prompt/assets/dnd/spells/system.md`
|
|
||||||
- `internal/framework/prompt/assets/dnd/spells/user.md`
|
|
||||||
- `internal/modules/extract/dnd/spells/extractor.go`
|
|
||||||
- `internal/modules/extract/dnd/spells/prompt.go`
|
|
||||||
- `internal/modules/extract/dnd/spells/prompt_test.go`
|
|
||||||
- `internal/modules/extract/dnd/spells/schema_test.go`
|
|
||||||
- `docs/integrations/dnd-spells.md`, only if prompt/schema ownership text needs
|
|
||||||
to be corrected.
|
|
||||||
|
|
||||||
### Required Design
|
|
||||||
|
|
||||||
Refactor `internal/framework/llm` so it can load schemas from caller-owned
|
|
||||||
embedded files.
|
|
||||||
|
|
||||||
Add or expose a generic constructor similar to:
|
|
||||||
|
|
||||||
```go
|
|
||||||
func LoadResponseSchema(fsys fs.FS, def ResponseSchemaDefinition) (ResponseSchema, error)
|
|
||||||
```
|
|
||||||
|
|
||||||
where `ResponseSchemaDefinition` carries:
|
|
||||||
|
|
||||||
- key;
|
|
||||||
- ID;
|
|
||||||
- version;
|
|
||||||
- name;
|
|
||||||
- asset path.
|
|
||||||
|
|
||||||
The existing framework registry may keep test schemas, but it must not include
|
|
||||||
`DNDSpellsSchemaKey` or `dnd_spells.v1.json`.
|
|
||||||
|
|
||||||
Refactor `internal/framework/prompt` so it can compile/render prompt pairs from
|
|
||||||
caller-owned embedded files.
|
|
||||||
|
|
||||||
Add or expose a generic constructor/rendering type similar to:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type Bundle struct { ... }
|
|
||||||
func LoadBundle(fsys fs.FS, def Definition) (*Bundle, error)
|
|
||||||
func (b *Bundle) RenderUserSystem(data any) (system string, user string, metadata Metadata, err error)
|
|
||||||
```
|
|
||||||
|
|
||||||
The framework prompt package may continue to own shared hardening text if that
|
|
||||||
is useful, but it must not include `DNDSpellsPromptID` or D&D prompt paths.
|
|
||||||
|
|
||||||
Move D&D prompt and schema assets under `internal/modules/extract/dnd/spells`.
|
|
||||||
Recommended paths:
|
|
||||||
|
|
||||||
```text
|
|
||||||
internal/modules/extract/dnd/spells/assets/prompts/system.md
|
|
||||||
internal/modules/extract/dnd/spells/assets/prompts/user.md
|
|
||||||
internal/modules/extract/dnd/spells/assets/schemas/dnd_spells.v1.json
|
|
||||||
```
|
|
||||||
|
|
||||||
The spells package should define its own stable identifiers:
|
|
||||||
|
|
||||||
```go
|
|
||||||
const PromptID = "dnd.spells"
|
|
||||||
const ResponseSchemaKey = "dnd_spells"
|
|
||||||
const ResponseSchemaID = "notarius.dnd.spells"
|
|
||||||
const ResponseSchemaName = "notarius_dnd_spells_v1"
|
|
||||||
```
|
|
||||||
|
|
||||||
The spells extractor must call module-owned prompt/schema helpers and pass only
|
|
||||||
generic framework values into the LLM client.
|
|
||||||
|
|
||||||
### Required Tests
|
|
||||||
|
|
||||||
- Framework LLM schema tests prove test schemas still load, sort, clone, and
|
|
||||||
omit raw schema content from diagnostics.
|
|
||||||
- Framework LLM schema tests prove looking up `dnd_spells` in the framework
|
|
||||||
registry fails.
|
|
||||||
- Framework prompt tests prove test prompts still render and missing template
|
|
||||||
data still errors.
|
|
||||||
- Framework prompt tests contain no D&D prompt assertions.
|
|
||||||
- Spells package schema tests load the module-owned D&D schema and verify:
|
|
||||||
- key;
|
|
||||||
- ID;
|
|
||||||
- version;
|
|
||||||
- response schema name;
|
|
||||||
- valid JSON;
|
|
||||||
- clone/mutation safety;
|
|
||||||
- diagnostics omit raw schema content.
|
|
||||||
- Spells package prompt tests render the module-owned prompt and verify
|
|
||||||
hardening text and prompt metadata.
|
|
||||||
- Existing spells extractor tests still pass without importing framework-owned
|
|
||||||
D&D constants.
|
|
||||||
|
|
||||||
### Validation
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
gofmt -w internal/framework/llm internal/framework/prompt internal/modules/extract/dnd/spells
|
|
||||||
go test ./internal/framework/llm ./internal/framework/prompt ./internal/modules/extract/dnd/spells
|
|
||||||
go test ./...
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 2: Add MVP Manifest And Logical Output File Contracts
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Make output and manifest contracts capable of representing the MVP's durable
|
|
||||||
run output before implementing the production JSON encoder or CLI writing.
|
|
||||||
|
|
||||||
### Files To Update
|
|
||||||
|
|
||||||
Expected files:
|
|
||||||
|
|
||||||
- `internal/framework/contracts/contracts.go`
|
|
||||||
- `internal/core/artifacts/*.go`
|
|
||||||
- `internal/framework/pipeline/runner.go`
|
|
||||||
- `internal/framework/pipeline/runner_test.go`
|
|
||||||
- `internal/framework/contracts/contracts_test.go`
|
|
||||||
- `docs/policy/architecture.md`, only if the implemented output contract
|
|
||||||
requires clarifying policy text.
|
|
||||||
|
|
||||||
### Required Design
|
|
||||||
|
|
||||||
Extend the output contract to support logical files:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type OutputFile struct {
|
|
||||||
Name string `json:"name"`
|
|
||||||
ContentType string `json:"content_type,omitempty"`
|
|
||||||
Bytes []byte `json:"-"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type OutputResult struct {
|
|
||||||
Files []OutputFile `json:"files,omitempty"`
|
|
||||||
Warnings []Warning `json:"warnings,omitempty"`
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Remove or stop using the single `OutputResult.Bytes` / `ContentType` path once
|
|
||||||
all tests are updated. If keeping those fields temporarily reduces churn, mark
|
|
||||||
them as legacy in comments and make the runner prefer `Files`.
|
|
||||||
|
|
||||||
Add matching fields to `pipeline.RunOutput`:
|
|
||||||
|
|
||||||
```go
|
|
||||||
OutputFiles []contracts.OutputFile `json:"-"`
|
|
||||||
```
|
|
||||||
|
|
||||||
The runner should collect output-stage warnings exactly as it does now, after
|
|
||||||
calling the output encoder.
|
|
||||||
|
|
||||||
Define safe logical file names:
|
|
||||||
|
|
||||||
- names are slash-separated relative paths;
|
|
||||||
- names must not be empty, absolute, contain `..`, or contain `\`;
|
|
||||||
- names are validated before the runner returns them;
|
|
||||||
- file names are sorted deterministically by the encoder that creates them.
|
|
||||||
|
|
||||||
Extend manifest data enough for MVP provenance:
|
|
||||||
|
|
||||||
- add `RunManifest.LLMProfiles []LLMProfileManifest`;
|
|
||||||
- add `ArtifactLaneManifest.Metadata map[string]any`;
|
|
||||||
- add `RunManifest.StartedAt`, `CompletedAt`, and `RunID` population support
|
|
||||||
in the runner input/output path.
|
|
||||||
|
|
||||||
Recommended structs:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type LLMProfileManifest struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Provider string `json:"provider,omitempty"`
|
|
||||||
Model string `json:"model,omitempty"`
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Add an optional metadata interface for modules:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type ManifestMetadataProvider interface {
|
|
||||||
ManifestMetadata() map[string]any
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
When a stage module implements the interface, the runner should include that
|
|
||||||
metadata in the appropriate manifest area. For the MVP, the D&D spells extractor
|
|
||||||
will use this to report prompt and response schema identifiers/hashes on its
|
|
||||||
artifact lane.
|
|
||||||
|
|
||||||
Update `pipeline.RunInput` to accept:
|
|
||||||
|
|
||||||
- `RunID string`;
|
|
||||||
- `StartedAt time.Time`;
|
|
||||||
- `LLMProfiles []artifacts.LLMProfileManifest`.
|
|
||||||
|
|
||||||
The runner should set `CompletedAt` when the run finishes or fails after a
|
|
||||||
manifest has been initialized.
|
|
||||||
|
|
||||||
### Required Tests
|
|
||||||
|
|
||||||
- `contracts.OutputFile` JSON shape omits bytes and includes name/content type.
|
|
||||||
- Runner rejects unsafe output file names returned by an encoder.
|
|
||||||
- Runner preserves output warnings out-of-band.
|
|
||||||
- Runner output contains logical files returned by the encoder.
|
|
||||||
- Manifest includes run ID, started/completed timestamps when supplied or
|
|
||||||
generated.
|
|
||||||
- Manifest includes LLM profile metadata supplied in `RunInput`.
|
|
||||||
- Manifest includes extractor-provided lane metadata when the extractor
|
|
||||||
implements `ManifestMetadataProvider`.
|
|
||||||
|
|
||||||
### Validation
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
gofmt -w internal/framework/contracts internal/core/artifacts internal/framework/pipeline
|
|
||||||
go test ./internal/framework/contracts ./internal/core/artifacts ./internal/framework/pipeline
|
|
||||||
go test ./...
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 3: Implement Production Default Stage Modules
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Make pipeline defaults real production modules instead of test-only fakes or
|
|
||||||
framework-only helpers.
|
|
||||||
|
|
||||||
### Files To Add Or Update
|
|
||||||
|
|
||||||
Expected packages:
|
|
||||||
|
|
||||||
- `internal/modules/chunk/generic`
|
|
||||||
- `internal/modules/merge/appendorder`
|
|
||||||
- `internal/modules/normalize/noop`
|
|
||||||
- `internal/modules/output/json`
|
|
||||||
|
|
||||||
Expected framework cleanup:
|
|
||||||
|
|
||||||
- `internal/framework/pipeline/generic_stages.go`
|
|
||||||
- pipeline tests that currently instantiate framework `AppendOrderMerger` or
|
|
||||||
`NoopNormalizer`.
|
|
||||||
|
|
||||||
### Required Design
|
|
||||||
|
|
||||||
#### `generic` chunker
|
|
||||||
|
|
||||||
Package: `internal/modules/chunk/generic`
|
|
||||||
|
|
||||||
Key: `generic`
|
|
||||||
|
|
||||||
Module spec:
|
|
||||||
|
|
||||||
- stage: `chunk`;
|
|
||||||
- requires: `source.transcript` is **not** required;
|
|
||||||
- provides: `chunks`.
|
|
||||||
|
|
||||||
Behavior:
|
|
||||||
|
|
||||||
- accepts any valid `SourceDocument`;
|
|
||||||
- preserves source-unit order;
|
|
||||||
- returns stable chunk IDs: `chunk-000001`, `chunk-000002`, and so on;
|
|
||||||
- copies source units defensively;
|
|
||||||
- adds chunk metadata:
|
|
||||||
- `start_unit_id`;
|
|
||||||
- `end_unit_id`;
|
|
||||||
- `unit_count`.
|
|
||||||
|
|
||||||
Options:
|
|
||||||
|
|
||||||
- `max_units`: positive integer, default `50`;
|
|
||||||
- `overlap_units`: non-negative integer, default `0`, must be less than
|
|
||||||
`max_units`.
|
|
||||||
|
|
||||||
If the source has no units, return a clear error. If options have the wrong type
|
|
||||||
or invalid values, return a clear module-specific error.
|
|
||||||
|
|
||||||
#### `appendorder` merger
|
|
||||||
|
|
||||||
Package: `internal/modules/merge/appendorder`
|
|
||||||
|
|
||||||
Key: `appendorder`
|
|
||||||
|
|
||||||
Module spec:
|
|
||||||
|
|
||||||
- stage: `merge`;
|
|
||||||
- requires: no artifact-type-specific capability;
|
|
||||||
- provides: `merged`.
|
|
||||||
|
|
||||||
Behavior:
|
|
||||||
|
|
||||||
- preserves chunk order as provided by the runner;
|
|
||||||
- preserves candidate order within each chunk;
|
|
||||||
- defensively copies candidates, payloads, source refs, and metadata;
|
|
||||||
- does not merge, deduplicate, or rewrite source references.
|
|
||||||
|
|
||||||
#### `noop` normalizer
|
|
||||||
|
|
||||||
Package: `internal/modules/normalize/noop`
|
|
||||||
|
|
||||||
Key: `noop`
|
|
||||||
|
|
||||||
Module spec:
|
|
||||||
|
|
||||||
- stage: `normalize`;
|
|
||||||
- requires: `merged`;
|
|
||||||
- provides: `normalized`.
|
|
||||||
|
|
||||||
Behavior:
|
|
||||||
|
|
||||||
- defensively copies candidates;
|
|
||||||
- does not deduplicate, rewrite, or validate domain content.
|
|
||||||
|
|
||||||
#### `json` output encoder
|
|
||||||
|
|
||||||
Package: `internal/modules/output/json`
|
|
||||||
|
|
||||||
Key: `json`
|
|
||||||
|
|
||||||
Module spec:
|
|
||||||
|
|
||||||
- stage: `output`;
|
|
||||||
- requires: `normalized`;
|
|
||||||
- provides: `encoded`.
|
|
||||||
|
|
||||||
Behavior:
|
|
||||||
|
|
||||||
- returns logical output files:
|
|
||||||
- `index.json`;
|
|
||||||
- `manifest.json`;
|
|
||||||
- `artifacts/<artifact_type>.json` for each approved artifact type;
|
|
||||||
- `rejected.json`;
|
|
||||||
- `warnings.json`.
|
|
||||||
- groups approved artifacts by `Artifact.ArtifactType`;
|
|
||||||
- sorts artifact-type file names by artifact type;
|
|
||||||
- preserves artifact order within each artifact type according to runner order;
|
|
||||||
- pretty-prints JSON with two-space indentation and trailing newline;
|
|
||||||
- uses content type `application/json`;
|
|
||||||
- includes rejected artifacts and warnings even when the arrays are empty;
|
|
||||||
- does not include output warnings inside artifact payloads.
|
|
||||||
|
|
||||||
File-name safety:
|
|
||||||
|
|
||||||
- artifact type may contain dots and hyphens;
|
|
||||||
- replace any character outside `[A-Za-z0-9._-]` with `_` for artifact file
|
|
||||||
names;
|
|
||||||
- if sanitization produces an empty name, return an error.
|
|
||||||
|
|
||||||
### Required Tests
|
|
||||||
|
|
||||||
- Generic chunker tests cover defaults, exact chunk boundaries, overlap,
|
|
||||||
invalid options, empty source, defensive copies, and stable IDs.
|
|
||||||
- Append-order merge tests cover ordering and defensive copies.
|
|
||||||
- Noop normalizer tests cover pass-through behavior and defensive copies.
|
|
||||||
- JSON output tests cover all logical files, grouping, sorted filenames,
|
|
||||||
rejected/warnings presence, pretty JSON, unsafe artifact type sanitization,
|
|
||||||
and no mutation of inputs.
|
|
||||||
- Pipeline config tests using defaults resolve when these module specs are
|
|
||||||
registered.
|
|
||||||
|
|
||||||
### Validation
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
gofmt -w internal/modules/chunk/generic internal/modules/merge/appendorder internal/modules/normalize/noop internal/modules/output/json internal/framework/pipeline
|
|
||||||
go test ./internal/modules/chunk/generic ./internal/modules/merge/appendorder ./internal/modules/normalize/noop ./internal/modules/output/json
|
|
||||||
go test ./internal/framework/pipeline
|
|
||||||
go test ./...
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 4: Add Production CLI Catalog And Runtime Wiring
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Make implemented modules selectable by real CLI commands without test-injected
|
|
||||||
catalogs.
|
|
||||||
|
|
||||||
### Files To Add Or Update
|
|
||||||
|
|
||||||
Expected files:
|
|
||||||
|
|
||||||
- `internal/cli/run.go`
|
|
||||||
- new `internal/cli/catalog.go` or equivalent;
|
|
||||||
- `internal/cli/run_test.go`;
|
|
||||||
- module registry tests as needed.
|
|
||||||
|
|
||||||
### Required Design
|
|
||||||
|
|
||||||
Add production wiring in `internal/cli`:
|
|
||||||
|
|
||||||
```go
|
|
||||||
func productionRegistries() (pipeline.Registries, error)
|
|
||||||
func productionCatalog() (pipeline.ModuleCatalog, error)
|
|
||||||
```
|
|
||||||
|
|
||||||
The production wiring must register:
|
|
||||||
|
|
||||||
- input: `seriatim`;
|
|
||||||
- chunk: `generic`;
|
|
||||||
- extract: `dnd/spells`;
|
|
||||||
- merge: `appendorder`;
|
|
||||||
- normalize: `noop`;
|
|
||||||
- output: `json`.
|
|
||||||
|
|
||||||
Keep all business logic in module packages. `internal/cli` should only compose
|
|
||||||
registries/catalogs and command behavior.
|
|
||||||
|
|
||||||
Update `cli.Options` so tests may inject registries/catalog/runtime without
|
|
||||||
disabling production defaults unintentionally.
|
|
||||||
|
|
||||||
Recommended option fields:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type Options struct {
|
|
||||||
Catalog pipeline.ModuleCatalog
|
|
||||||
Registries pipeline.Registries
|
|
||||||
LLMClientFactory LLMClientFactory
|
|
||||||
LookupEnv func(string) (string, bool)
|
|
||||||
Now func() time.Time
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
If `Catalog` or `Registries` is empty in normal `Run`, use production wiring.
|
|
||||||
If tests provide either, use the provided value.
|
|
||||||
|
|
||||||
Define `LLMClientFactory` in `internal/cli` or a small local file:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type LLMClientFactory func(ctx context.Context, cfg config.Config, profileID string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error)
|
|
||||||
```
|
|
||||||
|
|
||||||
The production factory should:
|
|
||||||
|
|
||||||
- read the selected LLM profile from effective config;
|
|
||||||
- construct `llm.OpenAICompatibleClient`;
|
|
||||||
- construct `llm.Scheduler` using the most specific configured concurrency:
|
|
||||||
profile `max_concurrency` if set, otherwise global `concurrency.total_llm`,
|
|
||||||
otherwise `1`;
|
|
||||||
- wrap the client in a scheduled client so every completion acquires/releases a
|
|
||||||
scheduler permit;
|
|
||||||
- return manifest-safe LLM metadata with profile ID, provider, and model.
|
|
||||||
|
|
||||||
Add a scheduled client wrapper in `internal/framework/llm` if it does not
|
|
||||||
already exist:
|
|
||||||
|
|
||||||
```go
|
|
||||||
func NewScheduledClient(client contracts.StructuredLLMClient, scheduler *Scheduler) contracts.StructuredLLMClient
|
|
||||||
```
|
|
||||||
|
|
||||||
### Required CLI Behavior
|
|
||||||
|
|
||||||
- `notarius config validate --config <file> --pipeline <id>` uses the
|
|
||||||
production catalog by default.
|
|
||||||
- `notarius pipelines list --config <file>` still lists configured pipeline
|
|
||||||
IDs and validates config shape.
|
|
||||||
- `notarius pipelines list --config <file> --json` remains stable.
|
|
||||||
|
|
||||||
Do not implement `notarius run` in this stage.
|
|
||||||
|
|
||||||
### Required Tests
|
|
||||||
|
|
||||||
- Production catalog includes the six MVP modules and their module specs.
|
|
||||||
- `config validate --pipeline` succeeds for a real MVP config fixture using no
|
|
||||||
injected catalog.
|
|
||||||
- Unknown module keys still fail with stage/pipeline context.
|
|
||||||
- Production LLM client factory rejects missing/invalid LLM profiles with clear
|
|
||||||
errors.
|
|
||||||
- Scheduled client wrapper enforces scheduler use and propagates errors.
|
|
||||||
- Existing CLI tests using injected catalogs still pass.
|
|
||||||
|
|
||||||
### Validation
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
gofmt -w internal/cli internal/framework/llm
|
|
||||||
go test ./internal/cli ./internal/framework/llm
|
|
||||||
go test ./...
|
|
||||||
go vet ./...
|
|
||||||
go build ./cmd/notarius
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 5: Implement `notarius run` Without Durable File Writing
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Add the user-facing run command and prove it can drive the configured pipeline
|
|
||||||
with injected fake runtime pieces. This stage should return/run data in memory
|
|
||||||
or through test buffers, but durable file writing may be completed in Stage 6.
|
|
||||||
|
|
||||||
### Files To Update
|
|
||||||
|
|
||||||
Expected files:
|
|
||||||
|
|
||||||
- `internal/cli/run.go`
|
|
||||||
- `internal/cli/run_test.go`
|
|
||||||
- `cmd/notarius/main.go`, only if command wiring requires it.
|
|
||||||
|
|
||||||
### Required Command Shape
|
|
||||||
|
|
||||||
Support:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
notarius run <pipeline-id> --input path/to/source.json
|
|
||||||
notarius run <pipeline-id> --input path/to/source.json --only spells
|
|
||||||
```
|
|
||||||
|
|
||||||
Supported flags:
|
|
||||||
|
|
||||||
- `--config path`;
|
|
||||||
- `--input path`, required;
|
|
||||||
- `--only lane-a,lane-b`;
|
|
||||||
- `--output-dir path`, parsed and passed through metadata for Stage 6;
|
|
||||||
- `--diagnostics-dir path`, overrides config diagnostics work dir for this run;
|
|
||||||
- `--llm-profile profile-id`, operational override for MVP runs.
|
|
||||||
|
|
||||||
Do not add flags for structural module selection, such as `--extractor`,
|
|
||||||
`--chunker`, `--merge`, or `--output`.
|
|
||||||
|
|
||||||
### Required Behavior
|
|
||||||
|
|
||||||
- Missing pipeline ID returns exit code `2`.
|
|
||||||
- Missing `--input` returns exit code `2`.
|
|
||||||
- Unknown flags return exit code `2`.
|
|
||||||
- Config/load/resolve/runtime failures return exit code `1`.
|
|
||||||
- Successful runs return exit code `0`.
|
|
||||||
- `--only` uses existing lane selection behavior.
|
|
||||||
- Extend `config.ResolveInput` with `LLMProfileOverride string` or an
|
|
||||||
equivalent option. When `--llm-profile` is provided, apply it to every
|
|
||||||
resolved module binding before the resolved pipeline digest is computed. This
|
|
||||||
keeps the override operational rather than structural while still making the
|
|
||||||
effective pipeline digest truthful.
|
|
||||||
- After resolution, collect the distinct effective LLM profile IDs used by the
|
|
||||||
selected pipeline. For the MVP, require exactly one distinct profile ID and
|
|
||||||
fail clearly if more than one is present.
|
|
||||||
- Input file is read as raw bytes and passed to the runner with path metadata.
|
|
||||||
- Source ID defaults to the input path or basename only if the input adapter
|
|
||||||
needs one; do not invent transcript-specific source IDs in the CLI.
|
|
||||||
- The command resolves the selected pipeline with the production catalog.
|
|
||||||
- The command constructs the LLM client through `LLMClientFactory`.
|
|
||||||
- The command invokes `pipeline.New(registries).Run(...)`.
|
|
||||||
- The command prints a concise success message that includes at least:
|
|
||||||
- pipeline ID;
|
|
||||||
- approved artifact count;
|
|
||||||
- rejected artifact count.
|
|
||||||
- If warnings exist, print a concise warning count to stderr.
|
|
||||||
|
|
||||||
### Required Tests
|
|
||||||
|
|
||||||
Use fake LLM/runtime injection; do not call external services.
|
|
||||||
|
|
||||||
- Missing pipeline ID.
|
|
||||||
- Missing input flag.
|
|
||||||
- Unknown pipeline.
|
|
||||||
- Unknown `--only` lane.
|
|
||||||
- Invalid input file path.
|
|
||||||
- Successful run invokes runner path through real registries and fake LLM.
|
|
||||||
- `--only spells` runs only the selected lane.
|
|
||||||
- LLM factory failure is reported clearly.
|
|
||||||
- Validation rejection produces a failed/non-zero or successful-with-rejections
|
|
||||||
behavior according to current runner semantics. For MVP, keep runner
|
|
||||||
semantics: a run with rejected artifacts completes successfully with
|
|
||||||
`ValidationStatus` set to `rejected`, unless an error occurs.
|
|
||||||
|
|
||||||
### Validation
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
gofmt -w internal/cli
|
|
||||||
go test ./internal/cli
|
|
||||||
go test ./...
|
|
||||||
go vet ./...
|
|
||||||
go build ./cmd/notarius
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 6: Write Durable Output And Diagnostics For `notarius run`
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Complete the MVP run workflow by writing output files and diagnostics.
|
|
||||||
|
|
||||||
### Files To Update
|
|
||||||
|
|
||||||
Expected files:
|
|
||||||
|
|
||||||
- `internal/cli/run.go`
|
|
||||||
- `internal/cli/run_test.go`
|
|
||||||
- `internal/core/diagnostics/*.go`, only if helper methods are needed.
|
|
||||||
|
|
||||||
### Required Design
|
|
||||||
|
|
||||||
Output directory behavior:
|
|
||||||
|
|
||||||
- default root: `./notarius-output`;
|
|
||||||
- override: `--output-dir`;
|
|
||||||
- each run writes to `<output-root>/<run-id>/`;
|
|
||||||
- run ID comes from diagnostics run directory when available or from a
|
|
||||||
generated UTC nanosecond timestamp using the same style as diagnostics;
|
|
||||||
- create directories with `0755`;
|
|
||||||
- write files with `0644`;
|
|
||||||
- write each file atomically where practical.
|
|
||||||
|
|
||||||
Logical output files from `pipeline.RunOutput.OutputFiles` should be written
|
|
||||||
under the run output directory. Reject unsafe logical file names before writing:
|
|
||||||
|
|
||||||
- empty;
|
|
||||||
- absolute;
|
|
||||||
- contains `..`;
|
|
||||||
- contains backslash;
|
|
||||||
- escapes the run output directory after path cleaning.
|
|
||||||
|
|
||||||
Diagnostics behavior:
|
|
||||||
|
|
||||||
- create a diagnostics run directory at command start unless retention is
|
|
||||||
`never` and the implementation can still reliably capture failures; simplest
|
|
||||||
MVP behavior is to create it and then apply retention at the end;
|
|
||||||
- write invocation metadata;
|
|
||||||
- write redacted effective config;
|
|
||||||
- write resolved pipeline;
|
|
||||||
- write run manifest;
|
|
||||||
- write warnings;
|
|
||||||
- write run report containing output path, counts, and validation status;
|
|
||||||
- write error log on failure;
|
|
||||||
- apply retention with existing diagnostics policy.
|
|
||||||
|
|
||||||
`--diagnostics-dir` should override `Config.Diagnostics.WorkDir` after file and
|
|
||||||
environment config have been applied, without changing structural pipeline
|
|
||||||
definition or pipeline digest.
|
|
||||||
|
|
||||||
Success output:
|
|
||||||
|
|
||||||
- stdout includes the durable output run directory path;
|
|
||||||
- stderr includes warning count when warnings are present;
|
|
||||||
- no raw prompt text, raw API keys, or large source payloads should be printed.
|
|
||||||
|
|
||||||
### Required Tests
|
|
||||||
|
|
||||||
- Successful `notarius run` writes output files under a temp output directory.
|
|
||||||
- Output write rejects unsafe logical file names from a fake encoder.
|
|
||||||
- Writes are atomic enough that no temporary files remain after success.
|
|
||||||
- Diagnostics artifacts are written on success.
|
|
||||||
- Error log is written on failure after diagnostics directory creation.
|
|
||||||
- Retention `never` removes successful warning-free diagnostics directories.
|
|
||||||
- Warnings are present in diagnostics and are reported to stderr.
|
|
||||||
- `--diagnostics-dir` overrides config diagnostics directory.
|
|
||||||
|
|
||||||
### Validation
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
gofmt -w internal/cli internal/core/diagnostics
|
|
||||||
go test ./internal/cli ./internal/core/diagnostics
|
|
||||||
go test ./...
|
|
||||||
go vet ./...
|
|
||||||
go build ./cmd/notarius
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 7: Add MVP Fixtures And End-To-End Acceptance Coverage
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Make the MVP path continuously testable without network access.
|
|
||||||
|
|
||||||
### Files To Add Or Update
|
|
||||||
|
|
||||||
Expected fixtures:
|
|
||||||
|
|
||||||
- `examples/seriatim-minimal-transcript.json`, if the example can be kept
|
|
||||||
accurate before the deferred documentation pass;
|
|
||||||
- `examples/dnd-spells.config.yml`, if config examples are tested in this
|
|
||||||
stage;
|
|
||||||
- or equivalent `internal/cli/testdata/...` fixtures if examples are deferred.
|
|
||||||
|
|
||||||
Expected tests:
|
|
||||||
|
|
||||||
- `internal/cli/run_test.go`
|
|
||||||
- `internal/modules/extract/dnd/spells/runner_test.go`
|
|
||||||
- config tests as needed.
|
|
||||||
|
|
||||||
### Required Design
|
|
||||||
|
|
||||||
Add a maintained MVP config fixture:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
version: 1
|
|
||||||
llm_profiles:
|
|
||||||
default:
|
|
||||||
provider: openai-compatible
|
|
||||||
base_url: http://127.0.0.1:1
|
|
||||||
model: fake-model
|
|
||||||
pipelines:
|
|
||||||
dnd-session:
|
|
||||||
input: seriatim
|
|
||||||
chunk:
|
|
||||||
module: generic
|
|
||||||
options:
|
|
||||||
max_units: 50
|
|
||||||
artifacts:
|
|
||||||
spells:
|
|
||||||
extract: dnd/spells
|
|
||||||
```
|
|
||||||
|
|
||||||
The fixture may use a fake base URL because tests should inject a fake LLM
|
|
||||||
client factory. Do not require a real network call.
|
|
||||||
|
|
||||||
Acceptance tests should execute the public CLI entry path with:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
notarius run dnd-session --config <fixture> --input <fixture> --output-dir <tmp>
|
|
||||||
notarius run dnd-session --config <fixture> --input <fixture> --only spells --output-dir <tmp>
|
|
||||||
notarius config validate --config <fixture> --pipeline dnd-session
|
|
||||||
notarius pipelines list --config <fixture>
|
|
||||||
```
|
|
||||||
|
|
||||||
The fake LLM should return deterministic D&D spell output with valid source
|
|
||||||
references. The resulting output files should be parsed as JSON and checked for:
|
|
||||||
|
|
||||||
- manifest pipeline ID and digest;
|
|
||||||
- spell artifact payload;
|
|
||||||
- source references;
|
|
||||||
- prompt/schema metadata in manifest or artifact metadata;
|
|
||||||
- validation status;
|
|
||||||
- warning behavior.
|
|
||||||
|
|
||||||
### Required Failure Coverage
|
|
||||||
|
|
||||||
Add fixture-driven tests for:
|
|
||||||
|
|
||||||
- missing config;
|
|
||||||
- unknown pipeline;
|
|
||||||
- invalid Seriatim input;
|
|
||||||
- invalid `--only` lane;
|
|
||||||
- fake LLM failure;
|
|
||||||
- malformed LLM response;
|
|
||||||
- invalid source reference rejection.
|
|
||||||
|
|
||||||
### Validation
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
gofmt -w internal/cli internal/modules/extract/dnd/spells
|
|
||||||
go test ./internal/cli ./internal/modules/extract/dnd/spells
|
|
||||||
go test ./...
|
|
||||||
go vet ./...
|
|
||||||
go build ./cmd/notarius
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 8: MVP Final Review And Roadmap Cleanup
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Confirm the MVP is complete enough to trigger the deferred documentation pass.
|
|
||||||
|
|
||||||
### Required Review
|
|
||||||
|
|
||||||
Perform a code review against:
|
|
||||||
|
|
||||||
- [`mvp.md`](mvp.md);
|
|
||||||
- [`../policy/architecture.md`](../policy/architecture.md);
|
|
||||||
- [`../policy/documentation.md`](../policy/documentation.md).
|
|
||||||
|
|
||||||
Check specifically:
|
|
||||||
|
|
||||||
- no D&D prompt/schema assets or constants remain in framework packages;
|
|
||||||
- production CLI commands use production wiring by default;
|
|
||||||
- `config validate --pipeline` works with the MVP fixture;
|
|
||||||
- `pipelines list` works with the MVP fixture;
|
|
||||||
- `notarius run` writes durable output and diagnostics;
|
|
||||||
- default modules resolve without test-only registration;
|
|
||||||
- output warnings remain out-of-band from artifact payloads;
|
|
||||||
- no private data or secrets appear in fixtures;
|
|
||||||
- docs outside `docs/roadmap/` describe only implemented behavior.
|
|
||||||
|
|
||||||
### Required Validation
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./...
|
|
||||||
go vet ./...
|
|
||||||
go build ./cmd/notarius
|
|
||||||
```
|
|
||||||
|
|
||||||
### Required Roadmap Update
|
|
||||||
|
|
||||||
After the MVP is implemented and reviewed:
|
|
||||||
|
|
||||||
- update [`mvp.md`](mvp.md) to mark MVP functionality complete or reduce it to
|
|
||||||
remaining release/documentation work;
|
|
||||||
- keep the full documentation pass deferred until this review passes;
|
|
||||||
- do not tag alpha `0.1.0` until the documentation pass is complete.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None. The plan above makes the required MVP implementation choices explicitly.
|
|
||||||
@@ -1,632 +0,0 @@
|
|||||||
# Initial Architecture Roadmap
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
This document captures proposed architecture and implementation sequencing for
|
|
||||||
Notarius. It describes planned work, not implemented behavior.
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Notarius should extract structured JSON artifacts from primary source inputs
|
|
||||||
using modular, LLM-backed extractors.
|
|
||||||
|
|
||||||
The first MVP should target audio transcripts generated by Seriatim. That
|
|
||||||
choice should be implemented as an input-stage module, not as a
|
|
||||||
transcript-specific assumption in the application core. Later input sources,
|
|
||||||
such as unstructured Markdown notes or Obsidian documents, should be addable
|
|
||||||
through new input and extract modules without reshaping the framework.
|
|
||||||
|
|
||||||
The first extraction domain should be D&D session analysis, starting with spell
|
|
||||||
casts. That domain should live in extract-stage modules and related schemas, not
|
|
||||||
in core framework packages.
|
|
||||||
|
|
||||||
The application should follow the same broad architecture as Audita:
|
|
||||||
|
|
||||||
- deterministic core packages for source documents, artifacts, and configuration once needed;
|
|
||||||
- input-stage modules that translate external source formats into a small internal source model;
|
|
||||||
- reusable framework packages for contracts, orchestration, LLM runtime, structured output, and validation;
|
|
||||||
- independent extract-stage modules that own domain-specific behavior;
|
|
||||||
- independent validator packages;
|
|
||||||
- embedded prompt and JSON schema assets;
|
|
||||||
- CLI orchestration that wires the pieces together without owning domain logic.
|
|
||||||
|
|
||||||
The main domain difference from Audita is that Notarius emits extracted
|
|
||||||
artifacts rather than proposing and applying transcript corrections.
|
|
||||||
|
|
||||||
## Architectural Principles
|
|
||||||
|
|
||||||
- Keep the core input model generic: ordered text units plus metadata.
|
|
||||||
- Keep source-format details in hexagonal input modules.
|
|
||||||
- Keep extraction-domain details in extract modules.
|
|
||||||
- Treat evidence as source references, not transcript references.
|
|
||||||
- Prefer narrow, useful abstractions over a universal document model.
|
|
||||||
- Preserve enough provenance for validation, replay, and downstream inspection.
|
|
||||||
|
|
||||||
## Proposed Package Shape
|
|
||||||
|
|
||||||
```text
|
|
||||||
cmd/notarius
|
|
||||||
internal/cli
|
|
||||||
|
|
||||||
internal/core/source
|
|
||||||
internal/core/artifacts
|
|
||||||
|
|
||||||
internal/framework/contracts
|
|
||||||
internal/framework/pipeline
|
|
||||||
internal/framework/validate
|
|
||||||
internal/framework/llm
|
|
||||||
internal/framework/prompt
|
|
||||||
|
|
||||||
internal/modules/input/seriatim
|
|
||||||
internal/modules/input/markdown
|
|
||||||
|
|
||||||
internal/modules/chunk/generic
|
|
||||||
internal/modules/chunk/dndtranscript
|
|
||||||
|
|
||||||
internal/modules/extract/dnd/spells
|
|
||||||
internal/modules/extract/dnd/items
|
|
||||||
internal/modules/extract/dnd/npcs
|
|
||||||
internal/modules/extract/dnd/combat
|
|
||||||
|
|
||||||
internal/modules/merge/appendorder
|
|
||||||
internal/modules/merge/dnd/spells
|
|
||||||
|
|
||||||
internal/modules/normalize/noop
|
|
||||||
internal/modules/normalize/dnd/spells
|
|
||||||
|
|
||||||
internal/modules/output/json
|
|
||||||
|
|
||||||
internal/validators/source_refs
|
|
||||||
internal/validators/schema_validity
|
|
||||||
internal/validators/domain_consistency
|
|
||||||
internal/validators/llm_review
|
|
||||||
|
|
||||||
examples
|
|
||||||
docs/internal
|
|
||||||
```
|
|
||||||
|
|
||||||
The `markdown` input module and D&D-specific chunk, merge, normalize, and
|
|
||||||
output modules are listed as likely future packages. The MVP should implement
|
|
||||||
only the stage modules needed by the checkpoint sequence.
|
|
||||||
|
|
||||||
`internal/core/config` should be added when production configuration exists.
|
|
||||||
|
|
||||||
The framework package list is intentionally consolidated. `pipeline` should own
|
|
||||||
runner orchestration, stage registries, and small merge/normalize/output helpers
|
|
||||||
until those boundaries prove they need separate packages. `llm` should own
|
|
||||||
structured output and response-schema mechanics until those concerns become too
|
|
||||||
large or import-heavy. `prompt` should own prompt assets and rendering helpers
|
|
||||||
once prompt assets exist.
|
|
||||||
|
|
||||||
## Core Concepts
|
|
||||||
|
|
||||||
### SourceDocument
|
|
||||||
|
|
||||||
Canonical internal representation of source material. This should be the object
|
|
||||||
extractors receive, regardless of whether the original input was a transcript,
|
|
||||||
Markdown file, note export, or another source type.
|
|
||||||
|
|
||||||
```go
|
|
||||||
type SourceDocument struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Kind string `json:"kind"`
|
|
||||||
Format string `json:"format"`
|
|
||||||
Digest string `json:"digest"`
|
|
||||||
Units []SourceUnit `json:"units"`
|
|
||||||
Metadata map[string]any `json:"metadata,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type SourceUnit struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Kind string `json:"kind"`
|
|
||||||
Text string `json:"text"`
|
|
||||||
Metadata map[string]any `json:"metadata,omitempty"`
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Initial source-unit assumptions:
|
|
||||||
|
|
||||||
- units are ordered;
|
|
||||||
- unit IDs are stable within a source document;
|
|
||||||
- each unit has extractable text;
|
|
||||||
- adapter-specific metadata may carry speaker, timestamps, heading paths, page
|
|
||||||
numbers, or other source details.
|
|
||||||
|
|
||||||
Core source metadata should remain `map[string]any`. Notarius should not define
|
|
||||||
a universal document model. Instead, the project should document well-known
|
|
||||||
metadata keys, such as `speaker`, `start`, `end`, and `heading_path`, as
|
|
||||||
conventions. Input modules may export typed accessor helpers for their own
|
|
||||||
metadata, such as `seriatim.SpeakerOf(unit)`, without leaking those helpers into
|
|
||||||
core framework contracts.
|
|
||||||
|
|
||||||
### Input Module / Adapter Contract
|
|
||||||
|
|
||||||
Hexagonal boundary for external source formats.
|
|
||||||
|
|
||||||
```go
|
|
||||||
type InputAdapter interface {
|
|
||||||
Key() string
|
|
||||||
Parse(ctx context.Context, req ParseRequest) (*source.SourceDocument, error)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The MVP input module should target Seriatim minimal transcript JSON. Seriatim segment
|
|
||||||
fields should map as follows:
|
|
||||||
|
|
||||||
- `id` becomes `SourceUnit.ID`;
|
|
||||||
- `text` becomes `SourceUnit.Text`;
|
|
||||||
- `speaker`, `start`, and `end` become unit metadata;
|
|
||||||
- Seriatim output metadata becomes document metadata.
|
|
||||||
|
|
||||||
The core runner should not know that these units came from transcript segments.
|
|
||||||
|
|
||||||
### SourceRef
|
|
||||||
|
|
||||||
Grounding reference from an extracted fact back to source units.
|
|
||||||
|
|
||||||
```go
|
|
||||||
type SourceRef struct {
|
|
||||||
SourceID string `json:"source_id"`
|
|
||||||
StartUnitID string `json:"start_unit_id"`
|
|
||||||
EndUnitID string `json:"end_unit_id"`
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Initial source-reference validation should require:
|
|
||||||
|
|
||||||
- source ID exists for the current run;
|
|
||||||
- start and end unit IDs exist;
|
|
||||||
- start is less than or equal to end in document order;
|
|
||||||
- the referenced range is contiguous within the source document;
|
|
||||||
- every extracted fact has at least one source reference unless its schema
|
|
||||||
explicitly allows ungrounded metadata.
|
|
||||||
|
|
||||||
Transcript-oriented output can still present these as transcript segment ranges
|
|
||||||
when the adapter metadata makes that interpretation available.
|
|
||||||
|
|
||||||
Source references should preserve the exact ranges produced by extractors and
|
|
||||||
validators. Overlapping ranges should not be merged or rewritten by generic
|
|
||||||
pipeline code. If a domain module wants a derived compact range later, that
|
|
||||||
should be additional output, not a replacement for the original evidence.
|
|
||||||
|
|
||||||
### Extractor
|
|
||||||
|
|
||||||
Reusable module contract for producing one artifact type.
|
|
||||||
|
|
||||||
```go
|
|
||||||
type Extractor interface {
|
|
||||||
Key() string
|
|
||||||
ArtifactType() string
|
|
||||||
SchemaVersion() string
|
|
||||||
Validators() []Validator
|
|
||||||
Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
An extractor should receive either a whole source document or a source chunk,
|
|
||||||
depending on processing mode. It should return typed artifact candidates plus
|
|
||||||
warnings. It should not mutate the source document.
|
|
||||||
|
|
||||||
`ExtractionRequest` should be designed now to carry both the active chunk and
|
|
||||||
optional ambient context, even if the MVP leaves that context empty. Useful
|
|
||||||
ambient context may include a document synopsis, prior-chunk summaries, known
|
|
||||||
entities, or other module-provided state. D&D spell extraction can likely work
|
|
||||||
per chunk, but combat, NPC, and identity-oriented extraction will need broader
|
|
||||||
context. Adding the field later would force churn across every extractor.
|
|
||||||
|
|
||||||
Extract modules own domain concepts. For example, D&D spell extraction should
|
|
||||||
live under `internal/modules/extract/dnd/spells`; a future to-do extractor for
|
|
||||||
notes should live under a different extract-module path and use the same
|
|
||||||
framework contract.
|
|
||||||
|
|
||||||
### Chunker
|
|
||||||
|
|
||||||
Reusable stage contract for splitting a source document into ordered source
|
|
||||||
chunks.
|
|
||||||
|
|
||||||
Chunking is a first-class pipeline concern because source documents may exceed a
|
|
||||||
single LLM extraction pass. Chunkers should preserve source-unit order and
|
|
||||||
produce stable chunk metadata suitable for diagnostics and replay.
|
|
||||||
|
|
||||||
### Merger
|
|
||||||
|
|
||||||
Reusable stage contract for combining per-chunk artifact candidates into one
|
|
||||||
merged candidate collection.
|
|
||||||
|
|
||||||
Merge should combine outputs without doing semantic reconciliation. A generic
|
|
||||||
append-in-chunk-order merger should be sufficient for many artifact streams,
|
|
||||||
including the likely first D&D spell-cast extractor.
|
|
||||||
|
|
||||||
### Normalizer
|
|
||||||
|
|
||||||
Reusable stage contract for reconciling merged artifact candidates.
|
|
||||||
|
|
||||||
Normalize is distinct from merge. Normalizers may deduplicate repeated facts,
|
|
||||||
resolve aliases, reconcile conflicting fields, check cross-chunk consistency,
|
|
||||||
or attach normalization warnings.
|
|
||||||
|
|
||||||
### Validator
|
|
||||||
|
|
||||||
Reusable validation contract for artifact candidates.
|
|
||||||
|
|
||||||
Validators should cover:
|
|
||||||
|
|
||||||
- JSON/schema validity;
|
|
||||||
- source-reference validity;
|
|
||||||
- required-field and shape checks;
|
|
||||||
- domain consistency;
|
|
||||||
- optional LLM review for high-risk or ambiguous artifacts.
|
|
||||||
|
|
||||||
Validator output should follow Audita's decision-cardinality model: each
|
|
||||||
candidate artifact receives exactly one decision per validator.
|
|
||||||
|
|
||||||
LLM-backed review should be modeled as part of a module's validator chain, not
|
|
||||||
as a separate global review phase. Extract modules should be able to attach one
|
|
||||||
or more deterministic or LLM-backed validators. Normalize-stage modules may also
|
|
||||||
run validator chains, including LLM-backed validators, when semantic
|
|
||||||
reconciliation needs review.
|
|
||||||
|
|
||||||
### Artifact
|
|
||||||
|
|
||||||
Final approved JSON output from one or more extractors.
|
|
||||||
|
|
||||||
Artifacts should preserve enough metadata to support downstream validation,
|
|
||||||
debugging, and replay.
|
|
||||||
|
|
||||||
The pipeline should carry artifact candidates through a generic envelope with a
|
|
||||||
`json.RawMessage` payload. Extract modules should own typed Go structs at their
|
|
||||||
module boundary, then encode those typed records into the generic artifact
|
|
||||||
candidate envelope before returning to framework code. This keeps stage
|
|
||||||
contracts simple and avoids generic type plumbing across unrelated artifact
|
|
||||||
families.
|
|
||||||
|
|
||||||
Final durable output should be one file per artifact type plus a run-level
|
|
||||||
manifest/index file. This supports partial success and lets downstream consumers
|
|
||||||
read only the artifact types they need. Each artifact file should include its
|
|
||||||
artifact type, extractor key, extractor schema version, envelope format version,
|
|
||||||
records, source references, and enough provenance to connect it to the run
|
|
||||||
manifest.
|
|
||||||
|
|
||||||
Every artifact record should require source references unless that artifact
|
|
||||||
schema explicitly opts into ungrounded fields. Artifact-level metadata, counts,
|
|
||||||
run information, and other derived summary fields are exempt from the per-record
|
|
||||||
grounding rule.
|
|
||||||
|
|
||||||
Schemas should be versioned per extractor, with a separate envelope/manifest
|
|
||||||
format version. A single global schema version would couple unrelated extractor
|
|
||||||
release cadence.
|
|
||||||
|
|
||||||
### RunManifest
|
|
||||||
|
|
||||||
Per-run provenance record.
|
|
||||||
|
|
||||||
```go
|
|
||||||
type RunManifest struct {
|
|
||||||
EnvelopeVersion string `json:"envelope_version"`
|
|
||||||
PipelineID string `json:"pipeline_id"`
|
|
||||||
PipelineDigest string `json:"pipeline_digest"`
|
|
||||||
InputModule string `json:"input_module"`
|
|
||||||
Chunker string `json:"chunker"`
|
|
||||||
SourceDigests []string `json:"source_digests"`
|
|
||||||
Extractors []string `json:"extractors"`
|
|
||||||
Merger string `json:"merger"`
|
|
||||||
Normalizer string `json:"normalizer"`
|
|
||||||
OutputEncoder string `json:"output_encoder"`
|
|
||||||
SchemaVersion string `json:"schema_version"`
|
|
||||||
ValidationStatus string `json:"validation_status"`
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The manifest should eventually include model names, prompt IDs, prompt hashes,
|
|
||||||
response schema versions, config source, redacted resolved config digest,
|
|
||||||
started/completed timestamps, and diagnostics paths.
|
|
||||||
|
|
||||||
## Initial Extractor Targets
|
|
||||||
|
|
||||||
### D&D Spells
|
|
||||||
|
|
||||||
Recommended first vertical slice because it is narrow but representative.
|
|
||||||
|
|
||||||
```go
|
|
||||||
type SpellCast struct {
|
|
||||||
Player string `json:"player"`
|
|
||||||
Spell string `json:"spell"`
|
|
||||||
Effect string `json:"effect"`
|
|
||||||
NarrativeDescription string `json:"narrative_description"`
|
|
||||||
SourceRefs []SourceRef `json:"source_refs"`
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The spell extractor should be D&D-specific. The framework should not know what a
|
|
||||||
spell is.
|
|
||||||
|
|
||||||
### D&D Items
|
|
||||||
|
|
||||||
Tracks items gained, lost, transferred, consumed, or transformed.
|
|
||||||
|
|
||||||
Open questions:
|
|
||||||
|
|
||||||
- Should currency be represented as items or as its own artifact type?
|
|
||||||
- Should item ownership be a required field?
|
|
||||||
- How should ambiguous ownership changes be represented?
|
|
||||||
|
|
||||||
### D&D NPCs
|
|
||||||
|
|
||||||
Tracks NPCs interacted with, newly introduced, renamed, described, or otherwise
|
|
||||||
made relevant to campaign state.
|
|
||||||
|
|
||||||
Open questions:
|
|
||||||
|
|
||||||
- Should NPC identity resolution happen inside this extractor or in a later
|
|
||||||
deduplication stage?
|
|
||||||
- Should location/faction/relationship facts be separate artifact types?
|
|
||||||
|
|
||||||
### D&D Combat
|
|
||||||
|
|
||||||
Likely warrants a dedicated schema rather than a generic event list.
|
|
||||||
|
|
||||||
Proposed first shape:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type CombatTurn struct {
|
|
||||||
Actor string `json:"actor"`
|
|
||||||
Action string `json:"action"`
|
|
||||||
Outcome string `json:"outcome"`
|
|
||||||
NarrativeDescription string `json:"narrative_description"`
|
|
||||||
SourceRefs []SourceRef `json:"source_refs"`
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Open questions:
|
|
||||||
|
|
||||||
- Should combat be extracted as turns, rounds, encounters, or all three?
|
|
||||||
- Should mechanical fields such as damage, conditions, saves, attacks, and spell
|
|
||||||
slots be normalized immediately or added later?
|
|
||||||
- How should uncertain initiative order be represented?
|
|
||||||
|
|
||||||
### Future Non-D&D Extractors
|
|
||||||
|
|
||||||
The architecture should support extractors outside the D&D domain. Examples:
|
|
||||||
|
|
||||||
- to-do items from Markdown or Obsidian notes;
|
|
||||||
- decisions and action items from meeting transcripts;
|
|
||||||
- named people, places, and dates from research notes.
|
|
||||||
|
|
||||||
These should be addable as extract modules without changing runner,
|
|
||||||
validator, source-reference, or LLM framework contracts.
|
|
||||||
|
|
||||||
## Configuration Model
|
|
||||||
|
|
||||||
Notarius should use named pipeline profiles selected by ID at the CLI. A
|
|
||||||
pipeline is a fixed-shape template for the known application workflow, not a
|
|
||||||
free-form list of steps:
|
|
||||||
|
|
||||||
```text
|
|
||||||
input -> chunk -> extract -> merge -> normalize -> output
|
|
||||||
```
|
|
||||||
|
|
||||||
A pipeline profile should define one shared front end and one or more artifact
|
|
||||||
lanes:
|
|
||||||
|
|
||||||
- shared input module;
|
|
||||||
- shared chunk module by default;
|
|
||||||
- artifact lanes containing extract, merge, normalize, and validator behavior;
|
|
||||||
- shared output module.
|
|
||||||
|
|
||||||
The MVP should use one shared chunk module per pipeline. Per-lane chunk
|
|
||||||
overrides can be added later if an artifact lane, such as combat, proves it
|
|
||||||
needs a different chunking strategy.
|
|
||||||
|
|
||||||
Example shape:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
llm_profiles:
|
|
||||||
default:
|
|
||||||
model: example-model
|
|
||||||
max_concurrency: 4
|
|
||||||
|
|
||||||
pipelines:
|
|
||||||
dnd-session:
|
|
||||||
input: seriatim
|
|
||||||
chunk: dnd/transcript
|
|
||||||
artifacts:
|
|
||||||
spells:
|
|
||||||
extract: dnd/spells
|
|
||||||
normalize: dnd/spells
|
|
||||||
npcs:
|
|
||||||
extract: dnd/npcs
|
|
||||||
items:
|
|
||||||
extract: dnd/items
|
|
||||||
```
|
|
||||||
|
|
||||||
The CLI should run named pipelines:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
notarius run dnd-session --input session-014.json
|
|
||||||
notarius run dnd-session --input session-014.json --only spells,npcs
|
|
||||||
```
|
|
||||||
|
|
||||||
`--only` should select configured artifact lanes. It should not create an
|
|
||||||
ad hoc pipeline. Structural module selection should come from config, while CLI
|
|
||||||
flags may override operational knobs such as model, concurrency, output
|
|
||||||
directory, and diagnostics directory.
|
|
||||||
|
|
||||||
Initial defaults:
|
|
||||||
|
|
||||||
- `chunk`: `generic`;
|
|
||||||
- lane `merge`: `appendorder`;
|
|
||||||
- lane `normalize`: `noop`;
|
|
||||||
- `output`: `json`;
|
|
||||||
- `llm_profile`: `default` where an LLM profile is needed.
|
|
||||||
|
|
||||||
Module bindings should support both string shorthand and object form:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
extract: dnd/spells
|
|
||||||
```
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
extract:
|
|
||||||
module: dnd/spells
|
|
||||||
llm_profile: fast
|
|
||||||
prompt_version: v1
|
|
||||||
```
|
|
||||||
|
|
||||||
Both forms should normalize into a single internal `ModuleBinding` shape before
|
|
||||||
validation and manifest hashing.
|
|
||||||
|
|
||||||
Pipeline validation should use module metadata declared through registries.
|
|
||||||
Modules should expose flat string capability metadata, such as `speaker` or
|
|
||||||
`timestamps`, without requiring module construction. Config validation should
|
|
||||||
fail fast for:
|
|
||||||
|
|
||||||
- unknown pipeline IDs;
|
|
||||||
- unknown module keys;
|
|
||||||
- missing required slots;
|
|
||||||
- missing required capabilities;
|
|
||||||
- unknown LLM profiles;
|
|
||||||
- empty artifact-lane sets;
|
|
||||||
- `--only` lane names that do not exist in the selected pipeline.
|
|
||||||
|
|
||||||
The MVP should keep pipelines config-file-only. Built-in pipeline profiles can
|
|
||||||
be added later if the project needs embedded defaults, but that introduces
|
|
||||||
merge/override semantics that the MVP does not need.
|
|
||||||
|
|
||||||
The resolved pipeline definition should be hashed after defaults and lane
|
|
||||||
selection are applied. The run manifest should record both `pipeline_id` and
|
|
||||||
`pipeline_digest`; a pipeline ID alone is not stable provenance.
|
|
||||||
|
|
||||||
## Proposed Pipeline Flow
|
|
||||||
|
|
||||||
The application workflow should be first-class:
|
|
||||||
|
|
||||||
```text
|
|
||||||
input -> chunk -> extract -> merge -> normalize -> output
|
|
||||||
```
|
|
||||||
|
|
||||||
Proposed runner flow:
|
|
||||||
|
|
||||||
1. Load effective config.
|
|
||||||
2. Resolve the selected pipeline profile by ID.
|
|
||||||
3. Apply defaults and `--only` lane selection.
|
|
||||||
4. Validate module keys, lane definitions, LLM profiles, and capabilities.
|
|
||||||
5. Hash the resolved pipeline definition.
|
|
||||||
6. Create diagnostics run directory.
|
|
||||||
7. Resolve the configured input module through the input adapter registry.
|
|
||||||
8. Read source input.
|
|
||||||
9. Parse source input into a `SourceDocument`.
|
|
||||||
10. Validate source-document invariants.
|
|
||||||
11. Resolve the configured chunker.
|
|
||||||
12. Chunk source units into deterministic source chunks.
|
|
||||||
13. Resolve configured artifact lanes through registries.
|
|
||||||
14. Extract, merge, normalize, and validate each selected artifact lane.
|
|
||||||
15. Retain approved artifacts and rejected-artifact diagnostics.
|
|
||||||
16. Serialize output files and run-level manifest/index.
|
|
||||||
17. Write diagnostics and optional report JSON.
|
|
||||||
|
|
||||||
The runner should operate on source documents and source chunks only. Any
|
|
||||||
transcript-specific behavior should happen before the runner, inside the input
|
|
||||||
adapter, or after the runner, inside output rendering that understands source
|
|
||||||
metadata.
|
|
||||||
|
|
||||||
## Audita Patterns To Reuse
|
|
||||||
|
|
||||||
Reuse these architectural patterns:
|
|
||||||
|
|
||||||
- deterministic parsing and schema validation style;
|
|
||||||
- deterministic chunking of ordered source units;
|
|
||||||
- explicit extractor registry;
|
|
||||||
- explicit pipeline stage contracts;
|
|
||||||
- `contracts` package for transport-neutral interfaces;
|
|
||||||
- OpenAI-compatible structured LLM client;
|
|
||||||
- scheduler for bounded LLM concurrency;
|
|
||||||
- embedded prompt registry with prompt metadata and hashes;
|
|
||||||
- embedded response-schema registry with schema metadata and hashes;
|
|
||||||
- diagnostics run directory with redacted effective config;
|
|
||||||
- validator decision cardinality and deterministic validator ordering;
|
|
||||||
- CLI tests and fixture-driven integration tests.
|
|
||||||
|
|
||||||
The fixture-driven integration-test pattern should remain part of the codebase:
|
|
||||||
walking skeleton tests over fake modules and fake LLM clients should be
|
|
||||||
preserved as real Seriatim, runtime, and D&D modules are added, so the
|
|
||||||
end-to-end contract coverage is not lost.
|
|
||||||
|
|
||||||
Avoid copying these Audita concepts directly:
|
|
||||||
|
|
||||||
- transcript-specific core types;
|
|
||||||
- correction proposals;
|
|
||||||
- replacement policies;
|
|
||||||
- deterministic transcript mutation;
|
|
||||||
- correction ledger terminology.
|
|
||||||
|
|
||||||
Those concepts are specific to Audita's transcript-editing role and should be
|
|
||||||
replaced with source-document, artifact-candidate, artifact-validation, and
|
|
||||||
extraction-report concepts.
|
|
||||||
|
|
||||||
## Checkpoint Roadmap
|
|
||||||
|
|
||||||
The initial six checkpoint roadmap has been implemented and retired. The
|
|
||||||
checkpoint files have been removed from `docs/roadmap/` so the active roadmap
|
|
||||||
does not compete with completed implementation history.
|
|
||||||
|
|
||||||
The remaining work needed to reach the first functional MVP is tracked in
|
|
||||||
[`mvp.md`](mvp.md). Future staged implementation plans should be written to
|
|
||||||
[`implementation.md`](implementation.md) from that active MVP roadmap.
|
|
||||||
|
|
||||||
## Architecture Decisions
|
|
||||||
|
|
||||||
- Final durable output should use one artifact file per artifact type plus a
|
|
||||||
run-level manifest/index file.
|
|
||||||
- Framework artifact flow should use a generic envelope with `json.RawMessage`
|
|
||||||
payloads. Extract modules should use typed Go structs at their own boundaries.
|
|
||||||
- Schemas should be versioned per extractor, with a separate envelope/manifest
|
|
||||||
format version.
|
|
||||||
- Artifact records should require source references by default. Individual
|
|
||||||
schemas may explicitly opt into ungrounded fields. Artifact-level metadata is
|
|
||||||
exempt.
|
|
||||||
- Source-reference ranges should be preserved exactly. Generic pipeline code
|
|
||||||
should not merge or rewrite overlapping ranges.
|
|
||||||
- `ExtractionRequest` should carry the active chunk plus optional ambient
|
|
||||||
context for document synopsis, prior-chunk summaries, known entities, or
|
|
||||||
similar module-provided state.
|
|
||||||
- LLM-backed review should be part of module-owned validator chains. Extract
|
|
||||||
modules and normalize modules may both use deterministic and LLM-backed
|
|
||||||
validators.
|
|
||||||
- The Seriatim MVP should support only the minimal Seriatim schema. Broader
|
|
||||||
Seriatim schema support should be added later without changing core source
|
|
||||||
contracts.
|
|
||||||
- Core source metadata should remain `map[string]any`. Well-known metadata keys
|
|
||||||
should be documented as conventions, and input modules may expose typed
|
|
||||||
accessor helpers for their own metadata.
|
|
||||||
- Configuration should use named pipeline profiles selected by ID at the CLI.
|
|
||||||
- A pipeline profile should be a fixed template, not a free-form DAG: shared
|
|
||||||
input and chunk stages, one or more artifact lanes, and shared output.
|
|
||||||
- `--only` should select configured artifact lanes without creating ad hoc
|
|
||||||
pipelines.
|
|
||||||
- Module bindings should support string shorthand and inline object settings,
|
|
||||||
normalized into one internal binding shape.
|
|
||||||
- Registries should expose flat capability metadata so config can fail fast on
|
|
||||||
invalid module combinations.
|
|
||||||
- The run manifest should record both `pipeline_id` and a digest of the resolved
|
|
||||||
pipeline definition after defaults and lane selection.
|
|
||||||
|
|
||||||
## Open Design Questions
|
|
||||||
|
|
||||||
- Which artifact types should use generic append-in-chunk-order merge, and which
|
|
||||||
should use domain-specific merge?
|
|
||||||
- Which artifact types need domain-specific normalization for deduplication,
|
|
||||||
identity resolution, or consistency?
|
|
||||||
- Which operational settings should be allowed as CLI/environment overrides
|
|
||||||
without weakening pipeline provenance?
|
|
||||||
|
|
||||||
## Near-Term Documentation Tasks
|
|
||||||
|
|
||||||
Once behavior is implemented, move implemented contracts out of roadmap docs and
|
|
||||||
into canonical docs:
|
|
||||||
|
|
||||||
- `README.md` for purpose and shortest useful command;
|
|
||||||
- `docs/cli.md` for CLI behavior;
|
|
||||||
- `docs/config.md` for config fields and precedence;
|
|
||||||
- `docs/internal/` for implemented architecture and package boundaries;
|
|
||||||
- `docs/integrations/` for source input and artifact file formats;
|
|
||||||
- `examples/` for maintained source, config, and artifact examples.
|
|
||||||
@@ -1,157 +1,29 @@
|
|||||||
# MVP Roadmap
|
# Future Work
|
||||||
|
|
||||||
## Status
|
Current Notarius behavior is documented in the canonical README, CLI,
|
||||||
|
configuration, operations, internal, and integration docs. This roadmap records
|
||||||
|
future work only.
|
||||||
|
|
||||||
The first functional Notarius MVP implementation is complete enough to start the
|
## Candidate Product Work
|
||||||
deferred documentation pass.
|
|
||||||
|
|
||||||
The previous numbered checkpoint roadmaps have been implemented and retired.
|
- Additional input adapters, such as Markdown or note-export formats.
|
||||||
This document now records the implemented MVP scope and the remaining
|
- Additional D&D extractors beyond spell casts.
|
||||||
release/documentation work before alpha `0.1.0`.
|
- Cross-lane entity normalization.
|
||||||
|
- Cross-chunk semantic deduplication.
|
||||||
|
- Configurable validator chains with production validator modules.
|
||||||
|
- Multiple effective LLM profiles in one run.
|
||||||
|
- Parallel execution where it preserves deterministic manifests and diagnostics.
|
||||||
|
- Additional output encoders.
|
||||||
|
|
||||||
Implementation staging belongs in [`implementation.md`](implementation.md).
|
## Candidate Operational Work
|
||||||
|
|
||||||
## MVP Goal
|
- Packaged release artifacts for alpha distribution.
|
||||||
|
- A documented versioning and release process.
|
||||||
|
- Optional generated example output fixtures with a regeneration procedure.
|
||||||
|
- Additional diagnostics or reporting views if operator workflows need them.
|
||||||
|
|
||||||
The MVP should let a user run Notarius against a Seriatim minimal transcript
|
## Non-Goals To Revisit Deliberately
|
||||||
JSON file, select a configured pipeline profile, extract D&D spell-cast
|
|
||||||
artifacts with an LLM-backed extractor, validate those artifacts, and write
|
|
||||||
durable JSON output plus diagnostics.
|
|
||||||
|
|
||||||
The intended command shape is:
|
- A general workflow language.
|
||||||
|
- Structural module selection through ad hoc run flags.
|
||||||
```sh
|
- Storing secrets in config files, diagnostics, manifests, or examples.
|
||||||
notarius run dnd-session --input session-014.json
|
|
||||||
notarius run dnd-session --input session-014.json --only spells
|
|
||||||
```
|
|
||||||
|
|
||||||
The MVP remains intentionally narrow:
|
|
||||||
|
|
||||||
- one production input module: `seriatim`;
|
|
||||||
- one production extract module: `dnd/spells`;
|
|
||||||
- one generic chunk module sufficient for transcript-scale processing;
|
|
||||||
- generic append-order merge;
|
|
||||||
- generic noop normalization;
|
|
||||||
- generic JSON output;
|
|
||||||
- config-driven pipeline profiles;
|
|
||||||
- OpenAI-compatible structured LLM execution through the existing LLM client.
|
|
||||||
|
|
||||||
## Implemented MVP Behavior
|
|
||||||
|
|
||||||
The MVP now includes:
|
|
||||||
|
|
||||||
- framework/domain asset boundaries: D&D spell prompt and response schema assets
|
|
||||||
are owned by `internal/modules/extract/dnd/spells`, while framework prompt and
|
|
||||||
LLM packages provide only generic primitives;
|
|
||||||
- production CLI wiring that registers `seriatim`, `dnd/spells`, and the
|
|
||||||
default `generic`, `appendorder`, `noop`, and `json` modules;
|
|
||||||
- a config-driven `notarius run` command that reads Seriatim input, resolves a
|
|
||||||
configured pipeline, invokes the runner, and writes durable output plus
|
|
||||||
diagnostics;
|
|
||||||
- fixture-driven CLI acceptance coverage using maintained example config and
|
|
||||||
transcript fixtures with a fake LLM path, so `go test ./...` exercises the MVP
|
|
||||||
without network access.
|
|
||||||
|
|
||||||
The production pipeline defaults are:
|
|
||||||
|
|
||||||
- `chunk: generic`;
|
|
||||||
- `merge: appendorder`;
|
|
||||||
- `normalize: noop`;
|
|
||||||
- `output: json`.
|
|
||||||
|
|
||||||
The implemented command shape is:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
notarius run <pipeline-id> --input path/to/source.json
|
|
||||||
notarius run <pipeline-id> --input path/to/source.json --only spells
|
|
||||||
```
|
|
||||||
|
|
||||||
Current output and diagnostics behavior:
|
|
||||||
|
|
||||||
- output is written under `<output-root>/<run-id>/`, defaulting to
|
|
||||||
`./notarius-output`;
|
|
||||||
- JSON output includes a manifest, grouped approved artifacts, rejected
|
|
||||||
artifacts, and warnings;
|
|
||||||
- artifact records include generic source references;
|
|
||||||
- output-stage warnings remain out-of-band from artifact payloads and are
|
|
||||||
captured for CLI reporting and diagnostics;
|
|
||||||
- run manifest data includes source digest, resolved pipeline digest, LLM
|
|
||||||
profile/model metadata, prompt/schema identifiers, and validation status;
|
|
||||||
- diagnostics redact secrets and include invocation metadata, redacted effective
|
|
||||||
config, resolved pipeline, manifest, warnings, run report, and error logs.
|
|
||||||
|
|
||||||
Maintained MVP fixtures:
|
|
||||||
|
|
||||||
- `examples/dnd-spells.config.yml`;
|
|
||||||
- `examples/seriatim-minimal-transcript.json`;
|
|
||||||
- `internal/cli/testdata/invalid-seriatim-empty-segments.json`.
|
|
||||||
|
|
||||||
## Remaining Release And Documentation Work
|
|
||||||
|
|
||||||
The full documentation pass is intentionally deferred until MVP functionality
|
|
||||||
exists. It should happen before tagging alpha `0.1.0`.
|
|
||||||
|
|
||||||
Remaining work before alpha `0.1.0`:
|
|
||||||
|
|
||||||
- move implemented CLI behavior into `docs/cli.md`;
|
|
||||||
- move implemented config behavior into `docs/config.md`;
|
|
||||||
- document run output and diagnostics behavior in canonical docs;
|
|
||||||
- update integration docs for Seriatim input, D&D spell artifacts, and JSON
|
|
||||||
output where needed;
|
|
||||||
- update `README.md` with a shortest useful command based on the maintained
|
|
||||||
examples;
|
|
||||||
- keep examples validated by tests.
|
|
||||||
|
|
||||||
## Out Of Scope For MVP
|
|
||||||
|
|
||||||
- D&D item extraction;
|
|
||||||
- NPC extraction;
|
|
||||||
- combat extraction;
|
|
||||||
- D&D rules validation beyond the spell extractor's deterministic checks;
|
|
||||||
- Markdown or Obsidian input;
|
|
||||||
- cross-lane entity normalization;
|
|
||||||
- cross-chunk semantic deduplication beyond whatever a simple normalizer can
|
|
||||||
safely support;
|
|
||||||
- a general DAG or workflow engine;
|
|
||||||
- ad hoc CLI flags for structural module selection;
|
|
||||||
- release-quality documentation before the MVP behavior is implemented.
|
|
||||||
|
|
||||||
## MVP Done Criteria
|
|
||||||
|
|
||||||
- D&D prompt and response schema assets are owned by the D&D spells module, not
|
|
||||||
by framework packages.
|
|
||||||
- Production CLI commands use a real app catalog rather than test-injected
|
|
||||||
module catalogs.
|
|
||||||
- A config profile can bind `input: seriatim` and an artifact lane with
|
|
||||||
`extract: dnd/spells`.
|
|
||||||
- Default `generic`, `appendorder`, `noop`, and `json` modules resolve through
|
|
||||||
production wiring.
|
|
||||||
- `notarius config validate --config <file> --pipeline <id>` works with the
|
|
||||||
MVP config.
|
|
||||||
- `notarius pipelines list --config <file>` works with the MVP config.
|
|
||||||
- `notarius run <pipeline-id> --input <file>` reads a Seriatim transcript,
|
|
||||||
extracts D&D spell artifacts, validates them, and writes JSON output.
|
|
||||||
- `notarius run <pipeline-id> --input <file> --only spells` runs only the
|
|
||||||
selected artifact lane.
|
|
||||||
- The run manifest records source digest, resolved pipeline digest, LLM profile
|
|
||||||
and model, prompt/schema identifiers, and validation status.
|
|
||||||
- Output warnings are available to CLI/diagnostics without becoming artifact
|
|
||||||
payload fields.
|
|
||||||
- MVP fixture tests cover the full path without network access.
|
|
||||||
- `go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` pass.
|
|
||||||
|
|
||||||
## Deferred Documentation Pass
|
|
||||||
|
|
||||||
Before alpha `0.1.0`, complete a full documentation pass/rewrite. That pass
|
|
||||||
should move implemented behavior out of roadmap documents and into canonical
|
|
||||||
docs required by
|
|
||||||
[`../policy/documentation.md`](../policy/documentation.md), including at least:
|
|
||||||
|
|
||||||
- `README.md`;
|
|
||||||
- `docs/cli.md`;
|
|
||||||
- `docs/config.md`;
|
|
||||||
- `docs/operations.md`, if diagnostics/run recovery behavior warrants it;
|
|
||||||
- `docs/internal/` architecture and package-boundary docs;
|
|
||||||
- `docs/integrations/` updates for Seriatim input, D&D spell artifacts, and
|
|
||||||
JSON output;
|
|
||||||
- maintained `examples/` files.
|
|
||||||
|
|||||||
215
docs/troubleshooting.md
Normal file
215
docs/troubleshooting.md
Normal file
@@ -0,0 +1,215 @@
|
|||||||
|
# Troubleshooting
|
||||||
|
|
||||||
|
This guide maps common implemented failure modes to inspection steps and fixes.
|
||||||
|
For command syntax, see [CLI Reference](cli.md). For YAML fields and
|
||||||
|
environment overrides, see [Configuration](config.md). For output and
|
||||||
|
diagnostics layout, see [Operations](operations.md).
|
||||||
|
|
||||||
|
## Config File Not Found
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
```text
|
||||||
|
notarius: config file not found; pass --config or set NOTARIUS_CONFIG
|
||||||
|
```
|
||||||
|
|
||||||
|
Fix:
|
||||||
|
|
||||||
|
- Pass `--config path/to/config.yml`.
|
||||||
|
- Or set `NOTARIUS_CONFIG` to a readable file.
|
||||||
|
- Or install a config at `/usr/local/etc/notarius/config.yml`.
|
||||||
|
|
||||||
|
If the message says the config path is a directory or is not available, correct
|
||||||
|
the path or file permissions.
|
||||||
|
|
||||||
|
## Unsupported Or Invalid Config
|
||||||
|
|
||||||
|
Symptoms include:
|
||||||
|
|
||||||
|
- `unsupported config version`
|
||||||
|
- `config version is required`
|
||||||
|
- `field <name> not found`
|
||||||
|
- `total LLM concurrency must be greater than zero`
|
||||||
|
- `diagnostics retention "<value>" is not supported`
|
||||||
|
|
||||||
|
Fix:
|
||||||
|
|
||||||
|
- Use `version: 1`.
|
||||||
|
- Remove unknown YAML fields.
|
||||||
|
- Validate with:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/notarius config validate --config path/to/config.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Unknown Pipeline
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
```text
|
||||||
|
notarius: pipeline "..." is not configured
|
||||||
|
```
|
||||||
|
|
||||||
|
Fix:
|
||||||
|
|
||||||
|
- List configured pipeline IDs:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/notarius pipelines list --config path/to/config.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
- Use one of those IDs in `notarius run <pipeline-id>`.
|
||||||
|
- Check indentation under the top-level `pipelines` map.
|
||||||
|
|
||||||
|
## Unknown Or Incompatible Module
|
||||||
|
|
||||||
|
Symptoms mention a module key, pipeline slot, lane, capability, or `not
|
||||||
|
registered`.
|
||||||
|
|
||||||
|
Fix:
|
||||||
|
|
||||||
|
- Validate the pipeline against the production module catalog:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/notarius config validate \
|
||||||
|
--config path/to/config.yml \
|
||||||
|
--pipeline dnd-session
|
||||||
|
```
|
||||||
|
|
||||||
|
- Use only implemented production module keys listed in
|
||||||
|
[Configuration](config.md#implemented-production-modules).
|
||||||
|
- Check that artifact lanes include an `extract` binding.
|
||||||
|
|
||||||
|
## Invalid `--only`
|
||||||
|
|
||||||
|
Symptoms include:
|
||||||
|
|
||||||
|
- `--only must contain comma-separated non-empty artifact lane IDs`
|
||||||
|
- `--only requires --pipeline`
|
||||||
|
- `selected artifact lane`
|
||||||
|
|
||||||
|
Fix:
|
||||||
|
|
||||||
|
- Use comma-separated lane IDs with no empty entries:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/notarius run dnd-session \
|
||||||
|
--config path/to/config.yml \
|
||||||
|
--input path/to/input.json \
|
||||||
|
--only spells
|
||||||
|
```
|
||||||
|
|
||||||
|
- For `config validate`, include `--pipeline` when using `--only`.
|
||||||
|
- Confirm the lane ID exists under `pipelines.<id>.artifacts`.
|
||||||
|
|
||||||
|
## Seriatim Input Validation Failure
|
||||||
|
|
||||||
|
Symptoms include `seriatim input`, `parse JSON`, `segments must not be empty`,
|
||||||
|
or validation errors naming a segment field.
|
||||||
|
|
||||||
|
Fix:
|
||||||
|
|
||||||
|
- Compare the input to
|
||||||
|
[examples/seriatim-minimal-transcript.json](../examples/seriatim-minimal-transcript.json).
|
||||||
|
- Ensure the JSON has a `metadata` object and a non-empty `segments` array.
|
||||||
|
- Each segment needs a non-empty `id`, non-empty `speaker`, non-empty `text`,
|
||||||
|
non-negative numeric `start`, and non-negative numeric `end`.
|
||||||
|
- Segment IDs must be unique and must not contain leading or trailing
|
||||||
|
whitespace.
|
||||||
|
- `end` must be greater than or equal to `start`.
|
||||||
|
|
||||||
|
## Missing LLM Base URL Or Model
|
||||||
|
|
||||||
|
Symptoms include:
|
||||||
|
|
||||||
|
- `LLM profile "default" base URL must not be empty`
|
||||||
|
- `LLM profile "default" model must not be empty`
|
||||||
|
- `base URL must be valid`
|
||||||
|
|
||||||
|
Fix:
|
||||||
|
|
||||||
|
- Set `base_url` and `model` in `llm_profiles.default`.
|
||||||
|
- Or set `NOTARIUS_LLM_DEFAULT_BASE_URL` and
|
||||||
|
`NOTARIUS_LLM_DEFAULT_MODEL`.
|
||||||
|
- If a profile needs authentication, set `api_key_env` in YAML or set
|
||||||
|
`NOTARIUS_LLM_DEFAULT_API_KEY`.
|
||||||
|
|
||||||
|
## LLM Profile Override Failure
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
```text
|
||||||
|
notarius: LLM profile override "..." is not configured
|
||||||
|
```
|
||||||
|
|
||||||
|
Fix:
|
||||||
|
|
||||||
|
- Add the profile under `llm_profiles`.
|
||||||
|
- Or use an existing profile ID with `--llm-profile`.
|
||||||
|
|
||||||
|
Current runs require exactly one distinct effective LLM profile. If a pipeline
|
||||||
|
uses several profiles, run with `--llm-profile <id>` or align the bindings in
|
||||||
|
configuration.
|
||||||
|
|
||||||
|
## Provider HTTP Or Response Failure
|
||||||
|
|
||||||
|
Symptoms include:
|
||||||
|
|
||||||
|
- `provider request failed`
|
||||||
|
- `provider returned status 400`
|
||||||
|
- `provider returned status 403`
|
||||||
|
- `provider response missing choices`
|
||||||
|
- `provider response assistant message content is not valid JSON`
|
||||||
|
- `decode structured output`
|
||||||
|
|
||||||
|
Fix:
|
||||||
|
|
||||||
|
- Confirm the `base_url` points to an OpenAI-compatible endpoint root. Notarius
|
||||||
|
posts to `<base_url>/chat/completions`.
|
||||||
|
- Check `model` and provider credentials.
|
||||||
|
- Inspect the retained diagnostics `error.log`.
|
||||||
|
- For 400 and 403 responses, fix the request configuration or credentials.
|
||||||
|
- For 429 and 5xx responses, the client retries according to `max_retries`; if
|
||||||
|
the failure persists, inspect the provider response and adjust capacity,
|
||||||
|
credentials, or model settings.
|
||||||
|
- The assistant message content must decode as JSON matching the extractor's
|
||||||
|
structured response schema.
|
||||||
|
|
||||||
|
Provider error messages are redacted for configured API key values.
|
||||||
|
|
||||||
|
## Output Write Failure
|
||||||
|
|
||||||
|
Symptoms include:
|
||||||
|
|
||||||
|
- `create output directory`
|
||||||
|
- `write output file`
|
||||||
|
- `output file name must`
|
||||||
|
|
||||||
|
Fix:
|
||||||
|
|
||||||
|
- Ensure `--output-dir` points to a directory path or a path that can be
|
||||||
|
created.
|
||||||
|
- Check filesystem permissions and available disk space.
|
||||||
|
- If diagnostics were retained, inspect `run-report.json`, `run-manifest.json`,
|
||||||
|
and `error.log`.
|
||||||
|
|
||||||
|
The CLI rejects unsafe logical output paths before writing files.
|
||||||
|
|
||||||
|
## Diagnostics Directory Surprise
|
||||||
|
|
||||||
|
Symptom: the diagnostics directory is missing after a successful run.
|
||||||
|
|
||||||
|
Fix:
|
||||||
|
|
||||||
|
- Check `diagnostics.retention`.
|
||||||
|
- With `auto`, successful runs without warnings are removed.
|
||||||
|
- Use `diagnostics.retention: always` when every diagnostics run directory
|
||||||
|
should be kept.
|
||||||
|
- Use `--diagnostics-dir` to override the configured work directory for a run.
|
||||||
|
|
||||||
|
Symptom: diagnostics exist even with `retention: never`.
|
||||||
|
|
||||||
|
Explanation:
|
||||||
|
|
||||||
|
- Failed runs are retained so that `error.log` and available context can be
|
||||||
|
inspected.
|
||||||
Reference in New Issue
Block a user