32 Commits

Author SHA1 Message Date
b85e826c1c Bugfix in the seriatim input adapter 2026-07-03 22:27:54 -05:00
4d0b2c69e6 Clean up completed documentation roadmaps 2026-07-04 03:13:27 +00:00
30e98a4d99 Add integration and artifact contract documentation 2026-07-04 03:11:29 +00:00
6ceabf40bb Add developer and internal architecture documentation 2026-07-04 03:07:56 +00:00
2d75f6ad13 Add operations and troubleshooting documentation 2026-07-04 03:04:01 +00:00
4dd00347e0 Add user documentation for Notarius CLI and config 2026-07-04 03:01:38 +00:00
5fab9936d0 Add a roadmap for a full documentation pass 2026-07-03 21:55:58 -05:00
b738dbc1eb Update policy documentation in advance of a full documentation pass 2026-07-03 21:46:27 -05:00
4db7805a95 Clean up MVP output handling 2026-07-03 21:26:05 -05:00
5424aae3de Mark MVP implementation complete in roadmap 2026-07-04 01:17:51 +00:00
6310e49fce Add fixture-driven MVP acceptance coverage 2026-07-04 01:15:59 +00:00
7de41eb3bd Write run outputs and diagnostics 2026-07-04 01:11:57 +00:00
ae218d7c57 Add in-memory pipeline run command 2026-07-04 01:06:04 +00:00
361b1f53f4 Wire production CLI catalog and scheduled LLM client 2026-07-04 00:59:32 +00:00
0ad96618fc Add production default pipeline modules 2026-07-04 00:55:18 +00:00
bc4203a264 Add run manifest and logical output file contracts 2026-07-04 00:45:21 +00:00
ac14667797 Move D&D spell assets into extractor module 2026-07-04 00:39:31 +00:00
07b3264b6b Add a staged implementation plan for MVP 2026-07-03 19:33:53 -05:00
0a4a29a2df Document D&D spells extraction contract 2026-07-03 23:48:38 +00:00
373ba13562 Add D&D spells pipeline integration tests 2026-07-03 23:47:06 +00:00
c51c2d7fe5 Add D&D spells validators 2026-07-03 23:42:12 +00:00
b2012b9b2c Implement D&D spells extraction mapping 2026-07-03 23:38:41 +00:00
404bfc8da5 Add D&D spells prompt assets 2026-07-03 23:34:10 +00:00
382ca3fb6c Add D&D spells response schema 2026-07-03 23:31:19 +00:00
d9b5c25cfe Add D&D spells extractor skeleton 2026-07-03 23:29:08 +00:00
a4d64f6f16 Plan D&D spells extractor implementation 2026-07-03 23:25:32 +00:00
39fcfba605 Tighten Seriatim input validation 2026-07-03 23:14:33 +00:00
91bec3ae52 Document Seriatim input integration 2026-07-03 21:18:24 +00:00
c804cb4bca Add Seriatim runner integration coverage 2026-07-03 21:17:10 +00:00
ef8bd91d48 Add Seriatim pipeline config coverage 2026-07-03 21:15:06 +00:00
95a54505cf Implement Seriatim transcript parsing 2026-07-03 21:13:12 +00:00
47013daa04 Add Seriatim input module skeleton 2026-07-03 21:09:43 +00:00
92 changed files with 9783 additions and 2750 deletions

View File

@@ -1,2 +1,36 @@
# go-application-template
# Notarius
Notarius is a Go CLI for extracting structured artifacts from source material
with explicit, configurable pipeline modules.
The current implementation reads Seriatim transcript JSON, chunks the source
units, extracts D&D spell-cast artifacts with an OpenAI-compatible LLM, and
writes JSON output plus diagnostics for each run.
```sh
NOTARIUS_LLM_DEFAULT_BASE_URL=http://127.0.0.1:8080/v1 \
NOTARIUS_LLM_DEFAULT_MODEL=your-model \
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json
```
If the provider requires authentication, set
`NOTARIUS_LLM_DEFAULT_API_KEY` in the environment before running the command.
Outputs are written under `./notarius-output/<run-id>/` unless `--output-dir`
is provided.
Useful references:
- [CLI reference](docs/cli.md)
- [Configuration reference](docs/config.md)
- [Operations](docs/operations.md)
- [Troubleshooting](docs/troubleshooting.md)
- [Seriatim input contract](docs/integrations/seriatim.md)
- [OpenAI-compatible provider contract](docs/integrations/openai-compatible.md)
- [JSON output contract](docs/integrations/json-output.md)
- [D&D spell artifact contract](docs/integrations/dnd-spell-artifacts.md)
- [Developer workflow](docs/policy/development.md)
- [Internal architecture docs](docs/internal/overview.md)
- [Maintained example config](examples/dnd-spells.config.yml)
- [Maintained example input](examples/seriatim-minimal-transcript.json)

129
docs/cli.md Normal file
View File

@@ -0,0 +1,129 @@
# CLI Reference
This is the canonical reference for the implemented Notarius command-line
interface.
## Quick Run
```sh
NOTARIUS_LLM_DEFAULT_BASE_URL=http://127.0.0.1:8080/v1 \
NOTARIUS_LLM_DEFAULT_MODEL=your-model \
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json
```
Set `NOTARIUS_LLM_DEFAULT_API_KEY` if the OpenAI-compatible provider requires
a bearer token.
## Commands
```text
notarius help
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b]
notarius config validate --config path/to/config.yml [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list --config path/to/config.yml [--json]
```
Running `notarius` with no arguments, `notarius help`, `notarius --help`, or
`notarius -h` prints usage and exits successfully.
## `run`
`notarius run <pipeline-id>` executes a configured pipeline against one input
file.
Flags:
- `--input path`: required source input file.
- `--config path`: config file path. If omitted, Notarius checks
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are
comma-separated and must be non-empty.
- `--output-dir path`: output root. The run writes to `<path>/<run-id>/`.
Defaults to `./notarius-output`.
- `--diagnostics-dir path`: diagnostics work directory override for this
invocation.
- `--llm-profile id`: override every effective module binding to use one LLM
profile.
On success, the command prints the completed pipeline ID, approved and rejected
artifact counts, and the output directory. If the run completes with warnings,
the warning count is printed to stderr.
For durable output, diagnostics, retention, and failure inspection, see
[Operations](operations.md).
The current `run` command requires the resolved pipeline to use exactly one
distinct LLM profile after defaults and overrides are applied.
## `config validate`
`notarius config validate` loads and validates configuration.
Flags:
- `--config path`: config file path. If omitted, discovery uses
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
- `--pipeline pipeline-id`: additionally resolve one configured pipeline against
the production module catalog.
- `--only lane-a,lane-b`: validate resolution for selected artifact lanes. This
flag requires `--pipeline`.
Examples:
```sh
go run ./cmd/notarius config validate \
--config examples/dnd-spells.config.yml
go run ./cmd/notarius config validate \
--config examples/dnd-spells.config.yml \
--pipeline dnd-session \
--only spells
```
## `pipelines list`
`notarius pipelines list` prints configured pipeline IDs in sorted order.
Flags:
- `--config path`: config file path. If omitted, discovery uses
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
- `--json`: print `{"pipelines":[...]}` instead of one ID per line.
Examples:
```sh
go run ./cmd/notarius pipelines list \
--config examples/dnd-spells.config.yml
go run ./cmd/notarius pipelines list \
--config examples/dnd-spells.config.yml \
--json
```
## Exit Codes
- `0`: command succeeded.
- `1`: command syntax was valid, but loading config, resolving modules, running
the pipeline, calling the provider, writing output, or writing diagnostics
failed.
- `2`: command syntax was invalid, a command was unknown, a required argument
was missing, or a flag value was malformed.
## Implemented Production Pipeline Modules
The production CLI currently registers these module keys:
- input: `seriatim`
- chunk: `generic`
- extract: `dnd/spells`
- merge: `appendorder`
- normalize: `noop`
- output: `json`
The production CLI does not currently register validator modules.
For YAML structure, defaults, environment overrides, and module binding syntax,
see [Configuration](config.md).

213
docs/config.md Normal file
View File

@@ -0,0 +1,213 @@
# Configuration
This is the canonical reference for implemented Notarius configuration.
Notarius reads YAML config files with `version: 1`. File config is applied over
built-in defaults, then environment overrides are applied.
## Discovery
Commands that accept `--config` load configuration in this order:
1. the `--config` path, when provided;
2. `NOTARIUS_CONFIG`, when set to a non-empty path;
3. `/usr/local/etc/notarius/config.yml`.
If none is available, the command fails with a config file not found error.
## Minimal Example
```yaml
version: 1
llm_profiles:
default:
provider: openai-compatible
base_url: http://127.0.0.1:8080/v1
model: your-model
pipelines:
dnd-session:
input: seriatim
chunk:
module: generic
options:
max_units: 50
artifacts:
spells:
extract: dnd/spells
```
The maintained fixture is [examples/dnd-spells.config.yml](../examples/dnd-spells.config.yml).
## Top-Level Fields
- `version`: required. The only supported value is `1`.
- `llm_profiles`: optional map of LLM profile IDs to profile settings.
- `pipelines`: optional map of pipeline IDs to pipeline definitions.
- `concurrency`: optional global concurrency settings.
- `diagnostics`: optional diagnostics settings.
Unknown YAML fields are rejected.
## Defaults
Built-in defaults:
```yaml
llm_profiles:
default:
provider: openai-compatible
timeout: 600
max_retries: 3
max_concurrency: 1
concurrency:
total_llm: 1
diagnostics:
work_dir: /tmp/notarius
retention: auto
```
No pipelines are built in. A run requires a configured pipeline.
## LLM Profiles
Each `llm_profiles` entry may contain:
- `provider`: optional provider key. Empty means `openai-compatible`; any other
non-empty value must be `openai-compatible`.
- `base_url`: provider base URL. Required for actual LLM calls.
- `model`: provider model name. Required for actual LLM calls.
- `api_key_env`: environment variable name to read for the API key.
- `timeout`: request timeout as whole seconds or a Go-style duration string such
as `10m`.
- `max_retries`: retry count for provider calls. Must be zero or greater.
- `max_concurrency`: per-profile LLM concurrency. Must be zero or greater; when
zero, Notarius uses `concurrency.total_llm`.
Raw API keys are not accepted as file config fields. Use `api_key_env` or an
environment override.
## Environment Overrides
These environment variables are applied after the config file:
- `NOTARIUS_CONFIG`: config discovery path.
- `NOTARIUS_LLM_DEFAULT_API_KEY`: API key for the `default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_BASE_URL`: base URL for the `default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_MODEL`: model for the `default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_TIMEOUT_SECONDS`: integer timeout seconds for the
`default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_MAX_RETRIES`: integer retry count for the `default` LLM
profile.
- `NOTARIUS_LLM_DEFAULT_MAX_CONCURRENCY`: integer max concurrency for the
`default` LLM profile.
- `NOTARIUS_TOTAL_LLM_CONCURRENCY`: integer global LLM concurrency.
- `NOTARIUS_WORK_DIR`: diagnostics work directory.
- `NOTARIUS_DIAGNOSTICS_RETENTION`: diagnostics retention mode.
Integer environment values must parse as base-10 integers.
## Pipelines
A pipeline defines the fixed Notarius workflow:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
Pipeline fields:
- `input`: required module binding.
- `chunk`: optional module binding. Default module is `generic`.
- `artifacts`: required for pipeline resolution. It maps artifact lane IDs to
lane definitions.
- `output`: optional module binding. Default module is `json`.
Artifact lane fields:
- `extract`: required module binding.
- `merge`: optional module binding. Default module is `appendorder`.
- `normalize`: optional module binding. Default module is `noop`.
- `validators`: optional list of module bindings. The production CLI currently
does not register validator modules.
`notarius run` and `notarius config validate --pipeline` resolve the pipeline
against the production module catalog and fail fast for unknown or incompatible
module keys.
## Module Bindings
Every module binding may use shorthand:
```yaml
input: seriatim
```
or object form:
```yaml
chunk:
module: generic
llm_profile: default
options:
max_units: 50
```
Binding fields:
- `module`: module key.
- `llm_profile`: optional LLM profile ID. Empty means `default`.
- `options`: optional module-specific settings.
The `--llm-profile` run flag overrides every effective module binding to use
one configured profile.
## Implemented Production Modules
| Slot | Key | Notes |
| --- | --- | --- |
| input | `seriatim` | Reads Seriatim transcript JSON. |
| chunk | `generic` | Splits source units into ordered chunks. |
| extract | `dnd/spells` | Extracts `dnd.spell_cast` artifacts. |
| merge | `appendorder` | Keeps candidates in append order. |
| normalize | `noop` | Passes merged artifacts through unchanged. |
| output | `json` | Produces JSON output files. |
The `generic` chunker accepts:
- `max_units`: positive integer, default `50`;
- `overlap_units`: non-negative integer, default `0`, and must be less than
`max_units`.
## Diagnostics
`diagnostics` fields:
- `work_dir`: directory for per-run diagnostics. Default: `/tmp/notarius`.
- `retention`: `auto`, `always`, or `never`. Empty uses `auto`.
`auto` retains diagnostics for failed runs and successful runs with warnings.
`always` retains diagnostics for every run. `never` removes diagnostics for
successful runs without regard to warnings; failed runs are retained.
The `--diagnostics-dir` run flag overrides `diagnostics.work_dir` for that
invocation.
## Validation
Configuration validation checks:
- supported config version and known YAML fields;
- non-empty, non-duplicated IDs after trimming;
- supported LLM provider and non-negative profile limits;
- positive global LLM concurrency;
- supported diagnostics retention and non-empty work directory;
- module binding LLM profiles refer to configured profiles.
Pipeline resolution additionally checks:
- the pipeline ID exists;
- at least one artifact lane is declared and selected;
- selected lanes exist when `--only` is used;
- required module keys are present;
- module keys are registered for the expected slot;
- module capability requirements are satisfied.

View File

@@ -0,0 +1,152 @@
# D&D Spell-Cast Artifacts
This document is the durable artifact contract for approved
`dnd.spell_cast` artifacts produced by the implemented `dnd/spells` extractor.
## Artifact Identity
- Extractor key: `dnd/spells`
- Artifact type: `dnd.spell_cast`
- Schema version: `v1`
- Prompt ID: `dnd.spells`
- Response schema key: `dnd_spells`
- Response schema ID: `notarius.dnd.spells`
- Response schema name: `notarius_dnd_spells_v1`
The extractor requires source chunks and transcript source capability. It
returns generic artifact candidates that are serialized by the JSON output
module.
## Artifact Envelope
Approved artifacts use the generic artifact envelope documented in
[JSON Output](json-output.md#artifact-files):
```json
{
"extractor_key": "dnd/spells",
"artifact_type": "dnd.spell_cast",
"schema_version": "v1",
"payload": {
"caster": "Aria",
"spell": "Cure Wounds",
"effect": "heals an injured ally",
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds."
},
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": "seg-001",
"end_unit_id": "seg-001"
}
]
}
```
## Payload Fields
The `payload` object contains:
- `caster`: in-world character or creature casting the spell;
- `spell`: spell name;
- `effect`: concise spell effect in the scene;
- `narrative_description`: short description of the spell cast in context.
All payload fields are strings and must be non-empty after trimming.
`caster` is the in-world caster, not the transcript speaker.
## Source References
Source references live on the artifact envelope as `source_refs`; they are not
duplicated inside the `payload`.
Each source reference uses the generic source-reference shape:
- `source_id`
- `start_unit_id`
- `end_unit_id`
Validation requires:
- at least one source reference;
- non-empty source ID and unit IDs;
- source ID matching the source document ID;
- start and end unit IDs existing in the source document;
- start unit appearing before or at the same position as end unit.
## Structured LLM Response Shape
The extractor asks the LLM for this top-level response shape:
```json
{
"spell_casts": [
{
"caster": "Aria",
"spell": "Cure Wounds",
"effect": "heals an injured ally",
"narrative_description": "Aria raises her holy symbol and casts Cure Wounds.",
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": "seg-001",
"end_unit_id": "seg-001"
}
]
}
]
}
```
`spell_casts` must be present. It may be empty when no spell casts are found.
The response schema asset is embedded at
`internal/modules/extract/dnd/spells/assets/schemas/dnd_spells.v1.json`.
## Validators
The extractor supplies two deterministic validators by default:
- `dnd/spells/shape`
- `dnd/spells/source_refs`
Rejection reason codes:
- `invalid_payload`: payload JSON cannot be decoded as a spell-cast payload.
- `missing_required_field`: `caster`, `spell`, `effect`, or
`narrative_description` is blank.
- `missing_source_ref`: candidate has no source references.
- `invalid_source_ref`: at least one source reference fails generic source
reference validation.
Rejected candidates are written to `rejected.json` by the JSON output module.
## Manifest Metadata
The extractor adds prompt and response-schema provenance under the artifact lane
manifest metadata:
```json
{
"metadata": {
"extractor": {
"prompt_id": "dnd.spells",
"prompt_version": "v1",
"prompt_sha256": "sha256:...",
"response_schema_key": "dnd_spells",
"response_schema_id": "notarius.dnd.spells",
"response_schema_name": "notarius_dnd_spells_v1",
"response_schema_version": "v1",
"response_schema_sha256": "sha256:..."
}
}
}
```
Raw prompt and schema content are not included in manifest metadata.
## Compatibility Limit
This contract covers only `dnd.spell_cast` artifacts produced by the
implemented spell-cast extractor.

View File

@@ -0,0 +1,192 @@
# JSON Output
This document is the durable JSON output file-format contract produced by the
implemented `json` output module and written by the CLI.
## Output Directory
The CLI writes logical output files under:
```text
<output-root>/<run-id>/
```
The default output root is `./notarius-output`. Operational behavior is covered
in [Operations](../operations.md).
## Files
The `json` output module writes:
- `index.json`
- `manifest.json`
- `artifacts/<artifact-type>.json`, one file per approved artifact type
- `rejected.json`
- `warnings.json`
Files are pretty-printed JSON with a trailing newline.
## `index.json`
Shape:
```json
{
"manifest_file": "manifest.json",
"artifact_files": [
{
"artifact_type": "dnd.spell_cast",
"file": "artifacts/dnd.spell_cast.json"
}
],
"rejected_file": "rejected.json",
"warnings_file": "warnings.json"
}
```
`artifact_files` is sorted by artifact type. It is empty when no artifacts are
approved.
## `manifest.json`
`manifest.json` contains a run manifest:
```json
{
"run_id": "run-123",
"pipeline_id": "dnd-session",
"pipeline_digest": "sha256:...",
"input_module": "seriatim",
"chunker": "generic",
"source_digests": ["sha256:..."],
"extractors": ["dnd/spells"],
"merger": "appendorder",
"normalizer": "noop",
"output_encoder": "json",
"artifact_lanes": [
{
"id": "spells",
"extractor": "dnd/spells",
"merger": "appendorder",
"normalizer": "noop"
}
],
"llm_profiles": [
{
"id": "default",
"provider": "openai-compatible",
"model": "configured-model"
}
],
"validation_status": "approved",
"started_at": "2026-01-01T00:00:00Z",
"completed_at": "2026-01-01T00:00:01Z"
}
```
Fields with empty values may be omitted by JSON encoding.
`validation_status` is `approved` when no candidates were rejected and
`rejected` when one or more candidates were rejected.
## Artifact Files
Each artifact file has this shape:
```json
{
"artifact_type": "dnd.spell_cast",
"artifacts": [
{
"extractor_key": "dnd/spells",
"artifact_type": "dnd.spell_cast",
"schema_version": "v1",
"payload": {},
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": "seg-001",
"end_unit_id": "seg-001"
}
]
}
]
}
```
Artifact envelope fields:
- `extractor_key`: extractor module key.
- `artifact_type`: artifact type.
- `schema_version`: artifact schema version.
- `payload`: artifact-type-specific JSON payload.
- `source_refs`: optional generic source references.
- `metadata`: optional artifact metadata.
Artifact file names are produced by sanitizing the artifact type:
- characters outside `A-Z`, `a-z`, `0-9`, `.`, `_`, and `-` become `_`;
- repeated `..` sequences are replaced;
- leading and trailing `.`, `_`, and `-` are trimmed;
- empty sanitized names are rejected.
For current D&D spell-cast artifacts, the file is
`artifacts/dnd.spell_cast.json`.
## `rejected.json`
Shape:
```json
{
"rejected": [
{
"candidate": {
"index": 0,
"extractor_key": "dnd/spells",
"artifact_type": "dnd.spell_cast",
"schema_version": "v1",
"payload": {},
"source_refs": []
},
"validator_name": "dnd/spells/source_refs",
"reason_code": "missing_source_ref",
"message": "spell cast candidate must include at least one source ref"
}
]
}
```
`rejected` is an empty array when no candidates are rejected.
## `warnings.json`
Shape:
```json
{
"warnings": [
{
"scope": "output",
"reason_code": "example_warning",
"message": "warning message"
}
]
}
```
`warnings` is an empty array when no warnings are reported.
## Path Safety
The output module returns slash-separated logical paths. The CLI also validates
logical output names before writing:
- names must be non-empty;
- names must be relative;
- names must be clean;
- names must use `/`, not `\`;
- names must not contain `..`;
- resolved paths must stay under the run output directory.
Durable writes are atomic per file.

View File

@@ -0,0 +1,128 @@
# OpenAI-Compatible Structured Output
This document describes the external LLM provider contract implemented by the
production Notarius LLM client.
## Provider
- Provider key: `openai-compatible`
- HTTP method: `POST`
- Endpoint: `<base_url>/chat/completions`
- Request body: JSON
- Response mode: chat completions with structured JSON schema output
`base_url` is trimmed of trailing slashes before `/chat/completions` is
appended. Configure provider settings in [Configuration](../config.md).
## Request
The client sends a JSON object with:
```json
{
"model": "configured-model",
"messages": [
{
"role": "system",
"content": "..."
},
{
"role": "user",
"content": "..."
}
],
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "schema_name",
"strict": true,
"schema": {}
}
}
}
```
Implemented request behavior:
- `model` comes from the structured completion request when set, otherwise from
the configured LLM profile.
- `messages` must be non-empty; each role and content must be non-empty after
trimming.
- `response_format.type` is always `json_schema`.
- `response_format.json_schema.strict` is always `true`.
- `response_format.json_schema.name` and `schema` come from the extractor or
validator making the call.
If an API key is configured, the client sends:
```text
Authorization: Bearer <api-key>
```
The client always sends `Content-Type: application/json`.
## Response
The client expects a JSON response with at least one choice:
```json
{
"model": "provider-model",
"choices": [
{
"message": {
"content": "{\"field\":\"value\"}"
}
}
],
"usage": {
"prompt_tokens": 10,
"completion_tokens": 5,
"total_tokens": 15
}
}
```
`choices[0].message.content` may be either:
- a JSON string whose contents are valid JSON; or
- raw JSON.
The decoded content is unmarshaled into the caller-provided structured output
target. If `usage` is present, prompt, completion, and total token counts are
copied into the completion response.
## Errors And Retries
The client validates base URL, model, response schema name, response schema
JSON, messages, and output target before or during the call.
Retryable failures:
- HTTP request failure;
- response body read failure;
- HTTP `429`;
- HTTP `5xx`;
- malformed provider response envelope;
- missing choices;
- missing, empty, or invalid assistant JSON content;
- structured-output decode failure.
Non-retryable provider status codes include non-`429` `4xx` responses.
Provider error bodies are parsed for `error.message` or `message` when present.
Configured API key values and bearer-token values are redacted from returned
provider errors.
## Timeouts And Concurrency
The configured profile timeout is applied per provider request when greater
than zero. Context cancellation is respected.
The production CLI wraps the provider client with the LLM scheduler. Effective
concurrency is described in [LLM runtime internals](../internal/llm.md).
## Limits
This contract documents only the fields the implemented client sends and reads.
Provider-specific extensions are ignored unless they affect those fields.

View File

@@ -0,0 +1,120 @@
# Seriatim Transcript JSON
This document is the external input contract for the implemented `seriatim`
input adapter.
## Adapter
- Module key: `seriatim`
- Document kind: `transcript`
- Unit kind: `transcript_segment`
- Source format: `application/vnd.seriatim+json`
The adapter parses raw Seriatim JSON into a generic source document. It owns
transcript-specific JSON parsing and metadata mapping; core source and pipeline
code stay source-format agnostic.
## Accepted Shape
The input must be one JSON object with top-level `metadata` and `segments`
fields. This covers the maintained minimal fixture and Seriatim intermediate
output that provides the same required segment fields.
```json
{
"metadata": {
"id": "session-alpha",
"title": "Synthetic D&D spell session"
},
"segments": [
{
"id": "seg-001",
"start": 0,
"end": 4,
"speaker": "Aria",
"text": "Aria raises her holy symbol and casts Cure Wounds."
}
]
}
```
The maintained example is
[examples/seriatim-minimal-transcript.json](../../examples/seriatim-minimal-transcript.json).
Top-level metadata entries are preserved. Other segment fields, such as
`categories`, are ignored.
Multiple top-level JSON values are rejected.
## Validation
The adapter rejects:
- empty raw input;
- malformed JSON;
- top-level JSON that is not an object;
- missing, null, or non-object `metadata`;
- missing, null, non-array, or empty `segments`;
- segment values that are not objects;
- segment `id` values that are neither strings nor numbers;
- non-string `speaker` or `text`;
- empty segment IDs;
- segment IDs with leading or trailing whitespace;
- duplicate segment IDs;
- missing or empty `speaker`;
- missing, empty, invalid, non-finite, or negative `start`;
- missing, empty, invalid, non-finite, or negative `end`;
- `end` values before `start`;
- missing or empty `text`.
Segment text is preserved as provided, but it must not be empty after trimming.
## Source Mapping
The adapter maps input to `SourceDocument`:
- `metadata` becomes `SourceDocument.Metadata`;
- `SourceDocument.Kind` is `transcript`;
- `SourceDocument.Format` is `application/vnd.seriatim+json`;
- `SourceDocument.Digest` is `sha256:<hex>` of the exact raw input bytes.
`SourceDocument.ID` is selected in this order:
1. the parse request source ID, after trimming;
2. `metadata.id`, when it is a non-empty string after trimming;
3. `metadata.source_id`, when it is a non-empty string after trimming;
4. `seriatim:<first-16-hex-chars-of-raw-sha256>`.
Each segment becomes one `SourceUnit`:
- `segment.id` becomes `SourceUnit.ID`; numeric IDs are converted to their JSON
number text, so `1` becomes `"1"`;
- `segment.text` becomes `SourceUnit.Text`;
- `SourceUnit.Kind` is `transcript_segment`;
- `speaker`, `start`, and `end` are stored in source-unit metadata.
## Metadata Keys
Seriatim unit metadata uses these keys:
- `speaker`: string speaker label;
- `start`: `json.Number` start value;
- `end`: `json.Number` end value.
The `internal/modules/input/seriatim` package exposes typed accessors for these
values.
## Capabilities
The module declares these provided capabilities:
- `source.transcript`
- `transcript.speaker`
- `transcript.timestamps`
## Compatibility Limit
This contract covers only Seriatim transcript JSON with the top-level
`metadata` object and `segments` array described here. Broader Seriatim output
schemas are compatible only when they provide these required fields with the
accepted types.

View File

@@ -0,0 +1,88 @@
# Diagnostics Internals
Diagnostics internals live in `internal/core/diagnostics`. Operator-facing run
behavior is documented in [Operations](../operations.md).
## Purpose
Diagnostics provide local inspection artifacts for a run without becoming the
durable output contract. Durable user output is produced by output modules and
written by the CLI.
Diagnostics must not expose secrets.
## Run Directory
`NewRunDirectory(workDir, retention)` creates:
```text
<workDir>/run-<unix-nanoseconds>/
```
If `workDir` is empty, it defaults to `/tmp/notarius`. Empty retention defaults
to `auto`.
The writer makes the work directory if needed, then attempts to create a unique
run directory. It retries run ID creation a bounded number of times if a
collision occurs.
## Artifact Writers
Implemented artifact names:
- `invocation.json`
- `effective-config.json`
- `resolved-pipeline.json`
- `source-document.json`
- `run-manifest.json`
- `run-report.json`
- `warnings.json`
- `error.log`
JSON artifacts are encoded with indentation and a trailing newline. Writes are
atomic through a temporary file in the target directory followed by rename.
Artifact names must be single relative file names. Absolute paths, path
separators, and names resolving outside the run directory are rejected.
## Redacted Effective Config
Diagnostics writers accept payloads that implement
`RedactedDiagnosticsPayload`. `internal/core/config` uses this to redact API
keys in effective config diagnostics while preserving resolved pipeline context.
The redaction path clones config data before replacing secret values.
## Retention
Retention is decided by `ShouldRetainRunDirectory`.
- Failed runs are always retained.
- `always` retains successful runs.
- `never` removes successful runs.
- `auto` retains successful runs only when warnings exist.
- Unknown retention values are treated as retain by the retention decision, but
config validation rejects unsupported values before normal runs.
`ApplyRetention` removes only the specific run directory.
## CLI Failure Behavior
The CLI creates the diagnostics run directory after config loading and before
pipeline resolution. Failures before that point do not have diagnostics.
After diagnostics creation, run failures call `WriteErrorLog` and apply
retention with `RunSucceeded: false`, so the run directory remains available.
When the pipeline returns a partial manifest on failure, the CLI writes that
manifest before logging the failure.
## Invariants
- Diagnostics paths must be narrow and run-directory scoped.
- Writes should be atomic where practical.
- Secrets must be redacted.
- Diagnostics write failures are command failures because they can hide the
information needed for recovery.
- Durable output file contracts belong to output modules and integration docs,
not to diagnostics.

116
docs/internal/llm.md Normal file
View File

@@ -0,0 +1,116 @@
# LLM Runtime
The implemented LLM runtime lives in `internal/framework/llm`. It provides
transport-neutral structured completion contracts, an OpenAI-compatible HTTP
adapter, concurrency scheduling, schema registry helpers, retry behavior, and
secret redaction.
## Contract
Modules depend on `contracts.StructuredLLMClient`:
```go
CompleteStructured(ctx, request, out) (response, error)
```
The request contains messages, optional model override, response schema name,
and response schema JSON. The caller supplies a pointer target for decoded
structured output.
Extractors own prompts and schemas. Provider adapters should not contain
domain-specific prompt logic.
## Production Client Construction
`internal/cli` builds the production LLM client from the effective config:
1. find the effective LLM profile;
2. build `OpenAICompatibleClientConfig`;
3. create an OpenAI-compatible client;
4. create a scheduler from profile or global concurrency;
5. wrap the client with `NewScheduledClient`;
6. return non-secret LLM profile manifest metadata.
The current run command requires exactly one distinct effective LLM profile for
the resolved pipeline.
## OpenAI-Compatible Adapter
`OpenAICompatibleClient` posts JSON to:
```text
<base_url>/chat/completions
```
It sends:
- `model`
- `messages`
- `response_format.type = "json_schema"`
- `response_format.json_schema.name`
- `response_format.json_schema.strict = true`
- `response_format.json_schema.schema`
If an API key is configured, the adapter sends an `Authorization: Bearer ...`
header.
The adapter accepts assistant content either as a JSON string containing JSON or
as raw JSON content. It then unmarshals that content into the caller-provided
target.
External wire-contract details belong in the
[OpenAI-compatible integration doc](../integrations/openai-compatible.md).
## Retries And Timeouts
The adapter retries:
- provider request failures;
- response read failures;
- HTTP `429`;
- HTTP `5xx`;
- malformed provider envelopes;
- malformed assistant JSON;
- structured-output decode failures.
Non-retryable `4xx` responses are returned without retry. Request timeout comes
from the effective LLM profile. Context cancellation is respected.
## Scheduler
`Scheduler` bounds concurrent provider calls. It tracks in-flight calls and a
FIFO queue of waiters. Cancellation removes queued waiters or releases granted
permits.
`NewScheduledClient` wraps any structured LLM client and runs each completion
inside the scheduler.
Effective concurrency is:
1. `llm_profiles.<id>.max_concurrency`, when greater than zero;
2. `concurrency.total_llm`, when greater than zero;
3. `1`.
## Schema Registry
The framework schema registry embeds generic test schemas. It also exposes
helpers for caller-owned schemas:
- `LoadResponseSchema`
- `LookupResponseSchema`
- `MustLookupResponseSchema`
- `ResponseSchema.DiagnosticsMap`
`DiagnosticsMap` omits raw schema content and includes metadata such as key,
ID, version, name, and SHA-256.
The D&D spell extractor owns and loads its own embedded response schema.
## Secret Redaction
Provider errors are passed through `ErrorWithSecretsRedacted` with the API key
and bearer-token value. Config diagnostics use redacted effective config
payloads.
Do not add raw provider request bodies, response bodies, API keys, or prompt
payloads to diagnostics by default.

165
docs/internal/modules.md Normal file
View File

@@ -0,0 +1,165 @@
# Modules
Production modules live under `internal/modules`. Each module implements one
contract from `internal/framework/contracts`, exposes a `ModuleSpec`, and
registers itself with the matching pipeline registry.
The CLI production catalog currently registers only the modules listed here.
## Contract Pattern
A production module package should provide:
- a stable module key;
- a constructor such as `New`;
- the relevant contract implementation;
- `ModuleSpec`;
- `Register`;
- focused tests for registration, options, contract behavior, and errors.
Module specs should describe capabilities accurately. Resolution uses specs to
reject incompatible pipelines before execution.
## `seriatim` Input
Package: `internal/modules/input/seriatim`
The `seriatim` adapter parses Seriatim transcript JSON into a generic source
document. It owns transcript JSON details, source ID selection, source digest
creation, transcript segment validation, and segment metadata mapping.
Provides:
- `source.transcript`
- `transcript.speaker`
- `transcript.timestamps`
External JSON shape belongs in the Seriatim integration doc.
## `generic` Chunker
Package: `internal/modules/chunk/generic`
The `generic` chunker splits source units into ordered chunks. It validates the
source document, clones source units, assigns chunk IDs such as `chunk-000001`,
and records chunk metadata for start unit, end unit, and unit count.
Options:
- `max_units`: positive integer, default `50`;
- `overlap_units`: non-negative integer, default `0`, and less than
`max_units`.
Provides:
- `chunks`
## `dnd/spells` Extractor
Package: `internal/modules/extract/dnd/spells`
The `dnd/spells` extractor owns D&D spell-cast artifact semantics. It renders
embedded prompts, loads the embedded structured response schema, calls the
structured LLM client, converts spell-cast responses into artifact candidates,
and supplies deterministic validators.
Requires:
- `chunks`
- `source.transcript`
Provides:
- `dnd.spell_casts`
Artifact type and schema version:
- artifact type: `dnd.spell_cast`
- schema version: `v1`
The extractor adds prompt and response-schema provenance to lane manifest
metadata. Durable artifact payload details belong in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
## D&D Spell Validators
The spell extractor returns two built-in validators:
- `dnd/spells/shape`: rejects malformed payloads and missing required fields.
- `dnd/spells/source_refs`: rejects candidates without valid source references.
Reason codes include:
- `invalid_payload`
- `missing_required_field`
- `missing_source_ref`
- `invalid_source_ref`
These validators are supplied by the extractor when no validators are configured
for the lane.
## `appendorder` Merger
Package: `internal/modules/merge/appendorder`
The `appendorder` merger clones and appends candidates in chunk order. It does
not deduplicate or reconcile candidates.
Provides:
- `merged`
## `noop` Normalizer
Package: `internal/modules/normalize/noop`
The `noop` normalizer clones merged candidates and returns them unchanged.
Requires:
- `merged`
Provides:
- `normalized`
## `json` Output
Package: `internal/modules/output/json`
The `json` output encoder converts approved artifacts, rejected artifacts,
warnings, and the run manifest into logical JSON output files. It groups
approved artifacts by artifact type and sanitizes artifact-type file names.
Requires:
- `normalized`
Provides:
- `encoded`
Durable output file shapes belong in the
[JSON output contract](../integrations/json-output.md). Operator behavior
belongs in [Operations](../operations.md).
## Production Registration
Production registration is centralized in `internal/cli/catalog.go`.
Do not make framework code import production modules. The CLI wires production
modules at the application boundary; tests may provide fake registries or fake
catalogs directly.
## Adding A Module
When adding a module, keep source-format and extraction-domain boundaries clear:
- input modules may know external source formats;
- extract modules may know artifact semantics and prompt/schema assets;
- merge and normalize modules own candidate combination and reconciliation;
- output modules own serialization, not diagnostics or CLI reporting.
Update [Development](../policy/development.md), [Configuration](../config.md),
internal docs, integration docs, and examples when the new module becomes
implemented production behavior.

86
docs/internal/overview.md Normal file
View File

@@ -0,0 +1,86 @@
# Internal Overview
This directory documents implemented Notarius internals for developers and LLM
coding agents. It complements [Architecture](../policy/architecture.md), which
is the durable policy for boundaries and invariants.
## Executable And CLI
`cmd/notarius` calls the CLI package. `internal/cli` owns:
- command parsing and usage;
- config discovery and loading;
- production module catalog and registry wiring;
- production LLM client construction;
- run directory creation;
- durable output writes;
- user-facing stdout, stderr, and exit codes.
The CLI should stay thin around framework contracts. Domain extraction behavior
belongs in modules, not in command handlers.
## Core Packages
- `internal/core/artifacts`: artifact candidates, approved artifacts, rejected
artifacts, validation decisions, and run manifests.
- `internal/core/config`: defaults, YAML config parsing, environment overrides,
validation, redaction, and resolved pipeline config.
- `internal/core/diagnostics`: per-run diagnostics directory creation,
diagnostics artifact writers, atomic writes, and retention decisions.
- `internal/core/source`: source documents, source units, source references, and
validation.
Core packages should remain deterministic and concrete. They should not import
production modules.
## Framework Packages
- `internal/framework/contracts`: interfaces and request/result structs for
input adapters, chunkers, extractors, mergers, normalizers, validators, output
encoders, and structured LLM clients.
- `internal/framework/pipeline`: module registries, module specs, profile
resolution, capability checks, run orchestration, warnings, validation, and
manifest population.
- `internal/framework/llm`: OpenAI-compatible structured-output client,
scheduler, schema registry, retries, and secret redaction.
- `internal/framework/prompt`: embedded prompt registry and template rendering.
- `internal/framework/validate`: validator decision helpers and cardinality
enforcement.
Framework code should stay source-agnostic and domain-agnostic.
## Module Packages
Production module packages live under `internal/modules`:
- `input/seriatim`
- `chunk/generic`
- `extract/dnd/spells`
- `merge/appendorder`
- `normalize/noop`
- `output/json`
Each module package owns its contract implementation, module spec,
registration, options, focused tests, and module-specific errors.
## Fixtures And Tests
The repository uses focused package tests plus a fixture-driven CLI workflow.
- CLI acceptance tests cover maintained examples under `examples/`.
- Pipeline tests cover registry composition and end-to-end framework behavior
with fakes.
- Module tests cover implemented module contracts without requiring real
provider calls.
- LLM tests use local test servers and fakes.
Do not use real external services in tests. Use fakes, fixtures, or local test
servers.
## Boundary Reminders
- Source-format details stay in input modules and integration docs.
- Extraction-domain details stay in extract modules and artifact docs.
- Provider wire details stay in the LLM runtime and provider integration docs.
- Durable output contracts belong in integration docs.
- Operator procedures belong in `docs/operations.md`, not internal docs.

127
docs/internal/pipeline.md Normal file
View File

@@ -0,0 +1,127 @@
# Pipeline Internals
The implemented pipeline runner lives in `internal/framework/pipeline`. It
executes the fixed workflow defined by the architecture policy:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
Pipeline execution is serial. The runner executes the resolved lanes one after
another in the fixed workflow order.
## Profile Resolution
Config loading produces `pipeline.PipelineProfile` values. Resolution happens
before execution:
1. `internal/core/config.Config.Resolve` validates config and finds the named
pipeline.
2. The optional lane selection is passed to `pipeline.ResolvePipeline`.
3. Module bindings are defaulted:
- chunk: `generic`
- merge: `appendorder`
- normalize: `noop`
- output: `json`
- LLM profile: `default`
4. The module catalog is checked for each bound module key.
5. Module capabilities are checked in workflow order.
6. A digest is calculated from the resolved pipeline without the digest field.
The CLI writes the resolved pipeline and digest to diagnostics.
## Registries And Module Specs
`pipeline.Registries` holds concrete constructors for execution. A
`pipeline.ModuleCatalog` exposes module specs for config validation and
resolution.
Every production module registers a `ModuleSpec` with:
- `Key`: module key used in config;
- `Stage`: module kind such as input, chunk, extract, merge, normalize,
validate, or output;
- `Provides`: capabilities added after that module runs;
- `Requires`: capabilities that must already be available.
Capability checks prevent incompatible pipeline composition before a run starts.
## Runner Input And Output
`pipeline.RunInput` carries:
- a `ResolvedPipeline`;
- optional source ID, input path, and raw input bytes;
- a structured LLM client;
- run ID, start time, LLM profile manifest metadata, and CLI metadata.
`pipeline.RunOutput` carries:
- run manifest;
- approved artifacts;
- rejected artifacts;
- warnings;
- logical output files returned by the output encoder.
The CLI owns durable file writes and diagnostics writes after the runner returns.
## Execution
The runner:
1. validates run input and registries;
2. builds the input adapter and parses the raw input into a source document;
3. validates the source document;
4. builds the chunker and produces source chunks;
5. runs each selected artifact lane in sorted resolved order;
6. builds the output encoder and validates logical output file names.
Within an artifact lane, the runner:
1. builds the extractor, merger, and normalizer;
2. records module manifest metadata when modules provide it;
3. extracts candidates from each chunk;
4. normalizes candidate envelope fields such as index, extractor key, artifact
type, and schema version;
5. merges candidates;
6. normalizes merged candidates;
7. validates candidate envelope consistency;
8. runs validators;
9. converts approved candidates to artifacts.
## Validators
If a lane declares validators in config, the runner builds those validators from
the validator registry. Otherwise it uses validators returned by the extractor.
Each validator must return exactly one decision for each eligible candidate. The
runner enforces decision cardinality with `internal/framework/validate`.
Rejected candidates are removed before the next validator runs. Approved
candidates continue through the chain.
The production CLI currently registers no standalone validator modules. The
current D&D spell extractor supplies deterministic shape and source-reference
validators.
## Warnings And Failures
Warnings from chunking, extraction, merging, normalization, validation, and
output encoding are accumulated in `RunOutput.Warnings`.
Errors wrap the operation and module key or lane context. If execution fails
after a manifest exists, the returned manifest is marked `failed` and receives a
completion timestamp.
On successful execution, the manifest validation status is:
- `approved` when no candidates were rejected;
- `rejected` when at least one candidate was rejected.
## Manifest Population
The manifest records run ID, pipeline ID, pipeline digest, module keys, artifact
lanes, LLM profile metadata, source digest, validation status, and timing.
Modules can add non-secret manifest metadata by implementing
`contracts.ManifestMetadataProvider`. The D&D spell extractor uses this for
prompt and response-schema provenance.

132
docs/operations.md Normal file
View File

@@ -0,0 +1,132 @@
# Operations
This is the canonical reference for operating implemented Notarius runs.
## Normal Run
A run reads one source file, resolves one configured pipeline, calls the
configured OpenAI-compatible LLM profile, writes durable JSON output, and writes
diagnostics for inspection.
```sh
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json \
--output-dir ./notarius-output \
--diagnostics-dir /tmp/notarius
```
The command prints a success line with the pipeline ID, approved and rejected
artifact counts, and the output path.
## Output Directory
Durable output is written to:
```text
<output-root>/<run-id>/
```
The default output root is `./notarius-output`. Use `--output-dir` to choose a
different root.
The `json` output module writes these files:
- `index.json`: file index with paths to the manifest, artifact files,
rejected artifacts, and warnings.
- `manifest.json`: run manifest with resolved pipeline provenance, module keys,
validation status, and timing.
- `artifacts/<artifact-type>.json`: approved artifacts grouped by artifact
type. For the current D&D spell extractor, this includes
`artifacts/dnd.spell_cast.json` when spell-cast artifacts are approved.
- `rejected.json`: rejected candidates and validator decisions.
- `warnings.json`: warnings reported by pipeline modules or the output encoder.
Output writes are atomic per file. Logical output file names must be clean,
relative, slash-separated paths and must not contain `..`.
## Diagnostics Directory
Diagnostics are written under:
```text
<diagnostics-work-dir>/<run-id>/
```
The default diagnostics work directory is `/tmp/notarius`. It can be set with
`diagnostics.work_dir`, `NOTARIUS_WORK_DIR`, or `--diagnostics-dir`.
Implemented diagnostics artifacts:
- `invocation.json`: command metadata such as operation, config path, input
path, selected lanes, run ID, and pipeline digest when available.
- `effective-config.json`: resolved config with API keys redacted.
- `resolved-pipeline.json`: resolved module bindings and pipeline digest.
- `run-manifest.json`: the same run manifest written to durable output when it
is available.
- `warnings.json`: warning list.
- `run-report.json`: counts, status, output path, diagnostics path, and run ID.
- `error.log`: failure message, written after diagnostics directory creation
when a run fails.
`source-document.json` is supported by the diagnostics writer but is not written
by the current CLI run workflow.
## Retention
Diagnostics retention is configured with `diagnostics.retention`,
`NOTARIUS_DIAGNOSTICS_RETENTION`, or the default `auto`.
- `auto`: keep failed runs and successful runs with warnings; remove successful
warning-free runs.
- `always`: keep every diagnostics run directory.
- `never`: remove successful run directories; failed runs are still retained.
Unknown retention values are rejected during config validation.
## Failures
Failures before diagnostics directory creation, such as a missing config file or
an unusable diagnostics work directory, are printed to stderr and may not have a
diagnostics run directory.
Failures after diagnostics directory creation are printed to stderr and written
to `error.log`. Depending on where the failure occurred, diagnostics may also
include invocation metadata, redacted effective config, resolved pipeline data,
the run manifest, warnings, and a run report.
If durable output writing fails after the pipeline completes, diagnostics are
retained for inspection and may include `run-manifest.json`, `warnings.json`,
`run-report.json`, and `error.log`.
## Warnings
A successful run with warnings exits with code `0`, prints a warning count to
stderr, and writes warnings to durable output and diagnostics when retained.
The run manifest `validation_status` indicates whether final artifacts were
approved or rejected after validation.
## Cleanup
It is safe to remove specific old run directories after their output and
diagnostics are no longer needed:
```sh
rm -rf /tmp/notarius/run-1234567890
rm -rf ./notarius-output/run-1234567890
```
Use exact run-directory paths. Avoid broad cleanup commands against parent
directories unless they are part of your own operational policy.
## Operational Limits
There is no command to resume a failed run. Re-run `notarius run` after fixing
the cause.
Provider retries are limited to the OpenAI-compatible client retry behavior
configured by the effective LLM profile. There is no separate CLI retry command.
Notarius writes local files only. Remote storage and archive management are not
part of the implemented CLI.

View File

@@ -1,19 +1,22 @@
# Architecture
This document defines the development principles for Notarius. It is
inward-facing: developers and LLM coding agents should use it to preserve the
project's shape, boundaries, and invariants as the code evolves.
This document defines Notarius development policy. It is inward-facing:
developers and LLM coding agents should use it to preserve the project's shape,
boundaries, and invariants as the code evolves.
Keep this document concise. It should describe durable architectural rules, not
CLI syntax, configuration reference material, module catalogs, or roadmap items.
## Project Shape
Notarius is a small, explicit, dependency-light Go application for extracting
structured artifacts from source material using modular extractors.
structured artifacts from source material using modular pipeline stages.
The application should be contract-first but not abstraction-heavy. Add
interfaces and extension points when they protect a real boundary:
The application is contract-first but not abstraction-heavy. Add interfaces and
extension points when they protect a real boundary:
- external source formats;
- extractor modules;
- pipeline stage modules;
- validators;
- LLM providers and runtime plumbing;
- output schemas and embedded assets.
@@ -23,95 +26,43 @@ contracts that can be exercised by tests and real modules.
## Core Invariants
The core framework must remain source-agnostic and domain-agnostic.
The framework must remain source-agnostic and domain-agnostic.
Source-format details belong in input modules. Transcript-specific concepts
such as segments, speakers, timestamps, and transcript schemas must not spread
into runner, extractor, or validator framework code.
Source-format details belong in input modules. Transcript-specific concepts such
as segments, speakers, timestamps, and transcript schemas must not spread into
runner, extractor, validator, or LLM framework code.
Extraction-domain details belong in extract modules. D&D-specific concepts
such as spells, NPCs, items, combat turns, and encounters must not spread into
core source, runner, or LLM framework packages.
Extraction-domain details belong in domain modules. D&D-specific concepts such
as spells, NPCs, items, combat turns, and encounters must not spread into core
source, runner, or LLM framework packages.
Extracted facts should be grounded with source references. Source references
should point to generic source units, not to transcript-only structures.
Artifact records should require source references by default unless their schema
explicitly opts into ungrounded fields. Generic pipeline code should preserve
source-reference ranges exactly and should not merge or rewrite overlapping
ranges.
should point to generic source units, not transcript-only structures. Framework
code should preserve source-reference ranges exactly and should not merge or
rewrite overlapping ranges unless a module explicitly owns that behavior.
The application workflow is:
The application workflow is fixed:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
These stages should remain explicit in the architecture. Chunking, merging, and
normalization must not be hidden inside domain extract modules when they represent
normalization must not be hidden inside domain extractors when they represent
general pipeline behavior.
## Dependency Policy
Pipelines are fixed-shape templates for this workflow, not arbitrary DAGs or a
general workflow language. Module selection should be configuration- and
registry-driven, not scattered through conditionals.
Prefer the Go standard library where practical.
## Package Boundaries
Use external dependencies only when justified by correctness, security,
interoperability, or substantial complexity reduction. Good reasons include
widely used file formats, complex validation behavior, or secure transport
handling.
Prefer fewer, larger framework packages until a boundary proves itself through
import direction, ownership, test seams, or substantial file size.
Avoid dependencies for small conveniences. Do not let external dependency types
leak across internal package boundaries unless the dependency is itself the
explicit public contract of that package.
## Package Layout
Use this layout unless a change documents a better project-specific reason.
CLI and executable entrypoint:
- `cmd/notarius`: executable entrypoint.
- `internal/cli`: CLI command definitions, flags, argument parsing, and command wiring.
Core deterministic model and policy:
- `internal/core/source`: source document, source unit, and source reference types.
- `internal/core/artifacts`: artifact envelope, artifact candidates, rejected artifacts, and manifests.
- `internal/core/config`: configuration structs, defaults, loading, precedence, and validation, once config exists.
Reusable framework plumbing:
- `internal/framework/contracts`: core interfaces and transport-neutral request/response contracts.
- `internal/framework/pipeline`: runner, pipeline-stage orchestration, registries, and small shared stage helpers.
- `internal/framework/validate`: shared validator runtime behavior and decision checks.
- `internal/framework/llm`: LLM clients, scheduling, structured-output parsing, and response-schema registry, once LLM runtime exists.
- `internal/framework/prompt`: embedded prompt assets, prompt registry, and prompt rendering helpers, once prompt assets exist.
Domain implementations:
- `internal/modules/input/<name>`: input-stage modules that parse external input into core source documents.
- `internal/modules/chunk/<name>`: chunk-stage modules.
- `internal/modules/extract/<domain>/<name>`: extract-stage extractor modules.
- `internal/modules/merge/<name>` or `internal/modules/merge/<domain>/<name>`: merge-stage modules.
- `internal/modules/normalize/<name>` or `internal/modules/normalize/<domain>/<name>`: normalize-stage modules.
- `internal/modules/output/<name>`: output-stage modules.
- `internal/validators/<validator>`: built-in validator implementations.
- `internal/transport/http`: shared HTTP client code, if needed by provider integrations.
Package-private implementation constants may live near the package that owns
them, preferably in `constants.go` when useful.
Start with fewer, larger framework packages. Split a package only when a real
boundary proves itself through import direction, ownership, test seams, or
substantial file size. Do not create catalog, diagnostics, reporting,
structured-output, response-schema, output, merge, normalize, extraction, or
warnings packages merely because the concepts exist in the architecture.
## Stage Modules
Concrete business logic should live under `internal/modules/<stage>/...`.
Stage-oriented module layout is preferred because it makes the application
workflow visible in the filesystem:
Core packages should contain deterministic models and policy. Framework
packages should contain reusable orchestration and provider plumbing. Concrete
business logic should live under stage-oriented module packages:
```text
internal/modules/input/...
@@ -122,120 +73,43 @@ internal/modules/normalize/...
internal/modules/output/...
```
Use short, lowercase, idiomatic Go package names. Prefer names such as
`dndtranscript`, `appendorder`, and `spells` over names like `dnd_transcript`,
`serial_merge`, or `spell_extractor` that repeat parent-stage context.
## Input Modules
Use a hexagonal architecture style for source input.
Use short, lowercase, idiomatic Go package names. Avoid package names that repeat
parent-stage context.
Input modules translate external source formats into the core source model.
They may know about external schema details, source-specific metadata, and
format-specific validation rules. They should not own extraction-domain
decisions.
Other packages should interact with source input through adapter contracts and
core source types. Input module implementation details and external dependency
types must not leak into framework or extract module packages.
Extract modules own artifact semantics, prompt usage, structured response schema
selection, validator defaults, and domain-specific interpretation. They should
depend on framework contracts and core source/artifact types, not concrete input
module packages.
Input module metadata may preserve source-specific facts such as transcript
speaker, timestamps, Markdown heading path, page number, or block ID. Framework
code may carry metadata through, but should not require a specific adapter's
metadata shape.
Merge modules combine extracted candidates. Normalize modules reconcile merged
candidates for semantic consistency. Generic behavior may exist for simple
artifact types, but domain-specific behavior belongs in modules for the relevant
stage.
Core source metadata should remain `map[string]any`. Document well-known keys
as conventions, and let input modules expose typed accessor helpers for their
own metadata when useful.
Output modules serialize final artifacts and may report warnings out of band.
CLI, diagnostics, and reporting layers are responsible for surfacing those
warnings.
## Extractors
Extractors are independent modules that process source chunks or whole source
documents and produce one kind of structured artifact candidate.
Each extract module owns:
- its artifact semantics;
- its prompt usage;
- its structured response schema selection;
- its validator chain;
- any domain-specific mapping or interpretation.
Extract modules should depend on framework contracts and core source/artifact
types. They should not depend on concrete input module packages.
Extraction requests should carry the active source chunk plus optional ambient
context, such as a document synopsis, prior-chunk summaries, known entities, or
other module-provided state. The context may be empty for simple modules, but
the contract should not assume extraction is always chunk-local.
Extractors should not be the only place where chunking, merging, or
normalization happens. They may choose processing mode or provide domain-specific
merge/normalization behavior when generic behavior is insufficient, but the
pipeline stages themselves are framework concepts.
The runner should be able to compose, skip, resume, or run individual extractors
when their prerequisites are satisfied. Ordering should be explicit through
configuration, a default sequence, or documented orchestration rules.
Extractor selection must go through a registry or equivalent mechanism rather
than scattered conditionals.
## Pipeline Stages
The pipeline has six conceptual stages:
1. input: external source material becomes a `SourceDocument`;
2. chunk: a `SourceDocument` becomes ordered source chunks;
3. extract: extractors produce artifact candidates from chunks or whole documents;
4. merge: per-chunk candidates become a merged candidate collection;
5. normalize: merged candidates are reconciled for duplicates, aliases, consistency, or cross-chunk issues;
6. output: final artifacts are serialized.
Chunking is first-class because source documents may be too large for a single
LLM pass. Chunkers should preserve source-unit order and produce stable chunk
metadata.
Merge and normalize are separate concerns. Merge combines per-chunk results into
a deterministic collection. Normalize performs semantic reconciliation after
merge. Generic append-in-chunk-order merge and no-op normalization should be
available for simple artifact types, while domain-specific behavior can be
provided where needed.
The framework should allow serial and parallel chunk processing. The first
implementation may execute chunks serially for determinism, but contracts should
not prevent later parallel execution.
Final durable output should use one artifact file per artifact type plus a
run-level manifest/index file. Framework artifact flow should use a generic
envelope with `json.RawMessage` payloads; extract modules should own typed Go
structs at their boundaries and encode into that generic envelope before
returning to framework code.
Output-stage warnings are intentionally out-of-band. An `OutputEncoder` may
return warnings about serialization, formatting, truncation, or destination
concerns, but those warnings are appended to the runner result after encoding
and are not expected to appear inside the encoded artifact bytes. CLI,
diagnostic, or reporting layers should surface output-stage warnings from the
runner result.
Schemas should be versioned per extractor, with a separate envelope/manifest
format version.
## Validators
## Validation
Validators should be independently testable and composable.
Deterministic validators should run before LLM-backed validators when both are
present. Validator decision semantics should be explicit: each candidate
artifact should receive exactly one decision from each validator that evaluates
it.
artifact evaluated by a validator should receive exactly one decision from that
validator.
LLM-backed review belongs in module-owned validator chains, not in a separate
global review phase. Extract modules and normalize modules may both use
deterministic and LLM-backed validators.
LLM-backed review belongs in module-owned validator chains, not in an implicit
global review phase. Extract and normalize modules may both use deterministic
and LLM-backed validators.
Shared validator runtime mechanics belong under `internal/framework/validate`.
Concrete validator behavior belongs under `internal/validators/<validator>`.
Shared validator runtime mechanics belong in framework code. Concrete validator
behavior belongs in module or validator implementation packages.
## LLM Runtime
@@ -243,131 +117,64 @@ LLM provider details belong behind transport-neutral framework contracts.
Provider-specific HTTP request and response types should stay inside the LLM
runtime package. Prompt construction should stay in extractors, validators, or
shared prompt-context helpers; provider adapters should not own domain prompt
logic.
shared prompt helpers; provider adapters should not own domain prompt logic.
Errors, diagnostics, reports, and redacted config must not expose secrets.
Errors, diagnostics, reports, manifests, and redacted configuration must not
expose secrets.
## Configuration
Centralize configuration loading, processing, precedence, defaults, and
validation in `internal/core/config`.
Configuration should make pipeline composition explicit and discoverable.
The goal is to make configuration discoverable and avoid implicit or hidden
operational values. User-visible defaults and cross-package operational defaults
should be defined in config code.
Centralize configuration loading, precedence, defaults, and validation. Structural
pipeline choices should come from named pipeline definitions, not ad hoc command
flags. Operational overrides may be handled separately when they do not obscure
the configured pipeline structure.
Configuration should be organized around named pipeline profiles. A pipeline is
a fixed-shape template for the application workflow, not a free-form DAG or
general workflow program. The six-stage flow remains fixed:
Module registries should expose module metadata and capabilities without
requiring module construction. Configuration validation should fail fast when a
pipeline binds incompatible or unknown modules.
```text
input -> chunk -> extract -> merge -> normalize -> output
```
Run manifests should record enough resolved pipeline provenance to make a run
auditable after named configuration changes over time.
A pipeline profile should bind registered modules to those stage slots:
## Dependencies
- one shared input module;
- one shared chunk module by default;
- one or more artifact lanes, each with extract, merge, normalize, and
validator behavior;
- one output module.
Prefer the Go standard library where practical.
The MVP should use one shared chunk module per pipeline. Per-lane chunk
overrides are a future extension and should be added only if a real artifact
lane needs different chunking.
Use external dependencies only when justified by correctness, security,
interoperability, or substantial complexity reduction. Good reasons include
widely used file formats, complex validation behavior, or secure transport
handling.
The CLI should select a named pipeline by ID, such as
`notarius run dnd-session --input session.json`. Structural module selection
should come from configuration, not ad hoc CLI flags. CLI flags may select a
subset of configured artifact lanes, such as `--only spells,npcs`, and may
override operational settings such as model, concurrency, output directory, or
diagnostics directory.
Pipeline definitions should support compact defaults:
- `chunk`: `generic`;
- lane `merge`: `appendorder`;
- lane `normalize`: `noop`;
- `output`: `json`;
- `llm_profile`: `default` where an LLM profile is needed.
Module bindings should support both string shorthand and object form. For
example, `extract: dnd/spells` and
`extract: {module: dnd/spells, llm_profile: fast}` should normalize to the same
internal binding type.
Module registries should expose module metadata, including flat string
capabilities, without requiring module construction. Config validation should
fail fast on unknown module keys, unknown pipeline IDs, missing required slots,
missing capabilities, unknown LLM profiles, empty artifact-lane sets, or
`--only` lane names that do not exist in the selected pipeline.
Keep capabilities as a flat string set. Do not evolve capabilities into a type
system unless real module interactions prove the need.
Unless documented otherwise, precedence from lowest to highest is:
1. built-in defaults
2. configuration file
3. environment variables
4. CLI flags
Prefer YAML configuration unless the project has a strong reason to use another
format. Config files should be discoverable at
`/usr/local/etc/notarius/config.yml`, with a CLI override via `--config`.
Configuration files should not contain raw secrets unless the application is
explicitly designed for that. Prefer environment variables or secret files for
secrets.
Stage-module-specific configuration should remain inline with the pipeline slot
that owns it. Do not add named module instances until repeated inline settings
create real drift or duplication. LLM profiles are the justified top-level
exception because model settings are cross-cutting.
The run manifest should record the selected `pipeline_id` and a digest of the
resolved pipeline definition after defaults and lane selection are applied.
`pipeline_id` alone is not sufficient provenance because a named pipeline can
change over time.
## Embedded Assets
Store embedded JSON schemas, Markdown prompts, templates, and similar assets as
separate files, not inline string literals, unless there is a strong reason
otherwise.
Embedded prompts and response schemas should have stable IDs, versions, source
metadata, and hashes suitable for diagnostics and run manifests.
## Errors and Logging
Errors should be actionable and preserve context. Wrap errors with operation and
path/resource context. CLI code should convert internal errors into concise
user-facing messages.
Errors and logs must not expose secrets.
Use structured logging where practical. Logs should describe operations, paths,
external calls, retries, and failure causes, but should not include large source
or artifact payloads by default.
## Context, Timeouts, and Cancellation
Long-running operations should accept `context.Context`. External calls,
subprocesses, HTTP requests, storage operations, LLM calls, and multi-stage
workflows should respect cancellation and timeouts.
Avoid dependencies for small conveniences. Do not let external dependency types
leak across internal package boundaries unless the dependency is itself the
explicit contract of that package.
## State, Files, and Safety
If the application writes durable state, writes should be atomic where
practical. Multi-step workflows should preserve enough state to support
inspection, retry, or resume after failure.
practical. Multi-step workflows should preserve enough diagnostics to support
inspection after failure.
Code that deletes, moves, or overwrites files must use narrow, explicit paths.
Avoid broad parent-directory operations. Cleanup that can cause data loss must
be opt-in.
## Errors and Logging
Errors should be actionable and preserve context. Wrap errors with operation and
path or resource context. CLI code should convert internal errors into concise
user-facing messages.
Errors and logs must not expose secrets. Logs should describe operations,
external calls, retries, and failure causes, but should not include large source
or artifact payloads by default.
Long-running operations should accept `context.Context`. External calls,
subprocesses, HTTP requests, storage operations, LLM calls, and multi-stage
workflows should respect cancellation and timeouts.
## Testing
Core logic should be testable without real external services. Use fakes,
@@ -375,18 +182,15 @@ fixtures, or local test doubles for input modules, extract modules, validators,
and LLM clients where practical.
Contract-first work should include fake implementations that prove interfaces
compose before real input modules or extract modules depend on them.
compose before real modules depend on them.
Once the pipeline-stage contracts exist, maintain a fixture-driven walking
skeleton that exercises input, chunk, extract, merge, normalize, and output
stages with fake modules and fake external clients. This test should protect
stage composition continuously while real modules are introduced over later
checkpoints.
Maintain a fixture-driven walking skeleton that exercises the full pipeline with
fake modules and fake external clients. This protects stage composition as real
modules evolve.
Config examples should be load-tested once config files exist. Important CLI
workflows should have parser or command tests. Adapter, extractor, and validator
contracts should have focused tests that do not require running the full
application unless end-to-end coverage is intentional.
Important CLI and configuration workflows should have tests. Adapter, extractor,
validator, and stage contracts should have focused tests that do not require
running the full application unless end-to-end coverage is intentional.
## Documentation

139
docs/policy/development.md Normal file
View File

@@ -0,0 +1,139 @@
# Development
This document defines contributor workflow for Notarius. For architectural
invariants and package boundaries, read [Architecture](architecture.md) first.
## Required Reading
Before changing the repository, review:
- [Architecture](architecture.md)
- [Documentation Policy](documentation.md)
Keep current-behavior documentation limited to implemented behavior. Put planned
or deferred behavior under `docs/roadmap/`.
## Repository Layout
- `cmd/notarius`: executable entry point.
- `internal/cli`: CLI parsing, production catalog wiring, config loading, run
command orchestration, output writes, and user-facing errors.
- `internal/core`: deterministic models and policy for artifacts, source
documents, config, and diagnostics.
- `internal/framework`: reusable contracts, pipeline orchestration, prompt
helpers, validation helpers, and LLM runtime plumbing.
- `internal/modules`: concrete input, chunk, extract, merge, normalize, and
output modules.
- `docs`: policy, user/operator docs, internal docs, integration docs, and
roadmap files.
- `examples`: maintained, secret-free examples covered by tests where practical.
## Validation Commands
Run focused tests for the area changed, then run the broader checks when the
change affects shared contracts, CLI behavior, or documentation examples.
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
```
Useful focused checks:
```sh
go test ./internal/cli
go test ./internal/core/config
go test ./internal/framework/pipeline
go test ./internal/framework/llm
go test ./internal/modules/input/seriatim
go test ./internal/modules/extract/dnd/spells
go test ./internal/modules/output/json
```
## Go Conventions
- Prefer the standard library unless a dependency is justified by correctness,
security, interoperability, or substantial complexity reduction.
- Keep package names short, lowercase, and idiomatic.
- Preserve import direction: framework and core code must not depend on concrete
production modules.
- Use `context.Context` for long-running operations and external calls.
- Return contextual errors that name the operation and relevant module, path, or
resource.
- Do not include secrets in errors, logs, diagnostics, manifests, or docs.
## Adding Config Fields
Config behavior is centralized under `internal/core/config`.
When adding a file config field:
1. Update file config structs and YAML parsing in `file_config.go`.
2. Apply the field over defaults in config application code.
3. Add validation in `validation.go` when the field has constraints.
4. Add environment override support in `env.go` only for operational overrides.
5. Update redaction if the field can contain secrets.
6. Add focused config tests.
7. Update [Configuration](../config.md) and maintained examples when behavior
changes.
Pipeline composition should remain config-driven. Do not add command flags that
silently replace structural pipeline definitions.
## Adding CLI Flags Or Commands
CLI behavior lives in `internal/cli`.
When adding CLI surface:
1. Keep syntax explicit and update usage text.
2. Validate arguments before running expensive work.
3. Convert internal errors into concise user-facing messages.
4. Add CLI tests for success, syntax errors, and failure modes.
5. Update [CLI Reference](../cli.md), and update
[Operations](../operations.md) or [Troubleshooting](../troubleshooting.md)
if run behavior changes.
## Adding Modules Or Adapters
Concrete modules live under `internal/modules/<kind>/...` and implement the
interfaces in `internal/framework/contracts`.
For a new production module:
1. Implement the relevant contract.
2. Expose a `ModuleSpec` with the correct module key, module kind, provided
capabilities, and required capabilities.
3. Expose a `Register` function that registers the module with its registry.
4. Add focused module tests for contract behavior, registration, options,
validation, and errors.
5. Register the module in `internal/cli/catalog.go` only when it is production
ready.
6. Update internal docs and user-facing docs only for implemented behavior.
Source-format behavior belongs in input modules and integration docs.
Extraction-domain behavior belongs in extract modules and artifact docs.
## Updating Examples
Examples must be valid, secret-free, and small.
- Prefer environment-based secret configuration.
- Keep `examples/dnd-spells.config.yml` loadable by CLI tests.
- Keep `examples/seriatim-minimal-transcript.json` compatible with the Seriatim
adapter.
- Do not add expected-output fixtures unless they are validated or have a clear
regeneration procedure.
## Documentation Updates
Update docs in the same change when behavior changes.
- CLI syntax: `docs/cli.md`
- Config fields and defaults: `docs/config.md`
- Output, diagnostics, retention, or recovery: `docs/operations.md`
- Common user-facing failures: `docs/troubleshooting.md`
- Internal architecture and contracts: `docs/internal/`
- External file formats and durable integration contracts: `docs/integrations/`
- Future or planned work only: `docs/roadmap/`

View File

@@ -188,7 +188,9 @@ It should include:
- architectural invariants;
- explicit non-goals, if useful.
For small projects, this file may be brief. It may simply state that the project is intentionally narrow, monolithic, and dependency-light.
Notably, this file should prescribe a core development *policy* that should remain unchanged as the application evolves. It is not a place for details (e.g., CLI flags) that could change over time.
The contents of `architecture.md` should be trim and concise. LLMs may be directed to review it routinely via AGENTS.md, CLAUDE.md, or similar.
### docs/api.md

View File

@@ -1,70 +0,0 @@
# Checkpoint 1: Core Contracts And Skeleton
## Status
This document describes planned work, not implemented behavior.
## Goal
Define the stable vocabulary and core interfaces that adapters, extractors,
validators, and runners will build against.
This checkpoint should produce a compileable Go repository with a minimal CLI
shell and contract-level tests. It does not need to process real input or
produce useful artifacts.
## Scope
In scope:
- Go module bootstrap;
- executable entrypoint;
- minimal CLI package;
- core source, artifact, manifest, and contract types;
- fake implementation tests proving the interfaces are usable.
Out of scope:
- real input modules;
- real extract modules;
- LLM provider calls;
- prompt or response-schema assets;
- diagnostics run directory;
- production config loading.
## Target End State
The repository should contain a compileable Go application shell and stable core
contract packages:
- `cmd/notarius` provides the executable entrypoint.
- `internal/cli` provides a minimal CLI shell.
- `internal/core/source` defines generic source documents, source units, source
references, and source validation helpers.
- `internal/core/artifacts` defines extractor-neutral artifact candidate,
approved artifact, rejected artifact, and run manifest types.
- `internal/framework/contracts` defines the adapter, extractor, validator, and
structured LLM interfaces used by later checkpoints.
The contracts should be proven with fake implementations in tests. Those tests
should demonstrate composition without real input modules, real extract modules,
LLM provider calls, prompt assets, or diagnostics infrastructure.
Implementation staging belongs in
[`implementation.md`](implementation.md).
## Done Criteria
- `go test ./...` passes.
- `go build ./cmd/notarius` passes.
- Core types and contracts exist in stable package locations.
- Tests prove fake implementations can compose at the type-contract level.
- No real Seriatim, D&D, LLM, or Audita-specific behavior has been added yet.
## Review Questions
- Are the interfaces small enough?
- Are source-format details absent from core packages?
- Are D&D concepts absent from framework and core packages?
- Is the shell compileable without placeholder behavior that will be hard to
unwind?

View File

@@ -1,75 +0,0 @@
# Checkpoint 2: Framework Composition
## Status
This document describes planned work, not implemented behavior.
## Goal
Prove the core contracts compose before adding real stage modules or portable
Audita infrastructure.
This checkpoint should produce a minimal runner that can execute fake registered
components from source input to artifact output in tests.
## Scope
In scope:
- input adapter registry;
- extractor registry;
- validator decision model;
- decision-cardinality checks;
- minimal runner;
- fake-component runner tests.
Out of scope:
- real input parsing;
- real input modules;
- real extract modules;
- pipeline-profile config loading;
- module capability validation;
- real LLM calls;
- prompt assets;
- response schema assets;
- diagnostics run directory;
- real D&D artifact schemas.
## Target End State
The repository should contain a minimal framework composition layer:
- `internal/framework/pipeline` registers and builds input adapter constructors
and extractor constructors by stable key.
- `internal/framework/validate` provides shared validator decision helpers and
cardinality checks.
- `internal/framework/pipeline` executes configured extractors against a
`SourceDocument`, applies validator chains, and returns approved and rejected
artifacts.
The runner should operate on already parsed source documents in this checkpoint.
Raw input parsing and concrete input module behavior remain deferred to the
Seriatim input module checkpoint.
Pipeline-profile resolution, module metadata, and capability validation are
deferred to checkpoint 3. This checkpoint only needs constructor registries and
minimal runner composition.
Implementation staging belongs in
[`implementation.md`](implementation.md).
## Done Criteria
- `go test ./...` passes.
- Fake adapter/extractor/validator registrations work in tests.
- The runner operates on `SourceDocument`, not transcript-specific structures.
- The runner does not import concrete D&D extract module packages.
## Review Questions
- Does the runner know only about sources, extractors, validators, and artifacts?
- Are registries simple enough to evolve?
- Are validation decisions expressive enough for deterministic and LLM-backed
validators?
- Is any domain-specific behavior creeping into framework packages?

View File

@@ -1,181 +0,0 @@
# Checkpoint 3: Pipeline Stages, Chunking, Merge, And Normalize
## Status
This document describes planned work, not implemented behavior.
## Goal
Make Notarius's application workflow first-class before adding real input
modules or extract modules.
The workflow should be:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
Checkpoint 3 should define the contracts and minimal fake-tested framework
behavior for chunking, per-chunk extraction, merging, and normalization. It
should also introduce a fixture-driven walking skeleton that exercises the full
stage sequence with fake modules and a fake LLM client. It should not add real
input modules, real domain extract modules, LLM provider code, or production
output modules.
## Scope
In scope:
- source chunk model;
- chunker contract;
- extract-stage contract for extractors operating on chunks;
- merge-stage contract;
- normalize-stage contract;
- output-stage contract and fake output encoder for pipeline completeness;
- resolved pipeline definition types for a fixed-shape pipeline template;
- module binding and module metadata types, including flat capability strings;
- default application for `chunk`, lane `merge`, lane `normalize`, `output`,
and `llm_profile`;
- lane selection behavior equivalent to future `--only`;
- runner/pipeline updates that exercise these stages with fake components;
- generic append/chronological merge behavior for artifact candidates when
appropriate;
- fixture-driven walking skeleton test for
`input -> chunk -> extract -> merge -> normalize -> output`;
- fake `StructuredLLMClient` wired through a trivial extractor.
Out of scope:
- Seriatim parsing;
- D&D spell extraction;
- LLM provider calls;
- prompt assets;
- response schema assets;
- diagnostics run directory;
- production config file loading;
- real CLI command behavior;
- production output serialization or durable output writing.
## Target End State
The repository should contain explicit pipeline-stage contracts:
- `InputAdapter`: external source input to `SourceDocument`.
- `Chunker`: `SourceDocument` to ordered `SourceChunk` values.
- `Extractor`: `SourceChunk` to artifact candidates.
- `Merger`: per-chunk candidates to merged candidates.
- `Normalizer`: merged candidates to normalized candidates.
- `OutputEncoder`: final artifact bundle to bytes.
The repository should also contain a resolved pipeline model that represents:
- `pipeline_id`;
- shared input binding;
- shared chunk binding;
- selected artifact lanes;
- lane extract, merge, normalize, and validator bindings;
- output binding;
- resolved defaults;
- resolved pipeline digest input.
The runner should orchestrate fake implementations through chunk, extract,
merge, normalize, and approval/validation behavior in tests.
The checkpoint should include a fixture-driven walking skeleton that starts from
fixture input bytes and ends at encoded output bytes. The walking skeleton should
exercise the stage contracts, resolved pipeline model, module metadata,
capability validation, defaults, lane selection, and fake LLM client wiring. It
is contract coverage, not useful user-facing behavior.
## Design Intent
Chunking is a core application concern because many source documents, especially
transcripts, will be too large for a single LLM extraction pass.
Chunk processing may be serial or parallel depending on extractor needs. The
architecture should support both, while checkpoint 3 may execute
deterministically in series until a later checkpoint introduces concurrency.
Merge and normalize are separate stages:
- merge combines per-chunk extracted candidates into one stream or collection;
- normalize reconciles the merged output by checking duplicates, consistency,
ordering, identity resolution, or other cross-chunk concerns.
For some artifact types, merge may be generic append-in-source-order behavior.
For other artifact types, merge may be domain-specific. Normalization is where
deduplication and consistency checks should live.
## Processing Modes
The architecture should leave room for extractor-level processing modes:
- whole-document processing;
- serial chunk processing;
- parallel chunk processing.
Checkpoint 3 may execute chunks serially for deterministic behavior.
The contracts should not bake in a single-pass assumption or prevent later
parallel execution.
## Generic Merge Behavior
A generic merger should be able to concatenate candidates in deterministic
chunk order and candidate order. This is likely sufficient for early spell-cast
extraction, where chronological serialization is useful.
Domain-specific mergers may be added later when generic ordering is not enough.
## Generic Normalize Behavior
A no-op normalizer should be available as the default.
Domain-specific normalizers may later:
- deduplicate repeated extracted facts;
- resolve aliases;
- reconcile conflicting candidate fields;
- enforce chronological or source-reference consistency;
- attach normalization warnings.
## Walking Skeleton
The fixture-driven skeleton should prove the staged architecture continuously as
new contracts are added. It should remain deliberately small and use fake modules
only. It should validate module keys and flat capability requirements before
execution, using registry metadata rather than constructing modules. Capability
values should remain simple strings.
Implementation staging for the walking skeleton belongs in
[`implementation.md`](implementation.md).
## Done Criteria
- `go test ./...` passes.
- Pipeline-stage contracts are explicit and source/domain agnostic.
- A resolved pipeline profile model exists for fixed-shape pipeline templates.
- Pipeline defaults and lane selection are covered by fake tests.
- Module capability validation is covered by fake tests.
- Fake tests prove input source documents can be chunked, extracted, merged, and
normalized.
- A fixture-driven walking skeleton proves fake input, chunk, extract, merge,
normalize, and output modules compose end to end with a fake LLM client.
- Merge and normalize are distinct concepts in code and tests.
- The runner no longer implies whole-document-only extraction as the core
application model.
- No concrete input module, domain extract module, LLM provider, prompt,
response schema, diagnostics, config, or D&D artifact code is added.
## Review Questions
- Is the workflow clearly represented as input, chunk, extract, merge,
normalize, and output?
- Are merge and normalize cleanly separated?
- Does the resolved pipeline model avoid becoming a general-purpose workflow
engine?
- Are module capabilities simple flat strings?
- Does lane selection avoid creating ad hoc pipelines?
- Can a generic merger handle simple chronological artifact streams?
- Can a later domain-specific normalizer handle duplicates and consistency
without changing core runner contracts?
- Does the design allow serial and parallel chunk processing later?

View File

@@ -1,114 +0,0 @@
# Checkpoint 4: Portable Audita Infrastructure
## Status
This document describes planned work, not implemented behavior.
## Goal
Port or adapt reusable Audita infrastructure that directly supports Notarius
contracts while avoiding Audita's transcript-correction model.
This checkpoint should add reusable runtime plumbing, not real extraction
behavior.
## Scope
In scope:
- structured LLM client interface implementation;
- LLM scheduler;
- prompt registry pattern;
- response-schema registry pattern;
- diagnostics run directory pattern;
- config loading and validation for named pipeline profiles and implemented
runtime pieces.
Out of scope:
- correction proposals;
- replacement policies;
- transcript mutation;
- correction ledger terminology;
- Audita module or validator behavior;
- real D&D prompts or schemas;
- domain-specific prompt or schema assets;
- embedded built-in pipeline profiles.
## Target End State
The repository should contain reusable runtime infrastructure adapted from
Audita where it directly supports Notarius contracts:
- an OpenAI-compatible structured-output LLM client behind the existing
`StructuredLLMClient` interface;
- an LLM scheduler for bounded concurrency;
- an embedded response-schema registry pattern;
- an embedded prompt registry pattern;
- a diagnostics run directory pattern using extraction-oriented artifact names;
- config structs, loading, defaults, redaction, and validation for named
pipeline profiles.
Framework code should remain source-agnostic and domain-agnostic. Provider HTTP
details should stay inside the LLM runtime package. Prompt and schema registries
should use placeholder/test assets until real extractor prompts and schemas are
introduced by later checkpoints.
Config should support:
- `llm_profiles`;
- `pipelines.<pipeline_id>.input`;
- `pipelines.<pipeline_id>.chunk`;
- `pipelines.<pipeline_id>.artifacts.<lane>.extract`;
- lane `merge`, `normalize`, and validator settings;
- output module selection;
- inline module-binding object form and string shorthand;
- default `chunk`, `merge`, `normalize`, `output`, and `llm_profile`;
- selected pipeline ID and lane filtering for runtime use;
- concurrency;
- work directory;
- diagnostics retention.
Config loading should support the standard precedence model:
1. built-in defaults
2. configuration file
3. environment variables
4. CLI flags
Structural module selection should come from pipeline config. CLI flags may
override operational settings and artifact lane selection, but should not offer
ad hoc `--extractor` or `--chunker` wiring.
Config validation should fail fast for unknown pipeline IDs, unknown module
keys, missing required slots, missing capabilities, unknown LLM profiles, empty
artifact-lane sets, and invalid lane selections.
If the CLI shell is ready, the checkpoint should expose discovery/validation
commands for config and pipeline profiles:
```sh
notarius config validate
notarius pipelines list
```
Implementation staging belongs in [`implementation.md`](implementation.md).
## Done Criteria
- `go test ./...` passes.
- Audita runtime pieces are adapted to Notarius package names and contracts.
- No correction proposal, replacement policy, transcript mutation, or correction
ledger code has been copied.
- Runtime tests cover secret redaction, schema registry lookup, prompt metadata,
and scheduler behavior where applicable.
- Config tests cover named pipeline profiles, defaults, lane selection,
capability validation, and resolved pipeline digesting.
## Review Questions
- Did we copy only reusable infrastructure?
- Do provider-specific types stay behind adapter/runtime boundaries?
- Are diagnostics names and report concepts extraction-oriented?
- Is config limited to named pipeline profiles and implemented behavior?
- Are structural pipeline changes kept out of ad hoc CLI flags?

View File

@@ -1,107 +0,0 @@
# Checkpoint 5: Seriatim Input Module
## Status
This document describes planned work, not implemented behavior.
## Goal
Add the first real input source while keeping transcript-specific behavior
isolated inside an input-stage module.
This checkpoint should allow Seriatim minimal transcript JSON to become a
generic `SourceDocument`.
## Scope
In scope:
- `internal/modules/input/seriatim`;
- parser for Seriatim minimal output JSON;
- mapping into `SourceDocument` and `SourceUnit`;
- source-document validation;
- input adapter registry wiring;
- module metadata/capabilities for pipeline validation;
- fixtures and tests;
- config compatibility through named pipeline profiles.
Out of scope:
- D&D extraction;
- LLM extraction calls;
- transcript-specific behavior in runner/core packages;
- support for every possible Seriatim schema variant.
## Target End State
The repository should contain a real Seriatim input-stage module that translates
Seriatim minimal transcript JSON into the generic source model.
The Seriatim module should be registered under the stable input adapter key
`seriatim`. It should be selectable through the existing input adapter registry
and through pipeline-profile resolution when a profile binds `input: seriatim`.
The module should accept the Seriatim minimal output shape:
- top-level `metadata`;
- top-level `segments`;
- segment `id`;
- segment `start`;
- segment `end`;
- segment `speaker`;
- segment `text`.
The module should map Seriatim data into generic source values:
- segment `id` becomes `SourceUnit.ID`;
- segment `text` becomes `SourceUnit.Text`;
- the document and unit kind strings identify transcript-like source material
without adding transcript-specific fields or types to core packages;
- `speaker`, `start`, and `end` become source-unit metadata;
- top-level Seriatim metadata becomes source-document metadata;
- the resulting source document passes core source validation.
The module should reject invalid Seriatim input with clear module-specific
errors. Validation should cover:
- valid JSON;
- required top-level metadata and segments;
- required segment fields;
- unique segment IDs;
- non-empty segment text;
- valid start and end values.
The module should declare flat capabilities for pipeline validation. Initial
capabilities should describe transcript-oriented source properties preserved by
the adapter, including speaker and timestamp metadata.
Implementation staging belongs in
[`implementation.md`](implementation.md).
## Fixtures And Tests
The checkpoint should add synthetic fixtures and focused tests for:
- valid Seriatim minimal transcript;
- malformed JSON;
- missing metadata;
- missing or duplicate segment IDs;
- empty segment text;
- source-reference compatibility with generated unit IDs.
## Done Criteria
- `go test ./...` passes.
- Seriatim minimal transcript JSON maps into `SourceDocument`.
- Transcript fields do not appear in core runner contracts.
- The input module is selectable through the registry and pipeline-profile
configuration when config support exists.
- The input module declares capabilities needed for pipeline validation.
- Tests prove transcript-specific assumptions are isolated to the input module.
## Review Questions
- Are segment, speaker, and timestamp assumptions contained inside the input module?
- Are unit IDs stable and suitable for source references?
- Does the input module preserve enough metadata for transcript-oriented output later?
- Should the input module accept only Seriatim minimal output for now?

View File

@@ -1,136 +0,0 @@
# Checkpoint 6: D&D Spells Extractor
## Status
This document describes planned work, not implemented behavior.
## Goal
Implement the first useful extract-stage module: D&D spell casts from a
Seriatim transcript source document.
This checkpoint should produce the first meaningful vertical slice from real
source input to validated artifact output.
## Scope
In scope:
- D&D spell artifact schema and Go structs;
- structured response schema asset;
- prompt assets;
- `internal/modules/extract/dnd/spells`;
- module metadata/capability requirements for pipeline validation;
- source-reference and schema validators in the extractor chain;
- fake LLM tests;
- CLI-level integration test if the CLI path is ready.
Out of scope:
- D&D item extraction;
- NPC extraction;
- combat extraction;
- cross-slice deduplication beyond simple deterministic merging;
- broad D&D rules validation.
## Proposed Stages
### Stage 1: Spell Artifact Schema
Define the D&D spell artifact model.
Initial shape:
```go
type SpellCast struct {
Player string `json:"player"`
Spell string `json:"spell"`
Effect string `json:"effect"`
NarrativeDescription string `json:"narrative_description"`
SourceRefs []SourceRef `json:"source_refs"`
}
```
Keep this schema inside the D&D spells extract module or a D&D artifact package,
not inside core framework packages.
### Stage 2: Structured Response Schema
Add a structured response schema asset for spell extraction.
The schema should require:
- spell-cast array;
- non-empty player, spell, effect, and narrative description fields;
- at least one source reference per spell cast.
### Stage 3: Prompt Assets
Add embedded prompt assets for D&D spell extraction.
Prompts should:
- describe the generic source-unit input format;
- explain that source references must use source-unit IDs;
- avoid relying on transcript-specific fields except as optional metadata;
- request only spell-cast artifacts.
### Stage 4: Process Module Implementation
Implement `internal/modules/extract/dnd/spells`.
The extractor should:
- satisfy the framework `Extractor` contract;
- declare module metadata for pipeline-profile validation;
- build LLM messages from a source document or source chunk;
- call the structured LLM client;
- return artifact candidates with source references;
- attach its validator chain.
### Stage 5: Validators And Tests
Wire deterministic validators:
- schema/shape validation;
- source-reference validation;
- required-field validation if not covered by schema handling.
Add tests using a fake structured LLM client:
- successful spell extraction;
- empty result;
- invalid source reference rejection;
- malformed structured output handling;
- stable output ordering.
### Stage 6: CLI Integration
If the CLI path is ready, add an end-to-end test using:
```sh
notarius run dnd-session --input ./transcript.json --only spells
```
The test should use fake LLM wiring, fixture input, and a named pipeline profile
with a `spells` artifact lane.
## Done Criteria
- `go test ./...` passes.
- Seriatim input can flow through the runner into the D&D spells extractor.
- Spell artifacts include valid source references.
- D&D concepts are contained in extract module/artifact packages and docs.
- The spells module can be selected as a named artifact lane in pipeline
configuration.
- The first meaningful vertical slice is available through tests, and through
CLI if the CLI path is ready.
## Review Questions
- Is the spell extract module domain-specific without making the framework
D&D-specific?
- Are source references valid and useful for downstream validation?
- Is prompt/schema ownership clear?
- Does this vertical slice reveal contract changes needed before adding items,
NPCs, or combat?

View File

@@ -1,199 +0,0 @@
# Documentation Roadmap
## Status
This document captures planned documentation decisions for Notarius. It records
policy choices while the application architecture is still being shaped. It does
not describe implemented behavior.
## Documentation Goals
Notarius documentation should make three boundaries obvious:
- source-format support belongs to input-stage modules;
- extraction-domain behavior belongs to extract-stage modules;
- core framework behavior is source-agnostic and domain-agnostic.
Documentation should avoid making the MVP look more transcript-specific or
D&D-specific than the architecture intends.
## Current Policy Decisions
### Planned Work Stays In Roadmap Docs
Until code exists, planned behavior belongs under `docs/roadmap/`.
Implemented behavior should later move into canonical docs. Roadmap files may
then link to those docs or be reduced to remaining future work.
### Core Docs Should Use Generic Terms
Core architecture docs should prefer:
- source document;
- source unit;
- source reference;
- input adapter;
- extractor;
- chunker;
- merger;
- normalizer;
- output encoder;
- artifact;
- validator;
- run manifest.
Core docs should avoid transcript-specific terms such as segment, speaker,
timestamp, and transcript range unless discussing an input adapter or an example.
Core docs should avoid D&D-specific terms such as spell, NPC, item, combat, and
encounter unless discussing extract modules, artifact docs, or examples.
### Input Module Docs Own Source Formats
Each implemented input-stage module should have a canonical integration
document.
Likely future files:
```text
docs/integrations/seriatim-transcript.md
docs/integrations/markdown-source.md
```
Input module docs should cover:
- accepted external schema or file shape;
- mapping into `SourceDocument` and `SourceUnit`;
- metadata preserved by the module;
- validation rules and failure behavior;
- examples.
The Seriatim input module doc should reference the Seriatim schema it supports and
explain how transcript segment IDs become source-unit IDs.
### Stage Module Docs Own Business Logic
Each implemented stage-module family should have canonical internal or
integration docs.
Likely future files:
```text
docs/internal/stage-modules.md
docs/integrations/artifacts-dnd.md
```
Stage module docs should cover:
- module key;
- stage;
- artifact type;
- schema version;
- required source-reference behavior;
- validator chain;
- prompt and response-schema ownership;
- examples.
D&D concepts should be documented in D&D extract-module or artifact docs, not in
generic runner or framework docs.
### CLI Docs Should Reflect Extensibility
The CLI reference should present named pipeline profiles as the primary
user-facing abstraction. Individual stage modules should be visible through
pipeline configuration and discovery commands, not through ad hoc structural
run flags.
Provisional command shape:
```sh
notarius run dnd-session --input ./source.json
notarius run dnd-session --input ./source.json --only spells,npcs
notarius config validate
notarius pipelines list
```
Once implemented, `docs/cli.md` should document:
- pipeline ID selection;
- required input path flags;
- `--only` artifact-lane selection;
- config path behavior;
- operational overrides such as output path, model, concurrency, and diagnostics
directory;
- output path behavior;
- diagnostics and report behavior;
- exit codes.
### Config Docs Should Separate Framework And Plugin-Like Options
`docs/config.md` should describe named pipeline profiles and the resolved
pipeline model.
It should cover:
- config file locations and precedence;
- `llm_profiles`;
- `pipelines.<pipeline_id>.input`;
- `pipelines.<pipeline_id>.chunk`;
- `pipelines.<pipeline_id>.artifacts.<lane>.extract`;
- lane `merge`, `normalize`, and validator settings;
- output module selection;
- string shorthand versus inline module-binding object form;
- defaults for omitted slots;
- capability validation;
- pipeline digest and manifest provenance.
Module-specific config should stay inline with the pipeline slot that owns it.
Top-level named module instances should not be introduced until repeated inline
settings create real drift. `llm_profiles` are the cross-cutting exception.
### Examples Should Stay Real
Examples should be added only when the matching behavior exists and should be
covered by tests where practical.
Likely future examples:
```text
examples/seriatim-minimal-transcript.json
examples/minimal-config.yml
examples/dnd-spells.artifacts.json
examples/dnd-session.config.yml
```
Examples should be secret-free and should use the same command shapes documented
in `docs/cli.md`.
## Canonical Documentation Targets
When the first vertical slice is implemented, add or update:
- `README.md`: concise purpose, shortest useful command, links.
- `docs/cli.md`: implemented command behavior.
- `docs/config.md`: implemented config behavior.
- `docs/operations.md`: diagnostics, retention, failure inspection.
- `docs/troubleshooting.md`: common failures.
- `docs/internal/overview.md`: implemented package map.
- `docs/internal/pipeline.md`: implemented extraction flow.
- `docs/internal/stage-modules.md`: stage contracts and implemented modules.
- `docs/internal/input-modules.md`: input adapter contract and implemented input modules.
- `docs/internal/validators.md`: validator contract and built-ins.
- `docs/integrations/seriatim-transcript.md`: Seriatim input contract.
- `docs/integrations/artifacts.md`: output artifact envelope.
## Review Checklist For Future Documentation Changes
Before merging docs, check:
- Does the document describe implemented behavior outside `docs/roadmap/`?
- Are source-format details isolated to input module or integration docs?
- Are D&D details isolated to extract module or artifact docs?
- Is there one canonical home for the topic?
- Do command examples match implemented CLI syntax?
- Do config examples use named pipeline profiles rather than ad hoc module
flags?
- Are examples valid, maintained, and free of secrets?
- Did any architecture, config, CLI, stage module, validator, or artifact
contract change require a docs update?

View File

@@ -1,443 +0,0 @@
# Implementation Plan: Checkpoint 5 Seriatim Input Module
## Status
This is a staged implementation plan for
[`5-seriatim-input-module.md`](5-seriatim-input-module.md). It is intended for
an LLM coding agent to follow stage by stage.
This plan implements only checkpoint 5. Do not add D&D extraction, real domain
prompts or schemas, LLM extraction calls, a `notarius run` command, broad
Seriatim schema support, or transcript-specific behavior in core framework
packages in this checkpoint.
## Policy Context
Follow:
- [`docs/policy/architecture.md`](../policy/architecture.md)
- [`docs/policy/documentation.md`](../policy/documentation.md)
Required boundaries:
- keep Seriatim JSON schema details inside `internal/modules/input/seriatim`;
- keep core source, runner, pipeline, extractor, validator, LLM, and config
packages source-agnostic and domain-agnostic;
- do not add transcript-specific typed fields to `SourceDocument`,
`SourceUnit`, runner contracts, or pipeline contracts;
- preserve transcript-specific values only as source metadata conventions;
- register the input module through the existing input adapter registry instead
of adding ad hoc conditionals;
- use flat capability strings in module metadata;
- keep future or planned behavior in `docs/roadmap/` until implemented.
## Global Implementation Decisions
- Add no new third-party dependency. Use `encoding/json` with
`Decoder.UseNumber` for Seriatim JSON parsing.
- Use `seriatim` as the stable input adapter key.
- Put all concrete Seriatim input code under
`internal/modules/input/seriatim`.
- Expose a small module API:
```go
const Key = "seriatim"
func New() *Adapter
func ModuleSpec() pipeline.ModuleSpec
func Register(registry *pipeline.InputAdapterRegistry) error
```
- `ModuleSpec()` must return stage `pipeline.StageInput`, no required
capabilities, and these provided capabilities:
`source.transcript`, `transcript.speaker`, and `transcript.timestamps`.
- The Seriatim adapter should satisfy `contracts.InputAdapter`.
- Use `source.SourceDocument.Kind = "transcript"`.
- Use `source.SourceDocument.Format =
"application/vnd.seriatim.minimal+json"`.
- Use `source.SourceUnit.Kind = "transcript_segment"`.
- Compute `SourceDocument.Digest` from the exact raw input bytes as
`sha256:<hex>`.
- Resolve `SourceDocument.ID` in this order:
1. trimmed `contracts.ParseRequest.SourceID`, if non-empty;
2. trimmed string `metadata.id`, if present and non-empty;
3. trimmed string `metadata.source_id`, if present and non-empty;
4. deterministic fallback `seriatim:<first-16-hex-chars-of-raw-sha256>`.
- Segment IDs become source unit IDs exactly after validation. Reject segment
IDs with leading or trailing whitespace rather than silently rewriting them.
- Copy top-level Seriatim `metadata` into `SourceDocument.Metadata`.
- Store segment `speaker`, `start`, and `end` in `SourceUnit.Metadata` under
keys with those exact names.
- Store `start` and `end` as `json.Number` values so JSON serialization remains
numeric and the original decimal representation is preserved.
- Require top-level `metadata` to be present and be an object, but do not
require any specific metadata keys in checkpoint 5.
- Require top-level `segments` to be present and contain at least one segment.
- Reject unknown or extra JSON fields only if they prevent parsing the minimal
shape. Otherwise ignore them so the module can tolerate compatible Seriatim
additions.
- Return module-specific errors prefixed with useful Seriatim context, for
example `seriatim input: segment "s1" text must not be empty`.
- Keep examples free of private transcript content. Use synthetic fixture text.
## Stage 1: Seriatim Package Skeleton And External Model
### Goal
Create the Seriatim input module package, define the module-local JSON model,
and add registry-facing module metadata without changing framework contracts.
### Files To Add
- `internal/modules/input/seriatim/adapter.go`
- `internal/modules/input/seriatim/model.go`
- `internal/modules/input/seriatim/metadata.go`
- `internal/modules/input/seriatim/registry_test.go`
### Required API
Add:
```go
package seriatim
const Key = "seriatim"
const (
DocumentKind = "transcript"
UnitKind = "transcript_segment"
Format = "application/vnd.seriatim.minimal+json"
)
const (
MetadataSpeaker = "speaker"
MetadataStart = "start"
MetadataEnd = "end"
)
type Adapter struct{}
func New() *Adapter
func (a *Adapter) Key() string
func (a *Adapter) Parse(ctx context.Context, req contracts.ParseRequest) (*source.SourceDocument, error)
func ModuleSpec() pipeline.ModuleSpec
func Register(registry *pipeline.InputAdapterRegistry) error
```
Add typed metadata helpers:
```go
func Speaker(unit source.SourceUnit) (string, bool)
func Start(unit source.SourceUnit) (json.Number, bool)
func End(unit source.SourceUnit) (json.Number, bool)
```
### Seriatim JSON Shape
Define module-local structs for the minimal external shape:
```go
type transcript struct {
Metadata map[string]any `json:"metadata"`
Segments []segment `json:"segments"`
}
type segment struct {
ID string `json:"id"`
Start json.Number `json:"start"`
End json.Number `json:"end"`
Speaker string `json:"speaker"`
Text string `json:"text"`
}
```
Use an internal decode helper based on `json.NewDecoder(bytes.NewReader(raw))`
and `UseNumber`.
### Required Behavior
- `New()` returns a non-nil adapter.
- `Adapter.Key()` returns `Key`.
- `ModuleSpec()` returns defensive slices and the capability set listed in the
global decisions.
- `Register()` calls `InputAdapterRegistry.RegisterWithSpec(ModuleSpec(), ...)`.
- `Register(nil)` returns an error from the registry path rather than panicking.
- Keep external JSON structs unexported.
### Required Tests
- `New()` returns an adapter whose key is `seriatim`.
- `ModuleSpec()` uses input stage and declares the required provided
capabilities.
- `Register()` makes the adapter buildable from an `InputAdapterRegistry`.
- Registry lookup returns the Seriatim module spec.
### Validation
Run:
```sh
gofmt -w internal/modules/input/seriatim
go test ./internal/modules/input/seriatim
go test ./...
```
## Stage 2: Parse, Validate, And Map To SourceDocument
### Goal
Implement the Seriatim parser and mapper from minimal Seriatim JSON into the
generic source model.
### Files To Add Or Update
- `internal/modules/input/seriatim/adapter.go`
- `internal/modules/input/seriatim/model.go`
- `internal/modules/input/seriatim/adapter_test.go`
- `internal/modules/input/seriatim/testdata/valid_minimal.json`
- `internal/modules/input/seriatim/testdata/duplicate_segment_id.json`
### Required Validation
Reject:
- nil or canceled context before parsing;
- empty raw input;
- malformed JSON;
- valid JSON with trailing non-whitespace data;
- missing, null, or non-object top-level `metadata`;
- missing, null, empty, or non-array top-level `segments`;
- segment IDs that are empty after trimming;
- segment IDs with leading or trailing whitespace;
- duplicate segment IDs;
- missing or empty `speaker`;
- missing, empty, non-numeric, negative, or non-finite `start`;
- missing, empty, non-numeric, negative, or non-finite `end`;
- segments where `end < start`;
- missing or empty `text`.
The parser may preserve leading and trailing whitespace in segment text as long
as the text is not empty after trimming.
### Mapping Rules
- `segment.id` becomes `SourceUnit.ID`.
- `segment.text` becomes `SourceUnit.Text`.
- `speaker`, `start`, and `end` become unit metadata under the exact keys
defined in stage 1.
- The document metadata is a shallow copy of top-level Seriatim metadata.
- The document digest is based on raw input bytes, not normalized JSON.
- Call `source.ValidateDocument` before returning the document and wrap any
validation failure with Seriatim context.
### Required Tests
Add tests for:
- valid minimal transcript parses to a source document with expected ID, kind,
format, digest, units, and metadata;
- `ParseRequest.SourceID` overrides metadata-derived IDs;
- fallback document ID is deterministic and has prefix `seriatim:`;
- malformed JSON returns an actionable Seriatim parse error;
- missing metadata is rejected;
- missing or empty segments is rejected;
- duplicate segment IDs are rejected;
- empty segment text is rejected;
- missing speaker is rejected;
- invalid timestamp values are rejected;
- `end < start` is rejected;
- typed metadata helpers return the expected speaker and timestamp values;
- a `source.SourceRef` using the first and last generated unit IDs validates
with `source.ValidateRef`.
### Validation
Run:
```sh
gofmt -w internal/modules/input/seriatim
go test ./internal/modules/input/seriatim
go test ./...
```
## Stage 3: Pipeline Resolution And Config Compatibility
### Goal
Prove the Seriatim input module participates in pipeline-profile resolution and
capability validation through existing registries and config loading.
### Files To Add Or Update
- `internal/modules/input/seriatim/config_test.go`
- `internal/modules/input/seriatim/testdata/pipeline.yml`
### Required Test Catalog
Build a test-only module catalog with:
- Seriatim input registered through `seriatim.Register`;
- a fake chunker requiring `source.transcript` and providing `chunks`;
- a fake extractor requiring `chunks`, `transcript.speaker`, and
`transcript.timestamps`, and providing `fake.artifacts`;
- `pipeline.AppendOrderMerger` registered as `appendorder`, requiring
`fake.artifacts`;
- `pipeline.NoopNormalizer` registered as `noop`;
- a fake `json` output encoder registered as output stage.
Do not add real extract, chunk, normalize, or output modules for this checkpoint.
### YAML Fixture
Use a synthetic pipeline fixture shaped like:
```yaml
version: 1
pipelines:
seriatim-fixture:
input: seriatim
chunk: fake/chunk
artifacts:
events:
extract: fake/extract
merge: appendorder
normalize: noop
output: json
```
The default LLM profile supplied by `config.Default()` is sufficient. Do not
add real provider settings to this fixture.
### Required Tests
- `config.ParseFileConfigYAML` and `Config.ApplyFileConfig` load the fixture.
- `Config.Resolve` succeeds with pipeline ID `seriatim-fixture` and the
test-only catalog.
- The resolved pipeline input module is `seriatim`.
- The resolved pipeline digest is non-empty and stable across repeated
resolution.
- Removing `transcript.timestamps` from the Seriatim module spec in the
test-only catalog causes resolution to fail with a missing capability error.
- Selecting an unknown `--only` lane still fails through existing resolution
behavior.
### Validation
Run:
```sh
gofmt -w internal/modules/input/seriatim
go test ./internal/modules/input/seriatim
go test ./internal/core/config
go test ./...
```
## Stage 4: Runner Integration With Fake Downstream Stages
### Goal
Prove real Seriatim input can flow through the existing runner into fake
downstream stages while preserving source-unit IDs and metadata.
### Files To Add Or Update
- `internal/modules/input/seriatim/runner_test.go`
### Required Behavior
Use the same Seriatim fixture from stage 2 and a resolved pipeline from stage 3.
Register fake downstream stages only inside the test.
The fake extractor should:
- inspect the received `SourceDocument` and `SourceChunk`;
- assert that unit IDs match Seriatim segment IDs;
- assert that speaker and timestamp metadata are present;
- return one generic artifact candidate with a source reference pointing at
existing Seriatim-derived unit IDs.
The test should then assert:
- `Runner.Run` succeeds;
- the manifest records input module `seriatim`;
- the manifest source digest equals the parsed document digest;
- approved artifacts preserve valid source references;
- no transcript-specific type has been added outside the module.
### Required Tests
- successful runner execution from Seriatim JSON through fake chunk, extract,
merge, normalize, and output stages;
- runner failure when the Seriatim adapter returns an invalid source document,
using a malformed fixture or test input;
- validation of the fake candidate's source reference with
`source.ValidateRef`.
### Validation
Run:
```sh
gofmt -w internal/modules/input/seriatim
go test ./internal/modules/input/seriatim
go test ./internal/framework/pipeline
go test ./...
```
## Stage 5: Documentation And Final Verification
### Goal
Document the implemented Seriatim integration contract once the module exists,
without describing unimplemented D&D extraction or run-command behavior.
### Files To Add Or Update
- `docs/integrations/seriatim.md`
- `docs/roadmap/5-seriatim-input-module.md`
### Required Documentation
Create `docs/integrations/seriatim.md` as implemented-behavior documentation
with:
- accepted minimal JSON shape;
- required fields and validation rules;
- mapping from Seriatim fields to `SourceDocument` and `SourceUnit`;
- metadata key conventions for `speaker`, `start`, and `end`;
- capability strings declared by the module;
- note that broader Seriatim schema variants are not yet supported.
Update `docs/roadmap/5-seriatim-input-module.md` only if implementation
reveals a real scope or policy correction. Keep future D&D extraction behavior
out of the integration doc.
### Final Validation
Run:
```sh
gofmt -w internal/modules/input/seriatim
go test ./...
go build ./cmd/notarius
rm -f ./notarius
```
### Done Criteria
- `go test ./...` passes.
- `go build ./cmd/notarius` passes.
- Seriatim minimal transcript JSON maps into `SourceDocument`.
- Unit IDs are stable and validate in source references.
- Transcript fields do not appear in core runner contracts.
- The input module is selectable through the input registry and pipeline-profile
resolution.
- The input module declares transcript-oriented flat capabilities for pipeline
validation.
- Tests prove transcript-specific assumptions are isolated to
`internal/modules/input/seriatim`.
## Open Questions
None. This plan chooses the checkpoint-5 behavior needed to implement the
feature without requiring additional product decisions.

View File

@@ -1,642 +0,0 @@
# Initial Architecture Roadmap
## Status
This document captures proposed architecture and implementation sequencing for
Notarius. It describes planned work, not implemented behavior.
## Goal
Notarius should extract structured JSON artifacts from primary source inputs
using modular, LLM-backed extractors.
The first MVP should target audio transcripts generated by Seriatim. That
choice should be implemented as an input-stage module, not as a
transcript-specific assumption in the application core. Later input sources,
such as unstructured Markdown notes or Obsidian documents, should be addable
through new input and extract modules without reshaping the framework.
The first extraction domain should be D&D session analysis, starting with spell
casts. That domain should live in extract-stage modules and related schemas, not
in core framework packages.
The application should follow the same broad architecture as Audita:
- deterministic core packages for source documents, artifacts, and configuration once needed;
- input-stage modules that translate external source formats into a small internal source model;
- reusable framework packages for contracts, orchestration, LLM runtime, structured output, and validation;
- independent extract-stage modules that own domain-specific behavior;
- independent validator packages;
- embedded prompt and JSON schema assets;
- CLI orchestration that wires the pieces together without owning domain logic.
The main domain difference from Audita is that Notarius emits extracted
artifacts rather than proposing and applying transcript corrections.
## Architectural Principles
- Keep the core input model generic: ordered text units plus metadata.
- Keep source-format details in hexagonal input modules.
- Keep extraction-domain details in extract modules.
- Treat evidence as source references, not transcript references.
- Prefer narrow, useful abstractions over a universal document model.
- Preserve enough provenance for validation, replay, and downstream inspection.
## Proposed Package Shape
```text
cmd/notarius
internal/cli
internal/core/source
internal/core/artifacts
internal/framework/contracts
internal/framework/pipeline
internal/framework/validate
internal/framework/llm
internal/framework/prompt
internal/modules/input/seriatim
internal/modules/input/markdown
internal/modules/chunk/generic
internal/modules/chunk/dndtranscript
internal/modules/extract/dnd/spells
internal/modules/extract/dnd/items
internal/modules/extract/dnd/npcs
internal/modules/extract/dnd/combat
internal/modules/merge/appendorder
internal/modules/merge/dnd/spells
internal/modules/normalize/noop
internal/modules/normalize/dnd/spells
internal/modules/output/json
internal/validators/source_refs
internal/validators/schema_validity
internal/validators/domain_consistency
internal/validators/llm_review
examples
docs/internal
```
The `markdown` input module and D&D-specific chunk, merge, normalize, and
output modules are listed as likely future packages. The MVP should implement
only the stage modules needed by the checkpoint sequence.
`internal/core/config` should be added when production configuration exists.
The framework package list is intentionally consolidated. `pipeline` should own
runner orchestration, stage registries, and small merge/normalize/output helpers
until those boundaries prove they need separate packages. `llm` should own
structured output and response-schema mechanics until those concerns become too
large or import-heavy. `prompt` should own prompt assets and rendering helpers
once prompt assets exist.
## Core Concepts
### SourceDocument
Canonical internal representation of source material. This should be the object
extractors receive, regardless of whether the original input was a transcript,
Markdown file, note export, or another source type.
```go
type SourceDocument struct {
ID string `json:"id"`
Kind string `json:"kind"`
Format string `json:"format"`
Digest string `json:"digest"`
Units []SourceUnit `json:"units"`
Metadata map[string]any `json:"metadata,omitempty"`
}
type SourceUnit struct {
ID string `json:"id"`
Kind string `json:"kind"`
Text string `json:"text"`
Metadata map[string]any `json:"metadata,omitempty"`
}
```
Initial source-unit assumptions:
- units are ordered;
- unit IDs are stable within a source document;
- each unit has extractable text;
- adapter-specific metadata may carry speaker, timestamps, heading paths, page
numbers, or other source details.
Core source metadata should remain `map[string]any`. Notarius should not define
a universal document model. Instead, the project should document well-known
metadata keys, such as `speaker`, `start`, `end`, and `heading_path`, as
conventions. Input modules may export typed accessor helpers for their own
metadata, such as `seriatim.SpeakerOf(unit)`, without leaking those helpers into
core framework contracts.
### Input Module / Adapter Contract
Hexagonal boundary for external source formats.
```go
type InputAdapter interface {
Key() string
Parse(ctx context.Context, req ParseRequest) (*source.SourceDocument, error)
}
```
The MVP input module should target Seriatim minimal transcript JSON. Seriatim segment
fields should map as follows:
- `id` becomes `SourceUnit.ID`;
- `text` becomes `SourceUnit.Text`;
- `speaker`, `start`, and `end` become unit metadata;
- Seriatim output metadata becomes document metadata.
The core runner should not know that these units came from transcript segments.
### SourceRef
Grounding reference from an extracted fact back to source units.
```go
type SourceRef struct {
SourceID string `json:"source_id"`
StartUnitID string `json:"start_unit_id"`
EndUnitID string `json:"end_unit_id"`
}
```
Initial source-reference validation should require:
- source ID exists for the current run;
- start and end unit IDs exist;
- start is less than or equal to end in document order;
- the referenced range is contiguous within the source document;
- every extracted fact has at least one source reference unless its schema
explicitly allows ungrounded metadata.
Transcript-oriented output can still present these as transcript segment ranges
when the adapter metadata makes that interpretation available.
Source references should preserve the exact ranges produced by extractors and
validators. Overlapping ranges should not be merged or rewritten by generic
pipeline code. If a domain module wants a derived compact range later, that
should be additional output, not a replacement for the original evidence.
### Extractor
Reusable module contract for producing one artifact type.
```go
type Extractor interface {
Key() string
ArtifactType() string
SchemaVersion() string
Validators() []Validator
Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error)
}
```
An extractor should receive either a whole source document or a source chunk,
depending on processing mode. It should return typed artifact candidates plus
warnings. It should not mutate the source document.
`ExtractionRequest` should be designed now to carry both the active chunk and
optional ambient context, even if the MVP leaves that context empty. Useful
ambient context may include a document synopsis, prior-chunk summaries, known
entities, or other module-provided state. D&D spell extraction can likely work
per chunk, but combat, NPC, and identity-oriented extraction will need broader
context. Adding the field later would force churn across every extractor.
Extract modules own domain concepts. For example, D&D spell extraction should
live under `internal/modules/extract/dnd/spells`; a future to-do extractor for
notes should live under a different extract-module path and use the same
framework contract.
### Chunker
Reusable stage contract for splitting a source document into ordered source
chunks.
Chunking is a first-class pipeline concern because source documents may exceed a
single LLM extraction pass. Chunkers should preserve source-unit order and
produce stable chunk metadata suitable for diagnostics and replay.
### Merger
Reusable stage contract for combining per-chunk artifact candidates into one
merged candidate collection.
Merge should combine outputs without doing semantic reconciliation. A generic
append-in-chunk-order merger should be sufficient for many artifact streams,
including the likely first D&D spell-cast extractor.
### Normalizer
Reusable stage contract for reconciling merged artifact candidates.
Normalize is distinct from merge. Normalizers may deduplicate repeated facts,
resolve aliases, reconcile conflicting fields, check cross-chunk consistency,
or attach normalization warnings.
### Validator
Reusable validation contract for artifact candidates.
Validators should cover:
- JSON/schema validity;
- source-reference validity;
- required-field and shape checks;
- domain consistency;
- optional LLM review for high-risk or ambiguous artifacts.
Validator output should follow Audita's decision-cardinality model: each
candidate artifact receives exactly one decision per validator.
LLM-backed review should be modeled as part of a module's validator chain, not
as a separate global review phase. Extract modules should be able to attach one
or more deterministic or LLM-backed validators. Normalize-stage modules may also
run validator chains, including LLM-backed validators, when semantic
reconciliation needs review.
### Artifact
Final approved JSON output from one or more extractors.
Artifacts should preserve enough metadata to support downstream validation,
debugging, and replay.
The pipeline should carry artifact candidates through a generic envelope with a
`json.RawMessage` payload. Extract modules should own typed Go structs at their
module boundary, then encode those typed records into the generic artifact
candidate envelope before returning to framework code. This keeps stage
contracts simple and avoids generic type plumbing across unrelated artifact
families.
Final durable output should be one file per artifact type plus a run-level
manifest/index file. This supports partial success and lets downstream consumers
read only the artifact types they need. Each artifact file should include its
artifact type, extractor key, extractor schema version, envelope format version,
records, source references, and enough provenance to connect it to the run
manifest.
Every artifact record should require source references unless that artifact
schema explicitly opts into ungrounded fields. Artifact-level metadata, counts,
run information, and other derived summary fields are exempt from the per-record
grounding rule.
Schemas should be versioned per extractor, with a separate envelope/manifest
format version. A single global schema version would couple unrelated extractor
release cadence.
### RunManifest
Per-run provenance record.
```go
type RunManifest struct {
EnvelopeVersion string `json:"envelope_version"`
PipelineID string `json:"pipeline_id"`
PipelineDigest string `json:"pipeline_digest"`
InputModule string `json:"input_module"`
Chunker string `json:"chunker"`
SourceDigests []string `json:"source_digests"`
Extractors []string `json:"extractors"`
Merger string `json:"merger"`
Normalizer string `json:"normalizer"`
OutputEncoder string `json:"output_encoder"`
SchemaVersion string `json:"schema_version"`
ValidationStatus string `json:"validation_status"`
}
```
The manifest should eventually include model names, prompt IDs, prompt hashes,
response schema versions, config source, redacted resolved config digest,
started/completed timestamps, and diagnostics paths.
## Initial Extractor Targets
### D&D Spells
Recommended first vertical slice because it is narrow but representative.
```go
type SpellCast struct {
Player string `json:"player"`
Spell string `json:"spell"`
Effect string `json:"effect"`
NarrativeDescription string `json:"narrative_description"`
SourceRefs []SourceRef `json:"source_refs"`
}
```
The spell extractor should be D&D-specific. The framework should not know what a
spell is.
### D&D Items
Tracks items gained, lost, transferred, consumed, or transformed.
Open questions:
- Should currency be represented as items or as its own artifact type?
- Should item ownership be a required field?
- How should ambiguous ownership changes be represented?
### D&D NPCs
Tracks NPCs interacted with, newly introduced, renamed, described, or otherwise
made relevant to campaign state.
Open questions:
- Should NPC identity resolution happen inside this extractor or in a later
deduplication stage?
- Should location/faction/relationship facts be separate artifact types?
### D&D Combat
Likely warrants a dedicated schema rather than a generic event list.
Proposed first shape:
```go
type CombatTurn struct {
Actor string `json:"actor"`
Action string `json:"action"`
Outcome string `json:"outcome"`
NarrativeDescription string `json:"narrative_description"`
SourceRefs []SourceRef `json:"source_refs"`
}
```
Open questions:
- Should combat be extracted as turns, rounds, encounters, or all three?
- Should mechanical fields such as damage, conditions, saves, attacks, and spell
slots be normalized immediately or added later?
- How should uncertain initiative order be represented?
### Future Non-D&D Extractors
The architecture should support extractors outside the D&D domain. Examples:
- to-do items from Markdown or Obsidian notes;
- decisions and action items from meeting transcripts;
- named people, places, and dates from research notes.
These should be addable as extract modules without changing runner,
validator, source-reference, or LLM framework contracts.
## Configuration Model
Notarius should use named pipeline profiles selected by ID at the CLI. A
pipeline is a fixed-shape template for the known application workflow, not a
free-form list of steps:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
A pipeline profile should define one shared front end and one or more artifact
lanes:
- shared input module;
- shared chunk module by default;
- artifact lanes containing extract, merge, normalize, and validator behavior;
- shared output module.
The MVP should use one shared chunk module per pipeline. Per-lane chunk
overrides can be added later if an artifact lane, such as combat, proves it
needs a different chunking strategy.
Example shape:
```yaml
llm_profiles:
default:
model: example-model
max_concurrency: 4
pipelines:
dnd-session:
input: seriatim
chunk: dnd/transcript
artifacts:
spells:
extract: dnd/spells
normalize: dnd/spells
npcs:
extract: dnd/npcs
items:
extract: dnd/items
```
The CLI should run named pipelines:
```sh
notarius run dnd-session --input session-014.json
notarius run dnd-session --input session-014.json --only spells,npcs
```
`--only` should select configured artifact lanes. It should not create an
ad hoc pipeline. Structural module selection should come from config, while CLI
flags may override operational knobs such as model, concurrency, output
directory, and diagnostics directory.
Initial defaults:
- `chunk`: `generic`;
- lane `merge`: `appendorder`;
- lane `normalize`: `noop`;
- `output`: `json`;
- `llm_profile`: `default` where an LLM profile is needed.
Module bindings should support both string shorthand and object form:
```yaml
extract: dnd/spells
```
```yaml
extract:
module: dnd/spells
llm_profile: fast
prompt_version: v1
```
Both forms should normalize into a single internal `ModuleBinding` shape before
validation and manifest hashing.
Pipeline validation should use module metadata declared through registries.
Modules should expose flat string capability metadata, such as `speaker` or
`timestamps`, without requiring module construction. Config validation should
fail fast for:
- unknown pipeline IDs;
- unknown module keys;
- missing required slots;
- missing required capabilities;
- unknown LLM profiles;
- empty artifact-lane sets;
- `--only` lane names that do not exist in the selected pipeline.
The MVP should keep pipelines config-file-only. Built-in pipeline profiles can
be added later if the project needs embedded defaults, but that introduces
merge/override semantics that the MVP does not need.
The resolved pipeline definition should be hashed after defaults and lane
selection are applied. The run manifest should record both `pipeline_id` and
`pipeline_digest`; a pipeline ID alone is not stable provenance.
## Proposed Pipeline Flow
The application workflow should be first-class:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
Proposed runner flow:
1. Load effective config.
2. Resolve the selected pipeline profile by ID.
3. Apply defaults and `--only` lane selection.
4. Validate module keys, lane definitions, LLM profiles, and capabilities.
5. Hash the resolved pipeline definition.
6. Create diagnostics run directory.
7. Resolve the configured input module through the input adapter registry.
8. Read source input.
9. Parse source input into a `SourceDocument`.
10. Validate source-document invariants.
11. Resolve the configured chunker.
12. Chunk source units into deterministic source chunks.
13. Resolve configured artifact lanes through registries.
14. Extract, merge, normalize, and validate each selected artifact lane.
15. Retain approved artifacts and rejected-artifact diagnostics.
16. Serialize output files and run-level manifest/index.
17. Write diagnostics and optional report JSON.
The runner should operate on source documents and source chunks only. Any
transcript-specific behavior should happen before the runner, inside the input
adapter, or after the runner, inside output rendering that understands source
metadata.
## Audita Patterns To Reuse
Reuse these architectural patterns:
- deterministic parsing and schema validation style;
- deterministic chunking of ordered source units;
- explicit extractor registry;
- explicit pipeline stage contracts;
- `contracts` package for transport-neutral interfaces;
- OpenAI-compatible structured LLM client;
- scheduler for bounded LLM concurrency;
- embedded prompt registry with prompt metadata and hashes;
- embedded response-schema registry with schema metadata and hashes;
- diagnostics run directory with redacted effective config;
- validator decision cardinality and deterministic validator ordering;
- CLI tests and fixture-driven integration tests.
The fixture-driven integration-test pattern should begin at checkpoint 3 with a
walking skeleton over fake modules and a fake LLM client. Later checkpoints
should replace fake pieces with real Seriatim, runtime, and D&D modules without
losing that end-to-end contract coverage.
Avoid copying these Audita concepts directly:
- transcript-specific core types;
- correction proposals;
- replacement policies;
- deterministic transcript mutation;
- correction ledger terminology.
Those concepts are specific to Audita's transcript-editing role and should be
replaced with source-document, artifact-candidate, artifact-validation, and
extraction-report concepts.
## Checkpoint Roadmap
The initial implementation should proceed through six coherent checkpoints.
Each checkpoint should leave the repository in a reviewable state, with the code
compiling and targeted tests covering the newly introduced contracts or behavior.
1. [Core Contracts And Skeleton](1-core-contracts-and-skeleton.md)
2. [Framework Composition](2-framework-composition.md)
3. [Pipeline Stages, Chunking, Merge, And Normalize](3-pipeline-stages-chunking-merge-normalize.md)
4. [Portable Audita Infrastructure](4-portable-audita-infrastructure.md)
5. [Seriatim Input Module](5-seriatim-input-module.md)
6. [D&D Spells Extractor](6-dnd-spells-extractor.md)
The first contract-level walking skeleton should arrive at checkpoint 3: fixture
input through fake input, chunk, extract, merge, normalize, and output modules
with a fake LLM client. The first useful vertical slice should arrive at
checkpoint 6: Seriatim transcript input to validated D&D spell artifact output.
Earlier checkpoints remain contract-first and may not produce useful user output
yet.
## Architecture Decisions
- Final durable output should use one artifact file per artifact type plus a
run-level manifest/index file.
- Framework artifact flow should use a generic envelope with `json.RawMessage`
payloads. Extract modules should use typed Go structs at their own boundaries.
- Schemas should be versioned per extractor, with a separate envelope/manifest
format version.
- Artifact records should require source references by default. Individual
schemas may explicitly opt into ungrounded fields. Artifact-level metadata is
exempt.
- Source-reference ranges should be preserved exactly. Generic pipeline code
should not merge or rewrite overlapping ranges.
- `ExtractionRequest` should carry the active chunk plus optional ambient
context for document synopsis, prior-chunk summaries, known entities, or
similar module-provided state.
- LLM-backed review should be part of module-owned validator chains. Extract
modules and normalize modules may both use deterministic and LLM-backed
validators.
- The Seriatim MVP should support only the minimal Seriatim schema. Broader
Seriatim schema support should be added later without changing core source
contracts.
- Core source metadata should remain `map[string]any`. Well-known metadata keys
should be documented as conventions, and input modules may expose typed
accessor helpers for their own metadata.
- Configuration should use named pipeline profiles selected by ID at the CLI.
- A pipeline profile should be a fixed template, not a free-form DAG: shared
input and chunk stages, one or more artifact lanes, and shared output.
- `--only` should select configured artifact lanes without creating ad hoc
pipelines.
- Module bindings should support string shorthand and inline object settings,
normalized into one internal binding shape.
- Registries should expose flat capability metadata so config can fail fast on
invalid module combinations.
- The run manifest should record both `pipeline_id` and a digest of the resolved
pipeline definition after defaults and lane selection.
## Open Design Questions
- Which artifact types should use generic append-in-chunk-order merge, and which
should use domain-specific merge?
- Which artifact types need domain-specific normalization for deduplication,
identity resolution, or consistency?
- Which operational settings should be allowed as CLI/environment overrides
without weakening pipeline provenance?
## Near-Term Documentation Tasks
Once behavior is implemented, move implemented contracts out of roadmap docs and
into canonical docs:
- `README.md` for purpose and shortest useful command;
- `docs/cli.md` for CLI behavior;
- `docs/config.md` for config fields and precedence;
- `docs/internal/` for implemented architecture and package boundaries;
- `docs/integrations/` for source input and artifact file formats;
- `examples/` for maintained source, config, and artifact examples.

29
docs/roadmap/mvp.md Normal file
View File

@@ -0,0 +1,29 @@
# Future Work
Current Notarius behavior is documented in the canonical README, CLI,
configuration, operations, internal, and integration docs. This roadmap records
future work only.
## Candidate Product Work
- Additional input adapters, such as Markdown or note-export formats.
- Additional D&D extractors beyond spell casts.
- Cross-lane entity normalization.
- Cross-chunk semantic deduplication.
- Configurable validator chains with production validator modules.
- Multiple effective LLM profiles in one run.
- Parallel execution where it preserves deterministic manifests and diagnostics.
- Additional output encoders.
## Candidate Operational Work
- Packaged release artifacts for alpha distribution.
- A documented versioning and release process.
- Optional generated example output fixtures with a regeneration procedure.
- Additional diagnostics or reporting views if operator workflows need them.
## Non-Goals To Revisit Deliberately
- A general workflow language.
- Structural module selection through ad hoc run flags.
- Storing secrets in config files, diagnostics, manifests, or examples.

215
docs/troubleshooting.md Normal file
View File

@@ -0,0 +1,215 @@
# Troubleshooting
This guide maps common implemented failure modes to inspection steps and fixes.
For command syntax, see [CLI Reference](cli.md). For YAML fields and
environment overrides, see [Configuration](config.md). For output and
diagnostics layout, see [Operations](operations.md).
## Config File Not Found
Symptom:
```text
notarius: config file not found; pass --config or set NOTARIUS_CONFIG
```
Fix:
- Pass `--config path/to/config.yml`.
- Or set `NOTARIUS_CONFIG` to a readable file.
- Or install a config at `/usr/local/etc/notarius/config.yml`.
If the message says the config path is a directory or is not available, correct
the path or file permissions.
## Unsupported Or Invalid Config
Symptoms include:
- `unsupported config version`
- `config version is required`
- `field <name> not found`
- `total LLM concurrency must be greater than zero`
- `diagnostics retention "<value>" is not supported`
Fix:
- Use `version: 1`.
- Remove unknown YAML fields.
- Validate with:
```sh
go run ./cmd/notarius config validate --config path/to/config.yml
```
## Unknown Pipeline
Symptom:
```text
notarius: pipeline "..." is not configured
```
Fix:
- List configured pipeline IDs:
```sh
go run ./cmd/notarius pipelines list --config path/to/config.yml
```
- Use one of those IDs in `notarius run <pipeline-id>`.
- Check indentation under the top-level `pipelines` map.
## Unknown Or Incompatible Module
Symptoms mention a module key, pipeline slot, lane, capability, or `not
registered`.
Fix:
- Validate the pipeline against the production module catalog:
```sh
go run ./cmd/notarius config validate \
--config path/to/config.yml \
--pipeline dnd-session
```
- Use only implemented production module keys listed in
[Configuration](config.md#implemented-production-modules).
- Check that artifact lanes include an `extract` binding.
## Invalid `--only`
Symptoms include:
- `--only must contain comma-separated non-empty artifact lane IDs`
- `--only requires --pipeline`
- `selected artifact lane`
Fix:
- Use comma-separated lane IDs with no empty entries:
```sh
go run ./cmd/notarius run dnd-session \
--config path/to/config.yml \
--input path/to/input.json \
--only spells
```
- For `config validate`, include `--pipeline` when using `--only`.
- Confirm the lane ID exists under `pipelines.<id>.artifacts`.
## Seriatim Input Validation Failure
Symptoms include `seriatim input`, `parse JSON`, `segments must not be empty`,
or validation errors naming a segment field.
Fix:
- Compare the input to
[examples/seriatim-minimal-transcript.json](../examples/seriatim-minimal-transcript.json).
- Ensure the JSON has a `metadata` object and a non-empty `segments` array.
- Each segment needs a non-empty `id`, non-empty `speaker`, non-empty `text`,
non-negative numeric `start`, and non-negative numeric `end`.
- Segment IDs must be unique and must not contain leading or trailing
whitespace.
- `end` must be greater than or equal to `start`.
## Missing LLM Base URL Or Model
Symptoms include:
- `LLM profile "default" base URL must not be empty`
- `LLM profile "default" model must not be empty`
- `base URL must be valid`
Fix:
- Set `base_url` and `model` in `llm_profiles.default`.
- Or set `NOTARIUS_LLM_DEFAULT_BASE_URL` and
`NOTARIUS_LLM_DEFAULT_MODEL`.
- If a profile needs authentication, set `api_key_env` in YAML or set
`NOTARIUS_LLM_DEFAULT_API_KEY`.
## LLM Profile Override Failure
Symptom:
```text
notarius: LLM profile override "..." is not configured
```
Fix:
- Add the profile under `llm_profiles`.
- Or use an existing profile ID with `--llm-profile`.
Current runs require exactly one distinct effective LLM profile. If a pipeline
uses several profiles, run with `--llm-profile <id>` or align the bindings in
configuration.
## Provider HTTP Or Response Failure
Symptoms include:
- `provider request failed`
- `provider returned status 400`
- `provider returned status 403`
- `provider response missing choices`
- `provider response assistant message content is not valid JSON`
- `decode structured output`
Fix:
- Confirm the `base_url` points to an OpenAI-compatible endpoint root. Notarius
posts to `<base_url>/chat/completions`.
- Check `model` and provider credentials.
- Inspect the retained diagnostics `error.log`.
- For 400 and 403 responses, fix the request configuration or credentials.
- For 429 and 5xx responses, the client retries according to `max_retries`; if
the failure persists, inspect the provider response and adjust capacity,
credentials, or model settings.
- The assistant message content must decode as JSON matching the extractor's
structured response schema.
Provider error messages are redacted for configured API key values.
## Output Write Failure
Symptoms include:
- `create output directory`
- `write output file`
- `output file name must`
Fix:
- Ensure `--output-dir` points to a directory path or a path that can be
created.
- Check filesystem permissions and available disk space.
- If diagnostics were retained, inspect `run-report.json`, `run-manifest.json`,
and `error.log`.
The CLI rejects unsafe logical output paths before writing files.
## Diagnostics Directory Surprise
Symptom: the diagnostics directory is missing after a successful run.
Fix:
- Check `diagnostics.retention`.
- With `auto`, successful runs without warnings are removed.
- Use `diagnostics.retention: always` when every diagnostics run directory
should be kept.
- Use `--diagnostics-dir` to override the configured work directory for a run.
Symptom: diagnostics exist even with `retention: never`.
Explanation:
- Failed runs are retained so that `error.log` and available context can be
inspected.

View File

@@ -0,0 +1,16 @@
version: 1
llm_profiles:
default:
provider: openai-compatible
base_url: http://127.0.0.1:1
model: fake-model
pipelines:
dnd-session:
input: seriatim
chunk:
module: generic
options:
max_units: 50
artifacts:
spells:
extract: dnd/spells

View File

@@ -0,0 +1,22 @@
{
"metadata": {
"id": "session-alpha",
"title": "Synthetic D&D spell session"
},
"segments": [
{
"id": "seg-001",
"start": 0,
"end": 4,
"speaker": "Aria",
"text": "Aria raises her holy symbol and casts Cure Wounds."
},
{
"id": "seg-002",
"start": 4,
"end": 8,
"speaker": "DM",
"text": "The bandit mage casts Shield as the blow lands."
}
]
}

172
internal/cli/catalog.go Normal file
View File

@@ -0,0 +1,172 @@
package cli
import (
"context"
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/chunk/generic"
"gitea.maximumdirect.net/eric/notarius/internal/modules/extract/dnd/spells"
"gitea.maximumdirect.net/eric/notarius/internal/modules/input/seriatim"
"gitea.maximumdirect.net/eric/notarius/internal/modules/merge/appendorder"
"gitea.maximumdirect.net/eric/notarius/internal/modules/normalize/noop"
jsonoutput "gitea.maximumdirect.net/eric/notarius/internal/modules/output/json"
)
func productionRegistries() (pipeline.Registries, error) {
registries := pipeline.Registries{
Inputs: pipeline.NewInputAdapterRegistry(),
Chunkers: pipeline.NewChunkerRegistry(),
Extractors: pipeline.NewExtractorRegistry(),
Mergers: pipeline.NewMergerRegistry(),
Normalizers: pipeline.NewNormalizerRegistry(),
Validators: pipeline.NewValidatorRegistry(),
Outputs: pipeline.NewOutputEncoderRegistry(),
}
if err := seriatim.Register(registries.Inputs); err != nil {
return pipeline.Registries{}, fmt.Errorf("register seriatim input: %w", err)
}
if err := generic.Register(registries.Chunkers); err != nil {
return pipeline.Registries{}, fmt.Errorf("register generic chunker: %w", err)
}
if err := spells.Register(registries.Extractors); err != nil {
return pipeline.Registries{}, fmt.Errorf("register dnd spells extractor: %w", err)
}
if err := appendorder.Register(registries.Mergers); err != nil {
return pipeline.Registries{}, fmt.Errorf("register appendorder merger: %w", err)
}
if err := noop.Register(registries.Normalizers); err != nil {
return pipeline.Registries{}, fmt.Errorf("register noop normalizer: %w", err)
}
if err := jsonoutput.Register(registries.Outputs); err != nil {
return pipeline.Registries{}, fmt.Errorf("register json output encoder: %w", err)
}
return registries, nil
}
func productionCatalog() (pipeline.ModuleCatalog, error) {
registries, err := productionRegistries()
if err != nil {
return pipeline.ModuleCatalog{}, err
}
return catalogFromRegistries(registries), nil
}
func effectiveCatalog(opts Options) (pipeline.ModuleCatalog, error) {
if !isEmptyCatalog(opts.Catalog) {
return opts.Catalog, nil
}
if !isEmptyRegistries(opts.Registries) {
return catalogFromRegistries(opts.Registries), nil
}
return productionCatalog()
}
func effectiveRegistries(opts Options) (pipeline.Registries, error) {
if !isEmptyRegistries(opts.Registries) {
return opts.Registries, nil
}
if !isEmptyCatalog(opts.Catalog) {
return registriesFromCatalog(opts.Catalog), nil
}
return productionRegistries()
}
func catalogFromRegistries(registries pipeline.Registries) pipeline.ModuleCatalog {
return pipeline.ModuleCatalog{
Inputs: registries.Inputs,
Chunkers: registries.Chunkers,
Extractors: registries.Extractors,
Mergers: registries.Mergers,
Normalizers: registries.Normalizers,
Validators: registries.Validators,
Outputs: registries.Outputs,
}
}
func registriesFromCatalog(catalog pipeline.ModuleCatalog) pipeline.Registries {
return pipeline.Registries{
Inputs: catalog.Inputs,
Chunkers: catalog.Chunkers,
Extractors: catalog.Extractors,
Mergers: catalog.Mergers,
Normalizers: catalog.Normalizers,
Validators: catalog.Validators,
Outputs: catalog.Outputs,
}
}
func isEmptyCatalog(catalog pipeline.ModuleCatalog) bool {
return catalog.Inputs == nil &&
catalog.Chunkers == nil &&
catalog.Extractors == nil &&
catalog.Mergers == nil &&
catalog.Normalizers == nil &&
catalog.Validators == nil &&
catalog.Outputs == nil
}
func isEmptyRegistries(registries pipeline.Registries) bool {
return registries.Inputs == nil &&
registries.Chunkers == nil &&
registries.Extractors == nil &&
registries.Mergers == nil &&
registries.Normalizers == nil &&
registries.Validators == nil &&
registries.Outputs == nil
}
func productionLLMClientFactory(ctx context.Context, cfg config.Config, profileID string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) {
if err := ctx.Err(); err != nil {
return nil, nil, err
}
trimmedID := strings.TrimSpace(profileID)
if trimmedID == "" {
trimmedID = pipeline.DefaultLLMProfile
}
profile, ok := cfg.LLMProfile(trimmedID)
if !ok {
return nil, nil, fmt.Errorf("LLM profile %q is not configured", trimmedID)
}
clientCfg, err := cfg.OpenAICompatibleClientConfig(trimmedID)
if err != nil {
return nil, nil, err
}
client, err := llm.NewOpenAICompatibleClient(clientCfg)
if err != nil {
return nil, nil, fmt.Errorf("create LLM client for profile %q: %w", trimmedID, err)
}
scheduler, err := llm.NewScheduler(effectiveLLMConcurrency(cfg, profile))
if err != nil {
return nil, nil, fmt.Errorf("create LLM scheduler for profile %q: %w", trimmedID, err)
}
provider := strings.TrimSpace(profile.Provider)
if provider == "" {
provider = "openai-compatible"
}
metadata := []artifacts.LLMProfileManifest{
{
ID: trimmedID,
Provider: provider,
Model: strings.TrimSpace(profile.Model),
},
}
return llm.NewScheduledClient(client, scheduler), metadata, nil
}
func effectiveLLMConcurrency(cfg config.Config, profile config.LLMProfile) int {
if profile.MaxConcurrency > 0 {
return profile.MaxConcurrency
}
if cfg.Concurrency.TotalLLM > 0 {
return cfg.Concurrency.TotalLLM
}
return 1
}

View File

@@ -1,31 +1,45 @@
package cli
import (
"context"
"encoding/json"
"flag"
"fmt"
"io"
"os"
"path"
"path/filepath"
"sort"
"strings"
"time"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const defaultConfigPath = "/usr/local/etc/notarius/config.yml"
const defaultOutputRoot = "./notarius-output"
const usage = `Usage:
notarius help
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b]
notarius config validate --config path/to/config.yml [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list --config path/to/config.yml [--json]
`
type Options struct {
Catalog pipeline.ModuleCatalog
LookupEnv func(string) (string, bool)
Catalog pipeline.ModuleCatalog
Registries pipeline.Registries
LLMClientFactory LLMClientFactory
LookupEnv func(string) (string, bool)
Now func() time.Time
}
type LLMClientFactory func(ctx context.Context, cfg config.Config, profileID string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error)
// Run executes the command-line interface and returns a process exit code.
func Run(args []string, stdout, stderr io.Writer) int {
return RunWithOptions(args, stdout, stderr, Options{})
@@ -46,6 +60,8 @@ func RunWithOptions(args []string, stdout, stderr io.Writer, opts Options) int {
return runConfig(args[1:], stdout, stderr, opts)
case "pipelines":
return runPipelines(args[1:], stdout, stderr, opts)
case "run":
return runPipelineCommand(args[1:], stdout, stderr, opts)
default:
fmt.Fprintf(stderr, "notarius: unknown command %q\n", args[0])
writeUsage(stderr)
@@ -61,9 +77,389 @@ func normalizeOptions(opts Options) Options {
if opts.LookupEnv == nil {
opts.LookupEnv = os.LookupEnv
}
if opts.Now == nil {
opts.Now = time.Now
}
if opts.LLMClientFactory == nil {
opts.LLMClientFactory = productionLLMClientFactory
}
return opts
}
func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) int {
fs := flag.NewFlagSet("run", flag.ContinueOnError)
fs.SetOutput(io.Discard)
configPath := fs.String("config", "", "config file path")
inputPath := fs.String("input", "", "source input file path")
onlyRaw := fs.String("only", "", "comma-separated artifact lanes")
outputDir := fs.String("output-dir", "", "output directory")
diagnosticsDir := fs.String("diagnostics-dir", "", "diagnostics directory")
llmProfile := fs.String("llm-profile", "", "LLM profile override")
if err := fs.Parse(reorderRunArgs(args)); err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err)
return 2
}
if fs.NArg() == 0 {
fmt.Fprintln(stderr, "notarius: run requires a pipeline ID")
return 2
}
if fs.NArg() > 1 {
fmt.Fprintf(stderr, "notarius: unexpected argument %q\n", fs.Arg(1))
return 2
}
pipelineID := strings.TrimSpace(fs.Arg(0))
if pipelineID == "" {
fmt.Fprintln(stderr, "notarius: run requires a pipeline ID")
return 2
}
if strings.TrimSpace(*inputPath) == "" {
fmt.Fprintln(stderr, "notarius: run requires --input")
return 2
}
only, err := parseOnly(*onlyRaw)
if err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err)
return 2
}
cfg, loadedConfigPath, err := loadConfig(*configPath, opts)
if err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err)
return 1
}
if dir := strings.TrimSpace(*diagnosticsDir); dir != "" {
cfg.Diagnostics.WorkDir = dir
}
startedAt := opts.Now().UTC()
runDir, err := diagnostics.NewRunDirectory(cfg.Diagnostics.WorkDir, cfg.Diagnostics.Retention)
if err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err)
return 1
}
invocation := diagnostics.InvocationMetadata{
Operation: "run",
PipelineID: pipelineID,
InputPath: strings.TrimSpace(*inputPath),
ConfigPath: loadedConfigPath,
ConfigSource: configSource(*configPath),
OnlyLanes: append([]string(nil), only...),
RunID: runDir.RunID(),
StartedAt: startedAt,
}
if err := runDir.WriteInvocationMetadata(invocation); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics invocation metadata: %w", err))
}
catalog, err := effectiveCatalog(opts)
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
}
effective, err := cfg.Resolve(config.ResolveInput{
PipelineID: pipelineID,
Only: only,
Catalog: catalog,
LLMProfileOverride: *llmProfile,
})
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
}
invocation.PipelineDigest = effective.ResolvedPipeline.Digest
if err := runDir.WriteInvocationMetadata(invocation); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics invocation metadata: %w", err))
}
if err := runDir.WriteRedactedEffectiveConfig(effective); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics effective config: %w", err))
}
if err := runDir.WriteResolvedPipeline(effective.ResolvedPipeline); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics resolved pipeline: %w", err))
}
profileIDs := effectiveLLMProfileIDs(effective.ResolvedPipeline)
if len(profileIDs) != 1 {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("pipeline %q uses %d distinct LLM profiles; current runs require exactly one: %s", pipelineID, len(profileIDs), strings.Join(profileIDs, ", ")))
}
rawInput, err := os.ReadFile(strings.TrimSpace(*inputPath))
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("read input %q: %w", strings.TrimSpace(*inputPath), err))
}
registries, err := effectiveRegistries(opts)
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
}
ctx := context.Background()
llmClient, llmProfiles, err := opts.LLMClientFactory(ctx, effective.Config, profileIDs[0])
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("create LLM client for profile %q: %w", profileIDs[0], err))
}
output, err := pipeline.New(registries).Run(ctx, pipeline.RunInput{
Pipeline: effective.ResolvedPipeline,
Path: strings.TrimSpace(*inputPath),
RawInput: rawInput,
LLMClient: llmClient,
RunID: runDir.RunID(),
StartedAt: startedAt,
LLMProfiles: llmProfiles,
Metadata: runMetadata(*outputDir, *diagnosticsDir),
})
if err != nil {
if output.Manifest.PipelineID != "" {
_ = runDir.WriteRunManifest(output.Manifest)
}
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("run pipeline %q: %w", pipelineID, err))
}
runOutputDir := filepath.Join(outputRoot(*outputDir), runDir.RunID())
if err := runDir.WriteRunManifest(output.Manifest); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics run manifest: %w", err))
}
if err := runDir.WriteWarnings(output.Warnings); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics warnings: %w", err))
}
if err := runDir.WriteRunReport(runReport{
RunID: runDir.RunID(),
PipelineID: effective.PipelineID,
OutputPath: runOutputDir,
DiagnosticsPath: runDir.Path(),
ApprovedCount: len(output.Approved),
RejectedCount: len(output.Rejected),
WarningCount: len(output.Warnings),
ValidationStatus: output.Manifest.ValidationStatus,
}); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics run report: %w", err))
}
if err := writeOutputFiles(runOutputDir, output.OutputFiles); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
}
if err := runDir.ApplyRetention(diagnostics.RetentionDecisionInput{
RetentionMode: cfg.Diagnostics.Retention,
RunSucceeded: true,
HasWarnings: len(output.Warnings) > 0,
}); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("apply diagnostics retention: %w", err))
}
fmt.Fprintf(stdout, "pipeline %q complete: approved=%d rejected=%d output=%s\n", effective.PipelineID, len(output.Approved), len(output.Rejected), runOutputDir)
if len(output.Warnings) > 0 {
fmt.Fprintf(stderr, "notarius: run completed with %d warning(s)\n", len(output.Warnings))
}
return 0
}
type runReport struct {
RunID string `json:"run_id"`
PipelineID string `json:"pipeline_id"`
OutputPath string `json:"output_path"`
DiagnosticsPath string `json:"diagnostics_path,omitempty"`
ApprovedCount int `json:"approved_count"`
RejectedCount int `json:"rejected_count"`
WarningCount int `json:"warning_count"`
ValidationStatus string `json:"validation_status,omitempty"`
}
func failPipelineCommand(stderr io.Writer, runDir *diagnostics.RunDirectory, retention diagnostics.RetentionMode, err error) int {
fmt.Fprintf(stderr, "notarius: %v\n", err)
if runDir != nil {
if logErr := runDir.WriteErrorLog(err.Error()); logErr != nil {
fmt.Fprintf(stderr, "notarius: write diagnostics error log: %v\n", logErr)
}
if retentionErr := runDir.ApplyRetention(diagnostics.RetentionDecisionInput{
RetentionMode: retention,
RunSucceeded: false,
}); retentionErr != nil {
fmt.Fprintf(stderr, "notarius: apply diagnostics retention: %v\n", retentionErr)
}
}
return 1
}
func configSource(configPath string) string {
if strings.TrimSpace(configPath) != "" {
return "flag"
}
return "discovered"
}
func outputRoot(outputDir string) string {
if dir := strings.TrimSpace(outputDir); dir != "" {
return dir
}
return defaultOutputRoot
}
func writeOutputFiles(runOutputDir string, files []contracts.OutputFile) error {
type outputTarget struct {
path string
file contracts.OutputFile
}
targets := make([]outputTarget, 0, len(files))
for _, file := range files {
targetPath, err := outputFilePath(runOutputDir, file.Name)
if err != nil {
return err
}
targets = append(targets, outputTarget{path: targetPath, file: file})
}
if err := os.MkdirAll(runOutputDir, 0o755); err != nil {
return fmt.Errorf("create output directory %q: %w", runOutputDir, err)
}
for _, target := range targets {
if err := os.MkdirAll(filepath.Dir(target.path), 0o755); err != nil {
return fmt.Errorf("create output directory %q: %w", filepath.Dir(target.path), err)
}
if err := writeFileAtomic(target.path, target.file.Bytes, 0o644); err != nil {
return fmt.Errorf("write output file %q: %w", target.file.Name, err)
}
}
return nil
}
func outputFilePath(runOutputDir, logicalName string) (string, error) {
name := strings.TrimSpace(logicalName)
if name == "" {
return "", fmt.Errorf("output file name must not be empty")
}
if strings.Contains(name, `\`) {
return "", fmt.Errorf("output file name %q must use slash-separated relative paths", name)
}
if path.IsAbs(name) || filepath.IsAbs(name) {
return "", fmt.Errorf("output file name %q must be relative", name)
}
if strings.Contains(name, "..") {
return "", fmt.Errorf("output file name %q must not contain ..", name)
}
cleaned := path.Clean(name)
if cleaned == "." || cleaned != name {
return "", fmt.Errorf("output file name %q must be clean", name)
}
root, err := filepath.Abs(runOutputDir)
if err != nil {
return "", fmt.Errorf("resolve output directory %q: %w", runOutputDir, err)
}
target, err := filepath.Abs(filepath.Join(root, filepath.FromSlash(cleaned)))
if err != nil {
return "", fmt.Errorf("resolve output file %q: %w", name, err)
}
rel, err := filepath.Rel(root, target)
if err != nil {
return "", fmt.Errorf("resolve output file %q: %w", name, err)
}
if rel == "." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) || rel == ".." {
return "", fmt.Errorf("output file name %q resolves outside output directory", name)
}
return target, nil
}
func writeFileAtomic(path string, data []byte, perm os.FileMode) error {
dir := filepath.Dir(path)
temp, err := os.CreateTemp(dir, "."+filepath.Base(path)+".tmp-*")
if err != nil {
return err
}
tempPath := temp.Name()
removeTemp := true
defer func() {
if removeTemp {
_ = os.Remove(tempPath)
}
}()
if _, err := temp.Write(data); err != nil {
_ = temp.Close()
return err
}
if err := temp.Chmod(perm); err != nil {
_ = temp.Close()
return err
}
if err := temp.Close(); err != nil {
return err
}
if err := os.Rename(tempPath, path); err != nil {
return err
}
removeTemp = false
return nil
}
func reorderRunArgs(args []string) []string {
var flags []string
var positionals []string
for i := 0; i < len(args); i++ {
arg := args[i]
if arg == "--" {
positionals = append(positionals, args[i+1:]...)
break
}
if strings.HasPrefix(arg, "-") {
flags = append(flags, arg)
if runFlagTakesValue(arg) && !strings.Contains(arg, "=") && i+1 < len(args) {
i++
flags = append(flags, args[i])
}
continue
}
positionals = append(positionals, arg)
}
return append(flags, positionals...)
}
func runFlagTakesValue(arg string) bool {
switch arg {
case "--config", "--input", "--only", "--output-dir", "--diagnostics-dir", "--llm-profile":
return true
default:
return false
}
}
func effectiveLLMProfileIDs(resolved pipeline.ResolvedPipeline) []string {
seen := make(map[string]struct{})
add := func(binding pipeline.ModuleBinding) {
id := strings.TrimSpace(binding.LLMProfile)
if id != "" {
seen[id] = struct{}{}
}
}
add(resolved.Input)
add(resolved.Chunk)
add(resolved.Output)
for _, lane := range resolved.ArtifactLanes {
add(lane.Extract)
add(lane.Merge)
add(lane.Normalize)
for _, validator := range lane.Validators {
add(validator)
}
}
ids := make([]string, 0, len(seen))
for id := range seen {
ids = append(ids, id)
}
sort.Strings(ids)
return ids
}
func runMetadata(outputDir, diagnosticsDir string) map[string]any {
metadata := make(map[string]any)
if dir := strings.TrimSpace(outputDir); dir != "" {
metadata["output_dir"] = dir
}
if dir := strings.TrimSpace(diagnosticsDir); dir != "" {
metadata["diagnostics_dir"] = dir
}
if len(metadata) == 0 {
return nil
}
return metadata
}
func runConfig(args []string, stdout, stderr io.Writer, opts Options) int {
if len(args) == 0 {
fmt.Fprintln(stderr, "notarius: config requires a subcommand")
@@ -111,10 +507,15 @@ func runConfigValidate(args []string, stdout, stderr io.Writer, opts Options) in
}
if strings.TrimSpace(*pipelineID) != "" {
catalog, err := effectiveCatalog(opts)
if err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err)
return 1
}
if _, err := cfg.Resolve(config.ResolveInput{
PipelineID: *pipelineID,
Only: only,
Catalog: opts.Catalog,
Catalog: catalog,
}); err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err)
return 1

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,6 @@
{
"metadata": {
"id": "session-alpha"
},
"segments": []
}

View File

@@ -34,11 +34,18 @@ type RejectedArtifact struct {
}
type ArtifactLaneManifest struct {
ID string `json:"id"`
Extractor string `json:"extractor"`
Merger string `json:"merger"`
Normalizer string `json:"normalizer"`
Validators []string `json:"validators,omitempty"`
ID string `json:"id"`
Extractor string `json:"extractor"`
Merger string `json:"merger"`
Normalizer string `json:"normalizer"`
Validators []string `json:"validators,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
}
type LLMProfileManifest struct {
ID string `json:"id"`
Provider string `json:"provider,omitempty"`
Model string `json:"model,omitempty"`
}
type RunManifest struct {
@@ -53,6 +60,7 @@ type RunManifest struct {
Normalizer string `json:"normalizer,omitempty"`
OutputEncoder string `json:"output_encoder,omitempty"`
ArtifactLanes []ArtifactLaneManifest `json:"artifact_lanes,omitempty"`
LLMProfiles []LLMProfileManifest `json:"llm_profiles,omitempty"`
SchemaVersion string `json:"schema_version,omitempty"`
ValidationStatus string `json:"validation_status,omitempty"`
StartedAt *time.Time `json:"started_at,omitempty"`

View File

@@ -127,6 +127,9 @@ func TestRunManifestIncludesPipelineAndArtifactLaneFields(t *testing.T) {
manifest := RunManifest{
PipelineID: "pipeline-1",
PipelineDigest: "sha256:abc123",
LLMProfiles: []LLMProfileManifest{
{ID: "default", Provider: "openai-compatible", Model: "model-a"},
},
ArtifactLanes: []ArtifactLaneManifest{
{
ID: "events",
@@ -134,6 +137,9 @@ func TestRunManifestIncludesPipelineAndArtifactLaneFields(t *testing.T) {
Merger: "appendorder",
Normalizer: "noop",
Validators: []string{"grounded"},
Metadata: map[string]any{
"extractor": map[string]any{"prompt_id": "test.prompt"},
},
},
},
}
@@ -148,7 +154,20 @@ func TestRunManifestIncludesPipelineAndArtifactLaneFields(t *testing.T) {
t.Fatalf("json.Unmarshal() error = %v", err)
}
assertHasKeys(t, got, "pipeline_id", "pipeline_digest", "artifact_lanes")
assertHasKeys(t, got, "pipeline_id", "pipeline_digest", "artifact_lanes", "llm_profiles")
profiles, ok := got["llm_profiles"].([]any)
if !ok {
t.Fatalf("llm_profiles = %#v, want array", got["llm_profiles"])
}
if len(profiles) != 1 {
t.Fatalf("len(llm_profiles) = %d, want 1", len(profiles))
}
profile, ok := profiles[0].(map[string]any)
if !ok {
t.Fatalf("llm_profiles[0] = %#v, want object", profiles[0])
}
assertHasKeys(t, profile, "id", "provider", "model")
lanes, ok := got["artifact_lanes"].([]any)
if !ok {
@@ -161,7 +180,7 @@ func TestRunManifestIncludesPipelineAndArtifactLaneFields(t *testing.T) {
if !ok {
t.Fatalf("artifact_lanes[0] = %#v, want object", lanes[0])
}
assertHasKeys(t, lane, "id", "extractor", "merger", "normalizer", "validators")
assertHasKeys(t, lane, "id", "extractor", "merger", "normalizer", "validators", "metadata")
}
func assertHasKeys(t *testing.T, values map[string]any, keys ...string) {

View File

@@ -10,9 +10,10 @@ import (
)
type ResolveInput struct {
PipelineID string
Only []string
Catalog pipeline.ModuleCatalog
PipelineID string
Only []string
Catalog pipeline.ModuleCatalog
LLMProfileOverride string
}
type EffectiveConfig struct {
@@ -38,6 +39,12 @@ func (c Config) Resolve(input ResolveInput) (EffectiveConfig, error) {
}
profile = clonePipelineProfile(profile)
profile.ID = pipelineID
if override := strings.TrimSpace(input.LLMProfileOverride); override != "" {
if !hasLLMProfile(c.LLMProfiles, override) {
return EffectiveConfig{}, fmt.Errorf("LLM profile override %q is not configured", override)
}
applyLLMProfileOverride(&profile, override)
}
resolved, err := pipeline.ResolvePipeline(profile, pipeline.ResolveOptions{Only: input.Only}, input.Catalog)
if err != nil {
@@ -52,6 +59,21 @@ func (c Config) Resolve(input ResolveInput) (EffectiveConfig, error) {
}, nil
}
func applyLLMProfileOverride(profile *pipeline.PipelineProfile, profileID string) {
profile.Input.LLMProfile = profileID
profile.Chunk.LLMProfile = profileID
profile.Output.LLMProfile = profileID
for laneID, lane := range profile.Artifacts {
lane.Extract.LLMProfile = profileID
lane.Merge.LLMProfile = profileID
lane.Normalize.LLMProfile = profileID
for i := range lane.Validators {
lane.Validators[i].LLMProfile = profileID
}
profile.Artifacts[laneID] = lane
}
}
func lookupPipelineProfile(profiles map[string]pipeline.PipelineProfile, pipelineID string) (pipeline.PipelineProfile, bool) {
pipelineID = strings.TrimSpace(pipelineID)
for rawID, profile := range profiles {

View File

@@ -118,6 +118,51 @@ func TestResolveDigestChangesWhenEffectiveConfigChanges(t *testing.T) {
}
}
func TestResolveLLMProfileOverrideAppliesBeforeDigest(t *testing.T) {
cfg := validConfig()
cfg.LLMProfiles["runtime"] = LLMProfile{Provider: "openai-compatible"}
base, err := cfg.Resolve(ResolveInput{PipelineID: "example", Catalog: fakeCatalog(t)})
if err != nil {
t.Fatalf("Resolve base: %v", err)
}
effective, err := cfg.Resolve(ResolveInput{
PipelineID: "example",
Catalog: fakeCatalog(t),
LLMProfileOverride: "runtime",
})
if err != nil {
t.Fatalf("Resolve override: %v", err)
}
if base.ResolvedPipeline.Digest == effective.ResolvedPipeline.Digest {
t.Fatalf("expected digest to change after LLM profile override")
}
for _, binding := range resolvedBindings(effective.ResolvedPipeline) {
if binding.LLMProfile != "runtime" {
t.Fatalf("binding profile = %q, want runtime", binding.LLMProfile)
}
}
_, err = cfg.Resolve(ResolveInput{
PipelineID: "example",
Catalog: fakeCatalog(t),
LLMProfileOverride: "missing",
})
if err == nil || !strings.Contains(err.Error(), "LLM profile override") {
t.Fatalf("expected override profile error, got %v", err)
}
}
func resolvedBindings(resolved pipeline.ResolvedPipeline) []pipeline.ModuleBinding {
bindings := []pipeline.ModuleBinding{resolved.Input, resolved.Chunk, resolved.Output}
for _, lane := range resolved.ArtifactLanes {
bindings = append(bindings, lane.Extract, lane.Merge, lane.Normalize)
bindings = append(bindings, lane.Validators...)
}
return bindings
}
func TestOpenAICompatibleClientConfigRejectsIncompleteDefaultProfile(t *testing.T) {
cfg := Default()

View File

@@ -173,7 +173,7 @@ func (r *RunDirectory) WriteErrorLog(errorMessage string) error {
if err != nil {
return err
}
if err := os.WriteFile(path, []byte(errorMessage+"\n"), 0o644); err != nil {
if err := writeFileAtomic(path, []byte(errorMessage+"\n"), 0o644); err != nil {
return fmt.Errorf("write diagnostics artifact %q: %w", ArtifactErrorLog, err)
}
return nil
@@ -193,7 +193,7 @@ func (r *RunDirectory) WriteJSONArtifact(name string, payload any) error {
return fmt.Errorf("marshal diagnostics artifact %q: %w", name, err)
}
data = append(data, '\n')
if err := os.WriteFile(path, data, 0o644); err != nil {
if err := writeFileAtomic(path, data, 0o644); err != nil {
return fmt.Errorf("write diagnostics artifact %q: %w", name, err)
}
return nil
@@ -241,3 +241,39 @@ func (r *RunDirectory) artifactPath(name string) (string, error) {
}
return artifactPath, nil
}
func writeFileAtomic(path string, data []byte, perm os.FileMode) error {
dir := filepath.Dir(path)
if err := os.MkdirAll(dir, 0o755); err != nil {
return err
}
temp, err := os.CreateTemp(dir, "."+filepath.Base(path)+".tmp-*")
if err != nil {
return err
}
tempPath := temp.Name()
removeTemp := true
defer func() {
if removeTemp {
_ = os.Remove(tempPath)
}
}()
if _, err := temp.Write(data); err != nil {
_ = temp.Close()
return err
}
if err := temp.Chmod(perm); err != nil {
_ = temp.Close()
return err
}
if err := temp.Close(); err != nil {
return err
}
if err := os.Rename(tempPath, path); err != nil {
return err
}
removeTemp = false
return nil
}

View File

@@ -118,6 +118,24 @@ func TestWriteJSONArtifactWritesIndentedNewlineTerminatedJSON(t *testing.T) {
}
}
func TestWriteJSONArtifactLeavesNoTemporaryFiles(t *testing.T) {
runDir := newTestRunDirectory(t)
if err := runDir.WriteJSONArtifact("artifact.json", map[string]any{"value": "ok"}); err != nil {
t.Fatalf("WriteJSONArtifact: %v", err)
}
entries, err := os.ReadDir(runDir.Path())
if err != nil {
t.Fatalf("read run directory: %v", err)
}
for _, entry := range entries {
if strings.Contains(entry.Name(), ".tmp-") {
t.Fatalf("temporary diagnostics file remains after success: %s", entry.Name())
}
}
}
func TestWriteInvocationMetadataFillsMissingRunIDAndStartTime(t *testing.T) {
runDir := newTestRunDirectory(t)

View File

@@ -124,10 +124,13 @@ func TestContractsComposeAcrossPackages(t *testing.T) {
if err != nil {
t.Fatalf("Encode() error = %v, want nil", err)
}
if output.ContentType != "application/json" {
t.Fatalf("ContentType = %q, want application/json", output.ContentType)
if len(output.Files) != 1 {
t.Fatalf("len(Files) = %d, want 1", len(output.Files))
}
if len(output.Bytes) == 0 {
if output.Files[0].ContentType != "application/json" {
t.Fatalf("ContentType = %q, want application/json", output.Files[0].ContentType)
}
if len(output.Files[0].Bytes) == 0 {
t.Fatal("len(Bytes) = 0, want encoded bytes")
}
}
@@ -293,7 +296,12 @@ func (encoder compositionOutputEncoder) Encode(ctx context.Context, req contract
}
return contracts.OutputResult{
Bytes: encoded,
ContentType: "application/json",
Files: []contracts.OutputFile{
{
Name: "artifacts/generic.json",
ContentType: "application/json",
Bytes: encoded,
},
},
}, nil
}

View File

@@ -182,13 +182,22 @@ type OutputRequest struct {
Metadata map[string]any `json:"metadata,omitempty"`
}
type OutputFile struct {
Name string `json:"name"`
ContentType string `json:"content_type,omitempty"`
Bytes []byte `json:"-"`
}
type OutputResult struct {
Bytes []byte `json:"-"`
ContentType string `json:"content_type,omitempty"`
Warnings []Warning `json:"warnings,omitempty"`
Files []OutputFile `json:"files,omitempty"`
Warnings []Warning `json:"warnings,omitempty"`
}
type OutputEncoder interface {
Key() string
Encode(ctx context.Context, req OutputRequest) (OutputResult, error)
}
type ManifestMetadataProvider interface {
ManifestMetadata() map[string]any
}

View File

@@ -230,11 +230,44 @@ func TestFakeMergeNormalizeAndOutputContracts(t *testing.T) {
if encoder.Key() != "generic-output" {
t.Fatalf("OutputEncoder.Key() = %q, want generic-output", encoder.Key())
}
if encoded.ContentType != "application/json" {
t.Fatalf("ContentType = %q, want application/json", encoded.ContentType)
if len(encoded.Files) != 1 {
t.Fatalf("len(Files) = %d, want 1", len(encoded.Files))
}
if string(encoded.Bytes) != `{"run_id":"run-1","approved_count":1}` {
t.Fatalf("Bytes = %s, want encoded output", encoded.Bytes)
if encoded.Files[0].ContentType != "application/json" {
t.Fatalf("ContentType = %q, want application/json", encoded.Files[0].ContentType)
}
if string(encoded.Files[0].Bytes) != `{"run_id":"run-1","approved_count":1}` {
t.Fatalf("Bytes = %s, want encoded output", encoded.Files[0].Bytes)
}
}
func TestOutputFileJSONShapeOmitsBytes(t *testing.T) {
file := OutputFile{
Name: "artifacts/events.json",
ContentType: "application/json",
Bytes: []byte(`{"ignored":true}`),
}
encoded, err := json.Marshal(file)
if err != nil {
t.Fatalf("json.Marshal() error = %v, want nil", err)
}
var got map[string]any
if err := json.Unmarshal(encoded, &got); err != nil {
t.Fatalf("json.Unmarshal() error = %v, want nil", err)
}
if got["name"] != "artifacts/events.json" {
t.Fatalf("name = %#v, want logical file name", got["name"])
}
if got["content_type"] != "application/json" {
t.Fatalf("content_type = %#v, want application/json", got["content_type"])
}
if _, ok := got["Bytes"]; ok {
t.Fatalf("encoded output file leaked Bytes: %s", encoded)
}
if _, ok := got["bytes"]; ok {
t.Fatalf("encoded output file leaked bytes: %s", encoded)
}
}
@@ -397,7 +430,12 @@ func (encoder fakeOutputEncoder) Key() string {
func (encoder fakeOutputEncoder) Encode(ctx context.Context, req OutputRequest) (OutputResult, error) {
return OutputResult{
Bytes: []byte(`{"run_id":"` + req.Manifest.RunID + `","approved_count":1}`),
ContentType: "application/json",
Files: []OutputFile{
{
Name: "artifacts/generic.json",
ContentType: "application/json",
Bytes: []byte(`{"run_id":"` + req.Manifest.RunID + `","approved_count":1}`),
},
},
}, nil
}

View File

@@ -0,0 +1,43 @@
package llm
import (
"context"
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
type scheduledClient struct {
client contracts.StructuredLLMClient
scheduler *Scheduler
}
func NewScheduledClient(client contracts.StructuredLLMClient, scheduler *Scheduler) contracts.StructuredLLMClient {
return &scheduledClient{
client: client,
scheduler: scheduler,
}
}
func (c *scheduledClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
if c == nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("scheduled LLM client must not be nil")
}
if c.client == nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("scheduled LLM client inner client must not be nil")
}
if c.scheduler == nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("scheduled LLM client scheduler must not be nil")
}
var response contracts.StructuredCompletionResponse
err := c.scheduler.Run(ctx, func(ctx context.Context) error {
var callErr error
response, callErr = c.client.CompleteStructured(ctx, req, out)
return callErr
})
if err != nil {
return contracts.StructuredCompletionResponse{}, err
}
return response, nil
}

View File

@@ -0,0 +1,115 @@
package llm
import (
"context"
"encoding/json"
"errors"
"sync"
"sync/atomic"
"testing"
"time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
func TestScheduledClientEnforcesSchedulerLimit(t *testing.T) {
scheduler, err := NewScheduler(1)
if err != nil {
t.Fatalf("NewScheduler() error = %v, want nil", err)
}
inner := &blockingStructuredClient{
release: make(chan struct{}),
}
client := NewScheduledClient(inner, scheduler)
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func() {
defer wg.Done()
var out map[string]any
if _, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{}, &out); err != nil {
t.Errorf("CompleteStructured() error = %v, want nil", err)
}
}()
}
waitForAtomicAtLeast(t, &inner.calls, 1)
time.Sleep(20 * time.Millisecond)
if got := atomic.LoadInt32(&inner.maxInFlight); got > 1 {
t.Fatalf("max in-flight calls = %d, want <= 1", got)
}
close(inner.release)
wg.Wait()
if got := atomic.LoadInt32(&inner.calls); got != 3 {
t.Fatalf("calls = %d, want 3", got)
}
}
func TestScheduledClientPropagatesClientError(t *testing.T) {
scheduler, err := NewScheduler(1)
if err != nil {
t.Fatalf("NewScheduler() error = %v, want nil", err)
}
expected := errors.New("provider unavailable")
client := NewScheduledClient(&errorStructuredClient{err: expected}, scheduler)
_, err = client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{}, &struct{}{})
if !errors.Is(err, expected) {
t.Fatalf("CompleteStructured() error = %v, want %v", err, expected)
}
}
func TestScheduledClientPropagatesSchedulerError(t *testing.T) {
scheduler, err := NewScheduler(1)
if err != nil {
t.Fatalf("NewScheduler() error = %v, want nil", err)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
client := NewScheduledClient(&errorStructuredClient{}, scheduler)
_, err = client.CompleteStructured(ctx, contracts.StructuredCompletionRequest{}, &struct{}{})
if !errors.Is(err, context.Canceled) {
t.Fatalf("CompleteStructured() error = %v, want context canceled", err)
}
}
type blockingStructuredClient struct {
release chan struct{}
inFlight int32
maxInFlight int32
calls int32
}
func (c *blockingStructuredClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
atomic.AddInt32(&c.calls, 1)
current := atomic.AddInt32(&c.inFlight, 1)
for {
seen := atomic.LoadInt32(&c.maxInFlight)
if current <= seen || atomic.CompareAndSwapInt32(&c.maxInFlight, seen, current) {
break
}
}
defer atomic.AddInt32(&c.inFlight, -1)
select {
case <-c.release:
case <-ctx.Done():
return contracts.StructuredCompletionResponse{}, ctx.Err()
}
if target, ok := out.(*map[string]any); ok {
*target = map[string]any{"ok": true}
}
return contracts.StructuredCompletionResponse{
Content: json.RawMessage(`{"ok":true}`),
}, nil
}
type errorStructuredClient struct {
err error
}
func (c *errorStructuredClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
return contracts.StructuredCompletionResponse{}, c.err
}

View File

@@ -6,6 +6,7 @@ import (
"encoding/hex"
"encoding/json"
"fmt"
"io/fs"
"sort"
"strings"
)
@@ -23,6 +24,15 @@ const (
schemaVersionV1 = "v1"
)
// ResponseSchemaDefinition identifies a caller-owned structured response schema asset.
type ResponseSchemaDefinition struct {
Key ResponseSchemaKey
ID string
Version string
Name string
AssetPath string
}
// ResponseSchema describes one registered structured response schema.
type ResponseSchema struct {
Key ResponseSchemaKey `json:"key"`
@@ -34,20 +44,20 @@ type ResponseSchema struct {
}
var responseSchemaRegistry = map[ResponseSchemaKey]ResponseSchema{
TestArtifactSchemaKey: mustLoadResponseSchema(
TestArtifactSchemaKey,
"notarius.test_artifact",
schemaVersionV1,
"notarius_test_artifact_v1",
"assets/schemas/test_artifact.v1.json",
),
TestValidatorDecisionSchemaKey: mustLoadResponseSchema(
TestValidatorDecisionSchemaKey,
"notarius.test_validator_decision",
schemaVersionV1,
"notarius_test_validator_decision_v1",
"assets/schemas/test_validator_decision.v1.json",
),
TestArtifactSchemaKey: mustLoadResponseSchema(schemaAssets, ResponseSchemaDefinition{
Key: TestArtifactSchemaKey,
ID: "notarius.test_artifact",
Version: schemaVersionV1,
Name: "notarius_test_artifact_v1",
AssetPath: "assets/schemas/test_artifact.v1.json",
}),
TestValidatorDecisionSchemaKey: mustLoadResponseSchema(schemaAssets, ResponseSchemaDefinition{
Key: TestValidatorDecisionSchemaKey,
ID: "notarius.test_validator_decision",
Version: schemaVersionV1,
Name: "notarius_test_validator_decision_v1",
AssetPath: "assets/schemas/test_validator_decision.v1.json",
}),
}
// RegisteredResponseSchemas returns all registered response schemas sorted by key.
@@ -94,40 +104,35 @@ func (s ResponseSchema) DiagnosticsMap() map[string]any {
}
}
func mustLoadResponseSchema(
key ResponseSchemaKey,
id string,
version string,
name string,
path string,
) ResponseSchema {
key = ResponseSchemaKey(strings.TrimSpace(string(key)))
id = strings.TrimSpace(id)
version = strings.TrimSpace(version)
name = strings.TrimSpace(name)
path = strings.TrimSpace(path)
// LoadResponseSchema loads a structured response schema from a caller-owned filesystem.
func LoadResponseSchema(fsys fs.FS, def ResponseSchemaDefinition) (ResponseSchema, error) {
key := ResponseSchemaKey(strings.TrimSpace(string(def.Key)))
id := strings.TrimSpace(def.ID)
version := strings.TrimSpace(def.Version)
name := strings.TrimSpace(def.Name)
path := strings.TrimSpace(def.AssetPath)
if key == "" {
panic("response schema key must not be empty")
return ResponseSchema{}, fmt.Errorf("response schema key must not be empty")
}
if id == "" {
panic("response schema id must not be empty")
return ResponseSchema{}, fmt.Errorf("response schema id must not be empty")
}
if version == "" {
panic("response schema version must not be empty")
return ResponseSchema{}, fmt.Errorf("response schema version must not be empty")
}
if name == "" {
panic("response schema name must not be empty")
return ResponseSchema{}, fmt.Errorf("response schema name must not be empty")
}
if path == "" {
panic("response schema asset path must not be empty")
return ResponseSchema{}, fmt.Errorf("response schema asset path must not be empty")
}
rawSchema, err := schemaAssets.ReadFile(path)
rawSchema, err := fs.ReadFile(fsys, path)
if err != nil {
panic(fmt.Sprintf("read response schema %s: %v", path, err))
return ResponseSchema{}, fmt.Errorf("read response schema %s: %w", path, err)
}
if !json.Valid(rawSchema) {
panic(fmt.Sprintf("response schema %s is not valid JSON", path))
return ResponseSchema{}, fmt.Errorf("response schema %s is not valid JSON", path)
}
hash := sha256.Sum256(rawSchema)
@@ -138,7 +143,15 @@ func mustLoadResponseSchema(
Name: name,
JSONSchema: append(json.RawMessage(nil), rawSchema...),
SHA256: "sha256:" + hex.EncodeToString(hash[:]),
}, nil
}
func mustLoadResponseSchema(fsys fs.FS, def ResponseSchemaDefinition) ResponseSchema {
schema, err := LoadResponseSchema(fsys, def)
if err != nil {
panic(err)
}
return schema
}
func cloneResponseSchema(in ResponseSchema) ResponseSchema {

View File

@@ -7,7 +7,7 @@ import (
"testing"
)
func TestLookupResponseSchemaSucceedsForTestSchemas(t *testing.T) {
func TestLookupResponseSchemaSucceedsForRegisteredSchemas(t *testing.T) {
tests := []ResponseSchemaKey{
TestArtifactSchemaKey,
TestValidatorDecisionSchemaKey,
@@ -38,6 +38,12 @@ func TestLookupResponseSchemaUnknownReturnsFalse(t *testing.T) {
}
}
func TestLookupResponseSchemaDNDSpellsIsNotFrameworkRegistered(t *testing.T) {
if schema, ok := LookupResponseSchema("dnd_spells"); ok {
t.Fatalf("expected D&D spells schema lookup to fail in framework registry, got %+v", schema)
}
}
func TestMustLookupResponseSchemaPanicsForUnknownKey(t *testing.T) {
defer func() {
if recover() == nil {
@@ -55,12 +61,17 @@ func TestRegisteredResponseSchemasSortedByKey(t *testing.T) {
}
keys := make([]string, len(schemas))
seen := make(map[ResponseSchemaKey]bool, len(schemas))
for i, schema := range schemas {
keys[i] = string(schema.Key)
seen[schema.Key] = true
}
if !sort.StringsAreSorted(keys) {
t.Fatalf("expected sorted keys, got %v", keys)
}
if !seen[TestArtifactSchemaKey] || !seen[TestValidatorDecisionSchemaKey] {
t.Fatalf("registered schemas = %v, want test schemas", keys)
}
}
func TestResponseSchemaContentIsValidJSON(t *testing.T) {
@@ -72,26 +83,30 @@ func TestResponseSchemaContentIsValidJSON(t *testing.T) {
}
func TestResponseSchemaJSONIsMutationSafe(t *testing.T) {
first := MustLookupResponseSchema(TestArtifactSchemaKey)
first.JSONSchema[0] = '['
for _, key := range []ResponseSchemaKey{TestArtifactSchemaKey, TestValidatorDecisionSchemaKey} {
t.Run(string(key), func(t *testing.T) {
first := MustLookupResponseSchema(key)
first.JSONSchema[0] = '['
second := MustLookupResponseSchema(TestArtifactSchemaKey)
if !json.Valid(second.JSONSchema) {
t.Fatalf("schema JSON was mutated: %s", second.JSONSchema)
}
if len(second.JSONSchema) > 0 && second.JSONSchema[0] == '[' {
t.Fatalf("schema JSON did not use defensive copy")
}
second := MustLookupResponseSchema(key)
if !json.Valid(second.JSONSchema) {
t.Fatalf("schema JSON was mutated: %s", second.JSONSchema)
}
if len(second.JSONSchema) > 0 && second.JSONSchema[0] == '[' {
t.Fatalf("schema JSON did not use defensive copy")
}
registered := RegisteredResponseSchemas()
for i := range registered {
if registered[i].Key == TestArtifactSchemaKey {
registered[i].JSONSchema[0] = '['
}
}
again := MustLookupResponseSchema(TestArtifactSchemaKey)
if !json.Valid(again.JSONSchema) || again.JSONSchema[0] == '[' {
t.Fatalf("registered schema JSON did not use defensive copy")
registered := RegisteredResponseSchemas()
for i := range registered {
if registered[i].Key == key {
registered[i].JSONSchema[0] = '['
}
}
again := MustLookupResponseSchema(key)
if !json.Valid(again.JSONSchema) || again.JSONSchema[0] == '[' {
t.Fatalf("registered schema JSON did not use defensive copy")
}
})
}
}

View File

@@ -0,0 +1,124 @@
package pipeline_test
import (
"context"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/chunk/generic"
"gitea.maximumdirect.net/eric/notarius/internal/modules/merge/appendorder"
"gitea.maximumdirect.net/eric/notarius/internal/modules/normalize/noop"
jsonoutput "gitea.maximumdirect.net/eric/notarius/internal/modules/output/json"
)
func TestPipelineConfigResolvesWithProductionDefaultsRegistered(t *testing.T) {
cfg := config.Default()
cfg.Pipelines = map[string]pipeline.PipelineProfile{
"defaults": {
Input: pipeline.Binding("input"),
Artifacts: map[string]pipeline.ArtifactLaneProfile{
"events": {Extract: pipeline.Binding("extract")},
},
},
}
resolved, err := cfg.Resolve(config.ResolveInput{
PipelineID: "defaults",
Catalog: defaultModuleCatalog(t),
})
if err != nil {
t.Fatalf("Resolve() error = %v, want nil", err)
}
pipeline := resolved.ResolvedPipeline
if pipeline.Chunk.Module != generic.Key {
t.Fatalf("Chunk.Module = %q, want %q", pipeline.Chunk.Module, generic.Key)
}
if pipeline.Output.Module != jsonoutput.Key {
t.Fatalf("Output.Module = %q, want %q", pipeline.Output.Module, jsonoutput.Key)
}
lane := pipeline.ArtifactLanes[0]
if lane.Merge.Module != appendorder.Key {
t.Fatalf("Merge.Module = %q, want %q", lane.Merge.Module, appendorder.Key)
}
if lane.Normalize.Module != noop.Key {
t.Fatalf("Normalize.Module = %q, want %q", lane.Normalize.Module, noop.Key)
}
}
func defaultModuleCatalog(t *testing.T) pipeline.ModuleCatalog {
t.Helper()
inputs := pipeline.NewInputAdapterRegistry()
chunkers := pipeline.NewChunkerRegistry()
extractors := pipeline.NewExtractorRegistry()
mergers := pipeline.NewMergerRegistry()
normalizers := pipeline.NewNormalizerRegistry()
outputs := pipeline.NewOutputEncoderRegistry()
if err := inputs.RegisterWithSpec(pipeline.ModuleSpec{
Key: "input",
Stage: pipeline.StageInput,
Provides: []string{"source"},
}, func() (contracts.InputAdapter, error) {
return defaultInput{}, nil
}); err != nil {
t.Fatalf("register input: %v", err)
}
if err := generic.Register(chunkers); err != nil {
t.Fatalf("register generic chunker: %v", err)
}
if err := extractors.RegisterWithSpec(pipeline.ModuleSpec{
Key: "extract",
Stage: pipeline.StageExtract,
Requires: []string{"chunks"},
Provides: []string{"records"},
}, func() (contracts.Extractor, error) {
return defaultExtractor{}, nil
}); err != nil {
t.Fatalf("register extractor: %v", err)
}
if err := appendorder.Register(mergers); err != nil {
t.Fatalf("register appendorder merger: %v", err)
}
if err := noop.Register(normalizers); err != nil {
t.Fatalf("register noop normalizer: %v", err)
}
if err := jsonoutput.Register(outputs); err != nil {
t.Fatalf("register json output: %v", err)
}
return pipeline.ModuleCatalog{
Inputs: inputs,
Chunkers: chunkers,
Extractors: extractors,
Mergers: mergers,
Normalizers: normalizers,
Outputs: outputs,
}
}
type defaultInput struct{}
func (defaultInput) Key() string { return "input" }
func (defaultInput) Parse(ctx context.Context, req contracts.ParseRequest) (*source.SourceDocument, error) {
return nil, nil
}
type defaultExtractor struct{}
func (defaultExtractor) Key() string { return "extract" }
func (defaultExtractor) ArtifactType() string { return "record" }
func (defaultExtractor) SchemaVersion() string { return "v1" }
func (defaultExtractor) Validators() []contracts.Validator { return nil }
func (defaultExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
return contracts.ExtractionResult{}, nil
}

View File

@@ -1,70 +0,0 @@
package pipeline
import (
"context"
"encoding/json"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
type AppendOrderMerger struct{}
func (m AppendOrderMerger) Key() string {
return DefaultMergeModule
}
func (m AppendOrderMerger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) {
var candidates []artifacts.ArtifactCandidate
for _, chunkArtifacts := range req.ChunkArtifacts {
candidates = append(candidates, copyArtifactCandidates(chunkArtifacts.Candidates)...)
}
return contracts.MergeResult{Candidates: candidates}, nil
}
type NoopNormalizer struct{}
func (n NoopNormalizer) Key() string {
return DefaultNormalizeModule
}
func (n NoopNormalizer) Normalize(ctx context.Context, req contracts.NormalizeRequest) (contracts.NormalizeResult, error) {
return contracts.NormalizeResult{Candidates: copyArtifactCandidates(req.Candidates)}, nil
}
func copyArtifactCandidates(candidates []artifacts.ArtifactCandidate) []artifacts.ArtifactCandidate {
if len(candidates) == 0 {
return nil
}
copied := make([]artifacts.ArtifactCandidate, 0, len(candidates))
for _, candidate := range candidates {
copied = append(copied, copyArtifactCandidate(candidate))
}
return copied
}
func copyArtifactCandidate(candidate artifacts.ArtifactCandidate) artifacts.ArtifactCandidate {
return artifacts.ArtifactCandidate{
Index: candidate.Index,
ExtractorKey: candidate.ExtractorKey,
ArtifactType: candidate.ArtifactType,
SchemaVersion: candidate.SchemaVersion,
Payload: append(json.RawMessage(nil), candidate.Payload...),
SourceRefs: append([]source.SourceRef(nil), candidate.SourceRefs...),
Metadata: copyArtifactMetadata(candidate.Metadata),
}
}
func copyArtifactMetadata(metadata map[string]any) map[string]any {
if len(metadata) == 0 {
return nil
}
copied := make(map[string]any, len(metadata))
for key, value := range metadata {
copied[key] = value
}
return copied
}

View File

@@ -1,221 +0,0 @@
package pipeline
import (
"context"
"encoding/json"
"reflect"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
func TestGenericMergeAndNormalizeKeys(t *testing.T) {
merger := AppendOrderMerger{}
normalizer := NoopNormalizer{}
if merger.Key() != DefaultMergeModule {
t.Fatalf("AppendOrderMerger.Key() = %q, want %q", merger.Key(), DefaultMergeModule)
}
if normalizer.Key() != DefaultNormalizeModule {
t.Fatalf("NoopNormalizer.Key() = %q, want %q", normalizer.Key(), DefaultNormalizeModule)
}
}
func TestAppendOrderMergerConcatenatesByChunkAndCandidateOrder(t *testing.T) {
merger := AppendOrderMerger{}
chunks := []contracts.ChunkArtifacts{
{
Chunk: sourceChunk(0),
Candidates: []artifacts.ArtifactCandidate{
candidate(2, "first-b"),
candidate(1, "first-a"),
},
},
{
Chunk: sourceChunk(1),
Candidates: []artifacts.ArtifactCandidate{
candidate(4, "second-b"),
candidate(3, "second-a"),
},
},
}
result, err := merger.Merge(context.Background(), contracts.MergeRequest{ChunkArtifacts: chunks})
if err != nil {
t.Fatalf("Merge() error = %v, want nil", err)
}
if len(result.Warnings) != 0 {
t.Fatalf("Warnings = %#v, want none", result.Warnings)
}
got := candidateNames(result.Candidates)
want := []string{"first-b", "first-a", "second-b", "second-a"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("candidate order = %#v, want %#v", got, want)
}
}
func TestAppendOrderMergerReturnsMutationSafeCandidates(t *testing.T) {
merger := AppendOrderMerger{}
input := []contracts.ChunkArtifacts{
{
Chunk: sourceChunk(0),
Candidates: []artifacts.ArtifactCandidate{
candidate(1, "original"),
},
},
}
result, err := merger.Merge(context.Background(), contracts.MergeRequest{ChunkArtifacts: input})
if err != nil {
t.Fatalf("Merge() error = %v, want nil", err)
}
if len(result.Candidates) != 1 {
t.Fatalf("len(Candidates) = %d, want 1", len(result.Candidates))
}
input[0].Candidates[0].Index = 99
input[0].Candidates[0].Payload[0] = '['
input[0].Candidates[0].SourceRefs[0].StartUnitID = "changed"
input[0].Candidates[0].Metadata["name"] = "changed"
got := result.Candidates[0]
if got.Index != 1 {
t.Fatalf("Index = %d, want 1", got.Index)
}
if string(got.Payload) != `{"name":"original"}` {
t.Fatalf("Payload = %s, want original payload", got.Payload)
}
if got.SourceRefs[0].StartUnitID != "u1" {
t.Fatalf("SourceRefs = %#v, want original source ref", got.SourceRefs)
}
if got.Metadata["name"] != "original" {
t.Fatalf("Metadata = %#v, want original metadata", got.Metadata)
}
}
func TestNoopNormalizerPreservesOrderAndValues(t *testing.T) {
normalizer := NoopNormalizer{}
input := []artifacts.ArtifactCandidate{
candidate(3, "third"),
candidate(1, "first"),
candidate(2, "second"),
}
result, err := normalizer.Normalize(context.Background(), contracts.NormalizeRequest{Candidates: input})
if err != nil {
t.Fatalf("Normalize() error = %v, want nil", err)
}
if len(result.Warnings) != 0 {
t.Fatalf("Warnings = %#v, want none", result.Warnings)
}
got := candidateNames(result.Candidates)
want := []string{"third", "first", "second"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("candidate order = %#v, want %#v", got, want)
}
if !reflect.DeepEqual(result.Candidates[0].SourceRefs, input[0].SourceRefs) {
t.Fatalf("SourceRefs = %#v, want %#v", result.Candidates[0].SourceRefs, input[0].SourceRefs)
}
if !reflect.DeepEqual(result.Candidates[0].Metadata, input[0].Metadata) {
t.Fatalf("Metadata = %#v, want %#v", result.Candidates[0].Metadata, input[0].Metadata)
}
}
func TestNoopNormalizerReturnsMutationSafeCandidates(t *testing.T) {
normalizer := NoopNormalizer{}
input := []artifacts.ArtifactCandidate{candidate(1, "original")}
result, err := normalizer.Normalize(context.Background(), contracts.NormalizeRequest{Candidates: input})
if err != nil {
t.Fatalf("Normalize() error = %v, want nil", err)
}
if len(result.Candidates) != 1 {
t.Fatalf("len(Candidates) = %d, want 1", len(result.Candidates))
}
input[0].Index = 99
input[0].Payload[0] = '['
input[0].SourceRefs[0].EndUnitID = "changed"
input[0].Metadata["name"] = "changed"
got := result.Candidates[0]
if got.Index != 1 {
t.Fatalf("Index = %d, want 1", got.Index)
}
if string(got.Payload) != `{"name":"original"}` {
t.Fatalf("Payload = %s, want original payload", got.Payload)
}
if got.SourceRefs[0].EndUnitID != "u1" {
t.Fatalf("SourceRefs = %#v, want original source ref", got.SourceRefs)
}
if got.Metadata["name"] != "original" {
t.Fatalf("Metadata = %#v, want original metadata", got.Metadata)
}
}
func TestGenericMergeAndNormalizeHandleEmptyInput(t *testing.T) {
merger := AppendOrderMerger{}
normalizer := NoopNormalizer{}
mergeResult, err := merger.Merge(context.Background(), contracts.MergeRequest{})
if err != nil {
t.Fatalf("Merge() error = %v, want nil", err)
}
if len(mergeResult.Candidates) != 0 {
t.Fatalf("len(mergeResult.Candidates) = %d, want 0", len(mergeResult.Candidates))
}
if len(mergeResult.Warnings) != 0 {
t.Fatalf("merge warnings = %#v, want none", mergeResult.Warnings)
}
normalizeResult, err := normalizer.Normalize(context.Background(), contracts.NormalizeRequest{})
if err != nil {
t.Fatalf("Normalize() error = %v, want nil", err)
}
if len(normalizeResult.Candidates) != 0 {
t.Fatalf("len(normalizeResult.Candidates) = %d, want 0", len(normalizeResult.Candidates))
}
if len(normalizeResult.Warnings) != 0 {
t.Fatalf("normalize warnings = %#v, want none", normalizeResult.Warnings)
}
}
func candidate(index int, name string) artifacts.ArtifactCandidate {
return artifacts.ArtifactCandidate{
Index: index,
ExtractorKey: "generic-extractor",
ArtifactType: "generic-artifact",
SchemaVersion: "v1",
Payload: json.RawMessage(`{"name":"` + name + `"}`),
SourceRefs: []source.SourceRef{
{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u1"},
},
Metadata: map[string]any{
"name": name,
},
}
}
func candidateNames(candidates []artifacts.ArtifactCandidate) []string {
names := make([]string, 0, len(candidates))
for _, candidate := range candidates {
names = append(names, candidate.Metadata["name"].(string))
}
return names
}
func sourceChunk(index int) contracts.SourceChunk {
return contracts.SourceChunk{
ID: "chunk",
SourceID: "source-1",
Index: index,
Units: []source.SourceUnit{
{ID: "u1", Kind: "unit", Text: "Source unit."},
},
}
}

View File

@@ -193,7 +193,11 @@ func (output integrationOutput) Key() string {
}
func (output integrationOutput) Encode(ctx context.Context, req contracts.OutputRequest) (contracts.OutputResult, error) {
return contracts.OutputResult{Bytes: []byte(`{}`), ContentType: "application/json"}, nil
return contracts.OutputResult{
Files: []contracts.OutputFile{
{Name: "output.json", ContentType: "application/json", Bytes: []byte(`{}`)},
},
}, nil
}
type integrationValidator struct {

View File

@@ -3,6 +3,9 @@ package pipeline
import (
"context"
"fmt"
"path"
"strings"
"time"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
@@ -29,21 +32,23 @@ func New(registries Registries) *Runner {
}
type RunInput struct {
Pipeline ResolvedPipeline
SourceID string
Path string
RawInput []byte
LLMClient contracts.StructuredLLMClient
Metadata map[string]any
Pipeline ResolvedPipeline
SourceID string
Path string
RawInput []byte
LLMClient contracts.StructuredLLMClient
RunID string
StartedAt time.Time
LLMProfiles []artifacts.LLMProfileManifest
Metadata map[string]any
}
type RunOutput struct {
Manifest artifacts.RunManifest `json:"manifest"`
Approved []artifacts.Artifact `json:"approved,omitempty"`
Rejected []artifacts.RejectedArtifact `json:"rejected,omitempty"`
Warnings []contracts.Warning `json:"warnings,omitempty"`
EncodedOutput []byte `json:"-"`
ContentType string `json:"content_type,omitempty"`
Manifest artifacts.RunManifest `json:"manifest"`
Approved []artifacts.Artifact `json:"approved,omitempty"`
Rejected []artifacts.RejectedArtifact `json:"rejected,omitempty"`
Warnings []contracts.Warning `json:"warnings,omitempty"`
OutputFiles []contracts.OutputFile `json:"-"`
}
func (r *Runner) Run(ctx context.Context, input RunInput) (RunOutput, error) {
@@ -58,7 +63,7 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (RunOutput, error) {
return output, err
}
output.Manifest = manifestFromPipeline(input.Pipeline)
output.Manifest = manifestFromPipeline(input)
adapter, err := r.registries.Inputs.Build(input.Pipeline.Input.Module)
if err != nil {
@@ -110,6 +115,7 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (RunOutput, error) {
} else {
output.Manifest.ValidationStatus = "approved"
}
output.Manifest.CompletedAt = timePtr(time.Now().UTC())
encoder, err := r.registries.Outputs.Build(input.Pipeline.Output.Module)
if err != nil {
@@ -128,8 +134,11 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (RunOutput, error) {
if err != nil {
return failOutput(output), fmt.Errorf("encode output with encoder %q: %w", encoder.Key(), err)
}
output.EncodedOutput = encoded.Bytes
output.ContentType = encoded.ContentType
files, err := outputFilesFromResult(encoded)
if err != nil {
return failOutput(output), fmt.Errorf("validate output files from encoder %q: %w", encoder.Key(), err)
}
output.OutputFiles = files
return output, nil
}
@@ -147,6 +156,7 @@ func (r *Runner) runLane(ctx context.Context, input RunInput, doc *source.Source
if err != nil {
return fmt.Errorf("build normalizer %q for lane %q: %w", lane.Normalize.Module, lane.ID, err)
}
setLaneManifestMetadata(output, lane.ID, extractor, merger, normalizer)
var validators []validatorExecution
if len(lane.Validators) > 0 {
@@ -315,7 +325,17 @@ func validateRunInput(input RunInput) error {
return nil
}
func manifestFromPipeline(pipeline ResolvedPipeline) artifacts.RunManifest {
func manifestFromPipeline(input RunInput) artifacts.RunManifest {
startedAt := input.StartedAt
if startedAt.IsZero() {
startedAt = time.Now().UTC()
}
runID := strings.TrimSpace(input.RunID)
if runID == "" {
runID = fmt.Sprintf("run-%d", startedAt.UnixNano())
}
pipeline := input.Pipeline
manifest := artifacts.RunManifest{
PipelineID: pipeline.ID,
PipelineDigest: pipeline.Digest,
@@ -323,6 +343,9 @@ func manifestFromPipeline(pipeline ResolvedPipeline) artifacts.RunManifest {
Chunker: pipeline.Chunk.Module,
OutputEncoder: pipeline.Output.Module,
ArtifactLanes: make([]artifacts.ArtifactLaneManifest, 0, len(pipeline.ArtifactLanes)),
RunID: runID,
StartedAt: timePtr(startedAt),
LLMProfiles: cloneLLMProfiles(input.LLMProfiles),
}
for _, lane := range pipeline.ArtifactLanes {
@@ -343,10 +366,113 @@ func manifestFromPipeline(pipeline ResolvedPipeline) artifacts.RunManifest {
func failOutput(output RunOutput) RunOutput {
if output.Manifest.PipelineID != "" {
output.Manifest.ValidationStatus = "failed"
output.Manifest.CompletedAt = timePtr(time.Now().UTC())
}
return output
}
func setLaneManifestMetadata(output *RunOutput, laneID string, modules ...any) {
if output == nil {
return
}
for i := range output.Manifest.ArtifactLanes {
if output.Manifest.ArtifactLanes[i].ID != laneID {
continue
}
metadata := make(map[string]any)
for _, module := range modules {
provider, ok := module.(contracts.ManifestMetadataProvider)
if !ok {
continue
}
moduleMetadata := cloneMetadata(provider.ManifestMetadata())
if len(moduleMetadata) == 0 {
continue
}
key := manifestMetadataKey(module)
if key == "" {
continue
}
metadata[key] = moduleMetadata
}
if len(metadata) > 0 {
output.Manifest.ArtifactLanes[i].Metadata = metadata
}
return
}
}
func manifestMetadataKey(module any) string {
switch module.(type) {
case contracts.Extractor:
return "extractor"
case contracts.Merger:
return "merger"
case contracts.Normalizer:
return "normalizer"
default:
return ""
}
}
func outputFilesFromResult(result contracts.OutputResult) ([]contracts.OutputFile, error) {
out := make([]contracts.OutputFile, 0, len(result.Files))
for _, file := range result.Files {
if err := validateOutputFileName(file.Name); err != nil {
return nil, err
}
out = append(out, contracts.OutputFile{
Name: file.Name,
ContentType: file.ContentType,
Bytes: append([]byte(nil), file.Bytes...),
})
}
return out, nil
}
func validateOutputFileName(name string) error {
if strings.TrimSpace(name) == "" {
return fmt.Errorf("output file name must not be empty")
}
if strings.Contains(name, "\\") {
return fmt.Errorf("output file name %q must use slash-separated relative paths", name)
}
if path.IsAbs(name) {
return fmt.Errorf("output file name %q must be relative", name)
}
if strings.Contains(name, "..") {
return fmt.Errorf("output file name %q must not contain ..", name)
}
cleaned := path.Clean(name)
if cleaned == "." || cleaned != name {
return fmt.Errorf("output file name %q must be clean", name)
}
return nil
}
func cloneMetadata(metadata map[string]any) map[string]any {
if len(metadata) == 0 {
return nil
}
out := make(map[string]any, len(metadata))
for key, value := range metadata {
out[key] = value
}
return out
}
func cloneLLMProfiles(profiles []artifacts.LLMProfileManifest) []artifacts.LLMProfileManifest {
if len(profiles) == 0 {
return nil
}
return append([]artifacts.LLMProfileManifest(nil), profiles...)
}
func timePtr(t time.Time) *time.Time {
return &t
}
func pipelineUsesConfiguredValidators(pipeline ResolvedPipeline) bool {
for _, lane := range pipeline.ArtifactLanes {
if len(lane.Validators) > 0 {

View File

@@ -6,6 +6,7 @@ import (
"reflect"
"strings"
"testing"
"time"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
@@ -28,18 +29,17 @@ func TestNewAndDataTypes(t *testing.T) {
Metadata: map[string]any{"request": "test"},
}
output := RunOutput{
Manifest: artifacts.RunManifest{PipelineID: "pipeline-1"},
Approved: []artifacts.Artifact{{ExtractorKey: "extract-alpha"}},
Rejected: []artifacts.RejectedArtifact{{ValidatorName: "validator"}},
Warnings: []contracts.Warning{{ReasonCode: "note", Message: "message"}},
EncodedOutput: []byte(`{}`),
ContentType: "application/json",
Manifest: artifacts.RunManifest{PipelineID: "pipeline-1"},
Approved: []artifacts.Artifact{{ExtractorKey: "extract-alpha"}},
Rejected: []artifacts.RejectedArtifact{{ValidatorName: "validator"}},
Warnings: []contracts.Warning{{ReasonCode: "note", Message: "message"}},
OutputFiles: []contracts.OutputFile{{Name: "artifacts/generic.json", ContentType: "application/json", Bytes: []byte(`{}`)}},
}
if input.Pipeline.ID != "pipeline-1" || input.SourceID != "source-1" {
t.Fatalf("RunInput = %#v, want constructed fields", input)
}
if output.Manifest.PipelineID != "pipeline-1" || len(output.Approved) != 1 || len(output.Rejected) != 1 || len(output.Warnings) != 1 {
if output.Manifest.PipelineID != "pipeline-1" || len(output.Approved) != 1 || len(output.Rejected) != 1 || len(output.Warnings) != 1 || len(output.OutputFiles) != 1 {
t.Fatalf("RunOutput = %#v, want constructed fields", output)
}
}
@@ -604,11 +604,18 @@ func TestRunOutputEncoderReceivesManifestAndArtifacts(t *testing.T) {
t.Fatalf("Run() error = %v, want nil", err)
}
if output.ContentType != "application/json" {
t.Fatalf("ContentType = %q, want application/json", output.ContentType)
if len(output.OutputFiles) != 1 {
t.Fatalf("len(OutputFiles) = %d, want 1", len(output.OutputFiles))
}
if string(output.EncodedOutput) != `{"encoded":true}` {
t.Fatalf("EncodedOutput = %s, want encoded payload", output.EncodedOutput)
file := output.OutputFiles[0]
if file.Name != "artifacts/generic.json" {
t.Fatalf("OutputFiles[0].Name = %q, want artifacts/generic.json", file.Name)
}
if file.ContentType != "application/json" {
t.Fatalf("ContentType = %q, want application/json", file.ContentType)
}
if string(file.Bytes) != `{"encoded":true}` {
t.Fatalf("OutputFiles[0].Bytes = %s, want encoded payload", file.Bytes)
}
if len(modules.output.requests) != 1 {
t.Fatalf("len(output requests) = %d, want 1", len(modules.output.requests))
@@ -622,6 +629,35 @@ func TestRunOutputEncoderReceivesManifestAndArtifacts(t *testing.T) {
}
}
func TestRunRejectsUnsafeOutputFileNames(t *testing.T) {
tests := []struct {
name string
fileName string
}{
{name: "empty", fileName: ""},
{name: "absolute", fileName: "/tmp/output.json"},
{name: "parent", fileName: "artifacts/../manifest.json"},
{name: "backslash", fileName: `artifacts\manifest.json`},
{name: "unclean", fileName: "artifacts//manifest.json"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
modules := defaultRunnerModules()
modules.output.files = []contracts.OutputFile{
{Name: test.fileName, ContentType: "application/json", Bytes: []byte(`{}`)},
}
output, err := New(newRunnerRegistries(t, modules)).Run(context.Background(), RunInput{Pipeline: resolvedPipeline()})
assertRunError(t, err, "output file name")
if output.Manifest.ValidationStatus != "failed" {
t.Fatalf("ValidationStatus = %q, want failed", output.Manifest.ValidationStatus)
}
})
}
}
func TestRunReturnsFailedManifestWhenOutputEncoderFails(t *testing.T) {
modules := defaultRunnerModules()
modules.output.err = errors.New("encode failed")
@@ -632,6 +668,9 @@ func TestRunReturnsFailedManifestWhenOutputEncoderFails(t *testing.T) {
if output.Manifest.ValidationStatus != "failed" {
t.Fatalf("ValidationStatus = %q, want failed", output.Manifest.ValidationStatus)
}
if output.Manifest.CompletedAt == nil {
t.Fatal("CompletedAt = nil, want failed run completion timestamp")
}
if len(output.Approved) != 2 {
t.Fatalf("len(Approved) = %d, want partial approved output", len(output.Approved))
}
@@ -668,6 +707,76 @@ func TestRunManifestIncludesPipelineAndLaneDetails(t *testing.T) {
}
}
func TestRunManifestIncludesRunTimingAndLLMProfiles(t *testing.T) {
startedAt := time.Now().Add(-time.Minute).UTC()
profiles := []artifacts.LLMProfileManifest{
{ID: "default", Provider: "openai-compatible", Model: "model-a"},
}
output, err := New(newRunnerRegistries(t, nil)).Run(context.Background(), RunInput{
Pipeline: resolvedPipeline(),
RunID: "run-test",
StartedAt: startedAt,
LLMProfiles: profiles,
})
if err != nil {
t.Fatalf("Run() error = %v, want nil", err)
}
manifest := output.Manifest
if manifest.RunID != "run-test" {
t.Fatalf("RunID = %q, want run-test", manifest.RunID)
}
if manifest.StartedAt == nil || !manifest.StartedAt.Equal(startedAt) {
t.Fatalf("StartedAt = %v, want %s", manifest.StartedAt, startedAt)
}
if manifest.CompletedAt == nil || manifest.CompletedAt.Before(startedAt) {
t.Fatalf("CompletedAt = %v, want timestamp after start", manifest.CompletedAt)
}
if !reflect.DeepEqual(manifest.LLMProfiles, profiles) {
t.Fatalf("LLMProfiles = %#v, want %#v", manifest.LLMProfiles, profiles)
}
}
func TestRunManifestGeneratesRunIDAndTimestamps(t *testing.T) {
output, err := New(newRunnerRegistries(t, nil)).Run(context.Background(), RunInput{Pipeline: resolvedPipeline()})
if err != nil {
t.Fatalf("Run() error = %v, want nil", err)
}
if !strings.HasPrefix(output.Manifest.RunID, "run-") {
t.Fatalf("RunID = %q, want generated run ID", output.Manifest.RunID)
}
if output.Manifest.StartedAt == nil {
t.Fatal("StartedAt = nil, want generated timestamp")
}
if output.Manifest.CompletedAt == nil {
t.Fatal("CompletedAt = nil, want generated timestamp")
}
}
func TestRunManifestIncludesExtractorMetadata(t *testing.T) {
modules := defaultRunnerModules()
modules.extractors["extract-alpha"].manifestMetadata = map[string]any{
"prompt_id": "test.prompt",
"response_schema_name": "test_schema",
}
output, err := New(newRunnerRegistries(t, modules)).Run(context.Background(), RunInput{Pipeline: resolvedPipeline()})
if err != nil {
t.Fatalf("Run() error = %v, want nil", err)
}
lane := output.Manifest.ArtifactLanes[0]
extractorMetadata, ok := lane.Metadata["extractor"].(map[string]any)
if !ok {
t.Fatalf("lane metadata = %#v, want extractor metadata", lane.Metadata)
}
if extractorMetadata["prompt_id"] != "test.prompt" || extractorMetadata["response_schema_name"] != "test_schema" {
t.Fatalf("extractor metadata = %#v, want prompt and schema metadata", extractorMetadata)
}
}
func TestRunReturnsPartialOutputWhenLaterLaneFails(t *testing.T) {
modules := defaultRunnerModules()
modules.extractors["extract-beta"] = &runnerExtractor{key: "extract-beta", artifactType: "artifact", schemaVersion: "v1", err: errors.New("extract failed")}
@@ -783,7 +892,12 @@ func defaultRunnerModules() *runnerModules {
"configured": {name: "configured", decisions: approveAll},
"second-validator": {name: "second-validator", decisions: approveAll},
},
output: &runnerOutputEncoder{key: "output", bytes: []byte(`{"encoded":true}`), contentType: "application/json"},
output: &runnerOutputEncoder{
key: "output",
files: []contracts.OutputFile{
{Name: "artifacts/generic.json", ContentType: "application/json", Bytes: []byte(`{"encoded":true}`)},
},
},
}
}
@@ -885,17 +999,18 @@ func (chunker *runnerChunker) Chunk(ctx context.Context, req contracts.ChunkRequ
}
type runnerExtractor struct {
key string
artifactType string
schemaVersion string
candidates []artifacts.ArtifactCandidate
validators []contracts.Validator
warnings []contracts.Warning
err error
requests []contracts.ExtractionRequest
seenChunkIDs []string
seenLLMClients []contracts.StructuredLLMClient
seenMetadata []map[string]any
key string
artifactType string
schemaVersion string
manifestMetadata map[string]any
candidates []artifacts.ArtifactCandidate
validators []contracts.Validator
warnings []contracts.Warning
err error
requests []contracts.ExtractionRequest
seenChunkIDs []string
seenLLMClients []contracts.StructuredLLMClient
seenMetadata []map[string]any
}
func (extractor *runnerExtractor) Key() string {
@@ -910,6 +1025,10 @@ func (extractor *runnerExtractor) SchemaVersion() string {
return extractor.schemaVersion
}
func (extractor *runnerExtractor) ManifestMetadata() map[string]any {
return extractor.manifestMetadata
}
func (extractor *runnerExtractor) Validators() []contracts.Validator {
return extractor.validators
}
@@ -1019,12 +1138,11 @@ func (validator *runnerValidator) Validate(ctx context.Context, req contracts.Va
}
type runnerOutputEncoder struct {
key string
bytes []byte
contentType string
warnings []contracts.Warning
err error
requests []contracts.OutputRequest
key string
files []contracts.OutputFile
warnings []contracts.Warning
err error
requests []contracts.OutputRequest
}
func (encoder *runnerOutputEncoder) Key() string {
@@ -1034,9 +1152,8 @@ func (encoder *runnerOutputEncoder) Key() string {
func (encoder *runnerOutputEncoder) Encode(ctx context.Context, req contracts.OutputRequest) (contracts.OutputResult, error) {
encoder.requests = append(encoder.requests, req)
return contracts.OutputResult{
Bytes: encoder.bytes,
ContentType: encoder.contentType,
Warnings: encoder.warnings,
Files: encoder.files,
Warnings: encoder.warnings,
}, encoder.err
}

View File

@@ -36,10 +36,13 @@ func TestWalkingSkeletonFixture(t *testing.T) {
if err != nil {
t.Fatalf("Run() error = %v, want nil", err)
}
if output.ContentType != "application/json" {
t.Fatalf("ContentType = %q, want application/json", output.ContentType)
if len(output.OutputFiles) != 1 {
t.Fatalf("len(OutputFiles) = %d, want 1", len(output.OutputFiles))
}
assertStructuralJSONEqual(t, output.EncodedOutput, expectedBytes)
if output.OutputFiles[0].ContentType != "application/json" {
t.Fatalf("ContentType = %q, want application/json", output.OutputFiles[0].ContentType)
}
assertStructuralJSONEqual(t, output.OutputFiles[0].Bytes, expectedBytes)
if llmClient.calls != 2 {
t.Fatalf("LLM calls = %d, want chunk count 2", llmClient.calls)
}
@@ -135,7 +138,7 @@ func walkingSkeletonCatalog(t *testing.T) ModuleCatalog {
Stage: StageMerge,
Requires: []string{"fake_artifacts"},
}, func() (contracts.Merger, error) {
return AppendOrderMerger{}, nil
return walkingSkeletonMerger{}, nil
}); err != nil {
t.Fatalf("register append-order merger: %v", err)
}
@@ -143,7 +146,7 @@ func walkingSkeletonCatalog(t *testing.T) ModuleCatalog {
Key: DefaultNormalizeModule,
Stage: StageNormalize,
}, func() (contracts.Normalizer, error) {
return NoopNormalizer{}, nil
return walkingSkeletonNormalizer{}, nil
}); err != nil {
t.Fatalf("register no-op normalizer: %v", err)
}
@@ -309,6 +312,30 @@ func (client *walkingSkeletonLLMClient) CompleteStructured(ctx context.Context,
}, nil
}
type walkingSkeletonMerger struct{}
func (merger walkingSkeletonMerger) Key() string {
return DefaultMergeModule
}
func (merger walkingSkeletonMerger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) {
var candidates []artifacts.ArtifactCandidate
for _, chunkArtifacts := range req.ChunkArtifacts {
candidates = append(candidates, chunkArtifacts.Candidates...)
}
return contracts.MergeResult{Candidates: candidates}, nil
}
type walkingSkeletonNormalizer struct{}
func (normalizer walkingSkeletonNormalizer) Key() string {
return DefaultNormalizeModule
}
func (normalizer walkingSkeletonNormalizer) Normalize(ctx context.Context, req contracts.NormalizeRequest) (contracts.NormalizeResult, error) {
return contracts.NormalizeResult{Candidates: req.Candidates}, nil
}
type walkingSkeletonOutput struct{}
func (output walkingSkeletonOutput) Key() string {
@@ -332,8 +359,9 @@ func (output walkingSkeletonOutput) Encode(ctx context.Context, req contracts.Ou
return contracts.OutputResult{}, err
}
return contracts.OutputResult{
Bytes: encoded,
ContentType: "application/json",
Files: []contracts.OutputFile{
{Name: "output.json", ContentType: "application/json", Bytes: encoded},
},
}, nil
}

View File

@@ -5,6 +5,7 @@ import (
"embed"
"encoding/hex"
"fmt"
"io/fs"
"path"
"sort"
"strings"
@@ -40,21 +41,31 @@ func (m Metadata) DiagnosticsMap() map[string]any {
}
}
type definition struct {
id string
version string
embeddedDir string
systemPath string
userPath string
// Definition identifies a caller-owned system/user prompt bundle.
type Definition struct {
PromptID string
Version string
EmbeddedPath string
SystemPath string
UserPath string
}
type compiledPrompt struct {
// Bundle is a compiled system/user prompt pair.
type Bundle struct {
systemTmpl *template.Template
userTmpl *template.Template
metadata Metadata
}
var promptRegistry map[string]compiledPrompt
// Metadata returns metadata for the compiled prompt bundle.
func (b *Bundle) Metadata() Metadata {
if b == nil {
return Metadata{}
}
return b.metadata
}
var promptRegistry map[string]*Bundle
var sharedHardening string
func init() {
@@ -64,23 +75,23 @@ func init() {
panic(err)
}
defs := []definition{
defs := []Definition{
{
id: TestGenericPromptID,
version: VersionV1,
embeddedDir: "assets/test/generic",
systemPath: "assets/test/generic/system.md",
userPath: "assets/test/generic/user.md",
PromptID: TestGenericPromptID,
Version: VersionV1,
EmbeddedPath: "assets/test/generic",
SystemPath: "assets/test/generic/system.md",
UserPath: "assets/test/generic/user.md",
},
}
promptRegistry = make(map[string]compiledPrompt, len(defs))
promptRegistry = make(map[string]*Bundle, len(defs))
for _, def := range defs {
compiled, compileErr := compilePrompt(def)
compiled, compileErr := LoadBundle(embeddedAssets, def)
if compileErr != nil {
panic(compileErr)
}
promptRegistry[def.id] = compiled
promptRegistry[compiled.metadata.PromptID] = compiled
}
}
@@ -130,51 +141,68 @@ func readAsset(assetPath string) (string, error) {
return string(content), nil
}
func compilePrompt(def definition) (compiledPrompt, error) {
if strings.TrimSpace(def.id) == "" {
return compiledPrompt{}, fmt.Errorf("prompt id must not be empty")
// LoadBundle compiles a system/user prompt bundle from a caller-owned filesystem.
func LoadBundle(fsys fs.FS, def Definition) (*Bundle, error) {
promptID := strings.TrimSpace(def.PromptID)
version := strings.TrimSpace(def.Version)
embeddedPath := strings.TrimSpace(def.EmbeddedPath)
systemPath := strings.TrimSpace(def.SystemPath)
userPath := strings.TrimSpace(def.UserPath)
if promptID == "" {
return nil, fmt.Errorf("prompt id must not be empty")
}
if strings.TrimSpace(def.version) == "" {
return compiledPrompt{}, fmt.Errorf("prompt version must not be empty")
if version == "" {
return nil, fmt.Errorf("prompt version must not be empty")
}
if strings.TrimSpace(def.embeddedDir) == "" {
return compiledPrompt{}, fmt.Errorf("prompt embedded path must not be empty")
if embeddedPath == "" {
return nil, fmt.Errorf("prompt embedded path must not be empty")
}
systemSource, err := readAsset(def.systemPath)
systemSource, err := readPromptAsset(fsys, systemPath)
if err != nil {
return compiledPrompt{}, err
return nil, err
}
userSource, err := readAsset(def.userPath)
userSource, err := readPromptAsset(fsys, userPath)
if err != nil {
return compiledPrompt{}, err
return nil, err
}
funcs := template.FuncMap{
"hardening": func() string { return sharedHardening },
}
systemTmpl, err := template.New(path.Base(def.systemPath)).Option("missingkey=error").Funcs(funcs).Parse(systemSource)
systemTmpl, err := template.New(path.Base(systemPath)).Option("missingkey=error").Funcs(funcs).Parse(systemSource)
if err != nil {
return compiledPrompt{}, fmt.Errorf("parse embedded system prompt %q: %w", def.systemPath, err)
return nil, fmt.Errorf("parse embedded system prompt %q: %w", systemPath, err)
}
userTmpl, err := template.New(path.Base(def.userPath)).Option("missingkey=error").Funcs(funcs).Parse(userSource)
userTmpl, err := template.New(path.Base(userPath)).Option("missingkey=error").Funcs(funcs).Parse(userSource)
if err != nil {
return compiledPrompt{}, fmt.Errorf("parse embedded user prompt %q: %w", def.userPath, err)
return nil, fmt.Errorf("parse embedded user prompt %q: %w", userPath, err)
}
hashInput := systemSource + "\n\n" + userSource
hash := sha256.Sum256([]byte(hashInput))
metadata := Metadata{
PromptID: strings.TrimSpace(def.id),
PromptVersion: strings.TrimSpace(def.version),
PromptID: promptID,
PromptVersion: version,
PromptSource: SourceBuiltin,
EmbeddedPath: strings.TrimSpace(def.embeddedDir),
EmbeddedPath: embeddedPath,
SHA256: "sha256:" + hex.EncodeToString(hash[:]),
}
return compiledPrompt{
return &Bundle{
systemTmpl: systemTmpl,
userTmpl: userTmpl,
metadata: metadata,
}, nil
}
func readPromptAsset(fsys fs.FS, assetPath string) (string, error) {
if strings.TrimSpace(assetPath) == "" {
return "", fmt.Errorf("prompt asset path must not be empty")
}
content, err := fs.ReadFile(fsys, assetPath)
if err != nil {
return "", fmt.Errorf("read embedded prompt asset %q: %w", assetPath, err)
}
return string(content), nil
}

View File

@@ -6,26 +6,37 @@ import (
"testing"
)
func TestLookupMetadataSucceedsForGenericPrompt(t *testing.T) {
metadata, ok := LookupMetadata(TestGenericPromptID)
if !ok {
t.Fatalf("expected metadata for %q", TestGenericPromptID)
func TestLookupMetadataSucceedsForRegisteredPrompts(t *testing.T) {
tests := []struct {
promptID string
embeddedPath string
}{
{promptID: TestGenericPromptID, embeddedPath: "assets/test/generic"},
}
if metadata.PromptID != TestGenericPromptID {
t.Fatalf("unexpected prompt ID: %q", metadata.PromptID)
}
if metadata.PromptVersion != VersionV1 {
t.Fatalf("unexpected prompt version: %q", metadata.PromptVersion)
}
if metadata.PromptSource != SourceBuiltin {
t.Fatalf("unexpected prompt source: %q", metadata.PromptSource)
}
if metadata.EmbeddedPath != "assets/test/generic" {
t.Fatalf("unexpected embedded path: %q", metadata.EmbeddedPath)
}
if !strings.HasPrefix(metadata.SHA256, "sha256:") {
t.Fatalf("expected prefixed hash, got %q", metadata.SHA256)
for _, tc := range tests {
t.Run(tc.promptID, func(t *testing.T) {
metadata, ok := LookupMetadata(tc.promptID)
if !ok {
t.Fatalf("expected metadata for %q", tc.promptID)
}
if metadata.PromptID != tc.promptID {
t.Fatalf("unexpected prompt ID: %q", metadata.PromptID)
}
if metadata.PromptVersion != VersionV1 {
t.Fatalf("unexpected prompt version: %q", metadata.PromptVersion)
}
if metadata.PromptSource != SourceBuiltin {
t.Fatalf("unexpected prompt source: %q", metadata.PromptSource)
}
if metadata.EmbeddedPath != tc.embeddedPath {
t.Fatalf("unexpected embedded path: %q", metadata.EmbeddedPath)
}
if !strings.HasPrefix(metadata.SHA256, "sha256:") {
t.Fatalf("expected prefixed hash, got %q", metadata.SHA256)
}
})
}
}
@@ -52,12 +63,17 @@ func TestRegisteredMetadataSortedByPromptID(t *testing.T) {
}
ids := make([]string, len(registered))
seen := make(map[string]bool, len(registered))
for i, metadata := range registered {
ids[i] = metadata.PromptID
seen[metadata.PromptID] = true
}
if !sort.StringsAreSorted(ids) {
t.Fatalf("expected sorted prompt IDs, got %v", ids)
}
if !seen[TestGenericPromptID] {
t.Fatalf("registered prompt IDs = %v, want %q", ids, TestGenericPromptID)
}
}
func TestHardeningTextAvailable(t *testing.T) {

View File

@@ -13,16 +13,23 @@ func RenderUserSystem(promptID string, data any) (system string, user string, me
if !ok {
return "", "", Metadata{}, fmt.Errorf("unknown prompt id %q", promptID)
}
return compiled.RenderUserSystem(data)
}
// RenderUserSystem renders the bundle's system and user prompts.
func (b *Bundle) RenderUserSystem(data any) (system string, user string, metadata Metadata, err error) {
if b == nil {
return "", "", Metadata{}, fmt.Errorf("prompt bundle must not be nil")
}
var systemBuf bytes.Buffer
if err := compiled.systemTmpl.Execute(&systemBuf, data); err != nil {
return "", "", Metadata{}, fmt.Errorf("render system prompt %q: %w", trimmedID, err)
if err := b.systemTmpl.Execute(&systemBuf, data); err != nil {
return "", "", Metadata{}, fmt.Errorf("render system prompt %q: %w", b.metadata.PromptID, err)
}
var userBuf bytes.Buffer
if err := compiled.userTmpl.Execute(&userBuf, data); err != nil {
return "", "", Metadata{}, fmt.Errorf("render user prompt %q: %w", trimmedID, err)
if err := b.userTmpl.Execute(&userBuf, data); err != nil {
return "", "", Metadata{}, fmt.Errorf("render user prompt %q: %w", b.metadata.PromptID, err)
}
return strings.TrimSpace(systemBuf.String()), strings.TrimSpace(userBuf.String()), compiled.metadata, nil
return strings.TrimSpace(systemBuf.String()), strings.TrimSpace(userBuf.String()), b.metadata, nil
}

View File

@@ -0,0 +1,241 @@
package generic
import (
"context"
"encoding/json"
"fmt"
"math"
"strconv"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const Key = "generic"
const (
defaultMaxUnits = 50
defaultOverlapUnits = 0
)
var _ contracts.Chunker = (*Chunker)(nil)
type Chunker struct{}
func New() *Chunker {
return &Chunker{}
}
func (c *Chunker) Key() string {
return Key
}
func (c *Chunker) Chunk(ctx context.Context, req contracts.ChunkRequest) (contracts.ChunkResult, error) {
if c == nil {
return contracts.ChunkResult{}, chunkerErrorf("chunker must not be nil")
}
if ctx == nil {
return contracts.ChunkResult{}, chunkerErrorf("context must not be nil")
}
if err := ctx.Err(); err != nil {
return contracts.ChunkResult{}, chunkerErrorf("context error before chunking: %w", err)
}
if req.Source == nil {
return contracts.ChunkResult{}, chunkerErrorf("source must not be nil")
}
if len(req.Source.Units) == 0 {
return contracts.ChunkResult{}, chunkerErrorf("source units must not be empty")
}
if err := source.ValidateDocument(req.Source); err != nil {
return contracts.ChunkResult{}, chunkerErrorf("validate source document: %w", err)
}
opts, err := chunkOptionsFrom(req.Options)
if err != nil {
return contracts.ChunkResult{}, err
}
step := opts.maxUnits - opts.overlapUnits
chunks := make([]contracts.SourceChunk, 0, (len(req.Source.Units)+step-1)/step)
for start := 0; start < len(req.Source.Units); start += step {
end := start + opts.maxUnits
if end > len(req.Source.Units) {
end = len(req.Source.Units)
}
units := cloneUnits(req.Source.Units[start:end])
chunks = append(chunks, contracts.SourceChunk{
ID: fmt.Sprintf("chunk-%06d", len(chunks)+1),
SourceID: req.Source.ID,
Index: len(chunks),
Units: units,
Metadata: map[string]any{
"start_unit_id": units[0].ID,
"end_unit_id": units[len(units)-1].ID,
"unit_count": len(units),
},
})
if end == len(req.Source.Units) {
break
}
}
return contracts.ChunkResult{Chunks: chunks}, nil
}
func ModuleSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageChunk,
Provides: []string{"chunks"},
}
}
func Register(registry *pipeline.ChunkerRegistry) error {
return registry.RegisterWithSpec(ModuleSpec(), func() (contracts.Chunker, error) {
return New(), nil
})
}
type chunkOptions struct {
maxUnits int
overlapUnits int
}
func chunkOptionsFrom(options map[string]any) (chunkOptions, error) {
opts := chunkOptions{
maxUnits: defaultMaxUnits,
overlapUnits: defaultOverlapUnits,
}
var err error
if value, ok := options["max_units"]; ok {
opts.maxUnits, err = positiveIntOption("max_units", value)
if err != nil {
return chunkOptions{}, err
}
}
if value, ok := options["overlap_units"]; ok {
opts.overlapUnits, err = nonNegativeIntOption("overlap_units", value)
if err != nil {
return chunkOptions{}, err
}
}
if opts.overlapUnits >= opts.maxUnits {
return chunkOptions{}, chunkerErrorf("overlap_units must be less than max_units")
}
return opts, nil
}
func positiveIntOption(name string, value any) (int, error) {
got, err := intOption(name, value)
if err != nil {
return 0, err
}
if got <= 0 {
return 0, chunkerErrorf("%s must be positive", name)
}
return got, nil
}
func nonNegativeIntOption(name string, value any) (int, error) {
got, err := intOption(name, value)
if err != nil {
return 0, err
}
if got < 0 {
return 0, chunkerErrorf("%s must be non-negative", name)
}
return got, nil
}
func intOption(name string, value any) (int, error) {
switch typed := value.(type) {
case int:
return typed, nil
case int8:
return int(typed), nil
case int16:
return int(typed), nil
case int32:
return int(typed), nil
case int64:
if typed > maxInt() || typed < minInt() {
return 0, chunkerErrorf("%s is outside supported integer range", name)
}
return int(typed), nil
case uint:
if uint64(typed) > uint64(maxInt()) {
return 0, chunkerErrorf("%s is outside supported integer range", name)
}
return int(typed), nil
case uint8:
return int(typed), nil
case uint16:
return int(typed), nil
case uint32:
if uint64(typed) > uint64(maxInt()) {
return 0, chunkerErrorf("%s is outside supported integer range", name)
}
return int(typed), nil
case uint64:
if typed > uint64(maxInt()) {
return 0, chunkerErrorf("%s is outside supported integer range", name)
}
return int(typed), nil
case float64:
if typed != math.Trunc(typed) {
return 0, chunkerErrorf("%s must be an integer", name)
}
if typed > float64(maxInt()) || typed < float64(minInt()) {
return 0, chunkerErrorf("%s is outside supported integer range", name)
}
return int(typed), nil
case json.Number:
parsed, err := typed.Int64()
if err != nil {
return 0, chunkerErrorf("%s must be an integer", name)
}
if parsed > maxInt() || parsed < minInt() {
return 0, chunkerErrorf("%s is outside supported integer range", name)
}
return int(parsed), nil
default:
return 0, chunkerErrorf("%s must be an integer", name)
}
}
func maxInt() int64 {
return int64(1<<(strconv.IntSize-1) - 1)
}
func minInt() int64 {
return -maxInt() - 1
}
func cloneUnits(units []source.SourceUnit) []source.SourceUnit {
out := make([]source.SourceUnit, 0, len(units))
for _, unit := range units {
out = append(out, source.SourceUnit{
ID: unit.ID,
Kind: unit.Kind,
Text: unit.Text,
Metadata: cloneMetadata(unit.Metadata),
})
}
return out
}
func cloneMetadata(metadata map[string]any) map[string]any {
if len(metadata) == 0 {
return nil
}
out := make(map[string]any, len(metadata))
for key, value := range metadata {
out[key] = value
}
return out
}
func chunkerErrorf(format string, args ...any) error {
return fmt.Errorf("generic chunker: "+format, args...)
}

View File

@@ -0,0 +1,213 @@
package generic
import (
"context"
"encoding/json"
"fmt"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestModuleSpecAndRegister(t *testing.T) {
want := pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageChunk,
Provides: []string{"chunks"},
}
if got := ModuleSpec(); !reflect.DeepEqual(got, want) {
t.Fatalf("ModuleSpec() = %#v, want %#v", got, want)
}
registry := pipeline.NewChunkerRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
spec, ok := registry.Spec(Key)
if !ok {
t.Fatalf("Spec(%q) ok = false, want true", Key)
}
if !reflect.DeepEqual(spec, want) {
t.Fatalf("registered spec = %#v, want %#v", spec, want)
}
chunker, err := registry.Build(Key)
if err != nil {
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
}
if chunker.Key() != Key {
t.Fatalf("Key() = %q, want %q", chunker.Key(), Key)
}
}
func TestChunkUsesDefaultsForSingleChunk(t *testing.T) {
result, err := New().Chunk(context.Background(), contracts.ChunkRequest{Source: testSource(3), Options: nil})
if err != nil {
t.Fatalf("Chunk() error = %v, want nil", err)
}
if got := chunkIDs(result.Chunks); !reflect.DeepEqual(got, []string{"chunk-000001"}) {
t.Fatalf("chunk IDs = %#v, want one stable ID", got)
}
chunk := result.Chunks[0]
if chunk.Index != 0 || chunk.SourceID != "source-1" {
t.Fatalf("chunk = %#v, want source and index fields", chunk)
}
if got := unitIDs(chunk.Units); !reflect.DeepEqual(got, []string{"u001", "u002", "u003"}) {
t.Fatalf("unit IDs = %#v, want all units", got)
}
if chunk.Metadata["start_unit_id"] != "u001" || chunk.Metadata["end_unit_id"] != "u003" || chunk.Metadata["unit_count"] != 3 {
t.Fatalf("metadata = %#v, want chunk bounds", chunk.Metadata)
}
}
func TestChunkExactBoundaries(t *testing.T) {
result, err := New().Chunk(context.Background(), contracts.ChunkRequest{
Source: testSource(6),
Options: map[string]any{"max_units": 2},
})
if err != nil {
t.Fatalf("Chunk() error = %v, want nil", err)
}
if got := chunkIDs(result.Chunks); !reflect.DeepEqual(got, []string{"chunk-000001", "chunk-000002", "chunk-000003"}) {
t.Fatalf("chunk IDs = %#v, want stable IDs", got)
}
gotUnits := [][]string{unitIDs(result.Chunks[0].Units), unitIDs(result.Chunks[1].Units), unitIDs(result.Chunks[2].Units)}
wantUnits := [][]string{{"u001", "u002"}, {"u003", "u004"}, {"u005", "u006"}}
if !reflect.DeepEqual(gotUnits, wantUnits) {
t.Fatalf("chunk units = %#v, want %#v", gotUnits, wantUnits)
}
}
func TestChunkOverlap(t *testing.T) {
result, err := New().Chunk(context.Background(), contracts.ChunkRequest{
Source: testSource(7),
Options: map[string]any{"max_units": 3, "overlap_units": 1},
})
if err != nil {
t.Fatalf("Chunk() error = %v, want nil", err)
}
gotUnits := make([][]string, 0, len(result.Chunks))
for _, chunk := range result.Chunks {
gotUnits = append(gotUnits, unitIDs(chunk.Units))
}
wantUnits := [][]string{{"u001", "u002", "u003"}, {"u003", "u004", "u005"}, {"u005", "u006", "u007"}}
if !reflect.DeepEqual(gotUnits, wantUnits) {
t.Fatalf("chunk units = %#v, want %#v", gotUnits, wantUnits)
}
}
func TestChunkRejectsInvalidOptions(t *testing.T) {
tests := []struct {
name string
options map[string]any
want string
}{
{name: "max wrong type", options: map[string]any{"max_units": "2"}, want: "max_units"},
{name: "max fractional", options: map[string]any{"max_units": 1.5}, want: "integer"},
{name: "max zero", options: map[string]any{"max_units": 0}, want: "positive"},
{name: "overlap negative", options: map[string]any{"overlap_units": -1}, want: "non-negative"},
{name: "overlap too large", options: map[string]any{"max_units": 2, "overlap_units": 2}, want: "less than"},
{name: "json number", options: map[string]any{"max_units": json.Number("bad")}, want: "integer"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
_, err := New().Chunk(context.Background(), contracts.ChunkRequest{
Source: testSource(3),
Options: test.options,
})
if err == nil {
t.Fatal("Chunk() error = nil, want error")
}
if !strings.Contains(err.Error(), "generic chunker") || !strings.Contains(err.Error(), test.want) {
t.Fatalf("Chunk() error = %q, want module context and %q", err.Error(), test.want)
}
})
}
}
func TestChunkRejectsEmptySource(t *testing.T) {
doc := testSource(1)
doc.Units = nil
_, err := New().Chunk(context.Background(), contracts.ChunkRequest{Source: doc})
if err == nil {
t.Fatal("Chunk() error = nil, want empty source error")
}
if !strings.Contains(err.Error(), "generic chunker") || !strings.Contains(err.Error(), "units") {
t.Fatalf("Chunk() error = %q, want empty source context", err.Error())
}
}
func TestChunkDefensivelyCopiesUnits(t *testing.T) {
doc := testSource(2)
result, err := New().Chunk(context.Background(), contracts.ChunkRequest{
Source: doc,
Options: map[string]any{"max_units": 1},
})
if err != nil {
t.Fatalf("Chunk() error = %v, want nil", err)
}
if len(result.Chunks) != 2 {
t.Fatalf("len(Chunks) = %d, want 2", len(result.Chunks))
}
doc.Units[0].ID = "changed"
doc.Units[0].Metadata["speaker"] = "changed"
if result.Chunks[0].Units[0].ID != "u001" {
t.Fatalf("chunk unit ID changed after source mutation: %#v", result.Chunks[0].Units[0])
}
if result.Chunks[0].Units[0].Metadata["speaker"] != "speaker-001" {
t.Fatalf("chunk unit metadata changed after source mutation: %#v", result.Chunks[0].Units[0].Metadata)
}
}
func testSource(count int) *source.SourceDocument {
units := make([]source.SourceUnit, 0, count)
for i := 1; i <= count; i++ {
id := "u" + zeroPad3(i)
units = append(units, source.SourceUnit{
ID: id,
Kind: "unit",
Text: "Text for " + id,
Metadata: map[string]any{
"speaker": "speaker-" + zeroPad3(i),
},
})
}
return &source.SourceDocument{
ID: "source-1",
Kind: "document",
Format: "text/plain",
Digest: "sha256:source",
Units: units,
}
}
func zeroPad3(value int) string {
return fmt.Sprintf("%03d", value)
}
func chunkIDs(chunks []contracts.SourceChunk) []string {
ids := make([]string, 0, len(chunks))
for _, chunk := range chunks {
ids = append(ids, chunk.ID)
}
return ids
}
func unitIDs(units []source.SourceUnit) []string {
ids := make([]string, 0, len(units))
for _, unit := range units {
ids = append(ids, unit.ID)
}
return ids
}

View File

@@ -0,0 +1,6 @@
package spells
import "embed"
//go:embed assets/prompts/*.md assets/schemas/*.json
var embeddedAssets embed.FS

View File

@@ -0,0 +1,9 @@
You extract D&D spell-cast artifacts from source units.
{{ hardening }}
Extract only spell casts that are supported by the provided source text. Do not
infer spells from general D&D knowledge or from table chatter that does not
identify a spell being cast.
Source references must use the source-unit IDs exactly as provided.

View File

@@ -0,0 +1,21 @@
Source document ID: {{ .SourceID }}
{{ if .HasChunk }}
Chunk ID: {{ .ChunkID }}
Chunk index: {{ .ChunkIndex }}
{{ end }}
Source units:
{{ range .Units }}
- Unit ID: {{ .ID }}
Text: {{ .Text }}
{{ if .Metadata }}
Metadata:
{{ range .Metadata }}
- {{ .Key }}: {{ .Value }}
{{ end }}
{{ end }}
{{ end }}
Return only D&D spell-cast artifacts. For each spell cast, identify the in-world
caster, spell name, effect, narrative description, and source references using
source_id, start_unit_id, and end_unit_id.

View File

@@ -0,0 +1,64 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.spells",
"type": "object",
"additionalProperties": false,
"required": ["spell_casts"],
"properties": {
"spell_casts": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"caster",
"spell",
"effect",
"narrative_description",
"source_refs"
],
"properties": {
"caster": {
"type": "string",
"minLength": 1
},
"spell": {
"type": "string",
"minLength": 1
},
"effect": {
"type": "string",
"minLength": 1
},
"narrative_description": {
"type": "string",
"minLength": 1
},
"source_refs": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"additionalProperties": false,
"required": ["source_id", "start_unit_id", "end_unit_id"],
"properties": {
"source_id": {
"type": "string",
"minLength": 1
},
"start_unit_id": {
"type": "string",
"minLength": 1
},
"end_unit_id": {
"type": "string",
"minLength": 1
}
}
}
}
}
}
}
}
}

View File

@@ -0,0 +1,279 @@
package spells
import (
"context"
"os"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/input/seriatim"
"gitea.maximumdirect.net/eric/notarius/internal/modules/merge/appendorder"
"gitea.maximumdirect.net/eric/notarius/internal/modules/normalize/noop"
)
func TestPipelineConfigLoadsAndResolvesWithDNDSpellsExtractor(t *testing.T) {
data, err := os.ReadFile("testdata/pipeline.yml")
if err != nil {
t.Fatalf("ReadFile(pipeline.yml) error = %v, want nil", err)
}
fileCfg, err := config.ParseFileConfigYAML(data)
if err != nil {
t.Fatalf("ParseFileConfigYAML() error = %v, want nil", err)
}
cfg := config.Default()
if err := cfg.ApplyFileConfig(fileCfg); err != nil {
t.Fatalf("ApplyFileConfig() error = %v, want nil", err)
}
resolved, err := cfg.Resolve(config.ResolveInput{
PipelineID: "dnd-spells-fixture",
Catalog: dndSpellsTestCatalog(t, dndSpellsCatalogSpecs{}),
})
if err != nil {
t.Fatalf("Resolve() error = %v, want nil", err)
}
if len(resolved.ResolvedPipeline.ArtifactLanes) != 1 {
t.Fatalf("len(ArtifactLanes) = %d, want 1", len(resolved.ResolvedPipeline.ArtifactLanes))
}
lane := resolved.ResolvedPipeline.ArtifactLanes[0]
if lane.ID != "spells" {
t.Fatalf("lane ID = %q, want spells", lane.ID)
}
if lane.Extract.Module != Key {
t.Fatalf("extract module = %q, want %q", lane.Extract.Module, Key)
}
if resolved.ResolvedPipeline.Digest == "" {
t.Fatal("resolved digest is empty")
}
again, err := cfg.Resolve(config.ResolveInput{
PipelineID: "dnd-spells-fixture",
Catalog: dndSpellsTestCatalog(t, dndSpellsCatalogSpecs{}),
})
if err != nil {
t.Fatalf("second Resolve() error = %v, want nil", err)
}
if resolved.ResolvedPipeline.Digest != again.ResolvedPipeline.Digest {
t.Fatalf("resolved digest = %q, second digest = %q; want stable digest", resolved.ResolvedPipeline.Digest, again.ResolvedPipeline.Digest)
}
}
func TestPipelineConfigRejectsMissingTranscriptCapabilityForDNDSpells(t *testing.T) {
inputSpec := seriatim.ModuleSpec()
inputSpec.Provides = withoutCapability(inputSpec.Provides, "source.transcript")
chunkSpec := dndSpellsChunkerSpec()
chunkSpec.Requires = nil
_, err := loadDNDSpellsPipelineConfig(t).Resolve(config.ResolveInput{
PipelineID: "dnd-spells-fixture",
Catalog: dndSpellsTestCatalog(t, dndSpellsCatalogSpecs{
input: inputSpec,
chunk: chunkSpec,
}),
})
if err == nil {
t.Fatal("Resolve() error = nil, want missing capability error")
}
if !strings.Contains(err.Error(), "missing capability") ||
!strings.Contains(err.Error(), "source.transcript") ||
!strings.Contains(err.Error(), Key) {
t.Fatalf("Resolve() error = %q, want dnd/spells missing source.transcript capability", err.Error())
}
}
func TestPipelineConfigRejectsMissingSpellCastsCapabilityForAppendOrder(t *testing.T) {
extractorSpec := ModuleSpec()
extractorSpec.Provides = withoutCapability(extractorSpec.Provides, "dnd.spell_casts")
_, err := loadDNDSpellsPipelineConfig(t).Resolve(config.ResolveInput{
PipelineID: "dnd-spells-fixture",
Catalog: dndSpellsTestCatalog(t, dndSpellsCatalogSpecs{
extractor: extractorSpec,
}),
})
if err == nil {
t.Fatal("Resolve() error = nil, want missing capability error")
}
if !strings.Contains(err.Error(), "missing capability") ||
!strings.Contains(err.Error(), "dnd.spell_casts") ||
!strings.Contains(err.Error(), pipeline.DefaultMergeModule) {
t.Fatalf("Resolve() error = %q, want appendorder missing dnd.spell_casts capability", err.Error())
}
}
func TestPipelineConfigRejectsUnknownLaneSelection(t *testing.T) {
_, err := loadDNDSpellsPipelineConfig(t).Resolve(config.ResolveInput{
PipelineID: "dnd-spells-fixture",
Only: []string{"missing"},
Catalog: dndSpellsTestCatalog(t, dndSpellsCatalogSpecs{}),
})
if err == nil {
t.Fatal("Resolve() error = nil, want unknown lane error")
}
if !strings.Contains(err.Error(), "selected artifact lane") || !strings.Contains(err.Error(), "missing") {
t.Fatalf("Resolve() error = %q, want unknown lane context", err.Error())
}
}
func loadDNDSpellsPipelineConfig(t *testing.T) config.Config {
t.Helper()
data, err := os.ReadFile("testdata/pipeline.yml")
if err != nil {
t.Fatalf("ReadFile(pipeline.yml) error = %v, want nil", err)
}
fileCfg, err := config.ParseFileConfigYAML(data)
if err != nil {
t.Fatalf("ParseFileConfigYAML() error = %v, want nil", err)
}
cfg := config.Default()
if err := cfg.ApplyFileConfig(fileCfg); err != nil {
t.Fatalf("ApplyFileConfig() error = %v, want nil", err)
}
return cfg
}
type dndSpellsCatalogSpecs struct {
input pipeline.ModuleSpec
chunk pipeline.ModuleSpec
extractor pipeline.ModuleSpec
}
func dndSpellsTestCatalog(t *testing.T, specs dndSpellsCatalogSpecs) pipeline.ModuleCatalog {
t.Helper()
inputs := pipeline.NewInputAdapterRegistry()
chunkers := pipeline.NewChunkerRegistry()
extractors := pipeline.NewExtractorRegistry()
mergers := pipeline.NewMergerRegistry()
normalizers := pipeline.NewNormalizerRegistry()
outputs := pipeline.NewOutputEncoderRegistry()
if specs.input.Key == "" {
if err := seriatim.Register(inputs); err != nil {
t.Fatalf("register seriatim input: %v", err)
}
} else if err := inputs.RegisterWithSpec(specs.input, func() (contracts.InputAdapter, error) {
return seriatim.New(), nil
}); err != nil {
t.Fatalf("register seriatim input override: %v", err)
}
chunkSpec := specs.chunk
if chunkSpec.Key == "" {
chunkSpec = dndSpellsChunkerSpec()
}
if err := chunkers.RegisterWithSpec(chunkSpec, func() (contracts.Chunker, error) {
return dndSpellsChunker{}, nil
}); err != nil {
t.Fatalf("register chunker: %v", err)
}
if specs.extractor.Key == "" {
if err := Register(extractors); err != nil {
t.Fatalf("register dnd spells extractor: %v", err)
}
} else if err := extractors.RegisterWithSpec(specs.extractor, func() (contracts.Extractor, error) {
return New(), nil
}); err != nil {
t.Fatalf("register dnd spells extractor override: %v", err)
}
if err := mergers.RegisterWithSpec(pipeline.ModuleSpec{
Key: pipeline.DefaultMergeModule,
Stage: pipeline.StageMerge,
Requires: []string{"dnd.spell_casts"},
}, func() (contracts.Merger, error) {
return appendorder.New(), nil
}); err != nil {
t.Fatalf("register merger: %v", err)
}
if err := normalizers.RegisterWithSpec(pipeline.ModuleSpec{
Key: pipeline.DefaultNormalizeModule,
Stage: pipeline.StageNormalize,
}, func() (contracts.Normalizer, error) {
return noop.New(), nil
}); err != nil {
t.Fatalf("register normalizer: %v", err)
}
if err := outputs.RegisterWithSpec(pipeline.ModuleSpec{
Key: pipeline.DefaultOutputModule,
Stage: pipeline.StageOutput,
}, func() (contracts.OutputEncoder, error) {
return dndSpellsOutput{}, nil
}); err != nil {
t.Fatalf("register output: %v", err)
}
return pipeline.ModuleCatalog{
Inputs: inputs,
Chunkers: chunkers,
Extractors: extractors,
Mergers: mergers,
Normalizers: normalizers,
Outputs: outputs,
}
}
func dndSpellsChunkerSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{
Key: "fake/chunk",
Stage: pipeline.StageChunk,
Requires: []string{"source.transcript"},
Provides: []string{"chunks"},
}
}
type dndSpellsChunker struct{}
func (dndSpellsChunker) Key() string {
return "fake/chunk"
}
func (dndSpellsChunker) Chunk(ctx context.Context, req contracts.ChunkRequest) (contracts.ChunkResult, error) {
return contracts.ChunkResult{
Chunks: []contracts.SourceChunk{
{
ID: req.Source.ID + ":chunk:0",
SourceID: req.Source.ID,
Index: 0,
Units: append([]source.SourceUnit(nil), req.Source.Units...),
},
},
}, nil
}
type dndSpellsOutput struct{}
func (dndSpellsOutput) Key() string {
return pipeline.DefaultOutputModule
}
func (dndSpellsOutput) Encode(ctx context.Context, req contracts.OutputRequest) (contracts.OutputResult, error) {
return contracts.OutputResult{
Files: []contracts.OutputFile{
{Name: "output.json", ContentType: "application/json", Bytes: []byte(`{"encoded":true}`)},
},
}, nil
}
func withoutCapability(capabilities []string, capability string) []string {
filtered := make([]string, 0, len(capabilities))
for _, candidate := range capabilities {
if candidate != capability {
filtered = append(filtered, candidate)
}
}
return filtered
}
var (
_ contracts.Chunker = dndSpellsChunker{}
_ contracts.OutputEncoder = dndSpellsOutput{}
)

View File

@@ -0,0 +1,163 @@
package spells
import (
"context"
"encoding/json"
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const Key = "dnd/spells"
const ArtifactType = "dnd.spell_cast"
const SchemaVersion = "v1"
var requiredCapabilities = []string{
"chunks",
"source.transcript",
}
var providedCapabilities = []string{
"dnd.spell_casts",
}
var _ contracts.Extractor = (*Extractor)(nil)
type Extractor struct{}
func New() *Extractor {
return &Extractor{}
}
func (e *Extractor) Key() string {
return Key
}
func (e *Extractor) ArtifactType() string {
return ArtifactType
}
func (e *Extractor) SchemaVersion() string {
return SchemaVersion
}
func (e *Extractor) ManifestMetadata() map[string]any {
promptMetadata := spellsPromptBundle.Metadata()
metadata := map[string]any{
"prompt_id": PromptID,
"prompt_version": promptMetadata.PromptVersion,
"prompt_sha256": promptMetadata.SHA256,
"response_schema_key": string(ResponseSchemaKey),
"response_schema_id": ResponseSchemaID,
"response_schema_name": ResponseSchemaName,
}
if schema, err := loadResponseSchema(); err == nil {
metadata["response_schema_version"] = schema.Version
metadata["response_schema_sha256"] = schema.SHA256
}
return metadata
}
func (e *Extractor) Validators() []contracts.Validator {
return []contracts.Validator{
ShapeValidator{},
SourceRefValidator{},
}
}
func (e *Extractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
if e == nil {
return contracts.ExtractionResult{}, extractorErrorf("extractor must not be nil")
}
if ctx == nil {
return contracts.ExtractionResult{}, extractorErrorf("context must not be nil")
}
if err := ctx.Err(); err != nil {
return contracts.ExtractionResult{}, extractorErrorf("context error before extraction: %w", err)
}
if req.Source == nil {
return contracts.ExtractionResult{}, extractorErrorf("source must not be nil")
}
if req.Chunk == nil {
return contracts.ExtractionResult{}, extractorErrorf("chunk must not be nil")
}
if len(req.Chunk.Units) == 0 {
return contracts.ExtractionResult{}, extractorErrorf("chunk %q units must not be empty", req.Chunk.ID)
}
if req.LLMClient == nil {
return contracts.ExtractionResult{}, extractorErrorf("LLM client must not be nil")
}
system, user, _, err := renderPrompt(req)
if err != nil {
return contracts.ExtractionResult{}, extractorErrorf("render prompt: %w", err)
}
schema, err := loadResponseSchema()
if err != nil {
return contracts.ExtractionResult{}, extractorErrorf("load response schema %q: %w", ResponseSchemaKey, err)
}
var response extractionResponse
if _, err := req.LLMClient.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
StageName: Key,
Messages: []contracts.LLMMessage{
{Role: "system", Content: system},
{Role: "user", Content: user},
},
ResponseSchemaName: schema.Name,
ResponseSchema: schema.JSONSchema,
}, &response); err != nil {
return contracts.ExtractionResult{}, extractorErrorf("complete structured output: %w", err)
}
if response.SpellCasts == nil {
return contracts.ExtractionResult{}, extractorErrorf("malformed structured output: spell_casts must be present")
}
if len(response.SpellCasts) == 0 {
return contracts.ExtractionResult{}, nil
}
candidates := make([]artifacts.ArtifactCandidate, 0, len(response.SpellCasts))
for i, spellCast := range response.SpellCasts {
payload, err := spellCastPayload(spellCast)
if err != nil {
return contracts.ExtractionResult{}, extractorErrorf("marshal spell cast[%d]: %w", i, err)
}
candidates = append(candidates, artifacts.ArtifactCandidate{
Payload: payload,
SourceRefs: append([]source.SourceRef(nil), spellCast.SourceRefs...),
})
}
return contracts.ExtractionResult{Candidates: candidates}, nil
}
func ModuleSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageExtract,
Requires: append([]string(nil), requiredCapabilities...),
Provides: append([]string(nil), providedCapabilities...),
}
}
func Register(registry *pipeline.ExtractorRegistry) error {
return registry.RegisterWithSpec(ModuleSpec(), func() (contracts.Extractor, error) {
return New(), nil
})
}
func spellCastPayload(spellCast spellCastResponse) (json.RawMessage, error) {
return json.Marshal(SpellCast{
Caster: strings.TrimSpace(spellCast.Caster),
Spell: strings.TrimSpace(spellCast.Spell),
Effect: strings.TrimSpace(spellCast.Effect),
NarrativeDescription: strings.TrimSpace(spellCast.NarrativeDescription),
})
}
func extractorErrorf(format string, args ...any) error {
return fmt.Errorf("dnd spells extractor: "+format, args...)
}

View File

@@ -0,0 +1,301 @@
package spells
import (
"bytes"
"context"
"encoding/json"
"errors"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
func TestExtractReturnsSpellCandidateFromStructuredOutput(t *testing.T) {
client := &fakeSpellsLLMClient{
response: extractionResponse{
SpellCasts: []spellCastResponse{
{
Caster: " Aria ",
Spell: " Cure Wounds ",
Effect: " Heals an injured ally. ",
NarrativeDescription: " Aria restores the fighter after the fight. ",
SourceRefs: []source.SourceRef{
{SourceID: "session-alpha", StartUnitID: "seg-001", EndUnitID: "seg-002"},
},
},
},
},
}
result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
if len(client.requests) != 1 {
t.Fatalf("LLM calls = %d, want 1", len(client.requests))
}
req := client.requests[0]
if req.StageName != Key {
t.Fatalf("StageName = %q, want %q", req.StageName, Key)
}
schema, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
if req.ResponseSchemaName != schema.Name {
t.Fatalf("ResponseSchemaName = %q, want %q", req.ResponseSchemaName, schema.Name)
}
if !bytes.Equal(req.ResponseSchema, schema.JSONSchema) {
t.Fatal("ResponseSchema does not match registered D&D spells schema")
}
if len(req.Messages) != 2 {
t.Fatalf("len(Messages) = %d, want 2", len(req.Messages))
}
if req.Messages[0].Role != "system" || req.Messages[1].Role != "user" {
t.Fatalf("Messages roles = %#v, want system then user", req.Messages)
}
if !strings.Contains(req.Messages[0].Content, "D&D spell-cast") {
t.Fatalf("system message = %q, want D&D spell context", req.Messages[0].Content)
}
for _, want := range []string{"session-alpha", "session-alpha:chunk:0", "seg-001", "Cure Wounds"} {
if !strings.Contains(req.Messages[1].Content, want) {
t.Fatalf("user message = %q, want substring %q", req.Messages[1].Content, want)
}
}
if len(result.Candidates) != 1 {
t.Fatalf("len(Candidates) = %d, want 1", len(result.Candidates))
}
candidate := result.Candidates[0]
if candidate.Index != 0 || candidate.ExtractorKey != "" || candidate.ArtifactType != "" || candidate.SchemaVersion != "" {
t.Fatalf("candidate envelope fields = %#v, want runner-normalized zero values", candidate)
}
var payload SpellCast
if err := json.Unmarshal(candidate.Payload, &payload); err != nil {
t.Fatalf("Unmarshal(Payload) error = %v, want nil", err)
}
wantPayload := SpellCast{
Caster: "Aria",
Spell: "Cure Wounds",
Effect: "Heals an injured ally.",
NarrativeDescription: "Aria restores the fighter after the fight.",
}
if payload != wantPayload {
t.Fatalf("payload = %#v, want %#v", payload, wantPayload)
}
wantRef := source.SourceRef{SourceID: "session-alpha", StartUnitID: "seg-001", EndUnitID: "seg-002"}
if len(candidate.SourceRefs) != 1 || candidate.SourceRefs[0] != wantRef {
t.Fatalf("SourceRefs = %#v, want %#v", candidate.SourceRefs, []source.SourceRef{wantRef})
}
}
func TestExtractorManifestMetadataIncludesPromptAndSchemaProvenance(t *testing.T) {
metadata := New().ManifestMetadata()
tests := map[string]string{
"prompt_id": PromptID,
"prompt_version": SchemaVersion,
"response_schema_key": string(ResponseSchemaKey),
"response_schema_id": ResponseSchemaID,
"response_schema_name": ResponseSchemaName,
"response_schema_version": SchemaVersion,
}
for key, want := range tests {
if metadata[key] != want {
t.Fatalf("metadata[%q] = %#v, want %q", key, metadata[key], want)
}
}
for _, key := range []string{"prompt_sha256", "response_schema_sha256"} {
value, ok := metadata[key].(string)
if !ok || !strings.HasPrefix(value, "sha256:") {
t.Fatalf("metadata[%q] = %#v, want sha256 value", key, metadata[key])
}
}
}
func TestExtractReturnsNoCandidatesForEmptyResponse(t *testing.T) {
client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}}
result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
if len(result.Candidates) != 0 {
t.Fatalf("Candidates = %#v, want none", result.Candidates)
}
}
func TestExtractRejectsMissingSpellCasts(t *testing.T) {
client := &fakeSpellsLLMClient{response: extractionResponse{}}
_, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err == nil {
t.Fatal("Extract() error = nil, want malformed output error")
}
if !strings.Contains(err.Error(), "dnd spells") || !strings.Contains(err.Error(), "spell_casts") {
t.Fatalf("Extract() error = %q, want spell_casts context", err.Error())
}
}
func TestExtractWrapsLLMClientError(t *testing.T) {
client := &fakeSpellsLLMClient{err: errors.New("provider unavailable")}
_, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err == nil {
t.Fatal("Extract() error = nil, want LLM error")
}
if !strings.Contains(err.Error(), "dnd spells") || !strings.Contains(err.Error(), "provider unavailable") {
t.Fatalf("Extract() error = %q, want wrapped LLM context", err.Error())
}
}
func TestExtractRejectsInvalidRequests(t *testing.T) {
validClient := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}}
validReq := extractionRequestWithClient(validClient)
canceledCtx, cancel := context.WithCancel(context.Background())
cancel()
tests := []struct {
name string
extractor *Extractor
ctx context.Context
req contracts.ExtractionRequest
want string
}{
{name: "nil extractor", extractor: nil, ctx: context.Background(), req: validReq, want: "extractor"},
{name: "nil context", extractor: New(), ctx: nil, req: validReq, want: "context"},
{name: "canceled context", extractor: New(), ctx: canceledCtx, req: validReq, want: "context"},
{name: "nil source", extractor: New(), ctx: context.Background(), req: contracts.ExtractionRequest{Chunk: validReq.Chunk, LLMClient: validReq.LLMClient}, want: "source"},
{name: "nil chunk", extractor: New(), ctx: context.Background(), req: contracts.ExtractionRequest{Source: validReq.Source, LLMClient: validReq.LLMClient}, want: "chunk"},
{name: "empty chunk units", extractor: New(), ctx: context.Background(), req: emptyChunkRequest(validReq), want: "units"},
{name: "nil LLM client", extractor: New(), ctx: context.Background(), req: contracts.ExtractionRequest{Source: validReq.Source, Chunk: validReq.Chunk}, want: "LLM client"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := tt.extractor.Extract(tt.ctx, tt.req)
if err == nil {
t.Fatal("Extract() error = nil, want error")
}
if !strings.Contains(err.Error(), "dnd spells") || !strings.Contains(err.Error(), tt.want) {
t.Fatalf("Extract() error = %q, want %q context", err.Error(), tt.want)
}
})
}
}
func TestExtractPreservesResponseOrder(t *testing.T) {
client := &fakeSpellsLLMClient{
response: extractionResponse{
SpellCasts: []spellCastResponse{
{
Caster: "Aria",
Spell: "Cure Wounds",
Effect: "Heals.",
NarrativeDescription: "First spell.",
SourceRefs: []source.SourceRef{{SourceID: "session-alpha", StartUnitID: "seg-001", EndUnitID: "seg-001"}},
},
{
Caster: "Bandit Shaman",
Spell: "Fire Bolt",
Effect: "Burns.",
NarrativeDescription: "Second spell.",
SourceRefs: []source.SourceRef{{SourceID: "session-alpha", StartUnitID: "seg-002", EndUnitID: "seg-002"}},
},
},
},
}
result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
if len(result.Candidates) != 2 {
t.Fatalf("len(Candidates) = %d, want 2", len(result.Candidates))
}
var first, second SpellCast
if err := json.Unmarshal(result.Candidates[0].Payload, &first); err != nil {
t.Fatalf("Unmarshal(first) error = %v, want nil", err)
}
if err := json.Unmarshal(result.Candidates[1].Payload, &second); err != nil {
t.Fatalf("Unmarshal(second) error = %v, want nil", err)
}
if first.Spell != "Cure Wounds" || second.Spell != "Fire Bolt" {
t.Fatalf("candidate order = %q, %q; want response order", first.Spell, second.Spell)
}
}
func TestExtractCopiesCandidateSourceRefs(t *testing.T) {
client := &fakeSpellsLLMClient{
response: extractionResponse{
SpellCasts: []spellCastResponse{
{
Caster: "Aria",
Spell: "Cure Wounds",
Effect: "Heals.",
NarrativeDescription: "Aria heals.",
SourceRefs: []source.SourceRef{{SourceID: "session-alpha", StartUnitID: "seg-001", EndUnitID: "seg-002"}},
},
},
},
}
result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
client.response.SpellCasts[0].SourceRefs[0].StartUnitID = "mutated"
if got := result.Candidates[0].SourceRefs[0].StartUnitID; got != "seg-001" {
t.Fatalf("candidate source ref start = %q, want copied seg-001", got)
}
}
func extractionRequestWithClient(client contracts.StructuredLLMClient) contracts.ExtractionRequest {
req := promptExtractionRequest()
req.LLMClient = client
return req
}
func emptyChunkRequest(req contracts.ExtractionRequest) contracts.ExtractionRequest {
req.Chunk = &contracts.SourceChunk{
ID: req.Chunk.ID,
SourceID: req.Chunk.SourceID,
Index: req.Chunk.Index,
}
return req
}
type fakeSpellsLLMClient struct {
response extractionResponse
err error
requests []contracts.StructuredCompletionRequest
}
func (client *fakeSpellsLLMClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
client.requests = append(client.requests, contracts.StructuredCompletionRequest{
StageName: req.StageName,
Messages: append([]contracts.LLMMessage(nil), req.Messages...),
Model: req.Model,
ResponseSchemaName: req.ResponseSchemaName,
ResponseSchema: append(json.RawMessage(nil), req.ResponseSchema...),
})
if client.err != nil {
return contracts.StructuredCompletionResponse{}, client.err
}
target, ok := out.(*extractionResponse)
if !ok {
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
}
*target = client.response
content, err := json.Marshal(client.response)
if err != nil {
return contracts.StructuredCompletionResponse{}, err
}
return contracts.StructuredCompletionResponse{Content: content}, nil
}

View File

@@ -0,0 +1,22 @@
package spells
import "gitea.maximumdirect.net/eric/notarius/internal/core/source"
type SpellCast struct {
Caster string `json:"caster"`
Spell string `json:"spell"`
Effect string `json:"effect"`
NarrativeDescription string `json:"narrative_description"`
}
type extractionResponse struct {
SpellCasts []spellCastResponse `json:"spell_casts"`
}
type spellCastResponse struct {
Caster string `json:"caster"`
Spell string `json:"spell"`
Effect string `json:"effect"`
NarrativeDescription string `json:"narrative_description"`
SourceRefs []source.SourceRef `json:"source_refs"`
}

View File

@@ -0,0 +1,106 @@
package spells
import (
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/prompt"
)
type promptData struct {
SourceID string
HasChunk bool
ChunkID string
ChunkIndex int
Units []promptUnit
}
type promptUnit struct {
ID string
Text string
Metadata []promptMetadata
}
type promptMetadata struct {
Key string
Value string
}
var spellsPromptBundle = mustLoadPromptBundle()
func mustLoadPromptBundle() *prompt.Bundle {
bundle, err := prompt.LoadBundle(embeddedAssets, prompt.Definition{
PromptID: PromptID,
Version: SchemaVersion,
EmbeddedPath: "assets/prompts",
SystemPath: "assets/prompts/system.md",
UserPath: "assets/prompts/user.md",
})
if err != nil {
panic(err)
}
return bundle
}
func buildPromptData(req contracts.ExtractionRequest) (promptData, error) {
if req.Source == nil {
return promptData{}, fmt.Errorf("dnd spells prompt: source must not be nil")
}
if req.Chunk == nil {
return promptData{}, fmt.Errorf("dnd spells prompt: chunk must not be nil")
}
data := promptData{
SourceID: req.Source.ID,
HasChunk: true,
ChunkID: req.Chunk.ID,
ChunkIndex: req.Chunk.Index,
Units: make([]promptUnit, 0, len(req.Chunk.Units)),
}
for _, unit := range req.Chunk.Units {
data.Units = append(data.Units, promptUnit{
ID: unit.ID,
Text: unit.Text,
Metadata: selectedMetadata(unit),
})
}
return data, nil
}
func renderPrompt(req contracts.ExtractionRequest) (system string, user string, metadata prompt.Metadata, err error) {
data, err := buildPromptData(req)
if err != nil {
return "", "", prompt.Metadata{}, err
}
system, user, metadata, err = spellsPromptBundle.RenderUserSystem(data)
if err != nil {
return "", "", prompt.Metadata{}, fmt.Errorf("dnd spells prompt: %w", err)
}
return system, user, metadata, nil
}
func selectedMetadata(unit source.SourceUnit) []promptMetadata {
if len(unit.Metadata) == 0 {
return nil
}
keys := []string{"speaker", "start", "end"}
metadata := make([]promptMetadata, 0, len(keys))
for _, key := range keys {
value, ok := unit.Metadata[key]
if !ok {
continue
}
rendered := strings.TrimSpace(fmt.Sprint(value))
if rendered == "" {
continue
}
metadata = append(metadata, promptMetadata{
Key: key,
Value: rendered,
})
}
return metadata
}

View File

@@ -0,0 +1,165 @@
package spells
import (
"encoding/json"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/prompt"
)
func TestBuildPromptDataFromGenericSourceChunk(t *testing.T) {
req := promptExtractionRequest()
data, err := buildPromptData(req)
if err != nil {
t.Fatalf("buildPromptData() error = %v, want nil", err)
}
if data.SourceID != "session-alpha" {
t.Fatalf("SourceID = %q, want session-alpha", data.SourceID)
}
if !data.HasChunk || data.ChunkID != "session-alpha:chunk:0" || data.ChunkIndex != 0 {
t.Fatalf("chunk data = %#v, want fixture chunk", data)
}
if len(data.Units) != 2 {
t.Fatalf("len(Units) = %d, want 2", len(data.Units))
}
first := data.Units[0]
if first.ID != "seg-001" || first.Text != "Aria raises her hand and casts Cure Wounds." {
t.Fatalf("first unit = %#v, want source unit data", first)
}
wantMetadata := []promptMetadata{
{Key: "speaker", Value: "Alice"},
{Key: "start", Value: "1.25"},
{Key: "end", Value: "3.5"},
}
if !reflect.DeepEqual(first.Metadata, wantMetadata) {
t.Fatalf("first.Metadata = %#v, want %#v", first.Metadata, wantMetadata)
}
if len(data.Units[1].Metadata) != 0 {
t.Fatalf("second.Metadata = %#v, want no selected metadata", data.Units[1].Metadata)
}
}
func TestBuildPromptDataDoesNotMutateRequest(t *testing.T) {
req := promptExtractionRequest()
beforeSource := mustJSON(t, req.Source)
beforeChunk := mustJSON(t, req.Chunk)
beforeRequest := mustJSON(t, req)
if _, err := buildPromptData(req); err != nil {
t.Fatalf("buildPromptData() error = %v, want nil", err)
}
afterSource := mustJSON(t, req.Source)
afterChunk := mustJSON(t, req.Chunk)
afterRequest := mustJSON(t, req)
if beforeSource != afterSource || beforeChunk != afterChunk || beforeRequest != afterRequest {
t.Fatalf(
"request mutated:\nsource before: %s\nsource after: %s\nchunk before: %s\nchunk after: %s\nrequest before: %s\nrequest after: %s",
beforeSource,
afterSource,
beforeChunk,
afterChunk,
beforeRequest,
afterRequest,
)
}
}
func TestRenderPromptIncludesSourceContext(t *testing.T) {
system, user, metadata, err := renderPrompt(promptExtractionRequest())
if err != nil {
t.Fatalf("renderPrompt() error = %v, want nil", err)
}
if !strings.Contains(system, prompt.HardeningText()) {
t.Fatalf("system prompt = %q, want hardening text", system)
}
for _, want := range []string{
"session-alpha",
"session-alpha:chunk:0",
"seg-001",
"Aria raises her hand and casts Cure Wounds.",
"speaker: Alice",
"start: 1.25",
"end: 3.5",
} {
if !strings.Contains(user, want) {
t.Fatalf("user prompt = %q, want substring %q", user, want)
}
}
if metadata.PromptID != PromptID {
t.Fatalf("metadata.PromptID = %q, want %q", metadata.PromptID, PromptID)
}
if metadata.PromptVersion != SchemaVersion {
t.Fatalf("metadata.PromptVersion = %q, want %q", metadata.PromptVersion, SchemaVersion)
}
if metadata.EmbeddedPath != "assets/prompts" {
t.Fatalf("metadata.EmbeddedPath = %q, want assets/prompts", metadata.EmbeddedPath)
}
}
func TestBuildPromptDataRejectsMissingSourceContext(t *testing.T) {
if _, err := buildPromptData(contracts.ExtractionRequest{}); err == nil || !strings.Contains(err.Error(), "source") {
t.Fatalf("buildPromptData() error = %v, want source error", err)
}
if _, err := buildPromptData(contracts.ExtractionRequest{Source: promptSourceDocument()}); err == nil || !strings.Contains(err.Error(), "chunk") {
t.Fatalf("buildPromptData() error = %v, want chunk error", err)
}
}
func promptExtractionRequest() contracts.ExtractionRequest {
doc := promptSourceDocument()
chunk := &contracts.SourceChunk{
ID: "session-alpha:chunk:0",
SourceID: doc.ID,
Index: 0,
Units: append([]source.SourceUnit(nil), doc.Units...),
Metadata: map[string]any{"ignored": "chunk metadata"},
}
return contracts.ExtractionRequest{
Source: doc,
Chunk: chunk,
}
}
func promptSourceDocument() *source.SourceDocument {
return &source.SourceDocument{
ID: "session-alpha",
Kind: "transcript",
Format: "application/vnd.seriatim.minimal+json",
Digest: "sha256:test",
Units: []source.SourceUnit{
{
ID: "seg-001",
Kind: "transcript_segment",
Text: "Aria raises her hand and casts Cure Wounds.",
Metadata: map[string]any{
"speaker": "Alice",
"start": json.Number("1.25"),
"end": json.Number("3.5"),
"ignored": "not rendered",
},
},
{
ID: "seg-002",
Kind: "transcript_segment",
Text: "The fighter's wounds begin to close.",
Metadata: map[string]any{"ignored": "not rendered"},
},
},
}
}
func mustJSON(t *testing.T, value any) string {
t.Helper()
encoded, err := json.Marshal(value)
if err != nil {
t.Fatalf("Marshal() error = %v, want nil", err)
}
return string(encoded)
}

View File

@@ -0,0 +1,93 @@
package spells
import (
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestNewReturnsExtractorWithMetadata(t *testing.T) {
extractor := New()
if extractor == nil {
t.Fatal("New() = nil, want extractor")
}
if extractor.Key() != Key {
t.Fatalf("extractor.Key() = %q, want %q", extractor.Key(), Key)
}
if extractor.ArtifactType() != ArtifactType {
t.Fatalf("extractor.ArtifactType() = %q, want %q", extractor.ArtifactType(), ArtifactType)
}
if extractor.SchemaVersion() != SchemaVersion {
t.Fatalf("extractor.SchemaVersion() = %q, want %q", extractor.SchemaVersion(), SchemaVersion)
}
}
func TestModuleSpec(t *testing.T) {
got := ModuleSpec()
want := pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageExtract,
Requires: []string{
"chunks",
"source.transcript",
},
Provides: []string{
"dnd.spell_casts",
},
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("ModuleSpec() = %#v, want %#v", got, want)
}
got.Requires[0] = "changed"
got.Provides[0] = "changed"
again := ModuleSpec()
if !reflect.DeepEqual(again, want) {
t.Fatalf("ModuleSpec() after caller mutation = %#v, want %#v", again, want)
}
}
func TestRegisterMakesExtractorBuildable(t *testing.T) {
registry := pipeline.NewExtractorRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
extractor, err := registry.Build(Key)
if err != nil {
t.Fatalf("Build() error = %v, want nil", err)
}
if extractor.Key() != Key {
t.Fatalf("extractor.Key() = %q, want %q", extractor.Key(), Key)
}
}
func TestRegisterStoresModuleSpec(t *testing.T) {
registry := pipeline.NewExtractorRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
got, ok := registry.Spec(Key)
if !ok {
t.Fatal("Spec() ok = false, want true")
}
want := ModuleSpec()
if !reflect.DeepEqual(got, want) {
t.Fatalf("Spec() = %#v, want %#v", got, want)
}
}
func TestRegisterNilRegistryReturnsError(t *testing.T) {
err := Register(nil)
if err == nil {
t.Fatal("Register(nil) error = nil, want error")
}
if !strings.Contains(err.Error(), "extractor registry") {
t.Fatalf("Register(nil) error = %q, want registry context", err.Error())
}
}

View File

@@ -0,0 +1,226 @@
package spells
import (
"context"
"encoding/json"
"os"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/input/seriatim"
)
func TestRunnerProcessesSeriatimInputWithDNDSpellsExtractor(t *testing.T) {
raw := readDNDSpellsFixture(t)
expectedDoc := parseDNDSpellsFixture(t, raw)
resolved := resolveDNDSpellsPipeline(t)
llmClient := &fakeSpellsLLMClient{
response: extractionResponse{
SpellCasts: []spellCastResponse{
{
Caster: "Aria",
Spell: "Cure Wounds",
Effect: "Heals an injured ally.",
NarrativeDescription: "Aria restores the fighter after the fight.",
SourceRefs: []source.SourceRef{
{SourceID: expectedDoc.ID, StartUnitID: "seg-001", EndUnitID: "seg-001"},
},
},
{
Caster: "Borin",
Spell: "Fire Bolt",
Effect: "Scorches the wight.",
NarrativeDescription: "Borin hurls fire at the wight.",
SourceRefs: []source.SourceRef{
{SourceID: expectedDoc.ID, StartUnitID: "seg-003", EndUnitID: "seg-003"},
},
},
},
},
}
output, err := pipeline.New(dndSpellsRunnerRegistries(t)).Run(context.Background(), pipeline.RunInput{
Pipeline: resolved.ResolvedPipeline,
RawInput: raw,
LLMClient: llmClient,
})
if err != nil {
t.Fatalf("Run() error = %v, want nil", err)
}
if len(output.Approved) != 2 {
t.Fatalf("len(Approved) = %d, want 2", len(output.Approved))
}
var first, second SpellCast
if err := json.Unmarshal(output.Approved[0].Payload, &first); err != nil {
t.Fatalf("Unmarshal(first payload) error = %v, want nil", err)
}
if err := json.Unmarshal(output.Approved[1].Payload, &second); err != nil {
t.Fatalf("Unmarshal(second payload) error = %v, want nil", err)
}
if first.Spell != "Cure Wounds" || second.Spell != "Fire Bolt" {
t.Fatalf("approved spell order = %q, %q; want response order", first.Spell, second.Spell)
}
if first.Caster != "Aria" || second.Caster != "Borin" {
t.Fatalf("approved casters = %q, %q; want spell data", first.Caster, second.Caster)
}
for _, artifact := range output.Approved {
if artifact.ExtractorKey != Key || artifact.ArtifactType != ArtifactType || artifact.SchemaVersion != SchemaVersion {
t.Fatalf("approved artifact envelope = %#v, want dnd spells envelope", artifact)
}
if len(artifact.SourceRefs) != 1 {
t.Fatalf("len(SourceRefs) = %d, want 1", len(artifact.SourceRefs))
}
if err := source.ValidateRef(expectedDoc, artifact.SourceRefs[0]); err != nil {
t.Fatalf("ValidateRef() error = %v, want nil", err)
}
}
if output.Manifest.InputModule != seriatim.Key {
t.Fatalf("manifest input module = %q, want %q", output.Manifest.InputModule, seriatim.Key)
}
if output.Manifest.ValidationStatus != "approved" {
t.Fatalf("ValidationStatus = %q, want approved", output.Manifest.ValidationStatus)
}
if len(output.Manifest.ArtifactLanes) != 1 {
t.Fatalf("len(ArtifactLanes) = %d, want 1", len(output.Manifest.ArtifactLanes))
}
lane := output.Manifest.ArtifactLanes[0]
if lane.ID != "spells" || lane.Extractor != Key {
t.Fatalf("manifest lane = %#v, want spells lane with dnd/spells extractor", lane)
}
extractorMetadata, ok := lane.Metadata["extractor"].(map[string]any)
if !ok {
t.Fatalf("manifest lane metadata = %#v, want extractor metadata", lane.Metadata)
}
if extractorMetadata["prompt_id"] != PromptID ||
extractorMetadata["response_schema_key"] != string(ResponseSchemaKey) ||
extractorMetadata["response_schema_name"] != ResponseSchemaName {
t.Fatalf("extractor metadata = %#v, want prompt/schema identifiers", extractorMetadata)
}
if len(output.OutputFiles) != 1 {
t.Fatalf("len(OutputFiles) = %d, want 1", len(output.OutputFiles))
}
if output.OutputFiles[0].ContentType != "application/json" {
t.Fatalf("ContentType = %q, want application/json", output.OutputFiles[0].ContentType)
}
}
func TestRunnerRejectsDNDSpellCastWithInvalidSourceRef(t *testing.T) {
raw := readDNDSpellsFixture(t)
resolved := resolveDNDSpellsPipeline(t)
llmClient := &fakeSpellsLLMClient{
response: extractionResponse{
SpellCasts: []spellCastResponse{
{
Caster: "Aria",
Spell: "Cure Wounds",
Effect: "Heals an injured ally.",
NarrativeDescription: "Aria restores the fighter after the fight.",
SourceRefs: []source.SourceRef{
{SourceID: "spell-session", StartUnitID: "seg-999", EndUnitID: "seg-999"},
},
},
},
},
}
output, err := pipeline.New(dndSpellsRunnerRegistries(t)).Run(context.Background(), pipeline.RunInput{
Pipeline: resolved.ResolvedPipeline,
RawInput: raw,
LLMClient: llmClient,
})
if err != nil {
t.Fatalf("Run() error = %v, want nil", err)
}
if len(output.Approved) != 0 {
t.Fatalf("len(Approved) = %d, want 0", len(output.Approved))
}
if len(output.Rejected) != 1 {
t.Fatalf("len(Rejected) = %d, want 1", len(output.Rejected))
}
rejected := output.Rejected[0]
if rejected.ValidatorName != sourceRefValidatorName {
t.Fatalf("ValidatorName = %q, want %q", rejected.ValidatorName, sourceRefValidatorName)
}
if rejected.ReasonCode != reasonInvalidSourceRef {
t.Fatalf("ReasonCode = %q, want %q", rejected.ReasonCode, reasonInvalidSourceRef)
}
if output.Manifest.ValidationStatus != "rejected" {
t.Fatalf("ValidationStatus = %q, want rejected", output.Manifest.ValidationStatus)
}
}
func TestRunnerFailsWhenDNDSpellsExtractorReturnsMalformedOutput(t *testing.T) {
raw := readDNDSpellsFixture(t)
resolved := resolveDNDSpellsPipeline(t)
llmClient := &fakeSpellsLLMClient{response: extractionResponse{}}
output, err := pipeline.New(dndSpellsRunnerRegistries(t)).Run(context.Background(), pipeline.RunInput{
Pipeline: resolved.ResolvedPipeline,
RawInput: raw,
LLMClient: llmClient,
})
if err == nil {
t.Fatal("Run() error = nil, want malformed extraction error")
}
if !strings.Contains(err.Error(), "extract lane") ||
!strings.Contains(err.Error(), "dnd spells") ||
!strings.Contains(err.Error(), "spell_casts") {
t.Fatalf("Run() error = %q, want D&D spells extraction context", err.Error())
}
if output.Manifest.ValidationStatus != "failed" {
t.Fatalf("ValidationStatus = %q, want failed", output.Manifest.ValidationStatus)
}
}
func resolveDNDSpellsPipeline(t *testing.T) config.EffectiveConfig {
t.Helper()
resolved, err := loadDNDSpellsPipelineConfig(t).Resolve(config.ResolveInput{
PipelineID: "dnd-spells-fixture",
Catalog: dndSpellsTestCatalog(t, dndSpellsCatalogSpecs{}),
})
if err != nil {
t.Fatalf("Resolve() error = %v, want nil", err)
}
return resolved
}
func dndSpellsRunnerRegistries(t *testing.T) pipeline.Registries {
t.Helper()
catalog := dndSpellsTestCatalog(t, dndSpellsCatalogSpecs{})
return pipeline.Registries{
Inputs: catalog.Inputs,
Chunkers: catalog.Chunkers,
Extractors: catalog.Extractors,
Mergers: catalog.Mergers,
Normalizers: catalog.Normalizers,
Outputs: catalog.Outputs,
}
}
func readDNDSpellsFixture(t *testing.T) []byte {
t.Helper()
raw, err := os.ReadFile("testdata/seriatim_spell_session.json")
if err != nil {
t.Fatalf("ReadFile(seriatim_spell_session.json) error = %v, want nil", err)
}
return raw
}
func parseDNDSpellsFixture(t *testing.T, raw []byte) *source.SourceDocument {
t.Helper()
doc, err := seriatim.New().Parse(context.Background(), contracts.ParseRequest{Raw: raw})
if err != nil {
t.Fatalf("Parse() error = %v, want nil", err)
}
return doc
}

View File

@@ -0,0 +1,20 @@
package spells
import "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
const (
PromptID = "dnd.spells"
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_spells")
ResponseSchemaID = "notarius.dnd.spells"
ResponseSchemaName = "notarius_dnd_spells_v1"
)
func loadResponseSchema() (llm.ResponseSchema, error) {
return llm.LoadResponseSchema(embeddedAssets, llm.ResponseSchemaDefinition{
Key: ResponseSchemaKey,
ID: ResponseSchemaID,
Version: SchemaVersion,
Name: ResponseSchemaName,
AssetPath: "assets/schemas/dnd_spells.v1.json",
})
}

View File

@@ -0,0 +1,74 @@
package spells
import (
"encoding/json"
"strings"
"testing"
)
func TestLoadResponseSchemaForSpells(t *testing.T) {
schema, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
if schema.Key != ResponseSchemaKey {
t.Fatalf("schema.Key = %q, want %q", schema.Key, ResponseSchemaKey)
}
if schema.ID != ResponseSchemaID {
t.Fatalf("schema.ID = %q, want %q", schema.ID, ResponseSchemaID)
}
if schema.Version != SchemaVersion {
t.Fatalf("schema.Version = %q, want %q", schema.Version, SchemaVersion)
}
if schema.Name != ResponseSchemaName {
t.Fatalf("schema.Name = %q, want %q", schema.Name, ResponseSchemaName)
}
if !strings.HasPrefix(schema.SHA256, "sha256:") {
t.Fatalf("schema.SHA256 = %q, want sha256 prefix", schema.SHA256)
}
if !json.Valid(schema.JSONSchema) {
t.Fatalf("schema.JSONSchema is invalid JSON: %s", schema.JSONSchema)
}
}
func TestResponseSchemaJSONIsMutationSafe(t *testing.T) {
first, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
first.JSONSchema[0] = '['
second, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
if !json.Valid(second.JSONSchema) {
t.Fatalf("schema JSON was mutated: %s", second.JSONSchema)
}
if len(second.JSONSchema) > 0 && second.JSONSchema[0] == '[' {
t.Fatalf("schema JSON did not use defensive copy")
}
}
func TestResponseSchemaDiagnosticsOmitRawSchema(t *testing.T) {
schema, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
diagnostics := schema.DiagnosticsMap()
if diagnostics["key"] != ResponseSchemaKey {
t.Fatalf("diagnostics[key] = %#v, want %q", diagnostics["key"], ResponseSchemaKey)
}
for _, key := range []string{"id", "version", "name", "sha256"} {
if diagnostics[key] == "" {
t.Fatalf("diagnostics[%q] = %#v, want value", key, diagnostics[key])
}
}
if _, ok := diagnostics["json_schema"]; ok {
t.Fatalf("diagnostics should omit raw schema content: %#v", diagnostics)
}
if _, ok := diagnostics["JSONSchema"]; ok {
t.Fatalf("diagnostics should omit raw schema content: %#v", diagnostics)
}
}

View File

@@ -0,0 +1,11 @@
version: 1
pipelines:
dnd-spells-fixture:
input: seriatim
chunk: fake/chunk
artifacts:
spells:
extract: dnd/spells
merge: appendorder
normalize: noop
output: json

View File

@@ -0,0 +1,29 @@
{
"metadata": {
"id": "spell-session",
"title": "Synthetic D&D spell session"
},
"segments": [
{
"id": "seg-001",
"start": 0,
"end": 4,
"speaker": "Alice",
"text": "Aria raises her holy symbol and casts Cure Wounds."
},
{
"id": "seg-002",
"start": 4,
"end": 8,
"speaker": "DM",
"text": "The bandit mage casts Shield as the blow lands."
},
{
"id": "seg-003",
"start": 8,
"end": 12,
"speaker": "Bob",
"text": "Borin points at the wight and casts Fire Bolt."
}
]
}

View File

@@ -0,0 +1,104 @@
package spells
import (
"context"
"encoding/json"
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/validate"
)
const (
shapeValidatorName = "dnd/spells/shape"
sourceRefValidatorName = "dnd/spells/source_refs"
reasonInvalidPayload = "invalid_payload"
reasonMissingRequiredField = "missing_required_field"
reasonMissingSourceRef = "missing_source_ref"
reasonInvalidSourceRef = "invalid_source_ref"
)
var _ contracts.Validator = ShapeValidator{}
var _ contracts.Validator = SourceRefValidator{}
type ShapeValidator struct{}
func (validator ShapeValidator) Name() string {
return shapeValidatorName
}
func (validator ShapeValidator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
decisions := make([]contracts.ValidationDecision, 0, len(req.Candidates))
for _, candidate := range req.Candidates {
decisions = append(decisions, validateShape(candidate))
}
return contracts.ValidationResult{
ValidatorName: validator.Name(),
Decisions: decisions,
}, nil
}
type SourceRefValidator struct{}
func (validator SourceRefValidator) Name() string {
return sourceRefValidatorName
}
func (validator SourceRefValidator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
if req.Source == nil {
return contracts.ValidationResult{}, fmt.Errorf("dnd spells source refs validator: source must not be nil")
}
decisions := make([]contracts.ValidationDecision, 0, len(req.Candidates))
for _, candidate := range req.Candidates {
decisions = append(decisions, validateSourceRefs(req.Source, candidate))
}
return contracts.ValidationResult{
ValidatorName: validator.Name(),
Decisions: decisions,
}, nil
}
func validateShape(candidate artifacts.ArtifactCandidate) contracts.ValidationDecision {
var payload SpellCast
if err := json.Unmarshal(candidate.Payload, &payload); err != nil {
return validate.Rejected(candidate.Index, reasonInvalidPayload, fmt.Sprintf("invalid spell cast payload: %v", err))
}
for _, field := range requiredSpellCastFields(payload) {
if strings.TrimSpace(field.value) == "" {
return validate.Rejected(candidate.Index, reasonMissingRequiredField, fmt.Sprintf("missing required field %q", field.name))
}
}
return validate.Approved(candidate.Index)
}
func validateSourceRefs(doc *source.SourceDocument, candidate artifacts.ArtifactCandidate) contracts.ValidationDecision {
if len(candidate.SourceRefs) == 0 {
return validate.Rejected(candidate.Index, reasonMissingSourceRef, "spell cast candidate must include at least one source ref")
}
for _, ref := range candidate.SourceRefs {
if err := source.ValidateRef(doc, ref); err != nil {
return validate.Rejected(candidate.Index, reasonInvalidSourceRef, err.Error())
}
}
return validate.Approved(candidate.Index)
}
func requiredSpellCastFields(payload SpellCast) []struct {
name string
value string
} {
return []struct {
name string
value string
}{
{name: "caster", value: payload.Caster},
{name: "spell", value: payload.Spell},
{name: "effect", value: payload.Effect},
{name: "narrative_description", value: payload.NarrativeDescription},
}
}

View File

@@ -0,0 +1,271 @@
package spells
import (
"context"
"encoding/json"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/validate"
)
func TestExtractorValidatorsReturnsExpectedChain(t *testing.T) {
validators := New().Validators()
if len(validators) != 2 {
t.Fatalf("len(Validators()) = %d, want 2", len(validators))
}
if validators[0].Name() != shapeValidatorName {
t.Fatalf("Validators()[0].Name() = %q, want %q", validators[0].Name(), shapeValidatorName)
}
if validators[1].Name() != sourceRefValidatorName {
t.Fatalf("Validators()[1].Name() = %q, want %q", validators[1].Name(), sourceRefValidatorName)
}
validators[0] = nil
again := New().Validators()
if len(again) != 2 || again[0] == nil || again[0].Name() != shapeValidatorName {
t.Fatalf("Validators() after caller mutation = %#v, want fresh validators", again)
}
}
func TestValidatorsApproveValidCandidate(t *testing.T) {
candidate := validSpellCandidate(7)
shapeResult, err := ShapeValidator{}.Validate(context.Background(), contracts.ValidationRequest{
Candidates: []artifacts.ArtifactCandidate{candidate},
})
if err != nil {
t.Fatalf("ShapeValidator.Validate() error = %v, want nil", err)
}
assertSingleDecision(t, shapeResult, shapeValidatorName, 7, true, validate.ReasonApproved)
sourceRefResult, err := SourceRefValidator{}.Validate(context.Background(), contracts.ValidationRequest{
Source: promptSourceDocument(),
Candidates: []artifacts.ArtifactCandidate{candidate},
})
if err != nil {
t.Fatalf("SourceRefValidator.Validate() error = %v, want nil", err)
}
assertSingleDecision(t, sourceRefResult, sourceRefValidatorName, 7, true, validate.ReasonApproved)
}
func TestShapeValidatorRejectsMalformedPayload(t *testing.T) {
candidate := validSpellCandidate(3)
candidate.Payload = json.RawMessage(`{"caster":`)
result, err := ShapeValidator{}.Validate(context.Background(), contracts.ValidationRequest{
Candidates: []artifacts.ArtifactCandidate{candidate},
})
if err != nil {
t.Fatalf("ShapeValidator.Validate() error = %v, want nil", err)
}
assertSingleDecision(t, result, shapeValidatorName, 3, false, reasonInvalidPayload)
}
func TestShapeValidatorRejectsBlankRequiredFields(t *testing.T) {
tests := []struct {
name string
mutate func(*SpellCast)
}{
{name: "caster", mutate: func(payload *SpellCast) { payload.Caster = " \t" }},
{name: "spell", mutate: func(payload *SpellCast) { payload.Spell = "" }},
{name: "effect", mutate: func(payload *SpellCast) { payload.Effect = "\n" }},
{name: "narrative description", mutate: func(payload *SpellCast) { payload.NarrativeDescription = " " }},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
payload := validSpellPayload()
tt.mutate(&payload)
candidate := validSpellCandidate(5)
candidate.Payload = mustSpellPayload(t, payload)
result, err := ShapeValidator{}.Validate(context.Background(), contracts.ValidationRequest{
Candidates: []artifacts.ArtifactCandidate{candidate},
})
if err != nil {
t.Fatalf("ShapeValidator.Validate() error = %v, want nil", err)
}
assertSingleDecision(t, result, shapeValidatorName, 5, false, reasonMissingRequiredField)
})
}
}
func TestShapeValidatorDoesNotRequireSourceDocument(t *testing.T) {
result, err := ShapeValidator{}.Validate(context.Background(), contracts.ValidationRequest{
Candidates: []artifacts.ArtifactCandidate{validSpellCandidate(11)},
})
if err != nil {
t.Fatalf("ShapeValidator.Validate() error = %v, want nil", err)
}
assertSingleDecision(t, result, shapeValidatorName, 11, true, validate.ReasonApproved)
}
func TestSourceRefValidatorRejectsMissingRefs(t *testing.T) {
candidate := validSpellCandidate(13)
candidate.SourceRefs = nil
result, err := SourceRefValidator{}.Validate(context.Background(), contracts.ValidationRequest{
Source: promptSourceDocument(),
Candidates: []artifacts.ArtifactCandidate{candidate},
})
if err != nil {
t.Fatalf("SourceRefValidator.Validate() error = %v, want nil", err)
}
assertSingleDecision(t, result, sourceRefValidatorName, 13, false, reasonMissingSourceRef)
}
func TestSourceRefValidatorRejectsInvalidRefs(t *testing.T) {
tests := []struct {
name string
ref source.SourceRef
want string
}{
{
name: "unknown source id",
ref: source.SourceRef{SourceID: "session-beta", StartUnitID: "seg-001", EndUnitID: "seg-002"},
want: "does not match",
},
{
name: "unknown start unit",
ref: source.SourceRef{SourceID: "session-alpha", StartUnitID: "seg-999", EndUnitID: "seg-002"},
want: "start_unit_id",
},
{
name: "unknown end unit",
ref: source.SourceRef{SourceID: "session-alpha", StartUnitID: "seg-001", EndUnitID: "seg-999"},
want: "end_unit_id",
},
{
name: "reversed unit range",
ref: source.SourceRef{SourceID: "session-alpha", StartUnitID: "seg-002", EndUnitID: "seg-001"},
want: "appears after",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
candidate := validSpellCandidate(17)
candidate.SourceRefs = []source.SourceRef{tt.ref}
result, err := SourceRefValidator{}.Validate(context.Background(), contracts.ValidationRequest{
Source: promptSourceDocument(),
Candidates: []artifacts.ArtifactCandidate{candidate},
})
if err != nil {
t.Fatalf("SourceRefValidator.Validate() error = %v, want nil", err)
}
assertSingleDecision(t, result, sourceRefValidatorName, 17, false, reasonInvalidSourceRef)
if !strings.Contains(result.Decisions[0].Message, tt.want) {
t.Fatalf("Message = %q, want substring %q", result.Decisions[0].Message, tt.want)
}
})
}
}
func TestSourceRefValidatorRequiresSourceDocument(t *testing.T) {
_, err := SourceRefValidator{}.Validate(context.Background(), contracts.ValidationRequest{
Candidates: []artifacts.ArtifactCandidate{validSpellCandidate(19)},
})
if err == nil {
t.Fatal("SourceRefValidator.Validate() error = nil, want source error")
}
if !strings.Contains(err.Error(), "dnd spells") || !strings.Contains(err.Error(), "source") {
t.Fatalf("SourceRefValidator.Validate() error = %q, want source context", err.Error())
}
}
func TestValidatorsPreserveCandidateIndexes(t *testing.T) {
candidates := []artifacts.ArtifactCandidate{
validSpellCandidate(23),
validSpellCandidate(29),
}
shapeResult, err := ShapeValidator{}.Validate(context.Background(), contracts.ValidationRequest{
Candidates: candidates,
})
if err != nil {
t.Fatalf("ShapeValidator.Validate() error = %v, want nil", err)
}
assertDecisionIndexes(t, shapeResult.Decisions, []int{23, 29})
sourceRefResult, err := SourceRefValidator{}.Validate(context.Background(), contracts.ValidationRequest{
Source: promptSourceDocument(),
Candidates: candidates,
})
if err != nil {
t.Fatalf("SourceRefValidator.Validate() error = %v, want nil", err)
}
assertDecisionIndexes(t, sourceRefResult.Decisions, []int{23, 29})
}
func validSpellCandidate(index int) artifacts.ArtifactCandidate {
return artifacts.ArtifactCandidate{
Index: index,
Payload: spellPayload(validSpellPayload()),
SourceRefs: []source.SourceRef{
{SourceID: "session-alpha", StartUnitID: "seg-001", EndUnitID: "seg-002"},
},
}
}
func validSpellPayload() SpellCast {
return SpellCast{
Caster: "Aria",
Spell: "Cure Wounds",
Effect: "Heals an injured ally.",
NarrativeDescription: "Aria restores the fighter after the fight.",
}
}
func mustSpellPayload(t *testing.T, payload SpellCast) json.RawMessage {
t.Helper()
return spellPayload(payload)
}
func spellPayload(payload SpellCast) json.RawMessage {
encoded, err := json.Marshal(payload)
if err != nil {
panic(err)
}
return encoded
}
func assertSingleDecision(t *testing.T, result contracts.ValidationResult, wantName string, wantIndex int, wantApproved bool, wantReason string) {
t.Helper()
if result.ValidatorName != wantName {
t.Fatalf("ValidatorName = %q, want %q", result.ValidatorName, wantName)
}
if len(result.Decisions) != 1 {
t.Fatalf("len(Decisions) = %d, want 1", len(result.Decisions))
}
decision := result.Decisions[0]
if decision.CandidateIndex != wantIndex {
t.Fatalf("CandidateIndex = %d, want %d", decision.CandidateIndex, wantIndex)
}
if decision.Approved != wantApproved {
t.Fatalf("Approved = %t, want %t", decision.Approved, wantApproved)
}
if decision.ReasonCode != wantReason {
t.Fatalf("ReasonCode = %q, want %q", decision.ReasonCode, wantReason)
}
}
func assertDecisionIndexes(t *testing.T, decisions []contracts.ValidationDecision, want []int) {
t.Helper()
if len(decisions) != len(want) {
t.Fatalf("len(Decisions) = %d, want %d", len(decisions), len(want))
}
for i := range want {
if decisions[i].CandidateIndex != want[i] {
t.Fatalf("Decisions[%d].CandidateIndex = %d, want %d", i, decisions[i].CandidateIndex, want[i])
}
}
}

View File

@@ -0,0 +1,209 @@
package seriatim
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"math"
"math/big"
"strconv"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const Key = "seriatim"
const (
DocumentKind = "transcript"
UnitKind = "transcript_segment"
Format = "application/vnd.seriatim+json"
)
var providedCapabilities = []string{
"source.transcript",
"transcript.speaker",
"transcript.timestamps",
}
var _ contracts.InputAdapter = (*Adapter)(nil)
type Adapter struct{}
func New() *Adapter {
return &Adapter{}
}
func (a *Adapter) Key() string {
return Key
}
func (a *Adapter) Parse(ctx context.Context, req contracts.ParseRequest) (*source.SourceDocument, error) {
if ctx == nil {
return nil, inputErrorf("context must not be nil")
}
if err := ctx.Err(); err != nil {
return nil, inputErrorf("context error before parsing: %w", err)
}
if len(req.Raw) == 0 {
return nil, inputErrorf("raw input must not be empty")
}
parsed, err := decodeTranscript(req.Raw)
if err != nil {
return nil, inputErrorf("parse JSON: %w", err)
}
if len(parsed.Segments) == 0 {
return nil, inputErrorf("segments must not be empty")
}
rawDigest := digest(req.Raw)
doc := &source.SourceDocument{
ID: documentID(req.SourceID, parsed.Metadata, rawDigest),
Kind: DocumentKind,
Format: Format,
Digest: rawDigest,
Metadata: copyMetadata(parsed.Metadata),
}
seenSegmentIDs := make(map[string]struct{}, len(parsed.Segments))
for i, segment := range parsed.Segments {
unit, err := sourceUnit(segment, i, seenSegmentIDs)
if err != nil {
return nil, err
}
doc.Units = append(doc.Units, unit)
}
if err := source.ValidateDocument(doc); err != nil {
return nil, inputErrorf("validate source document: %w", err)
}
return doc, nil
}
func ModuleSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageInput,
Provides: append([]string(nil), providedCapabilities...),
}
}
func Register(registry *pipeline.InputAdapterRegistry) error {
return registry.RegisterWithSpec(ModuleSpec(), func() (contracts.InputAdapter, error) {
return New(), nil
})
}
func sourceUnit(segment segment, index int, seen map[string]struct{}) (source.SourceUnit, error) {
segmentLabel := fmt.Sprintf("segment[%d]", index)
segmentID := strings.TrimSpace(segment.ID)
if segmentID == "" {
return source.SourceUnit{}, inputErrorf("%s id must not be empty", segmentLabel)
}
if segmentID != segment.ID {
return source.SourceUnit{}, inputErrorf("%s id %q must not contain leading or trailing whitespace", segmentLabel, segment.ID)
}
if _, ok := seen[segment.ID]; ok {
return source.SourceUnit{}, inputErrorf("segment id %q is duplicated", segment.ID)
}
seen[segment.ID] = struct{}{}
speaker := strings.TrimSpace(segment.Speaker)
if speaker == "" {
return source.SourceUnit{}, inputErrorf("segment %q speaker must not be empty", segment.ID)
}
start, err := validTimestamp(segment.Start, fmt.Sprintf("segment %q start", segment.ID))
if err != nil {
return source.SourceUnit{}, err
}
end, err := validTimestamp(segment.End, fmt.Sprintf("segment %q end", segment.ID))
if err != nil {
return source.SourceUnit{}, err
}
if end.Cmp(start) < 0 {
return source.SourceUnit{}, inputErrorf("segment %q end must be greater than or equal to start", segment.ID)
}
if strings.TrimSpace(segment.Text) == "" {
return source.SourceUnit{}, inputErrorf("segment %q text must not be empty", segment.ID)
}
return source.SourceUnit{
ID: segment.ID,
Kind: UnitKind,
Text: segment.Text,
Metadata: map[string]any{
MetadataSpeaker: segment.Speaker,
MetadataStart: segment.Start,
MetadataEnd: segment.End,
},
}, nil
}
func validTimestamp(value fmt.Stringer, label string) (*big.Rat, error) {
raw := strings.TrimSpace(value.String())
if raw == "" {
return nil, inputErrorf("%s must not be empty", label)
}
parsed, err := strconv.ParseFloat(raw, 64)
if err != nil {
return nil, inputErrorf("%s must be a valid number: %w", label, err)
}
if math.IsInf(parsed, 0) || math.IsNaN(parsed) {
return nil, inputErrorf("%s must be finite", label)
}
if parsed < 0 {
return nil, inputErrorf("%s must not be negative", label)
}
rat, ok := new(big.Rat).SetString(raw)
if !ok {
return nil, inputErrorf("%s must be a valid number", label)
}
return rat, nil
}
func documentID(requestedID string, metadata map[string]any, rawDigest string) string {
if id := strings.TrimSpace(requestedID); id != "" {
return id
}
if id := stringMetadata(metadata, "id"); id != "" {
return id
}
if id := stringMetadata(metadata, "source_id"); id != "" {
return id
}
return "seriatim:" + strings.TrimPrefix(rawDigest, "sha256:")[:16]
}
func stringMetadata(metadata map[string]any, key string) string {
value, ok := metadata[key].(string)
if !ok {
return ""
}
return strings.TrimSpace(value)
}
func copyMetadata(metadata map[string]any) map[string]any {
if len(metadata) == 0 {
return nil
}
copied := make(map[string]any, len(metadata))
for key, value := range metadata {
copied[key] = value
}
return copied
}
func digest(raw []byte) string {
sum := sha256.Sum256(raw)
return "sha256:" + hex.EncodeToString(sum[:])
}
func inputErrorf(format string, args ...any) error {
return fmt.Errorf("seriatim input: "+format, args...)
}

View File

@@ -0,0 +1,324 @@
package seriatim
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"os"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
func TestParseValidMinimalTranscript(t *testing.T) {
raw := readFixture(t, "testdata/valid_minimal.json")
doc, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw})
if err != nil {
t.Fatalf("Parse() error = %v, want nil", err)
}
if doc.ID != "session-alpha" {
t.Fatalf("doc.ID = %q, want session-alpha", doc.ID)
}
if doc.Kind != DocumentKind {
t.Fatalf("doc.Kind = %q, want %q", doc.Kind, DocumentKind)
}
if doc.Format != Format {
t.Fatalf("doc.Format = %q, want %q", doc.Format, Format)
}
if doc.Digest != testDigest(raw) {
t.Fatalf("doc.Digest = %q, want %q", doc.Digest, testDigest(raw))
}
if got := doc.Metadata["title"]; got != "Synthetic session transcript" {
t.Fatalf("doc.Metadata[title] = %#v, want Synthetic session transcript", got)
}
if len(doc.Units) != 2 {
t.Fatalf("len(doc.Units) = %d, want 2", len(doc.Units))
}
first := doc.Units[0]
if first.ID != "seg-001" {
t.Fatalf("first.ID = %q, want seg-001", first.ID)
}
if first.Kind != UnitKind {
t.Fatalf("first.Kind = %q, want %q", first.Kind, UnitKind)
}
if first.Text != "The stone door opens." {
t.Fatalf("first.Text = %q, want fixture text", first.Text)
}
if speaker, ok := Speaker(first); !ok || speaker != "Narrator" {
t.Fatalf("Speaker(first) = %q, %v; want Narrator, true", speaker, ok)
}
if start, ok := Start(first); !ok || start != json.Number("0") {
t.Fatalf("Start(first) = %q, %v; want 0, true", start, ok)
}
if end, ok := End(first); !ok || end != json.Number("4.5") {
t.Fatalf("End(first) = %q, %v; want 4.5, true", end, ok)
}
ref := source.SourceRef{
SourceID: doc.ID,
StartUnitID: doc.Units[0].ID,
EndUnitID: doc.Units[len(doc.Units)-1].ID,
}
if err := source.ValidateRef(doc, ref); err != nil {
t.Fatalf("ValidateRef() error = %v, want nil", err)
}
}
func TestParseAcceptsNumericSegmentIDs(t *testing.T) {
raw := []byte(`{"metadata":{"application":"seriatim","version":"v1.5.0","output_schema":"seriatim-intermediate"},"segments":[{"id":1,"start":451.821,"end":469.685,"speaker":"Narrator","text":"The stone door opens.","categories":["scene"]},{"id":2,"start":470,"end":471,"speaker":"Player","text":"I step inside."}]}`)
doc, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw})
if err != nil {
t.Fatalf("Parse() error = %v, want nil", err)
}
if got, want := doc.Metadata["output_schema"], "seriatim-intermediate"; got != want {
t.Fatalf("doc.Metadata[output_schema] = %#v, want %q", got, want)
}
if doc.Format != Format {
t.Fatalf("doc.Format = %q, want %q", doc.Format, Format)
}
if len(doc.Units) != 2 {
t.Fatalf("len(doc.Units) = %d, want 2", len(doc.Units))
}
if doc.Units[0].ID != "1" || doc.Units[1].ID != "2" {
t.Fatalf("unit IDs = %#v, want numeric IDs normalized to strings", []string{doc.Units[0].ID, doc.Units[1].ID})
}
ref := source.SourceRef{
SourceID: doc.ID,
StartUnitID: "1",
EndUnitID: "2",
}
if err := source.ValidateRef(doc, ref); err != nil {
t.Fatalf("ValidateRef() error = %v, want nil", err)
}
}
func TestParseRequestSourceIDOverridesMetadataIDs(t *testing.T) {
raw := readFixture(t, "testdata/valid_minimal.json")
doc, err := New().Parse(context.Background(), contracts.ParseRequest{
SourceID: " requested-source ",
Raw: raw,
})
if err != nil {
t.Fatalf("Parse() error = %v, want nil", err)
}
if doc.ID != "requested-source" {
t.Fatalf("doc.ID = %q, want requested-source", doc.ID)
}
}
func TestParseFallbackDocumentIDIsDeterministic(t *testing.T) {
raw := []byte(`{"metadata":{},"segments":[{"id":"s1","start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."}]}`)
first, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw})
if err != nil {
t.Fatalf("first Parse() error = %v, want nil", err)
}
second, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw})
if err != nil {
t.Fatalf("second Parse() error = %v, want nil", err)
}
if first.ID != second.ID {
t.Fatalf("fallback IDs differ: %q vs %q", first.ID, second.ID)
}
if !strings.HasPrefix(first.ID, "seriatim:") {
t.Fatalf("fallback ID = %q, want seriatim prefix", first.ID)
}
if first.ID != "seriatim:"+strings.TrimPrefix(testDigest(raw), "sha256:")[:16] {
t.Fatalf("fallback ID = %q, want digest-derived ID", first.ID)
}
}
func TestParseUsesMetadataSourceIDWhenMetadataIDIsAbsent(t *testing.T) {
raw := []byte(`{"metadata":{"source_id":" source-from-metadata "},"segments":[{"id":"s1","start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."}]}`)
doc, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw})
if err != nil {
t.Fatalf("Parse() error = %v, want nil", err)
}
if doc.ID != "source-from-metadata" {
t.Fatalf("doc.ID = %q, want source-from-metadata", doc.ID)
}
}
func TestParseRejectsInvalidInput(t *testing.T) {
tests := []struct {
name string
raw []byte
wantErr []string
}{
{
name: "malformed JSON",
raw: []byte(`{"metadata":`),
wantErr: []string{"seriatim input", "parse JSON"},
},
{
name: "trailing JSON",
raw: []byte(`{"metadata":{},"segments":[]} {}`),
wantErr: []string{"seriatim input", "trailing"},
},
{
name: "missing metadata",
raw: []byte(`{"segments":[]}`),
wantErr: []string{"metadata"},
},
{
name: "null metadata",
raw: []byte(`{"metadata":null,"segments":[]}`),
wantErr: []string{"metadata", "object"},
},
{
name: "metadata wrong type",
raw: []byte(`{"metadata":[],"segments":[]}`),
wantErr: []string{"metadata", "object"},
},
{
name: "missing segments",
raw: []byte(`{"metadata":{}}`),
wantErr: []string{"segments"},
},
{
name: "null segments",
raw: []byte(`{"metadata":{},"segments":null}`),
wantErr: []string{"segments", "array"},
},
{
name: "segments wrong type",
raw: []byte(`{"metadata":{},"segments":{}}`),
wantErr: []string{"segments", "array"},
},
{
name: "empty segments",
raw: []byte(`{"metadata":{},"segments":[]}`),
wantErr: []string{"segments", "empty"},
},
{
name: "missing segment id",
raw: validJSONWithSegment(`"start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"id", "empty"},
},
{
name: "invalid segment id type",
raw: validJSONWithSegment(`"id":{},"start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"id", "string or number"},
},
{
name: "whitespace segment id",
raw: validJSONWithSegment(`"id":" s1 ","start":0,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"id", "whitespace"},
},
{
name: "empty text",
raw: validJSONWithSegment(`"id":"s1","start":0,"end":1,"speaker":"Narrator","text":" "`),
wantErr: []string{"text", "empty"},
},
{
name: "missing speaker",
raw: validJSONWithSegment(`"id":"s1","start":0,"end":1,"text":"Synthetic text."`),
wantErr: []string{"speaker", "empty"},
},
{
name: "missing start",
raw: validJSONWithSegment(`"id":"s1","end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"start", "empty"},
},
{
name: "missing end",
raw: validJSONWithSegment(`"id":"s1","start":0,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"end", "empty"},
},
{
name: "negative start",
raw: validJSONWithSegment(`"id":"s1","start":-1,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"start", "negative"},
},
{
name: "non-numeric end",
raw: validJSONWithSegment(`"id":"s1","start":0,"end":"late","speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"segment[0]", "end", "number"},
},
{
name: "non-finite timestamp",
raw: validJSONWithSegment(`"id":"s1","start":1e10000,"end":1e10000,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"start", "valid number"},
},
{
name: "end before start",
raw: validJSONWithSegment(`"id":"s1","start":2,"end":1,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"end", "start"},
},
{
name: "end before start beyond float precision",
raw: validJSONWithSegment(`"id":"s1","start":9007199254740993,"end":9007199254740992,"speaker":"Narrator","text":"Synthetic text."`),
wantErr: []string{"end", "start"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: tt.raw})
if err == nil {
t.Fatal("Parse() error = nil, want error")
}
for _, want := range tt.wantErr {
if !strings.Contains(err.Error(), want) {
t.Fatalf("Parse() error = %q, want substring %q", err.Error(), want)
}
}
})
}
}
func TestParseRejectsDuplicateSegmentIDs(t *testing.T) {
raw := readFixture(t, "testdata/duplicate_segment_id.json")
_, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw})
if err == nil {
t.Fatal("Parse() error = nil, want duplicate ID error")
}
if !strings.Contains(err.Error(), "duplicated") || !strings.Contains(err.Error(), "seg-001") {
t.Fatalf("Parse() error = %q, want duplicate segment context", err.Error())
}
}
func TestParseRejectsInvalidContextOrEmptyInput(t *testing.T) {
if _, err := New().Parse(nil, contracts.ParseRequest{Raw: []byte(`{}`)}); err == nil {
t.Fatal("Parse(nil context) error = nil, want error")
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
if _, err := New().Parse(ctx, contracts.ParseRequest{Raw: []byte(`{}`)}); err == nil {
t.Fatal("Parse(canceled context) error = nil, want error")
}
if _, err := New().Parse(context.Background(), contracts.ParseRequest{}); err == nil {
t.Fatal("Parse(empty input) error = nil, want error")
}
}
func readFixture(t *testing.T, path string) []byte {
t.Helper()
raw, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile(%q) error = %v, want nil", path, err)
}
return raw
}
func validJSONWithSegment(segmentFields string) []byte {
return []byte(`{"metadata":{"id":"fixture"},"segments":[{` + segmentFields + `}]}`)
}
func testDigest(raw []byte) string {
sum := sha256.Sum256(raw)
return "sha256:" + hex.EncodeToString(sum[:])
}

View File

@@ -0,0 +1,252 @@
package seriatim
import (
"context"
"os"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/merge/appendorder"
"gitea.maximumdirect.net/eric/notarius/internal/modules/normalize/noop"
)
func TestPipelineConfigLoadsAndResolvesWithSeriatimInput(t *testing.T) {
cfg := loadPipelineConfig(t)
resolved, err := cfg.Resolve(config.ResolveInput{
PipelineID: "seriatim-fixture",
Catalog: seriatimTestCatalog(t, ModuleSpec()),
})
if err != nil {
t.Fatalf("Resolve() error = %v, want nil", err)
}
if resolved.ResolvedPipeline.Input.Module != Key {
t.Fatalf("resolved input module = %q, want %q", resolved.ResolvedPipeline.Input.Module, Key)
}
if resolved.ResolvedPipeline.Digest == "" {
t.Fatal("resolved digest is empty")
}
again, err := cfg.Resolve(config.ResolveInput{
PipelineID: "seriatim-fixture",
Catalog: seriatimTestCatalog(t, ModuleSpec()),
})
if err != nil {
t.Fatalf("second Resolve() error = %v, want nil", err)
}
if resolved.ResolvedPipeline.Digest != again.ResolvedPipeline.Digest {
t.Fatalf("resolved digest = %q, second digest = %q; want stable digest", resolved.ResolvedPipeline.Digest, again.ResolvedPipeline.Digest)
}
}
func TestPipelineConfigRejectsMissingSeriatimCapability(t *testing.T) {
spec := ModuleSpec()
spec.Provides = withoutCapability(spec.Provides, "transcript.timestamps")
cfg := loadPipelineConfig(t)
_, err := cfg.Resolve(config.ResolveInput{
PipelineID: "seriatim-fixture",
Catalog: seriatimTestCatalog(t, spec),
})
if err == nil {
t.Fatal("Resolve() error = nil, want missing capability error")
}
if !strings.Contains(err.Error(), "missing capability") || !strings.Contains(err.Error(), "transcript.timestamps") {
t.Fatalf("Resolve() error = %q, want missing transcript.timestamps capability", err.Error())
}
}
func TestPipelineConfigRejectsUnknownLaneSelection(t *testing.T) {
cfg := loadPipelineConfig(t)
_, err := cfg.Resolve(config.ResolveInput{
PipelineID: "seriatim-fixture",
Only: []string{"missing"},
Catalog: seriatimTestCatalog(t, ModuleSpec()),
})
if err == nil {
t.Fatal("Resolve() error = nil, want unknown lane error")
}
if !strings.Contains(err.Error(), "selected artifact lane") || !strings.Contains(err.Error(), "missing") {
t.Fatalf("Resolve() error = %q, want unknown lane context", err.Error())
}
}
func loadPipelineConfig(t *testing.T) config.Config {
t.Helper()
data, err := os.ReadFile("testdata/pipeline.yml")
if err != nil {
t.Fatalf("ReadFile(pipeline.yml) error = %v, want nil", err)
}
fileCfg, err := config.ParseFileConfigYAML(data)
if err != nil {
t.Fatalf("ParseFileConfigYAML() error = %v, want nil", err)
}
cfg := config.Default()
if err := cfg.ApplyFileConfig(fileCfg); err != nil {
t.Fatalf("ApplyFileConfig() error = %v, want nil", err)
}
profile, ok := cfg.Pipelines["seriatim-fixture"]
if !ok {
t.Fatal("pipeline seriatim-fixture was not loaded")
}
if profile.Input.Module != Key {
t.Fatalf("loaded input module = %q, want %q", profile.Input.Module, Key)
}
return cfg
}
func seriatimTestCatalog(t *testing.T, inputSpec pipeline.ModuleSpec) pipeline.ModuleCatalog {
t.Helper()
inputs := pipeline.NewInputAdapterRegistry()
chunkers := pipeline.NewChunkerRegistry()
extractors := pipeline.NewExtractorRegistry()
mergers := pipeline.NewMergerRegistry()
normalizers := pipeline.NewNormalizerRegistry()
outputs := pipeline.NewOutputEncoderRegistry()
if reflect.DeepEqual(inputSpec, ModuleSpec()) {
if err := Register(inputs); err != nil {
t.Fatalf("register seriatim input: %v", err)
}
} else if err := inputs.RegisterWithSpec(inputSpec, func() (contracts.InputAdapter, error) {
return New(), nil
}); err != nil {
t.Fatalf("register seriatim input override: %v", err)
}
mustRegisterChunker(t, chunkers, pipeline.ModuleSpec{
Key: "fake/chunk",
Stage: pipeline.StageChunk,
Requires: []string{"source.transcript"},
Provides: []string{"chunks"},
})
mustRegisterExtractor(t, extractors, pipeline.ModuleSpec{
Key: "fake/extract",
Stage: pipeline.StageExtract,
Requires: []string{"chunks", "transcript.speaker", "transcript.timestamps"},
Provides: []string{"fake.artifacts"},
})
mustRegisterMerger(t, mergers, pipeline.ModuleSpec{
Key: pipeline.DefaultMergeModule,
Stage: pipeline.StageMerge,
Requires: []string{"fake.artifacts"},
})
mustRegisterNormalizer(t, normalizers, pipeline.ModuleSpec{
Key: pipeline.DefaultNormalizeModule,
Stage: pipeline.StageNormalize,
})
mustRegisterOutput(t, outputs, pipeline.ModuleSpec{
Key: pipeline.DefaultOutputModule,
Stage: pipeline.StageOutput,
})
return pipeline.ModuleCatalog{
Inputs: inputs,
Chunkers: chunkers,
Extractors: extractors,
Mergers: mergers,
Normalizers: normalizers,
Outputs: outputs,
}
}
func mustRegisterChunker(t *testing.T, registry *pipeline.ChunkerRegistry, spec pipeline.ModuleSpec) {
t.Helper()
if err := registry.RegisterWithSpec(spec, func() (contracts.Chunker, error) {
return fakeChunker{}, nil
}); err != nil {
t.Fatalf("register chunker: %v", err)
}
}
func mustRegisterExtractor(t *testing.T, registry *pipeline.ExtractorRegistry, spec pipeline.ModuleSpec) {
t.Helper()
if err := registry.RegisterWithSpec(spec, func() (contracts.Extractor, error) {
return fakeExtractor{}, nil
}); err != nil {
t.Fatalf("register extractor: %v", err)
}
}
func mustRegisterMerger(t *testing.T, registry *pipeline.MergerRegistry, spec pipeline.ModuleSpec) {
t.Helper()
if err := registry.RegisterWithSpec(spec, func() (contracts.Merger, error) {
return appendorder.New(), nil
}); err != nil {
t.Fatalf("register merger: %v", err)
}
}
func mustRegisterNormalizer(t *testing.T, registry *pipeline.NormalizerRegistry, spec pipeline.ModuleSpec) {
t.Helper()
if err := registry.RegisterWithSpec(spec, func() (contracts.Normalizer, error) {
return noop.New(), nil
}); err != nil {
t.Fatalf("register normalizer: %v", err)
}
}
func mustRegisterOutput(t *testing.T, registry *pipeline.OutputEncoderRegistry, spec pipeline.ModuleSpec) {
t.Helper()
if err := registry.RegisterWithSpec(spec, func() (contracts.OutputEncoder, error) {
return fakeOutput{}, nil
}); err != nil {
t.Fatalf("register output: %v", err)
}
}
type fakeChunker struct{}
func (fakeChunker) Key() string { return "fake/chunk" }
func (fakeChunker) Chunk(ctx context.Context, req contracts.ChunkRequest) (contracts.ChunkResult, error) {
return contracts.ChunkResult{}, nil
}
type fakeExtractor struct{}
func (fakeExtractor) Key() string { return "fake/extract" }
func (fakeExtractor) ArtifactType() string { return "fake" }
func (fakeExtractor) SchemaVersion() string { return "v1" }
func (fakeExtractor) Validators() []contracts.Validator { return nil }
func (fakeExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
return contracts.ExtractionResult{}, nil
}
type fakeOutput struct{}
func (fakeOutput) Key() string { return pipeline.DefaultOutputModule }
func (fakeOutput) Encode(ctx context.Context, req contracts.OutputRequest) (contracts.OutputResult, error) {
return contracts.OutputResult{}, nil
}
func withoutCapability(capabilities []string, capability string) []string {
filtered := make([]string, 0, len(capabilities))
for _, candidate := range capabilities {
if candidate != capability {
filtered = append(filtered, candidate)
}
}
return filtered
}
var (
_ contracts.Chunker = fakeChunker{}
_ contracts.Extractor = fakeExtractor{}
_ contracts.OutputEncoder = fakeOutput{}
)

View File

@@ -0,0 +1,28 @@
package seriatim
import (
"encoding/json"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
)
const (
MetadataSpeaker = "speaker"
MetadataStart = "start"
MetadataEnd = "end"
)
func Speaker(unit source.SourceUnit) (string, bool) {
value, ok := unit.Metadata[MetadataSpeaker].(string)
return value, ok
}
func Start(unit source.SourceUnit) (json.Number, bool) {
value, ok := unit.Metadata[MetadataStart].(json.Number)
return value, ok
}
func End(unit source.SourceUnit) (json.Number, bool) {
value, ok := unit.Metadata[MetadataEnd].(json.Number)
return value, ok
}

View File

@@ -0,0 +1,148 @@
package seriatim
import (
"bytes"
"encoding/json"
"fmt"
"io"
)
type transcript struct {
Metadata map[string]any `json:"metadata"`
Segments []segment `json:"segments"`
}
type segment struct {
ID string `json:"id"`
Start json.Number `json:"start"`
End json.Number `json:"end"`
Speaker string `json:"speaker"`
Text string `json:"text"`
}
func decodeTranscript(raw []byte) (transcript, error) {
var fields map[string]json.RawMessage
if err := decodeJSON(raw, &fields); err != nil {
return transcript{}, err
}
if fields == nil {
return transcript{}, fmt.Errorf("top-level value must be an object")
}
metadataRaw, ok := fields["metadata"]
if !ok {
return transcript{}, fmt.Errorf("metadata is required")
}
var metadata map[string]any
if err := decodeJSON(metadataRaw, &metadata); err != nil {
return transcript{}, fmt.Errorf("metadata must be an object: %w", err)
}
if metadata == nil {
return transcript{}, fmt.Errorf("metadata must be an object")
}
segmentsRaw, ok := fields["segments"]
if !ok {
return transcript{}, fmt.Errorf("segments are required")
}
var segmentValues []json.RawMessage
if err := decodeJSON(segmentsRaw, &segmentValues); err != nil {
return transcript{}, fmt.Errorf("segments must be an array: %w", err)
}
if segmentValues == nil {
return transcript{}, fmt.Errorf("segments must be an array")
}
segments := make([]segment, 0, len(segmentValues))
for i, rawSegment := range segmentValues {
segment, err := decodeSegment(rawSegment, i)
if err != nil {
return transcript{}, err
}
segments = append(segments, segment)
}
return transcript{
Metadata: metadata,
Segments: segments,
}, nil
}
func decodeSegment(raw []byte, index int) (segment, error) {
var fields map[string]json.RawMessage
if err := decodeJSON(raw, &fields); err != nil {
return segment{}, fmt.Errorf("segment[%d] must be an object: %w", index, err)
}
if fields == nil {
return segment{}, fmt.Errorf("segment[%d] must be an object", index)
}
var decoded segment
if err := decodeOptionalSegmentID(fields, "id", &decoded.ID); err != nil {
return segment{}, fmt.Errorf("segment[%d] id must be a string or number: %w", index, err)
}
if err := decodeOptionalNumber(fields, "start", &decoded.Start); err != nil {
return segment{}, fmt.Errorf("segment[%d] start must be a number: %w", index, err)
}
if err := decodeOptionalNumber(fields, "end", &decoded.End); err != nil {
return segment{}, fmt.Errorf("segment[%d] end must be a number: %w", index, err)
}
if err := decodeOptionalString(fields, "speaker", &decoded.Speaker); err != nil {
return segment{}, fmt.Errorf("segment[%d] speaker must be a string: %w", index, err)
}
if err := decodeOptionalString(fields, "text", &decoded.Text); err != nil {
return segment{}, fmt.Errorf("segment[%d] text must be a string: %w", index, err)
}
return decoded, nil
}
func decodeOptionalString(fields map[string]json.RawMessage, key string, out *string) error {
raw, ok := fields[key]
if !ok {
return nil
}
return decodeJSON(raw, out)
}
func decodeOptionalSegmentID(fields map[string]json.RawMessage, key string, out *string) error {
raw, ok := fields[key]
if !ok {
return nil
}
var text string
if err := decodeJSON(raw, &text); err == nil {
*out = text
return nil
}
var number json.Number
if err := decodeJSON(raw, &number); err == nil {
*out = number.String()
return nil
}
return fmt.Errorf("must be a string or number")
}
func decodeOptionalNumber(fields map[string]json.RawMessage, key string, out *json.Number) error {
raw, ok := fields[key]
if !ok {
return nil
}
return decodeJSON(raw, out)
}
func decodeJSON(raw []byte, out any) error {
decoder := json.NewDecoder(bytes.NewReader(raw))
decoder.UseNumber()
if err := decoder.Decode(out); err != nil {
return err
}
if err := decoder.Decode(&struct{}{}); err != io.EOF {
if err == nil {
return fmt.Errorf("unexpected trailing JSON value")
}
return err
}
return nil
}

View File

@@ -0,0 +1,106 @@
package seriatim
import (
"encoding/json"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestNewReturnsAdapterWithKey(t *testing.T) {
adapter := New()
if adapter == nil {
t.Fatal("New() = nil, want adapter")
}
if adapter.Key() != Key {
t.Fatalf("adapter.Key() = %q, want %q", adapter.Key(), Key)
}
}
func TestModuleSpec(t *testing.T) {
got := ModuleSpec()
want := pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageInput,
Provides: []string{
"source.transcript",
"transcript.speaker",
"transcript.timestamps",
},
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("ModuleSpec() = %#v, want %#v", got, want)
}
got.Provides[0] = "changed"
again := ModuleSpec()
if !reflect.DeepEqual(again, want) {
t.Fatalf("ModuleSpec() after caller mutation = %#v, want %#v", again, want)
}
}
func TestRegisterMakesAdapterBuildable(t *testing.T) {
registry := pipeline.NewInputAdapterRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
adapter, err := registry.Build(Key)
if err != nil {
t.Fatalf("Build() error = %v, want nil", err)
}
if adapter.Key() != Key {
t.Fatalf("adapter.Key() = %q, want %q", adapter.Key(), Key)
}
}
func TestRegisterStoresModuleSpec(t *testing.T) {
registry := pipeline.NewInputAdapterRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
got, ok := registry.Spec(Key)
if !ok {
t.Fatal("Spec() ok = false, want true")
}
want := ModuleSpec()
if !reflect.DeepEqual(got, want) {
t.Fatalf("Spec() = %#v, want %#v", got, want)
}
}
func TestRegisterNilRegistryReturnsError(t *testing.T) {
err := Register(nil)
if err == nil {
t.Fatal("Register(nil) error = nil, want error")
}
if !strings.Contains(err.Error(), "input adapter registry") {
t.Fatalf("Register(nil) error = %q, want registry context", err.Error())
}
}
func TestMetadataHelpers(t *testing.T) {
unit := source.SourceUnit{
Metadata: map[string]any{
MetadataSpeaker: "Narrator",
MetadataStart: json.Number("1.25"),
MetadataEnd: json.Number("2.5"),
},
}
if got, ok := Speaker(unit); !ok || got != "Narrator" {
t.Fatalf("Speaker() = %q, %v; want Narrator, true", got, ok)
}
if got, ok := Start(unit); !ok || got != json.Number("1.25") {
t.Fatalf("Start() = %q, %v; want 1.25, true", got, ok)
}
if got, ok := End(unit); !ok || got != json.Number("2.5") {
t.Fatalf("End() = %q, %v; want 2.5, true", got, ok)
}
}

View File

@@ -0,0 +1,273 @@
package seriatim
import (
"context"
"encoding/json"
"fmt"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/merge/appendorder"
"gitea.maximumdirect.net/eric/notarius/internal/modules/normalize/noop"
)
func TestRunnerProcessesSeriatimInputWithFakeModules(t *testing.T) {
raw := readFixture(t, "testdata/valid_minimal.json")
expectedDoc, err := New().Parse(context.Background(), contracts.ParseRequest{Raw: raw})
if err != nil {
t.Fatalf("Parse() error = %v, want nil", err)
}
resolved, err := loadPipelineConfig(t).Resolve(configResolveInput(t))
if err != nil {
t.Fatalf("Resolve() error = %v, want nil", err)
}
extractor := &runnerSeriatimExtractor{}
output, err := pipeline.New(seriatimRunnerRegistries(t, extractor)).Run(context.Background(), pipeline.RunInput{
Pipeline: resolved.ResolvedPipeline,
RawInput: raw,
})
if err != nil {
t.Fatalf("Run() error = %v, want nil", err)
}
if output.Manifest.InputModule != Key {
t.Fatalf("manifest input module = %q, want %q", output.Manifest.InputModule, Key)
}
if got := output.Manifest.SourceDigests; len(got) != 1 || got[0] != expectedDoc.Digest {
t.Fatalf("manifest source digests = %#v, want %q", got, expectedDoc.Digest)
}
if len(output.Approved) != 1 {
t.Fatalf("len(Approved) = %d, want 1", len(output.Approved))
}
artifact := output.Approved[0]
if artifact.ExtractorKey != "fake/extract" || artifact.ArtifactType != "fake.event" || artifact.SchemaVersion != "v1" {
t.Fatalf("approved artifact envelope = %#v, want fake extractor envelope", artifact)
}
if len(artifact.SourceRefs) != 1 {
t.Fatalf("len(SourceRefs) = %d, want 1", len(artifact.SourceRefs))
}
if err := source.ValidateRef(expectedDoc, artifact.SourceRefs[0]); err != nil {
t.Fatalf("ValidateRef() error = %v, want nil", err)
}
if artifact.SourceRefs[0].StartUnitID != "seg-001" || artifact.SourceRefs[0].EndUnitID != "seg-002" {
t.Fatalf("SourceRefs[0] = %#v, want Seriatim unit IDs", artifact.SourceRefs[0])
}
if extractor.calls != 1 {
t.Fatalf("extractor calls = %d, want 1", extractor.calls)
}
if len(output.OutputFiles) != 1 {
t.Fatalf("len(OutputFiles) = %d, want 1", len(output.OutputFiles))
}
if output.OutputFiles[0].ContentType != "application/json" {
t.Fatalf("ContentType = %q, want application/json", output.OutputFiles[0].ContentType)
}
}
func TestRunnerFailsOnInvalidSeriatimInput(t *testing.T) {
resolved, err := loadPipelineConfig(t).Resolve(configResolveInput(t))
if err != nil {
t.Fatalf("Resolve() error = %v, want nil", err)
}
output, err := pipeline.New(seriatimRunnerRegistries(t, &runnerSeriatimExtractor{})).Run(context.Background(), pipeline.RunInput{
Pipeline: resolved.ResolvedPipeline,
RawInput: []byte(`{"metadata":{},"segments":[]}`),
SourceID: "invalid-source",
LLMClient: nil,
})
if err == nil {
t.Fatal("Run() error = nil, want invalid input error")
}
if !strings.Contains(err.Error(), "parse input with adapter") || !strings.Contains(err.Error(), "seriatim input") {
t.Fatalf("Run() error = %q, want Seriatim parse context", err.Error())
}
if output.Manifest.ValidationStatus != "failed" {
t.Fatalf("ValidationStatus = %q, want failed", output.Manifest.ValidationStatus)
}
}
func configResolveInput(t *testing.T) config.ResolveInput {
t.Helper()
return config.ResolveInput{
PipelineID: "seriatim-fixture",
Catalog: seriatimTestCatalog(t, ModuleSpec()),
}
}
func seriatimRunnerRegistries(t *testing.T, extractor contracts.Extractor) pipeline.Registries {
t.Helper()
inputs := pipeline.NewInputAdapterRegistry()
chunkers := pipeline.NewChunkerRegistry()
extractors := pipeline.NewExtractorRegistry()
mergers := pipeline.NewMergerRegistry()
normalizers := pipeline.NewNormalizerRegistry()
outputs := pipeline.NewOutputEncoderRegistry()
if err := Register(inputs); err != nil {
t.Fatalf("register seriatim input: %v", err)
}
if err := chunkers.Register("fake/chunk", func() (contracts.Chunker, error) {
return runnerSeriatimChunker{}, nil
}); err != nil {
t.Fatalf("register chunker: %v", err)
}
if err := extractors.Register("fake/extract", func() (contracts.Extractor, error) {
return extractor, nil
}); err != nil {
t.Fatalf("register extractor: %v", err)
}
if err := mergers.Register(pipeline.DefaultMergeModule, func() (contracts.Merger, error) {
return appendorder.New(), nil
}); err != nil {
t.Fatalf("register merger: %v", err)
}
if err := normalizers.Register(pipeline.DefaultNormalizeModule, func() (contracts.Normalizer, error) {
return noop.New(), nil
}); err != nil {
t.Fatalf("register normalizer: %v", err)
}
if err := outputs.Register(pipeline.DefaultOutputModule, func() (contracts.OutputEncoder, error) {
return runnerSeriatimOutput{}, nil
}); err != nil {
t.Fatalf("register output: %v", err)
}
return pipeline.Registries{
Inputs: inputs,
Chunkers: chunkers,
Extractors: extractors,
Mergers: mergers,
Normalizers: normalizers,
Outputs: outputs,
}
}
type runnerSeriatimChunker struct{}
func (runnerSeriatimChunker) Key() string {
return "fake/chunk"
}
func (runnerSeriatimChunker) Chunk(ctx context.Context, req contracts.ChunkRequest) (contracts.ChunkResult, error) {
return contracts.ChunkResult{
Chunks: []contracts.SourceChunk{
{
ID: req.Source.ID + ":chunk:0",
SourceID: req.Source.ID,
Index: 0,
Units: append([]source.SourceUnit(nil), req.Source.Units...),
},
},
}, nil
}
type runnerSeriatimExtractor struct {
calls int
}
func (e *runnerSeriatimExtractor) Key() string {
return "fake/extract"
}
func (e *runnerSeriatimExtractor) ArtifactType() string {
return "fake.event"
}
func (e *runnerSeriatimExtractor) SchemaVersion() string {
return "v1"
}
func (e *runnerSeriatimExtractor) Validators() []contracts.Validator {
return nil
}
func (e *runnerSeriatimExtractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
e.calls++
if req.Source == nil {
return contracts.ExtractionResult{}, fmt.Errorf("source must not be nil")
}
if req.Chunk == nil {
return contracts.ExtractionResult{}, fmt.Errorf("chunk must not be nil")
}
if got := unitIDs(req.Source.Units); !equalStrings(got, []string{"seg-001", "seg-002"}) {
return contracts.ExtractionResult{}, fmt.Errorf("source unit IDs = %#v, want Seriatim segment IDs", got)
}
if got := unitIDs(req.Chunk.Units); !equalStrings(got, []string{"seg-001", "seg-002"}) {
return contracts.ExtractionResult{}, fmt.Errorf("chunk unit IDs = %#v, want Seriatim segment IDs", got)
}
for _, unit := range req.Chunk.Units {
if speaker, ok := Speaker(unit); !ok || speaker == "" {
return contracts.ExtractionResult{}, fmt.Errorf("unit %q missing speaker metadata", unit.ID)
}
if _, ok := Start(unit); !ok {
return contracts.ExtractionResult{}, fmt.Errorf("unit %q missing start metadata", unit.ID)
}
if _, ok := End(unit); !ok {
return contracts.ExtractionResult{}, fmt.Errorf("unit %q missing end metadata", unit.ID)
}
}
return contracts.ExtractionResult{
Candidates: []artifacts.ArtifactCandidate{
{
Payload: json.RawMessage(`{"value":"seriatim-source-ref"}`),
SourceRefs: []source.SourceRef{
{
SourceID: req.Source.ID,
StartUnitID: req.Chunk.Units[0].ID,
EndUnitID: req.Chunk.Units[len(req.Chunk.Units)-1].ID,
},
},
},
},
}, nil
}
type runnerSeriatimOutput struct{}
func (runnerSeriatimOutput) Key() string {
return pipeline.DefaultOutputModule
}
func (runnerSeriatimOutput) Encode(ctx context.Context, req contracts.OutputRequest) (contracts.OutputResult, error) {
return contracts.OutputResult{
Files: []contracts.OutputFile{
{Name: "output.json", ContentType: "application/json", Bytes: []byte(`{"encoded":true}`)},
},
}, nil
}
func unitIDs(units []source.SourceUnit) []string {
ids := make([]string, 0, len(units))
for _, unit := range units {
ids = append(ids, unit.ID)
}
return ids
}
func equalStrings(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
var (
_ contracts.Chunker = runnerSeriatimChunker{}
_ contracts.Extractor = (*runnerSeriatimExtractor)(nil)
_ contracts.OutputEncoder = runnerSeriatimOutput{}
)

View File

@@ -0,0 +1,21 @@
{
"metadata": {
"id": "duplicate-segment-fixture"
},
"segments": [
{
"id": "seg-001",
"start": 0,
"end": 1,
"speaker": "Narrator",
"text": "First segment."
},
{
"id": "seg-001",
"start": 1,
"end": 2,
"speaker": "Player",
"text": "Duplicate segment."
}
]
}

View File

@@ -0,0 +1,11 @@
version: 1
pipelines:
seriatim-fixture:
input: seriatim
chunk: fake/chunk
artifacts:
events:
extract: fake/extract
merge: appendorder
normalize: noop
output: json

View File

@@ -0,0 +1,23 @@
{
"metadata": {
"id": "session-alpha",
"source_id": "fallback-session",
"title": "Synthetic session transcript"
},
"segments": [
{
"id": "seg-001",
"start": 0,
"end": 4.5,
"speaker": "Narrator",
"text": "The stone door opens."
},
{
"id": "seg-002",
"start": 4.5,
"end": 8,
"speaker": "Player",
"text": "I cast light."
}
]
}

View File

@@ -0,0 +1,97 @@
package appendorder
import (
"context"
"encoding/json"
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const Key = "appendorder"
var _ contracts.Merger = (*Merger)(nil)
type Merger struct{}
func New() *Merger {
return &Merger{}
}
func (m *Merger) Key() string {
return Key
}
func (m *Merger) Merge(ctx context.Context, req contracts.MergeRequest) (contracts.MergeResult, error) {
if m == nil {
return contracts.MergeResult{}, mergerErrorf("merger must not be nil")
}
if ctx == nil {
return contracts.MergeResult{}, mergerErrorf("context must not be nil")
}
if err := ctx.Err(); err != nil {
return contracts.MergeResult{}, mergerErrorf("context error before merge: %w", err)
}
var candidates []artifacts.ArtifactCandidate
for _, chunkArtifacts := range req.ChunkArtifacts {
candidates = append(candidates, cloneCandidates(chunkArtifacts.Candidates)...)
}
return contracts.MergeResult{Candidates: candidates}, nil
}
func ModuleSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageMerge,
Provides: []string{"merged"},
}
}
func Register(registry *pipeline.MergerRegistry) error {
return registry.RegisterWithSpec(ModuleSpec(), func() (contracts.Merger, error) {
return New(), nil
})
}
func cloneCandidates(candidates []artifacts.ArtifactCandidate) []artifacts.ArtifactCandidate {
if len(candidates) == 0 {
return nil
}
out := make([]artifacts.ArtifactCandidate, 0, len(candidates))
for _, candidate := range candidates {
out = append(out, cloneCandidate(candidate))
}
return out
}
func cloneCandidate(candidate artifacts.ArtifactCandidate) artifacts.ArtifactCandidate {
return artifacts.ArtifactCandidate{
Index: candidate.Index,
ExtractorKey: candidate.ExtractorKey,
ArtifactType: candidate.ArtifactType,
SchemaVersion: candidate.SchemaVersion,
Payload: append(json.RawMessage(nil), candidate.Payload...),
SourceRefs: append([]source.SourceRef(nil), candidate.SourceRefs...),
Metadata: cloneMetadata(candidate.Metadata),
}
}
func cloneMetadata(metadata map[string]any) map[string]any {
if len(metadata) == 0 {
return nil
}
out := make(map[string]any, len(metadata))
for key, value := range metadata {
out[key] = value
}
return out
}
func mergerErrorf(format string, args ...any) error {
return fmt.Errorf("appendorder merger: "+format, args...)
}

View File

@@ -0,0 +1,147 @@
package appendorder
import (
"context"
"encoding/json"
"reflect"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestModuleSpecAndRegister(t *testing.T) {
want := pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageMerge,
Provides: []string{"merged"},
}
if got := ModuleSpec(); !reflect.DeepEqual(got, want) {
t.Fatalf("ModuleSpec() = %#v, want %#v", got, want)
}
registry := pipeline.NewMergerRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
spec, ok := registry.Spec(Key)
if !ok {
t.Fatalf("Spec(%q) ok = false, want true", Key)
}
if !reflect.DeepEqual(spec, want) {
t.Fatalf("registered spec = %#v, want %#v", spec, want)
}
}
func TestMergePreservesChunkAndCandidateOrder(t *testing.T) {
result, err := New().Merge(context.Background(), contracts.MergeRequest{
ChunkArtifacts: []contracts.ChunkArtifacts{
{
Chunk: sourceChunk(0),
Candidates: []artifacts.ArtifactCandidate{candidate(2, "first-b"), candidate(1, "first-a")},
},
{
Chunk: sourceChunk(1),
Candidates: []artifacts.ArtifactCandidate{candidate(4, "second-b"), candidate(3, "second-a")},
},
},
})
if err != nil {
t.Fatalf("Merge() error = %v, want nil", err)
}
got := candidateNames(result.Candidates)
want := []string{"first-b", "first-a", "second-b", "second-a"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("candidate order = %#v, want %#v", got, want)
}
if len(result.Warnings) != 0 {
t.Fatalf("Warnings = %#v, want none", result.Warnings)
}
}
func TestMergeDefensivelyCopiesCandidates(t *testing.T) {
input := []contracts.ChunkArtifacts{
{
Chunk: sourceChunk(0),
Candidates: []artifacts.ArtifactCandidate{candidate(1, "original")},
},
}
result, err := New().Merge(context.Background(), contracts.MergeRequest{ChunkArtifacts: input})
if err != nil {
t.Fatalf("Merge() error = %v, want nil", err)
}
if len(result.Candidates) != 1 {
t.Fatalf("len(Candidates) = %d, want 1", len(result.Candidates))
}
input[0].Candidates[0].Index = 99
input[0].Candidates[0].Payload[0] = '['
input[0].Candidates[0].SourceRefs[0].StartUnitID = "changed"
input[0].Candidates[0].Metadata["name"] = "changed"
got := result.Candidates[0]
if got.Index != 1 {
t.Fatalf("Index = %d, want 1", got.Index)
}
if string(got.Payload) != `{"name":"original"}` {
t.Fatalf("Payload = %s, want original payload", got.Payload)
}
if got.SourceRefs[0].StartUnitID != "u1" {
t.Fatalf("SourceRefs = %#v, want original source ref", got.SourceRefs)
}
if got.Metadata["name"] != "original" {
t.Fatalf("Metadata = %#v, want original metadata", got.Metadata)
}
}
func TestMergeHandlesEmptyInput(t *testing.T) {
result, err := New().Merge(context.Background(), contracts.MergeRequest{})
if err != nil {
t.Fatalf("Merge() error = %v, want nil", err)
}
if len(result.Candidates) != 0 {
t.Fatalf("len(Candidates) = %d, want 0", len(result.Candidates))
}
if len(result.Warnings) != 0 {
t.Fatalf("Warnings = %#v, want none", result.Warnings)
}
}
func candidate(index int, name string) artifacts.ArtifactCandidate {
return artifacts.ArtifactCandidate{
Index: index,
ExtractorKey: "generic-extractor",
ArtifactType: "generic-artifact",
SchemaVersion: "v1",
Payload: json.RawMessage(`{"name":"` + name + `"}`),
SourceRefs: []source.SourceRef{
{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u1"},
},
Metadata: map[string]any{
"name": name,
},
}
}
func candidateNames(candidates []artifacts.ArtifactCandidate) []string {
names := make([]string, 0, len(candidates))
for _, candidate := range candidates {
names = append(names, candidate.Metadata["name"].(string))
}
return names
}
func sourceChunk(index int) contracts.SourceChunk {
return contracts.SourceChunk{
ID: "chunk",
SourceID: "source-1",
Index: index,
Units: []source.SourceUnit{
{ID: "u1", Kind: "unit", Text: "Source unit."},
},
}
}

View File

@@ -0,0 +1,89 @@
package noop
import (
"context"
"encoding/json"
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const Key = "noop"
var _ contracts.Normalizer = (*Normalizer)(nil)
type Normalizer struct{}
func New() *Normalizer {
return &Normalizer{}
}
func (n *Normalizer) Key() string {
return Key
}
func (n *Normalizer) Normalize(ctx context.Context, req contracts.NormalizeRequest) (contracts.NormalizeResult, error) {
if n == nil {
return contracts.NormalizeResult{}, normalizerErrorf("normalizer must not be nil")
}
if ctx == nil {
return contracts.NormalizeResult{}, normalizerErrorf("context must not be nil")
}
if err := ctx.Err(); err != nil {
return contracts.NormalizeResult{}, normalizerErrorf("context error before normalize: %w", err)
}
return contracts.NormalizeResult{Candidates: cloneCandidates(req.Candidates)}, nil
}
func ModuleSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageNormalize,
Requires: []string{"merged"},
Provides: []string{"normalized"},
}
}
func Register(registry *pipeline.NormalizerRegistry) error {
return registry.RegisterWithSpec(ModuleSpec(), func() (contracts.Normalizer, error) {
return New(), nil
})
}
func cloneCandidates(candidates []artifacts.ArtifactCandidate) []artifacts.ArtifactCandidate {
if len(candidates) == 0 {
return nil
}
out := make([]artifacts.ArtifactCandidate, 0, len(candidates))
for _, candidate := range candidates {
out = append(out, artifacts.ArtifactCandidate{
Index: candidate.Index,
ExtractorKey: candidate.ExtractorKey,
ArtifactType: candidate.ArtifactType,
SchemaVersion: candidate.SchemaVersion,
Payload: append(json.RawMessage(nil), candidate.Payload...),
SourceRefs: append([]source.SourceRef(nil), candidate.SourceRefs...),
Metadata: cloneMetadata(candidate.Metadata),
})
}
return out
}
func cloneMetadata(metadata map[string]any) map[string]any {
if len(metadata) == 0 {
return nil
}
out := make(map[string]any, len(metadata))
for key, value := range metadata {
out[key] = value
}
return out
}
func normalizerErrorf(format string, args ...any) error {
return fmt.Errorf("noop normalizer: "+format, args...)
}

View File

@@ -0,0 +1,133 @@
package noop
import (
"context"
"encoding/json"
"reflect"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestModuleSpecAndRegister(t *testing.T) {
want := pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageNormalize,
Requires: []string{"merged"},
Provides: []string{"normalized"},
}
if got := ModuleSpec(); !reflect.DeepEqual(got, want) {
t.Fatalf("ModuleSpec() = %#v, want %#v", got, want)
}
registry := pipeline.NewNormalizerRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
spec, ok := registry.Spec(Key)
if !ok {
t.Fatalf("Spec(%q) ok = false, want true", Key)
}
if !reflect.DeepEqual(spec, want) {
t.Fatalf("registered spec = %#v, want %#v", spec, want)
}
}
func TestNormalizePassesThroughOrderAndValues(t *testing.T) {
input := []artifacts.ArtifactCandidate{
candidate(3, "third"),
candidate(1, "first"),
candidate(2, "second"),
}
result, err := New().Normalize(context.Background(), contracts.NormalizeRequest{Candidates: input})
if err != nil {
t.Fatalf("Normalize() error = %v, want nil", err)
}
if len(result.Warnings) != 0 {
t.Fatalf("Warnings = %#v, want none", result.Warnings)
}
got := candidateNames(result.Candidates)
want := []string{"third", "first", "second"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("candidate order = %#v, want %#v", got, want)
}
if !reflect.DeepEqual(result.Candidates[0].SourceRefs, input[0].SourceRefs) {
t.Fatalf("SourceRefs = %#v, want %#v", result.Candidates[0].SourceRefs, input[0].SourceRefs)
}
if !reflect.DeepEqual(result.Candidates[0].Metadata, input[0].Metadata) {
t.Fatalf("Metadata = %#v, want %#v", result.Candidates[0].Metadata, input[0].Metadata)
}
}
func TestNormalizeDefensivelyCopiesCandidates(t *testing.T) {
input := []artifacts.ArtifactCandidate{candidate(1, "original")}
result, err := New().Normalize(context.Background(), contracts.NormalizeRequest{Candidates: input})
if err != nil {
t.Fatalf("Normalize() error = %v, want nil", err)
}
if len(result.Candidates) != 1 {
t.Fatalf("len(Candidates) = %d, want 1", len(result.Candidates))
}
input[0].Index = 99
input[0].Payload[0] = '['
input[0].SourceRefs[0].EndUnitID = "changed"
input[0].Metadata["name"] = "changed"
got := result.Candidates[0]
if got.Index != 1 {
t.Fatalf("Index = %d, want 1", got.Index)
}
if string(got.Payload) != `{"name":"original"}` {
t.Fatalf("Payload = %s, want original payload", got.Payload)
}
if got.SourceRefs[0].EndUnitID != "u1" {
t.Fatalf("SourceRefs = %#v, want original source ref", got.SourceRefs)
}
if got.Metadata["name"] != "original" {
t.Fatalf("Metadata = %#v, want original metadata", got.Metadata)
}
}
func TestNormalizeHandlesEmptyInput(t *testing.T) {
result, err := New().Normalize(context.Background(), contracts.NormalizeRequest{})
if err != nil {
t.Fatalf("Normalize() error = %v, want nil", err)
}
if len(result.Candidates) != 0 {
t.Fatalf("len(Candidates) = %d, want 0", len(result.Candidates))
}
if len(result.Warnings) != 0 {
t.Fatalf("Warnings = %#v, want none", result.Warnings)
}
}
func candidate(index int, name string) artifacts.ArtifactCandidate {
return artifacts.ArtifactCandidate{
Index: index,
ExtractorKey: "generic-extractor",
ArtifactType: "generic-artifact",
SchemaVersion: "v1",
Payload: json.RawMessage(`{"name":"` + name + `"}`),
SourceRefs: []source.SourceRef{
{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u1"},
},
Metadata: map[string]any{
"name": name,
},
}
}
func candidateNames(candidates []artifacts.ArtifactCandidate) []string {
names := make([]string, 0, len(candidates))
for _, candidate := range candidates {
names = append(names, candidate.Metadata["name"].(string))
}
return names
}

View File

@@ -0,0 +1,253 @@
package json
import (
"context"
stdjson "encoding/json"
"fmt"
"regexp"
"sort"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const Key = "json"
const contentTypeJSON = "application/json"
var safeArtifactFileChar = regexp.MustCompile(`[^A-Za-z0-9._-]`)
var _ contracts.OutputEncoder = (*Encoder)(nil)
type Encoder struct{}
func New() *Encoder {
return &Encoder{}
}
func (e *Encoder) Key() string {
return Key
}
func (e *Encoder) Encode(ctx context.Context, req contracts.OutputRequest) (contracts.OutputResult, error) {
if e == nil {
return contracts.OutputResult{}, encoderErrorf("encoder must not be nil")
}
if ctx == nil {
return contracts.OutputResult{}, encoderErrorf("context must not be nil")
}
if err := ctx.Err(); err != nil {
return contracts.OutputResult{}, encoderErrorf("context error before encoding: %w", err)
}
files, err := logicalFiles(req)
if err != nil {
return contracts.OutputResult{}, err
}
return contracts.OutputResult{Files: files}, nil
}
func ModuleSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageOutput,
Requires: []string{"normalized"},
Provides: []string{"encoded"},
}
}
func Register(registry *pipeline.OutputEncoderRegistry) error {
return registry.RegisterWithSpec(ModuleSpec(), func() (contracts.OutputEncoder, error) {
return New(), nil
})
}
type indexFile struct {
ManifestFile string `json:"manifest_file"`
ArtifactFiles []artifactFileIndex `json:"artifact_files"`
RejectedFile string `json:"rejected_file"`
WarningsFile string `json:"warnings_file"`
}
type artifactFileIndex struct {
ArtifactType string `json:"artifact_type"`
File string `json:"file"`
}
type artifactFile struct {
ArtifactType string `json:"artifact_type"`
Artifacts []artifacts.Artifact `json:"artifacts"`
}
type rejectedFile struct {
Rejected []artifacts.RejectedArtifact `json:"rejected"`
}
type warningsFile struct {
Warnings []contracts.Warning `json:"warnings"`
}
func logicalFiles(req contracts.OutputRequest) ([]contracts.OutputFile, error) {
artifactsByType := make(map[string][]artifacts.Artifact)
for _, artifact := range req.Approved {
artifactsByType[artifact.ArtifactType] = append(artifactsByType[artifact.ArtifactType], cloneArtifact(artifact))
}
artifactTypes := make([]string, 0, len(artifactsByType))
for artifactType := range artifactsByType {
artifactTypes = append(artifactTypes, artifactType)
}
sort.Strings(artifactTypes)
artifactIndexes := make([]artifactFileIndex, 0, len(artifactTypes))
files := make([]contracts.OutputFile, 0, len(artifactTypes)+4)
manifestFile, err := jsonFile("manifest.json", req.Manifest)
if err != nil {
return nil, err
}
files = append(files, manifestFile)
usedArtifactFiles := make(map[string]string, len(artifactTypes))
for _, artifactType := range artifactTypes {
name, err := artifactFileName(artifactType)
if err != nil {
return nil, err
}
if existingType, ok := usedArtifactFiles[name]; ok {
return nil, encoderErrorf("artifact types %q and %q produce duplicate output file %q", existingType, artifactType, name)
}
usedArtifactFiles[name] = artifactType
artifactIndexes = append(artifactIndexes, artifactFileIndex{
ArtifactType: artifactType,
File: name,
})
file, err := jsonFile(name, artifactFile{
ArtifactType: artifactType,
Artifacts: artifactsByType[artifactType],
})
if err != nil {
return nil, err
}
files = append(files, file)
}
index := indexFile{
ManifestFile: "manifest.json",
ArtifactFiles: artifactIndexes,
RejectedFile: "rejected.json",
WarningsFile: "warnings.json",
}
indexOutput, err := jsonFile("index.json", index)
if err != nil {
return nil, err
}
rejectedOutput, err := jsonFile("rejected.json", rejectedFile{Rejected: cloneRejected(req.Rejected)})
if err != nil {
return nil, err
}
warningsOutput, err := jsonFile("warnings.json", warningsFile{Warnings: cloneWarnings(req.Warnings)})
if err != nil {
return nil, err
}
files = append(files, indexOutput, rejectedOutput, warningsOutput)
sort.Slice(files, func(i, j int) bool {
return files[i].Name < files[j].Name
})
return files, nil
}
func jsonFile(name string, value any) (contracts.OutputFile, error) {
data, err := marshalPretty(value)
if err != nil {
return contracts.OutputFile{}, encoderErrorf("encode %s: %w", name, err)
}
return contracts.OutputFile{
Name: name,
ContentType: contentTypeJSON,
Bytes: data,
}, nil
}
func marshalPretty(value any) ([]byte, error) {
data, err := stdjson.MarshalIndent(value, "", " ")
if err != nil {
return nil, err
}
return append(data, '\n'), nil
}
func artifactFileName(artifactType string) (string, error) {
sanitized := safeArtifactFileChar.ReplaceAllString(strings.TrimSpace(artifactType), "_")
for strings.Contains(sanitized, "..") {
sanitized = strings.ReplaceAll(sanitized, "..", "__")
}
sanitized = strings.Trim(sanitized, "._")
if sanitized == "" {
return "", encoderErrorf("artifact type %q cannot produce a safe file name", artifactType)
}
return "artifacts/" + sanitized + ".json", nil
}
func cloneArtifact(artifact artifacts.Artifact) artifacts.Artifact {
return artifacts.Artifact{
ExtractorKey: artifact.ExtractorKey,
ArtifactType: artifact.ArtifactType,
SchemaVersion: artifact.SchemaVersion,
Payload: append(stdjson.RawMessage(nil), artifact.Payload...),
SourceRefs: append([]source.SourceRef(nil), artifact.SourceRefs...),
Metadata: cloneMetadata(artifact.Metadata),
}
}
func cloneRejected(rejected []artifacts.RejectedArtifact) []artifacts.RejectedArtifact {
if len(rejected) == 0 {
return []artifacts.RejectedArtifact{}
}
out := make([]artifacts.RejectedArtifact, 0, len(rejected))
for _, item := range rejected {
out = append(out, artifacts.RejectedArtifact{
Candidate: cloneCandidate(item.Candidate),
ValidatorName: item.ValidatorName,
ReasonCode: item.ReasonCode,
Message: item.Message,
})
}
return out
}
func cloneCandidate(candidate artifacts.ArtifactCandidate) artifacts.ArtifactCandidate {
return artifacts.ArtifactCandidate{
Index: candidate.Index,
ExtractorKey: candidate.ExtractorKey,
ArtifactType: candidate.ArtifactType,
SchemaVersion: candidate.SchemaVersion,
Payload: append(stdjson.RawMessage(nil), candidate.Payload...),
SourceRefs: append([]source.SourceRef(nil), candidate.SourceRefs...),
Metadata: cloneMetadata(candidate.Metadata),
}
}
func cloneWarnings(warnings []contracts.Warning) []contracts.Warning {
if len(warnings) == 0 {
return []contracts.Warning{}
}
return append([]contracts.Warning(nil), warnings...)
}
func cloneMetadata(metadata map[string]any) map[string]any {
if len(metadata) == 0 {
return nil
}
out := make(map[string]any, len(metadata))
for key, value := range metadata {
out[key] = value
}
return out
}
func encoderErrorf(format string, args ...any) error {
return fmt.Errorf("json output encoder: "+format, args...)
}

View File

@@ -0,0 +1,316 @@
package json
import (
"context"
stdjson "encoding/json"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestModuleSpecAndRegister(t *testing.T) {
want := pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageOutput,
Requires: []string{"normalized"},
Provides: []string{"encoded"},
}
if got := ModuleSpec(); !reflect.DeepEqual(got, want) {
t.Fatalf("ModuleSpec() = %#v, want %#v", got, want)
}
registry := pipeline.NewOutputEncoderRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
spec, ok := registry.Spec(Key)
if !ok {
t.Fatalf("Spec(%q) ok = false, want true", Key)
}
if !reflect.DeepEqual(spec, want) {
t.Fatalf("registered spec = %#v, want %#v", spec, want)
}
}
func TestEncodeReturnsLogicalFilesGroupedByArtifactType(t *testing.T) {
req := contracts.OutputRequest{
Manifest: artifacts.RunManifest{RunID: "run-1", PipelineID: "pipeline-1"},
Approved: []artifacts.Artifact{
artifact("dnd.spell-cast", "first"),
artifact("notes/item", "item"),
artifact("dnd.spell-cast", "second"),
},
Rejected: []artifacts.RejectedArtifact{
{
Candidate: candidate("bad type", "bad"),
ValidatorName: "validator",
ReasonCode: "invalid",
Message: "not accepted",
},
},
Warnings: []contracts.Warning{{ReasonCode: "warning", Message: "check source"}},
}
result, err := New().Encode(context.Background(), req)
if err != nil {
t.Fatalf("Encode() error = %v, want nil", err)
}
wantNames := []string{
"artifacts/dnd.spell-cast.json",
"artifacts/notes_item.json",
"index.json",
"manifest.json",
"rejected.json",
"warnings.json",
}
if got := outputFileNames(result.Files); !reflect.DeepEqual(got, wantNames) {
t.Fatalf("file names = %#v, want %#v", got, wantNames)
}
for _, file := range result.Files {
if file.ContentType != contentTypeJSON {
t.Fatalf("%s ContentType = %q, want %q", file.Name, file.ContentType, contentTypeJSON)
}
if !strings.HasSuffix(string(file.Bytes), "\n") {
t.Fatalf("%s does not end with newline: %q", file.Name, string(file.Bytes))
}
if !stdjson.Valid(file.Bytes) {
t.Fatalf("%s has invalid JSON: %s", file.Name, file.Bytes)
}
}
spellFile := decodeObject(t, fileBytes(t, result.Files, "artifacts/dnd.spell-cast.json"))
if spellFile["artifact_type"] != "dnd.spell-cast" {
t.Fatalf("artifact_type = %#v, want dnd.spell-cast", spellFile["artifact_type"])
}
spells := spellFile["artifacts"].([]any)
if len(spells) != 2 {
t.Fatalf("len(spells) = %d, want 2", len(spells))
}
firstPayload := spells[0].(map[string]any)["payload"].(map[string]any)
secondPayload := spells[1].(map[string]any)["payload"].(map[string]any)
if firstPayload["name"] != "first" || secondPayload["name"] != "second" {
t.Fatalf("spell order payloads = %#v then %#v, want runner order", firstPayload, secondPayload)
}
index := decodeObject(t, fileBytes(t, result.Files, "index.json"))
artifactFiles := index["artifact_files"].([]any)
if len(artifactFiles) != 2 {
t.Fatalf("len(index artifact_files) = %d, want 2", len(artifactFiles))
}
firstIndex := artifactFiles[0].(map[string]any)
secondIndex := artifactFiles[1].(map[string]any)
if firstIndex["artifact_type"] != "dnd.spell-cast" || secondIndex["artifact_type"] != "notes/item" {
t.Fatalf("artifact_files = %#v, want sorted by artifact type", artifactFiles)
}
}
func TestEncodeIncludesRejectedAndWarningsWhenEmpty(t *testing.T) {
result, err := New().Encode(context.Background(), contracts.OutputRequest{
Manifest: artifacts.RunManifest{RunID: "run-1"},
})
if err != nil {
t.Fatalf("Encode() error = %v, want nil", err)
}
rejected := decodeObject(t, fileBytes(t, result.Files, "rejected.json"))
if got := rejected["rejected"].([]any); len(got) != 0 {
t.Fatalf("rejected = %#v, want empty array", got)
}
warnings := decodeObject(t, fileBytes(t, result.Files, "warnings.json"))
if got := warnings["warnings"].([]any); len(got) != 0 {
t.Fatalf("warnings = %#v, want empty array", got)
}
}
func TestEncodePrettyPrintsJSON(t *testing.T) {
result, err := New().Encode(context.Background(), contracts.OutputRequest{
Manifest: artifacts.RunManifest{RunID: "run-1"},
})
if err != nil {
t.Fatalf("Encode() error = %v, want nil", err)
}
manifest := string(fileBytes(t, result.Files, "manifest.json"))
if !strings.Contains(manifest, "\n \"run_id\": \"run-1\"\n") {
t.Fatalf("manifest JSON = %q, want two-space indentation", manifest)
}
}
func TestEncodeRejectsArtifactTypeWithoutSafeFileName(t *testing.T) {
_, err := New().Encode(context.Background(), contracts.OutputRequest{
Approved: []artifacts.Artifact{artifact("///", "unsafe")},
})
if err == nil {
t.Fatal("Encode() error = nil, want unsafe artifact type error")
}
if !strings.Contains(err.Error(), "json output encoder") || !strings.Contains(err.Error(), "safe file name") {
t.Fatalf("Encode() error = %q, want safe file name context", err.Error())
}
}
func TestEncodeSanitizesParentPathSequences(t *testing.T) {
result, err := New().Encode(context.Background(), contracts.OutputRequest{
Approved: []artifacts.Artifact{artifact("dnd..spell.", "spell")},
})
if err != nil {
t.Fatalf("Encode() error = %v, want nil", err)
}
if got := outputFileNames(result.Files); !containsString(got, "artifacts/dnd__spell.json") {
t.Fatalf("file names = %#v, want sanitized artifact filename", got)
}
}
func TestEncodeRejectsSanitizedFilenameCollisions(t *testing.T) {
_, err := New().Encode(context.Background(), contracts.OutputRequest{
Approved: []artifacts.Artifact{
artifact("a/b", "slash"),
artifact("a?b", "question"),
},
})
if err == nil {
t.Fatal("Encode() error = nil, want duplicate file error")
}
if !strings.Contains(err.Error(), "duplicate output file") {
t.Fatalf("Encode() error = %q, want duplicate file context", err.Error())
}
}
func TestEncodeDoesNotMutateInputs(t *testing.T) {
req := contracts.OutputRequest{
Manifest: artifacts.RunManifest{RunID: "run-1"},
Approved: []artifacts.Artifact{
artifact("dnd.spell", "original"),
},
Rejected: []artifacts.RejectedArtifact{
{
Candidate: candidate("bad", "rejected"),
ValidatorName: "validator",
ReasonCode: "invalid",
Message: "not accepted",
},
},
Warnings: []contracts.Warning{{ReasonCode: "warning", Message: "message"}},
}
before := mustMarshal(t, req)
result, err := New().Encode(context.Background(), req)
if err != nil {
t.Fatalf("Encode() error = %v, want nil", err)
}
after := mustMarshal(t, req)
if before != after {
t.Fatalf("request mutated:\nbefore: %s\nafter: %s", before, after)
}
req.Approved[0].Payload[0] = '['
req.Approved[0].SourceRefs[0].StartUnitID = "changed"
req.Approved[0].Metadata["name"] = "changed"
req.Rejected[0].Candidate.Payload[0] = '['
req.Warnings[0].Message = "changed"
if !stdjson.Valid(fileBytes(t, result.Files, "artifacts/dnd.spell.json")) {
t.Fatal("artifact output changed after request mutation")
}
warnings := decodeObject(t, fileBytes(t, result.Files, "warnings.json"))
gotWarnings := warnings["warnings"].([]any)
if gotWarnings[0].(map[string]any)["message"] != "message" {
t.Fatalf("warnings output changed after request mutation: %#v", gotWarnings)
}
}
func TestArtifactFilesDoNotContainWarnings(t *testing.T) {
result, err := New().Encode(context.Background(), contracts.OutputRequest{
Approved: []artifacts.Artifact{artifact("dnd.spell", "spell")},
Warnings: []contracts.Warning{
{ReasonCode: "pipeline-warning", Message: "warning"},
},
})
if err != nil {
t.Fatalf("Encode() error = %v, want nil", err)
}
artifactFile := decodeObject(t, fileBytes(t, result.Files, "artifacts/dnd.spell.json"))
if _, ok := artifactFile["warnings"]; ok {
t.Fatalf("artifact file contains warnings: %#v", artifactFile)
}
}
func artifact(artifactType, name string) artifacts.Artifact {
return artifacts.Artifact{
ExtractorKey: "extractor",
ArtifactType: artifactType,
SchemaVersion: "v1",
Payload: stdjson.RawMessage(`{"name":"` + name + `"}`),
SourceRefs: []source.SourceRef{
{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u1"},
},
Metadata: map[string]any{"name": name},
}
}
func candidate(artifactType, name string) artifacts.ArtifactCandidate {
return artifacts.ArtifactCandidate{
Index: 1,
ExtractorKey: "extractor",
ArtifactType: artifactType,
SchemaVersion: "v1",
Payload: stdjson.RawMessage(`{"name":"` + name + `"}`),
SourceRefs: []source.SourceRef{
{SourceID: "source-1", StartUnitID: "u1", EndUnitID: "u1"},
},
Metadata: map[string]any{"name": name},
}
}
func outputFileNames(files []contracts.OutputFile) []string {
names := make([]string, 0, len(files))
for _, file := range files {
names = append(names, file.Name)
}
return names
}
func containsString(values []string, want string) bool {
for _, value := range values {
if value == want {
return true
}
}
return false
}
func fileBytes(t *testing.T, files []contracts.OutputFile, name string) []byte {
t.Helper()
for _, file := range files {
if file.Name == name {
return file.Bytes
}
}
t.Fatalf("file %q not found in %#v", name, outputFileNames(files))
return nil
}
func decodeObject(t *testing.T, data []byte) map[string]any {
t.Helper()
var got map[string]any
if err := stdjson.Unmarshal(data, &got); err != nil {
t.Fatalf("Unmarshal() error = %v, want nil\n%s", err, data)
}
return got
}
func mustMarshal(t *testing.T, value any) string {
t.Helper()
data, err := stdjson.Marshal(value)
if err != nil {
t.Fatalf("Marshal() error = %v, want nil", err)
}
return string(data)
}