eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: eric:286fb9dce7f31c115f660ec14afd29434e388092
Branches Tags
eric:main
eric:v0.4.0 eric:v0.3.0 eric:v0.2.0 eric:v0.1.0
...
compare: eric:882494891075b98d59e3609e47836e5d9a9a2773
Branches Tags
eric:main
eric:v0.4.0 eric:v0.3.0 eric:v0.2.0 eric:v0.1.0

6 Commits
286fb9dce7 ... 8824948910

Author SHA1 Message Date
Eric Rakestraw
8824948910 Document output cache and debug state model 2026-07-18 13:08:27 +00:00
Eric Rakestraw
8cb11e60e4 Harden output cache and debug state integration 2026-07-18 13:00:04 +00:00
Eric Rakestraw
26142f0e05 Remove legacy workspace and diagnostics implementation 2026-07-18 12:48:21 +00:00
Eric Rakestraw
1542a12497 Cut configuration and CLI over to output cache and debug surfaces 2026-07-18 12:42:10 +00:00
Eric Rakestraw
a9250206d5 Add internal debug bundle collaborators 2026-07-18 04:57:50 +00:00
Eric Rakestraw
5bd0ba7a72 Decouple checkpoint storage from workspace state 2026-07-18 04:55:33 +00:00
67 changed files with 2363 additions and 11741 deletions
Expand all files Collapse all files

3
README.md
View File

@@ -5,7 +5,8 @@ with explicit, configurable pipeline modules.
The current implementation reads Seriatim transcript JSON, chunks the source
units, extracts D&D spell-cast artifacts with a Scriptorium-backed LLM runtime,
and writes JSON output plus diagnostics when enabled.
and writes JSON output. Add `--debug` when a per-run inspection bundle is
needed.
```sh
OPENROUTER_API_KEY=... \

2
docs/adr/0006-separate-output-cache-and-debug-state.md
View File

@@ -1,6 +1,6 @@
# ADR-0006: Separate output, cache, and debug state
**Status:** Proposed
**Status:** Accepted
**Date:** 2026-07-17
## Context

37
docs/cli.md
View File

@@ -9,7 +9,7 @@ For the minimal end-to-end invocation, see the [README](../README.md).
```text
notarius help
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--chunk_cache auto|bypass|refresh] [--output-dir path] [--diagnostics-dir path] [--llm-profile id] [--resume] [--session-id id] [--reference selector=path] [--without-reference selector]
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--chunk_cache auto|bypass|refresh] [--output-dir path] [--resume] [--debug [--debug-dir path]] [--llm-profile id] [--session-id id] [--reference selector=path] [--without-reference selector]
notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list [--config path/to/config.yml] [--json]
```
@@ -35,11 +35,11 @@ Flags:
invocation. `auto` reuses a valid plan by canonical source digest, `bypass`
performs no plan-cache I/O, and `refresh` regenerates and replaces a valid
plan only after chunk validation succeeds. See
[Configuration](config.md#workspace) for the persistent setting, precedence,
[Configuration](config.md#state-surfaces) for the persistent setting, precedence,
and cache-root selection.
- `--output-dir path`: output root. Defaults to `./notarius-output`.
- `--diagnostics-dir path`: diagnostics work directory override for this
invocation. It does not change the workspace directory.
- `--debug`: allocate and retain one debug bundle for this invocation.
- `--debug-dir path`: debug-bundle root override. This flag requires `--debug`.
- `--llm-profile id`: override every effective LLM-capable pipeline module
binding with one Scriptorium profile ID. Validator-specific profiles are not
overridden.
@@ -52,8 +52,9 @@ Flags:
`=path`.
On success, the command prints the completed pipeline ID, normalized output and
rejected output counts, and the output directory. If the run completes with warnings,
the warning count is printed to stderr.
rejected output counts, and the output directory. A debug-enabled run also
prints `debug=<bundle-path>`. If the run completes with warnings, the warning
count is printed to stderr.
Reference flags are resolved against selected chunk, extractor, merger, and
normalizer targets before the run starts. Flat slot names are accepted only
@@ -122,7 +123,9 @@ go run ./cmd/notarius run dnd-session \
--session-id campaign-17-session-04
```
The resume flag can be added to an otherwise identical run invocation:
The resume flag can be added to an otherwise identical run invocation. It both
loads compatible checkpoints and records replacements for work executed by that
invocation; without it, the checkpoint root is not used:
```sh
go run ./cmd/notarius run dnd-session \
@@ -131,6 +134,18 @@ go run ./cmd/notarius run dnd-session \
--resume
```
Use `--debug` to retain the redacted summary and trace bundle for one run. The
bundle is allocated before pipeline resolution; once allocated, its path is
also printed to stderr if the command fails. Debug-write failures cause exit
code `1`.
```sh
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json \
--debug --debug-dir ./notarius-debug
```
Use `refresh` when intentionally replacing the cached plan for the same source:
```sh
@@ -149,8 +164,8 @@ go run ./cmd/notarius run dnd-session \
--chunk_cache bypass
```
For checkpoint behavior, durable output, diagnostics, retention, and failure
inspection, see [Operations](operations.md).
`--diagnostics-dir` has been removed. For checkpoint behavior, durable output,
debug-bundle lifecycle, and failure inspection, see [Operations](operations.md).
## `config validate`
@@ -202,8 +217,8 @@ go run ./cmd/notarius pipelines list \
- `0`: command succeeded.
- `1`: command syntax was valid, but loading config, resolving modules, running
the pipeline, calling the provider, writing output, or writing diagnostics
failed.
the pipeline, calling the provider, writing output, or writing a requested
debug bundle failed.
- `2`: command syntax was invalid, a command was unknown, a required argument
was missing, or a flag value was malformed.

200
docs/config.md
View File

@@ -2,8 +2,9 @@
This is the canonical reference for implemented Notarius configuration.
Notarius reads YAML config files with `version: 2`. File config is applied over
built-in defaults, then environment overrides are applied.
Notarius reads YAML config files with `version: 3`. File configuration is
applied over built-in defaults, then environment overrides are applied. Explicit
CLI overrides are applied last where the command supports them.
## Discovery
@@ -21,17 +22,18 @@ The explicit-path option is defined in the [CLI reference](cli.md).
- [Minimal D&D spell configuration](../examples/dnd-spells.config.yml)
- [Production-oriented D&D spell configuration](../examples/dnd-spells-production.config.yml)
Both complete files are validated by the CLI test suite. The fragments below
illustrate individual fields and are not alternate complete configurations.
Both are complete version 3 files. The fragments below illustrate individual
fields and are not alternate complete configurations.
## Top-Level Fields
- `version`: required. The only supported value is `2`.
- `version`: required. The only supported value is `3`.
- `scriptorium`: optional Scriptorium profile source settings.
- `pipelines`: optional map of pipeline IDs to pipeline definitions.
- `concurrency`: optional global concurrency settings.
- `workspace`: optional workspace settings for Notarius-owned local state.
- `diagnostics`: optional diagnostics settings.
- `output`: optional durable output placement.
- `cache`: optional chunk-plan and checkpoint cache placement.
- `debug`: optional debug-bundle placement. It does not enable debug capture.
Unknown YAML fields are rejected. The removed top-level `llm_profiles` field is
rejected; execution profiles now come from Scriptorium.
@@ -42,14 +44,13 @@ Built-in defaults:
- `concurrency.total_llm`: `1`
- `concurrency.stage_workers.extract`: effective `concurrency.total_llm`
- `diagnostics.work_dir`: `/tmp/notarius`
- `diagnostics.retention`: `auto`
- `workspace.directory`: unset
- `workspace.diagnostics.enabled`: `true`
- `workspace.resume.enabled`: `false`
- `workspace.debug.enabled`: `false`
- `workspace.chunk_cache.mode`: `auto`
- `workspace.chunk_cache.directory`: unset
- `output.directory`: `./notarius-output`
- `cache.chunk_plans.mode`: `auto`
- `cache.chunk_plans.directory`: unset, selecting
`<os.UserCacheDir>/notarius/chunk-plans`
- `cache.checkpoints.directory`: unset, selecting
`<os.UserCacheDir>/notarius/checkpoints`
- `debug.directory`: `./notarius-debug`
No pipelines are built in. A run requires a configured pipeline.
@@ -92,22 +93,15 @@ These environment variables are applied after the config file:
- `NOTARIUS_CONFIG`: config discovery path.
- `NOTARIUS_TOTAL_LLM_CONCURRENCY`: integer global LLM concurrency.
- `NOTARIUS_STAGE_WORKERS_EXTRACT`: integer extract worker limit.
- `NOTARIUS_WORKSPACE_DIR`: workspace directory.
- `NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED`: boolean diagnostics enablement.
- `NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION`: workspace diagnostics retention
mode.
- `NOTARIUS_WORKSPACE_RESUME_ENABLED`: boolean resume checkpointing
enablement.
- `NOTARIUS_WORKSPACE_DEBUG_ENABLED`: boolean debug artifact enablement.
- `NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE`: chunk-plan cache mode.
- `NOTARIUS_WORKSPACE_CHUNK_CACHE_DIR`: chunk-plan cache root.
- `NOTARIUS_WORK_DIR`: deprecated diagnostics work directory compatibility
override.
- `NOTARIUS_DIAGNOSTICS_RETENTION`: deprecated diagnostics retention
compatibility override.
- `NOTARIUS_OUTPUT_DIR`: durable output root.
- `NOTARIUS_CACHE_CHUNK_PLANS_MODE`: chunk-plan cache mode.
- `NOTARIUS_CACHE_CHUNK_PLANS_DIR`: chunk-plan cache root.
- `NOTARIUS_CACHE_CHECKPOINTS_DIR`: checkpoint cache root.
- `NOTARIUS_DEBUG_DIR`: debug-bundle root.
Integer environment values must parse as base-10 integers. Boolean environment
values must parse as Go booleans such as `true`, `false`, `1`, or `0`.
Integer environment values must parse as base-10 integers. Directory overrides
must be non-empty after trimming. Cache-directory fields in a file may be
empty, which deliberately selects the corresponding per-user default.
The removed `NOTARIUS_LLM_DEFAULT_*` variables are not read. Configure provider
endpoint, model, and credential environment variable names through Scriptorium
@@ -341,71 +335,111 @@ Both modules accept UTF-8 plain text, Markdown, YAML, or JSON reference files.
The extractor uses references only as supporting disambiguation material; spell
casts still must be present in the source transcript.
## Workspace
## State Surfaces
`workspace` fields:
The `output`, `cache`, and `debug` top-level fields select independent physical
roots. Their layout, permissions, lifecycle, and sensitive-data handling are
defined in [Operations](operations.md).
- `directory`: optional workspace root for Notarius-owned local state.
- `resume.enabled`: boolean resume checkpointing setting.
- `debug.enabled`: boolean debug artifact setting.
- `chunk_cache.mode`: persistent chunk-plan cache mode: `auto`, `bypass`, or
`refresh`. The default is `auto`.
- `chunk_cache.directory`: optional chunk-plan cache root. This value is the
root itself; Notarius does not append `chunk-plans` to it.
- `diagnostics`: optional diagnostics settings defined below.
```yaml
output:
directory: ./notarius-output
cache:
chunk_plans:
directory: ""
mode: auto
checkpoints:
directory: ""
debug:
directory: ./notarius-debug
```
`workspace.resume.enabled`, `workspace.debug.enabled`, and
`workspace.chunk_cache` are independent. `workspace.directory` does not affect
chunk-plan placement. For directory layout, state lifecycle, permissions, and
sensitive content, see [Operations](operations.md).
`output.directory` is the durable output root. Its precedence is
`--output-dir`, `NOTARIUS_OUTPUT_DIR`, the file value, then the default.
`chunk_cache.mode` accepts only `auto`, `bypass`, and `refresh`. In `auto`, a
valid source-addressed plan is reused and a missing or invalid record is
regenerated and published after chunk validation. `bypass` neither reads nor
writes plan-cache state. `refresh` always generates a new plan and publishes it
only after validation succeeds.
`cache.chunk_plans.mode` accepts `auto`, `bypass`, or `refresh`. Its precedence
is `--chunk_cache`, `NOTARIUS_CACHE_CHUNK_PLANS_MODE`, the file value, then
`auto`. `auto` reuses a valid source-addressed plan and regenerates missing or
invalid records; `bypass` performs no plan-cache I/O; `refresh` regenerates and
publishes a plan after chunk validation.
Configuration values are applied in file then environment order; an explicit
`--chunk_cache` CLI value has highest precedence for the mode. The cache root
is selected from environment, file, then the per-user default; there is no CLI
root override. Every supplied value is parsed strictly even when a higher
precedence value wins, so malformed configuration is still an error.
`cache.chunk_plans.directory` and `cache.checkpoints.directory` each name an
exact cache-family root. Their precedence is the corresponding environment
variable, the file value, then the family-specific per-user default. There is
no CLI cache-root override. The defaults are
`<os.UserCacheDir>/notarius/chunk-plans` and
`<os.UserCacheDir>/notarius/checkpoints`; on Unix, `os.UserCacheDir` ordinarily
uses an absolute `$XDG_CACHE_HOME` or falls back to `$HOME/.cache`. A relative
`XDG_CACHE_HOME` is an error.
When `chunk_cache.directory` is unset, the root is
`<os.UserCacheDir>/notarius/chunk-plans`. On Unix this is ordinarily
`$XDG_CACHE_HOME/notarius/chunk-plans` when `XDG_CACHE_HOME` is an absolute
path, or `$HOME/.cache/notarius/chunk-plans` when it is unset. A relative
`XDG_CACHE_HOME` is rejected by `os.UserCacheDir`; Notarius reports that as a
configuration error and does not fall back to another directory.
Checkpoint I/O occurs only for `notarius run --resume`. That invocation loads
compatible checkpoints and records work it executes. Without `--resume`,
Notarius does not resolve, create, load, or record the checkpoint root.
## Diagnostics
`debug.directory` chooses a root but never enables debug capture. Its precedence
is `--debug-dir`, `NOTARIUS_DEBUG_DIR`, the file value, then the default.
Only `--debug` requests a bundle; `--debug-dir` is valid only with `--debug`.
Preferred workspace diagnostics fields:
Every supplied file, environment, and CLI value is validated even when a
higher-precedence value wins.
- `workspace.diagnostics.enabled`: set to `false` to skip creating diagnostics
run directories and diagnostics artifacts.
- `workspace.diagnostics.retention`: `auto`, `always`, or `never`.
## Version 2 To Version 3 Migration
Defaults for workspace and diagnostics fields are listed in
[Defaults](#defaults).
Version 2 files are rejected. Move each setting to the surface it controls and
remove obsolete enablement and retention controls. This complete before/after
example preserves an existing chunk-plan cache and checkpoint directory while
choosing an output and debug root explicitly.
`workspace.diagnostics.retention` overrides legacy diagnostics retention when
set.
```yaml
# Version 2 (no longer accepted)
version: 2
workspace:
directory: /srv/notarius/state
resume:
enabled: true
debug:
enabled: true
chunk_cache:
directory: /srv/notarius/chunk-plans
mode: auto
diagnostics:
retention: always
diagnostics:
work_dir: /srv/notarius/inspection
```
`diagnostics` fields:
```yaml
# Version 3
version: 3
output:
directory: /srv/notarius/output
cache:
chunk_plans:
directory: /srv/notarius/chunk-plans
mode: auto
checkpoints:
directory: /srv/notarius/state/checkpoints
debug:
directory: /srv/notarius/debug
```
- `work_dir`: deprecated compatibility directory for per-run diagnostics.
- `retention`: deprecated compatibility retention mode. `auto`, `always`, or
`never`.
Run the migrated configuration with `--resume` when checkpoint reuse or
recording is wanted, and with `--debug` when a debug bundle is wanted.
Existing `diagnostics.work_dir`, `diagnostics.retention`, `NOTARIUS_WORK_DIR`,
and `NOTARIUS_DIAGNOSTICS_RETENTION` inputs remain supported for compatibility.
New configuration should use `workspace.directory` and
`workspace.diagnostics.retention` instead.
For retention behavior and the physical diagnostics layout, see
[Operations](operations.md#retention). For the invocation-specific diagnostics
override, see [CLI Reference](cli.md#run).
The removed fields are `workspace.directory`, `workspace.resume.enabled`,
`workspace.debug.enabled`, `workspace.chunk_cache.mode`,
`workspace.chunk_cache.directory`, `workspace.diagnostics.enabled`,
`workspace.diagnostics.retention`, `diagnostics.work_dir`, and
`diagnostics.retention`. The removed environment variables are `NOTARIUS_WORKSPACE_DIR`,
`NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED`,
`NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION`,
`NOTARIUS_WORKSPACE_RESUME_ENABLED`, `NOTARIUS_WORKSPACE_DEBUG_ENABLED`,
`NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE`,
`NOTARIUS_WORKSPACE_CHUNK_CACHE_DIR`, `NOTARIUS_WORK_DIR`, and
`NOTARIUS_DIAGNOSTICS_RETENTION`. The chunk-cache variables are replaced by
`NOTARIUS_CACHE_CHUNK_PLANS_MODE` and
`NOTARIUS_CACHE_CHUNK_PLANS_DIR`; the former shared directory has no direct
replacement.
## Validation
@@ -417,8 +451,8 @@ Configuration validation checks:
- positive global LLM concurrency;
- supported stage-worker keys and an effective extract worker count in the
inclusive range `1..concurrency.total_llm`;
- supported diagnostics retention and non-empty work directory;
- a supported chunk-cache mode and a chunk-cache directory without NUL bytes;
- non-empty output and debug directories;
- a supported chunk-cache mode and state-surface directories without NUL bytes;
- stale removed fields such as `llm_profiles`.
Pipeline resolution additionally checks:

2
docs/development.md
View File

@@ -19,7 +19,7 @@ implemented component map.
| Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. |
| Production modules or validators | [Module Internals](internal/modules.md) | It documents implemented module contracts, capabilities, assets, and registration. |
| LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and Scriptorium integration. |
| Diagnostics, workspace state, resume, or debug artifacts | [Diagnostics Internals](internal/diagnostics.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. |
| Output, cache, resume, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. |
| CLI or user-visible configuration behavior | [CLI Reference](cli.md) and [Configuration](config.md) | These are the canonical user and operator references. |
| External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. |
| Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. |

105
docs/internal/diagnostics.md
View File

@@ -1,105 +0,0 @@
# Diagnostics Internals
`internal/core/diagnostics` provides the scoped writer and retention decision
used by `internal/cli`. The physical layout, artifact inventory, retention
semantics, failure inspection, and cleanup procedures are canonical in
[Operations](../operations.md#diagnostics-directory). Configuration fields and
defaults are canonical in [Configuration](../config.md#diagnostics).
## Run Directory
`NewRunDirectory` normalizes empty constructor inputs, creates the effective
diagnostics root when needed, and allocates a unique timestamp-based child
directory. It retries a bounded number of collisions before failing. The
resulting `RunDirectory` retains its creation time and retention mode for later
metadata and cleanup decisions.
The package does not resolve workspace configuration. `internal/cli` derives
effective workspace settings first and passes the diagnostics root into the
constructor.
## Scoped Writers
Typed methods on `RunDirectory` write invocation metadata, redacted effective
configuration, resolved pipeline/reference data, checkpoint events, source data
when explicitly requested, manifests, reports, warnings, redacted chunk-plan
summaries, and error text. The
current filenames and their operator-facing contents are listed in
[Operations](../operations.md#diagnostics-directory).
The chunk-plan summary records the effective mode, source and candidate
digests, requested module, lookup decision, materialization action, validation
decision, and publication decision. Its closed decision values make failures
and recoverable invalid records inspectable without serializing plan ranges,
annotations, source content, reference content, prompts, model responses, or
raw invalid-file bytes. Lookup reasons are derived only from the lookup status:
`stored chunk plan is valid`, `chunk plan not found`, `stored chunk plan is
invalid`, or `chunk plan lookup skipped`. Store-provided reasons and malformed
record details never enter this artifact.
JSON methods indent their payload and append a newline. All artifact writes use
a temporary file in the target directory, apply the requested permissions, and
rename it into place. Artifact resolution accepts only a single relative base
name; absolute paths, separators, and paths escaping the run directory fail
before writing.
## Redacted Configuration
`WriteRedactedEffectiveConfig` accepts a `RedactedDiagnosticsPayload` provider
rather than a raw config value. `internal/core/config` implements that contract
by cloning effective config data and removing secret-shaped values before JSON
encoding. The diagnostics package therefore never needs configuration-specific
field knowledge.
## Retention Coordination
`ShouldRetainRunDirectory` is a pure decision over the effective retention mode,
run success, and warning presence. `ApplyRetention` uses that result to remove
only its own run directory. Unsupported modes retain data as a fail-safe, though
normal CLI execution rejects them during config validation.
The meaning of each supported mode belongs in
[Operations](../operations.md#retention); this package implements that contract
without loading config or inspecting run artifacts.
## CLI State Flow
When diagnostics are enabled, `internal/cli` creates the run directory after
configuration loading and before pipeline resolution. It then writes artifacts
as state becomes available: invocation data, effective resolution data,
pipeline results, and the final report. This ordering permits later failures to
retain the context already established.
Failures before construction have no `RunDirectory`. Later failures write an
error log, preserve any available partial manifest and chunk-plan summary, and
apply a failed-run
retention decision. A diagnostics write failure is itself a command failure so
the CLI does not report success after losing requested inspection data.
When diagnostics are disabled, the CLI carries a nil run directory and the
shared `writeDiagnostics` helper turns writes into no-ops. User-facing errors
still go to stderr; that invocation behavior is documented in
[Operations](../operations.md#failures).
## Package Guarantees
- A `RunDirectory` writes and removes only within its allocated directory.
- JSON and error artifacts use atomic replacement.
- Nil receivers and invalid typed payloads return errors rather than panicking.
- Retention never removes a failed run and never targets the diagnostics root.
- Diagnostics models contain inspection metadata, not the durable output
contract.
- Checkpoint and debug serializers remain separate framework components.
- Secret-handling follows the invariant in
[Architecture](../policy/architecture.md#state-output-and-safety).
## Tests To Inspect
- `internal/core/diagnostics/run_dir_test.go`: allocation, artifact confinement,
atomic writes, retention, and failure behavior.
- `internal/core/diagnostics/artifacts_test.go`: stable artifact identifiers.
- `internal/core/config/redaction_test.go`: clone-and-redact payload behavior.
- `internal/core/workspace/settings_test.go`: effective diagnostics-root and
enablement handoff.
- `internal/cli/run_test.go`: creation timing, artifact sequencing, disabled
diagnostics, overrides, failures, and retention integration.

9
docs/internal/llm.md
View File

@@ -106,14 +106,13 @@ schemas remain package-owned.
## Debug And Redaction Boundaries
The pipeline may wrap the client with a debug recorder that captures prepared
prompt/response material for an explicitly enabled debug run. Default
diagnostics and manifests receive identities, hashes, usage, and selected
profile summaries rather than prompt, source, reference, schema, or response
content.
prompt/response material for an explicitly requested debug run. Debug summaries
and manifests receive identities, hashes, usage, and selected profile summaries
rather than prompt, source, reference, schema, or response content.
The Scriptorium error wrapper removes bearer credential values from surfaced
provider errors; `RedactSecrets` and `ErrorWithSecretsRedacted` support known
secret values elsewhere in the runtime. Config diagnostics use a separate
secret values elsewhere in the runtime. Config summaries use a separate
clone-and-redact path in `internal/core/config`. These mechanisms implement the
security invariant in
[Architecture](../policy/architecture.md#state-output-and-safety); operator

27
docs/internal/overview.md
View File

@@ -11,8 +11,8 @@ boundaries and dependency direction belong in
`cmd/notarius` delegates to `internal/cli`, the production composition root.
The CLI loads configuration, builds the production catalogs and runtime
collaborators, invokes `internal/framework/pipeline`, and places the logical
output files returned by the runner. Diagnostics, checkpoints, and debug
recorders are optional side-channel collaborators supplied at this boundary.
output files returned by the runner. Cache and debug collaborators are supplied
at this boundary.
Resolution produces a fixed ordered workflow and a sorted set of artifact
lanes. Preparation constructs the complete module and validator set before the
@@ -25,7 +25,7 @@ normalize continuations that may overlap across lanes.
| Package | Implemented responsibility |
| --- | --- |
| `cmd/notarius` | Executable entry point and process exit delegation. |
| `internal/cli` | Command parsing, config discovery, package-family registrar invocation, LLM client construction, reference materialization, workspace collaborator setup, durable writes, and user-facing results. |
| `internal/cli` | Command parsing, config discovery, package-family registrar invocation, LLM client construction, reference materialization, state collaborator setup, durable writes, and user-facing results. |
## Core Packages
@@ -33,9 +33,9 @@ normalize continuations that may overlap across lanes.
| --- | --- |
| `internal/core/artifacts` | Run-manifest and provenance models. |
| `internal/core/config` | Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline resolution. |
| `internal/core/diagnostics` | Scoped run directories, diagnostics writers, atomic writes, and retention decisions. |
| `internal/core/debugbundle` | Explicit per-run debug-bundle allocation and redacted summary writing. |
| `internal/core/fileio` | Generic confined atomic file and JSON writes with caller-selected permissions. |
| `internal/core/source` | Generic source documents, units, chunks, canonical references, validation, deterministic source digests, and independent metadata materialization. |
| `internal/core/workspace` | Effective workspace settings, confined paths and writes, and checkpoint identity and manifest models. |
## Framework Packages
@@ -46,9 +46,9 @@ normalize continuations that may overlap across lanes.
| `internal/framework/validate` | Shared validator decision and cardinality helpers. |
| `internal/framework/llm` | Scriptorium-backed structured completions, prompt/schema registration, scheduling, profile recording, and secret redaction. |
| `internal/framework/promptfs` | Builds module prompt filesystems from module-owned and caller-provided shared prompt assets. |
| `internal/framework/checkpoint` | Workspace-backed checkpoint loading, recording, and payload serialization. |
| `internal/framework/checkpoint` | Root-based checkpoint loading, recording, identity, and payload serialization. |
| `internal/framework/chunkplan` | Source-addressed chunk-plan filesystem storage, envelope validation, and atomic publication. |
| `internal/framework/debug` | Workspace-backed framework and LLM debug recording. |
| `internal/framework/debug` | Root-based framework and LLM debug recording. |
Framework contracts provide typed artifact, provenance-wrapper, chunk-validator,
serialized-validator, and
@@ -122,14 +122,13 @@ Implementation details for all production extensions are in
| Surface | Implemented owners | Internal purpose |
| --- | --- | --- |
| Durable output | Output module, pipeline runner, and CLI writer | Return logical consumer files and place them for a run. |
| Diagnostics | `internal/core/diagnostics` and `internal/cli` | Record redacted invocation, resolution, result, and failure inspection data. |
| Checkpoints | `internal/framework/checkpoint` and `internal/core/workspace` | Validate and serialize reusable extract, merge, and normalize outcomes. |
| Cache checkpoints | `internal/framework/checkpoint` and `internal/cli` | Validate and serialize reusable extract, merge, and normalize outcomes. |
| Chunk-plan cache | `internal/framework/chunkplan` and `internal/cli` | Persist and select source-addressed plans before framework materialization. |
| Debug artifacts | `internal/framework/debug` and pipeline instrumentation | Capture sensitive framework-boundary and LLM-call material. |
| Debug bundles | `internal/core/debugbundle`, `internal/framework/debug`, and pipeline instrumentation | Persist redacted summaries and application-owned traces. |
Physical layout, retention, recovery, and sensitive-data handling are defined
Physical layout, cleanup, recovery, and sensitive-data handling are defined
in [Operations](../operations.md). Concrete stage modules receive recorder
interfaces and request data, not workspace paths.
interfaces and request data, not physical state roots.
## Focused Documentation
@@ -139,5 +138,5 @@ interfaces and request data, not workspace paths.
registration, and the contributor recipe for adding an extension.
- [LLM Runtime](llm.md): structured completion contracts, Scriptorium adapter,
assets, scheduling, profile recording, and redaction.
- [Diagnostics Internals](diagnostics.md): scoped writers, retention
coordination, CLI failure flow, and path safety.
- [Run State Internals](state.md): output, cache, debug collaborator
composition, and path safety.

6
docs/internal/pipeline.md
View File

@@ -143,8 +143,8 @@ reference spanning the first selected unit through the last.
`pipeline.RunOutput` carries the run manifest, accepted normalized serialized
artifacts with lane and normalizer provenance,
rejected results, warnings, checkpoint events, and logical files returned by the
output encoder. The CLI owns diagnostics and durable filesystem writes after the
runner returns.
output encoder. The CLI owns debug-summary and durable filesystem writes after
the runner returns.
## Execution Flow
@@ -199,7 +199,7 @@ configuration, requested chunker, options, references, lanes, validators, or
LLM profile do not prevent a source-digest hit. The manifest records both the
currently requested chunker and the effective plan producer. Cache state and
paths are configured and operated outside the runner; see
[Configuration](../config.md#workspace) and [Operations](../operations.md).
[Configuration](../config.md#state-surfaces) and [Operations](../operations.md).
The extract job channel has the same capacity as the effective extract worker
count, so dispatch applies backpressure. A fixed continuation executor prevents

56
docs/internal/state.md Normal file
View File

@@ -0,0 +1,56 @@
# Run State Internals
This document describes the implementation collaborators behind output, cache,
and debug state. User-visible fields belong in [Configuration](../config.md),
and layouts and lifecycle belong in [Operations](../operations.md).
## Composition
`internal/cli` is the only physical-path composition root. It resolves the
effective configuration, selects exact roots, allocates requested debug bundles,
constructs cache collaborators, writes logical output files, and reports paths.
Pipeline modules receive interfaces and request data, never output, cache, or
debug roots.
## Output And Cache
The pipeline runner returns logical output files. The CLI places them beneath
the selected output root with confined, atomic writes.
`internal/framework/chunkplan` owns source-addressed plan storage, validation,
and atomic publication. Its store is constructed only when the selected mode is
not `bypass`.
`internal/framework/checkpoint` owns checkpoint identity, manifests, payload
codecs, loader, and recorder. The CLI constructs both loader and recorder only
for a `--resume` invocation. The serialized
`workspace_schema_version` identifiers are frozen wire-compatibility fields;
they do not describe a current public state surface.
`internal/core/fileio` provides confined atomic file writes used by state
collaborators. The chunk-plan store retains its stronger entry validation.
## Debug Bundles
`internal/core/debugbundle` allocates an explicitly requested per-run bundle
with `summary/` and `trace/` roots. `SummaryWriter` persists redacted command,
resolution, run, warning, and failure artifacts. `internal/framework/debug`
implements the pipeline-facing trace recorder under the trace root.
The CLI allocates a bundle before pipeline resolution and treats requested
summary or trace persistence failures as command failures. The pipeline's debug
boundaries redact sensitive metadata and credential-shaped bytes while allowing
application-owned trace material. Debug data is never a checkpoint source or
cache input.
## Tests To Inspect
- `internal/cli/state_surfaces_test.go`: debug allocation and configuration
boundaries.
- `internal/cli/state_hardening_test.go`: independent roots, reuse, failures,
permissions, cleanup, and redaction.
- `internal/core/debugbundle/*_test.go`: bundle allocation and summary writes.
- `internal/framework/checkpoint/*_test.go`: checkpoint serialization and
reuse.
- `internal/framework/chunkplan/store_test.go`: plan envelope, confinement,
publication, and permissions.

347
docs/operations.md
View File

@@ -1,303 +1,170 @@
# Operations
This is the canonical reference for operating implemented Notarius runs.
This is the canonical guide to operating Notarius filesystem state. Command
syntax is in the [CLI reference](cli.md); field definitions and precedence are
in [Configuration](config.md).
## Normal Run
## State Model
A run reads one source file, resolves one configured pipeline, executes its
modules, writes durable output, and writes diagnostics when enabled. Start with
the [README quickstart](../README.md), then use the [CLI reference](cli.md) for
invocation options.
Notarius uses three independent filesystem surfaces:
For production, configure an application-owned workspace such as
`/var/lib/notarius` and ensure the Notarius process can create files below it.
For local development, prefer an ignored project-local workspace such as
`./.notarius/workspace`. See [Configuration](config.md#workspace) for workspace
fields.
- output is durable user data;
- cache is reconstructible chunk-plan and checkpoint state; and
- debug is explicitly requested inspection data.
## Output Directory
Choose separate roots and access controls for each surface. A normal run writes
durable output and may use the chunk-plan cache. It does not create checkpoint
or debug state unless its invocation includes `--resume` or `--debug`.
Durable output is written to:
## Output
Durable logical files are written under:
```text
<output-root>/<run-id>/
```
The output root and its invocation-specific override are defined in the
[CLI reference](cli.md#run). Output writes are atomic per file. The
[JSON output contract](integrations/json-output.md) defines the logical files,
paths, schemas, and media types inside each run directory.
Each output file is written atomically. Notarius never automatically removes
output. The [JSON output contract](integrations/json-output.md) owns the
logical file names, schemas, and media types inside a run directory.
## Diagnostics Directory
Diagnostics are written under:
```text
<diagnostics-work-dir>/<run-id>/
```
When a workspace directory is configured, diagnostics are written under
`<workspace.directory>/diagnostics/<run-id>/`. An invocation-specific override
changes only the diagnostics root, not the workspace root. Configuration and
environment controls are defined in [Configuration](config.md); the override
flag is defined in the [CLI reference](cli.md#run).
Diagnostics can be disabled through configuration. When disabled, Notarius
does not create a diagnostics run directory or write diagnostics artifacts;
concise failures are still printed to stderr.
Implemented diagnostics artifacts:
- `invocation.json`: command metadata such as operation, config path, input
path, selected lanes, run ID, and pipeline digest when available.
- `effective-config.json`: resolved config without raw API keys.
- `resolved-pipeline.json`: resolved module bindings and pipeline digest.
- `resolved-references.json`: resolved reference provenance, including target
stage, lane ID when present, origin, digest, media type, byte size, and
binding source, without reference content.
- `checkpoint-events.json`: checkpoint steps that were reused or executed
during an explicit resume invocation.
- `chunk-plan.json`: redacted plan-cache lookup, validation, and publication
summary. It contains identifiers and decisions, never source units, plan
annotations, reference content, prompts, model responses, or invalid-file
bytes.
- `run-manifest.json`: the same run manifest written to durable output when it
is available, including top-level module metadata when present.
- `warnings.json`: warning list.
- `run-report.json`: counts, status, output path, diagnostics path, and run ID.
- `error.log`: failure message, written after diagnostics directory creation
when a run fails.
Remove an output run directory only after its consumer data is no longer
needed. This is data deletion, not cache cleanup.
## Chunk-Plan Cache
The chunk-plan cache is independent of the workspace and checkpoints. Its
configuration and selection precedence are defined in
[Configuration](config.md#workspace); the invocation override is documented in
the [CLI reference](cli.md#run).
When no root is configured, a normal Linux user uses
`$XDG_CACHE_HOME/notarius/chunk-plans` when `XDG_CACHE_HOME` is a valid absolute
path, or `$HOME/.cache/notarius/chunk-plans` when it is unset. A relative
`XDG_CACHE_HOME` is a configuration error. A configured
`workspace.chunk_cache.directory` is the root itself, not a parent to which
Notarius adds a suffix.
Each source digest has one file:
Chunk plans are stored at:
```text
<chunk-plan-root>/<source-sha256-hex>/plan.json
```
Directories are created with `0700` permissions and plan files with `0600`.
`auto` reuses a complete valid plan or regenerates an absent or invalid one;
`refresh` deliberately regenerates; `bypass` performs no cache I/O. A stored
plan is still validated and materialized against the current source before use,
and the current run's chunk validators always run. Invalid state is recoverable:
an `auto` run regenerates and atomically replaces it only after validation
succeeds. Delete an exact cache root or digest directory only when regeneration
cost is acceptable.
`auto` reuses a complete valid plan or regenerates missing or invalid state.
`refresh` regenerates and atomically replaces a plan after chunk validation.
`bypass` performs no plan-cache I/O and does not resolve or create the root.
Plan selection is source-addressed and independent of checkpoint and debug
roots.
The configured root is the cache trust boundary. An operator-supplied root path
may itself resolve through a symlink, but cache-owned digest directories and
plan files must be real directory and regular-file entries. Links or other
unexpected entry types are rejected rather than followed.
When its directory is empty in configuration, the root is
`<os.UserCacheDir>/notarius/chunk-plans`. A configured directory is the exact
root; no suffix is appended. Directories and files created by the store use
`0700` and `0600` permissions on supported Unix systems. The configured root
is a trust boundary: do not share it among mutually untrusted users.
Publication uses atomic replacement. Concurrent readers observe a complete old
or new plan, and concurrent writers leave one complete valid winner; there is
no history, lock protocol, or rollback facility. Do not share a cache root
between mutually untrusted users because plans can contain source-derived
structure and annotations.
Remove an exact digest directory or the configured root only when accepting the
cost of recomputing plans and any chunk-stage work. Cache publication is atomic;
there is no history, locking, garbage collection, or rollback facility.
For a system-wide Linux deployment under a dedicated service account, configure
and provision a separate restrictive root such as:
For a Linux service account, provision a dedicated restrictive root such as:
```yaml
workspace:
chunk_cache:
cache:
chunk_plans:
directory: /var/cache/notarius/chunk-plans
```
`/var/cache/notarius/chunk-plans` is a recommended configured service root, not
the unprivileged default. The operator or package installer must create it with
restrictive service-account ownership and permissions before use.
## Checkpoint Cache
## Checkpoints
Checkpoint state is used only by an invocation with `--resume`. That invocation
loads compatible completed work and records checkpoints for work it executes.
Without `--resume`, Notarius neither resolves nor creates the checkpoint root,
and neither loads nor records checkpoints.
When checkpoint writing is enabled for a configured workspace, runs write
checkpoints under:
Checkpoints use the selected root and the existing identity hierarchy:
```text
<workspace.directory>/checkpoints/<pipeline-id>/<input-key>-<source-or-input-digest>/<pipeline-digest>/<identity-digest>/
<checkpoint-root>/<pipeline-id>/<input-key>-<source-or-input-digest>/<pipeline-digest>/<identity-digest>/...
```
Each workflow step owns its own manifest and payload files. There is no
root-level checkpoint summary. Ordinary invocations execute the pipeline
normally and refresh checkpoints. An explicit resume invocation reuses valid
checkpoints and executes any missing, invalid, or incompatible step normally.
Configuration controls checkpoint writing, while the explicit resume option is
defined in the [Configuration](config.md#workspace) and
[CLI](cli.md#run) references.
An empty configured directory selects
`<os.UserCacheDir>/notarius/checkpoints`. The root is exact when configured.
Created directories and files use `0700` and `0600` permissions on supported
Unix systems.
Checkpoints do not include raw prompts, raw reference contents, raw LLM request
payloads, or debug traces. They can still contain source text, intermediate
extracted content, rejected outputs, metadata, warnings, and content digests.
Treat checkpoint directories as sensitive local state.
Checkpoint payloads can contain source text, intermediate artifacts, metadata,
warnings, and content digests. Treat them as sensitive derived application
data. Compatible files from a former checkpoint root remain reusable when
`cache.checkpoints.directory` names that exact existing root. They are not
moved, migrated, or deleted automatically. The frozen serialized identifier
`workspace_schema_version` remains part of checkpoint compatibility; it is not
a configuration setting.
A checkpoint is reused only when its stored status, dependencies, payloads, and
digests match the current invocation. Changes to input bytes, the resolved
pipeline, selected lanes, the runtime LLM profile override, or bound reference
content invalidate reuse. The resolved pipeline identity includes effective
default and explicitly overridden validator chains, so adding, removing,
reordering, or reconfiguring a validator invalidates checkpoints even when the
pipeline profile itself is unchanged.
For a Linux service account, independently provision:
Typed artifact checkpoints additionally record codec-owned bytes, artifact
kind, schema ID and version, exact schema digest, and media type. A missing or
mismatched codec identity, or bytes the current codec cannot decode, is reported
as a checkpoint reuse miss. The affected operation executes normally and, when
checkpoint writing is enabled, replaces the incompatible checkpoint.
```yaml
cache:
checkpoints:
directory: /var/cache/notarius/checkpoints
```
Current checkpoint manifests use workspace schema `notarius.workspace.v2`.
Manifests written with `notarius.workspace.v1` are incompatible because their
chunk provenance has an older shape. On the first explicit resume after an
upgrade, each affected checkpoint is treated as a reuse miss and its workflow
step executes normally. The compatibility check does not migrate or delete the
v1 files; when checkpoint writing is enabled, normal execution refreshes the
affected checkpoint files in the current schema.
Remove an exact checkpoint identity directory or the configured root only when
recomputation is acceptable.
Runs do not reuse checkpoints unless explicitly requested. Without reuse, the
workflow executes normally and refreshes checkpoint files when checkpointing is
enabled.
## Debug Bundles
## Debug
When debug recording is enabled for a configured workspace, runs write debug
artifacts under:
Only `notarius run --debug` enables debug collection. The selected root contains
one retained bundle per invocation:
```text
<workspace.directory>/debug/<run-id>/
<debug-root>/<run-id>/
summary/
trace/
```
Debug output is per invocation. It is independent of checkpointing and is not
used for resume. Enabling debug does not write checkpoints, and enabling resume
checkpointing does not write debug output.
`summary/` contains redacted invocation, effective-configuration, resolved
pipeline and reference provenance, checkpoint and chunk-plan decisions, run
manifest, warnings, report, and any available error text. It excludes raw
source, references, annotations, prompts, model responses, credentials, and
malformed cache bytes.
Debug artifacts include inputs and outputs for source, chunk, extract, merge,
normalize, and output work, structured LLM request and response data, validator
requests and results, timing, and retry attempt metadata. LLM calls made inside
a module retry write `prompt-000N.json`, `response-000N.json`, and
`response-content-000N.*` files under that attempt directory and are linked
from its `llm_calls` array. Validator calls use separate attempt scopes under
`validate/` and are not duplicated into the module attempt. Prompt content is
written inline in the prompt artifact. The response metadata and body use the
paired files described above; the body is pretty-printed JSON when possible
and raw text otherwise. Retrying stages use these stable module-attempt paths:
`trace/` contains application-owned execution detail, including source and
stage material, plans, chunks, validator attempts, prompts, model responses,
timing, and serialized artifacts. It may retain application data omitted from
output. Credentials, credential-shaped values, sensitive metadata, unrelated
environment values, and unrelated filesystem content are not captured.
```text
chunk/attempt-<NN>.json
extract/<lane-id>/chunk-<NNNNNN>/attempt-<NN>.json
merge/<lane-id>/attempt-<NN>.json
merge/<lane-id>/attempt-<NN>/prompt-<NNNN>.json
merge/<lane-id>/attempt-<NN>/response-<NNNN>.json
merge/<lane-id>/attempt-<NN>/response-content-<NNNN>.<ext>
Bundles inherit the sensitivity of the application data they capture. Their
additional risk comes from copying and aggregating that data, so restrict
access, avoid shared roots between untrusted users, and define retention outside
Notarius. Created bundle directories use `0700` and files use `0600` on
supported Unix systems.
normalize/<lane-id>/attempt-<NN>.json
normalize/<lane-id>/attempt-<NN>/prompt-<NNNN>.json
normalize/<lane-id>/attempt-<NN>/response-<NNNN>.json
normalize/<lane-id>/attempt-<NN>/response-content-<NNNN>.<ext>
```
Notarius never automatically deletes a requested bundle. If allocation
succeeds, its path is reported on success and failure. A requested summary or
trace write failure makes the command fail, preserving whatever bundle data was
already written for inspection.
Every executed chunk, extract, merge, and normalize attempt has one terminal
envelope recording acceptance, validator rejection, or a module, validator,
candidate-serialization, or final-serialization error as applicable. It
includes attempt-local warnings and any available candidate or rejection. A
failure before a candidate exists has no candidate payload. If the envelope
cannot be persisted, the run does not retry that module attempt and reports the
debug failure together with any primary attempt error.
## Failures And Warnings
Chunk-plan candidates, materialized chunks, annotations, and chunk-attempt
details appear only in these opt-in debug artifacts. They are intentionally not
included in normal manifests or the `chunk-plan.json` diagnostics summary.
Checkpoint-reused extract, merge, and normalize work retains the stage-level
input and output artifacts but has no retry-attempt artifacts
because no module attempt executed. Debug artifacts may contain source
material, reference material, prompt inputs, model outputs, and other sensitive
data. Typed artifact
envelopes include domain-neutral codec identity, redacted metadata and content,
and digests of the stable codec bytes. API keys are not written, and obvious
credential-shaped values and sensitive map keys are redacted, but debug
directories should still be protected as sensitive local state.
## Retention
Diagnostics retention uses the effective mode selected through configuration;
see [Configuration](config.md#diagnostics) for the fields, environment
overrides, precedence, and default.
- `auto`: keep failed runs and successful runs with warnings; remove successful
warning-free runs.
- `always`: keep every diagnostics run directory.
- `never`: remove successful run directories; failed runs are still retained.
## Failures
Failures before diagnostics directory creation, such as a missing config file or
an unusable diagnostics work directory, are printed to stderr and may not have a
diagnostics run directory.
Failures after diagnostics directory creation are printed to stderr and written
to `error.log`. Depending on where the failure occurred, the directory may also
contain artifacts written before the failure.
If durable output writing fails after the pipeline completes, diagnostics are
retained for inspection.
## Warnings
A successful run with warnings exits with code `0`, prints a warning count to
stderr, and writes warnings to durable output and diagnostics when retained.
The [JSON output contract](integrations/json-output.md) defines durable warning
and validation-status fields.
Failures before debug allocation are reported on stderr without a bundle.
Failures after allocation report the bundle path on stderr and write `error.log`
when that summary write succeeds. An output-write failure leaves the allocated
bundle in place. A successful run with warnings exits `0`, reports a warning
count on stderr, and records warnings in durable output and any requested debug
summary.
## Cleanup
It is safe to remove specific old run directories after their output and
diagnostics are no longer needed:
Use exact paths for manual cleanup. Examples:
```sh
rm -rf /tmp/notarius/run-1234567890
rm -rf ./notarius-output/run-1234567890
rm -rf /var/cache/notarius/chunk-plans/0123abcd
rm -rf /var/cache/notarius/checkpoints/pipeline/input-0123/pipeline-4567/identity-89ab
rm -rf ./notarius-debug/run-1234567890
```
Workspace checkpoint and debug directories can also be removed when no longer
needed. Remove exact identity or run directories, for example:
```sh
rm -rf /var/lib/notarius/checkpoints/dnd-session/seriatim-abcdef123456/7890abcd1234/identityabcd1234
rm -rf /var/lib/notarius/debug/run-1234567890
```
Chunk-plan cache entries can likewise be removed by exact digest directory or
configured root. Removal is recoverable, but the next non-bypass run may need
to regenerate plans and repeat any chunk-stage LLM work.
Use exact run-directory paths. Avoid broad cleanup commands against parent
directories unless they are part of your own operational policy.
Avoid broad recursive cleanup against a parent root unless it is an explicit
operator policy. Output deletion is permanent user-data loss. Cache deletion is
recoverable but can repeat expensive work. Debug deletion removes troubleshooting
evidence and any retained application-data copy.
## Operational Limits
Provider retries and timeouts are handled by Scriptorium according to the
selected execution profile. Pipeline module retry settings are defined in
[Configuration](config.md#module-bindings). There is no separate CLI retry
command.
Extract worker concurrency and actual provider-call concurrency are separate
limits. Their configuration, defaults, and validation are defined in
[Configuration](config.md#concurrency). Cancellation stops undispatched extract
work; already started work is allowed to finish or observe cancellation before
the run reports failure.
Notarius writes local files only. Remote storage and archive management are not
part of the implemented CLI.
[Configuration](config.md#module-bindings). Extract worker concurrency and
actual provider-call concurrency are separate limits; their fields and
validation are defined in [Configuration](config.md#concurrency). Notarius
writes local files only; remote storage and archive management are outside the
implemented CLI.

47
docs/policy/architecture.md
View File

@@ -114,8 +114,8 @@ plan, while the current run still applies its configured chunk validators to
the materialized chunks.
The framework owns orchestration and handoff provenance. Modules return logical
results and warnings; they do not own CLI reporting, workspace paths, durable
file placement, checkpoints, or diagnostics.
results and warnings; they do not own CLI reporting, physical output, cache, or
debug roots, durable file placement, or checkpoint and debug lifecycle.
After pipeline-wide chunking, extraction uses bounded framework concurrency.
One run-wide worker pool receives chunk-scoped lane jobs in deterministic
@@ -188,31 +188,32 @@ record identities and summaries rather than secret or large payload content.
## State, Output, And Safety
Durable output, diagnostics, checkpoints, and debug artifacts are separate
surfaces with separate ownership:
Notarius exposes three filesystem surfaces with independent roots and
lifecycle:
- output modules define logical durable output; the application boundary owns
filesystem placement;
- diagnostics provide redacted run inspection and are not the durable output
contract;
- checkpoints support validated stage reuse and are not diagnostics;
- debug artifacts are opt-in inspection data and may contain sensitive source,
prompt, reference, and model-output content.
- output is durable user data; output modules define logical files and the CLI
owns their placement;
- cache is reconstructible state, with separate chunk-plan and checkpoint
families; and
- debug is explicitly requested inspection data, combining a redacted summary
with a detailed trace.
Chunk-plan cache state is an additional independent surface. It is keyed only
by canonical source digest, is not rooted under `workspace.directory`, and is
not a checkpoint or a diagnostic. A cache record is atomically replaced as one
complete plan envelope; it has no history, locking, or rollback interface.
Invalid records are recoverable cache misses rather than pipeline state.
Chunk plans are keyed only by canonical source digest. Checkpoints are used only
for an invocation that explicitly requests resume. Debug is never a cache input
and is never created without an explicit request. Pipeline modules receive
collaborator interfaces and never physical roots.
Writes of durable state are atomic where practical. Paths for writes, moves,
overwrites, and deletion must be narrow and explicit. Cleanup that can lose data
is opt-in.
Writes are atomic where practical. Paths for writes, moves, overwrites, and
deletion must be narrow and explicit. Notarius never automatically deletes
output or requested debug bundles; cache cleanup is explicit and recoverable.
Secrets must not appear in errors, logs, diagnostics, manifests,
documentation, examples, or redacted configuration. Default logs and
diagnostics must not include large source, prompt, reference, or artifact
payloads.
Secrets must not appear in errors, logs, output, cache, debug summaries,
traces, manifests, documentation, examples, or redacted configuration. Debug
collection is allowlisted to application-owned payloads and must not capture
unrelated process environment values or filesystem content. Trace data may
contain application data and therefore inherits its sensitivity; operators own
access controls and retention. Physical layout and operation are defined in
[Operations](../operations.md).
## Architectural Non-Goals

2
docs/policy/documentation.md
View File

@@ -63,7 +63,7 @@ secret values.
| Documentation organization | `docs/policy/documentation.md` | Documentation ownership, audience boundaries, maintenance rules, and ADR/document lifecycle. | Application architecture or product behavior. |
| CLI contract | `docs/cli.md` | Commands, arguments, flags, invocation semantics, and exit codes. | End-to-end operating procedures, configuration field definitions, runtime filesystem layout, module implementation details. |
| Configuration contract | `docs/config.md` | Discovery and precedence, file schema, fields, defaults, environment overrides, validation rules, and user-selectable module or validator keys. | Complete example files, CLI syntax, runtime state lifecycle, module implementation details. |
| Operations | `docs/operations.md` | Runtime workflows, physical filesystem and state layout, diagnostics use, retention, resume, cleanup, permissions, recovery, and operational limits. | CLI flag syntax, configuration field definitions, logical output schemas, implementation mechanics. |
| Operations | `docs/operations.md` | Runtime workflows, physical filesystem and state layout, output, cache, and debug handling, resume, cleanup, permissions, recovery, and operational limits. | CLI flag syntax, configuration field definitions, logical output schemas, implementation mechanics. |
| Public HTTP contract, if introduced | `docs/api.md` | Routes, authentication, media types, request and response schemas, status codes, pagination, caching, idempotency, rate limits, and HTTP retry semantics. | Client walkthroughs, upstream or downstream integration internals, implementation detail. |
| Consumer guidance, if a public package or API is introduced | `docs/consumers/` | Task-oriented use of the public interface, minimal client examples, and consumer responsibilities. | HTTP wire semantics, external protocol contracts, internal implementation detail. |
| External and durable integration contracts | `docs/integrations/` | External file formats and protocols, upstream and downstream contracts, logical output bundle paths and schemas, media types, and compatibility behavior. | Physical runtime placement and lifecycle, internal transformations, CLI syntax, configuration defaults. |

334
docs/roadmap/adr0006.md
View File

@@ -1,334 +0,0 @@
# ADR-0006 Feature Roadmap
This roadmap defines the intended end state for
[ADR-0006](../adr/0006-separate-output-cache-and-debug-state.md). The feature is
not implemented.
## Intent
Notarius should communicate filesystem behavior in terms of why data exists:
durable output, reconstructible cache, or explicitly requested debug material.
The public configuration and CLI should not expose internal distinctions such
as workspace state or a second diagnostics product.
The implementation may retain focused writers, serializers, and security
boundaries. This roadmap changes their public composition and lifecycle rather
than requiring every kind of data to share one package or physical root.
## Target Configuration
The configuration schema advances to version 3 and uses these top-level
surfaces:
```yaml
version: 3
output:
directory: ./notarius-output
cache:
chunk_plans:
directory: ""
mode: auto
checkpoints:
directory: ""
debug:
directory: ./notarius-debug
```
An empty cache directory selects that cache family's platform-appropriate
per-user default. A configured directory is the exact root; Notarius does not
append the cache-family name to it.
The `workspace` and `diagnostics` top-level configuration surfaces do not exist
in version 3. In particular, version 3 has no persistent debug enablement,
diagnostics enablement or retention, checkpoint enablement, or shared workspace
directory.
### Output configuration
`output.directory` selects the durable output root. Precedence is the explicit
`--output-dir` flag, `NOTARIUS_OUTPUT_DIR`, the file value, then
`./notarius-output`.
The pipeline's existing `pipelines.<id>.output` module binding remains separate:
it selects the output encoder, while top-level `output.directory` selects where
the CLI places that encoder's logical files.
### Chunk-plan cache configuration
`cache.chunk_plans.mode` accepts `auto`, `bypass`, or `refresh` with the
ADR-0005 semantics. Precedence is the explicit `--chunk_cache` flag,
`NOTARIUS_CACHE_CHUNK_PLANS_MODE`, the file value, then `auto`.
`cache.chunk_plans.directory` selects the exact canonical plan root. Precedence
is `NOTARIUS_CACHE_CHUNK_PLANS_DIR`, the file value, then
`<os.UserCacheDir>/notarius/chunk-plans`. On Linux, the default therefore uses a
valid absolute `$XDG_CACHE_HOME` or falls back to `$HOME/.cache` when that
variable is unset. A relative `XDG_CACHE_HOME` remains an error.
The recommended system-service value remains
`/var/cache/notarius/chunk-plans`. ADR-0005 plan identities, envelopes,
permissions, atomic publication, validation, provenance, and cleanup semantics
do not change.
### Checkpoint cache configuration
`cache.checkpoints.directory` selects the exact checkpoint root. Precedence is
`NOTARIUS_CACHE_CHECKPOINTS_DIR`, the file value, then
`<os.UserCacheDir>/notarius/checkpoints`.
For a system-wide Linux deployment under a dedicated service account, the
recommended configured root is `/var/cache/notarius/checkpoints`, provisioned
with restrictive service-account ownership independently from the chunk-plan
root.
Checkpoint directories and files use restrictive permissions because they may
contain source text and intermediate artifacts. They inherit the sensitivity of
the data captured in them. Their existing compatibility validation, pipeline
identity, source and lane payloads, and downstream dependency fingerprints
remain intact. Checkpoints do not store or select chunk results.
There is no `cache.checkpoints.enabled` setting. Checkpoint I/O is controlled by
the invocation's `--resume` flag.
### Debug configuration
`debug.directory` selects the debug root but does not enable debug output.
Precedence is `--debug-dir`, `NOTARIUS_DEBUG_DIR`, the file value, then
`./notarius-debug`.
There is no environment or file setting that enables debug. This prevents an
ambient production configuration from silently recording potentially sensitive
trace data and creating additional retained copies of application data.
## Target CLI
The run command retains `--output-dir`, `--chunk_cache`, and `--resume`, and
adds:
- `--debug`: enable the complete debug bundle for this invocation;
- `--debug-dir <path>`: override its root; valid only with `--debug`.
`--diagnostics-dir` is removed. There is no `--workspace`, `--cache-dir`, or
separate diagnostics flag.
`--resume` means “reuse compatible checkpoints when present and record
checkpoint state for work executed by this invocation.” On a first invocation
with no compatible state, it executes normally and creates checkpoints. A run
without `--resume` neither loads nor records checkpoints and does not resolve or
create the checkpoint root.
The CLI reports the durable output directory on success. When debug is enabled,
it also reports the allocated debug bundle path on success and includes that
path in failure reporting once allocation has succeeded.
Malformed values from every supplied configuration source remain errors even
when a higher-precedence source would otherwise override them.
## Filesystem Layout
### Output
Durable logical files retain the current per-run layout:
```text
<output-root>/<run-id>/...
```
Output creation remains atomic per file. The maintained logical JSON output
contract, including manifests, warnings, rejections, and normalized artifacts,
does not change solely because of this state-model refactor.
### Chunk plans
The existing ADR-0005 layout remains:
```text
<chunk-plan-root>/<source-sha256-hex>/plan.json
```
Chunk-plan state remains independent of checkpoint and debug roots.
### Checkpoints
The current identity hierarchy moves beneath the independently selected
checkpoint root:
```text
<checkpoint-root>/<pipeline-id>/<input-key>-<source-or-input-digest>/<pipeline-digest>/<identity-digest>/...
```
The configured root replaces the former implicit
`<workspace.directory>/checkpoints` prefix. Existing manifests and payload
schemas remain compatible when pointed at their prior physical checkpoint root.
### Debug bundle
One explicitly requested bundle is allocated before pipeline resolution at:
```text
<debug-root>/<run-id>/
summary/
trace/
```
`summary/` contains the redacted artifacts currently associated with
diagnostics: invocation metadata, redacted effective configuration, resolved
pipeline and reference provenance, checkpoint decisions, chunk-plan decisions,
run manifest, warnings, run report, and error text when available.
`trace/` contains the existing deep debug material: source and stage inputs and
outputs, plans and materialized chunks, annotations, validator attempts,
prompts, model responses, retry timing, and intermediate serialized artifacts.
Summary data never contains raw source, references, annotations, prompts, model
responses, credentials, or malformed cache bytes. Trace data may contain all
of those except credentials, which remain redacted. A debug bundle inherits the
sensitivity of the most sensitive data captured in it; it does not constitute a
separate, intrinsically higher sensitivity class. Because it aggregates and
retains an additional copy of application data, its root and files use
restrictive permissions by default.
Trace data may preserve source or reference content that the durable output
summarized, rejected, or intentionally omitted. This can make the bundle more
sensitive than the output alone, but not more sensitive than the complete set
of application data supplied to or produced by the run. Restrictive defaults
are therefore precautionary handling for data of unknown sensitivity, not a
classification of debug material as uniquely sensitive.
Debug persistence failures are command failures because the operator explicitly
requested the bundle. No debug directory is created without `--debug`. A
requested bundle is never automatically deleted based on success, warnings, or
failure; cleanup is explicit.
## Runtime Ownership
The CLI remains the composition root for physical paths and filesystem-backed
collaborators. Pipeline modules continue to return logical data and never
receive output, cache, or debug roots.
The framework may retain separate interfaces for checkpoint recording and
loading, chunk-plan storage, redacted summary recording, and deep trace
recording. The single debug facility may compose multiple internal recorders;
unification does not require weakening redaction or passing raw trace data
through summary models.
The public `workspace` type and terminology disappear from configuration,
invocation messages, debug metadata, and operator documentation. The standalone
diagnostics directory and retention surface also disappear. Domain contracts
that use “diagnostic” in another sense, such as a validator-provided diagnostic
artifact path, are not renamed solely by this decision. Generic confined-path
and atomic-write helpers may remain in an internal package but must not impose a
workspace abstraction on callers.
## Security and Lifecycle
- Output is durable user data. Notarius never automatically deletes it.
- Chunk plans and checkpoints are reconstructible cache. Exact entries or roots
may be removed, with the documented cost of recomputation.
- Debug is explicit inspection data. It is off by default, retained when
requested, and removed only by an explicit operator action.
- Checkpoints and debug bundles inherit the sensitivity of the application data
they capture. Their security concern is the additional retained copy and, for
debug bundles, aggregation of that data—not an intrinsically higher
sensitivity classification.
- Operators are responsible for choosing debug locations, access controls, and
retention appropriate to the data processed by the run. Documentation must
explain that a bundle can retain material omitted from the durable output.
- Cache and debug roots are separate trust boundaries and must not be shared
among mutually untrusted users.
- Credentials are excluded from output, cache, summaries, traces, logs, errors,
examples, and redacted configuration.
- Debug collection is allowlisted to application-owned data. It must not capture
unrelated process environment values, host secrets, or arbitrary filesystem
content merely because they are available to the process.
- Default runs may create output and use the default chunk-plan cache, but do
not create checkpoint or debug state unless their invocation flags request
those surfaces.
## Compatibility Policy
Version 3 is a deliberate configuration break. Version 2 files are rejected
with an actionable message directing operators to the migration documentation;
Notarius does not retain parallel legacy field parsing indefinitely.
The version 2 migration is:
- `workspace.chunk_cache.mode` -> `cache.chunk_plans.mode`;
- `workspace.chunk_cache.directory` -> `cache.chunk_plans.directory`;
- `<workspace.directory>/checkpoints` -> `cache.checkpoints.directory`, when
existing checkpoint reuse is desired;
- `--diagnostics-dir`, diagnostics fields, workspace diagnostics fields, and
their environment variables -> removal or the new debug controls;
- `workspace.debug.enabled` -> removal; invoke with `--debug`;
- `<workspace.directory>/debug` -> `debug.directory`, if the same parent is
desired; and
- `workspace.resume.enabled` -> removal; invoke with `--resume`.
Environment migration is:
- `NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE` ->
`NOTARIUS_CACHE_CHUNK_PLANS_MODE`;
- `NOTARIUS_WORKSPACE_CHUNK_CACHE_DIR` ->
`NOTARIUS_CACHE_CHUNK_PLANS_DIR`;
- `NOTARIUS_WORKSPACE_DIR` -> no direct replacement; configure the applicable
output, checkpoint-cache, and debug roots separately;
- `NOTARIUS_WORKSPACE_RESUME_ENABLED` and
`NOTARIUS_WORKSPACE_DEBUG_ENABLED` -> removal in favor of invocation flags;
and
- `NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED`,
`NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION`, `NOTARIUS_WORK_DIR`, and
`NOTARIUS_DIAGNOSTICS_RETENTION` -> removal with the standalone diagnostics
surface.
Existing chunk-plan files are reused without migration when the new directory
resolves to the same root. Existing checkpoint files are reusable when
`cache.checkpoints.directory` names the former checkpoint root and their normal
identity and schema checks pass. Existing diagnostics and debug directories are
not moved, merged, or deleted automatically.
The durable output schema and ADR-0005 cache envelope schema are not versioned
solely because configuration advances to version 3.
## Documentation Outcomes
When implemented:
- the CLI reference owns all flags and invocation semantics;
- configuration owns version 3 fields, environment variables, defaults, and
precedence;
- operations owns physical layouts, permissions, cleanup, failure inspection,
and per-user and system-service cache guidance;
- the integration contract continues to own durable logical output;
- internal pipeline and component documents explain collaborator composition
without reintroducing public workspace terminology; and
- migration documentation provides one complete version 2 to version 3 example.
Current-behavior documentation must not describe this target before the
corresponding behavior is implemented.
## Non-Goals
This feature does not introduce:
- remote output, cache, or debug storage;
- cache garbage collection, quotas, archival, history, or rollback;
- automatic debug retention or upload;
- a daemon logging, metrics, or telemetry system;
- a change to pipeline topology, artifact schemas, chunk-plan identity, or
module contracts; or
- a requirement that output, different cache families, and debug share one
physical parent directory.
## Completion Outcomes
The feature is complete when ordinary runs expose only durable output and
canonical chunk-plan cache behavior; checkpoint I/O occurs only under
`--resume`; one `--debug` invocation produces a complete retained summary and
trace bundle; no public workspace or diagnostics configuration remains; version
2 migration failures are actionable; existing compatible chunk plans and
checkpoints remain reusable at explicitly selected roots; and all configuration,
CLI, filesystem, security, compatibility, integration, and repository-wide
tests pass.

657
docs/roadmap/implementation.md
View File

@@ -1,657 +0,0 @@
# ADR-0006 Staged Implementation Plan
This document is the executable implementation plan for the target state in
[ADR-0006 Feature Roadmap](adr0006.md), governed by
[ADR-0006](../adr/0006-separate-output-cache-and-debug-state.md). The feature is
not implemented.
The audience is an LLM coding agent. Implement the stages in order. Each stage
is scoped to finish with a compiling, tested repository and may be assigned as
one implementation prompt.
## Execution Rules
Before every stage:
1. read [Development](../development.md), both documents under `docs/policy/`,
ADR-0006, the feature roadmap, this plan, and every stage-specific document
named below;
2. inspect `git status` and preserve user changes;
3. inspect the focused package contracts and tests before editing; and
4. confirm that all earlier stages are complete.
During every stage:
- implement only that stage and prerequisites discovered to be inseparable;
- keep physical path selection in the CLI composition root and keep pipeline
modules independent of output, cache, and debug roots;
- preserve checkpoint identity, chunk-plan identity, durable output schemas,
deterministic ordering, cancellation, redaction, confined paths, and atomic
writes unless this plan expressly changes them;
- treat debug persistence failures as command failures once debug is requested;
- add or update focused tests with every behavior change;
- keep unimplemented behavior under `docs/roadmap/` until the final
documentation stage; and
- do not begin the next stage after meeting the current exit criteria.
At the end of every stage:
1. run the focused tests listed for that stage;
2. run `go test ./...`;
3. run `go vet ./...`;
4. run `go build ./cmd/notarius`;
5. run `git diff --check`; and
6. mark the stage complete in this document only after all checks pass.
If repository reality conflicts with this plan, stop and update the plan in the
same change before implementing a materially different design. Do not silently
invent a new public contract.
## Fixed Decisions
The following choices are settled for this implementation.
### Public state model
- The only public filesystem surfaces are output, cache, and debug.
- Output is durable user data. Cache is reconstructible. Debug is explicitly
requested inspection data and is not a cache input.
- `workspace` and standalone `diagnostics` disappear from configuration, CLI
flags, operator messages, debug metadata, package names exposed to callers,
and current-behavior documentation.
- Domain uses of the word “diagnostic,” including validator diagnostic artifact
paths, are unchanged when they do not describe the removed filesystem
surface.
### Configuration version 3
- `SupportedFileConfigVersion` becomes `3`. Version 3 has these runtime types:
```go
type OutputConfig struct {
Directory string `json:"directory"`
}
type CacheConfig struct {
ChunkPlans ChunkPlanCacheConfig `json:"chunk_plans"`
Checkpoints CheckpointCacheConfig `json:"checkpoints"`
}
type ChunkPlanCacheConfig struct {
Directory string `json:"directory,omitempty"`
Mode pipeline.ChunkCacheMode `json:"mode"`
}
type CheckpointCacheConfig struct {
Directory string `json:"directory,omitempty"`
}
type DebugConfig struct {
Directory string `json:"directory"`
}
```
`Config` contains `Output`, `Cache`, and `Debug` fields and no `Workspace` or
`Diagnostics` field. File-config types mirror the YAML shape in the feature
roadmap.
- Defaults are `./notarius-output`, chunk-plan mode `auto`, empty cache
directories meaning per-user defaults, and `./notarius-debug`.
- There is no persistent checkpoint enablement, resume enablement, debug
enablement, diagnostics enablement, or retention setting.
- Output-directory precedence is explicit `--output-dir`,
`NOTARIUS_OUTPUT_DIR`, file, then `./notarius-output`.
- Chunk-plan mode precedence is explicit `--chunk_cache`,
`NOTARIUS_CACHE_CHUNK_PLANS_MODE`, file, then `auto`.
- Chunk-plan root precedence is `NOTARIUS_CACHE_CHUNK_PLANS_DIR`, file, then
`<os.UserCacheDir>/notarius/chunk-plans`.
- Checkpoint root precedence is `NOTARIUS_CACHE_CHECKPOINTS_DIR`, file, then
`<os.UserCacheDir>/notarius/checkpoints`.
- Debug-root precedence is explicit `--debug-dir`, `NOTARIUS_DEBUG_DIR`, file,
then `./notarius-debug`. A directory value never enables debug.
- A configured cache directory names the exact family root. No family suffix is
appended. Explicit relative paths retain the current CLI convention and are
interpreted relative to the process working directory.
- Empty cache directory fields in a version 3 file select their per-user
defaults. A supplied environment directory override and an explicit CLI
directory flag must be non-empty after trimming. Empty output or debug file
values and malformed modes are errors.
- Every supplied file, environment, and CLI value is validated even when a
higher-precedence value wins. Apply CLI overrides only after file and
environment parsing has succeeded.
- Parse the YAML version header before decoding the strict version 3 schema so
a version 2 file receives the intentional migration error rather than an
incidental unknown-field error. Continue to use `KnownFields(true)` for the
supported schema.
### Cache families
- ADR-0005 behavior and the chunk-plan envelope and path layout remain
byte-compatible. Move only configuration ownership and the default-root
helper; do not change lookup, validation, publication, provenance, or
materialization behavior.
- Chunk-plan `bypass` does not resolve or create its root. `auto` and `refresh`
resolve it lazily when the run constructs the store.
- Checkpoint I/O exists only for an invocation with `--resume`. Without that
flag, the CLI does not resolve the checkpoint root, compute a checkpoint
identity, construct a checkpoint store, load a checkpoint, or record one.
- With `--resume`, the CLI constructs both loader and recorder. Compatible
state is reused; missing or incompatible state executes normally and writes
replacement checkpoints.
- Checkpoint identities, relative paths, manifests, payloads, compatibility
rules, and dependency fingerprints remain unchanged. Existing files are
reusable when the new root points at the former
`<workspace.directory>/checkpoints` directory.
- Preserve the existing checkpoint JSON field
`workspace_schema_version` and values `notarius.workspace.v2` and
`notarius.workspace.v1`. They are frozen wire compatibility identifiers, not
public configuration or package terminology. Rename internal Go identifiers
only if doing so cannot alter serialized bytes or compatibility messages.
- The recommended Linux service roots are
`/var/cache/notarius/chunk-plans` and `/var/cache/notarius/checkpoints`,
provisioned independently for the dedicated service account.
### Debug bundle
- `--debug` is the only debug enablement control. `--debug-dir <path>` is valid
only when `--debug` is present; misuse is a CLI syntax error with exit code
`2`.
- One requested bundle is allocated before pipeline resolution at
`<debug-root>/<run-id>/`, with `summary/` and `trace/` children.
- Bundle directories are created with mode `0700` and files with mode `0600`.
Do not silently change permissions on a pre-existing operator-owned debug
parent; enforce restrictive permissions on directories and files created for
the bundle.
- `summary/` contains the current redacted diagnostics artifacts, using their
current logical filenames where applicable. `trace/` contains the current
deep debug paths. There is no retention mode or automatic deletion.
- The summary run report uses `debug_path`, not `diagnostics_path`. Invocation
and effective-config payloads use debug/state terminology and contain no
obsolete public workspace settings.
- The redacted summary excludes raw source, references, annotations, prompts,
model responses, credentials, and malformed cache bytes. Trace data may
contain application data but never credentials.
- Debug collection records only explicit application-owned payloads. It does
not dump the process environment, inspect unrelated files, or capture host
secrets. Existing sensitive-key and credential-shaped-value redaction stays
in force at every serialization boundary.
- A bundle inherits the sensitivity of its captured application data. The
additional operational risks are copying, aggregation, and retention, not an
intrinsically higher sensitivity class.
- If any requested summary or trace write fails, the command exits `1`. When a
primary operation and a debug write both fail, report both errors on stderr
without recursively attempting the same failed write.
- On success, stdout reports the durable output directory and, when enabled,
the debug bundle path. Once allocation succeeds, failure output also names
the bundle path.
### Compatibility and documentation
- Version 2 configuration is rejected; no legacy fields or environment aliases
remain after the implementation. The error directs the operator to the
version 2-to-3 migration section in `docs/config.md`.
- Existing chunk-plan and checkpoint state is never moved or deleted
automatically. Existing diagnostics and debug directories are likewise left
untouched.
- The durable output contract and chunk-plan envelope schema are not versioned
because of this refactor.
- `docs/config.md` owns the complete version 2-to-3 configuration and
environment migration example. `docs/operations.md` owns physical state reuse
and cleanup guidance. Other documents link to those owners rather than
duplicating volatile details.
## Planned Internal Boundaries
Use these package responsibilities unless an existing collision requires the
smallest obvious naming adjustment.
- `internal/core/fileio` owns generic confined relative-path construction and
atomic JSON/byte writes with caller-selected directory and file modes. Its
errors say “file” or “artifact,” never “workspace.” It must not replace the
chunk-plan store's stronger `os.Root`-based entry protections.
- `internal/framework/checkpoint` owns checkpoint identity, manifests, payload
codecs, filesystem loader, and filesystem recorder. Constructors accept an
exact checkpoint root and an identity; they do not accept a workspace or a
general state settings object.
- `internal/framework/chunkplan` continues to own durable chunk-plan storage.
Per-user cache-root default helpers belong with configuration/path resolution,
not in a workspace package.
- `internal/core/debugbundle` owns allocation of a per-run bundle and the
redacted summary writer. It exposes exact `Path`, `SummaryRoot`, and
`TraceRoot` values and has no retention API.
- `internal/framework/debug` remains the pipeline-facing deep-trace adapter but
becomes a root-based filesystem recorder aimed at the bundle's `trace/`
directory. Pipeline debug interfaces and synchronization remain unchanged.
- `internal/cli` remains the only composition root. It resolves effective
roots, allocates requested state, constructs collaborators, writes summary
artifacts, and reports paths.
After the cutover, delete `internal/core/workspace` and
`internal/core/diagnostics` if no remaining code has a distinct domain-neutral
reason to retain them. Do not preserve wrapper packages solely to avoid
updating imports.
## Stage 1: Accept ADR-0006 and decouple cache storage internals
**Status:** Not started
### Objective
Accept the architectural decision and remove checkpoint and chunk-plan storage
from workspace-specific constructors without changing current public
configuration or CLI behavior yet.
### Read first
- `internal/core/workspace/`
- `internal/framework/checkpoint/`
- `internal/framework/chunkplan/`
- `internal/framework/pipeline/checkpoint.go`
- `internal/cli/chunk_cache_test.go`
- the checkpoint sections of `docs/internal/pipeline.md` and
`docs/internal/diagnostics.md`
### Implement
1. Confirm ADR-0006 matches the finalized feature roadmap, then change only its
status from `Proposed` to `Accepted`.
2. Add the generic confined-write primitives in `internal/core/fileio`, with
caller-selected `0700`/`0600` support and focused path, atomicity, and
permission tests.
3. Move checkpoint identity, manifest, path, and persistence ownership into
`internal/framework/checkpoint`. Preserve every serialized field, schema
string, digest input, path component, validation reason, and payload byte.
4. Replace `NewWorkspaceRecorder` and `NewWorkspaceLoader` with root-based
filesystem constructors. During this preparatory stage, adapt the existing
CLI to pass the checkpoint root derived from its current workspace settings
so public behavior remains unchanged.
5. Move `DefaultChunkPlanRoot` out of `internal/core/workspace` into the
configuration/path-resolution boundary and retain the existing injected
`UserCacheDir` behavior and errors.
6. Remove checkpoint-specific and chunk-plan-specific files from
`internal/core/workspace`. Leave only pieces still required by the current
diagnostics/debug composition until later stages.
### Tests
Prove that:
- old and refactored checkpoint identities and relative paths are identical;
- representative manifests marshal to identical JSON and v1/v2 compatibility
behavior is unchanged;
- a root-based recorder's output is reusable by the root-based loader;
- confined writes reject absolute, unclean, traversal, and backslash paths;
- atomic writes use the requested modes and do not leave temporary files after
failure; and
- chunk-plan default-root and bypass tests remain unchanged in behavior.
Run at minimum:
```sh
go test ./internal/core/fileio
go test ./internal/framework/checkpoint ./internal/framework/chunkplan
go test ./internal/cli
```
### Exit criteria
ADR-0006 is accepted; cache persistence no longer depends on workspace-owned
identity or constructors; all current user-visible behavior remains unchanged;
and the repository-wide checks pass.
## Stage 2: Build the unified debug-bundle collaborators
**Status:** Not started
### Objective
Implement and test the target debug bundle behind internal constructors before
changing the public CLI enablement and configuration model.
### Read first
- `internal/core/diagnostics/`
- `internal/framework/debug/recorder.go`
- `internal/framework/pipeline/debug.go`
- `internal/framework/pipeline/runner_attempt_debug_test.go`
- `internal/framework/pipeline/runner_terminal_debug_test.go`
- `internal/core/config/redaction.go`
- [Diagnostics Internals](../internal/diagnostics.md)
### Implement
1. Add `internal/core/debugbundle` with a collision-safe allocator that accepts
the exact debug parent, uses the established `run-<unix-nanoseconds>` ID
shape, creates `<run-id>/summary` and `<run-id>/trace`, and returns a bundle
exposing those exact paths.
2. Port the current diagnostics artifact constants and redacted writer methods
into a summary writer rooted at `summary/`. Remove retention decisions from
the new type. Use `0600` files and confined atomic writes.
3. Define summary-owned invocation and run-report payloads. Rename
diagnostics-specific methods and fields to summary/debug terminology while
retaining the useful artifact filenames listed in the feature roadmap.
4. Replace the deep trace recorder's workspace-settings constructor with a
constructor accepting the exact `trace/` root. Keep the existing
`pipeline.DebugRecorder` contract and synchronization wrapper.
5. Rename `RedactedDiagnosticsPayload` to a neutral redacted-summary method and
update its tests. Ensure the v3 effective configuration planned for the next
stage can be represented without secrets or raw application payloads.
6. Do not expose `--debug` or change current run behavior in this stage. The new
bundle is an internal collaborator exercised directly by tests until the
atomic CLI cutover.
### Tests
Add focused tests for:
- collision retry and exhausted allocation;
- exact `summary/` and `trace/` layout;
- `0700` bundle directories and `0600` files;
- path traversal and symlink/path confinement behavior;
- every summary artifact and error log;
- absence of retention/deletion behavior;
- credential and sensitive-key redaction;
- no ambient environment or unrelated filesystem capture; and
- trace-recorder write failure propagation and concurrent synchronized writes.
Run at minimum:
```sh
go test ./internal/core/debugbundle
go test ./internal/core/config
go test ./internal/framework/debug ./internal/framework/pipeline
```
### Exit criteria
The target bundle can be allocated and populated through tested summary and
trace collaborators, no public invocation behavior has changed, and all
repository-wide checks pass.
## Stage 3: Cut configuration and CLI behavior over to the three surfaces
**Status:** Not started
### Objective
Make the version 3 configuration, new flags, independent roots, resume policy,
and opt-in debug bundle the complete public runtime behavior in one atomic
cutover.
### Read first
- all files under `internal/core/config/`
- `internal/cli/run.go`, `internal/cli/run_test.go`,
`internal/cli/chunk_cache_test.go`, and `internal/cli/compatibility_test.go`
- the collaborators completed in Stages 1 and 2
- `examples/*.config.yml`
- the Target Configuration, Target CLI, and Filesystem Layout sections of the
feature roadmap
### Implement
1. Replace the runtime and file configuration models with the fixed version 3
types. Remove legacy diagnostics/workspace defaults, merge logic,
validation, effective-config fields, and redaction output.
2. Implement strict two-pass version handling. Reject version 2 with an
actionable migration message; reject other unsupported versions clearly;
and strictly decode version 3 with unknown fields rejected.
3. Implement the five new environment variables and remove all environment
aliases listed for removal in the feature roadmap. Preserve malformed-source
validation before precedence selection.
4. Add `--debug` and `--debug-dir`; remove `--diagnostics-dir`; update usage,
flag reordering, value validation, and exit-code behavior. Keep
`--output-dir`, `--chunk_cache`, and `--resume` with their target semantics.
5. Resolve the effective output directory after configuration loading and CLI
overrides. Use it for durable placement and summary metadata; do not fall
back to a separate CLI-only default helper.
6. When `--debug` is present, resolve the effective debug root and allocate the
bundle after configuration succeeds but before catalog lookup or pipeline
resolution. Use its run ID for the entire invocation, construct the summary
writer and trace recorder, and record invocation metadata immediately.
Without `--debug`, use the existing injected clock to generate a run ID and
construct no debug collaborator or directory.
7. Resolve the chunk-plan root only for `auto` or `refresh`, using the new
configuration family. Preserve all ADR-0005 store construction behavior.
8. When `--resume` is absent, pass no-op checkpoint collaborators without
resolving `UserCacheDir` for checkpoints. When it is present, resolve the
configured or per-user checkpoint root, create the identity, and construct
both root-based loader and recorder.
9. Replace diagnostics writes in the command with conditional summary writes.
Populate invocation metadata, redacted effective configuration, resolved
pipeline, resolved references, checkpoint events, chunk-plan decisions, run
manifest, warnings, run report, and error text as each becomes available.
10. Replace retention-aware failure handling with one failure path that prints
the primary error, attempts a single summary error record when a bundle
exists, reports any secondary debug error, and includes the allocated bundle
path. Never delete a requested bundle.
11. Report output and debug paths as fixed above. Keep warning and exit-code
behavior otherwise unchanged.
12. Convert both maintained example configs and package testdata configs to
version 3 so every executable example uses the supported schema.
### Tests
Update or add CLI and config tests covering:
- exact v3 defaults, file fields, strict unknown fields, JSON/redacted shape,
and validation;
- precedence for output, chunk mode/root, checkpoint root, and debug root;
- malformed lower-precedence file or environment values despite valid CLI
overrides;
- actionable version 2 rejection before legacy unknown fields are decoded;
- rejection of every removed environment variable and configuration surface by
absence of effect, with obsolete file fields rejected as unknown;
- `--debug-dir` without `--debug`, empty explicit directory values, and removed
`--diagnostics-dir`;
- no debug allocation for ordinary runs and allocation before resolution
failures for debug runs;
- no checkpoint root resolution, loading, or recording without `--resume`;
- first-use resume writes checkpoints and later resume reuses them;
- `bypass` does not resolve a chunk-plan root;
- output directory selection and unchanged durable logical files;
- success and failure messages with and without a debug path; and
- requested summary or trace write failures producing exit code `1` without
masking a primary error.
Run at minimum:
```sh
go test ./internal/core/config
go test ./internal/cli
go test ./internal/modules/integration ./internal/modules/seriatim/input/transcript
```
### Exit criteria
Only version 3 configuration is accepted; ordinary runs expose output and
chunk-plan cache only; resume and debug are invocation-controlled; all examples
and CLI tests use the new public contract; and the repository-wide checks pass.
## Stage 4: Remove legacy workspace and diagnostics implementation
**Status:** Not started
### Objective
Complete the internal ownership refactor and prove that no obsolete public state
model remains hidden behind compatibility wrappers.
### Read first
- the complete import graph reported by `rg 'core/(workspace|diagnostics)'`
- `internal/core/workspace/`
- `internal/core/diagnostics/`
- `internal/framework/checkpoint/`
- `internal/core/debugbundle/`
- `internal/framework/debug/`
- `internal/cli/run.go`
### Implement
1. Delete the remaining workspace settings, path, writer, and tests after moving
any still-valid generic behavior to `fileio`, `checkpoint`, or
`debugbundle`.
2. Delete the old diagnostics package, retention modes, run directory, and
tests after confirming all desired redacted artifacts are owned and tested
by `debugbundle`.
3. Remove compatibility constructors, aliases, fields, environment names, and
dead CLI helpers introduced solely by the old state model. Do not retain
deprecated parsing.
4. Rename internal variables, metadata fields, recorder types, and errors that
still use workspace or diagnostics to describe the removed surfaces.
5. Retain and document the frozen checkpoint wire schema identifiers. Retain
unrelated domain “diagnostic” terminology.
6. Confirm generic framework and pipeline packages receive only collaborator
interfaces and never physical roots.
### Tests
- Run `rg` checks proving there are no remaining imports of deleted packages,
public configuration fields, removed environment names, removed flags, or
operator-facing workspace/diagnostics messages.
- Allow matches only in ADR/history, version 2 migration guidance, frozen
checkpoint wire compatibility, and unrelated domain diagnostic contracts.
- Run all checkpoint, debug, CLI, and integration tests after deletion.
### Exit criteria
The old packages and compatibility surfaces are gone; remaining historical
terms are intentional and documented; package ownership matches the Planned
Internal Boundaries; and all checks pass.
## Stage 5: Harden cross-surface security and compatibility
**Status:** Not started
### Objective
Exercise the completed feature across failures, permissions, reuse, redaction,
and surface independence before changing current-behavior documentation.
### Read first
- all focused tests changed in Stages 1 through 4
- `internal/cli/compatibility_test.go`
- `internal/framework/pipeline/runner_*debug*_test.go`
- `internal/framework/checkpoint/*_test.go`
- `internal/framework/chunkplan/store_test.go`
- the Security and Lifecycle and Compatibility Policy sections of the feature
roadmap
### Implement
1. Add an end-to-end state-surface matrix covering debug on/off, resume on/off,
and chunk modes `auto`, `bypass`, and `refresh`. Assert exactly which roots
are resolved and which files are created.
2. Add integration coverage proving chunk plans and checkpoints use independent
configured roots and that debug never affects cache selection or reuse.
3. Reuse a representative pre-refactor checkpoint fixture from an explicitly
selected old checkpoint root. Assert byte-compatible loading and normal
replacement behavior; do not create a migration path or rewrite untouched
entries.
4. Cover early failures before debug allocation, failures after allocation but
before pipeline execution, pipeline failures, output-write failures, summary
failures, and trace failures. Assert stable exit codes, stderr content, and
retained bundle state.
5. Inspect all summary artifacts for forbidden raw content and secrets. Inspect
trace artifacts to prove expected application data is present while API keys,
credential-shaped values, unrelated environment values, and unrelated file
contents are absent.
6. Assert created bundle and checkpoint modes on supported Unix systems and
preserve portable behavior on other supported platforms.
7. Confirm deletion is never automatic for output or debug, while manually
removing an exact cache entry only causes recomputation.
8. Confirm durable logical output and chunk-plan envelope bytes have not changed
solely because of ADR-0006.
### Tests and validation
Run focused packages while iterating, then run:
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
git diff --check
```
Run race-enabled tests for the concurrent debug and checkpoint collaborators if
supported by the environment:
```sh
go test -race ./internal/framework/debug ./internal/framework/checkpoint ./internal/cli
```
### Exit criteria
All three surfaces are independent under success and failure; compatibility and
permissions are proven; debug captures only intended data; durable contracts
are unchanged; and all repository-wide checks pass.
## Stage 6: Update canonical documentation and close the roadmap
**Status:** Not started
### Objective
Make current-behavior documentation and maintained examples describe the
implemented version 3 state model, then remove completed planning documents.
### Read first
- [Documentation Policy](../policy/documentation.md)
- [Architecture Policy](../policy/architecture.md)
- [CLI Reference](../cli.md)
- [Configuration](../config.md)
- [Operations](../operations.md)
- [Internal Overview](../internal/overview.md)
- [Diagnostics Internals](../internal/diagnostics.md)
- [Development](../development.md)
- `README.md`, `examples/`, and the final code and tests
### Implement
1. Update `docs/config.md` for version 3 fields, defaults, validation,
precedence, and the five supported state environment variables. Add one
complete version 2-to-3 before/after migration example and list every removed
setting and environment variable from the feature roadmap.
2. Update `docs/cli.md` for `--debug`, `--debug-dir`, target `--resume`
semantics, removed `--diagnostics-dir`, reporting, and failure behavior.
3. Rewrite the relevant `docs/operations.md` sections around output, chunk-plan
cache, checkpoint cache, and debug bundles. Document exact layouts,
permissions, inherited sensitivity, aggregation/copying/retention risk,
cleanup, compatible old-root reuse, per-user defaults, and the two
`/var/cache/notarius/...` Linux service recommendations.
4. Update `docs/policy/architecture.md` to describe the implemented three
surfaces and remove the obsolete five-surface model. Keep normative safety
rules concise and link operational details to Operations.
5. Replace `docs/internal/diagnostics.md` with a correctly named internal state
document, or distribute its content among focused internal documents if that
better matches the final packages. Update `docs/internal/overview.md`,
`docs/internal/pipeline.md`, `docs/development.md`, and all inbound links.
6. Update README orientation and examples only where they currently mention the
old schema or invocation. Do not duplicate the complete CLI, configuration,
or operations contracts.
7. Search the repository for stale flags, fields, environment names, paths,
package names, and workspace/diagnostics terminology. Preserve only ADR
history, explicit version 2 migration text, frozen checkpoint wire fields,
and unrelated domain diagnostics.
8. After implementation and documentation validation are complete, remove
`docs/roadmap/adr0006.md` and this implementation plan. They are completed
planning artifacts; the accepted ADR and canonical current-behavior
documents become authoritative.
### Tests and validation
- Validate every command, field, default, environment variable, path, and
example against code and focused tests.
- Validate all changed links and the task-specific reading map.
- Run any repository documentation or link checker if present.
- Run the complete repository-wide validation in the Execution Rules.
### Exit criteria
Canonical documentation describes only implemented behavior, migration and
operations each have one owner, all maintained examples are valid version 3,
no stale public state terminology remains, completed roadmap documents are
removed, and all checks pass.

21
examples/dnd-spells-production.config.yml
View File

@@ -1,19 +1,18 @@
version: 2
version: 3
concurrency:
total_llm: 1
stage_workers:
extract: 1
workspace:
directory: /var/lib/notarius
diagnostics:
enabled: true
retention: auto
resume:
enabled: false
debug:
enabled: false
chunk_cache:
output:
directory: ./notarius-output
cache:
chunk_plans:
directory: /var/cache/notarius/chunk-plans
mode: auto
checkpoints:
directory: /var/cache/notarius/checkpoints
debug:
directory: ./notarius-debug
pipelines:
dnd-session:
input: seriatim

12
examples/dnd-spells.config.yml
View File

@@ -1,7 +1,13 @@
version: 2
workspace:
chunk_cache:
version: 3
output:
directory: ./notarius-output
cache:
chunk_plans:
mode: bypass
checkpoints:
directory: ""
debug:
directory: ./notarius-debug
pipelines:
dnd-session:
input: seriatim

496
internal/cli/chunk_cache_test.go
View File

@@ -1,496 +0,0 @@
package cli
import (
"bytes"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
type recordingChunkPlanStore struct {
record pipeline.ChunkPlanRecord
decision pipeline.ChunkPlanDecision
loadErr error
saveErr error
loads int
saves int
}
func (s *recordingChunkPlanStore) Load(string) (pipeline.ChunkPlanRecord, pipeline.ChunkPlanDecision, error) {
s.loads++
return s.record, s.decision, s.loadErr
}
func (s *recordingChunkPlanStore) Save(record pipeline.ChunkPlanRecord) error {
s.saves++
s.record = record
return s.saveErr
}
type recordingChunkPlanFactory struct {
roots []string
store *recordingChunkPlanStore
err error
}
func (f *recordingChunkPlanFactory) build(root string) (pipeline.ChunkPlanStore, error) {
f.roots = append(f.roots, root)
if f.err != nil {
return nil, f.err
}
if f.store == nil {
f.store = &recordingChunkPlanStore{decision: pipeline.ChunkPlanDecision{Status: pipeline.ChunkPlanMissing}}
}
return f.store, nil
}
func TestRunChunkCachePrecedence(t *testing.T) {
tests := []struct {
name string
fileMode string
envMode string
flagMode string
wantMode pipeline.ChunkCacheMode
wantBuild bool
}{
{name: "file refresh", fileMode: "refresh", wantMode: pipeline.ChunkCacheRefresh, wantBuild: true},
{name: "environment over file", fileMode: "bypass", envMode: "refresh", wantMode: pipeline.ChunkCacheRefresh, wantBuild: true},
{name: "flag over environment", fileMode: "refresh", envMode: "auto", flagMode: "bypass", wantMode: pipeline.ChunkCacheBypass},
{name: "explicit auto", fileMode: "bypass", envMode: "refresh", flagMode: "auto", wantMode: pipeline.ChunkCacheAuto, wantBuild: true},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
root := filepath.Join(t.TempDir(), "plans")
configPath := writeTestConfig(t, cacheTestConfig(tc.fileMode, root, ""))
inputPath := writeFile(t, "input.txt", "input")
values := map[string]string{}
if tc.envMode != "" {
values["NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE"] = tc.envMode
}
factory := &recordingChunkPlanFactory{}
args := []string{"run", "example", "--config", configPath, "--input", inputPath, "--output-dir", t.TempDir()}
if tc.flagMode != "" {
args = append(args, "--chunk_cache", tc.flagMode)
}
code, stderr := runCacheCommand(t, args, cacheTestOptions(t, values, factory))
if code != 0 {
t.Fatalf("code = %d stderr = %q", code, stderr)
}
if got := len(factory.roots) > 0; got != tc.wantBuild {
t.Fatalf("store built = %t roots = %#v, want %t", got, factory.roots, tc.wantBuild)
}
if !tc.wantBuild {
return
}
if tc.wantMode == pipeline.ChunkCacheAuto && (factory.store.loads != 1 || factory.store.saves != 1) {
t.Fatalf("auto calls = load %d save %d", factory.store.loads, factory.store.saves)
}
if tc.wantMode == pipeline.ChunkCacheRefresh && (factory.store.loads != 0 || factory.store.saves != 1) {
t.Fatalf("refresh calls = load %d save %d", factory.store.loads, factory.store.saves)
}
})
}
}
func TestRunChunkCacheInvalidValuesHaveEstablishedExitCodes(t *testing.T) {
validConfig := writeTestConfig(t, cacheTestConfig("bypass", "", ""))
invalidFile := writeTestConfig(t, cacheTestConfig("sometimes", "", ""))
inputPath := writeFile(t, "input.txt", "input")
tests := []struct {
name string
config string
env map[string]string
flag string
want int
}{
{name: "flag", config: validConfig, flag: "sometimes", want: 2},
{name: "environment", config: validConfig, env: map[string]string{"NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE": "sometimes"}, want: 1},
{name: "file", config: invalidFile, want: 1},
{name: "flag does not mask invalid file", config: invalidFile, flag: "bypass", want: 1},
{name: "flag does not mask invalid environment", config: validConfig, env: map[string]string{"NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE": "sometimes"}, flag: "bypass", want: 1},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
args := []string{"run", "example", "--config", tc.config, "--input", inputPath, "--output-dir", t.TempDir()}
if tc.flag != "" {
args = append(args, "--chunk_cache", tc.flag)
}
code, _ := runCacheCommand(t, args, cacheTestOptions(t, tc.env, &recordingChunkPlanFactory{}))
if code != tc.want {
t.Fatalf("code = %d, want %d", code, tc.want)
}
})
}
}
func TestRunChunkPlanRootResolution(t *testing.T) {
t.Run("explicit file root", func(t *testing.T) {
factory := &recordingChunkPlanFactory{}
resolverCalls := 0
opts := cacheTestOptions(t, nil, factory)
opts.UserCacheDir = func() (string, error) { resolverCalls++; return "", errors.New("must not be called") }
configPath := writeTestConfig(t, cacheTestConfig("auto", "/var/cache/notarius/chunk-plans", ""))
code, stderr := runCacheCommand(t, cacheRunArgs(t, configPath), opts)
if code != 0 || stderr != "" || resolverCalls != 0 || len(factory.roots) != 1 || factory.roots[0] != "/var/cache/notarius/chunk-plans" {
t.Fatalf("code=%d stderr=%q resolver=%d roots=%#v", code, stderr, resolverCalls, factory.roots)
}
})
t.Run("environment root", func(t *testing.T) {
factory := &recordingChunkPlanFactory{}
environmentRoot := filepath.Join(t.TempDir(), "environment-plans")
configPath := writeTestConfig(t, cacheTestConfig("auto", filepath.Join(t.TempDir(), "file-plans"), ""))
code, stderr := runCacheCommand(t, cacheRunArgs(t, configPath), cacheTestOptions(t, map[string]string{"NOTARIUS_WORKSPACE_CHUNK_CACHE_DIR": environmentRoot}, factory))
if code != 0 || stderr != "" || len(factory.roots) != 1 || factory.roots[0] != environmentRoot {
t.Fatalf("code=%d stderr=%q roots=%#v", code, stderr, factory.roots)
}
})
t.Run("per-user default", func(t *testing.T) {
factory := &recordingChunkPlanFactory{}
cacheDir := filepath.Join(t.TempDir(), "user-cache")
opts := cacheTestOptions(t, nil, factory)
opts.UserCacheDir = func() (string, error) { return cacheDir, nil }
configPath := writeTestConfig(t, cacheTestConfig("auto", "", ""))
code, stderr := runCacheCommand(t, cacheRunArgs(t, configPath), opts)
want := filepath.Join(cacheDir, "notarius", "chunk-plans")
if code != 0 || stderr != "" || len(factory.roots) != 1 || factory.roots[0] != want {
t.Fatalf("code=%d stderr=%q roots=%#v want=%q", code, stderr, factory.roots, want)
}
})
}
func TestRunBypassSkipsRootAndStore(t *testing.T) {
factory := &recordingChunkPlanFactory{err: errors.New("must not build")}
resolverCalls := 0
opts := cacheTestOptions(t, nil, factory)
opts.UserCacheDir = func() (string, error) { resolverCalls++; return "", errors.New("must not resolve") }
configPath := writeTestConfig(t, cacheTestConfig("bypass", "", ""))
code, stderr := runCacheCommand(t, cacheRunArgs(t, configPath), opts)
if code != 0 || stderr != "" || resolverCalls != 0 || len(factory.roots) != 0 {
t.Fatalf("code=%d stderr=%q resolver=%d roots=%#v", code, stderr, resolverCalls, factory.roots)
}
}
func TestRunChunkPlanSetupFailures(t *testing.T) {
for _, mode := range []string{"auto", "refresh"} {
t.Run(mode+" resolver", func(t *testing.T) {
factory := &recordingChunkPlanFactory{}
opts := cacheTestOptions(t, nil, factory)
opts.UserCacheDir = func() (string, error) { return "", errors.New("cache unavailable") }
configPath := writeTestConfig(t, cacheTestConfig(mode, "", ""))
code, stderr := runCacheCommand(t, cacheRunArgs(t, configPath), opts)
if code != 1 || !strings.Contains(stderr, "cache unavailable") || len(factory.roots) != 0 {
t.Fatalf("code=%d stderr=%q roots=%#v", code, stderr, factory.roots)
}
})
}
t.Run("store", func(t *testing.T) {
factory := &recordingChunkPlanFactory{err: errors.New("unwritable store")}
configPath := writeTestConfig(t, cacheTestConfig("auto", t.TempDir(), ""))
code, stderr := runCacheCommand(t, cacheRunArgs(t, configPath), cacheTestOptions(t, nil, factory))
if code != 1 || !strings.Contains(stderr, "unwritable store") {
t.Fatalf("code=%d stderr=%q", code, stderr)
}
})
t.Run("unwritable filesystem store", func(t *testing.T) {
root := writeFile(t, "not-a-directory", "occupied")
configPath := writeTestConfig(t, `version: 2
workspace:
chunk_cache:
mode: auto
directory: `+root+`
pipelines:
dnd-session:
input: seriatim
artifacts:
spells:
extract: dnd/spells
`)
code, stderr := runCacheCommand(t, []string{"run", "dnd-session", "--config", configPath, "--input", writeSeriatimInput(t), "--output-dir", t.TempDir()}, Options{LLMClientFactory: fakeLLMFactory(newFakeRunLLMClient(false), nil), LookupEnv: mapLookup(nil)})
if code != 1 || (!strings.Contains(stderr, "load chunk plan") && !strings.Contains(stderr, "save chunk plan")) {
t.Fatalf("code=%d stderr=%q", code, stderr)
}
})
}
func TestConfigCommandsDoNotResolveOrCreateChunkPlanState(t *testing.T) {
configPath := writeTestConfig(t, cacheTestConfig("auto", "", ""))
for _, args := range [][]string{
{"config", "validate", "--config", configPath},
{"pipelines", "list", "--config", configPath},
} {
factory := &recordingChunkPlanFactory{err: errors.New("must not build")}
resolverCalls := 0
opts := cacheTestOptions(t, nil, factory)
opts.UserCacheDir = func() (string, error) { resolverCalls++; return "", errors.New("must not resolve") }
code, stderr := runCacheCommand(t, args, opts)
if code != 0 || stderr != "" || resolverCalls != 0 || len(factory.roots) != 0 {
t.Fatalf("args=%v code=%d stderr=%q resolver=%d roots=%#v", args, code, stderr, resolverCalls, factory.roots)
}
}
}
func TestRunRecordsExplicitChunkCacheOverrideAndEffectiveMode(t *testing.T) {
diagnosticsDir := t.TempDir()
configPath := writeTestConfig(t, cacheTestConfig("bypass", "", diagnosticsDir))
factory := &recordingChunkPlanFactory{}
args := append(cacheRunArgs(t, configPath), "--chunk_cache", "refresh")
code, stderr := runCacheCommand(t, args, cacheTestOptions(t, nil, factory))
if code != 0 || stderr != "" {
t.Fatalf("code=%d stderr=%q", code, stderr)
}
runDir := onlyChildDir(t, diagnosticsDir)
invocation := string(readFile(t, filepath.Join(runDir, diagnostics.ArtifactInvocationMetadata)))
effective := string(readFile(t, filepath.Join(runDir, diagnostics.ArtifactEffectiveConfig)))
if !strings.Contains(invocation, `"chunk_cache_override": "refresh"`) || !strings.Contains(effective, `"mode": "refresh"`) {
t.Fatalf("invocation=%s effective=%s", invocation, effective)
}
}
func TestRunChunkPlanDiagnosticsRedactStoreDecisionReason(t *testing.T) {
diagnosticsDir := t.TempDir()
configPath := writeTestConfig(t, cacheTestConfig("auto", t.TempDir(), diagnosticsDir))
factory := &recordingChunkPlanFactory{store: &recordingChunkPlanStore{
decision: pipeline.ChunkPlanDecision{Status: pipeline.ChunkPlanInvalid, Reason: "SENTINEL_INVALID_RECORD_CONTENT"},
}}
code, stderr := runCacheCommand(t, cacheRunArgs(t, configPath), cacheTestOptions(t, nil, factory))
if code != 0 || stderr != "" {
t.Fatalf("code=%d stderr=%q", code, stderr)
}
summary := string(readFile(t, filepath.Join(onlyChildDir(t, diagnosticsDir), diagnostics.ArtifactChunkPlan)))
if strings.Contains(summary, "SENTINEL_INVALID_RECORD_CONTENT") || !strings.Contains(summary, `"lookup_reason": "stored chunk plan is invalid"`) {
t.Fatalf("chunk plan diagnostic = %s", summary)
}
}
func TestDefaultAutoReusesPlanAcrossIndependentInvocations(t *testing.T) {
cacheBase := filepath.Join(t.TempDir(), "cache")
workspaceDir := filepath.Join(t.TempDir(), "workspace")
configPath := writeTestConfig(t, `version: 2
workspace:
directory: `+workspaceDir+`
debug:
enabled: true
pipelines:
dnd-session:
input: seriatim
chunk: dnd/scenes
artifacts:
spells:
extract: dnd/spells
`)
inputPath := writeFile(t, "source.json", `{
"metadata": {"id": "session-alpha"},
"segments": [
{"id": 1, "start": 0, "end": 1, "speaker": "Aria", "text": "Aria casts Cure Wounds."},
{"id": 2, "start": 1, "end": 2, "speaker": "Borin", "text": "Borin recovers."}
]
}`)
client := newFakeRunLLMClient(false)
opts := Options{
LLMClientFactory: fakeLLMFactory(client, nil),
LookupEnv: mapLookup(nil),
UserCacheDir: func() (string, error) { return cacheBase, nil },
}
for i := 0; i < 2; i++ {
code, stderr := runCacheCommand(t, []string{"run", "dnd-session", "--config", configPath, "--input", inputPath, "--output-dir", t.TempDir()}, opts)
if code != 0 {
t.Fatalf("run %d code=%d stderr=%q", i+1, code, stderr)
}
}
if client.calls != 3 {
t.Fatalf("LLM calls = %d, want chunk+extract then extract-only reuse", client.calls)
}
planRoot := filepath.Join(cacheBase, "notarius", "chunk-plans")
entries, err := os.ReadDir(planRoot)
if err != nil || len(entries) != 1 {
t.Fatalf("plan root entries = %v error=%v", entries, err)
}
if _, err := os.Stat(filepath.Join(planRoot, entries[0].Name(), "plan.json")); err != nil {
t.Fatalf("plan file: %v", err)
}
debugRuns := childDirs(t, filepath.Join(workspaceDir, "debug"))
if len(debugRuns) != 2 {
t.Fatalf("debug runs = %#v", debugRuns)
}
attemptCounts := 0
for _, runDir := range debugRuns {
if _, err := os.Stat(filepath.Join(runDir, "chunk", "attempt-01.json")); err == nil {
attemptCounts++
} else if !os.IsNotExist(err) {
t.Fatal(err)
}
}
if attemptCounts != 1 {
t.Fatalf("chunk attempt files across runs = %d, want only generating run", attemptCounts)
}
}
func TestChunkPlanReuseDependsOnlyOnSourceDigest(t *testing.T) {
cacheRoot := filepath.Join(t.TempDir(), "plans")
inputPath := writeSeriatimInput(t)
referencePath := writeFile(t, "players.txt", "Alyx")
profilePath := writeScriptoriumProfileFile(t, "chunk-profile", "http://127.0.0.1:8080/v1", "test-model")
seedConfig := writeTestConfig(t, `version: 2
workspace:
chunk_cache:
mode: auto
directory: `+cacheRoot+`
pipelines:
seed:
input: seriatim
chunk:
module: generic
options:
max_units: 1
overlap_units: 0
artifacts:
spells:
extract: dnd/spells
`)
changedConfig := writeTestConfig(t, `version: 2
scriptorium:
profile_file: `+profilePath+`
workspace:
chunk_cache:
mode: auto
directory: `+cacheRoot+`
pipelines:
changed:
input: seriatim
chunk:
module: dnd/scenes
llm_profile: chunk-profile
references:
players: `+referencePath+`
validators:
- generic/always_accept
artifacts:
spells:
extract: dnd/spells
`)
client := newFakeRunLLMClient(false)
opts := Options{LLMClientFactory: fakeLLMFactory(client, nil), LookupEnv: mapLookup(nil)}
for _, run := range []struct {
pipeline string
config string
}{{pipeline: "seed", config: seedConfig}, {pipeline: "changed", config: changedConfig}} {
code, stderr := runCacheCommand(t, []string{"run", run.pipeline, "--config", run.config, "--input", inputPath, "--output-dir", t.TempDir()}, opts)
if code != 0 {
t.Fatalf("pipeline %q code=%d stderr=%q", run.pipeline, code, stderr)
}
}
if client.calls != 2 {
t.Fatalf("LLM calls = %d, want one extractor call per run and no changed chunker call", client.calls)
}
}
func TestResumeAndChunkCacheModesRemainIndependent(t *testing.T) {
for _, mode := range []pipeline.ChunkCacheMode{pipeline.ChunkCacheAuto, pipeline.ChunkCacheBypass, pipeline.ChunkCacheRefresh} {
t.Run(string(mode), func(t *testing.T) {
workspaceDir := filepath.Join(t.TempDir(), "workspace")
cacheRoot := filepath.Join(t.TempDir(), "plans")
configPath := writeTestConfig(t, cacheTestConfigWithWorkspace(string(mode), cacheRoot, workspaceDir))
factory := &recordingChunkPlanFactory{}
args := append(cacheRunArgs(t, configPath), "--resume")
code, stderr := runCacheCommand(t, args, cacheTestOptions(t, nil, factory))
if code != 0 || stderr != "" {
t.Fatalf("code=%d stderr=%q", code, stderr)
}
if mode == pipeline.ChunkCacheBypass {
if len(factory.roots) != 0 {
t.Fatalf("bypass roots = %#v", factory.roots)
}
} else if len(factory.roots) != 1 || factory.roots[0] != cacheRoot {
t.Fatalf("roots = %#v, want %q", factory.roots, cacheRoot)
}
if entries := childDirs(t, filepath.Join(workspaceDir, "checkpoints")); len(entries) != 1 {
t.Fatalf("checkpoint roots = %#v", entries)
}
if strings.HasPrefix(cacheRoot, workspaceDir+string(filepath.Separator)) || strings.HasPrefix(workspaceDir, cacheRoot+string(filepath.Separator)) {
t.Fatalf("cache root %q and workspace root %q overlap", cacheRoot, workspaceDir)
}
})
}
}
func TestWorkspaceDirectoryDoesNotSelectChunkPlanRoot(t *testing.T) {
cacheBase := filepath.Join(t.TempDir(), "user-cache")
factory := &recordingChunkPlanFactory{}
for _, workspaceDir := range []string{filepath.Join(t.TempDir(), "workspace-one"), filepath.Join(t.TempDir(), "workspace-two")} {
configPath := writeTestConfig(t, cacheTestConfigWithWorkspace("auto", "", workspaceDir))
opts := cacheTestOptions(t, nil, factory)
opts.UserCacheDir = func() (string, error) { return cacheBase, nil }
code, stderr := runCacheCommand(t, cacheRunArgs(t, configPath), opts)
if code != 0 || stderr != "" {
t.Fatalf("workspace=%q code=%d stderr=%q", workspaceDir, code, stderr)
}
}
want := filepath.Join(cacheBase, "notarius", "chunk-plans")
if len(factory.roots) != 2 || factory.roots[0] != want || factory.roots[1] != want {
t.Fatalf("roots = %#v, want %q twice", factory.roots, want)
}
}
func cacheTestOptions(t *testing.T, env map[string]string, factory *recordingChunkPlanFactory) Options {
t.Helper()
registries := fakeExecutionRegistries(t)
return Options{
Catalog: catalogFromRegistries(registries),
Registries: registries,
LLMClientFactory: fakeLLMFactory(nil, nil),
LookupEnv: mapLookup(env),
UserCacheDir: func() (string, error) { return filepath.Join(t.TempDir(), "cache"), nil },
ChunkPlanStoreFactory: factory.build,
}
}
func cacheRunArgs(t *testing.T, configPath string) []string {
t.Helper()
return []string{"run", "example", "--config", configPath, "--input", writeFile(t, "input.txt", "input"), "--output-dir", t.TempDir()}
}
func runCacheCommand(t *testing.T, args []string, opts Options) (int, string) {
t.Helper()
var stdout bytes.Buffer
var stderr bytes.Buffer
code := RunWithOptions(args, &stdout, &stderr, opts)
return code, stderr.String()
}
func cacheTestConfig(mode, directory, diagnosticsDir string) string {
var b strings.Builder
b.WriteString("version: 2\n")
if mode != "" || directory != "" {
b.WriteString("workspace:\n chunk_cache:\n")
if mode != "" {
b.WriteString(" mode: " + mode + "\n")
}
if directory != "" {
b.WriteString(" directory: " + directory + "\n")
}
}
if diagnosticsDir != "" {
b.WriteString("diagnostics:\n work_dir: " + diagnosticsDir + "\n retention: always\n")
}
b.WriteString("pipelines:\n example:\n input: fake/input\n artifacts:\n spells:\n extract: fake/extract\n")
return b.String()
}
func cacheTestConfigWithWorkspace(mode, cacheRoot, workspaceDir string) string {
var b strings.Builder
b.WriteString("version: 2\nworkspace:\n directory: " + workspaceDir + "\n resume:\n enabled: true\n chunk_cache:\n mode: " + mode + "\n")
if cacheRoot != "" {
b.WriteString(" directory: " + cacheRoot + "\n")
}
b.WriteString("pipelines:\n example:\n input: fake/input\n artifacts:\n spells:\n extract: fake/extract\n")
return b.String()
}

600
internal/cli/compatibility_test.go
View File

@@ -1,600 +0,0 @@
package cli
import (
"bytes"
"context"
"encoding/json"
"io/fs"
"path/filepath"
"reflect"
"sort"
"strings"
"sync"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
frameworkllm "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
spellshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/shape"
spellsourcerefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/source_refs"
spellrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/source_relatedness"
validjson "gitea.maximumdirect.net/eric/notarius/internal/modules/generic/validate/valid_json"
validjsonschema "gitea.maximumdirect.net/eric/notarius/internal/modules/generic/validate/valid_json_schema"
)
func TestProductionCompatibilitySnapshot(t *testing.T) {
normalizedOptions, err := normalizeOptions(Options{})
if err != nil {
t.Fatalf("normalizeOptions() error = %v, want nil", err)
}
if normalizedOptions.Catalog.Inputs != normalizedOptions.Registries.Inputs ||
normalizedOptions.Catalog.Chunkers != normalizedOptions.Registries.Chunkers ||
normalizedOptions.Catalog.Extractors != normalizedOptions.Registries.Extractors ||
normalizedOptions.Catalog.Mergers != normalizedOptions.Registries.Mergers ||
normalizedOptions.Catalog.Normalizers != normalizedOptions.Registries.Normalizers ||
normalizedOptions.Catalog.Validators != normalizedOptions.Registries.Validators ||
normalizedOptions.Catalog.ValidatorChains != normalizedOptions.Registries.ValidatorChains ||
normalizedOptions.Catalog.Outputs != normalizedOptions.Registries.Outputs {
t.Fatal("production catalog and execution registries do not share one composition")
}
if normalizedOptions.LLMClientFactory == nil {
t.Fatal("production LLM client factory is nil")
}
registries, err := productionRegistries()
if err != nil {
t.Fatalf("productionRegistries() error = %v, want nil", err)
}
keySnapshots := []struct {
name string
got []string
want []string
}{
{name: "inputs", got: registries.Inputs.RegisteredKeys(), want: []string{"seriatim"}},
{name: "chunkers", got: registries.Chunkers.RegisteredKeys(), want: []string{"dnd/scenes", "generic"}},
{name: "extractors", got: registries.Extractors.RegisteredKeys(), want: []string{"dnd/spells"}},
{name: "mergers", got: registries.Mergers.RegisteredKeys(), want: []string{"appendorder"}},
{name: "normalizers", got: registries.Normalizers.RegisteredKeys(), want: []string{"noop"}},
{name: "validators", got: registries.Validators.RegisteredKeys(), want: []string{
"extract/dnd/spells/shape", "extract/dnd/spells/source_refs", "extract/dnd/spells/source_relatedness",
"generic/always_accept", "generic/always_reject", "generic/valid_json", "generic/valid_json_schema",
}},
{name: "outputs", got: registries.Outputs.RegisteredKeys(), want: []string{"json"}},
}
for _, snapshot := range keySnapshots {
t.Run(snapshot.name, func(t *testing.T) {
if !reflect.DeepEqual(snapshot.got, snapshot.want) {
t.Fatalf("registered keys = %#v, want compatibility snapshot %#v", snapshot.got, snapshot.want)
}
})
}
wantChain := []pipeline.ModuleBinding{
pipeline.Binding(validjson.Key),
pipeline.Binding(validjsonschema.Key),
pipeline.Binding(spellshape.Key),
pipeline.Binding(spellsourcerefs.Key),
pipeline.Binding(spellrelatedness.Key),
}
if got := registries.ValidatorChains.Validators(pipeline.StageExtract, spells.Key); !reflect.DeepEqual(got, wantChain) {
t.Fatalf("spell validator chain = %#v, want compatibility snapshot %#v", got, wantChain)
}
assets, err := productionPromptAssets()
if err != nil {
t.Fatalf("productionPromptAssets() error = %v, want nil", err)
}
assertAssetNames(t, assets.PromptFS, []string{
"dnd.scenes/dnd.scenes.yaml",
"dnd.scenes/instructions.md",
"dnd.scenes/sharedassets/common-dnd-references.md",
"dnd.scenes/sharedassets/common-dnd-system.md",
"dnd.scenes/sharedassets/common-dnd-transcript.md",
"dnd.scenes/task.md",
"dnd.spells/dnd.spells.yaml",
"dnd.spells/instructions.md",
"dnd.spells/sharedassets/common-dnd-references.md",
"dnd.spells/sharedassets/common-dnd-system.md",
"dnd.spells/sharedassets/common-dnd-transcript.md",
"dnd.spells/task.md",
})
assertAssetNames(t, assets.SchemaFS, []string{
"dnd_scenes.v1.json",
"dnd_spells_llm.v1.json",
})
identitySnapshot := map[string]map[string]any{
"scenes": sceneManifestMetadata(t),
"spells": spellManifestMetadata(t),
}
for name, metadata := range identitySnapshot {
for _, key := range []string{"prompt_id", "prompt_version", "prompt_sha256", "response_schema_key", "response_schema_id", "response_schema_name", "response_schema_version", "response_schema_sha256"} {
if value, ok := metadata[key].(string); !ok || value == "" {
t.Fatalf("%s metadata[%q] = %#v, want non-empty identity", name, key, metadata[key])
}
}
}
if got := []any{
identitySnapshot["scenes"]["prompt_id"], identitySnapshot["scenes"]["prompt_version"],
identitySnapshot["scenes"]["response_schema_key"], identitySnapshot["scenes"]["response_schema_id"], identitySnapshot["scenes"]["response_schema_name"], identitySnapshot["scenes"]["response_schema_version"],
}; !reflect.DeepEqual(got, []any{"dnd.scenes", "v1", "dnd_scenes", "notarius.dnd.scenes", "notarius_dnd_scenes_v1", "v1"}) {
t.Fatalf("scene identities = %#v, want compatibility snapshot", got)
}
if got := []any{
identitySnapshot["spells"]["prompt_id"], identitySnapshot["spells"]["prompt_version"],
identitySnapshot["spells"]["response_schema_key"], identitySnapshot["spells"]["response_schema_id"], identitySnapshot["spells"]["response_schema_name"], identitySnapshot["spells"]["response_schema_version"],
}; !reflect.DeepEqual(got, []any{"dnd.spells", "v1", "dnd_spells", "notarius.dnd.spells", "notarius_dnd_spells_v1", "v1"}) {
t.Fatalf("spell identities = %#v, want compatibility snapshot", got)
}
fileConfig, err := config.LoadFileConfig(fixturePath(t, "examples/dnd-spells.config.yml"))
if err != nil {
t.Fatalf("LoadFileConfig() error = %v, want nil", err)
}
cfg := config.Default()
if err := cfg.ApplyFileConfig(fileConfig); err != nil {
t.Fatalf("ApplyFileConfig() error = %v, want nil", err)
}
effective, err := cfg.Resolve(config.ResolveInput{PipelineID: "dnd-session", Catalog: catalogFromRegistries(registries)})
if err != nil {
t.Fatalf("Resolve() error = %v, want nil", err)
}
resolved := effective.ResolvedPipeline
if resolved.Input.Module != "seriatim" || resolved.Chunk.Module != "generic" || resolved.Output.Module != "json" || len(resolved.ArtifactLanes) != 1 {
t.Fatalf("resolved example = %#v, want maintained production topology", resolved)
}
lane := resolved.ArtifactLanes[0]
if lane.ID != "spells" || lane.Extract.Module != "dnd/spells" || lane.Merge.Module != "appendorder" || lane.Normalize.Module != "noop" {
t.Fatalf("resolved lane = %#v, want maintained spell lane", lane)
}
if got := resolvedValidatorKeys(resolved.ValidatorChains, pipeline.StageExtract, "spells", spells.Key); !reflect.DeepEqual(got, []string{
"generic/valid_json", "generic/valid_json_schema", "extract/dnd/spells/shape", "extract/dnd/spells/source_refs", "extract/dnd/spells/source_relatedness",
}) {
t.Fatalf("resolved validator keys = %#v, want compatibility snapshot", got)
}
productionFileConfig, err := config.LoadFileConfig(fixturePath(t, "examples/dnd-spells-production.config.yml"))
if err != nil {
t.Fatalf("LoadFileConfig(production example) error = %v, want nil", err)
}
productionConfig := config.Default()
if err := productionConfig.ApplyFileConfig(productionFileConfig); err != nil {
t.Fatalf("ApplyFileConfig(production example) error = %v, want nil", err)
}
productionEffective, err := productionConfig.Resolve(config.ResolveInput{PipelineID: "dnd-session", Catalog: catalogFromRegistries(registries)})
if err != nil {
t.Fatalf("Resolve(production example) error = %v, want nil", err)
}
productionResolved := productionEffective.ResolvedPipeline
if productionConfig.Concurrency.TotalLLM != 1 || productionConfig.Concurrency.StageWorkers["extract"] != 1 || !reflect.DeepEqual(productionResolved.Chunk.Options, map[string]any{"max_units": 50}) {
t.Fatalf("production example concurrency/options = %#v/%#v, want compatibility snapshot", productionConfig.Concurrency, productionResolved.Chunk.Options)
}
bindings := productionResolved.ArtifactLanes[0].ExtractReferences.Bindings
if len(bindings) != 2 || bindings[0].SlotName != "glossary" || bindings[0].Source != "./dnd-spells-glossary.txt" || bindings[1].SlotName != "party" || bindings[1].Source != "./dnd-spells-roster.txt" {
t.Fatalf("production example reference bindings = %#v, want maintained glossary and party bindings", bindings)
}
}
func TestMaintainedSeriatimToDNDCompatibilityBundle(t *testing.T) {
tests := []struct {
name string
client contracts.StructuredLLMClient
wantStatus string
wantLaneFile bool
wantRejectedCount int
}{
{name: "approved", client: newFakeRunLLMClient(false), wantStatus: "approved", wantLaneFile: true},
{name: "validator rejection is nonfatal", client: newFakeRunLLMClient(true), wantStatus: "rejected", wantRejectedCount: 1},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
outputDir := t.TempDir()
var stdout bytes.Buffer
var stderr bytes.Buffer
code := RunWithOptions([]string{
"run", "dnd-session",
"--config", fixturePath(t, "examples/dnd-spells.config.yml"),
"--input", fixturePath(t, "examples/seriatim-minimal-transcript.json"),
"--output-dir", outputDir,
"--diagnostics-dir", t.TempDir(),
}, &stdout, &stderr, Options{LLMClientFactory: fakeLLMFactory(test.client, nil)})
if code != 0 {
t.Fatalf("RunWithOptions() code = %d, stderr=%q", code, stderr.String())
}
runDir := onlyChildDir(t, outputDir)
wantFiles := []string{"index.json", "manifest.json", "rejected.json", "warnings.json"}
if test.wantLaneFile {
wantFiles = append(wantFiles, "lanes/spells.json")
}
sort.Strings(wantFiles)
if got := relativeFileNames(t, runDir); !reflect.DeepEqual(got, wantFiles) {
t.Fatalf("durable files = %#v, want compatibility snapshot %#v", got, wantFiles)
}
var manifest artifacts.RunManifest
readJSONFile(t, filepath.Join(runDir, "manifest.json"), &manifest)
if manifest.PipelineID != "dnd-session" || manifest.InputModule != "seriatim" || manifest.Chunker != "generic" || manifest.OutputEncoder != "json" {
t.Fatalf("manifest module provenance = %#v, want maintained production modules", manifest)
}
if len(manifest.ArtifactLanes) != 1 || manifest.ArtifactLanes[0].ID != "spells" {
t.Fatalf("artifact lanes = %#v, want one spells lane", manifest.ArtifactLanes)
}
laneManifest := manifest.ArtifactLanes[0]
if laneManifest.Extractor != "dnd/spells" || laneManifest.Merger != "appendorder" || laneManifest.Normalizer != "noop" {
t.Fatalf("manifest lane module provenance = %#v, want maintained production modules", laneManifest)
}
if len(manifest.Extractors) != 0 || manifest.Merger != "" || manifest.Normalizer != "" {
t.Fatalf("legacy top-level lane summaries = %#v/%q/%q, want empty compatibility snapshot", manifest.Extractors, manifest.Merger, manifest.Normalizer)
}
if manifest.ValidationStatus != test.wantStatus || len(manifest.RejectedOutputs) != test.wantRejectedCount {
t.Fatalf("manifest outcome = status %q rejected %#v, want %q/%d", manifest.ValidationStatus, manifest.RejectedOutputs, test.wantStatus, test.wantRejectedCount)
}
if !reflect.DeepEqual(manifest.SourceDigests, []string{"sha256:1c98d94ae632fb10a2b56f684cd4fb1019cedb1a629e57dc0977cf4a54135be0"}) {
t.Fatalf("source digests = %#v, want maintained fixture provenance", manifest.SourceDigests)
}
if got := manifestValidatorKeys(manifestValidatorChain(t, manifest, pipeline.StageExtract, "spells", spells.Key)); !reflect.DeepEqual(got, []string{
"generic/valid_json", "generic/valid_json_schema", "extract/dnd/spells/shape", "extract/dnd/spells/source_refs", "extract/dnd/spells/source_relatedness",
}) {
t.Fatalf("manifest validator chain = %#v, want compatibility snapshot", got)
}
var index struct {
ManifestFile string `json:"manifest_file"`
OutputFiles []struct {
LaneID string `json:"lane_id"`
MediaType string `json:"media_type"`
File string `json:"file"`
ModuleKey string `json:"module_key"`
SchemaID string `json:"schema_id"`
SchemaName string `json:"schema_name"`
SchemaVersion string `json:"schema_version"`
} `json:"output_files"`
RejectedFile string `json:"rejected_file"`
WarningsFile string `json:"warnings_file"`
}
readJSONFile(t, filepath.Join(runDir, "index.json"), &index)
if index.ManifestFile != "manifest.json" || index.RejectedFile != "rejected.json" || index.WarningsFile != "warnings.json" {
t.Fatalf("output index fixed files = %#v, want compatibility snapshot", index)
}
if test.wantLaneFile {
if len(index.OutputFiles) != 1 {
t.Fatalf("output index entries = %#v, want one", index.OutputFiles)
}
wantOutput := struct {
LaneID, MediaType, File, ModuleKey, SchemaID, SchemaName, SchemaVersion string
}{"spells", "application/json", "lanes/spells.json", "noop", "notarius.dnd.spells", "notarius_dnd_spells_v1", "v1"}
gotOutput := index.OutputFiles[0]
got := struct {
LaneID, MediaType, File, ModuleKey, SchemaID, SchemaName, SchemaVersion string
}{gotOutput.LaneID, gotOutput.MediaType, gotOutput.File, gotOutput.ModuleKey, gotOutput.SchemaID, gotOutput.SchemaName, gotOutput.SchemaVersion}
if got != wantOutput {
t.Fatalf("output index entries = %#v, want compatibility snapshot %#v", index.OutputFiles, wantOutput)
}
assertJSONEqual(t, readFile(t, filepath.Join(runDir, "lanes/spells.json")), []byte(`{
"spell_casts": [{
"caster": "Aria",
"spell": "Cure Wounds",
"effect": "Heals a wounded ally.",
"narrative_description": "Aria casts Cure Wounds.",
"source_refs": [{"source_id": "session-alpha", "start_unit_id": 1, "end_unit_id": 1}]
}]
}`))
} else if len(index.OutputFiles) != 0 {
t.Fatalf("output index entries = %#v, want none for rejected lane", index.OutputFiles)
}
var warnings struct {
Warnings []contracts.Warning `json:"warnings"`
}
readJSONFile(t, filepath.Join(runDir, "warnings.json"), &warnings)
if len(warnings.Warnings) != 0 {
t.Fatalf("warnings = %#v, want empty compatibility snapshot", warnings.Warnings)
}
var rejected struct {
Rejected []contracts.RejectedOutput `json:"rejected"`
}
readJSONFile(t, filepath.Join(runDir, "rejected.json"), &rejected)
if len(rejected.Rejected) != test.wantRejectedCount {
t.Fatalf("rejected outputs = %#v, want %d", rejected.Rejected, test.wantRejectedCount)
}
if test.wantRejectedCount == 1 {
got := rejected.Rejected[0]
if got.Stage != "extract" || got.LaneID != "spells" || got.ModuleKey != "dnd/spells" || got.ChunkID != "chunk-000001" || got.ChunkIndex != 0 || got.ValidatorName != "extract/dnd/spells/source_refs" || got.ReasonCode != "invalid_source_refs" || got.AttemptCount != 1 {
t.Fatalf("rejection = %#v, want maintained nonfatal validator outcome", got)
}
}
})
}
}
func TestProductionLLMCallersShareScheduledClient(t *testing.T) {
underlying := newBlockingProductionLLMClient()
scheduler, err := frameworkllm.NewScheduler(1)
if err != nil {
t.Fatalf("NewScheduler() error = %v, want nil", err)
}
client := frameworkllm.NewScheduledClient(underlying, scheduler)
doc := &source.SourceDocument{
ID: "session-alpha", Kind: "transcript", Format: "application/json", Digest: "sha256:source",
Units: []source.SourceUnit{
{ID: 1, Kind: "segment", Text: "Aria casts Cure Wounds.", Ref: source.SourceRef{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 1}},
{ID: 2, Kind: "segment", Text: "The spell takes effect.", Ref: source.SourceRef{SourceID: "session-alpha", StartUnitID: 2, EndUnitID: 2}},
},
}
chunk := source.Chunk{
ID: "session-alpha:chunk:0", SourceID: doc.ID, Index: 0, Ref: source.SourceRef{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 2},
Content: []byte(`{"scene":"Aria casts Cure Wounds."}`), MediaType: "application/json", Units: append([]source.SourceUnit(nil), doc.Units...),
}
var started sync.WaitGroup
started.Add(2)
errs := make(chan error, 2)
go func() {
started.Done()
chunker, err := scenes.New(client, scenes.Options{})
if err == nil {
_, err = chunker.Plan(context.Background(), contracts.ChunkRequest{Source: doc})
}
errs <- err
}()
go func() {
started.Done()
extractor, err := spells.New(client, spells.Options{})
if err == nil {
_, err = extractor.Extract(context.Background(), contracts.TypedExtractionRequest{Source: doc, Chunk: &chunk})
}
errs <- err
}()
started.Wait()
for i := 0; i < 2; i++ {
<-underlying.entered
underlying.release <- struct{}{}
}
for i := 0; i < 2; i++ {
if err := <-errs; err != nil {
t.Fatalf("production LLM caller error = %v, want nil", err)
}
}
if underlying.maxActive != 1 {
t.Fatalf("maximum concurrent provider calls = %d, want total_llm limit 1", underlying.maxActive)
}
sort.Strings(underlying.stageNames)
if !reflect.DeepEqual(underlying.stageNames, []string{"dnd/scenes", "dnd/spells"}) {
t.Fatalf("scheduled stage names = %#v, want both production LLM callers", underlying.stageNames)
}
}
func TestProductionBundlePreservesLaneAndChunkOrder(t *testing.T) {
configPath := writeTestConfig(t, `version: 2
pipelines:
dnd-session:
input: seriatim
chunk:
module: generic
options:
max_units: 1
artifacts:
zeta:
extract: dnd/spells
alpha:
extract: dnd/spells
`)
outputDir := t.TempDir()
var stdout bytes.Buffer
var stderr bytes.Buffer
code := RunWithOptions([]string{
"run", "dnd-session",
"--config", configPath,
"--input", fixturePath(t, "examples/seriatim-minimal-transcript.json"),
"--output-dir", outputDir,
"--diagnostics-dir", t.TempDir(),
}, &stdout, &stderr, Options{LLMClientFactory: fakeLLMFactory(orderingProductionLLMClient{}, nil)})
if code != 0 {
t.Fatalf("RunWithOptions() code = %d, stderr=%q", code, stderr.String())
}
runDir := onlyChildDir(t, outputDir)
var index struct {
OutputFiles []struct {
LaneID string `json:"lane_id"`
} `json:"output_files"`
}
readJSONFile(t, filepath.Join(runDir, "index.json"), &index)
if len(index.OutputFiles) != 2 {
t.Fatalf("output index entries = %#v, want two lanes", index.OutputFiles)
}
if got := []string{index.OutputFiles[0].LaneID, index.OutputFiles[1].LaneID}; !reflect.DeepEqual(got, []string{"alpha", "zeta"}) {
t.Fatalf("output lane order = %#v, want resolved lane order", got)
}
for _, laneID := range []string{"alpha", "zeta"} {
var payload struct {
SpellCasts []struct {
Spell string `json:"spell"`
SourceRefs []source.SourceRef `json:"source_refs"`
} `json:"spell_casts"`
}
readJSONFile(t, filepath.Join(runDir, "lanes", laneID+".json"), &payload)
if len(payload.SpellCasts) != 2 {
t.Fatalf("lane %q spell casts = %#v, want one per source chunk", laneID, payload.SpellCasts)
}
got := []any{
payload.SpellCasts[0].Spell, payload.SpellCasts[0].SourceRefs[0].StartUnitID,
payload.SpellCasts[1].Spell, payload.SpellCasts[1].SourceRefs[0].StartUnitID,
}
if !reflect.DeepEqual(got, []any{"Cure Wounds", 1, "Shield", 2}) {
t.Fatalf("lane %q chunk handoff order = %#v, want source chunk order", laneID, got)
}
}
}
type blockingProductionLLMClient struct {
mu sync.Mutex
active int
maxActive int
stageNames []string
entered chan struct{}
release chan struct{}
}
type orderingProductionLLMClient struct{}
func (orderingProductionLLMClient) CompleteStructured(_ context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
material := req.Inputs["transcript"]
unitID := 1
spellName := "Cure Wounds"
if strings.Contains(string(material.Content), "Shield") {
unitID = 2
spellName = "Shield"
}
payload, err := json.Marshal(map[string]any{
"spell_casts": []map[string]any{{
"caster": "Aria",
"spell": spellName,
"effect": "Fixture effect.",
"narrative_description": "Fixture spell cast.",
"source_refs": []map[string]any{{
"start_unit_id": unitID,
"end_unit_id": unitID,
}},
}},
})
if err != nil {
return contracts.StructuredCompletionResponse{}, err
}
if err := json.Unmarshal(payload, out); err != nil {
return contracts.StructuredCompletionResponse{}, err
}
return contracts.StructuredCompletionResponse{Content: payload}, nil
}
func newBlockingProductionLLMClient() *blockingProductionLLMClient {
return &blockingProductionLLMClient{entered: make(chan struct{}, 2), release: make(chan struct{}, 2)}
}
func (client *blockingProductionLLMClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
client.mu.Lock()
client.active++
if client.active > client.maxActive {
client.maxActive = client.active
}
client.stageNames = append(client.stageNames, req.StageName)
client.mu.Unlock()
client.entered <- struct{}{}
select {
case <-ctx.Done():
return contracts.StructuredCompletionResponse{}, ctx.Err()
case <-client.release:
}
client.mu.Lock()
client.active--
client.mu.Unlock()
var payload []byte
switch req.StageName {
case scenes.Key:
payload = []byte(`{"scenes":[{"start_unit_id":1,"end_unit_id":2,"short_title":"Spell","primary_mode":"Narrative","main_participants":["Aria"],"summary":"Aria casts a spell.","boundary_note":"Complete source.","boundary_confidence":"High"}],"boundary_caveats":[]}`)
case spells.Key:
payload = []byte(`{"spell_casts":[{"caster":"Aria","spell":"Cure Wounds","effect":"Healing","narrative_description":"Aria casts Cure Wounds.","source_refs":[{"start_unit_id":1,"end_unit_id":1}]}]}`)
}
if err := json.Unmarshal(payload, out); err != nil {
return contracts.StructuredCompletionResponse{}, err
}
return contracts.StructuredCompletionResponse{Content: payload}, nil
}
func sceneManifestMetadata(t *testing.T) map[string]any {
t.Helper()
chunker, err := scenes.New(orderingProductionLLMClient{}, scenes.Options{})
if err != nil {
t.Fatalf("construct scene chunker: %v", err)
}
return chunker.ManifestMetadata()
}
func spellManifestMetadata(t *testing.T) map[string]any {
t.Helper()
extractor, err := spells.New(orderingProductionLLMClient{}, spells.Options{})
if err != nil {
t.Fatalf("construct spell extractor: %v", err)
}
return extractor.ManifestMetadata()
}
func assertAssetNames(t *testing.T, getFS func() (fs.FS, error), want []string) {
t.Helper()
fSys, err := getFS()
if err != nil {
t.Fatalf("asset filesystem error = %v, want nil", err)
}
var got []string
if err := fs.WalkDir(fSys, ".", func(path string, entry fs.DirEntry, err error) error {
if err == nil && !entry.IsDir() {
got = append(got, path)
}
return err
}); err != nil {
t.Fatalf("walk assets: %v", err)
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("asset names = %#v, want compatibility snapshot %#v", got, want)
}
}
func resolvedValidatorKeys(chains []pipeline.ResolvedValidatorChain, stage pipeline.ModuleStage, laneID, moduleKey string) []string {
for _, chain := range chains {
if chain.Stage == stage && chain.LaneID == laneID && chain.ModuleKey == moduleKey {
keys := make([]string, 0, len(chain.Validators))
for _, validator := range chain.Validators {
keys = append(keys, validator.Binding.Module)
}
return keys
}
}
return nil
}
func relativeFileNames(t *testing.T, root string) []string {
t.Helper()
var names []string
if err := filepath.WalkDir(root, func(path string, entry fs.DirEntry, err error) error {
if err != nil || entry.IsDir() {
return err
}
rel, err := filepath.Rel(root, path)
if err != nil {
return err
}
names = append(names, filepath.ToSlash(rel))
return nil
}); err != nil {
t.Fatalf("walk durable output: %v", err)
}
sort.Strings(names)
return names
}
func assertJSONEqual(t *testing.T, got, want []byte) {
t.Helper()
var gotValue any
var wantValue any
if err := json.Unmarshal(got, &gotValue); err != nil {
t.Fatalf("unmarshal actual JSON: %v", err)
}
if err := json.Unmarshal(want, &wantValue); err != nil {
t.Fatalf("unmarshal expected JSON: %v", err)
}
if !reflect.DeepEqual(gotValue, wantValue) {
t.Fatalf("JSON = %#v, want compatibility snapshot %#v", gotValue, wantValue)
}
}

290
internal/cli/run.go
View File

@@ -17,8 +17,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
"gitea.maximumdirect.net/eric/notarius/internal/core/workspace"
"gitea.maximumdirect.net/eric/notarius/internal/core/debugbundle"
"gitea.maximumdirect.net/eric/notarius/internal/framework/checkpoint"
"gitea.maximumdirect.net/eric/notarius/internal/framework/chunkplan"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
@@ -27,11 +26,9 @@ import (
)
const defaultConfigPath = "/usr/local/etc/notarius/config.yml"
const defaultOutputRoot = "./notarius-output"
const usage = `Usage:
notarius help
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--chunk_cache auto|bypass|refresh] [--resume] [--session-id id] [--reference selector=path] [--without-reference selector]
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--output-dir path] [--chunk_cache auto|bypass|refresh] [--resume] [--debug [--debug-dir path]] [--only lane-a,lane-b] [--session-id id] [--reference selector=path] [--without-reference selector]
notarius config validate --config path/to/config.yml [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list --config path/to/config.yml [--json]
`
@@ -44,6 +41,7 @@ type Options struct {
Now func() time.Time
UserCacheDir func() (string, error)
ChunkPlanStoreFactory pipeline.ChunkPlanStoreFactory
DebugRecorderFactory func(string) (pipeline.DebugRecorder, error)
}
type LLMClientFactory func(ctx context.Context, cfg config.Config, profileID string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error)
@@ -99,6 +97,9 @@ func normalizeOptions(opts Options) (Options, error) {
if opts.ChunkPlanStoreFactory == nil {
opts.ChunkPlanStoreFactory = chunkplan.NewFilesystemStore
}
if opts.DebugRecorderFactory == nil {
opts.DebugRecorderFactory = frameworkdebug.NewFilesystemRecorder
}
if isEmptyCatalog(opts.Catalog) && isEmptyRegistries(opts.Registries) {
components, err := newProductionComponents()
if err != nil {
@@ -123,9 +124,10 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
inputPath := fs.String("input", "", "source input file path")
onlyRaw := fs.String("only", "", "comma-separated artifact lanes")
outputDir := fs.String("output-dir", "", "output directory")
diagnosticsDir := fs.String("diagnostics-dir", "", "diagnostics directory")
debug := fs.Bool("debug", false, "write a debug bundle")
debugDir := fs.String("debug-dir", "", "debug bundle directory")
llmProfile := fs.String("llm-profile", "", "LLM profile override")
resume := fs.Bool("resume", false, "reuse valid workspace checkpoints")
resume := fs.Bool("resume", false, "reuse and record compatible checkpoints")
chunkCache := chunkCacheFlag{}
sessionID := sessionIDFlag{}
referenceFlags := stringListFlag{}
@@ -159,6 +161,18 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
fmt.Fprintln(stderr, "notarius: run requires --input")
return 2
}
if strings.TrimSpace(*debugDir) != "" && !*debug {
fmt.Fprintln(stderr, "notarius: --debug-dir requires --debug")
return 2
}
if strings.TrimSpace(*outputDir) == "" && flagWasProvided(args, "--output-dir") {
fmt.Fprintln(stderr, "notarius: --output-dir must not be empty")
return 2
}
if strings.TrimSpace(*debugDir) == "" && flagWasProvided(args, "--debug-dir") {
fmt.Fprintln(stderr, "notarius: --debug-dir must not be empty")
return 2
}
if sessionID.set && strings.TrimSpace(sessionID.value) == "" {
fmt.Fprintln(stderr, "notarius: --session-id must not be empty")
return 2
@@ -185,26 +199,38 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
return 1
}
if chunkCache.set {
cfg.Workspace.ChunkCache.Mode = chunkCache.value
cfg.Cache.ChunkPlans.Mode = chunkCache.value
}
workspaceSettings := workspace.FromConfig(cfg)
if dir := strings.TrimSpace(*diagnosticsDir); dir != "" {
workspaceSettings.DiagnosticsRoot = dir
if dir := strings.TrimSpace(*outputDir); dir != "" {
cfg.Output.Directory = dir
}
if dir := strings.TrimSpace(*debugDir); dir != "" {
cfg.Debug.Directory = dir
}
if err := cfg.Validate(); err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err)
return 1
}
startedAt := opts.Now().UTC()
runID := fmt.Sprintf("run-%d", startedAt.UnixNano())
var runDir *diagnostics.RunDirectory
if workspaceSettings.DiagnosticsEnabled {
var err error
runDir, err = diagnostics.NewRunDirectory(workspaceSettings.DiagnosticsRoot, cfg.Diagnostics.Retention)
var summary *debugbundle.SummaryWriter
debugPath := ""
debugRecorder := pipeline.NoopDebugRecorder()
if *debug {
bundle, err := debugbundle.Allocate(cfg.Debug.Directory)
if err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err)
return 1
}
runID = runDir.RunID()
runID, debugPath, summary = bundle.RunID(), bundle.Path(), bundle.Summary()
debugRecorder, err = opts.DebugRecorderFactory(bundle.TraceRoot())
if err != nil {
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("create debug recorder: %w", err), true)
}
invocation := diagnostics.InvocationMetadata{
debugRecorder = pipeline.SynchronizedDebugRecorder(debugRecorder)
}
invocation := debugbundle.Invocation{
Operation: "run",
PipelineID: pipelineID,
InputPath: strings.TrimSpace(*inputPath),
@@ -216,25 +242,17 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
RunID: runID,
StartedAt: startedAt,
}
if err := writeDiagnostics(runDir, func() error { return runDir.WriteInvocationMetadata(invocation) }); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics invocation metadata: %w", err))
if err := writeSummary(summary, func() error { return summary.WriteInvocation(invocation) }); err != nil {
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug invocation metadata: %w", err), false)
}
if *resume && !workspaceSettings.ResumeEnabled {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("--resume requires workspace.resume.enabled: true"))
}
debugRecorder, err := frameworkdebug.NewWorkspaceRecorder(workspaceSettings, runID)
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("create debug recorder: %w", err))
}
debugRecorder = pipeline.SynchronizedDebugRecorder(debugRecorder)
catalog, err := effectiveCatalog(opts)
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
return failPipelineCommand(stderr, summary, debugPath, err, true)
}
referenceOverrides, referenceUnbinds, err := resolveCLIReferenceRequests(cfg, pipelineID, only, catalog, referenceRequests, referenceUnbindRequests)
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
return failPipelineCommand(stderr, summary, debugPath, err, true)
}
effective, err := cfg.Resolve(config.ResolveInput{
PipelineID: pipelineID,
@@ -245,43 +263,43 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
ReferenceUnbinds: referenceUnbinds,
})
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
return failPipelineCommand(stderr, summary, debugPath, err, true)
}
profileIDs := effectiveLLMProfileIDs(effective.ResolvedPipeline)
if err := validateExplicitScriptoriumProfiles(context.Background(), effective.Config, profileIDs); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
return failPipelineCommand(stderr, summary, debugPath, err, true)
}
workingDir, err := os.Getwd()
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("resolve working directory: %w", err))
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("resolve working directory: %w", err), true)
}
materialized, referenceWarnings, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalog, pipeline.ReferenceMaterializationOptions{
ConfigPath: loadedConfigPath,
WorkingDir: workingDir,
})
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
return failPipelineCommand(stderr, summary, debugPath, err, true)
}
effective.ResolvedPipeline = materialized
invocation.PipelineDigest = effective.ResolvedPipeline.Digest
if err := writeDiagnostics(runDir, func() error { return runDir.WriteInvocationMetadata(invocation) }); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics invocation metadata: %w", err))
if err := writeSummary(summary, func() error { return summary.WriteInvocation(invocation) }); err != nil {
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug invocation metadata: %w", err), false)
}
if err := writeDiagnostics(runDir, func() error { return runDir.WriteRedactedEffectiveConfig(effective) }); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics effective config: %w", err))
if err := writeSummary(summary, func() error { return summary.WriteRedactedEffectiveConfig(effective) }); err != nil {
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug effective config: %w", err), false)
}
if err := writeDiagnostics(runDir, func() error { return runDir.WriteResolvedPipeline(effective.ResolvedPipeline) }); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics resolved pipeline: %w", err))
if err := writeSummary(summary, func() error { return summary.WriteResolvedPipeline(effective.ResolvedPipeline) }); err != nil {
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug resolved pipeline: %w", err), false)
}
if err := writeDiagnostics(runDir, func() error {
return runDir.WriteResolvedReferences(pipeline.ReferenceProvenance(effective.ResolvedPipeline))
if err := writeSummary(summary, func() error {
return summary.WriteResolvedReferences(pipeline.ReferenceProvenance(effective.ResolvedPipeline))
}); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics resolved references: %w", err))
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug resolved references: %w", err), false)
}
registries, err := effectiveRegistries(opts)
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
return failPipelineCommand(stderr, summary, debugPath, err, true)
}
ctx := context.Background()
@@ -291,24 +309,24 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
}
llmClient, llmProfiles, err := opts.LLMClientFactory(ctx, effective.Config, factoryProfileID)
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("create LLM client for profile %q: %w", factoryProfileID, err))
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("create LLM client for profile %q: %w", factoryProfileID, err), true)
}
llmClient = pipeline.WithDebugLLMRecording(llmClient, debugRecorder)
prepared, err := pipeline.Prepare(effective.ResolvedPipeline, registries, pipeline.ModuleDependencies{LLM: llmClient})
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("prepare pipeline %q: %w", pipelineID, err))
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("prepare pipeline %q: %w", pipelineID, err), true)
}
rawInput, err := os.ReadFile(strings.TrimSpace(*inputPath))
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("read input %q: %w", strings.TrimSpace(*inputPath), err))
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("read input %q: %w", strings.TrimSpace(*inputPath), err), true)
}
chunkPlans, err := chunkPlanStoreForRun(effective.Config.Workspace.ChunkCache, opts)
chunkPlans, err := chunkPlanStoreForRun(effective.Config.Cache.ChunkPlans, opts)
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
return failPipelineCommand(stderr, summary, debugPath, err, true)
}
checkpointRecorder, checkpointLoader, err := checkpointHandlersForRun(workspaceSettings, effective.ResolvedPipeline, rawInput, only, llmProfiles, strings.TrimSpace(*llmProfile), strings.TrimSpace(sessionID.value), *resume)
checkpointRecorder, checkpointLoader, err := checkpointHandlersForRun(effective.Config.Cache.Checkpoints, opts, effective.ResolvedPipeline, rawInput, only, llmProfiles, strings.TrimSpace(*llmProfile), strings.TrimSpace(sessionID.value), *resume)
if err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
return failPipelineCommand(stderr, summary, debugPath, err, true)
}
output, err := pipeline.New().Run(ctx, pipeline.RunInput{
@@ -319,9 +337,9 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
RunID: runID,
StartedAt: startedAt,
LLMProfiles: llmProfiles,
Metadata: runMetadata(*outputDir, *diagnosticsDir),
Metadata: runMetadata(effective.Config.Output.Directory, debugPath),
Warnings: referenceWarnings,
ChunkCacheMode: effective.Config.Workspace.ChunkCache.Mode,
ChunkCacheMode: effective.Config.Cache.ChunkPlans.Mode,
ChunkPlans: chunkPlans,
Checkpoints: checkpointRecorder,
Checkpoint: checkpointLoader,
@@ -329,101 +347,78 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
ExtractWorkers: cfg.Concurrency.StageWorkers["extract"],
})
if err != nil {
if output.Manifest.PipelineID != "" && runDir != nil {
_ = runDir.WriteRunManifest(output.Manifest)
if output.ChunkPlan != nil {
_ = runDir.WriteChunkPlan(*output.ChunkPlan)
if output.Manifest.PipelineID != "" {
if summaryErr := writePartialSummary(summary, output); summaryErr != nil {
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("run pipeline %q: %w; write debug summary: %v", pipelineID, err, summaryErr), false)
}
_ = runDir.WriteCheckpointEvents(output.CheckpointEvents)
}
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("run pipeline %q: %w", pipelineID, err))
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("run pipeline %q: %w", pipelineID, err), true)
}
runOutputDir := filepath.Join(outputRoot(*outputDir), runID)
if err := writeDiagnostics(runDir, func() error { return runDir.WriteRunManifest(output.Manifest) }); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics run manifest: %w", err))
}
if output.ChunkPlan != nil {
if err := writeDiagnostics(runDir, func() error { return runDir.WriteChunkPlan(*output.ChunkPlan) }); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics chunk plan: %w", err))
}
}
if err := writeDiagnostics(runDir, func() error { return runDir.WriteWarnings(output.Warnings) }); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics warnings: %w", err))
}
if err := writeDiagnostics(runDir, func() error { return runDir.WriteCheckpointEvents(output.CheckpointEvents) }); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics checkpoint events: %w", err))
}
if err := writeDiagnostics(runDir, func() error {
return runDir.WriteRunReport(runReport{
RunID: runDir.RunID(),
PipelineID: effective.PipelineID,
OutputPath: runOutputDir,
DiagnosticsPath: runDir.Path(),
OutputCount: len(output.NormalizeOutputs),
RejectedCount: len(output.Rejected),
WarningCount: len(output.Warnings),
ValidationStatus: output.Manifest.ValidationStatus,
})
}); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("write diagnostics run report: %w", err))
runOutputDir := filepath.Join(effective.Config.Output.Directory, runID)
if err := writePartialSummary(summary, output); err != nil {
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug summary: %w", err), false)
}
if err := writeOutputFiles(runOutputDir, output.OutputFiles); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, err)
return failPipelineCommand(stderr, summary, debugPath, err, true)
}
if err := writeDiagnostics(runDir, func() error {
return runDir.ApplyRetention(diagnostics.RetentionDecisionInput{
RetentionMode: cfg.Diagnostics.Retention,
RunSucceeded: true,
HasWarnings: len(output.Warnings) > 0,
})
if err := writeSummary(summary, func() error {
return summary.WriteRunReport(debugbundle.RunReport{RunID: runID, PipelineID: effective.PipelineID, OutputPath: runOutputDir, DebugPath: debugPath, Succeeded: true, OutputCount: len(output.NormalizeOutputs), RejectedCount: len(output.Rejected), WarningCount: len(output.Warnings), ValidationStatus: output.Manifest.ValidationStatus})
}); err != nil {
return failPipelineCommand(stderr, runDir, cfg.Diagnostics.Retention, fmt.Errorf("apply diagnostics retention: %w", err))
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug run report: %w", err), false)
}
fmt.Fprintf(stdout, "pipeline %q complete: outputs=%d rejected=%d output=%s\n", effective.PipelineID, len(output.NormalizeOutputs), len(output.Rejected), runOutputDir)
if debugPath != "" {
fmt.Fprintf(stdout, "debug=%s\n", debugPath)
}
if len(output.Warnings) > 0 {
fmt.Fprintf(stderr, "notarius: run completed with %d warning(s)\n", len(output.Warnings))
}
return 0
}
type runReport struct {
RunID string `json:"run_id"`
PipelineID string `json:"pipeline_id"`
OutputPath string `json:"output_path"`
DiagnosticsPath string `json:"diagnostics_path,omitempty"`
OutputCount int `json:"output_count"`
RejectedCount int `json:"rejected_count"`
WarningCount int `json:"warning_count"`
ValidationStatus string `json:"validation_status,omitempty"`
}
func failPipelineCommand(stderr io.Writer, runDir *diagnostics.RunDirectory, retention diagnostics.RetentionMode, err error) int {
func failPipelineCommand(stderr io.Writer, summary *debugbundle.SummaryWriter, debugPath string, err error, recordError bool) int {
fmt.Fprintf(stderr, "notarius: %v\n", err)
if runDir != nil {
if logErr := runDir.WriteErrorLog(err.Error()); logErr != nil {
fmt.Fprintf(stderr, "notarius: write diagnostics error log: %v\n", logErr)
if recordError && summary != nil {
if summaryErr := summary.WriteError(err.Error()); summaryErr != nil {
fmt.Fprintf(stderr, "notarius: write debug error log: %v\n", summaryErr)
}
if retentionErr := runDir.ApplyRetention(diagnostics.RetentionDecisionInput{
RetentionMode: retention,
RunSucceeded: false,
}); retentionErr != nil {
fmt.Fprintf(stderr, "notarius: apply diagnostics retention: %v\n", retentionErr)
}
if debugPath != "" {
fmt.Fprintf(stderr, "notarius: debug=%s\n", debugPath)
}
return 1
}
func writeDiagnostics(runDir *diagnostics.RunDirectory, write func() error) error {
if runDir == nil {
func writeSummary(summary *debugbundle.SummaryWriter, write func() error) error {
if summary == nil {
return nil
}
return write()
}
func writePartialSummary(summary *debugbundle.SummaryWriter, output pipeline.RunOutput) error {
if summary == nil {
return nil
}
if err := summary.WriteRunManifest(output.Manifest); err != nil {
return err
}
if output.ChunkPlan != nil {
if err := summary.WriteChunkPlan(*output.ChunkPlan); err != nil {
return err
}
}
if err := summary.WriteWarnings(output.Warnings); err != nil {
return err
}
return summary.WriteCheckpointEvents(output.CheckpointEvents)
}
func checkpointHandlersForRun(
settings workspace.Settings,
settings config.CheckpointCacheConfig,
opts Options,
resolved pipeline.ResolvedPipeline,
rawInput []byte,
only []string,
@@ -432,7 +427,10 @@ func checkpointHandlersForRun(
sessionID string,
resume bool,
) (pipeline.CheckpointRecorder, pipeline.CheckpointLoader, error) {
identity, err := workspace.NewCheckpointIdentity(workspace.CheckpointIdentityInput{
if !resume {
return pipeline.NoopCheckpointRecorder(), pipeline.NoopCheckpointLoader(), nil
}
identity, err := checkpoint.NewIdentity(checkpoint.IdentityInput{
Pipeline: resolved,
InputKey: resolved.Input.Module,
RawInputDigest: rawInputDigest(rawInput),
@@ -444,17 +442,21 @@ func checkpointHandlersForRun(
if err != nil {
return nil, nil, fmt.Errorf("create checkpoint identity: %w", err)
}
recorder, err := checkpoint.NewWorkspaceRecorder(settings, identity)
checkpointRoot := strings.TrimSpace(settings.Directory)
if checkpointRoot == "" {
checkpointRoot, err = config.DefaultCheckpointRoot(opts.UserCacheDir)
if err != nil {
return nil, nil, fmt.Errorf("resolve checkpoint root: %w", err)
}
}
recorder, err := checkpoint.NewFilesystemRecorder(checkpointRoot, identity)
if err != nil {
return nil, nil, fmt.Errorf("create checkpoint recorder: %w", err)
}
loader := pipeline.NoopCheckpointLoader()
if resume {
loader, err = checkpoint.NewWorkspaceLoader(settings, identity)
loader, err := checkpoint.NewFilesystemLoader(checkpointRoot, identity)
if err != nil {
return nil, nil, fmt.Errorf("create checkpoint loader: %w", err)
}
}
return recorder, loader, nil
}
@@ -463,28 +465,28 @@ func rawInputDigest(data []byte) string {
return "sha256:" + hex.EncodeToString(sum[:])
}
func runtimeOverrideFingerprints(llmProfileOverride string, sessionID string) []workspace.Fingerprint {
var values []workspace.Fingerprint
func runtimeOverrideFingerprints(llmProfileOverride string, sessionID string) []checkpoint.Fingerprint {
var values []checkpoint.Fingerprint
if strings.TrimSpace(llmProfileOverride) != "" {
values = append(values, workspace.Fingerprint{Name: "llm_profile_override", Value: strings.TrimSpace(llmProfileOverride)})
values = append(values, checkpoint.Fingerprint{Name: "llm_profile_override", Value: strings.TrimSpace(llmProfileOverride)})
}
if strings.TrimSpace(sessionID) != "" {
values = append(values, workspace.Fingerprint{Name: "session_id", Value: strings.TrimSpace(sessionID)})
values = append(values, checkpoint.Fingerprint{Name: "session_id", Value: strings.TrimSpace(sessionID)})
}
return values
}
func llmProfileFingerprints(profiles []artifacts.LLMProfileManifest) []workspace.Fingerprint {
func llmProfileFingerprints(profiles []artifacts.LLMProfileManifest) []checkpoint.Fingerprint {
if len(profiles) == 0 {
return nil
}
values := make([]workspace.Fingerprint, 0, len(profiles))
values := make([]checkpoint.Fingerprint, 0, len(profiles))
for _, profile := range profiles {
id := strings.TrimSpace(profile.ID)
if id == "" {
continue
}
values = append(values, workspace.Fingerprint{
values = append(values, checkpoint.Fingerprint{
Name: "llm_profile:" + id,
Value: strings.TrimSpace(profile.Provider) + ":" + strings.TrimSpace(profile.Model),
})
@@ -499,13 +501,6 @@ func configSource(configPath string) string {
return "discovered"
}
func outputRoot(outputDir string) string {
if dir := strings.TrimSpace(outputDir); dir != "" {
return dir
}
return defaultOutputRoot
}
func writeOutputFiles(runOutputDir string, files []contracts.OutputFile) error {
type outputTarget struct {
path string
@@ -627,7 +622,7 @@ func reorderRunArgs(args []string) []string {
func runFlagTakesValue(arg string) bool {
switch arg {
case "--config", "--input", "--only", "--output-dir", "--diagnostics-dir", "--llm-profile", "--session-id", "--chunk_cache", "--reference", "--without-reference":
case "--config", "--input", "--only", "--output-dir", "--debug-dir", "--llm-profile", "--session-id", "--chunk_cache", "--reference", "--without-reference":
return true
default:
return false
@@ -663,14 +658,14 @@ func (f chunkCacheFlag) explicitValue() string {
return string(f.value)
}
func chunkPlanStoreForRun(cfg config.WorkspaceChunkCacheConfig, opts Options) (pipeline.ChunkPlanStore, error) {
func chunkPlanStoreForRun(cfg config.ChunkPlanCacheConfig, opts Options) (pipeline.ChunkPlanStore, error) {
if cfg.Mode == pipeline.ChunkCacheBypass {
return nil, nil
}
root := strings.TrimSpace(cfg.Directory)
if root == "" {
var err error
root, err = workspace.DefaultChunkPlanRoot(opts.UserCacheDir)
root, err = config.DefaultChunkPlanRoot(opts.UserCacheDir)
if err != nil {
return nil, fmt.Errorf("resolve chunk plan root: %w", err)
}
@@ -697,6 +692,15 @@ func validateRunFlagValues(args []string) error {
return nil
}
func flagWasProvided(args []string, name string) bool {
for _, arg := range args {
if arg == name || strings.HasPrefix(arg, name+"=") {
return true
}
}
return false
}
func effectiveLLMProfileIDs(resolved pipeline.ResolvedPipeline) []string {
seen := make(map[string]struct{})
add := func(binding pipeline.ModuleBinding) {
@@ -726,13 +730,13 @@ func effectiveLLMProfileIDs(resolved pipeline.ResolvedPipeline) []string {
return ids
}
func runMetadata(outputDir, diagnosticsDir string) map[string]any {
func runMetadata(outputDir, debugDir string) map[string]any {
metadata := make(map[string]any)
if dir := strings.TrimSpace(outputDir); dir != "" {
metadata["output_dir"] = dir
}
if dir := strings.TrimSpace(diagnosticsDir); dir != "" {
metadata["diagnostics_dir"] = dir
if dir := strings.TrimSpace(debugDir); dir != "" {
metadata["debug_dir"] = dir
}
if len(metadata) == 0 {
return nil

4256
internal/cli/run_test.go
View File

File diff suppressed because it is too large Load Diff

528
internal/cli/state_hardening_test.go Normal file
View File

@@ -0,0 +1,528 @@
package cli
import (
"bytes"
"context"
"errors"
"fmt"
"os"
"path/filepath"
"runtime"
"strings"
"sync"
"testing"
"time"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/chunkplan"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
frameworkdebug "gitea.maximumdirect.net/eric/notarius/internal/framework/debug"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const stateTestDigest = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
func TestRunStateSurfaceMatrix(t *testing.T) {
for _, debug := range []bool{false, true} {
for _, resume := range []bool{false, true} {
for _, mode := range []string{"auto", "bypass", "refresh"} {
name := fmt.Sprintf("debug=%t/resume=%t/cache=%s", debug, resume, mode)
t.Run(name, func(t *testing.T) {
roots := newStateTestRoots(t)
harness := newStateTestHarness()
opts := harness.options()
var storeRoots []string
opts.ChunkPlanStoreFactory = func(root string) (pipeline.ChunkPlanStore, error) {
storeRoots = append(storeRoots, root)
return chunkplan.NewFilesystemStore(root)
}
result := runStateTest(t, roots, opts, debug, resume, mode)
if result.code != 0 {
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
}
assertStateTestOutput(t, roots.output)
if mode == "bypass" {
assertAbsent(t, roots.plans)
if len(storeRoots) != 0 {
t.Fatalf("chunk plan store roots = %v, want none", storeRoots)
}
} else {
assertFile(t, filepath.Join(roots.plans, strings.TrimPrefix(stateTestDigest, "sha256:"), "plan.json"))
if len(storeRoots) != 1 || storeRoots[0] != roots.plans {
t.Fatalf("chunk plan store roots = %v, want [%q]", storeRoots, roots.plans)
}
}
if resume {
assertAnyFile(t, roots.checkpoints)
assertRestrictedTree(t, roots.checkpoints)
} else {
assertAbsent(t, roots.checkpoints)
}
if debug {
bundle := onlyChildDir(t, roots.debug)
assertFile(t, filepath.Join(bundle, "summary", "invocation.json"))
assertAnyFile(t, filepath.Join(bundle, "trace"))
assertRestrictedTree(t, roots.debug)
} else {
assertAbsent(t, roots.debug)
}
})
}
}
}
}
func TestRunKeepsStateRootsIndependentAndReusesSelectedCheckpointRoot(t *testing.T) {
roots := newStateTestRoots(t)
harness := newStateTestHarness()
first := runStateTest(t, roots, harness.options(), true, false, "auto")
if first.code != 0 {
t.Fatalf("first run code=%d stderr=%q", first.code, first.stderr)
}
planPath := filepath.Join(roots.plans, strings.TrimPrefix(stateTestDigest, "sha256:"), "plan.json")
initialPlan, err := os.ReadFile(planPath)
if err != nil {
t.Fatal(err)
}
firstBundle := onlyChildDir(t, roots.debug)
second := runStateTest(t, roots, harness.options(), false, false, "auto")
if second.code != 0 {
t.Fatalf("second run code=%d stderr=%q", second.code, second.stderr)
}
if harness.chunkCalls != 1 {
t.Fatalf("chunk calls after debug toggle = %d, want 1", harness.chunkCalls)
}
if got, err := os.ReadFile(planPath); err != nil || !bytes.Equal(got, initialPlan) {
t.Fatalf("chunk plan changed after debug toggle: %v", err)
}
if _, err := os.Stat(firstBundle); err != nil {
t.Fatalf("initial debug bundle was removed: %v", err)
}
checkpointRoot := roots.checkpoints
seed := runStateTest(t, roots, harness.options(), false, true, "auto")
if seed.code != 0 {
t.Fatalf("checkpoint seed code=%d stderr=%q", seed.code, seed.stderr)
}
extractCalls := harness.extractCalls
checkpointFiles := readTree(t, checkpointRoot)
reused := runStateTest(t, roots, harness.options(), false, true, "auto")
if reused.code != 0 {
t.Fatalf("checkpoint reuse code=%d stderr=%q", reused.code, reused.stderr)
}
if harness.extractCalls != extractCalls {
t.Fatalf("extract calls after checkpoint reuse = %d, want %d", harness.extractCalls, extractCalls)
}
if got := readTree(t, checkpointRoot); !sameFiles(got, checkpointFiles) {
t.Fatal("reused checkpoint was rewritten")
}
}
func TestRunRecomputesOnlyAfterExplicitChunkPlanRemoval(t *testing.T) {
roots := newStateTestRoots(t)
harness := newStateTestHarness()
first := runStateTest(t, roots, harness.options(), true, false, "auto")
if first.code != 0 {
t.Fatalf("first run code=%d stderr=%q", first.code, first.stderr)
}
firstOutput := onlyChildDir(t, roots.output)
firstBundle := onlyChildDir(t, roots.debug)
entry := filepath.Join(roots.plans, strings.TrimPrefix(stateTestDigest, "sha256:"))
if err := os.RemoveAll(entry); err != nil {
t.Fatal(err)
}
second := runStateTest(t, roots, harness.options(), false, false, "auto")
if second.code != 0 {
t.Fatalf("second run code=%d stderr=%q", second.code, second.stderr)
}
if harness.chunkCalls != 2 {
t.Fatalf("chunk calls = %d, want 2 after removing exact cache entry", harness.chunkCalls)
}
assertFile(t, filepath.Join(firstOutput, "result.json"))
assertFile(t, filepath.Join(firstBundle, "summary", "run-report.json"))
}
func TestRunRetainsDebugBundlesAcrossFailures(t *testing.T) {
t.Run("configuration failure precedes allocation", func(t *testing.T) {
root := filepath.Join(t.TempDir(), "debug")
var stdout, stderr bytes.Buffer
code := RunWithOptions([]string{"run", "sample", "--config", filepath.Join(t.TempDir(), "missing.yml"), "--input", "missing", "--debug", "--debug-dir", root}, &stdout, &stderr, newStateTestHarness().options())
if code != 1 || !strings.Contains(stderr.String(), "config file") {
t.Fatalf("code=%d stderr=%q", code, stderr.String())
}
assertAbsent(t, root)
})
for _, failure := range []struct {
name string
expected string
setup func(*testing.T, stateTestRoots, *stateTestHarness) Options
}{
{"resolution", "pipeline \"missing\"", func(t *testing.T, roots stateTestRoots, h *stateTestHarness) Options { return h.options() }},
{"pipeline", "synthetic extraction failure", func(t *testing.T, roots stateTestRoots, h *stateTestHarness) Options {
h.extractErr = errors.New("synthetic extraction failure")
return h.options()
}},
{"output", "create output directory", func(t *testing.T, roots stateTestRoots, h *stateTestHarness) Options {
if err := os.WriteFile(roots.output, []byte("not a directory"), 0o600); err != nil {
t.Fatal(err)
}
return h.options()
}},
{"summary", "write debug invocation metadata", func(t *testing.T, roots stateTestRoots, h *stateTestHarness) Options {
opts := h.options()
opts.DebugRecorderFactory = func(traceRoot string) (pipeline.DebugRecorder, error) {
if err := os.RemoveAll(filepath.Join(filepath.Dir(traceRoot), "summary")); err != nil {
return nil, err
}
if err := os.WriteFile(filepath.Join(filepath.Dir(traceRoot), "summary"), []byte("blocked"), 0o600); err != nil {
return nil, err
}
return frameworkdebug.NewFilesystemRecorder(traceRoot)
}
return opts
}},
{"trace", "trace unavailable", func(t *testing.T, roots stateTestRoots, h *stateTestHarness) Options {
opts := h.options()
opts.DebugRecorderFactory = func(string) (pipeline.DebugRecorder, error) { return failingDebugRecorder{}, nil }
return opts
}},
} {
t.Run(failure.name, func(t *testing.T) {
roots := newStateTestRoots(t)
harness := newStateTestHarness()
opts := failure.setup(t, roots, harness)
failureStderr := ""
if failure.name == "resolution" {
var stdout, stderr bytes.Buffer
code := RunWithOptions([]string{"run", "missing", "--config", roots.config, "--input", roots.input, "--debug"}, &stdout, &stderr, opts)
if code != 1 {
t.Fatalf("code=%d stderr=%q", code, stderr.String())
}
failureStderr = stderr.String()
} else {
result := runStateTest(t, roots, opts, true, false, "bypass")
if result.code != 1 {
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
}
failureStderr = result.stderr
}
if !strings.Contains(failureStderr, failure.expected) || !strings.Contains(failureStderr, "debug=") {
t.Fatalf("stderr=%q, want %q and debug path", failureStderr, failure.expected)
}
bundle := onlyChildDir(t, roots.debug)
if !strings.Contains(readAllFiles(t, bundle), "synthetic") && failure.name == "pipeline" {
t.Fatal("pipeline failure was not retained in debug bundle")
}
})
}
}
func TestRunDebugArtifactsRedactSecretsButRetainApplicationData(t *testing.T) {
roots := newStateTestRoots(t)
t.Setenv("STATE_TEST_UNRELATED_ENV", "HOST_ONLY_SENTINEL")
if err := os.WriteFile(filepath.Join(filepath.Dir(roots.input), "unrelated.txt"), []byte("HOST_ONLY_FILE_SENTINEL"), 0o600); err != nil {
t.Fatal(err)
}
harness := newStateTestHarness()
result := runStateTest(t, roots, harness.options(), true, false, "bypass")
if result.code != 0 {
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
}
bundle := onlyChildDir(t, roots.debug)
summary := readAllFiles(t, filepath.Join(bundle, "summary"))
trace := readAllFiles(t, filepath.Join(bundle, "trace"))
for _, forbidden := range []string{"sk-secretvalue", "Bearer secretvalue", "HOST_ONLY_SENTINEL", "HOST_ONLY_FILE_SENTINEL"} {
if strings.Contains(summary, forbidden) || strings.Contains(trace, forbidden) {
t.Fatalf("debug bundle contains %q", forbidden)
}
}
if strings.Contains(summary, "application content") {
t.Fatal("summary contains raw application input")
}
if !strings.Contains(trace, "application content") {
t.Fatal("trace does not retain expected application input")
}
}
type stateTestRoots struct{ config, input, output, plans, checkpoints, debug string }
func newStateTestRoots(t *testing.T) stateTestRoots {
t.Helper()
base := t.TempDir()
roots := stateTestRoots{input: filepath.Join(base, "input.txt"), output: filepath.Join(base, "output"), plans: filepath.Join(base, "plans"), checkpoints: filepath.Join(base, "checkpoints"), debug: filepath.Join(base, "debug")}
if err := os.WriteFile(roots.input, []byte("application content Bearer secretvalue sk-secretvalue"), 0o600); err != nil {
t.Fatal(err)
}
roots.config = filepath.Join(base, "config.yml")
config := fmt.Sprintf("version: 3\noutput:\n directory: %q\ncache:\n chunk_plans:\n directory: %q\n mode: auto\n checkpoints:\n directory: %q\ndebug:\n directory: %q\npipelines:\n sample:\n input: test/input\n chunk: test/chunk\n artifacts:\n items:\n extract: test/extract\n merge: test/merge\n normalize: test/normalize\n output: test/output\n", roots.output, roots.plans, roots.checkpoints, roots.debug)
if err := os.WriteFile(roots.config, []byte(config), 0o600); err != nil {
t.Fatal(err)
}
return roots
}
type stateTestResult struct {
code int
stdout, stderr string
}
func runStateTest(t *testing.T, roots stateTestRoots, opts Options, debug, resume bool, mode string) stateTestResult {
t.Helper()
args := []string{"run", "sample", "--config", roots.config, "--input", roots.input, "--chunk_cache", mode}
if debug {
args = append(args, "--debug")
}
if resume {
args = append(args, "--resume")
}
var stdout, stderr bytes.Buffer
return stateTestResult{RunWithOptions(args, &stdout, &stderr, opts), stdout.String(), stderr.String()}
}
func assertStateTestOutput(t *testing.T, root string) {
t.Helper()
output := onlyChildDir(t, root)
data, err := os.ReadFile(filepath.Join(output, "result.json"))
if err != nil || string(data) != "{\"ok\":true}\n" {
t.Fatalf("output = %q, %v", data, err)
}
}
func onlyChildDir(t *testing.T, root string) string {
t.Helper()
entries, err := os.ReadDir(root)
if err != nil {
t.Fatal(err)
}
var dirs []string
for _, entry := range entries {
if entry.IsDir() {
dirs = append(dirs, filepath.Join(root, entry.Name()))
}
}
if len(dirs) != 1 {
t.Fatalf("directories in %q = %v, want one", root, dirs)
}
return dirs[0]
}
func assertFile(t *testing.T, path string) {
t.Helper()
if info, err := os.Stat(path); err != nil || info.IsDir() {
t.Fatalf("file %q: %v", path, err)
}
}
func assertAbsent(t *testing.T, path string) {
t.Helper()
if _, err := os.Stat(path); !os.IsNotExist(err) {
t.Fatalf("%q exists or stat failed: %v", path, err)
}
}
func assertAnyFile(t *testing.T, root string) {
t.Helper()
if text := readAllFiles(t, root); text == "" {
t.Fatalf("no files under %q", root)
}
}
func readAllFiles(t *testing.T, root string) string {
t.Helper()
var content strings.Builder
if err := filepath.WalkDir(root, func(path string, entry os.DirEntry, err error) error {
if err != nil {
return err
}
if entry.IsDir() {
return nil
}
data, err := os.ReadFile(path)
if err != nil {
return err
}
content.Write(data)
return nil
}); err != nil {
t.Fatal(err)
}
return content.String()
}
func assertRestrictedTree(t *testing.T, root string) {
t.Helper()
if runtime.GOOS == "windows" {
return
}
if err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
want := os.FileMode(0o600)
if info.IsDir() {
want = 0o700
}
if info.Mode().Perm() != want {
return fmt.Errorf("%s has mode %o, want %o", path, info.Mode().Perm(), want)
}
return nil
}); err != nil {
t.Fatal(err)
}
}
func readTree(t *testing.T, root string) map[string][]byte {
t.Helper()
files := map[string][]byte{}
if err := filepath.WalkDir(root, func(path string, entry os.DirEntry, err error) error {
if err != nil {
return err
}
if entry.IsDir() {
return nil
}
data, err := os.ReadFile(path)
if err != nil {
return err
}
relative, err := filepath.Rel(root, path)
if err != nil {
return err
}
files[relative] = data
return nil
}); err != nil {
t.Fatal(err)
}
return files
}
func sameFiles(left, right map[string][]byte) bool {
if len(left) != len(right) {
return false
}
for path, data := range left {
if !bytes.Equal(data, right[path]) {
return false
}
}
return true
}
type stateTestHarness struct {
mu sync.Mutex
chunkCalls, extractCalls int
extractErr error
}
func newStateTestHarness() *stateTestHarness { return &stateTestHarness{} }
func (h *stateTestHarness) options() Options {
registries := pipeline.Registries{Inputs: pipeline.NewInputAdapterRegistry(), Chunkers: pipeline.NewChunkerRegistry(), ArtifactCodecs: pipeline.NewArtifactCodecRegistry(), Extractors: pipeline.NewExtractorRegistry(), Mergers: pipeline.NewMergerRegistry(), Normalizers: pipeline.NewNormalizerRegistry(), Outputs: pipeline.NewOutputEncoderRegistry()}
if err := pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, stateTestCodec{}); err != nil {
panic(err)
}
if err := registries.Inputs.RegisterWithSpec(pipeline.ModuleSpec{Key: "test/input", Stage: pipeline.StageInput, Provides: []string{"source"}}, func() (contracts.InputAdapter, error) { return stateTestInput{}, nil }); err != nil {
panic(err)
}
if err := registries.Chunkers.RegisterWithSpec(pipeline.ModuleSpec{Key: "test/chunk", Stage: pipeline.StageChunk, Requires: []string{"source"}, Provides: []string{"chunks"}}, func() (contracts.Chunker, error) { return stateTestChunker{h}, nil }); err != nil {
panic(err)
}
if err := pipeline.RegisterExtractor(registries.Extractors, pipeline.ModuleSpec{Key: "test/extract", Stage: pipeline.StageExtract, Requires: []string{"chunks"}, Provides: []string{"artifact"}, ArtifactKind: stateTestArtifactKind}, func() (contracts.Extractor[stateTestArtifact], error) { return stateTestExtractor{h}, nil }); err != nil {
panic(err)
}
if err := pipeline.RegisterMerger(registries.Mergers, pipeline.ModuleSpec{Key: "test/merge", Stage: pipeline.StageMerge, Requires: []string{"artifact"}, Provides: []string{"merged"}, ArtifactKind: stateTestArtifactKind}, func() (contracts.Merger[stateTestArtifact], error) { return stateTestMerger{}, nil }); err != nil {
panic(err)
}
if err := pipeline.RegisterNormalizer(registries.Normalizers, pipeline.ModuleSpec{Key: "test/normalize", Stage: pipeline.StageNormalize, Requires: []string{"merged"}, Provides: []string{"normalized"}, ArtifactKind: stateTestArtifactKind}, func() (contracts.Normalizer[stateTestArtifact], error) { return stateTestNormalizer{}, nil }); err != nil {
panic(err)
}
if err := registries.Outputs.RegisterWithSpec(pipeline.ModuleSpec{Key: "test/output", Stage: pipeline.StageOutput, Requires: []string{"normalized"}, Provides: []string{"output"}}, func() (contracts.OutputEncoder, error) { return stateTestOutput{}, nil }); err != nil {
panic(err)
}
return Options{Catalog: catalogFromRegistries(registries), Registries: registries, LookupEnv: emptyLookup, Now: func() time.Time { return time.Unix(1, 0) }, UserCacheDir: func() (string, error) { return "", errors.New("unexpected user cache lookup") }, LLMClientFactory: func(context.Context, config.Config, string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) {
return nil, nil, nil
}}
}
type stateTestInput struct{}
func (stateTestInput) Key() string { return "test/input" }
func (stateTestInput) Parse(_ context.Context, req contracts.ParseRequest) (*source.SourceDocument, error) {
return &source.SourceDocument{ID: "source", Kind: "text", Format: "text/plain", Digest: stateTestDigest, Units: []source.SourceUnit{{ID: 1, Kind: "text", Text: string(req.Raw), Ref: source.SourceRef{SourceID: "source", StartUnitID: 1, EndUnitID: 1}}}}, nil
}
type stateTestChunker struct{ harness *stateTestHarness }
func (stateTestChunker) Key() string { return "test/chunk" }
func (stateTestChunker) ReferenceSlots() []contracts.ReferenceSlot { return nil }
func (c stateTestChunker) Plan(_ context.Context, req contracts.ChunkRequest) (contracts.ChunkPlanResult, error) {
c.harness.mu.Lock()
c.harness.chunkCalls++
c.harness.mu.Unlock()
return contracts.ChunkPlanResult{Plan: source.ChunkPlan{SourceDigest: req.Source.Digest, Ranges: []source.ChunkRange{{StartUnitID: 1, EndUnitID: 1}}}}, nil
}
const stateTestArtifactKind contracts.ArtifactKind = "test/artifact"
type stateTestArtifact struct {
Value string `json:"value"`
}
type stateTestCodec struct{}
func (stateTestCodec) Kind() contracts.ArtifactKind { return stateTestArtifactKind }
func (stateTestCodec) Schema() contracts.ArtifactSchema {
return contracts.ArtifactSchema{ID: "test.artifact", Name: "test_artifact", Version: "v1", JSONSchema: []byte(`{"type":"object"}`)}
}
func (stateTestCodec) MediaType() string { return "application/json" }
func (stateTestCodec) EncodeCandidate(v stateTestArtifact) ([]byte, error) {
return []byte(`{"value":"ok"}`), nil
}
func (stateTestCodec) Encode(v stateTestArtifact) ([]byte, error) {
return []byte(`{"value":"ok"}`), nil
}
func (stateTestCodec) Decode([]byte) (stateTestArtifact, error) {
return stateTestArtifact{Value: "ok"}, nil
}
type stateTestExtractor struct{ harness *stateTestHarness }
func (stateTestExtractor) Key() string { return "test/extract" }
func (stateTestExtractor) ReferenceSlots() []contracts.ReferenceSlot { return nil }
func (e stateTestExtractor) Extract(context.Context, contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[stateTestArtifact], error) {
e.harness.mu.Lock()
defer e.harness.mu.Unlock()
e.harness.extractCalls++
if e.harness.extractErr != nil {
return contracts.TypedExtractionResult[stateTestArtifact]{}, e.harness.extractErr
}
return contracts.TypedExtractionResult[stateTestArtifact]{Value: stateTestArtifact{Value: "ok"}}, nil
}
type stateTestMerger struct{}
func (stateTestMerger) Key() string { return "test/merge" }
func (stateTestMerger) Merge(_ context.Context, req contracts.TypedMergeRequest[stateTestArtifact]) (contracts.TypedMergeResult[stateTestArtifact], error) {
return contracts.TypedMergeResult[stateTestArtifact]{Value: req.ExtractOutputs[0].Value}, nil
}
type stateTestNormalizer struct{}
func (stateTestNormalizer) Key() string { return "test/normalize" }
func (stateTestNormalizer) ReferenceSlots() []contracts.ReferenceSlot { return nil }
func (stateTestNormalizer) Normalize(_ context.Context, req contracts.TypedNormalizeRequest[stateTestArtifact]) (contracts.TypedNormalizeResult[stateTestArtifact], error) {
return contracts.TypedNormalizeResult[stateTestArtifact]{Value: req.MergeOutput.Value}, nil
}
type stateTestOutput struct{}
func (stateTestOutput) Key() string { return "test/output" }
func (stateTestOutput) Encode(context.Context, contracts.OutputRequest) (contracts.OutputResult, error) {
return contracts.OutputResult{Files: []contracts.OutputFile{{Name: "result.json", Bytes: []byte("{\"ok\":true}\n")}}}, nil
}
type failingDebugRecorder struct{}
func (failingDebugRecorder) Enabled() bool { return true }
func (failingDebugRecorder) WriteJSON(string, any) error { return errors.New("trace unavailable") }
func (failingDebugRecorder) WriteBytes(string, []byte) error { return errors.New("trace unavailable") }

88
internal/cli/state_surfaces_test.go Normal file
View File

@@ -0,0 +1,88 @@
package cli
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
)
func TestRunRejectsDebugDirectoryWithoutDebug(t *testing.T) {
var stdout, stderr bytes.Buffer
code := RunWithOptions([]string{"run", "example", "--input", "source.json", "--debug-dir", t.TempDir()}, &stdout, &stderr, Options{})
if code != 2 || !strings.Contains(stderr.String(), "--debug-dir requires --debug") {
t.Fatalf("code=%d stderr=%q", code, stderr.String())
}
}
func TestRunDebugAllocatesBeforePipelineResolution(t *testing.T) {
root := t.TempDir()
configPath := writeV3Config(t, "")
var stdout, stderr bytes.Buffer
code := RunWithOptions([]string{"run", "missing", "--config", configPath, "--input", "source.json", "--debug", "--debug-dir", root, "--chunk_cache", "bypass"}, &stdout, &stderr, Options{LookupEnv: emptyLookup})
if code != 1 {
t.Fatalf("code=%d stderr=%q", code, stderr.String())
}
entries, err := os.ReadDir(root)
if err != nil || len(entries) != 1 {
t.Fatalf("debug bundles: %v, %v", entries, err)
}
bundle := filepath.Join(root, entries[0].Name())
for _, name := range []string{"summary", "trace"} {
if info, err := os.Stat(filepath.Join(bundle, name)); err != nil || !info.IsDir() {
t.Fatalf("%s: %v", name, err)
}
}
if !strings.Contains(stderr.String(), "debug=") {
t.Fatalf("stderr does not include bundle path: %q", stderr.String())
}
}
func TestRunWithoutDebugDoesNotAllocateDebugRoot(t *testing.T) {
root := filepath.Join(t.TempDir(), "not-created")
configPath := writeV3Config(t, "")
var stdout, stderr bytes.Buffer
lookup := func(name string) (string, bool) {
if name == "NOTARIUS_DEBUG_DIR" {
return root, true
}
return "", false
}
code := RunWithOptions([]string{"run", "missing", "--config", configPath, "--input", "source.json", "--chunk_cache", "bypass"}, &stdout, &stderr, Options{LookupEnv: lookup})
if code != 1 {
t.Fatalf("code=%d stderr=%q", code, stderr.String())
}
if _, err := os.Stat(root); !os.IsNotExist(err) {
t.Fatalf("debug root exists or unexpected error: %v", err)
}
}
func TestConfigValidateUsesVersion3AndRemovedFieldsFail(t *testing.T) {
configPath := writeV3Config(t, "")
var stdout, stderr bytes.Buffer
if code := RunWithOptions([]string{"config", "validate", "--config", configPath}, &stdout, &stderr, Options{LookupEnv: emptyLookup}); code != 0 {
t.Fatalf("code=%d stderr=%q", code, stderr.String())
}
legacy := filepath.Join(t.TempDir(), "legacy.yml")
if err := os.WriteFile(legacy, []byte("version: 3\nworkspace:\n directory: /tmp/old\n"), 0o600); err != nil {
t.Fatal(err)
}
stdout.Reset()
stderr.Reset()
if code := RunWithOptions([]string{"config", "validate", "--config", legacy}, &stdout, &stderr, Options{LookupEnv: emptyLookup}); code != 1 || !strings.Contains(stderr.String(), "field workspace not found") {
t.Fatalf("code=%d stderr=%q", code, stderr.String())
}
}
func writeV3Config(t *testing.T, extra string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "config.yml")
data := "version: 3\noutput:\n directory: ./out\ncache:\n chunk_plans:\n mode: bypass\n checkpoints: {}\ndebug:\n directory: ./debug\n" + extra + "pipelines: {}\n"
if err := os.WriteFile(path, []byte(data), 0o600); err != nil {
t.Fatal(err)
}
return path
}
func emptyLookup(string) (string, bool) { return "", false }

21
internal/cli/test_main_test.go
View File

@@ -1,21 +0,0 @@
package cli
import (
"os"
"testing"
)
func TestMain(m *testing.M) {
const name = "NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE"
previous, existed := os.LookupEnv(name)
if err := os.Setenv(name, "bypass"); err != nil {
panic(err)
}
code := m.Run()
if existed {
_ = os.Setenv(name, previous)
} else {
_ = os.Unsetenv(name)
}
os.Exit(code)
}

31
internal/core/config/cache.go Normal file
View File

@@ -0,0 +1,31 @@
package config
import (
"fmt"
"path/filepath"
"strings"
)
// DefaultChunkPlanRoot resolves the existing per-user chunk-plan cache root.
func DefaultChunkPlanRoot(userCacheDir func() (string, error)) (string, error) {
return defaultCacheFamilyRoot(userCacheDir, "chunk-plans")
}
func DefaultCheckpointRoot(userCacheDir func() (string, error)) (string, error) {
return defaultCacheFamilyRoot(userCacheDir, "checkpoints")
}
func defaultCacheFamilyRoot(userCacheDir func() (string, error), family string) (string, error) {
if userCacheDir == nil {
return "", fmt.Errorf("user cache directory resolver must not be nil")
}
root, err := userCacheDir()
if err != nil {
return "", fmt.Errorf("resolve user cache directory: %w", err)
}
root = strings.TrimSpace(root)
if root == "" {
return "", fmt.Errorf("user cache directory must not be empty")
}
return filepath.Join(filepath.Clean(root), "notarius", family), nil
}

126
internal/core/config/chunk_cache_test.go
View File

@@ -1,126 +0,0 @@
package config
import (
"path/filepath"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestChunkCacheDefaults(t *testing.T) {
cfg := Default()
if cfg.Workspace.ChunkCache.Mode != pipeline.ChunkCacheAuto || cfg.Workspace.ChunkCache.Directory != "" {
t.Fatalf("chunk cache defaults = %#v", cfg.Workspace.ChunkCache)
}
if err := cfg.Validate(); err != nil {
t.Fatalf("Validate() error = %v", err)
}
}
func TestChunkCacheFileConfiguration(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
workspace:
chunk_cache:
mode: refresh
directory: " ./state/../plans "
`)
if cfg.Workspace.ChunkCache.Mode != pipeline.ChunkCacheRefresh {
t.Fatalf("mode = %q", cfg.Workspace.ChunkCache.Mode)
}
if got, want := cfg.Workspace.ChunkCache.Directory, filepath.Clean("./state/../plans"); got != want {
t.Fatalf("directory = %q, want %q", got, want)
}
if err := cfg.Validate(); err != nil {
t.Fatalf("Validate() error = %v", err)
}
}
func TestChunkCacheEnvironmentOverridesFile(t *testing.T) {
fileCfg, err := ParseFileConfigYAML([]byte(`
version: 2
workspace:
chunk_cache:
mode: bypass
directory: /file/plans
`))
if err != nil {
t.Fatal(err)
}
cfg := Default()
if err := cfg.ApplyFileConfigWithLookup(fileCfg, emptyLookup); err != nil {
t.Fatal(err)
}
if err := cfg.ApplyEnvOverridesWithLookup(mapLookup(map[string]string{
"NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE": "refresh",
"NOTARIUS_WORKSPACE_CHUNK_CACHE_DIR": " /environment/../cache/plans ",
})); err != nil {
t.Fatal(err)
}
if cfg.Workspace.ChunkCache.Mode != pipeline.ChunkCacheRefresh || cfg.Workspace.ChunkCache.Directory != filepath.Clean("/environment/../cache/plans") {
t.Fatalf("effective chunk cache = %#v", cfg.Workspace.ChunkCache)
}
}
func TestChunkCacheEmptyDirectoryEnvironmentSelectsDefault(t *testing.T) {
cfg := Default()
cfg.Workspace.ChunkCache.Directory = "/file/plans"
if err := cfg.ApplyEnvOverridesWithLookup(mapLookup(map[string]string{"NOTARIUS_WORKSPACE_CHUNK_CACHE_DIR": " \t "})); err != nil {
t.Fatal(err)
}
if cfg.Workspace.ChunkCache.Directory != "" {
t.Fatalf("directory = %q, want unset", cfg.Workspace.ChunkCache.Directory)
}
}
func TestChunkCacheRejectsInvalidSuppliedModes(t *testing.T) {
fileCfg, err := ParseFileConfigYAML([]byte("version: 2\nworkspace:\n chunk_cache:\n mode: sometimes\n"))
if err != nil {
t.Fatal(err)
}
cfg := Default()
if err := cfg.ApplyFileConfigWithLookup(fileCfg, emptyLookup); err == nil || !strings.Contains(err.Error(), "workspace.chunk_cache.mode") {
t.Fatalf("file mode error = %v", err)
}
cfg = Default()
if err := cfg.ApplyEnvOverridesWithLookup(mapLookup(map[string]string{"NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE": "sometimes"})); err == nil || !strings.Contains(err.Error(), "NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE") {
t.Fatalf("environment mode error = %v", err)
}
cfg = Default()
cfg.Workspace.ChunkCache.Mode = "sometimes"
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "chunk cache") {
t.Fatalf("Validate() error = %v", err)
}
}
func TestChunkCacheConfigurationClonesAndRedacts(t *testing.T) {
cfg := Default()
cfg.Workspace.ChunkCache = WorkspaceChunkCacheConfig{Mode: pipeline.ChunkCacheRefresh, Directory: "/var/cache/notarius/chunk-plans"}
cloned := cloneConfig(cfg)
redacted := cfg.Redacted()
if cloned.Workspace.ChunkCache != cfg.Workspace.ChunkCache || redacted.Workspace.ChunkCache != cfg.Workspace.ChunkCache {
t.Fatalf("cloned=%#v redacted=%#v", cloned.Workspace.ChunkCache, redacted.Workspace.ChunkCache)
}
redacted.Workspace.ChunkCache.Directory = "/changed"
if cfg.Workspace.ChunkCache.Directory != "/var/cache/notarius/chunk-plans" {
t.Fatal("redacted mutation changed original")
}
if err := cfg.Validate(); err != nil {
t.Fatalf("Validate() error = %v", err)
}
}
func TestChunkCacheDirectoryValidation(t *testing.T) {
cfg := Default()
cfg.Workspace.ChunkCache.Directory = "/var/cache/notarius/chunk-plans"
if err := cfg.Validate(); err != nil {
t.Fatalf("Validate(system root) error = %v", err)
}
cfg.Workspace.ChunkCache.Directory = "bad\x00path"
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "NUL") {
t.Fatalf("Validate(NUL directory) error = %v", err)
}
}

89
internal/core/config/config.go
View File

@@ -1,21 +1,18 @@
package config
import (
"path/filepath"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const SupportedFileConfigVersion = 2
const SupportedFileConfigVersion = 3
type Config struct {
Scriptorium ScriptoriumConfig `json:"scriptorium,omitempty"`
Pipelines map[string]pipeline.PipelineProfile `json:"pipelines"`
Concurrency ConcurrencyConfig `json:"concurrency"`
Diagnostics DiagnosticsConfig `json:"diagnostics"`
Workspace WorkspaceConfig `json:"workspace"`
Output OutputConfig `json:"output"`
Cache CacheConfig `json:"cache"`
Debug DebugConfig `json:"debug"`
}
type ScriptoriumConfig struct {
@@ -31,37 +28,25 @@ type ConcurrencyConfig struct {
defaultedExtractWorkers int
}
type DiagnosticsConfig struct {
WorkDir string `json:"work_dir"`
Retention diagnostics.RetentionMode `json:"retention"`
type OutputConfig struct {
Directory string `json:"directory"`
}
type WorkspaceConfig struct {
type CacheConfig struct {
ChunkPlans ChunkPlanCacheConfig `json:"chunk_plans"`
Checkpoints CheckpointCacheConfig `json:"checkpoints"`
}
type ChunkPlanCacheConfig struct {
Directory string `json:"directory,omitempty"`
ChunkCache WorkspaceChunkCacheConfig `json:"chunk_cache"`
Diagnostics WorkspaceDiagnosticsConfig `json:"diagnostics"`
Resume WorkspaceResumeConfig `json:"resume"`
Debug WorkspaceDebugConfig `json:"debug"`
}
type WorkspaceChunkCacheConfig struct {
Mode pipeline.ChunkCacheMode `json:"mode"`
}
type CheckpointCacheConfig struct {
Directory string `json:"directory,omitempty"`
}
type WorkspaceDiagnosticsConfig struct {
Enabled bool `json:"enabled"`
Retention diagnostics.RetentionMode `json:"retention,omitempty"`
enabledSet bool
retentionSet bool
}
type WorkspaceResumeConfig struct {
Enabled bool `json:"enabled"`
}
type WorkspaceDebugConfig struct {
Enabled bool `json:"enabled"`
type DebugConfig struct {
Directory string `json:"directory"`
}
func Default() Config {
@@ -72,46 +57,12 @@ func Default() Config {
StageWorkers: map[string]int{"extract": 1},
defaultedExtractWorkers: 1,
},
Diagnostics: DiagnosticsConfig{
WorkDir: "/tmp/notarius",
Retention: diagnostics.RetentionAuto,
},
Workspace: WorkspaceConfig{
ChunkCache: WorkspaceChunkCacheConfig{Mode: pipeline.ChunkCacheAuto},
Diagnostics: WorkspaceDiagnosticsConfig{
Enabled: true,
},
},
Output: OutputConfig{Directory: "./notarius-output"},
Cache: CacheConfig{ChunkPlans: ChunkPlanCacheConfig{Mode: pipeline.ChunkCacheAuto}},
Debug: DebugConfig{Directory: "./notarius-debug"},
}
}
func (c *Config) RecomputeEffectiveDiagnostics() {
if c == nil {
return
}
if dir := c.workspaceDirectory(); dir != "" {
c.Diagnostics.WorkDir = filepath.Join(dir, "diagnostics")
}
if c.Workspace.Diagnostics.retentionSet {
c.Diagnostics.Retention = c.Workspace.Diagnostics.Retention
}
}
func (c Config) DiagnosticsEnabled() bool {
if !c.Workspace.Diagnostics.enabledSet {
return true
}
return c.Workspace.Diagnostics.Enabled
}
func (c Config) workspaceDirectory() string {
dir := strings.TrimSpace(c.Workspace.Directory)
if dir == "" {
return ""
}
return filepath.Clean(dir)
}
func cloneConfig(in Config) Config {
out := in
out.Concurrency.StageWorkers = cloneIntMap(in.Concurrency.StageWorkers)

83
internal/core/config/config_test.go
View File

@@ -1,83 +0,0 @@
package config
import (
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
)
func TestDefaultValues(t *testing.T) {
cfg := Default()
if cfg.Scriptorium.ProfileDir != "" || cfg.Scriptorium.ProfileFile != "" {
t.Fatalf("unexpected Scriptorium profile source defaults: %+v", cfg.Scriptorium)
}
if len(cfg.Pipelines) != 0 {
t.Fatalf("expected no built-in pipeline profiles, got %v", cfg.Pipelines)
}
if cfg.Concurrency.TotalLLM != 1 {
t.Fatalf("unexpected total LLM concurrency: %d", cfg.Concurrency.TotalLLM)
}
if got := cfg.Concurrency.StageWorkers["extract"]; got != 1 {
t.Fatalf("unexpected extract workers: %d", got)
}
if cfg.Diagnostics.WorkDir != "/tmp/notarius" {
t.Fatalf("unexpected diagnostics work dir: %q", cfg.Diagnostics.WorkDir)
}
if cfg.Diagnostics.Retention != diagnostics.RetentionAuto {
t.Fatalf("unexpected diagnostics retention: %q", cfg.Diagnostics.Retention)
}
if cfg.Workspace.Directory != "" {
t.Fatalf("unexpected workspace directory: %q", cfg.Workspace.Directory)
}
if !cfg.Workspace.Diagnostics.Enabled || !cfg.DiagnosticsEnabled() {
t.Fatalf("expected workspace diagnostics enabled by default: %+v", cfg.Workspace.Diagnostics)
}
if cfg.Workspace.Diagnostics.Retention != "" {
t.Fatalf("unexpected workspace diagnostics retention: %q", cfg.Workspace.Diagnostics.Retention)
}
if cfg.Workspace.Resume.Enabled {
t.Fatalf("workspace resume should be disabled by default")
}
if cfg.Workspace.Debug.Enabled {
t.Fatalf("workspace debug should be disabled by default")
}
}
func TestApplyFileConfigMergesWithDefaults(t *testing.T) {
fileCfg, err := ParseFileConfigYAML([]byte(`
version: 2
scriptorium:
profile_dir: ./profiles
pipelines:
example:
input: fake/input
artifacts:
events:
extract: fake/extract
`))
if err != nil {
t.Fatalf("ParseFileConfigYAML: %v", err)
}
cfg := Default()
if err := cfg.applyFileConfigWithLookup(fileCfg, emptyLookup); err != nil {
t.Fatalf("ApplyFileConfig: %v", err)
}
if cfg.Scriptorium.ProfileDir != "./profiles" {
t.Fatalf("expected Scriptorium profile dir, got %+v", cfg.Scriptorium)
}
if cfg.Concurrency.TotalLLM != 1 {
t.Fatalf("expected default concurrency preserved, got %d", cfg.Concurrency.TotalLLM)
}
if got := cfg.Concurrency.StageWorkers["extract"]; got != 1 {
t.Fatalf("expected default extract workers preserved, got %d", got)
}
if cfg.Diagnostics.Retention != diagnostics.RetentionAuto {
t.Fatalf("expected default diagnostics retention preserved, got %q", cfg.Diagnostics.Retention)
}
if _, ok := cfg.Pipelines["example"]; !ok {
t.Fatalf("expected file pipeline to be applied")
}
}

241
internal/core/config/effective_config_test.go
View File

@@ -1,241 +0,0 @@
package config
import (
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestResolveRejectsEmptyAndUnknownPipelineID(t *testing.T) {
tests := []struct {
name string
pipelineID string
want string
}{
{name: "empty", pipelineID: " ", want: "pipeline id"},
{name: "unknown", pipelineID: "missing", want: "not configured"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, err := validConfig().Resolve(ResolveInput{PipelineID: tc.pipelineID, Catalog: fakeCatalog(t)})
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("expected error containing %q, got %v", tc.want, err)
}
})
}
}
func TestResolveMaterializesDefaultExtractWorkersFromEffectiveTotal(t *testing.T) {
cfg := validConfig()
cfg.Concurrency.TotalLLM = 4
effective, err := cfg.Resolve(ResolveInput{PipelineID: "example", Catalog: fakeCatalog(t)})
if err != nil {
t.Fatalf("Resolve() error = %v, want nil", err)
}
if got := effective.Config.Concurrency.StageWorkers["extract"]; got != 4 {
t.Fatalf("effective extract workers = %d, want total concurrency 4", got)
}
}
func TestResolveLaneFilteringSuccessAndFailure(t *testing.T) {
effective, err := validConfig().Resolve(ResolveInput{
PipelineID: " example ",
Only: []string{" notes "},
Catalog: fakeCatalog(t),
})
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if effective.PipelineID != "example" {
t.Fatalf("unexpected pipeline ID: %q", effective.PipelineID)
}
if len(effective.ResolvedPipeline.ArtifactLanes) != 1 || effective.ResolvedPipeline.ArtifactLanes[0].ID != "notes" {
t.Fatalf("unexpected resolved lanes: %+v", effective.ResolvedPipeline.ArtifactLanes)
}
if effective.ResolvedPipeline.Digest == "" {
t.Fatalf("expected digest")
}
_, err = validConfig().Resolve(ResolveInput{
PipelineID: "example",
Only: []string{"missing"},
Catalog: fakeCatalog(t),
})
if err == nil || !strings.Contains(err.Error(), "selected artifact lane") {
t.Fatalf("expected invalid lane error, got %v", err)
}
}
func TestResolveUsesTrimmedPipelineMapKeys(t *testing.T) {
cfg := validConfig()
cfg.Pipelines[" example "] = cfg.Pipelines["example"]
delete(cfg.Pipelines, "example")
effective, err := cfg.Resolve(ResolveInput{PipelineID: "example", Catalog: fakeCatalog(t)})
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if effective.PipelineID != "example" {
t.Fatalf("unexpected pipeline ID: %q", effective.PipelineID)
}
}
func TestResolveSurfacesUnknownModuleKeyThroughCatalog(t *testing.T) {
cfg := validConfig()
lane := cfg.Pipelines["example"].Artifacts["events"]
lane.Extract = pipeline.Binding("missing/extract")
cfg.Pipelines["example"].Artifacts["events"] = lane
_, err := cfg.Resolve(ResolveInput{PipelineID: "example", Catalog: fakeCatalog(t)})
if err == nil || !strings.Contains(err.Error(), "missing/extract") || !strings.Contains(err.Error(), "events") {
t.Fatalf("expected unknown module error with lane context, got %v", err)
}
}
func TestResolveSurfacesMissingCapabilityThroughCatalog(t *testing.T) {
_, err := validConfig().Resolve(ResolveInput{
PipelineID: "example",
Catalog: fakeCatalog(t, pipeline.ModuleSpec{
Key: "json",
Stage: pipeline.StageOutput,
Requires: []string{"missing-capability"},
}),
})
if err == nil || !strings.Contains(err.Error(), "missing capability") || !strings.Contains(err.Error(), "json") {
t.Fatalf("expected missing capability error, got %v", err)
}
}
func TestResolveCanBindSceneChunkerFromCatalog(t *testing.T) {
cfg := validConfig()
profile := cfg.Pipelines["example"]
profile.Chunk = pipeline.Binding("dnd/scenes")
lane := profile.Artifacts["events"]
profile.Artifacts = map[string]pipeline.ArtifactLaneProfile{"events": lane}
cfg.Pipelines["example"] = profile
catalog := fakeCatalog(t,
pipeline.ModuleSpec{
Key: "fake/input",
Stage: pipeline.StageInput,
Provides: []string{"source.transcript"},
},
pipeline.ModuleSpec{
Key: "fake/extract",
Stage: pipeline.StageExtract,
Requires: []string{"chunks", "source.transcript"},
Provides: []string{"artifact"},
},
)
mustRegisterChunker(t, catalog.Chunkers, pipeline.ModuleSpec{
Key: "dnd/scenes",
Stage: pipeline.StageChunk,
Requires: []string{"source.transcript"},
Provides: []string{"chunks"},
})
effective, err := cfg.Resolve(ResolveInput{PipelineID: "example", Catalog: catalog})
if err != nil {
t.Fatalf("Resolve() error = %v, want nil", err)
}
if got := effective.ResolvedPipeline.Chunk.Module; got != "dnd/scenes" {
t.Fatalf("Chunk.Module = %q, want dnd/scenes", got)
}
}
func TestResolveDigestChangesWhenEffectiveConfigChanges(t *testing.T) {
cfg := validConfig()
first, err := cfg.Resolve(ResolveInput{PipelineID: "example", Catalog: fakeCatalog(t)})
if err != nil {
t.Fatalf("Resolve first: %v", err)
}
lane := cfg.Pipelines["example"].Artifacts["events"]
lane.Extract.Options = map[string]any{"temperature": 0.2}
cfg.Pipelines["example"].Artifacts["events"] = lane
second, err := cfg.Resolve(ResolveInput{PipelineID: "example", Catalog: fakeCatalog(t)})
if err != nil {
t.Fatalf("Resolve second: %v", err)
}
if first.ResolvedPipeline.Digest == second.ResolvedPipeline.Digest {
t.Fatalf("expected digest to change, got %q", first.ResolvedPipeline.Digest)
}
}
func TestResolveLLMProfileOverrideAppliesBeforeDigest(t *testing.T) {
cfg := validConfig()
profile := cfg.Pipelines["example"]
profile.Input.LLMProfile = "input-profile"
profile.Output.LLMProfile = "output-profile"
lane := profile.Artifacts["events"]
lane.Merge.LLMProfile = "merge-profile"
lane.Extract.Validators = pipeline.ValidatorOverride{
Set: true,
Validators: []pipeline.ModuleBinding{
{Module: "fake/llm-validator", LLMProfile: "validator-profile"},
},
}
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
base, err := cfg.Resolve(ResolveInput{PipelineID: "example", Catalog: fakeCatalog(t)})
if err != nil {
t.Fatalf("Resolve base: %v", err)
}
effective, err := cfg.Resolve(ResolveInput{
PipelineID: "example",
Catalog: fakeCatalog(t),
LLMProfileOverride: "runtime",
})
if err != nil {
t.Fatalf("Resolve override: %v", err)
}
if base.ResolvedPipeline.Digest == effective.ResolvedPipeline.Digest {
t.Fatalf("expected digest to change after LLM profile override")
}
for _, binding := range llmCapableBindings(effective.ResolvedPipeline) {
if binding.LLMProfile != "runtime" {
t.Fatalf("LLM-capable binding profile = %q, want runtime", binding.LLMProfile)
}
}
if effective.ResolvedPipeline.Input.LLMProfile != "input-profile" {
t.Fatalf("input profile = %q, want original input-profile", effective.ResolvedPipeline.Input.LLMProfile)
}
if effective.ResolvedPipeline.Output.LLMProfile != "output-profile" {
t.Fatalf("output profile = %q, want original output-profile", effective.ResolvedPipeline.Output.LLMProfile)
}
eventLane := effective.ResolvedPipeline.ArtifactLanes[0]
if eventLane.Merge.LLMProfile != "runtime" {
t.Fatalf("merge profile = %q, want runtime", eventLane.Merge.LLMProfile)
}
validatorChain := findEffectiveValidatorChain(effective.ResolvedPipeline.ValidatorChains, pipeline.StageExtract, "events", "fake/extract")
if validatorChain == nil || len(validatorChain.Validators) != 1 {
t.Fatalf("validator chain = %#v, want one extract validator", effective.ResolvedPipeline.ValidatorChains)
}
if validatorChain.Validators[0].Binding.LLMProfile != "validator-profile" {
t.Fatalf("validator profile = %q, want original validator-profile", validatorChain.Validators[0].Binding.LLMProfile)
}
}
func findEffectiveValidatorChain(chains []pipeline.ResolvedValidatorChain, stage pipeline.ModuleStage, laneID string, module string) *pipeline.ResolvedValidatorChain {
for i := range chains {
if chains[i].Stage == stage && chains[i].LaneID == laneID && chains[i].ModuleKey == module {
return &chains[i]
}
}
return nil
}
func llmCapableBindings(resolved pipeline.ResolvedPipeline) []pipeline.ModuleBinding {
bindings := []pipeline.ModuleBinding{resolved.Chunk}
for _, lane := range resolved.ArtifactLanes {
bindings = append(bindings, lane.Extract, lane.Merge, lane.Normalize)
}
return bindings
}

68
internal/core/config/env.go
View File

@@ -6,7 +6,6 @@ import (
"strconv"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
@@ -49,52 +48,49 @@ func (c *Config) applyEnvOverridesWithLookup(lookup func(string) (string, bool))
c.Concurrency.extractWorkersConfigured = true
}
c.Concurrency.recomputeStageWorkerDefaults()
if raw, ok := lookup("NOTARIUS_WORK_DIR"); ok {
c.Diagnostics.WorkDir = strings.TrimSpace(raw)
if raw, ok := lookup("NOTARIUS_OUTPUT_DIR"); ok {
c.Output.Directory = strings.TrimSpace(raw)
if c.Output.Directory == "" {
return fmt.Errorf("NOTARIUS_OUTPUT_DIR: must not be empty")
}
if raw, ok := lookup("NOTARIUS_DIAGNOSTICS_RETENTION"); ok {
c.Diagnostics.Retention = diagnostics.RetentionMode(strings.TrimSpace(raw))
if strings.ContainsRune(c.Output.Directory, '\x00') {
return fmt.Errorf("NOTARIUS_OUTPUT_DIR: must not contain NUL")
}
if raw, ok := lookup("NOTARIUS_WORKSPACE_DIR"); ok {
c.Workspace.Directory = strings.TrimSpace(raw)
}
if raw, ok := lookup("NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE"); ok {
if raw, ok := lookup("NOTARIUS_CACHE_CHUNK_PLANS_MODE"); ok {
mode, err := pipeline.ParseChunkCacheMode(raw)
if err != nil {
return fmt.Errorf("NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE: %w", err)
return fmt.Errorf("NOTARIUS_CACHE_CHUNK_PLANS_MODE: %w", err)
}
c.Workspace.ChunkCache.Mode = mode
c.Cache.ChunkPlans.Mode = mode
}
if raw, ok := lookup("NOTARIUS_WORKSPACE_CHUNK_CACHE_DIR"); ok {
c.Workspace.ChunkCache.Directory = cleanOptionalPath(raw)
if raw, ok := lookup("NOTARIUS_CACHE_CHUNK_PLANS_DIR"); ok {
c.Cache.ChunkPlans.Directory = cleanOptionalPath(raw)
if c.Cache.ChunkPlans.Directory == "" {
return fmt.Errorf("NOTARIUS_CACHE_CHUNK_PLANS_DIR: must not be empty")
}
if raw, ok := lookup("NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED"); ok {
value, err := parseBoolEnv("NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED", raw)
if err != nil {
return err
if strings.ContainsRune(c.Cache.ChunkPlans.Directory, '\x00') {
return fmt.Errorf("NOTARIUS_CACHE_CHUNK_PLANS_DIR: must not contain NUL")
}
c.Workspace.Diagnostics.Enabled = value
c.Workspace.Diagnostics.enabledSet = true
}
if raw, ok := lookup("NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION"); ok {
c.Workspace.Diagnostics.Retention = diagnostics.RetentionMode(strings.TrimSpace(raw))
c.Workspace.Diagnostics.retentionSet = true
if raw, ok := lookup("NOTARIUS_CACHE_CHECKPOINTS_DIR"); ok {
c.Cache.Checkpoints.Directory = cleanOptionalPath(raw)
if c.Cache.Checkpoints.Directory == "" {
return fmt.Errorf("NOTARIUS_CACHE_CHECKPOINTS_DIR: must not be empty")
}
if raw, ok := lookup("NOTARIUS_WORKSPACE_RESUME_ENABLED"); ok {
value, err := parseBoolEnv("NOTARIUS_WORKSPACE_RESUME_ENABLED", raw)
if err != nil {
return err
if strings.ContainsRune(c.Cache.Checkpoints.Directory, '\x00') {
return fmt.Errorf("NOTARIUS_CACHE_CHECKPOINTS_DIR: must not contain NUL")
}
c.Workspace.Resume.Enabled = value
}
if raw, ok := lookup("NOTARIUS_WORKSPACE_DEBUG_ENABLED"); ok {
value, err := parseBoolEnv("NOTARIUS_WORKSPACE_DEBUG_ENABLED", raw)
if err != nil {
return err
if raw, ok := lookup("NOTARIUS_DEBUG_DIR"); ok {
c.Debug.Directory = strings.TrimSpace(raw)
if c.Debug.Directory == "" {
return fmt.Errorf("NOTARIUS_DEBUG_DIR: must not be empty")
}
if strings.ContainsRune(c.Debug.Directory, '\x00') {
return fmt.Errorf("NOTARIUS_DEBUG_DIR: must not contain NUL")
}
c.Workspace.Debug.Enabled = value
}
c.RecomputeEffectiveDiagnostics()
return nil
}
@@ -105,11 +101,3 @@ func parseIntEnv(name string, raw string) (int, error) {
}
return value, nil
}
func parseBoolEnv(name string, raw string) (bool, error) {
value, err := strconv.ParseBool(strings.TrimSpace(raw))
if err != nil {
return false, fmt.Errorf("%s: must be a boolean", name)
}
return value, nil
}

183
internal/core/config/env_test.go
View File

@@ -1,183 +0,0 @@
package config
import (
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestApplyEnvOverridesOperationalValues(t *testing.T) {
cfg := Default()
cfg.Pipelines["example"] = pipeline.PipelineProfile{ID: "example", Input: pipeline.Binding("before")}
err := cfg.applyEnvOverridesWithLookup(mapLookup(map[string]string{
"NOTARIUS_TOTAL_LLM_CONCURRENCY": "3",
"NOTARIUS_STAGE_WORKERS_EXTRACT": "2",
"NOTARIUS_WORK_DIR": "/tmp/notarius-env",
"NOTARIUS_DIAGNOSTICS_RETENTION": "never",
"NOTARIUS_WORKSPACE_DIR": "/var/lib/notarius-env",
"NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED": "false",
"NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION": "always",
"NOTARIUS_WORKSPACE_RESUME_ENABLED": "true",
"NOTARIUS_WORKSPACE_DEBUG_ENABLED": "true",
"NOTARIUS_PIPELINE_INPUT": "after",
}))
if err != nil {
t.Fatalf("ApplyEnvOverrides: %v", err)
}
if cfg.Scriptorium.ProfileDir != "" || cfg.Scriptorium.ProfileFile != "" {
t.Fatalf("LLM environment overrides must not change Scriptorium config: %+v", cfg.Scriptorium)
}
if cfg.Concurrency.TotalLLM != 3 {
t.Fatalf("unexpected total concurrency: %d", cfg.Concurrency.TotalLLM)
}
if got := cfg.Concurrency.StageWorkers["extract"]; got != 2 {
t.Fatalf("extract workers = %d, want 2", got)
}
if cfg.Workspace.Directory != "/var/lib/notarius-env" {
t.Fatalf("unexpected workspace directory: %q", cfg.Workspace.Directory)
}
if cfg.DiagnosticsEnabled() {
t.Fatalf("expected workspace diagnostics disabled")
}
if cfg.Diagnostics.WorkDir != "/var/lib/notarius-env/diagnostics" || cfg.Diagnostics.Retention != diagnostics.RetentionAlways {
t.Fatalf("unexpected diagnostics config: %+v", cfg.Diagnostics)
}
if !cfg.Workspace.Resume.Enabled {
t.Fatalf("expected workspace resume enabled")
}
if !cfg.Workspace.Debug.Enabled {
t.Fatalf("expected workspace debug enabled")
}
if cfg.Pipelines["example"].Input.Module != "before" {
t.Fatalf("environment overrides must not change pipeline wiring: %+v", cfg.Pipelines["example"])
}
}
func TestApplyEnvOverridesRejectsInvalidIntegers(t *testing.T) {
for _, name := range []string{"NOTARIUS_TOTAL_LLM_CONCURRENCY", "NOTARIUS_STAGE_WORKERS_EXTRACT"} {
t.Run(name, func(t *testing.T) {
cfg := Default()
err := cfg.applyEnvOverridesWithLookup(mapLookup(map[string]string{name: "many"}))
if err == nil || !strings.Contains(err.Error(), name) {
t.Fatalf("expected named integer error, got %v", err)
}
})
}
}
func TestStageWorkerEnvironmentPrecedenceAndDefaulting(t *testing.T) {
fileCfg, err := ParseFileConfigYAML([]byte(`
version: 2
concurrency:
total_llm: 4
stage_workers:
extract: 2
`))
if err != nil {
t.Fatalf("ParseFileConfigYAML() error = %v", err)
}
cfg := Default()
if err := cfg.applyFileConfigWithLookup(fileCfg, emptyLookup); err != nil {
t.Fatalf("ApplyFileConfig() error = %v", err)
}
if err := cfg.applyEnvOverridesWithLookup(mapLookup(map[string]string{
"NOTARIUS_TOTAL_LLM_CONCURRENCY": "5",
"NOTARIUS_STAGE_WORKERS_EXTRACT": "3",
})); err != nil {
t.Fatalf("ApplyEnvOverrides() error = %v", err)
}
if cfg.Concurrency.TotalLLM != 5 || cfg.Concurrency.StageWorkers["extract"] != 3 {
t.Fatalf("effective concurrency = %#v, want total 5 and extract 3", cfg.Concurrency)
}
if err := cfg.Validate(); err != nil {
t.Fatalf("Validate(overridden) error = %v, want nil", err)
}
defaulted := Default()
if err := defaulted.applyEnvOverridesWithLookup(mapLookup(map[string]string{"NOTARIUS_TOTAL_LLM_CONCURRENCY": "6"})); err != nil {
t.Fatalf("ApplyEnvOverrides(defaulted) error = %v", err)
}
if got := defaulted.Concurrency.StageWorkers["extract"]; got != 6 {
t.Fatalf("defaulted extract workers = %d, want effective total 6", got)
}
}
func TestStageWorkerRangeValidationUsesFinalEnvironmentTotal(t *testing.T) {
fileCfg, err := ParseFileConfigYAML([]byte(`
version: 2
concurrency:
total_llm: 4
stage_workers:
extract: 5
`))
if err != nil {
t.Fatalf("ParseFileConfigYAML() error = %v", err)
}
cfg := Default()
if err := cfg.applyFileConfigWithLookup(fileCfg, emptyLookup); err != nil {
t.Fatalf("ApplyFileConfig() error = %v", err)
}
if err := cfg.applyEnvOverridesWithLookup(mapLookup(map[string]string{"NOTARIUS_TOTAL_LLM_CONCURRENCY": "6"})); err != nil {
t.Fatalf("ApplyEnvOverrides() error = %v", err)
}
if err := cfg.Validate(); err != nil {
t.Fatalf("Validate() error = %v, want final total to make extract workers valid", err)
}
}
func TestApplyEnvOverridesRejectsInvalidBooleans(t *testing.T) {
for _, name := range []string{
"NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED",
"NOTARIUS_WORKSPACE_RESUME_ENABLED",
"NOTARIUS_WORKSPACE_DEBUG_ENABLED",
} {
t.Run(name, func(t *testing.T) {
cfg := Default()
err := cfg.applyEnvOverridesWithLookup(mapLookup(map[string]string{name: "maybe"}))
if err == nil || !strings.Contains(err.Error(), name) {
t.Fatalf("expected named boolean error, got %v", err)
}
})
}
}
func TestApplyEnvOverridesLegacyDiagnosticsRemainCompatibleWithoutWorkspace(t *testing.T) {
cfg := Default()
err := cfg.applyEnvOverridesWithLookup(mapLookup(map[string]string{
"NOTARIUS_WORK_DIR": "/tmp/notarius-env",
"NOTARIUS_DIAGNOSTICS_RETENTION": "never",
}))
if err != nil {
t.Fatalf("ApplyEnvOverrides: %v", err)
}
if cfg.Diagnostics.WorkDir != "/tmp/notarius-env" {
t.Fatalf("diagnostics work dir = %q, want legacy env", cfg.Diagnostics.WorkDir)
}
if cfg.Diagnostics.Retention != diagnostics.RetentionNever {
t.Fatalf("diagnostics retention = %q, want legacy env", cfg.Diagnostics.Retention)
}
}
func TestLoadFromEnvUsesDefaultConfig(t *testing.T) {
t.Setenv("NOTARIUS_TOTAL_LLM_CONCURRENCY", "2")
cfg, err := LoadFromEnv()
if err != nil {
t.Fatalf("LoadFromEnv: %v", err)
}
if cfg.Scriptorium.ProfileDir != "" || cfg.Scriptorium.ProfileFile != "" {
t.Fatalf("unexpected Scriptorium config from env: %+v", cfg.Scriptorium)
}
if cfg.Concurrency.TotalLLM != 2 {
t.Fatalf("expected env concurrency override, got %+v", cfg.Concurrency)
}
if got := cfg.Concurrency.StageWorkers["extract"]; got != 2 {
t.Fatalf("expected extract workers to default to total, got %d", got)
}
}

113
internal/core/config/file_config.go
View File

@@ -8,7 +8,6 @@ import (
"sort"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gopkg.in/yaml.v3"
)
@@ -18,8 +17,9 @@ type FileConfig struct {
Scriptorium *FileScriptoriumConfig `yaml:"scriptorium,omitempty"`
Pipelines map[string]FilePipelineProfile `yaml:"pipelines,omitempty"`
Concurrency *FileConcurrencyConfig `yaml:"concurrency,omitempty"`
Diagnostics *FileDiagnosticsConfig `yaml:"diagnostics,omitempty"`
Workspace *FileWorkspaceConfig `yaml:"workspace,omitempty"`
Output *FileOutputConfig `yaml:"output,omitempty"`
Cache *FileCacheConfig `yaml:"cache,omitempty"`
Debug *FileDebugConfig `yaml:"debug,omitempty"`
}
type FileScriptoriumConfig struct {
@@ -48,31 +48,22 @@ type FileConcurrencyConfig struct {
StageWorkers map[string]int `yaml:"stage_workers,omitempty"`
}
type FileDiagnosticsConfig struct {
WorkDir *string `yaml:"work_dir,omitempty"`
Retention *string `yaml:"retention,omitempty"`
}
type FileWorkspaceConfig struct {
type FileOutputConfig struct {
Directory *string `yaml:"directory,omitempty"`
ChunkCache *FileWorkspaceChunkCacheConfig `yaml:"chunk_cache,omitempty"`
Diagnostics *FileWorkspaceDiagnosticsConfig `yaml:"diagnostics,omitempty"`
Resume *FileWorkspaceEnabledConfig `yaml:"resume,omitempty"`
Debug *FileWorkspaceEnabledConfig `yaml:"debug,omitempty"`
}
type FileWorkspaceChunkCacheConfig struct {
type FileCacheConfig struct {
ChunkPlans *FileChunkPlanCacheConfig `yaml:"chunk_plans,omitempty"`
Checkpoints *FileCheckpointCacheConfig `yaml:"checkpoints,omitempty"`
}
type FileChunkPlanCacheConfig struct {
Directory *string `yaml:"directory,omitempty"`
Mode *string `yaml:"mode,omitempty"`
}
type FileCheckpointCacheConfig struct {
Directory *string `yaml:"directory,omitempty"`
}
type FileWorkspaceDiagnosticsConfig struct {
Enabled *bool `yaml:"enabled,omitempty"`
Retention *string `yaml:"retention,omitempty"`
}
type FileWorkspaceEnabledConfig struct {
Enabled *bool `yaml:"enabled,omitempty"`
type FileDebugConfig struct {
Directory *string `yaml:"directory,omitempty"`
}
type fileModuleBinding struct {
@@ -172,18 +163,27 @@ func LoadFileConfig(path string) (FileConfig, error) {
}
func ParseFileConfigYAML(data []byte) (FileConfig, error) {
var header struct {
Version int `yaml:"version"`
}
if err := yaml.Unmarshal(data, &header); err != nil {
return FileConfig{}, fmt.Errorf("decode yaml version header: %w", err)
}
if header.Version == 0 {
return FileConfig{}, fmt.Errorf("config version is required")
}
if header.Version == 2 {
return FileConfig{}, fmt.Errorf("config version 2 is no longer supported; migrate the file using the version 2-to-3 migration in docs/config.md")
}
if header.Version != SupportedFileConfigVersion {
return FileConfig{}, fmt.Errorf("unsupported config version %d (supported version is %d)", header.Version, SupportedFileConfigVersion)
}
var fileCfg FileConfig
decoder := yaml.NewDecoder(bytes.NewReader(data))
decoder.KnownFields(true)
if err := decoder.Decode(&fileCfg); err != nil {
return FileConfig{}, fmt.Errorf("decode yaml: %w", err)
}
if fileCfg.Version == 0 {
return FileConfig{}, fmt.Errorf("config version is required")
}
if fileCfg.Version != SupportedFileConfigVersion {
return FileConfig{}, fmt.Errorf("unsupported config version %d", fileCfg.Version)
}
return fileCfg, nil
}
@@ -333,48 +333,47 @@ func (c *Config) applyFileConfigWithLookup(fileCfg FileConfig, lookup func(strin
c.Concurrency.extractWorkersConfigured = configured
}
c.Concurrency.recomputeStageWorkerDefaults()
if fileCfg.Diagnostics != nil {
if fileCfg.Diagnostics.WorkDir != nil {
c.Diagnostics.WorkDir = strings.TrimSpace(*fileCfg.Diagnostics.WorkDir)
if fileCfg.Output != nil && fileCfg.Output.Directory != nil {
c.Output.Directory = strings.TrimSpace(*fileCfg.Output.Directory)
if c.Output.Directory == "" {
return fmt.Errorf("output.directory must not be empty")
}
if fileCfg.Diagnostics.Retention != nil {
c.Diagnostics.Retention = diagnostics.RetentionMode(strings.TrimSpace(*fileCfg.Diagnostics.Retention))
if strings.ContainsRune(c.Output.Directory, '\x00') {
return fmt.Errorf("output.directory must not contain NUL")
}
}
if fileCfg.Workspace != nil {
if fileCfg.Workspace.Directory != nil {
c.Workspace.Directory = strings.TrimSpace(*fileCfg.Workspace.Directory)
}
if fileCfg.Workspace.ChunkCache != nil {
if fileCfg.Workspace.ChunkCache.Mode != nil {
mode, err := pipeline.ParseChunkCacheMode(*fileCfg.Workspace.ChunkCache.Mode)
if fileCfg.Cache != nil {
if fileCfg.Cache.ChunkPlans != nil {
if fileCfg.Cache.ChunkPlans.Mode != nil {
mode, err := pipeline.ParseChunkCacheMode(*fileCfg.Cache.ChunkPlans.Mode)
if err != nil {
return fmt.Errorf("workspace.chunk_cache.mode: %w", err)
return fmt.Errorf("cache.chunk_plans.mode: %w", err)
}
c.Workspace.ChunkCache.Mode = mode
c.Cache.ChunkPlans.Mode = mode
}
if fileCfg.Workspace.ChunkCache.Directory != nil {
c.Workspace.ChunkCache.Directory = cleanOptionalPath(*fileCfg.Workspace.ChunkCache.Directory)
if fileCfg.Cache.ChunkPlans.Directory != nil {
c.Cache.ChunkPlans.Directory = cleanOptionalPath(*fileCfg.Cache.ChunkPlans.Directory)
if strings.ContainsRune(c.Cache.ChunkPlans.Directory, '\x00') {
return fmt.Errorf("cache.chunk_plans.directory must not contain NUL")
}
}
if fileCfg.Workspace.Diagnostics != nil {
if fileCfg.Workspace.Diagnostics.Enabled != nil {
c.Workspace.Diagnostics.Enabled = *fileCfg.Workspace.Diagnostics.Enabled
c.Workspace.Diagnostics.enabledSet = true
}
if fileCfg.Workspace.Diagnostics.Retention != nil {
c.Workspace.Diagnostics.Retention = diagnostics.RetentionMode(strings.TrimSpace(*fileCfg.Workspace.Diagnostics.Retention))
c.Workspace.Diagnostics.retentionSet = true
if fileCfg.Cache.Checkpoints != nil && fileCfg.Cache.Checkpoints.Directory != nil {
c.Cache.Checkpoints.Directory = cleanOptionalPath(*fileCfg.Cache.Checkpoints.Directory)
if strings.ContainsRune(c.Cache.Checkpoints.Directory, '\x00') {
return fmt.Errorf("cache.checkpoints.directory must not contain NUL")
}
}
if fileCfg.Workspace.Resume != nil && fileCfg.Workspace.Resume.Enabled != nil {
c.Workspace.Resume.Enabled = *fileCfg.Workspace.Resume.Enabled
}
if fileCfg.Workspace.Debug != nil && fileCfg.Workspace.Debug.Enabled != nil {
c.Workspace.Debug.Enabled = *fileCfg.Workspace.Debug.Enabled
if fileCfg.Debug != nil && fileCfg.Debug.Directory != nil {
c.Debug.Directory = strings.TrimSpace(*fileCfg.Debug.Directory)
if c.Debug.Directory == "" {
return fmt.Errorf("debug.directory must not be empty")
}
if strings.ContainsRune(c.Debug.Directory, '\x00') {
return fmt.Errorf("debug.directory must not contain NUL")
}
}
c.RecomputeEffectiveDiagnostics()
return nil
}

723
internal/core/config/file_config_test.go
View File

@@ -1,723 +0,0 @@
package config
import (
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
)
func TestParseMinimalValidConfig(t *testing.T) {
fileCfg, err := ParseFileConfigYAML([]byte(`
version: 2
`))
if err != nil {
t.Fatalf("ParseFileConfigYAML: %v", err)
}
if fileCfg.Version != SupportedFileConfigVersion {
t.Fatalf("unexpected version: %d", fileCfg.Version)
}
}
func TestLoadFileConfig(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.yml")
if err := os.WriteFile(path, []byte("version: 2\n"), 0o644); err != nil {
t.Fatalf("write config: %v", err)
}
fileCfg, err := LoadFileConfig(path)
if err != nil {
t.Fatalf("LoadFileConfig: %v", err)
}
if fileCfg.Version != SupportedFileConfigVersion {
t.Fatalf("unexpected version: %d", fileCfg.Version)
}
}
func TestParseFileConfigRejectsUnknownYAMLFields(t *testing.T) {
_, err := ParseFileConfigYAML([]byte(`
version: 2
unexpected: true
`))
if err == nil || !strings.Contains(err.Error(), "field unexpected not found") {
t.Fatalf("expected unknown field error, got %v", err)
}
}
func TestParseFileConfigRejectsUnknownModuleBindingFields(t *testing.T) {
_, err := ParseFileConfigYAML([]byte(`
version: 2
pipelines:
example:
input:
module: fake/input
unexpected: true
artifacts:
events:
extract: fake/extract
`))
if err == nil || !strings.Contains(err.Error(), "field unexpected not found") {
t.Fatalf("expected unknown binding field error, got %v", err)
}
}
func TestParseFileConfigRejectsMissingAndUnsupportedVersion(t *testing.T) {
tests := []struct {
name string
data string
want string
}{
{name: "missing", data: `scriptorium: {}`, want: "version is required"},
{name: "unsupported", data: `version: 1`, want: "unsupported config version"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, err := ParseFileConfigYAML([]byte(tc.data))
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("expected error containing %q, got %v", tc.want, err)
}
})
}
}
func TestParseFileConfigRejectsStaleLLMProfiles(t *testing.T) {
_, err := ParseFileConfigYAML([]byte(`
version: 2
llm_profiles:
default: {}
`))
if err == nil || !strings.Contains(err.Error(), "llm_profiles") {
t.Fatalf("expected stale llm_profiles error, got %v", err)
}
}
func TestParseFileConfigScriptoriumProfileSources(t *testing.T) {
t.Run("profile dir", func(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
scriptorium:
profile_dir: ./profiles
`)
if cfg.Scriptorium.ProfileDir != "./profiles" || cfg.Scriptorium.ProfileFile != "" {
t.Fatalf("Scriptorium = %+v, want profile_dir", cfg.Scriptorium)
}
})
t.Run("profile file", func(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
scriptorium:
profile_file: ./profiles.yml
`)
if cfg.Scriptorium.ProfileFile != "./profiles.yml" || cfg.Scriptorium.ProfileDir != "" {
t.Fatalf("Scriptorium = %+v, want profile_file", cfg.Scriptorium)
}
})
}
func TestParseFileConfigModuleBindingForms(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
pipelines:
example:
input: fake/input
chunk:
module: generic
retries: 2
options:
size: 10
flags:
- alpha
nested:
enabled: true
artifacts:
events:
extract:
module: fake/extract
llm_profile: fast
retries: 3
options:
temperature: 0
merge:
module: appendorder
retries: 1
normalize:
module: noop
output: json
`)
profile := cfg.Pipelines["example"]
if profile.Input.Module != "fake/input" {
t.Fatalf("unexpected input binding: %+v", profile.Input)
}
if profile.Chunk.Module != "generic" {
t.Fatalf("unexpected chunk binding: %+v", profile.Chunk)
}
if profile.Chunk.Retries != 2 {
t.Fatalf("chunk retries = %d, want 2", profile.Chunk.Retries)
}
if profile.Chunk.Options["size"] != 10 {
t.Fatalf("expected chunk options to preserve scalar, got %#v", profile.Chunk.Options)
}
if !reflect.DeepEqual(profile.Chunk.Options["flags"], []any{"alpha"}) {
t.Fatalf("expected list option, got %#v", profile.Chunk.Options["flags"])
}
nested, ok := profile.Chunk.Options["nested"].(map[string]any)
if !ok || nested["enabled"] != true {
t.Fatalf("expected nested map option, got %#v", profile.Chunk.Options["nested"])
}
lane := profile.Artifacts["events"]
if lane.Extract.Module != "fake/extract" || lane.Extract.LLMProfile != "fast" {
t.Fatalf("unexpected extract binding: %+v", lane.Extract)
}
if lane.Extract.Retries != 3 || lane.Merge.Retries != 1 {
t.Fatalf("unexpected retries: extract=%d merge=%d", lane.Extract.Retries, lane.Merge.Retries)
}
if lane.Extract.Options["temperature"] != 0 {
t.Fatalf("expected object options, got %#v", lane.Extract.Options)
}
if lane.Merge.Module != "appendorder" || lane.Normalize.Module != "noop" {
t.Fatalf("unexpected lane defaults: %+v", lane)
}
if profile.Output.Module != "json" {
t.Fatalf("unexpected output binding: %+v", profile.Output)
}
}
func TestParseFileConfigReferenceMaps(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
pipelines:
example:
input: fake/input
references:
" roster ": " ./shared-roster.yml "
artifacts:
events:
extract: fake/extract
references:
" lore ": " ./lore.md "
`)
profile := cfg.Pipelines["example"]
if !reflect.DeepEqual(profile.References, map[string]string{"roster": "./shared-roster.yml"}) {
t.Fatalf("pipeline references = %#v, want trimmed map", profile.References)
}
gotLaneRefs := profile.Artifacts["events"].References
if !reflect.DeepEqual(gotLaneRefs, map[string]string{"lore": "./lore.md"}) {
t.Fatalf("lane references = %#v, want trimmed map", gotLaneRefs)
}
}
func TestParseFileConfigStageLocalReferenceMaps(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
pipelines:
example:
input: fake/input
chunk:
module: generic
references:
" scene_guide ": " ./scenes.md "
artifacts:
events:
extract:
module: fake/extract
references:
" glossary ": " ./glossary.md "
" roster ": " ./extract-roster.yml "
references:
roster: ./legacy-roster.yml
lore: ./lore.md
merge:
module: appendorder
references:
" merge_notes ": " ./merge.md "
normalize:
module: noop
references:
" normalization_notes ": " ./normalization.md "
`)
profile := cfg.Pipelines["example"]
if !reflect.DeepEqual(profile.Chunk.References, map[string]string{"scene_guide": "./scenes.md"}) {
t.Fatalf("chunk references = %#v, want trimmed map", profile.Chunk.References)
}
lane := profile.Artifacts["events"]
if !reflect.DeepEqual(lane.References, map[string]string{"lore": "./lore.md", "roster": "./legacy-roster.yml"}) {
t.Fatalf("lane references = %#v, want trimmed map", lane.References)
}
wantExtract := map[string]string{
"glossary": "./glossary.md",
"lore": "./lore.md",
"roster": "./extract-roster.yml",
}
if !reflect.DeepEqual(lane.Extract.References, wantExtract) {
t.Fatalf("extract references = %#v, want legacy merged with extract override %#v", lane.Extract.References, wantExtract)
}
if !reflect.DeepEqual(lane.Merge.References, map[string]string{"merge_notes": "./merge.md"}) {
t.Fatalf("merge references = %#v, want trimmed map", lane.Merge.References)
}
if !reflect.DeepEqual(lane.Normalize.References, map[string]string{"normalization_notes": "./normalization.md"}) {
t.Fatalf("normalize references = %#v, want trimmed map", lane.Normalize.References)
}
}
func TestParseFileConfigValidatorMixedBindingForms(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
pipelines:
example:
input: fake/input
artifacts:
events:
extract: fake/extract
validators:
- fake/validator
- module: fake/llm-validator
llm_profile: careful
options:
threshold: 0.7
`)
validators := cfg.Pipelines["example"].Artifacts["events"].Validators
if len(validators) != 2 {
t.Fatalf("expected two validators, got %d", len(validators))
}
if validators[0].Module != "fake/validator" {
t.Fatalf("unexpected shorthand validator: %+v", validators[0])
}
if validators[1].Module != "fake/llm-validator" || validators[1].LLMProfile != "careful" {
t.Fatalf("unexpected object validator: %+v", validators[1])
}
if validators[1].Options["threshold"] != 0.7 {
t.Fatalf("unexpected validator options: %#v", validators[1].Options)
}
}
func TestParseFileConfigStageLocalValidatorOverrides(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
pipelines:
example:
input: fake/input
chunk:
module: generic
validators: []
artifacts:
events:
extract:
module: fake/extract
validators:
- fake/validator
- module: fake/llm-validator
llm_profile: careful
options:
threshold: 0.7
merge:
module: appendorder
validators: []
normalize:
module: noop
`)
profile := cfg.Pipelines["example"]
if !profile.Chunk.Validators.Set || len(profile.Chunk.Validators.Validators) != 0 {
t.Fatalf("chunk validator override = %#v, want explicit empty", profile.Chunk.Validators)
}
lane := profile.Artifacts["events"]
if !lane.Extract.Validators.Set {
t.Fatalf("extract validator override Set = false, want true")
}
validators := lane.Extract.Validators.Validators
if len(validators) != 2 {
t.Fatalf("extract validators = %#v, want two validators", validators)
}
if validators[0].Module != "fake/validator" {
t.Fatalf("first validator = %#v, want fake/validator", validators[0])
}
if validators[1].Module != "fake/llm-validator" || validators[1].LLMProfile != "careful" {
t.Fatalf("second validator = %#v, want LLM validator with profile", validators[1])
}
if validators[1].Options["threshold"] != 0.7 {
t.Fatalf("second validator options = %#v, want threshold", validators[1].Options)
}
if !lane.Merge.Validators.Set || len(lane.Merge.Validators.Validators) != 0 {
t.Fatalf("merge validator override = %#v, want explicit empty", lane.Merge.Validators)
}
if lane.Normalize.Validators.Set {
t.Fatalf("normalize validator override Set = true, want omitted")
}
}
func TestApplyFileConfigRejectsDuplicateTrimmedPipelineIDs(t *testing.T) {
fileCfg, err := ParseFileConfigYAML([]byte(`
version: 2
pipelines:
example:
input: fake/input
" example ":
input: fake/other-input
`))
if err != nil {
t.Fatalf("ParseFileConfigYAML: %v", err)
}
cfg := Default()
err = cfg.applyFileConfigWithLookup(fileCfg, emptyLookup)
if err == nil || !strings.Contains(err.Error(), "pipeline id") || !strings.Contains(err.Error(), "duplicated") {
t.Fatalf("expected duplicate pipeline ID error, got %v", err)
}
}
func TestApplyFileConfigRejectsDuplicateTrimmedArtifactLaneIDs(t *testing.T) {
fileCfg, err := ParseFileConfigYAML([]byte(`
version: 2
pipelines:
example:
input: fake/input
artifacts:
events:
extract: fake/extract
" events ":
extract: fake/other-extract
`))
if err != nil {
t.Fatalf("ParseFileConfigYAML: %v", err)
}
cfg := Default()
err = cfg.applyFileConfigWithLookup(fileCfg, emptyLookup)
if err == nil || !strings.Contains(err.Error(), `pipeline "example" artifact lane id`) || !strings.Contains(err.Error(), "duplicated") {
t.Fatalf("expected duplicate artifact lane ID error, got %v", err)
}
}
func TestApplyFileConfigRejectsDuplicateTrimmedReferenceSlots(t *testing.T) {
tests := []struct {
name string
raw string
want string
}{
{
name: "pipeline",
raw: `
version: 2
pipelines:
example:
input: fake/input
references:
roster: ./first.yml
" roster ": ./second.yml
`,
want: `pipeline "example" reference slot`,
},
{
name: "lane",
raw: `
version: 2
pipelines:
example:
input: fake/input
artifacts:
events:
extract: fake/extract
references:
roster: ./first.yml
" roster ": ./second.yml
`,
want: `pipeline "example" lane "events" reference slot`,
},
{
name: "chunk",
raw: `
version: 2
pipelines:
example:
input: fake/input
chunk:
module: generic
references:
roster: ./first.yml
" roster ": ./second.yml
`,
want: `pipeline "example" chunk reference slot`,
},
{
name: "extract",
raw: `
version: 2
pipelines:
example:
input: fake/input
artifacts:
events:
extract:
module: fake/extract
references:
roster: ./first.yml
" roster ": ./second.yml
`,
want: `pipeline "example" lane "events" extract reference slot`,
},
{
name: "normalize",
raw: `
version: 2
pipelines:
example:
input: fake/input
artifacts:
events:
extract: fake/extract
normalize:
module: noop
references:
roster: ./first.yml
" roster ": ./second.yml
`,
want: `pipeline "example" lane "events" normalize reference slot`,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
fileCfg, err := ParseFileConfigYAML([]byte(tc.raw))
if err != nil {
t.Fatalf("ParseFileConfigYAML: %v", err)
}
cfg := Default()
err = cfg.applyFileConfigWithLookup(fileCfg, emptyLookup)
if err == nil || !strings.Contains(err.Error(), tc.want) || !strings.Contains(err.Error(), "duplicated") {
t.Fatalf("expected duplicate reference slot error, got %v", err)
}
})
}
}
func TestApplyFileConfigRejectsInvalidScriptoriumSources(t *testing.T) {
tests := []struct {
name string
raw string
want string
}{
{name: "empty profile dir", raw: "profile_dir: ' '", want: "profile_dir"},
{name: "empty profile file", raw: "profile_file: ' '", want: "profile_file"},
{name: "both sources", raw: "profile_dir: ./profiles\n profile_file: ./profiles.yml", want: "mutually exclusive"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
cfg := Default()
fileCfg, err := ParseFileConfigYAML([]byte(`
version: 2
scriptorium:
` + tc.raw + `
`))
if err != nil {
t.Fatalf("ParseFileConfigYAML: %v", err)
}
err = cfg.applyFileConfigWithLookup(fileCfg, emptyLookup)
if err == nil {
err = cfg.Validate()
}
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("expected error containing %q, got %v", tc.want, err)
}
})
}
}
func TestApplyFileConfigOperationalSections(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
concurrency:
total_llm: 4
diagnostics:
work_dir: /tmp/notarius-test
retention: always
`)
if cfg.Concurrency.TotalLLM != 4 {
t.Fatalf("unexpected total concurrency: %d", cfg.Concurrency.TotalLLM)
}
if got := cfg.Concurrency.StageWorkers["extract"]; got != 4 {
t.Fatalf("default extract workers = %d, want total concurrency", got)
}
if cfg.Diagnostics.WorkDir != "/tmp/notarius-test" {
t.Fatalf("unexpected work dir: %q", cfg.Diagnostics.WorkDir)
}
if cfg.Diagnostics.Retention != diagnostics.RetentionAlways {
t.Fatalf("unexpected retention: %q", cfg.Diagnostics.Retention)
}
}
func TestApplyFileConfigStageWorkers(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
concurrency:
total_llm: 4
stage_workers:
extract: 3
`)
if got := cfg.Concurrency.StageWorkers["extract"]; got != 3 {
t.Fatalf("extract workers = %d, want 3", got)
}
if err := cfg.Validate(); err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
}
func TestApplyFileConfigEmptyStageWorkersDefaultsExtractToTotal(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
concurrency:
total_llm: 4
stage_workers: {}
`)
if got := cfg.Concurrency.StageWorkers["extract"]; got != 4 {
t.Fatalf("extract workers = %d, want total concurrency 4", got)
}
}
func TestApplyFileConfigRejectsUnsupportedStageWorkerKeys(t *testing.T) {
for _, test := range []struct {
name string
key string
want string
}{
{name: "empty", key: "' '", want: "must not be empty"},
{name: "unknown", key: "merge", want: "not supported"},
} {
t.Run(test.name, func(t *testing.T) {
fileCfg, err := ParseFileConfigYAML([]byte("version: 2\nconcurrency:\n stage_workers:\n " + test.key + ": 1\n"))
if err != nil {
t.Fatalf("ParseFileConfigYAML() error = %v", err)
}
cfg := Default()
err = cfg.applyFileConfigWithLookup(fileCfg, emptyLookup)
if err == nil || !strings.Contains(err.Error(), test.want) {
t.Fatalf("ApplyFileConfig() error = %v, want %q", err, test.want)
}
})
}
}
func TestApplyFileConfigWorkspaceSection(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
workspace:
directory: /var/lib/notarius
diagnostics:
enabled: false
retention: never
resume:
enabled: true
debug:
enabled: true
diagnostics:
work_dir: /tmp/legacy
retention: always
`)
if cfg.Workspace.Directory != "/var/lib/notarius" {
t.Fatalf("workspace directory = %q, want /var/lib/notarius", cfg.Workspace.Directory)
}
if cfg.DiagnosticsEnabled() {
t.Fatalf("expected diagnostics disabled")
}
if cfg.Diagnostics.WorkDir != "/var/lib/notarius/diagnostics" {
t.Fatalf("effective diagnostics work dir = %q, want workspace diagnostics root", cfg.Diagnostics.WorkDir)
}
if cfg.Diagnostics.Retention != diagnostics.RetentionNever {
t.Fatalf("effective diagnostics retention = %q, want workspace override", cfg.Diagnostics.Retention)
}
if !cfg.Workspace.Resume.Enabled {
t.Fatalf("expected resume enabled")
}
if !cfg.Workspace.Debug.Enabled {
t.Fatalf("expected debug enabled")
}
}
func TestApplyFileConfigLegacyDiagnosticsRemainCompatible(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
diagnostics:
work_dir: /tmp/legacy
retention: always
`)
if cfg.Workspace.Directory != "" {
t.Fatalf("workspace directory = %q, want unset", cfg.Workspace.Directory)
}
if !cfg.DiagnosticsEnabled() {
t.Fatalf("expected diagnostics enabled")
}
if cfg.Diagnostics.WorkDir != "/tmp/legacy" {
t.Fatalf("effective diagnostics work dir = %q, want legacy", cfg.Diagnostics.WorkDir)
}
if cfg.Diagnostics.Retention != diagnostics.RetentionAlways {
t.Fatalf("effective diagnostics retention = %q, want legacy", cfg.Diagnostics.Retention)
}
}
func TestApplyFileConfigWorkspaceRetentionOverridesLegacyRetentionOnlyWhenSet(t *testing.T) {
t.Run("legacy retained", func(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
workspace:
directory: /var/lib/notarius
diagnostics:
retention: never
`)
if cfg.Diagnostics.Retention != diagnostics.RetentionNever {
t.Fatalf("effective diagnostics retention = %q, want legacy", cfg.Diagnostics.Retention)
}
})
t.Run("workspace overrides", func(t *testing.T) {
cfg := parseAndApplyConfig(t, `
version: 2
workspace:
directory: /var/lib/notarius
diagnostics:
retention: always
diagnostics:
retention: never
`)
if cfg.Diagnostics.Retention != diagnostics.RetentionAlways {
t.Fatalf("effective diagnostics retention = %q, want workspace", cfg.Diagnostics.Retention)
}
})
}
func parseAndApplyConfig(t *testing.T, raw string) Config {
t.Helper()
fileCfg, err := ParseFileConfigYAML([]byte(raw))
if err != nil {
t.Fatalf("ParseFileConfigYAML: %v", err)
}
cfg := Default()
if err := cfg.applyFileConfigWithLookup(fileCfg, emptyLookup); err != nil {
t.Fatalf("ApplyFileConfig: %v", err)
}
return cfg
}
func emptyLookup(string) (string, bool) {
return "", false
}
func mapLookup(values map[string]string) func(string) (string, bool) {
return func(key string) (string, bool) {
value, ok := values[key]
return value, ok
}
}

90
internal/core/config/redaction.go
View File

@@ -1,16 +1,20 @@
package config
import "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
import (
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func (c Config) Redacted() Config {
return cloneConfig(c)
return redactConfig(cloneConfig(c))
}
func (c Config) RedactedDiagnosticsPayload() any {
func (c Config) RedactedSummaryPayload() any {
return c.Redacted()
}
func (e EffectiveConfig) RedactedDiagnosticsPayload() any {
func (e EffectiveConfig) RedactedSummaryPayload() any {
return EffectiveConfig{
Config: e.Config.Redacted(),
PipelineID: e.PipelineID,
@@ -23,10 +27,10 @@ func (e EffectiveConfig) RedactedDiagnosticsPayload() any {
func cloneResolvedPipeline(in pipeline.ResolvedPipeline) pipeline.ResolvedPipeline {
out := in
out.Input = cloneModuleBinding(in.Input)
out.Chunk = cloneModuleBinding(in.Chunk)
out.Input = redactBinding(cloneModuleBinding(in.Input))
out.Chunk = redactBinding(cloneModuleBinding(in.Chunk))
out.ChunkReferences = pipeline.CloneReferenceTarget(in.ChunkReferences)
out.Output = cloneModuleBinding(in.Output)
out.Output = redactBinding(cloneModuleBinding(in.Output))
if len(in.ValidatorChains) > 0 {
out.ValidatorChains = make([]pipeline.ResolvedValidatorChain, len(in.ValidatorChains))
for i, chain := range in.ValidatorChains {
@@ -48,7 +52,7 @@ func cloneResolvedValidatorChain(in pipeline.ResolvedValidatorChain) pipeline.Re
out.Validators = make([]pipeline.ResolvedValidator, len(in.Validators))
for i, validator := range in.Validators {
out.Validators[i] = pipeline.ResolvedValidator{
Binding: cloneModuleBinding(validator.Binding),
Binding: redactBinding(cloneModuleBinding(validator.Binding)),
ExecutionClass: validator.ExecutionClass,
Target: validator.Target,
ArtifactKind: validator.ArtifactKind,
@@ -60,17 +64,79 @@ func cloneResolvedValidatorChain(in pipeline.ResolvedValidatorChain) pipeline.Re
func cloneResolvedArtifactLane(in pipeline.ResolvedArtifactLane) pipeline.ResolvedArtifactLane {
out := in
out.Extract = cloneModuleBinding(in.Extract)
out.Merge = cloneModuleBinding(in.Merge)
out.Normalize = cloneModuleBinding(in.Normalize)
out.Extract = redactBinding(cloneModuleBinding(in.Extract))
out.Merge = redactBinding(cloneModuleBinding(in.Merge))
out.Normalize = redactBinding(cloneModuleBinding(in.Normalize))
out.ExtractReferences = pipeline.CloneReferenceTarget(in.ExtractReferences)
out.MergeReferences = pipeline.CloneReferenceTarget(in.MergeReferences)
out.NormalizeReferences = pipeline.CloneReferenceTarget(in.NormalizeReferences)
if len(in.Validators) > 0 {
out.Validators = make([]pipeline.ModuleBinding, len(in.Validators))
for i, binding := range in.Validators {
out.Validators[i] = cloneModuleBinding(binding)
out.Validators[i] = redactBinding(cloneModuleBinding(binding))
}
}
return out
}
func redactConfig(cfg Config) Config {
for id, profile := range cfg.Pipelines {
profile.Input = redactBinding(profile.Input)
profile.Chunk = redactBinding(profile.Chunk)
profile.Output = redactBinding(profile.Output)
for laneID, lane := range profile.Artifacts {
lane.Extract = redactBinding(lane.Extract)
lane.Merge = redactBinding(lane.Merge)
lane.Normalize = redactBinding(lane.Normalize)
for i := range lane.Validators {
lane.Validators[i] = redactBinding(lane.Validators[i])
}
profile.Artifacts[laneID] = lane
}
cfg.Pipelines[id] = profile
}
return cfg
}
func redactBinding(binding pipeline.ModuleBinding) pipeline.ModuleBinding {
binding.Options = redactOptions(binding.Options)
for i := range binding.Validators.Validators {
binding.Validators.Validators[i] = redactBinding(binding.Validators.Validators[i])
}
return binding
}
func redactOptions(values map[string]any) map[string]any {
if len(values) == 0 {
return nil
}
out := make(map[string]any, len(values))
for key, value := range values {
if sensitiveConfigKey(key) {
out[key] = "[REDACTED]"
continue
}
switch typed := value.(type) {
case map[string]any:
out[key] = redactOptions(typed)
case []any:
items := make([]any, len(typed))
for i, item := range typed {
if nested, ok := item.(map[string]any); ok {
items[i] = redactOptions(nested)
} else {
items[i] = item
}
}
out[key] = items
default:
out[key] = value
}
}
return out
}
func sensitiveConfigKey(key string) bool {
key = strings.ToLower(key)
return strings.Contains(key, "api_key") || strings.Contains(key, "apikey") || strings.Contains(key, "authorization") || strings.Contains(key, "bearer") || strings.Contains(key, "password") || strings.Contains(key, "secret") || strings.Contains(key, "token")
}

172
internal/core/config/redaction_test.go
View File

@@ -1,172 +0,0 @@
package config
import (
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestRedactedConfigCopiesScriptoriumConfig(t *testing.T) {
cfg := Default()
cfg.Scriptorium.ProfileDir = "./profiles"
cfg.Workspace.Directory = "/var/lib/notarius"
cfg.Workspace.Resume.Enabled = true
cfg.Concurrency.StageWorkers["extract"] = 1
redacted := cfg.Redacted()
if redacted.Scriptorium.ProfileDir != "./profiles" {
t.Fatalf("expected Scriptorium profile source preserved, got %+v", redacted.Scriptorium)
}
redacted.Scriptorium.ProfileDir = "./changed"
if cfg.Scriptorium.ProfileDir != "./profiles" {
t.Fatalf("redaction mutated original config")
}
if redacted.Workspace.Directory != "/var/lib/notarius" || !redacted.Workspace.Resume.Enabled {
t.Fatalf("expected workspace config preserved, got %+v", redacted.Workspace)
}
redacted.Workspace.Directory = "/changed"
if cfg.Workspace.Directory != "/var/lib/notarius" {
t.Fatalf("redaction mutated original workspace config")
}
redacted.Concurrency.StageWorkers["extract"] = 9
if cfg.Concurrency.StageWorkers["extract"] != 1 {
t.Fatalf("redaction aliased stage worker map")
}
}
func TestConfigRedactedDiagnosticsPayloadCopiesConfig(t *testing.T) {
cfg := Default()
cfg.Scriptorium.ProfileFile = "./profiles.yml"
payload, ok := cfg.RedactedDiagnosticsPayload().(Config)
if !ok {
t.Fatalf("expected Config payload, got %T", cfg.RedactedDiagnosticsPayload())
}
if payload.Scriptorium.ProfileFile != "./profiles.yml" {
t.Fatalf("expected Scriptorium profile file preserved, got %+v", payload.Scriptorium)
}
}
func TestEffectiveConfigRedactedDiagnosticsPayloadCopies(t *testing.T) {
cfg := validConfig()
cfg.Concurrency.TotalLLM = 4
cfg.Concurrency.StageWorkers["extract"] = 2
lane := cfg.Pipelines["example"].Artifacts["events"]
lane.Extract.Options = map[string]any{"temperature": 0.2}
lane.References = map[string]string{"roster": "./roster.yml"}
lane.Extract.References = map[string]string{"glossary": "./glossary.md"}
lane.Normalize.References = map[string]string{"notes": "./normalize.md"}
cfg.Pipelines["example"].Artifacts["events"] = lane
pipelineProfile := cfg.Pipelines["example"]
pipelineProfile.Chunk.References = map[string]string{"scene_guide": "./scene.md"}
cfg.Pipelines["example"] = pipelineProfile
effective, err := cfg.Resolve(ResolveInput{
PipelineID: "example",
Only: []string{"events"},
Catalog: fakeCatalog(t,
pipeline.ModuleSpec{
Key: "generic",
Stage: pipeline.StageChunk,
Requires: []string{"source"},
Provides: []string{"chunks"},
ReferenceSlots: []contracts.ReferenceSlot{
{Name: "scene_guide"},
},
},
pipeline.ModuleSpec{
Key: "fake/extract",
Stage: pipeline.StageExtract,
Requires: []string{"chunks"},
Provides: []string{"artifact"},
ReferenceSlots: []contracts.ReferenceSlot{
{Name: "glossary"},
{Name: "roster"},
},
},
pipeline.ModuleSpec{
Key: "noop",
Stage: pipeline.StageNormalize,
Requires: []string{"merged"},
Provides: []string{"normalized"},
ReferenceSlots: []contracts.ReferenceSlot{
{Name: "notes"},
},
},
),
})
if err != nil {
t.Fatalf("Resolve: %v", err)
}
payload, ok := effective.RedactedDiagnosticsPayload().(EffectiveConfig)
if !ok {
t.Fatalf("expected EffectiveConfig payload, got %T", effective.RedactedDiagnosticsPayload())
}
if payload.PipelineID != effective.PipelineID || payload.ResolvedPipeline.Digest != effective.ResolvedPipeline.Digest {
t.Fatalf("expected pipeline metadata preserved, got %+v", payload)
}
payload.Config.Concurrency.StageWorkers["extract"] = 4
if effective.Config.Concurrency.StageWorkers["extract"] != 2 {
t.Fatalf("expected effective stage worker map to be copied")
}
payload.Only[0] = "changed"
if effective.Only[0] != "events" {
t.Fatalf("expected only lanes to be copied")
}
payload.ResolvedPipeline.ArtifactLanes[0].Extract.Options["temperature"] = 1.0
if effective.ResolvedPipeline.ArtifactLanes[0].Extract.Options["temperature"] != 0.2 {
t.Fatalf("expected resolved pipeline options to be copied")
}
payload.ResolvedPipeline.ArtifactLanes[0].ExtractReferences.Bindings[0].Source = "./changed.yml"
if referenceBindingSource(effective.ResolvedPipeline.ArtifactLanes[0].ExtractReferences.Bindings, "roster") != "./roster.yml" {
t.Fatalf("expected resolved pipeline references to be copied")
}
payload.ResolvedPipeline.Chunk.References["scene_guide"] = "./changed-scene.md"
if effective.ResolvedPipeline.Chunk.References["scene_guide"] != "./scene.md" {
t.Fatalf("expected chunk references to be copied")
}
payload.ResolvedPipeline.ArtifactLanes[0].Extract.References["glossary"] = "./changed-glossary.md"
if effective.ResolvedPipeline.ArtifactLanes[0].Extract.References["glossary"] != "./glossary.md" {
t.Fatalf("expected extract references to be copied")
}
payload.ResolvedPipeline.ArtifactLanes[0].Normalize.References["notes"] = "./changed-normalize.md"
if effective.ResolvedPipeline.ArtifactLanes[0].Normalize.References["notes"] != "./normalize.md" {
t.Fatalf("expected normalize references to be copied")
}
effective.ResolvedPipeline.ArtifactLanes[0].ExtractReferences.ReferenceSet = contracts.ReferenceSet{
Slots: map[string]contracts.ResolvedReferenceSlot{
"roster": {
Slot: contracts.ReferenceSlot{Name: "roster"},
Items: []contracts.ReferenceItem{
{
SlotName: "roster",
Content: []byte("reference content"),
},
},
},
},
}
payload, ok = effective.RedactedDiagnosticsPayload().(EffectiveConfig)
if !ok {
t.Fatalf("expected EffectiveConfig payload, got %T", effective.RedactedDiagnosticsPayload())
}
payload.ResolvedPipeline.ArtifactLanes[0].ExtractReferences.ReferenceSet.Slots["roster"].Items[0].Content[0] = 'X'
got := effective.ResolvedPipeline.ArtifactLanes[0].ExtractReferences.ReferenceSet.Slots["roster"].Items[0].Content
if string(got) != "reference content" {
t.Fatalf("expected materialized reference content to be copied, got %q", got)
}
}
func referenceBindingSource(bindings []pipeline.ReferenceBinding, slotName string) string {
for _, binding := range bindings {
if binding.SlotName == slotName {
return binding.Source
}
}
return ""
}

124
internal/core/config/v3_test.go Normal file
View File

@@ -0,0 +1,124 @@
package config
import (
"encoding/json"
"path/filepath"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestVersion3DefaultsAndValidation(t *testing.T) {
cfg := Default()
if cfg.Output.Directory != "./notarius-output" || cfg.Debug.Directory != "./notarius-debug" || cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheAuto {
t.Fatalf("unexpected defaults: %#v", cfg)
}
if err := cfg.Validate(); err != nil {
t.Fatalf("Validate() error = %v", err)
}
}
func TestVersion3FileSchemaIsStrictAndRejectsVersion2BeforeDecode(t *testing.T) {
_, err := ParseFileConfigYAML([]byte("version: 2\nworkspace:\n directory: /tmp/old\n"))
if err == nil || !strings.Contains(err.Error(), "version 2-to-3 migration") {
t.Fatalf("version 2 error = %v", err)
}
_, err = ParseFileConfigYAML([]byte("version: 3\nworkspace:\n directory: /tmp/old\n"))
if err == nil || !strings.Contains(err.Error(), "field workspace not found") {
t.Fatalf("unknown field error = %v", err)
}
_, err = ParseFileConfigYAML([]byte("version: 4\n"))
if err == nil || !strings.Contains(err.Error(), "unsupported config version 4") {
t.Fatalf("version 4 error = %v", err)
}
}
func TestStatePrecedenceAndInvalidSources(t *testing.T) {
file, err := ParseFileConfigYAML([]byte(`version: 3
output:
directory: ./file-output
cache:
chunk_plans:
directory: ./plans
mode: refresh
checkpoints:
directory: ./checkpoints
debug:
directory: ./debug
`))
if err != nil {
t.Fatal(err)
}
cfg := Default()
if err := cfg.ApplyFileConfig(file); err != nil {
t.Fatal(err)
}
lookup := func(name string) (string, bool) {
values := map[string]string{
"NOTARIUS_OUTPUT_DIR": "/env/output", "NOTARIUS_CACHE_CHUNK_PLANS_MODE": "auto",
"NOTARIUS_CACHE_CHUNK_PLANS_DIR": "/env/plans", "NOTARIUS_CACHE_CHECKPOINTS_DIR": "/env/checkpoints", "NOTARIUS_DEBUG_DIR": "/env/debug",
}
v, ok := values[name]
return v, ok
}
if err := cfg.ApplyEnvOverridesWithLookup(lookup); err != nil {
t.Fatal(err)
}
if cfg.Output.Directory != "/env/output" || cfg.Cache.ChunkPlans.Directory != "/env/plans" || cfg.Cache.Checkpoints.Directory != "/env/checkpoints" || cfg.Debug.Directory != "/env/debug" || cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheAuto {
t.Fatalf("unexpected environment precedence: %#v", cfg)
}
bad := Default()
err = bad.ApplyEnvOverridesWithLookup(func(name string) (string, bool) {
if name == "NOTARIUS_DEBUG_DIR" {
return " ", true
}
return "", false
})
if err == nil || !strings.Contains(err.Error(), "NOTARIUS_DEBUG_DIR") {
t.Fatalf("empty debug environment error = %v", err)
}
invalidFile, err := ParseFileConfigYAML([]byte("version: 3\noutput:\n directory: ' '\n"))
if err != nil {
t.Fatal(err)
}
bad = Default()
if err := bad.ApplyFileConfig(invalidFile); err == nil || !strings.Contains(err.Error(), "output.directory") {
t.Fatalf("invalid file error = %v", err)
}
}
func TestRedactedSummaryContainsOnlyVersion3StateFields(t *testing.T) {
cfg := Default()
cfg.Pipelines["example"] = pipeline.PipelineProfile{Input: pipeline.ModuleBinding{Module: "input", Options: map[string]any{"api_key": "secret-value", "safe": "value"}}}
payload, err := json.Marshal(cfg.RedactedSummaryPayload())
if err != nil {
t.Fatal(err)
}
text := string(payload)
for _, forbidden := range []string{"workspace", "diagnostics"} {
if strings.Contains(text, forbidden) {
t.Fatalf("payload contains %q: %s", forbidden, text)
}
}
if strings.Contains(text, "secret-value") || !strings.Contains(text, "[REDACTED]") {
t.Fatalf("payload did not redact sensitive option: %s", text)
}
}
func TestCacheFamilyDefaultsAreIndependent(t *testing.T) {
base := filepath.Join(t.TempDir(), "cache")
resolver := func() (string, error) { return base, nil }
plans, err := DefaultChunkPlanRoot(resolver)
if err != nil {
t.Fatal(err)
}
checkpoints, err := DefaultCheckpointRoot(resolver)
if err != nil {
t.Fatal(err)
}
if plans == checkpoints || plans != filepath.Join(base, "notarius", "chunk-plans") || checkpoints != filepath.Join(base, "notarius", "checkpoints") {
t.Fatalf("roots = %q, %q", plans, checkpoints)
}
}

44
internal/core/config/validation.go
View File

@@ -5,7 +5,6 @@ import (
"sort"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
@@ -14,10 +13,7 @@ func (c Config) Validate() error {
if err := validateScriptorium(c.Scriptorium); err != nil {
return err
}
if err := validateWorkspace(c.Workspace); err != nil {
return err
}
if err := validateDiagnostics(c.Diagnostics); err != nil {
if err := validateStateSurfaces(c); err != nil {
return err
}
if c.Concurrency.TotalLLM <= 0 {
@@ -60,35 +56,29 @@ func validateScriptorium(cfg ScriptoriumConfig) error {
return nil
}
func validateWorkspace(cfg WorkspaceConfig) error {
if err := cfg.ChunkCache.Mode.Validate(); err != nil {
return fmt.Errorf("workspace chunk cache: %w", err)
func validateStateSurfaces(cfg Config) error {
if strings.TrimSpace(cfg.Output.Directory) == "" {
return fmt.Errorf("output.directory must not be empty")
}
if strings.ContainsRune(cfg.ChunkCache.Directory, '\x00') {
return fmt.Errorf("workspace chunk cache directory must not contain NUL")
if strings.TrimSpace(cfg.Debug.Directory) == "" {
return fmt.Errorf("debug.directory must not be empty")
}
if cfg.Diagnostics.retentionSet {
switch cfg.Diagnostics.Retention {
case "", diagnostics.RetentionAuto, diagnostics.RetentionAlways, diagnostics.RetentionNever:
default:
return fmt.Errorf("workspace diagnostics retention %q is not supported", cfg.Diagnostics.Retention)
if err := cfg.Cache.ChunkPlans.Mode.Validate(); err != nil {
return fmt.Errorf("cache.chunk_plans.mode: %w", err)
}
for name, value := range map[string]string{
"output.directory": cfg.Output.Directory,
"cache.chunk_plans.directory": cfg.Cache.ChunkPlans.Directory,
"cache.checkpoints.directory": cfg.Cache.Checkpoints.Directory,
"debug.directory": cfg.Debug.Directory,
} {
if strings.ContainsRune(value, '\x00') {
return fmt.Errorf("%s must not contain NUL", name)
}
}
return nil
}
func validateDiagnostics(cfg DiagnosticsConfig) error {
if strings.TrimSpace(cfg.WorkDir) == "" {
return fmt.Errorf("diagnostics work dir must not be empty")
}
switch cfg.Retention {
case "", diagnostics.RetentionAuto, diagnostics.RetentionAlways, diagnostics.RetentionNever:
return nil
default:
return fmt.Errorf("diagnostics retention %q is not supported", cfg.Retention)
}
}
func validatePipelineProfiles(profiles map[string]pipeline.PipelineProfile) error {
seen := make(map[string]struct{}, len(profiles))
for rawID, profile := range profiles {

664
internal/core/config/validation_test.go
View File

@@ -1,664 +0,0 @@
package config
import (
"fmt"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestValidateSuccessForValidConfig(t *testing.T) {
cfg := validConfig()
if err := cfg.Validate(); err != nil {
t.Fatalf("Validate: %v", err)
}
}
func TestValidateAllowsExplicitScriptoriumProfileIDOnBinding(t *testing.T) {
cfg := validConfig()
lane := cfg.Pipelines["example"].Artifacts["events"]
lane.Extract.LLMProfile = "scriptorium-profile"
cfg.Pipelines["example"].Artifacts["events"] = lane
if err := cfg.Validate(); err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
}
func TestValidateRejectsWhitespaceOnlyExplicitLLMProfile(t *testing.T) {
cfg := validConfig()
lane := cfg.Pipelines["example"].Artifacts["events"]
lane.Extract.LLMProfile = " "
cfg.Pipelines["example"].Artifacts["events"] = lane
err := cfg.Validate()
if err == nil || !strings.Contains(err.Error(), "llm_profile") || !strings.Contains(err.Error(), "events") {
t.Fatalf("expected llm_profile error with lane context, got %v", err)
}
}
func TestValidateRejectsInvalidNumericFields(t *testing.T) {
tests := []struct {
name string
mutate func(Config) Config
want string
}{
{
name: "total concurrency",
mutate: func(cfg Config) Config {
cfg.Concurrency.TotalLLM = 0
return cfg
},
want: "total LLM concurrency",
},
{
name: "negative retries",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
lane := profile.Artifacts["events"]
lane.Merge.Retries = -1
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
return cfg
},
want: "retries",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
err := tc.mutate(validConfig()).Validate()
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("expected error containing %q, got %v", tc.want, err)
}
})
}
}
func TestValidateStageWorkerBoundaries(t *testing.T) {
for _, test := range []struct {
name string
workers int
wantErr bool
}{
{name: "below minimum", workers: 0, wantErr: true},
{name: "minimum", workers: 1},
{name: "maximum", workers: 4},
{name: "above maximum", workers: 5, wantErr: true},
} {
t.Run(test.name, func(t *testing.T) {
cfg := validConfig()
cfg.Concurrency.TotalLLM = 4
cfg.Concurrency.StageWorkers["extract"] = test.workers
err := cfg.Validate()
if test.wantErr && (err == nil || !strings.Contains(err.Error(), "stage_workers.extract")) {
t.Fatalf("Validate() error = %v, want extract worker range error", err)
}
if !test.wantErr && err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
})
}
}
func TestValidateRejectsUnknownEffectiveStageWorkerKey(t *testing.T) {
cfg := validConfig()
cfg.Concurrency.StageWorkers["merge"] = 1
err := cfg.Validate()
if err == nil || !strings.Contains(err.Error(), "stage_workers key") || !strings.Contains(err.Error(), "merge") {
t.Fatalf("Validate() error = %v, want unknown stage worker key", err)
}
}
func TestValidateRejectsMutuallyExclusiveScriptoriumProfileSources(t *testing.T) {
cfg := validConfig()
cfg.Scriptorium.ProfileDir = "./profiles"
cfg.Scriptorium.ProfileFile = "./profiles.yml"
err := cfg.Validate()
if err == nil || !strings.Contains(err.Error(), "mutually exclusive") {
t.Fatalf("expected Scriptorium source conflict, got %v", err)
}
}
func TestValidateRejectsInvalidDiagnosticsRetention(t *testing.T) {
cfg := validConfig()
cfg.Diagnostics.Retention = diagnostics.RetentionMode("sometimes")
err := cfg.Validate()
if err == nil || !strings.Contains(err.Error(), "retention") {
t.Fatalf("expected retention error, got %v", err)
}
}
func TestValidateRejectsInvalidWorkspaceDiagnosticsRetention(t *testing.T) {
cfg := validConfig()
cfg.Workspace.Diagnostics.Retention = diagnostics.RetentionMode("sometimes")
cfg.Workspace.Diagnostics.retentionSet = true
err := cfg.Validate()
if err == nil || !strings.Contains(err.Error(), "workspace diagnostics retention") {
t.Fatalf("expected workspace retention error, got %v", err)
}
}
func TestValidateRejectsInvalidReferenceMaps(t *testing.T) {
tests := []struct {
name string
mutate func(Config) Config
want []string
}{
{
name: "empty chunk slot",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
profile.Chunk.References = map[string]string{" ": "./roster.yml"}
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "chunk", "reference slot", "empty"},
},
{
name: "empty chunk source",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
profile.Chunk.References = map[string]string{"roster": " "}
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "chunk", "roster", "source", "empty"},
},
{
name: "empty extract slot",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
lane := profile.Artifacts["events"]
lane.Extract.References = map[string]string{" ": "./roster.yml"}
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "events", "extract", "reference slot", "empty"},
},
{
name: "empty extract source",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
lane := profile.Artifacts["events"]
lane.Extract.References = map[string]string{"roster": " "}
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "events", "extract", "roster", "source", "empty"},
},
{
name: "empty normalize slot",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
lane := profile.Artifacts["events"]
lane.Normalize.References = map[string]string{" ": "./roster.yml"}
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "events", "normalize", "reference slot", "empty"},
},
{
name: "empty normalize source",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
lane := profile.Artifacts["events"]
lane.Normalize.References = map[string]string{"roster": " "}
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "events", "normalize", "roster", "source", "empty"},
},
{
name: "empty pipeline slot",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
profile.References = map[string]string{" ": "./roster.yml"}
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "reference slot", "empty"},
},
{
name: "empty pipeline source",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
profile.References = map[string]string{"roster": " "}
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "roster", "source", "empty"},
},
{
name: "empty lane slot",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
lane := profile.Artifacts["events"]
lane.References = map[string]string{" ": "./roster.yml"}
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "events", "reference slot", "empty"},
},
{
name: "empty lane source",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
lane := profile.Artifacts["events"]
lane.References = map[string]string{"roster": " "}
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "events", "roster", "source", "empty"},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
err := tc.mutate(validConfig()).Validate()
if err == nil {
t.Fatal("Validate() error = nil, want error")
}
for _, want := range tc.want {
if !strings.Contains(err.Error(), want) {
t.Fatalf("Validate() error = %q, want substring %q", err.Error(), want)
}
}
})
}
}
func TestValidateRejectsReferencesOnUnsupportedBindings(t *testing.T) {
tests := []struct {
name string
mutate func(Config) Config
want []string
}{
{
name: "input",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
profile.Input.References = map[string]string{"roster": "./roster.yml"}
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "input", "references", "not supported"},
},
{
name: "output",
mutate: func(cfg Config) Config {
profile := cfg.Pipelines["example"]
profile.Output.References = map[string]string{"roster": "./roster.yml"}
cfg.Pipelines["example"] = profile
return cfg
},
want: []string{"example", "output", "references", "not supported"},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
err := tc.mutate(validConfig()).Validate()
if err == nil {
t.Fatal("Validate() error = nil, want error")
}
for _, want := range tc.want {
if !strings.Contains(err.Error(), want) {
t.Fatalf("Validate() error = %q, want substring %q", err.Error(), want)
}
}
})
}
}
func TestValidateRejectsEmptyIDs(t *testing.T) {
tests := []struct {
name string
mutate func(Config) Config
want string
}{
{
name: "pipeline",
mutate: func(cfg Config) Config {
cfg.Pipelines[" "] = pipeline.PipelineProfile{}
return cfg
},
want: "pipeline id",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
err := tc.mutate(validConfig()).Validate()
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("expected error containing %q, got %v", tc.want, err)
}
})
}
}
func TestValidateRejectsIDsDuplicatedAfterTrimming(t *testing.T) {
tests := []struct {
name string
mutate func(Config) Config
want string
}{
{
name: "pipeline",
mutate: func(cfg Config) Config {
cfg.Pipelines[" example "] = cfg.Pipelines["example"]
return cfg
},
want: "duplicated",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
err := tc.mutate(validConfig()).Validate()
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("expected error containing %q, got %v", tc.want, err)
}
})
}
}
func TestValidateRejectsConfiguredValidators(t *testing.T) {
cfg := validConfig()
profile := cfg.Pipelines["example"]
lane := profile.Artifacts["events"]
lane.Validators = []pipeline.ModuleBinding{pipeline.Binding("fake/validator")}
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
err := cfg.Validate()
if err == nil {
t.Fatal("Validate() error = nil, want configured validators error")
}
for _, want := range []string{"example", "events", "validators", "extract.validators", "merge.validators", "normalize.validators"} {
if !strings.Contains(err.Error(), want) {
t.Fatalf("Validate() error = %q, want substring %q", err.Error(), want)
}
}
}
func TestValidateAcceptsStageLocalValidatorOverrides(t *testing.T) {
cfg := validConfig()
profile := cfg.Pipelines["example"]
profile.Chunk.Validators = pipeline.ValidatorOverride{Set: true}
lane := profile.Artifacts["events"]
lane.Extract.Validators = pipeline.ValidatorOverride{
Set: true,
Validators: []pipeline.ModuleBinding{
pipeline.Binding("fake/validator"),
{Module: "fake/llm-validator", LLMProfile: "careful", Options: map[string]any{"threshold": 0.7}},
},
}
lane.Merge.Validators = pipeline.ValidatorOverride{Set: true}
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
if err := cfg.Validate(); err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
}
func TestValidateRejectsInvalidValidatorBindings(t *testing.T) {
tests := []struct {
name string
binding pipeline.ModuleBinding
want string
}{
{
name: "empty module",
binding: pipeline.ModuleBinding{},
want: "module must not be empty",
},
{
name: "references",
binding: pipeline.ModuleBinding{Module: "fake/validator", References: map[string]string{"roster": "./roster.txt"}},
want: "references are not supported",
},
{
name: "nested validators",
binding: pipeline.ModuleBinding{Module: "fake/validator", Validators: pipeline.ValidatorOverride{Set: true}},
want: "nested validators are not supported",
},
{
name: "retries",
binding: pipeline.ModuleBinding{Module: "fake/validator", Retries: 1},
want: "retries are not supported",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
cfg := validConfig()
profile := cfg.Pipelines["example"]
lane := profile.Artifacts["events"]
lane.Extract.Validators = pipeline.ValidatorOverride{
Set: true,
Validators: []pipeline.ModuleBinding{test.binding},
}
profile.Artifacts["events"] = lane
cfg.Pipelines["example"] = profile
err := cfg.Validate()
if err == nil || !strings.Contains(err.Error(), test.want) {
t.Fatalf("Validate() error = %v, want %q", err, test.want)
}
})
}
}
func validConfig() Config {
cfg := Default()
cfg.Pipelines["example"] = pipeline.PipelineProfile{
Input: pipeline.Binding("fake/input"),
Artifacts: map[string]pipeline.ArtifactLaneProfile{
"events": {
Extract: pipeline.Binding("fake/extract"),
},
"notes": {
Extract: pipeline.Binding("fake/extract"),
},
},
}
return cfg
}
func fakeCatalog(t *testing.T, overrides ...pipeline.ModuleSpec) pipeline.ModuleCatalog {
t.Helper()
specs := map[string]pipeline.ModuleSpec{
"fake/input": {
Key: "fake/input",
Stage: pipeline.StageInput,
Provides: []string{"source"},
},
"generic": {
Key: "generic",
Stage: pipeline.StageChunk,
Requires: []string{"source"},
Provides: []string{"chunks"},
},
"fake/extract": {
Key: "fake/extract",
Stage: pipeline.StageExtract,
Requires: []string{"chunks"},
Provides: []string{"artifact"},
},
"appendorder": {
Key: "appendorder",
Stage: pipeline.StageMerge,
Requires: []string{"artifact"},
Provides: []string{"merged"},
},
"noop": {
Key: "noop",
Stage: pipeline.StageNormalize,
Requires: []string{"merged"},
Provides: []string{"normalized"},
},
"fake/validator": {
Key: "fake/validator",
Stage: pipeline.StageValidate,
Requires: []string{"normalized"},
Provides: []string{"validated"},
},
"fake/llm-validator": {
Key: "fake/llm-validator",
Stage: pipeline.StageValidate,
Requires: []string{"normalized"},
Provides: []string{"validated"},
},
"json": {
Key: "json",
Stage: pipeline.StageOutput,
Requires: []string{"normalized"},
},
}
for _, override := range overrides {
specs[override.Key] = override
}
for _, key := range []string{"fake/extract", "appendorder", "noop"} {
spec := specs[key]
spec.ArtifactKind = fakeArtifactKind
specs[key] = spec
}
inputs := pipeline.NewInputAdapterRegistry()
chunkers := pipeline.NewChunkerRegistry()
extractors := pipeline.NewExtractorRegistry()
mergers := pipeline.NewMergerRegistry()
normalizers := pipeline.NewNormalizerRegistry()
validators := pipeline.NewValidatorRegistry()
outputs := pipeline.NewOutputEncoderRegistry()
mustRegisterInput(t, inputs, specs["fake/input"])
mustRegisterChunker(t, chunkers, specs["generic"])
mustRegisterExtractor(t, extractors, specs["fake/extract"])
mustRegisterMerger(t, mergers, specs["appendorder"])
mustRegisterNormalizer(t, normalizers, specs["noop"])
mustRegisterValidator(t, validators, specs["fake/validator"])
mustRegisterValidator(t, validators, specs["fake/llm-validator"])
mustRegisterOutput(t, outputs, specs["json"])
codecs := pipeline.NewArtifactCodecRegistry()
if err := pipeline.RegisterArtifactCodec(codecs, fakeArtifactCodec{}); err != nil {
t.Fatalf("register artifact codec: %v", err)
}
return pipeline.ModuleCatalog{
Inputs: inputs,
Chunkers: chunkers,
ArtifactCodecs: codecs,
Extractors: extractors,
Mergers: mergers,
Normalizers: normalizers,
Validators: validators,
ValidatorChains: pipeline.NewValidatorChainRegistry(),
Outputs: outputs,
}
}
func mustRegisterInput(t *testing.T, registry *pipeline.InputAdapterRegistry, spec pipeline.ModuleSpec) {
t.Helper()
if err := registry.RegisterWithSpec(spec, func() (contracts.InputAdapter, error) { return nil, nil }); err != nil {
t.Fatalf("register input: %v", err)
}
}
func mustRegisterChunker(t *testing.T, registry *pipeline.ChunkerRegistry, spec pipeline.ModuleSpec) {
t.Helper()
if err := registry.RegisterWithSpec(spec, func() (contracts.Chunker, error) { return nil, nil }); err != nil {
t.Fatalf("register chunker: %v", err)
}
}
func mustRegisterExtractor(t *testing.T, registry *pipeline.ExtractorRegistry, spec pipeline.ModuleSpec) {
t.Helper()
validateOptions := func(options map[string]any) error {
if err := pipeline.RejectUnknownOptions(options, "temperature"); err != nil {
return err
}
if value, ok := options["temperature"]; ok {
if _, ok := value.(float64); !ok {
return fmt.Errorf("temperature must be a number")
}
}
return nil
}
if err := pipeline.RegisterExtractorBuilder[fakeArtifact](registry, spec, validateOptions, func(pipeline.BuildRequest) (contracts.Extractor[fakeArtifact], error) { return nil, nil }); err != nil {
t.Fatalf("register extractor: %v", err)
}
}
func mustRegisterMerger(t *testing.T, registry *pipeline.MergerRegistry, spec pipeline.ModuleSpec) {
t.Helper()
if err := pipeline.RegisterMerger[fakeArtifact](registry, spec, func() (contracts.Merger[fakeArtifact], error) { return nil, nil }); err != nil {
t.Fatalf("register merger: %v", err)
}
}
func mustRegisterNormalizer(t *testing.T, registry *pipeline.NormalizerRegistry, spec pipeline.ModuleSpec) {
t.Helper()
if err := pipeline.RegisterNormalizer[fakeArtifact](registry, spec, func() (contracts.Normalizer[fakeArtifact], error) { return nil, nil }); err != nil {
t.Fatalf("register normalizer: %v", err)
}
}
func mustRegisterValidator(t *testing.T, registry *pipeline.ValidatorRegistry, spec pipeline.ModuleSpec) {
t.Helper()
executionClass := contracts.ExecutionClassDeterministic
if spec.Key == "fake/llm-validator" {
executionClass = contracts.ExecutionClassLLMBacked
}
validatorSpec := pipeline.ValidatorSpec{Key: spec.Key, ExecutionClass: executionClass}
if err := pipeline.RegisterTypedValidator[fakeArtifact](registry, fakeArtifactKind, validatorSpec, func() (contracts.TypedValidator[fakeArtifact], error) { return nil, nil }); err != nil {
t.Fatalf("register validator: %v", err)
}
}
const fakeArtifactKind contracts.ArtifactKind = "test/artifact"
type fakeArtifact string
type fakeArtifactCodec struct{}
func (fakeArtifactCodec) Kind() contracts.ArtifactKind { return fakeArtifactKind }
func (fakeArtifactCodec) Schema() contracts.ArtifactSchema {
return contracts.ArtifactSchema{ID: "urn:notarius:test:artifact", Name: "Test artifact", Version: "1", JSONSchema: []byte(`{"type":"string"}`)}
}
func (fakeArtifactCodec) MediaType() string { return "application/json" }
func (fakeArtifactCodec) EncodeCandidate(value fakeArtifact) ([]byte, error) {
return []byte(fmt.Sprintf("%q", value)), nil
}
func (fakeArtifactCodec) Encode(value fakeArtifact) ([]byte, error) {
return []byte(fmt.Sprintf("%q", value)), nil
}
func (fakeArtifactCodec) Decode(content []byte) (fakeArtifact, error) {
if len(content) < 2 {
return "", fmt.Errorf("invalid test artifact")
}
return fakeArtifact(content[1 : len(content)-1]), nil
}
func mustRegisterOutput(t *testing.T, registry *pipeline.OutputEncoderRegistry, spec pipeline.ModuleSpec) {
t.Helper()
if err := registry.RegisterWithSpec(spec, func() (contracts.OutputEncoder, error) { return nil, nil }); err != nil {
t.Fatalf("register output: %v", err)
}
}

89
internal/core/debugbundle/bundle.go Normal file
View File

@@ -0,0 +1,89 @@
// Package debugbundle owns explicitly requested per-run debug bundles.
package debugbundle
import (
"fmt"
"os"
"path/filepath"
"strings"
"time"
)
const maxCreateAttempts = 16
var utcNow = func() time.Time { return time.Now().UTC() }
type Bundle struct {
path, summaryRoot, traceRoot string
createdAt time.Time
}
func Allocate(parent string) (*Bundle, error) {
parent = strings.TrimSpace(parent)
if parent == "" {
return nil, fmt.Errorf("debug parent must not be empty")
}
if err := os.MkdirAll(parent, 0o700); err != nil {
return nil, fmt.Errorf("create debug parent %q: %w", parent, err)
}
var last string
for attempt := 0; attempt < maxCreateAttempts; attempt++ {
createdAt := utcNow()
runID := fmt.Sprintf("run-%d", createdAt.UnixNano())
path := filepath.Join(parent, runID)
last = path
if err := os.Mkdir(path, 0o700); err != nil {
if os.IsExist(err) {
continue
}
return nil, fmt.Errorf("create debug bundle %q: %w", path, err)
}
summary, trace := filepath.Join(path, "summary"), filepath.Join(path, "trace")
if err := os.Mkdir(summary, 0o700); err != nil {
_ = os.Remove(path)
return nil, fmt.Errorf("create debug summary %q: %w", summary, err)
}
if err := os.Mkdir(trace, 0o700); err != nil {
_ = os.RemoveAll(path)
return nil, fmt.Errorf("create debug trace %q: %w", trace, err)
}
return &Bundle{path: path, summaryRoot: summary, traceRoot: trace, createdAt: createdAt}, nil
}
return nil, fmt.Errorf("create debug bundle %q: exhausted unique run ID attempts", last)
}
func (b *Bundle) Path() string {
if b == nil {
return ""
}
return b.path
}
func (b *Bundle) SummaryRoot() string {
if b == nil {
return ""
}
return b.summaryRoot
}
func (b *Bundle) TraceRoot() string {
if b == nil {
return ""
}
return b.traceRoot
}
func (b *Bundle) RunID() string {
if b == nil {
return ""
}
return filepath.Base(b.path)
}
func (b *Bundle) CreatedAt() time.Time {
if b == nil {
return time.Time{}
}
return b.createdAt
}
func (b *Bundle) Summary() *SummaryWriter {
if b == nil {
return nil
}
return &SummaryWriter{root: b.summaryRoot, runID: b.RunID(), createdAt: b.createdAt}
}

75
internal/core/debugbundle/bundle_test.go Normal file
View File

@@ -0,0 +1,75 @@
package debugbundle
import (
"os"
"path/filepath"
"testing"
"time"
)
func TestAllocateCreatesRestrictiveSummaryAndTrace(t *testing.T) {
parent := t.TempDir()
fixed := time.Unix(0, 42).UTC()
previous := utcNow
utcNow = func() time.Time { return fixed }
defer func() { utcNow = previous }()
bundle, err := Allocate(parent)
if err != nil {
t.Fatal(err)
}
if bundle.RunID() != "run-42" || bundle.SummaryRoot() != filepath.Join(bundle.Path(), "summary") || bundle.TraceRoot() != filepath.Join(bundle.Path(), "trace") {
t.Fatalf("bundle=%#v", bundle)
}
for _, path := range []string{bundle.Path(), bundle.SummaryRoot(), bundle.TraceRoot()} {
info, err := os.Stat(path)
if err != nil {
t.Fatal(err)
}
if info.Mode().Perm() != 0o700 {
t.Fatalf("%s mode=%#o", path, info.Mode().Perm())
}
}
if err := bundle.Summary().WriteError("failed"); err != nil {
t.Fatal(err)
}
info, err := os.Stat(filepath.Join(bundle.SummaryRoot(), ArtifactErrorLog))
if err != nil {
t.Fatal(err)
}
if info.Mode().Perm() != 0o600 {
t.Fatalf("file mode=%#o", info.Mode().Perm())
}
}
func TestAllocateRetriesAndDoesNotDeleteBundle(t *testing.T) {
parent := t.TempDir()
fixed := time.Unix(0, 9).UTC()
previous := utcNow
defer func() { utcNow = previous }()
calls := 0
utcNow = func() time.Time { calls++; return fixed.Add(time.Duration(calls-1) * time.Nanosecond) }
if err := os.Mkdir(filepath.Join(parent, "run-9"), 0o700); err != nil {
t.Fatal(err)
}
bundle, err := Allocate(parent)
if err != nil {
t.Fatal(err)
}
if bundle.RunID() != "run-10" {
t.Fatalf("run id=%q", bundle.RunID())
}
if _, err := os.Stat(bundle.Path()); err != nil {
t.Fatal(err)
}
}
func TestSummaryWriterConfinesArtifacts(t *testing.T) {
bundle, err := Allocate(t.TempDir())
if err != nil {
t.Fatal(err)
}
if err := bundle.Summary().WriteJSON("../outside.json", map[string]any{}); err == nil {
t.Fatal("accepted traversal")
}
if err := bundle.Summary().WriteBytes(`trace\\x`, []byte("x")); err == nil {
t.Fatal("accepted backslash")
}
}

120
internal/core/debugbundle/summary.go Normal file
View File

@@ -0,0 +1,120 @@
package debugbundle
import (
"fmt"
"strings"
"time"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/fileio"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
const (
ArtifactInvocationMetadata = "invocation.json"
ArtifactEffectiveConfig = "effective-config.json"
ArtifactResolvedPipeline = "resolved-pipeline.json"
ArtifactResolvedReferences = "resolved-references.json"
ArtifactCheckpointEvents = "checkpoint-events.json"
ArtifactRunManifest = "run-manifest.json"
ArtifactChunkPlan = "chunk-plan.json"
ArtifactRunReport = "run-report.json"
ArtifactWarnings = "warnings.json"
ArtifactErrorLog = "error.log"
)
type RedactedSummaryPayload interface{ RedactedSummaryPayload() any }
type Invocation struct {
Operation string `json:"operation"`
PipelineID string `json:"pipeline_id,omitempty"`
PipelineDigest string `json:"pipeline_digest,omitempty"`
Resume bool `json:"resume,omitempty"`
InputPath string `json:"input_path,omitempty"`
ConfigPath string `json:"config_path,omitempty"`
ConfigSource string `json:"config_source,omitempty"`
OnlyLanes []string `json:"only_lanes,omitempty"`
ChunkCacheOverride string `json:"chunk_cache_override,omitempty"`
RunID string `json:"run_id"`
StartedAt time.Time `json:"started_at"`
}
type RunReport struct {
RunID string `json:"run_id"`
PipelineID string `json:"pipeline_id"`
OutputPath string `json:"output_path,omitempty"`
DebugPath string `json:"debug_path,omitempty"`
Succeeded bool `json:"succeeded"`
OutputCount int `json:"output_count"`
RejectedCount int `json:"rejected_count"`
WarningCount int `json:"warning_count"`
ValidationStatus string `json:"validation_status,omitempty"`
}
type SummaryWriter struct {
root, runID string
createdAt time.Time
}
func (w *SummaryWriter) WriteInvocation(payload Invocation) error {
if w == nil {
return fmt.Errorf("debug summary writer must not be nil")
}
if payload.RunID == "" {
payload.RunID = w.runID
}
if payload.StartedAt.IsZero() {
payload.StartedAt = w.createdAt
}
return w.WriteJSON(ArtifactInvocationMetadata, payload)
}
func (w *SummaryWriter) WriteRedactedEffectiveConfig(payload RedactedSummaryPayload) error {
if payload == nil {
return fmt.Errorf("redacted summary payload must not be nil")
}
return w.WriteJSON(ArtifactEffectiveConfig, payload.RedactedSummaryPayload())
}
func (w *SummaryWriter) WriteResolvedPipeline(v any) error {
return w.WriteJSON(ArtifactResolvedPipeline, v)
}
func (w *SummaryWriter) WriteResolvedReferences(v any) error {
return w.WriteJSON(ArtifactResolvedReferences, v)
}
func (w *SummaryWriter) WriteCheckpointEvents(v any) error {
return w.WriteJSON(ArtifactCheckpointEvents, v)
}
func (w *SummaryWriter) WriteRunManifest(v artifacts.RunManifest) error {
return w.WriteJSON(ArtifactRunManifest, v)
}
func (w *SummaryWriter) WriteChunkPlan(v artifacts.ChunkPlanSummary) error {
return w.WriteJSON(ArtifactChunkPlan, v)
}
func (w *SummaryWriter) WriteRunReport(v RunReport) error { return w.WriteJSON(ArtifactRunReport, v) }
func (w *SummaryWriter) WriteWarnings(v []contracts.Warning) error {
return w.WriteJSON(ArtifactWarnings, v)
}
func (w *SummaryWriter) WriteError(message string) error {
return w.WriteBytes(ArtifactErrorLog, []byte(message+"\n"))
}
func (w *SummaryWriter) WriteJSON(name string, v any) error {
if w == nil {
return fmt.Errorf("debug summary writer must not be nil")
}
if err := fileio.WriteJSON(w.root, summaryName(name), v, 0o700, 0o600); err != nil {
return fmt.Errorf("write debug summary artifact %q: %w", name, err)
}
return nil
}
func (w *SummaryWriter) WriteBytes(name string, v []byte) error {
if w == nil {
return fmt.Errorf("debug summary writer must not be nil")
}
if err := fileio.WriteBytes(w.root, summaryName(name), v, 0o700, 0o600); err != nil {
return fmt.Errorf("write debug summary artifact %q: %w", name, err)
}
return nil
}
func summaryName(name string) string {
name = strings.TrimSpace(name)
if name == "" || strings.ContainsAny(name, "/\\") {
return "../invalid"
}
return name
}

15
internal/core/diagnostics/artifacts.go
View File

@@ -1,15 +0,0 @@
package diagnostics
const (
ArtifactInvocationMetadata = "invocation.json"
ArtifactEffectiveConfig = "effective-config.json"
ArtifactResolvedPipeline = "resolved-pipeline.json"
ArtifactResolvedReferences = "resolved-references.json"
ArtifactCheckpointEvents = "checkpoint-events.json"
ArtifactSourceDocument = "source-document.json"
ArtifactRunManifest = "run-manifest.json"
ArtifactChunkPlan = "chunk-plan.json"
ArtifactRunReport = "run-report.json"
ArtifactWarnings = "warnings.json"
ArtifactErrorLog = "error.log"
)

24
internal/core/diagnostics/artifacts_test.go
View File

@@ -1,24 +0,0 @@
package diagnostics
import "testing"
func TestArtifactNamesUseExtractionOrientedNames(t *testing.T) {
names := []string{
ArtifactInvocationMetadata,
ArtifactEffectiveConfig,
ArtifactResolvedPipeline,
ArtifactResolvedReferences,
ArtifactSourceDocument,
ArtifactRunManifest,
ArtifactChunkPlan,
ArtifactRunReport,
ArtifactWarnings,
ArtifactErrorLog,
}
for _, name := range names {
if name == "" {
t.Fatalf("artifact name must not be empty")
}
}
}

293
internal/core/diagnostics/run_dir.go
View File

@@ -1,293 +0,0 @@
package diagnostics
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"time"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
const (
defaultWorkDir = "/tmp/notarius"
maxRunDirectoryCreateAttempts = 16
)
var utcNow = func() time.Time {
return time.Now().UTC()
}
// RunDirectory represents a per-run diagnostics directory.
type RunDirectory struct {
path string
retention RetentionMode
createdAt time.Time
}
type RetentionMode string
const (
RetentionAuto RetentionMode = "auto"
RetentionAlways RetentionMode = "always"
RetentionNever RetentionMode = "never"
)
type RetentionDecisionInput struct {
RetentionMode RetentionMode
RunSucceeded bool
HasWarnings bool
}
type RedactedEffectiveConfigPayload interface {
RedactedDiagnosticsPayload() any
}
// InvocationMetadata captures non-secret invocation details for diagnostics.
type InvocationMetadata struct {
Operation string `json:"operation"`
PipelineID string `json:"pipeline_id,omitempty"`
PipelineDigest string `json:"pipeline_digest,omitempty"`
Resume bool `json:"resume,omitempty"`
InputPath string `json:"input_path,omitempty"`
ConfigPath string `json:"config_path,omitempty"`
ConfigSource string `json:"config_source,omitempty"`
OnlyLanes []string `json:"only_lanes,omitempty"`
ChunkCacheOverride string `json:"chunk_cache_override,omitempty"`
RunID string `json:"run_id"`
StartedAt time.Time `json:"started_at"`
}
func ShouldRetainRunDirectory(input RetentionDecisionInput) bool {
if !input.RunSucceeded {
return true
}
switch input.RetentionMode {
case RetentionAlways:
return true
case RetentionNever:
return false
case RetentionAuto, "":
return input.HasWarnings
default:
return true
}
}
func NewRunDirectory(workDir string, retention RetentionMode) (*RunDirectory, error) {
if strings.TrimSpace(workDir) == "" {
workDir = defaultWorkDir
}
if retention == "" {
retention = RetentionAuto
}
if err := os.MkdirAll(workDir, 0o755); err != nil {
return nil, fmt.Errorf("create diagnostics work directory %q: %w", workDir, err)
}
var lastRunPath string
for attempt := 0; attempt < maxRunDirectoryCreateAttempts; attempt++ {
createdAt := utcNow()
runID := fmt.Sprintf("run-%d", createdAt.UnixNano())
runPath := filepath.Join(workDir, runID)
lastRunPath = runPath
if err := os.Mkdir(runPath, 0o755); err != nil {
if os.IsExist(err) {
continue
}
return nil, fmt.Errorf("create diagnostics run directory %q: %w", runPath, err)
}
return &RunDirectory{
path: runPath,
retention: retention,
createdAt: createdAt,
}, nil
}
return nil, fmt.Errorf("create diagnostics run directory %q: exhausted unique run ID attempts", lastRunPath)
}
func (r *RunDirectory) Path() string {
if r == nil {
return ""
}
return r.path
}
func (r *RunDirectory) RunID() string {
if r == nil {
return ""
}
return filepath.Base(r.path)
}
func (r *RunDirectory) WriteInvocationMetadata(metadata InvocationMetadata) error {
if r == nil {
return fmt.Errorf("run directory must not be nil")
}
if metadata.RunID == "" {
metadata.RunID = r.RunID()
}
if metadata.StartedAt.IsZero() {
metadata.StartedAt = r.createdAt
}
return r.WriteJSONArtifact(ArtifactInvocationMetadata, metadata)
}
func (r *RunDirectory) WriteRedactedEffectiveConfig(payload RedactedEffectiveConfigPayload) error {
if payload == nil {
return fmt.Errorf("redacted effective config payload must not be nil")
}
return r.WriteJSONArtifact(ArtifactEffectiveConfig, payload.RedactedDiagnosticsPayload())
}
func (r *RunDirectory) WriteResolvedPipeline(payload any) error {
return r.WriteJSONArtifact(ArtifactResolvedPipeline, payload)
}
func (r *RunDirectory) WriteResolvedReferences(payload any) error {
return r.WriteJSONArtifact(ArtifactResolvedReferences, payload)
}
func (r *RunDirectory) WriteCheckpointEvents(payload any) error {
return r.WriteJSONArtifact(ArtifactCheckpointEvents, payload)
}
func (r *RunDirectory) WriteSourceDocument(payload any) error {
return r.WriteJSONArtifact(ArtifactSourceDocument, payload)
}
func (r *RunDirectory) WriteRunManifest(manifest artifacts.RunManifest) error {
return r.WriteJSONArtifact(ArtifactRunManifest, manifest)
}
func (r *RunDirectory) WriteChunkPlan(summary artifacts.ChunkPlanSummary) error {
return r.WriteJSONArtifact(ArtifactChunkPlan, summary)
}
func (r *RunDirectory) WriteRunReport(payload any) error {
return r.WriteJSONArtifact(ArtifactRunReport, payload)
}
func (r *RunDirectory) WriteWarnings(warnings []contracts.Warning) error {
return r.WriteJSONArtifact(ArtifactWarnings, warnings)
}
func (r *RunDirectory) WriteErrorLog(errorMessage string) error {
if r == nil {
return fmt.Errorf("run directory must not be nil")
}
path, err := r.artifactPath(ArtifactErrorLog)
if err != nil {
return err
}
if err := writeFileAtomic(path, []byte(errorMessage+"\n"), 0o644); err != nil {
return fmt.Errorf("write diagnostics artifact %q: %w", ArtifactErrorLog, err)
}
return nil
}
func (r *RunDirectory) WriteJSONArtifact(name string, payload any) error {
if r == nil {
return fmt.Errorf("run directory must not be nil")
}
path, err := r.artifactPath(name)
if err != nil {
return err
}
data, err := json.MarshalIndent(payload, "", " ")
if err != nil {
return fmt.Errorf("marshal diagnostics artifact %q: %w", name, err)
}
data = append(data, '\n')
if err := writeFileAtomic(path, data, 0o644); err != nil {
return fmt.Errorf("write diagnostics artifact %q: %w", name, err)
}
return nil
}
func (r *RunDirectory) ApplyRetention(input RetentionDecisionInput) error {
if r == nil {
return fmt.Errorf("run directory must not be nil")
}
decision := input
if decision.RetentionMode == "" {
decision.RetentionMode = r.retention
}
if ShouldRetainRunDirectory(decision) {
return nil
}
if err := os.RemoveAll(r.path); err != nil {
return fmt.Errorf("remove diagnostics run directory %q: %w", r.path, err)
}
return nil
}
func (r *RunDirectory) artifactPath(name string) (string, error) {
name = strings.TrimSpace(name)
if name == "" {
return "", fmt.Errorf("diagnostics artifact name must not be empty")
}
if filepath.IsAbs(name) {
return "", fmt.Errorf("diagnostics artifact name %q must not be absolute", name)
}
if name != filepath.Base(name) || strings.Contains(name, "/") || strings.Contains(name, `\`) {
return "", fmt.Errorf("diagnostics artifact name %q must not contain path separators", name)
}
runPath, err := filepath.Abs(r.path)
if err != nil {
return "", fmt.Errorf("resolve diagnostics run directory %q: %w", r.path, err)
}
artifactPath, err := filepath.Abs(filepath.Join(runPath, name))
if err != nil {
return "", fmt.Errorf("resolve diagnostics artifact %q: %w", name, err)
}
if filepath.Dir(artifactPath) != runPath {
return "", fmt.Errorf("diagnostics artifact name %q resolves outside run directory", name)
}
return artifactPath, nil
}
func writeFileAtomic(path string, data []byte, perm os.FileMode) error {
dir := filepath.Dir(path)
if err := os.MkdirAll(dir, 0o755); err != nil {
return err
}
temp, err := os.CreateTemp(dir, "."+filepath.Base(path)+".tmp-*")
if err != nil {
return err
}
tempPath := temp.Name()
removeTemp := true
defer func() {
if removeTemp {
_ = os.Remove(tempPath)
}
}()
if _, err := temp.Write(data); err != nil {
_ = temp.Close()
return err
}
if err := temp.Chmod(perm); err != nil {
_ = temp.Close()
return err
}
if err := temp.Close(); err != nil {
return err
}
if err := os.Rename(tempPath, path); err != nil {
return err
}
removeTemp = false
return nil
}

383
internal/core/diagnostics/run_dir_test.go
View File

@@ -1,383 +0,0 @@
package diagnostics
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"testing"
"time"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
func TestNewRunDirectoryCreatesRunDirectoryAndRunID(t *testing.T) {
workDir := t.TempDir()
runDir, err := NewRunDirectory(workDir, RetentionAuto)
if err != nil {
t.Fatalf("NewRunDirectory: %v", err)
}
if filepath.Dir(runDir.Path()) != workDir {
t.Fatalf("unexpected run directory parent: %q", runDir.Path())
}
if ok := regexp.MustCompile(`^run-\d+$`).MatchString(runDir.RunID()); !ok {
t.Fatalf("unexpected run ID: %q", runDir.RunID())
}
info, err := os.Stat(runDir.Path())
if err != nil {
t.Fatalf("stat run directory: %v", err)
}
if !info.IsDir() {
t.Fatalf("expected run path to be a directory")
}
}
func TestNewRunDirectoryRetriesOnRunIDCollision(t *testing.T) {
workDir := t.TempDir()
first := time.Unix(0, 100).UTC()
second := first.Add(time.Nanosecond)
if err := os.Mkdir(filepath.Join(workDir, fmt.Sprintf("run-%d", first.UnixNano())), 0o755); err != nil {
t.Fatalf("create existing run directory: %v", err)
}
restoreUTCNow := replaceUTCNow(func() func() time.Time {
calls := 0
return func() time.Time {
calls++
if calls == 1 {
return first
}
return second
}
}())
t.Cleanup(restoreUTCNow)
runDir, err := NewRunDirectory(workDir, RetentionAuto)
if err != nil {
t.Fatalf("NewRunDirectory: %v", err)
}
wantRunID := fmt.Sprintf("run-%d", second.UnixNano())
if runDir.RunID() != wantRunID {
t.Fatalf("RunID = %q, want %q", runDir.RunID(), wantRunID)
}
if _, err := os.Stat(runDir.Path()); err != nil {
t.Fatalf("stat run directory: %v", err)
}
}
func TestNewRunDirectoryReturnsErrorAfterRunIDCollisionsExhausted(t *testing.T) {
workDir := t.TempDir()
collisionTime := time.Unix(0, 200).UTC()
collisionPath := filepath.Join(workDir, fmt.Sprintf("run-%d", collisionTime.UnixNano()))
if err := os.Mkdir(collisionPath, 0o755); err != nil {
t.Fatalf("create existing run directory: %v", err)
}
restoreUTCNow := replaceUTCNow(func() time.Time {
return collisionTime
})
t.Cleanup(restoreUTCNow)
_, err := NewRunDirectory(workDir, RetentionAuto)
if err == nil || !strings.Contains(err.Error(), "exhausted unique run ID attempts") {
t.Fatalf("expected exhausted collision error, got %v", err)
}
}
func TestNewRunDirectoryUsesDefaultWorkDirectory(t *testing.T) {
runDir, err := NewRunDirectory("", RetentionAuto)
if err != nil {
t.Fatalf("NewRunDirectory: %v", err)
}
t.Cleanup(func() {
_ = os.RemoveAll(runDir.Path())
_ = os.Remove(defaultWorkDir)
})
if filepath.Dir(runDir.Path()) != defaultWorkDir {
t.Fatalf("expected default work directory %q, got %q", defaultWorkDir, filepath.Dir(runDir.Path()))
}
}
func TestWriteJSONArtifactWritesIndentedNewlineTerminatedJSON(t *testing.T) {
runDir := newTestRunDirectory(t)
if err := runDir.WriteJSONArtifact("artifact.json", map[string]any{"value": "ok"}); err != nil {
t.Fatalf("WriteJSONArtifact: %v", err)
}
data := readArtifact(t, runDir, "artifact.json")
if !strings.HasSuffix(string(data), "\n") {
t.Fatalf("expected trailing newline, got %q", data)
}
if !strings.Contains(string(data), "\n \"value\": \"ok\"\n") {
t.Fatalf("expected indented JSON, got %s", data)
}
}
func TestWriteJSONArtifactLeavesNoTemporaryFiles(t *testing.T) {
runDir := newTestRunDirectory(t)
if err := runDir.WriteJSONArtifact("artifact.json", map[string]any{"value": "ok"}); err != nil {
t.Fatalf("WriteJSONArtifact: %v", err)
}
entries, err := os.ReadDir(runDir.Path())
if err != nil {
t.Fatalf("read run directory: %v", err)
}
for _, entry := range entries {
if strings.Contains(entry.Name(), ".tmp-") {
t.Fatalf("temporary diagnostics file remains after success: %s", entry.Name())
}
}
}
func TestWriteInvocationMetadataFillsMissingRunIDAndStartTime(t *testing.T) {
runDir := newTestRunDirectory(t)
if err := runDir.WriteInvocationMetadata(InvocationMetadata{Operation: "validate"}); err != nil {
t.Fatalf("WriteInvocationMetadata: %v", err)
}
var got InvocationMetadata
if err := json.Unmarshal(readArtifact(t, runDir, ArtifactInvocationMetadata), &got); err != nil {
t.Fatalf("unmarshal invocation metadata: %v", err)
}
if got.RunID != runDir.RunID() {
t.Fatalf("unexpected run ID: got %q want %q", got.RunID, runDir.RunID())
}
if got.StartedAt.IsZero() {
t.Fatalf("expected started_at to be filled")
}
if got.Operation != "validate" {
t.Fatalf("unexpected operation: %q", got.Operation)
}
}
func TestWriteInvocationMetadataPreservesProvidedRunIDAndStartTime(t *testing.T) {
runDir := newTestRunDirectory(t)
startedAt := time.Date(2026, 7, 3, 12, 0, 0, 0, time.UTC)
if err := runDir.WriteInvocationMetadata(InvocationMetadata{
Operation: "validate",
RunID: "provided",
StartedAt: startedAt,
}); err != nil {
t.Fatalf("WriteInvocationMetadata: %v", err)
}
var got InvocationMetadata
if err := json.Unmarshal(readArtifact(t, runDir, ArtifactInvocationMetadata), &got); err != nil {
t.Fatalf("unmarshal invocation metadata: %v", err)
}
if got.RunID != "provided" {
t.Fatalf("unexpected run ID: %q", got.RunID)
}
if !got.StartedAt.Equal(startedAt) {
t.Fatalf("unexpected started_at: %s", got.StartedAt)
}
}
func TestWriteTypedArtifacts(t *testing.T) {
runDir := newTestRunDirectory(t)
if err := runDir.WriteRedactedEffectiveConfig(fakeRedactedEffectiveConfig{payload: map[string]any{"redacted": true}}); err != nil {
t.Fatalf("WriteRedactedEffectiveConfig: %v", err)
}
if err := runDir.WriteResolvedPipeline(map[string]any{"pipeline": "test"}); err != nil {
t.Fatalf("WriteResolvedPipeline: %v", err)
}
if err := runDir.WriteResolvedReferences([]artifacts.ReferenceProvenance{{LaneID: "events", SlotName: "roster"}}); err != nil {
t.Fatalf("WriteResolvedReferences: %v", err)
}
if err := runDir.WriteSourceDocument(map[string]any{"source_id": "source-1"}); err != nil {
t.Fatalf("WriteSourceDocument: %v", err)
}
if err := runDir.WriteRunManifest(artifacts.RunManifest{RunID: "run-1"}); err != nil {
t.Fatalf("WriteRunManifest: %v", err)
}
if err := runDir.WriteChunkPlan(artifacts.ChunkPlanSummary{Mode: "auto", RequestedModule: "chunk/test", LookupStatus: "invalid", LookupReason: "stored chunk plan failed validation", ValidationStatus: "not_run", PublicationStatus: "not_published"}); err != nil {
t.Fatalf("WriteChunkPlan: %v", err)
}
if err := runDir.WriteRunReport(map[string]any{"ok": true}); err != nil {
t.Fatalf("WriteRunReport: %v", err)
}
if err := runDir.WriteWarnings([]contracts.Warning{{ReasonCode: "test", Message: "warning"}}); err != nil {
t.Fatalf("WriteWarnings: %v", err)
}
for _, name := range []string{
ArtifactEffectiveConfig,
ArtifactResolvedPipeline,
ArtifactResolvedReferences,
ArtifactSourceDocument,
ArtifactRunManifest,
ArtifactChunkPlan,
ArtifactRunReport,
ArtifactWarnings,
} {
if _, err := os.Stat(filepath.Join(runDir.Path(), name)); err != nil {
t.Fatalf("expected artifact %q: %v", name, err)
}
}
chunkSummary, err := os.ReadFile(filepath.Join(runDir.Path(), ArtifactChunkPlan))
if err != nil {
t.Fatal(err)
}
for _, forbidden := range []string{"units", "annotations", "source content", "prompt", "response", "raw invalid"} {
if strings.Contains(string(chunkSummary), forbidden) {
t.Fatalf("chunk plan summary leaked %q: %s", forbidden, chunkSummary)
}
}
}
func TestWriteRedactedEffectiveConfigWritesPayloadReturnedByProvider(t *testing.T) {
runDir := newTestRunDirectory(t)
if err := runDir.WriteRedactedEffectiveConfig(fakeRedactedEffectiveConfig{
payload: map[string]any{
"api_key": "[REDACTED]",
"model": "test-model",
},
}); err != nil {
t.Fatalf("WriteRedactedEffectiveConfig: %v", err)
}
data := string(readArtifact(t, runDir, ArtifactEffectiveConfig))
if !strings.Contains(data, `"api_key": "[REDACTED]"`) || !strings.Contains(data, `"model": "test-model"`) {
t.Fatalf("unexpected effective config artifact: %s", data)
}
}
func TestWriteErrorLogWritesPlainTextWithTrailingNewline(t *testing.T) {
runDir := newTestRunDirectory(t)
if err := runDir.WriteErrorLog("something failed"); err != nil {
t.Fatalf("WriteErrorLog: %v", err)
}
if got := string(readArtifact(t, runDir, ArtifactErrorLog)); got != "something failed\n" {
t.Fatalf("unexpected error log: %q", got)
}
}
func TestArtifactPathRejectsUnsafeNames(t *testing.T) {
runDir := newTestRunDirectory(t)
tests := []string{
"",
" ",
"/absolute.json",
"nested/artifact.json",
`nested\artifact.json`,
"../escape.json",
}
for _, name := range tests {
t.Run(name, func(t *testing.T) {
if err := runDir.WriteJSONArtifact(name, map[string]any{}); err == nil {
t.Fatalf("expected unsafe artifact name %q to be rejected", name)
}
})
}
}
func TestShouldRetainRunDirectoryDecisions(t *testing.T) {
tests := []struct {
name string
input RetentionDecisionInput
want bool
}{
{name: "failed auto retained", input: RetentionDecisionInput{RetentionMode: RetentionAuto, RunSucceeded: false}, want: true},
{name: "failed always retained", input: RetentionDecisionInput{RetentionMode: RetentionAlways, RunSucceeded: false}, want: true},
{name: "failed never retained", input: RetentionDecisionInput{RetentionMode: RetentionNever, RunSucceeded: false}, want: true},
{name: "successful always retained", input: RetentionDecisionInput{RetentionMode: RetentionAlways, RunSucceeded: true}, want: true},
{name: "successful never removed", input: RetentionDecisionInput{RetentionMode: RetentionNever, RunSucceeded: true}, want: false},
{name: "successful auto without warnings removed", input: RetentionDecisionInput{RetentionMode: RetentionAuto, RunSucceeded: true}, want: false},
{name: "successful auto with warnings retained", input: RetentionDecisionInput{RetentionMode: RetentionAuto, RunSucceeded: true, HasWarnings: true}, want: true},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := ShouldRetainRunDirectory(tc.input); got != tc.want {
t.Fatalf("ShouldRetainRunDirectory() = %v, want %v", got, tc.want)
}
})
}
}
func TestApplyRetentionRemovesOnlyRunDirectory(t *testing.T) {
workDir := t.TempDir()
runDir, err := NewRunDirectory(workDir, RetentionNever)
if err != nil {
t.Fatalf("NewRunDirectory: %v", err)
}
siblingPath := filepath.Join(workDir, "sibling")
if err := os.WriteFile(siblingPath, []byte("keep"), 0o644); err != nil {
t.Fatalf("write sibling: %v", err)
}
if err := runDir.ApplyRetention(RetentionDecisionInput{RunSucceeded: true}); err != nil {
t.Fatalf("ApplyRetention: %v", err)
}
if _, err := os.Stat(runDir.Path()); !os.IsNotExist(err) {
t.Fatalf("expected run directory removed, stat err=%v", err)
}
if _, err := os.Stat(workDir); err != nil {
t.Fatalf("expected work directory retained: %v", err)
}
if _, err := os.Stat(siblingPath); err != nil {
t.Fatalf("expected sibling retained: %v", err)
}
}
func TestApplyRetentionKeepsRetainedRunDirectory(t *testing.T) {
runDir := newTestRunDirectory(t)
if err := runDir.ApplyRetention(RetentionDecisionInput{RunSucceeded: true, HasWarnings: true}); err != nil {
t.Fatalf("ApplyRetention: %v", err)
}
if _, err := os.Stat(runDir.Path()); err != nil {
t.Fatalf("expected run directory retained: %v", err)
}
}
func newTestRunDirectory(t *testing.T) *RunDirectory {
t.Helper()
runDir, err := NewRunDirectory(t.TempDir(), RetentionAuto)
if err != nil {
t.Fatalf("NewRunDirectory: %v", err)
}
return runDir
}
func replaceUTCNow(replacement func() time.Time) func() {
original := utcNow
utcNow = replacement
return func() {
utcNow = original
}
}
func readArtifact(t *testing.T, runDir *RunDirectory, name string) []byte {
t.Helper()
data, err := os.ReadFile(filepath.Join(runDir.Path(), name))
if err != nil {
t.Fatalf("read artifact %q: %v", name, err)
}
return data
}
type fakeRedactedEffectiveConfig struct {
payload any
}
func (f fakeRedactedEffectiveConfig) RedactedDiagnosticsPayload() any {
return f.payload
}

102
internal/core/fileio/fileio.go Normal file
View File

@@ -0,0 +1,102 @@
// Package fileio provides confined, atomic artifact writes.
package fileio
import (
"encoding/json"
"fmt"
"os"
"path"
"path/filepath"
"strings"
)
func SafePath(root, name string) (string, error) {
root = strings.TrimSpace(root)
if root == "" {
return "", fmt.Errorf("file root must not be empty")
}
name = strings.TrimSpace(name)
if name == "" {
return "", fmt.Errorf("artifact name must not be empty")
}
if strings.Contains(name, `\\`) {
return "", fmt.Errorf("artifact name %q must use slash-separated relative paths", name)
}
if path.IsAbs(name) || filepath.IsAbs(name) {
return "", fmt.Errorf("artifact name %q must be relative", name)
}
if name == "." || strings.Contains(name, "..") {
return "", fmt.Errorf("artifact name %q must not contain ..", name)
}
if path.Clean(name) != name {
return "", fmt.Errorf("artifact name %q must be clean", name)
}
absRoot, err := filepath.Abs(root)
if err != nil {
return "", fmt.Errorf("resolve file root %q: %w", root, err)
}
target, err := filepath.Abs(filepath.Join(absRoot, filepath.FromSlash(name)))
if err != nil {
return "", fmt.Errorf("resolve artifact %q: %w", name, err)
}
rel, err := filepath.Rel(absRoot, target)
if err != nil {
return "", fmt.Errorf("resolve artifact %q: %w", name, err)
}
if rel == "." || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return "", fmt.Errorf("artifact name %q resolves outside file root", name)
}
return target, nil
}
func WriteJSON(root, name string, payload any, dirMode, fileMode os.FileMode) error {
data, err := json.MarshalIndent(payload, "", " ")
if err != nil {
return fmt.Errorf("marshal artifact %q: %w", name, err)
}
return WriteBytes(root, name, append(data, '\n'), dirMode, fileMode)
}
func WriteBytes(root, name string, data []byte, dirMode, fileMode os.FileMode) error {
target, err := SafePath(root, name)
if err != nil {
return err
}
if err := writeAtomic(target, data, dirMode, fileMode); err != nil {
return fmt.Errorf("write artifact %q: %w", name, err)
}
return nil
}
func writeAtomic(target string, data []byte, dirMode, fileMode os.FileMode) error {
if err := os.MkdirAll(filepath.Dir(target), dirMode); err != nil {
return err
}
temp, err := os.CreateTemp(filepath.Dir(target), "."+filepath.Base(target)+".tmp-*")
if err != nil {
return err
}
tempPath := temp.Name()
keep := true
defer func() {
if keep {
_ = os.Remove(tempPath)
}
}()
if _, err := temp.Write(data); err != nil {
_ = temp.Close()
return err
}
if err := temp.Chmod(fileMode); err != nil {
_ = temp.Close()
return err
}
if err := temp.Close(); err != nil {
return err
}
if err := os.Rename(tempPath, target); err != nil {
return err
}
keep = false
return nil
}

41
internal/core/fileio/fileio_test.go Normal file
View File

@@ -0,0 +1,41 @@
package fileio
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestSafePathRejectsUnsafeNames(t *testing.T) {
for _, name := range []string{"/tmp/x", "a/../x", "a//x", `a\\x`} {
if _, err := SafePath(t.TempDir(), name); err == nil {
t.Fatalf("SafePath(%q) accepted unsafe path", name)
}
}
}
func TestWriteBytesIsAtomicAndUsesRequestedModes(t *testing.T) {
root := t.TempDir()
if err := WriteBytes(root, "nested/value", []byte("value"), 0o700, 0o600); err != nil {
t.Fatal(err)
}
for path, want := range map[string]os.FileMode{filepath.Join(root, "nested"): 0o700, filepath.Join(root, "nested", "value"): 0o600} {
info, err := os.Stat(path)
if err != nil {
t.Fatal(err)
}
if info.Mode().Perm() != want {
t.Fatalf("%s mode=%#o want %#o", path, info.Mode().Perm(), want)
}
}
entries, err := os.ReadDir(filepath.Join(root, "nested"))
if err != nil {
t.Fatal(err)
}
for _, entry := range entries {
if strings.Contains(entry.Name(), ".tmp-") {
t.Fatalf("temporary file remains: %s", entry.Name())
}
}
}

22
internal/core/workspace/chunk_plan.go
View File

@@ -1,22 +0,0 @@
package workspace
import (
"fmt"
"path/filepath"
"strings"
)
func DefaultChunkPlanRoot(userCacheDir func() (string, error)) (string, error) {
if userCacheDir == nil {
return "", fmt.Errorf("user cache directory resolver must not be nil")
}
root, err := userCacheDir()
if err != nil {
return "", fmt.Errorf("resolve user cache directory: %w", err)
}
root = strings.TrimSpace(root)
if root == "" {
return "", fmt.Errorf("user cache directory must not be empty")
}
return filepath.Join(filepath.Clean(root), "notarius", "chunk-plans"), nil
}

57
internal/core/workspace/chunk_plan_test.go
View File

@@ -1,57 +0,0 @@
package workspace
import (
"errors"
"path/filepath"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
)
func TestDefaultChunkPlanRoot(t *testing.T) {
got, err := DefaultChunkPlanRoot(func() (string, error) { return "/cache/user", nil })
if err != nil {
t.Fatalf("DefaultChunkPlanRoot() error = %v", err)
}
if want := filepath.Join("/cache/user", "notarius", "chunk-plans"); got != want {
t.Fatalf("root = %q, want %q", got, want)
}
}
func TestDefaultChunkPlanRootRejectsResolverFailures(t *testing.T) {
boom := errors.New("resolver failed")
if _, err := DefaultChunkPlanRoot(func() (string, error) { return "", boom }); !errors.Is(err, boom) {
t.Fatalf("resolver error = %v", err)
}
if _, err := DefaultChunkPlanRoot(func() (string, error) { return " \t ", nil }); err == nil || !strings.Contains(err.Error(), "empty") {
t.Fatalf("empty result error = %v", err)
}
if _, err := DefaultChunkPlanRoot(nil); err == nil {
t.Fatal("nil resolver error = nil")
}
}
func TestChunkPlanDirectoryIsIndependentFromWorkspaceSettings(t *testing.T) {
base := config.Default()
base.Workspace.Directory = "/workspace/one"
base.Workspace.ChunkCache.Directory = "/cache/plans"
first := FromConfig(base)
changedWorkspace := base
changedWorkspace.Workspace.Directory = "/workspace/two"
second := FromConfig(changedWorkspace)
if base.Workspace.ChunkCache.Directory != changedWorkspace.Workspace.ChunkCache.Directory {
t.Fatal("workspace directory changed chunk plan directory")
}
if first.RootDir == second.RootDir || first.CheckpointsRoot == second.CheckpointsRoot || first.DebugRoot == second.DebugRoot {
t.Fatalf("workspace settings did not follow workspace directory: %#v %#v", first, second)
}
changedCache := base
changedCache.Workspace.ChunkCache.Directory = "/cache/other"
third := FromConfig(changedCache)
if first != third {
t.Fatalf("chunk plan directory changed workspace settings: %#v %#v", first, third)
}
}

114
internal/core/workspace/files.go
View File

@@ -1,114 +0,0 @@
package workspace
import (
"encoding/json"
"fmt"
"os"
"path"
"path/filepath"
"strings"
)
func SafePath(root string, name string) (string, error) {
root = strings.TrimSpace(root)
if root == "" {
return "", fmt.Errorf("workspace root must not be empty")
}
name = strings.TrimSpace(name)
if name == "" {
return "", fmt.Errorf("workspace artifact name must not be empty")
}
if strings.Contains(name, `\`) {
return "", fmt.Errorf("workspace artifact name %q must use slash-separated relative paths", name)
}
if path.IsAbs(name) || filepath.IsAbs(name) {
return "", fmt.Errorf("workspace artifact name %q must be relative", name)
}
if name == "." || strings.Contains(name, "..") {
return "", fmt.Errorf("workspace artifact name %q must not contain ..", name)
}
cleaned := path.Clean(name)
if cleaned != name {
return "", fmt.Errorf("workspace artifact name %q must be clean", name)
}
absRoot, err := filepath.Abs(root)
if err != nil {
return "", fmt.Errorf("resolve workspace root %q: %w", root, err)
}
target, err := filepath.Abs(filepath.Join(absRoot, filepath.FromSlash(cleaned)))
if err != nil {
return "", fmt.Errorf("resolve workspace artifact %q: %w", name, err)
}
rel, err := filepath.Rel(absRoot, target)
if err != nil {
return "", fmt.Errorf("resolve workspace artifact %q: %w", name, err)
}
if rel == "." || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return "", fmt.Errorf("workspace artifact name %q resolves outside workspace root", name)
}
return target, nil
}
func WriteJSON(root string, name string, payload any) error {
target, err := SafePath(root, name)
if err != nil {
return err
}
data, err := json.MarshalIndent(payload, "", " ")
if err != nil {
return fmt.Errorf("marshal workspace artifact %q: %w", name, err)
}
data = append(data, '\n')
if err := writeFileAtomic(target, data, 0o644); err != nil {
return fmt.Errorf("write workspace artifact %q: %w", name, err)
}
return nil
}
func WriteBytes(root string, name string, data []byte) error {
target, err := SafePath(root, name)
if err != nil {
return err
}
if err := writeFileAtomic(target, data, 0o644); err != nil {
return fmt.Errorf("write workspace artifact %q: %w", name, err)
}
return nil
}
func writeFileAtomic(target string, data []byte, perm os.FileMode) error {
dir := filepath.Dir(target)
if err := os.MkdirAll(dir, 0o755); err != nil {
return err
}
temp, err := os.CreateTemp(dir, "."+filepath.Base(target)+".tmp-*")
if err != nil {
return err
}
tempPath := temp.Name()
removeTemp := true
defer func() {
if removeTemp {
_ = os.Remove(tempPath)
}
}()
if _, err := temp.Write(data); err != nil {
_ = temp.Close()
return err
}
if err := temp.Chmod(perm); err != nil {
_ = temp.Close()
return err
}
if err := temp.Close(); err != nil {
return err
}
if err := os.Rename(tempPath, target); err != nil {
return err
}
removeTemp = false
return nil
}

148
internal/core/workspace/files_test.go
View File

@@ -1,148 +0,0 @@
package workspace
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestSafePathAcceptsCleanRelativePaths(t *testing.T) {
root := t.TempDir()
got, err := SafePath(root, "source/manifest.json")
if err != nil {
t.Fatalf("SafePath: %v", err)
}
want := filepath.Join(root, "source", "manifest.json")
if got != want {
t.Fatalf("SafePath = %q, want %q", got, want)
}
}
func TestSafePathRejectsUnsafeNames(t *testing.T) {
root := t.TempDir()
tests := []struct {
name string
path string
want string
}{
{name: "empty", path: " ", want: "empty"},
{name: "absolute", path: filepath.Join(root, "artifact.json"), want: "relative"},
{name: "parent segment", path: "../artifact.json", want: ".."},
{name: "embedded parent", path: "source/../artifact.json", want: ".."},
{name: "backslash", path: `source\artifact.json`, want: "slash-separated"},
{name: "unclean", path: "source//artifact.json", want: "clean"},
{name: "dot", path: ".", want: ".."},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got, err := SafePath(root, tc.path)
if err == nil {
t.Fatalf("SafePath returned %q, want error", got)
}
if !strings.Contains(err.Error(), tc.want) {
t.Fatalf("SafePath error = %v, want containing %q", err, tc.want)
}
})
}
}
func TestSafePathRejectsEmptyRoot(t *testing.T) {
got, err := SafePath(" ", "artifact.json")
if err == nil {
t.Fatalf("SafePath returned %q, want error", got)
}
if !strings.Contains(err.Error(), "root") {
t.Fatalf("SafePath error = %v, want root error", err)
}
}
func TestSafePathDoesNotPermitEscapingRoot(t *testing.T) {
root := t.TempDir()
for _, name := range []string{
"..",
"../outside.json",
"nested/../../outside.json",
} {
t.Run(name, func(t *testing.T) {
got, err := SafePath(root, name)
if err == nil {
t.Fatalf("SafePath returned %q, want error", got)
}
})
}
}
func TestWriteJSONWritesIndentedAtomicArtifact(t *testing.T) {
root := t.TempDir()
err := WriteJSON(root, "source/manifest.json", map[string]any{
"status": "succeeded",
"count": 2,
})
if err != nil {
t.Fatalf("WriteJSON: %v", err)
}
got := string(readFile(t, filepath.Join(root, "source", "manifest.json")))
if !strings.HasSuffix(got, "\n") {
t.Fatalf("expected trailing newline, got %q", got)
}
if !strings.Contains(got, `"status": "succeeded"`) || !strings.Contains(got, `"count": 2`) {
t.Fatalf("unexpected JSON: %s", got)
}
assertNoTempFiles(t, filepath.Join(root, "source"))
}
func TestWriteBytesWritesNestedArtifact(t *testing.T) {
root := t.TempDir()
if err := WriteBytes(root, "chunk/chunks.json", []byte("payload")); err != nil {
t.Fatalf("WriteBytes: %v", err)
}
got := string(readFile(t, filepath.Join(root, "chunk", "chunks.json")))
if got != "payload" {
t.Fatalf("bytes = %q, want payload", got)
}
assertNoTempFiles(t, filepath.Join(root, "chunk"))
}
func TestWritersRejectUnsafePaths(t *testing.T) {
root := t.TempDir()
if err := WriteBytes(root, "../outside.json", []byte("payload")); err == nil {
t.Fatalf("WriteBytes accepted unsafe path")
}
if err := WriteJSON(root, `debug\trace.json`, map[string]string{"x": "y"}); err == nil {
t.Fatalf("WriteJSON accepted unsafe path")
}
if _, err := os.Stat(filepath.Join(root, "..", "outside.json")); !os.IsNotExist(err) {
t.Fatalf("outside path stat err = %v, want not exist", err)
}
}
func readFile(t *testing.T, path string) []byte {
t.Helper()
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %q: %v", path, err)
}
return data
}
func assertNoTempFiles(t *testing.T, dir string) {
t.Helper()
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatalf("read dir %q: %v", dir, err)
}
for _, entry := range entries {
if strings.Contains(entry.Name(), ".tmp-") {
t.Fatalf("temporary file was not cleaned up: %s", entry.Name())
}
}
}

276
internal/core/workspace/identity.go
View File

@@ -1,276 +0,0 @@
package workspace
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"path/filepath"
"sort"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const digestPrefixLength = 16
type Fingerprint struct {
Name string `json:"name"`
Value string `json:"value"`
}
type CheckpointIdentityInput struct {
Pipeline pipeline.ResolvedPipeline
InputKey string
RawInputDigest string
SourceDigest string
SelectedLanes []string
RuntimeOverrides []Fingerprint
References []artifacts.ReferenceProvenance
ProvenanceFingerprints []Fingerprint
}
type CheckpointIdentity struct {
Digest string `json:"digest"`
PipelineID string `json:"pipeline_id"`
PipelineDigest string `json:"pipeline_digest"`
InputKey string `json:"input_key"`
RawInputDigest string `json:"raw_input_digest,omitempty"`
SourceDigest string `json:"source_digest,omitempty"`
SelectedLanes []string `json:"selected_lanes,omitempty"`
RuntimeOverrides []Fingerprint `json:"runtime_overrides,omitempty"`
ReferenceDigests []Fingerprint `json:"reference_digests,omitempty"`
ProvenanceFingerprints []Fingerprint `json:"provenance_fingerprints,omitempty"`
}
func NewCheckpointIdentity(input CheckpointIdentityInput) (CheckpointIdentity, error) {
pipelineID := strings.TrimSpace(input.Pipeline.ID)
if pipelineID == "" {
return CheckpointIdentity{}, fmt.Errorf("checkpoint identity pipeline id must not be empty")
}
pipelineDigest := strings.TrimSpace(input.Pipeline.Digest)
if pipelineDigest == "" {
return CheckpointIdentity{}, fmt.Errorf("checkpoint identity pipeline digest must not be empty")
}
inputKey := strings.TrimSpace(input.InputKey)
if inputKey == "" {
inputKey = strings.TrimSpace(input.Pipeline.Input.Module)
}
if inputKey == "" {
return CheckpointIdentity{}, fmt.Errorf("checkpoint identity input key must not be empty")
}
rawInputDigest := strings.TrimSpace(input.RawInputDigest)
sourceDigest := strings.TrimSpace(input.SourceDigest)
if rawInputDigest == "" && sourceDigest == "" {
return CheckpointIdentity{}, fmt.Errorf("checkpoint identity raw input digest or source digest must be set")
}
identity := CheckpointIdentity{
PipelineID: pipelineID,
PipelineDigest: pipelineDigest,
InputKey: inputKey,
RawInputDigest: rawInputDigest,
SourceDigest: sourceDigest,
SelectedLanes: normalizedLanes(input.SelectedLanes, input.Pipeline.ArtifactLanes),
RuntimeOverrides: normalizeFingerprints(input.RuntimeOverrides),
ReferenceDigests: referenceFingerprints(input.References),
ProvenanceFingerprints: normalizeFingerprints(input.ProvenanceFingerprints),
}
digest, err := identityDigest(identity)
if err != nil {
return CheckpointIdentity{}, err
}
identity.Digest = digest
return identity, nil
}
func (s Settings) CheckpointDirectory(identity CheckpointIdentity) (string, error) {
if !s.ResumeEnabled || strings.TrimSpace(s.CheckpointsRoot) == "" {
return "", nil
}
relative, err := identity.RelativePath()
if err != nil {
return "", err
}
return SafePath(s.CheckpointsRoot, relative)
}
func (i CheckpointIdentity) RelativePath() (string, error) {
pipelineID, err := safePathComponent(i.PipelineID)
if err != nil {
return "", fmt.Errorf("checkpoint identity pipeline id: %w", err)
}
inputKey, err := safePathComponent(i.InputKey)
if err != nil {
return "", fmt.Errorf("checkpoint identity input key: %w", err)
}
sourceDigest := digestPrefix(i.SourceDigest)
if sourceDigest == "" {
sourceDigest = digestPrefix(i.RawInputDigest)
}
if sourceDigest == "" {
return "", fmt.Errorf("checkpoint identity source digest prefix must not be empty")
}
pipelineDigest := digestPrefix(i.PipelineDigest)
if pipelineDigest == "" {
return "", fmt.Errorf("checkpoint identity pipeline digest prefix must not be empty")
}
sourceComponent, err := safePathComponent(sourceDigest)
if err != nil {
return "", fmt.Errorf("checkpoint identity source digest: %w", err)
}
pipelineComponent, err := safePathComponent(pipelineDigest)
if err != nil {
return "", fmt.Errorf("checkpoint identity pipeline digest: %w", err)
}
identityDigest := digestPrefix(i.Digest)
if identityDigest == "" {
return "", fmt.Errorf("checkpoint identity digest prefix must not be empty")
}
identityComponent, err := safePathComponent(identityDigest)
if err != nil {
return "", fmt.Errorf("checkpoint identity digest: %w", err)
}
return filepath.ToSlash(filepath.Join(pipelineID, inputKey+"-"+sourceComponent, pipelineComponent, identityComponent)), nil
}
func identityDigest(identity CheckpointIdentity) (string, error) {
payload := identity
payload.Digest = ""
data, err := json.Marshal(payload)
if err != nil {
return "", fmt.Errorf("marshal checkpoint identity: %w", err)
}
sum := sha256.Sum256(data)
return "sha256:" + hex.EncodeToString(sum[:]), nil
}
func normalizedLanes(selected []string, resolved []pipeline.ResolvedArtifactLane) []string {
if len(selected) > 0 {
return normalizeStrings(selected)
}
lanes := make([]string, 0, len(resolved))
for _, lane := range resolved {
lanes = append(lanes, lane.ID)
}
return normalizeStrings(lanes)
}
func normalizeFingerprints(values []Fingerprint) []Fingerprint {
if len(values) == 0 {
return nil
}
byName := make(map[string]string, len(values))
for _, value := range values {
name := strings.TrimSpace(value.Name)
fingerprint := strings.TrimSpace(value.Value)
if name == "" || fingerprint == "" {
continue
}
byName[name] = fingerprint
}
if len(byName) == 0 {
return nil
}
names := make([]string, 0, len(byName))
for name := range byName {
names = append(names, name)
}
sort.Strings(names)
out := make([]Fingerprint, 0, len(names))
for _, name := range names {
out = append(out, Fingerprint{Name: name, Value: byName[name]})
}
return out
}
func referenceFingerprints(references []artifacts.ReferenceProvenance) []Fingerprint {
if len(references) == 0 {
return nil
}
values := make([]Fingerprint, 0, len(references))
for _, reference := range references {
digest := strings.TrimSpace(reference.Digest)
if digest == "" {
continue
}
parts := []string{
strings.TrimSpace(reference.Stage),
strings.TrimSpace(reference.LaneID),
strings.TrimSpace(reference.SlotName),
strings.TrimSpace(reference.OriginURI),
}
values = append(values, Fingerprint{
Name: strings.Join(parts, ":"),
Value: digest,
})
}
return normalizeFingerprints(values)
}
func normalizeStrings(values []string) []string {
if len(values) == 0 {
return nil
}
seen := make(map[string]struct{}, len(values))
for _, value := range values {
value = strings.TrimSpace(value)
if value == "" {
continue
}
seen[value] = struct{}{}
}
if len(seen) == 0 {
return nil
}
out := make([]string, 0, len(seen))
for value := range seen {
out = append(out, value)
}
sort.Strings(out)
return out
}
func digestPrefix(digest string) string {
digest = strings.TrimSpace(digest)
if digest == "" {
return ""
}
if idx := strings.Index(digest, ":"); idx >= 0 {
digest = digest[idx+1:]
}
digest = strings.TrimSpace(digest)
if len(digest) > digestPrefixLength {
return digest[:digestPrefixLength]
}
return digest
}
func safePathComponent(value string) (string, error) {
value = strings.TrimSpace(value)
if value == "" {
return "", fmt.Errorf("must not be empty")
}
var b strings.Builder
for _, r := range value {
switch {
case r >= 'a' && r <= 'z':
b.WriteRune(r)
case r >= 'A' && r <= 'Z':
b.WriteRune(r)
case r >= '0' && r <= '9':
b.WriteRune(r)
case r == '-' || r == '_' || r == '.':
b.WriteRune(r)
default:
b.WriteString(fmt.Sprintf("~%x", r))
}
}
encoded := b.String()
if encoded == "." || encoded == ".." || strings.Contains(encoded, "..") || strings.ContainsAny(encoded, `/\`) {
return "", fmt.Errorf("%q is not filesystem safe", value)
}
return encoded, nil
}

266
internal/core/workspace/identity_test.go
View File

@@ -1,266 +0,0 @@
package workspace
import (
"path/filepath"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestCheckpointIdentityIsDeterministic(t *testing.T) {
first := mustIdentity(t, identityInput())
second := mustIdentity(t, identityInput())
if first.Digest != second.Digest {
t.Fatalf("digest changed for same input: %q != %q", first.Digest, second.Digest)
}
if !strings.HasPrefix(first.Digest, "sha256:") {
t.Fatalf("digest = %q, want sha256 prefix", first.Digest)
}
}
func TestCheckpointIdentityChangesWhenInputsChange(t *testing.T) {
base := mustIdentity(t, identityInput())
tests := []struct {
name string
mutate func(CheckpointIdentityInput) CheckpointIdentityInput
}{
{
name: "pipeline digest",
mutate: func(input CheckpointIdentityInput) CheckpointIdentityInput {
input.Pipeline.Digest = "sha256:pipeline-b"
return input
},
},
{
name: "raw input digest",
mutate: func(input CheckpointIdentityInput) CheckpointIdentityInput {
input.RawInputDigest = "sha256:raw-b"
return input
},
},
{
name: "selected lanes",
mutate: func(input CheckpointIdentityInput) CheckpointIdentityInput {
input.SelectedLanes = []string{"items"}
return input
},
},
{
name: "reference digest",
mutate: func(input CheckpointIdentityInput) CheckpointIdentityInput {
input.References[0].Digest = "sha256:reference-b"
return input
},
},
{
name: "runtime override",
mutate: func(input CheckpointIdentityInput) CheckpointIdentityInput {
input.RuntimeOverrides = []Fingerprint{{Name: "llm_profile", Value: "careful"}}
return input
},
},
{
name: "provenance fingerprint",
mutate: func(input CheckpointIdentityInput) CheckpointIdentityInput {
input.ProvenanceFingerprints = []Fingerprint{{Name: "prompt:dnd.spells", Value: "sha256:prompt-b"}}
return input
},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
changed := mustIdentity(t, tc.mutate(identityInput()))
if changed.Digest == base.Digest {
t.Fatalf("digest did not change after %s mutation: %q", tc.name, changed.Digest)
}
})
}
}
func TestCheckpointIdentityNormalizesOrder(t *testing.T) {
input := identityInput()
input.SelectedLanes = []string{"spells", "items", "spells"}
input.RuntimeOverrides = []Fingerprint{
{Name: "z", Value: "2"},
{Name: "a", Value: "1"},
}
input.ProvenanceFingerprints = []Fingerprint{
{Name: "schema", Value: "sha256:schema"},
{Name: "prompt", Value: "sha256:prompt"},
}
identity := mustIdentity(t, input)
if got := strings.Join(identity.SelectedLanes, ","); got != "items,spells" {
t.Fatalf("selected lanes = %q, want sorted unique values", got)
}
if identity.RuntimeOverrides[0].Name != "a" || identity.ProvenanceFingerprints[0].Name != "prompt" {
t.Fatalf("fingerprints not sorted: runtime=%+v provenance=%+v", identity.RuntimeOverrides, identity.ProvenanceFingerprints)
}
}
func TestCheckpointIdentityPathIsFilesystemSafe(t *testing.T) {
input := identityInput()
input.Pipeline.ID = "campaign/main"
input.InputKey = "seriatim/input"
input.SourceDigest = "sha256:abcdef0123456789ffffffff"
input.Pipeline.Digest = "sha256:1234567890abcdefeeeeeeee"
identity := mustIdentity(t, input)
relative, err := identity.RelativePath()
if err != nil {
t.Fatalf("RelativePath: %v", err)
}
if strings.Contains(relative, `\`) || strings.Contains(relative, "..") {
t.Fatalf("relative path is not filesystem safe: %q", relative)
}
identityDigest := digestPrefix(identity.Digest)
wantRelative := "campaign~2fmain/seriatim~2finput-abcdef0123456789/1234567890abcdef/" + identityDigest
if relative != wantRelative {
t.Fatalf("relative path = %q", relative)
}
root := t.TempDir()
settings := Settings{
CheckpointsRoot: filepath.Join(root, "checkpoints"),
ResumeEnabled: true,
}
got, err := settings.CheckpointDirectory(identity)
if err != nil {
t.Fatalf("CheckpointDirectory: %v", err)
}
want := filepath.Join(root, "checkpoints", "campaign~2fmain", "seriatim~2finput-abcdef0123456789", "1234567890abcdef", identityDigest)
if got != want {
t.Fatalf("checkpoint directory = %q, want %q", got, want)
}
}
func TestCheckpointIdentityPathIncludesInvocationIdentity(t *testing.T) {
base := mustIdentity(t, identityInput())
changedInput := identityInput()
changedInput.References[0].Digest = "sha256:reference-b"
changed := mustIdentity(t, changedInput)
if base.Digest == changed.Digest {
t.Fatalf("test setup produced same identity digest: %q", base.Digest)
}
basePath, err := base.RelativePath()
if err != nil {
t.Fatalf("base RelativePath: %v", err)
}
changedPath, err := changed.RelativePath()
if err != nil {
t.Fatalf("changed RelativePath: %v", err)
}
if basePath == changedPath {
t.Fatalf("relative path did not change with invocation identity: %q", basePath)
}
}
func TestCheckpointDirectoryDisabledReturnsEmptyPath(t *testing.T) {
settings := Settings{CheckpointsRoot: filepath.Join(t.TempDir(), "checkpoints")}
got, err := settings.CheckpointDirectory(mustIdentity(t, identityInput()))
if err != nil {
t.Fatalf("CheckpointDirectory: %v", err)
}
if got != "" {
t.Fatalf("CheckpointDirectory = %q, want empty path", got)
}
}
func TestNewCheckpointIdentityRequiresCoreInputs(t *testing.T) {
tests := []struct {
name string
mutate func(CheckpointIdentityInput) CheckpointIdentityInput
want string
}{
{
name: "pipeline id",
mutate: func(input CheckpointIdentityInput) CheckpointIdentityInput {
input.Pipeline.ID = ""
return input
},
want: "pipeline id",
},
{
name: "pipeline digest",
mutate: func(input CheckpointIdentityInput) CheckpointIdentityInput {
input.Pipeline.Digest = ""
return input
},
want: "pipeline digest",
},
{
name: "input key",
mutate: func(input CheckpointIdentityInput) CheckpointIdentityInput {
input.InputKey = ""
input.Pipeline.Input.Module = ""
return input
},
want: "input key",
},
{
name: "input digest",
mutate: func(input CheckpointIdentityInput) CheckpointIdentityInput {
input.RawInputDigest = ""
input.SourceDigest = ""
return input
},
want: "raw input digest or source digest",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, err := NewCheckpointIdentity(tc.mutate(identityInput()))
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("expected error containing %q, got %v", tc.want, err)
}
})
}
}
func identityInput() CheckpointIdentityInput {
return CheckpointIdentityInput{
Pipeline: pipeline.ResolvedPipeline{
ID: "dnd-session",
Digest: "sha256:pipeline-a",
Input: pipeline.Binding("seriatim"),
ArtifactLanes: []pipeline.ResolvedArtifactLane{
{ID: "spells"},
{ID: "items"},
},
},
InputKey: "seriatim",
RawInputDigest: "sha256:raw-a",
SelectedLanes: []string{"spells"},
RuntimeOverrides: []Fingerprint{
{Name: "llm_profile", Value: "fast"},
},
References: []artifacts.ReferenceProvenance{
{
Stage: "extract",
LaneID: "spells",
SlotName: "party",
OriginURI: "file:///party.yml",
Digest: "sha256:reference-a",
},
},
ProvenanceFingerprints: []Fingerprint{
{Name: "prompt:dnd.spells", Value: "sha256:prompt-a"},
},
}
}
func mustIdentity(t *testing.T, input CheckpointIdentityInput) CheckpointIdentity {
t.Helper()
identity, err := NewCheckpointIdentity(input)
if err != nil {
t.Fatalf("NewCheckpointIdentity: %v", err)
}
return identity
}

137
internal/core/workspace/manifest_test.go
View File

@@ -1,137 +0,0 @@
package workspace
import (
"encoding/json"
"testing"
"time"
)
func TestStageManifestDefaults(t *testing.T) {
if WorkspaceSchemaVersion != "notarius.workspace.v2" {
t.Fatalf("current schema version = %q, want notarius.workspace.v2", WorkspaceSchemaVersion)
}
if WorkspaceSchemaVersionV1 != "notarius.workspace.v1" {
t.Fatalf("legacy schema version = %q, want notarius.workspace.v1", WorkspaceSchemaVersionV1)
}
manifest := NewStageManifest(StageExtract, StatusRunning)
if manifest.WorkspaceSchemaVersion != WorkspaceSchemaVersion {
t.Fatalf("schema version = %q, want %q", manifest.WorkspaceSchemaVersion, WorkspaceSchemaVersion)
}
if manifest.Stage != StageExtract {
t.Fatalf("stage = %q, want extract", manifest.Stage)
}
if manifest.Status != StatusRunning {
t.Fatalf("status = %q, want running", manifest.Status)
}
}
func TestManifestJSONRoundTrips(t *testing.T) {
started := time.Unix(100, 0).UTC()
completed := time.Unix(200, 0).UTC()
t.Run("source", func(t *testing.T) {
manifest := SourceManifest{
StageManifest: populatedManifest(StageSource, "", "seriatim", started, completed),
SourceID: "source-1",
}
var got SourceManifest
roundTripManifest(t, manifest, &got)
if got.SourceID != manifest.SourceID || got.Stage != StageSource {
t.Fatalf("round trip source manifest = %+v", got)
}
})
t.Run("extract", func(t *testing.T) {
manifest := ExtractLaneManifest{
StageManifest: populatedManifest(StageExtract, "spells", "dnd/spells", started, completed),
ChunkCount: 3,
OutputCount: 2,
}
var got ExtractLaneManifest
roundTripManifest(t, manifest, &got)
if got.LaneID != "spells" || got.OutputCount != manifest.OutputCount || got.Stage != StageExtract {
t.Fatalf("round trip extract manifest = %+v", got)
}
})
t.Run("merge", func(t *testing.T) {
manifest := MergeLaneManifest{
StageManifest: populatedManifest(StageMerge, "spells", "appendorder", started, completed),
InputCount: 2,
}
var got MergeLaneManifest
roundTripManifest(t, manifest, &got)
if got.InputCount != manifest.InputCount || got.Stage != StageMerge {
t.Fatalf("round trip merge manifest = %+v", got)
}
})
t.Run("normalize", func(t *testing.T) {
manifest := NormalizeLaneManifest{
StageManifest: populatedManifest(StageNormalize, "spells", "noop", started, completed),
InputCount: 1,
}
var got NormalizeLaneManifest
roundTripManifest(t, manifest, &got)
if got.InputCount != manifest.InputCount || got.Stage != StageNormalize {
t.Fatalf("round trip normalize manifest = %+v", got)
}
})
}
func TestStatusValues(t *testing.T) {
values := []StageStatus{
StatusPending,
StatusRunning,
StatusSucceeded,
StatusSucceededWithRejections,
StatusFailed,
StatusInvalidated,
}
want := []string{
"pending",
"running",
"succeeded",
"succeeded_with_rejections",
"failed",
"invalidated",
}
for i, value := range values {
if string(value) != want[i] {
t.Fatalf("status[%d] = %q, want %q", i, value, want[i])
}
}
}
func populatedManifest(stage StageName, laneID string, moduleKey string, started time.Time, completed time.Time) StageManifest {
manifest := NewStageManifest(stage, StatusSucceededWithRejections)
manifest.LaneID = laneID
manifest.ModuleKey = moduleKey
manifest.DependencyFingerprints = []Fingerprint{{Name: "source", Value: "sha256:source"}}
manifest.OutputDigests = []Fingerprint{{Name: "output", Value: "sha256:output"}}
manifest.ValidationStatus = "approved_with_warnings"
manifest.Rejections = []RejectionSummary{
{
ValidatorName: "shape",
ReasonCode: "invalid_shape",
Message: "invalid output shape",
Count: 1,
},
}
manifest.StartedAt = &started
manifest.CompletedAt = &completed
manifest.Metadata = map[string]string{"attempt": "1"}
return manifest
}
func roundTripManifest(t *testing.T, in any, out any) {
t.Helper()
data, err := json.Marshal(in)
if err != nil {
t.Fatalf("marshal manifest: %v", err)
}
if err := json.Unmarshal(data, out); err != nil {
t.Fatalf("unmarshal manifest: %v", err)
}
}

76
internal/core/workspace/settings.go
View File

@@ -1,76 +0,0 @@
package workspace
import (
"fmt"
"path/filepath"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
)
type Settings struct {
RootDir string
DiagnosticsRoot string
CheckpointsRoot string
DebugRoot string
DiagnosticsEnabled bool
ResumeEnabled bool
DebugEnabled bool
}
func FromConfig(cfg config.Config) Settings {
root := cleanPath(cfg.Workspace.Directory)
settings := Settings{
RootDir: root,
DiagnosticsEnabled: cfg.DiagnosticsEnabled(),
}
if settings.DiagnosticsEnabled {
settings.DiagnosticsRoot = cleanPath(cfg.Diagnostics.WorkDir)
}
if root == "" {
return settings
}
settings.CheckpointsRoot = filepath.Join(root, "checkpoints")
settings.DebugRoot = filepath.Join(root, "debug")
settings.ResumeEnabled = cfg.Workspace.Resume.Enabled
settings.DebugEnabled = cfg.Workspace.Debug.Enabled
return settings
}
func (s Settings) DiagnosticsRunDirectory(runID string) (string, error) {
if !s.DiagnosticsEnabled || strings.TrimSpace(s.DiagnosticsRoot) == "" {
return "", nil
}
return safeSingleDirectory(s.DiagnosticsRoot, runID, "diagnostics run ID")
}
func (s Settings) CheckpointIdentityDirectory(identity string) (string, error) {
if !s.ResumeEnabled || strings.TrimSpace(s.CheckpointsRoot) == "" {
return "", nil
}
return SafePath(s.CheckpointsRoot, identity)
}
func (s Settings) DebugRunDirectory(runID string) (string, error) {
if !s.DebugEnabled || strings.TrimSpace(s.DebugRoot) == "" {
return "", nil
}
return safeSingleDirectory(s.DebugRoot, runID, "debug run ID")
}
func cleanPath(path string) string {
path = strings.TrimSpace(path)
if path == "" {
return ""
}
return filepath.Clean(path)
}
func safeSingleDirectory(root string, name string, label string) (string, error) {
name = strings.TrimSpace(name)
if strings.Contains(name, "/") || strings.Contains(name, `\`) {
return "", fmt.Errorf("%s %q must be a single directory name", label, name)
}
return SafePath(root, name)
}

127
internal/core/workspace/settings_test.go
View File

@@ -1,127 +0,0 @@
package workspace
import (
"path/filepath"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
)
func TestFromConfigBuildsWorkspaceRoots(t *testing.T) {
cfg := config.Default()
cfg.Workspace.Directory = "/var/lib/notarius"
cfg.Workspace.Resume.Enabled = true
cfg.Workspace.Debug.Enabled = true
cfg.RecomputeEffectiveDiagnostics()
settings := FromConfig(cfg)
if settings.RootDir != "/var/lib/notarius" {
t.Fatalf("RootDir = %q, want /var/lib/notarius", settings.RootDir)
}
if settings.DiagnosticsRoot != "/var/lib/notarius/diagnostics" || !settings.DiagnosticsEnabled {
t.Fatalf("diagnostics settings = %+v, want workspace diagnostics root enabled", settings)
}
if settings.CheckpointsRoot != "/var/lib/notarius/checkpoints" || !settings.ResumeEnabled {
t.Fatalf("checkpoint settings = %+v, want workspace checkpoints root enabled", settings)
}
if settings.DebugRoot != "/var/lib/notarius/debug" || !settings.DebugEnabled {
t.Fatalf("debug settings = %+v, want workspace debug root enabled", settings)
}
}
func TestFromConfigKeepsLegacyDiagnosticsRootWithoutWorkspaceRoot(t *testing.T) {
cfg := config.Default()
cfg.Diagnostics.WorkDir = "/tmp/notarius-legacy"
cfg.Workspace.Resume.Enabled = true
cfg.Workspace.Debug.Enabled = true
settings := FromConfig(cfg)
if settings.RootDir != "" {
t.Fatalf("RootDir = %q, want empty", settings.RootDir)
}
if settings.DiagnosticsRoot != "/tmp/notarius-legacy" || !settings.DiagnosticsEnabled {
t.Fatalf("diagnostics settings = %+v, want legacy diagnostics root enabled", settings)
}
if settings.CheckpointsRoot != "" || settings.ResumeEnabled {
t.Fatalf("checkpoint settings = %+v, want disabled empty root", settings)
}
if settings.DebugRoot != "" || settings.DebugEnabled {
t.Fatalf("debug settings = %+v, want disabled empty root", settings)
}
}
func TestPathConstructors(t *testing.T) {
root := t.TempDir()
settings := Settings{
RootDir: root,
DiagnosticsRoot: filepath.Join(root, "diagnostics"),
CheckpointsRoot: filepath.Join(root, "checkpoints"),
DebugRoot: filepath.Join(root, "debug"),
DiagnosticsEnabled: true,
ResumeEnabled: true,
DebugEnabled: true,
}
diagnosticsDir, err := settings.DiagnosticsRunDirectory("run-123")
if err != nil {
t.Fatalf("DiagnosticsRunDirectory: %v", err)
}
if diagnosticsDir != filepath.Join(root, "diagnostics", "run-123") {
t.Fatalf("diagnostics dir = %q", diagnosticsDir)
}
checkpointDir, err := settings.CheckpointIdentityDirectory("pipeline/input-digest/pipeline-digest")
if err != nil {
t.Fatalf("CheckpointIdentityDirectory: %v", err)
}
if checkpointDir != filepath.Join(root, "checkpoints", "pipeline", "input-digest", "pipeline-digest") {
t.Fatalf("checkpoint dir = %q", checkpointDir)
}
debugDir, err := settings.DebugRunDirectory("run-456")
if err != nil {
t.Fatalf("DebugRunDirectory: %v", err)
}
if debugDir != filepath.Join(root, "debug", "run-456") {
t.Fatalf("debug dir = %q", debugDir)
}
}
func TestDisabledPathConstructorsReturnEmptyPaths(t *testing.T) {
settings := Settings{}
for name, call := range map[string]func() (string, error){
"diagnostics": func() (string, error) { return settings.DiagnosticsRunDirectory("run-1") },
"checkpoint": func() (string, error) { return settings.CheckpointIdentityDirectory("identity") },
"debug": func() (string, error) { return settings.DebugRunDirectory("run-1") },
} {
t.Run(name, func(t *testing.T) {
got, err := call()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != "" {
t.Fatalf("path = %q, want empty", got)
}
})
}
}
func TestRunDirectoryConstructorsRejectNestedNames(t *testing.T) {
root := t.TempDir()
settings := Settings{
DiagnosticsRoot: filepath.Join(root, "diagnostics"),
DebugRoot: filepath.Join(root, "debug"),
DiagnosticsEnabled: true,
DebugEnabled: true,
}
if got, err := settings.DiagnosticsRunDirectory("run-1/nested"); err == nil {
t.Fatalf("DiagnosticsRunDirectory returned %q, want error", got)
}
if got, err := settings.DebugRunDirectory("run-1/nested"); err == nil {
t.Fatalf("DebugRunDirectory returned %q, want error", got)
}
}

184
internal/framework/checkpoint/identity.go Normal file
View File

@@ -0,0 +1,184 @@
package checkpoint
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"path/filepath"
"sort"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const digestPrefixLength = 16
type Fingerprint struct {
Name string `json:"name"`
Value string `json:"value"`
}
type IdentityInput struct {
Pipeline pipeline.ResolvedPipeline
InputKey, RawInputDigest, SourceDigest string
SelectedLanes []string
RuntimeOverrides []Fingerprint
References []artifacts.ReferenceProvenance
ProvenanceFingerprints []Fingerprint
}
type Identity struct {
Digest string `json:"digest"`
PipelineID string `json:"pipeline_id"`
PipelineDigest string `json:"pipeline_digest"`
InputKey string `json:"input_key"`
RawInputDigest string `json:"raw_input_digest,omitempty"`
SourceDigest string `json:"source_digest,omitempty"`
SelectedLanes []string `json:"selected_lanes,omitempty"`
RuntimeOverrides []Fingerprint `json:"runtime_overrides,omitempty"`
ReferenceDigests []Fingerprint `json:"reference_digests,omitempty"`
ProvenanceFingerprints []Fingerprint `json:"provenance_fingerprints,omitempty"`
}
func NewIdentity(input IdentityInput) (Identity, error) {
pipelineID, pipelineDigest, inputKey := strings.TrimSpace(input.Pipeline.ID), strings.TrimSpace(input.Pipeline.Digest), strings.TrimSpace(input.InputKey)
if pipelineID == "" {
return Identity{}, fmt.Errorf("checkpoint identity pipeline id must not be empty")
}
if pipelineDigest == "" {
return Identity{}, fmt.Errorf("checkpoint identity pipeline digest must not be empty")
}
if inputKey == "" {
inputKey = strings.TrimSpace(input.Pipeline.Input.Module)
}
if inputKey == "" {
return Identity{}, fmt.Errorf("checkpoint identity input key must not be empty")
}
if strings.TrimSpace(input.RawInputDigest) == "" && strings.TrimSpace(input.SourceDigest) == "" {
return Identity{}, fmt.Errorf("checkpoint identity raw input digest or source digest must be set")
}
v := Identity{PipelineID: pipelineID, PipelineDigest: pipelineDigest, InputKey: inputKey, RawInputDigest: strings.TrimSpace(input.RawInputDigest), SourceDigest: strings.TrimSpace(input.SourceDigest), SelectedLanes: normalizedLanes(input.SelectedLanes, input.Pipeline.ArtifactLanes), RuntimeOverrides: normalizeIdentityFingerprints(input.RuntimeOverrides), ReferenceDigests: referenceFingerprints(input.References), ProvenanceFingerprints: normalizeIdentityFingerprints(input.ProvenanceFingerprints)}
data, err := json.Marshal(Identity{PipelineID: v.PipelineID, PipelineDigest: v.PipelineDigest, InputKey: v.InputKey, RawInputDigest: v.RawInputDigest, SourceDigest: v.SourceDigest, SelectedLanes: v.SelectedLanes, RuntimeOverrides: v.RuntimeOverrides, ReferenceDigests: v.ReferenceDigests, ProvenanceFingerprints: v.ProvenanceFingerprints})
if err != nil {
return Identity{}, fmt.Errorf("marshal checkpoint identity: %w", err)
}
sum := sha256.Sum256(data)
v.Digest = "sha256:" + hex.EncodeToString(sum[:])
return v, nil
}
func (i Identity) RelativePath() (string, error) {
p, err := safeComponent(i.PipelineID)
if err != nil {
return "", fmt.Errorf("checkpoint identity pipeline id: %w", err)
}
k, err := safeComponent(i.InputKey)
if err != nil {
return "", fmt.Errorf("checkpoint identity input key: %w", err)
}
s := digestPrefix(i.SourceDigest)
if s == "" {
s = digestPrefix(i.RawInputDigest)
}
d := digestPrefix(i.PipelineDigest)
x := digestPrefix(i.Digest)
if s == "" || d == "" || x == "" {
return "", fmt.Errorf("checkpoint identity digest prefix must not be empty")
}
s, err = safeComponent(s)
if err != nil {
return "", fmt.Errorf("checkpoint identity source digest: %w", err)
}
d, err = safeComponent(d)
if err != nil {
return "", fmt.Errorf("checkpoint identity pipeline digest: %w", err)
}
x, err = safeComponent(x)
if err != nil {
return "", fmt.Errorf("checkpoint identity digest: %w", err)
}
return filepath.ToSlash(filepath.Join(p, k+"-"+s, d, x)), nil
}
func normalizedLanes(selected []string, resolved []pipeline.ResolvedArtifactLane) []string {
if len(selected) == 0 {
for _, lane := range resolved {
selected = append(selected, lane.ID)
}
}
return normalizeStrings(selected)
}
func normalizeIdentityFingerprints(values []Fingerprint) []Fingerprint {
by := map[string]string{}
for _, v := range values {
if n, x := strings.TrimSpace(v.Name), strings.TrimSpace(v.Value); n != "" && x != "" {
by[n] = x
}
}
names := make([]string, 0, len(by))
for n := range by {
names = append(names, n)
}
sort.Strings(names)
out := make([]Fingerprint, 0, len(names))
for _, n := range names {
out = append(out, Fingerprint{Name: n, Value: by[n]})
}
if len(out) == 0 {
return nil
}
return out
}
func referenceFingerprints(refs []artifacts.ReferenceProvenance) []Fingerprint {
var values []Fingerprint
for _, r := range refs {
if d := strings.TrimSpace(r.Digest); d != "" {
values = append(values, Fingerprint{Name: strings.Join([]string{strings.TrimSpace(r.Stage), strings.TrimSpace(r.LaneID), strings.TrimSpace(r.SlotName), strings.TrimSpace(r.OriginURI)}, ":"), Value: d})
}
}
return normalizeIdentityFingerprints(values)
}
func normalizeStrings(values []string) []string {
seen := map[string]struct{}{}
for _, v := range values {
if v = strings.TrimSpace(v); v != "" {
seen[v] = struct{}{}
}
}
out := make([]string, 0, len(seen))
for v := range seen {
out = append(out, v)
}
sort.Strings(out)
if len(out) == 0 {
return nil
}
return out
}
func digestPrefix(v string) string {
v = strings.TrimSpace(v)
if n := strings.Index(v, ":"); n >= 0 {
v = v[n+1:]
}
if len(v) > digestPrefixLength {
return v[:digestPrefixLength]
}
return v
}
func safeComponent(v string) (string, error) {
v = strings.TrimSpace(v)
if v == "" {
return "", fmt.Errorf("must not be empty")
}
var b strings.Builder
for _, r := range v {
if r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9' || r == '-' || r == '_' || r == '.' {
b.WriteRune(r)
} else {
b.WriteString(fmt.Sprintf("~%x", r))
}
}
out := b.String()
if out == "." || out == ".." || strings.Contains(out, "..") || strings.ContainsAny(out, `/\\`) {
return "", fmt.Errorf("%q is not filesystem safe", v)
}
return out, nil
}

71
internal/framework/checkpoint/loader.go
View File

@@ -7,38 +7,43 @@ import (
"os"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/fileio"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
coreworkspace "gitea.maximumdirect.net/eric/notarius/internal/core/workspace"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
type WorkspaceLoader struct {
type FilesystemLoader struct {
root string
identityDigest string
}
func NewWorkspaceLoader(settings coreworkspace.Settings, identity coreworkspace.CheckpointIdentity) (pipeline.CheckpointLoader, error) {
root, err := settings.CheckpointDirectory(identity)
func NewFilesystemLoader(root string, identity Identity) (pipeline.CheckpointLoader, error) {
root = strings.TrimSpace(root)
if root == "" {
return pipeline.NoopCheckpointLoader(), nil
}
relative, err := identity.RelativePath()
if err != nil {
return nil, err
}
if strings.TrimSpace(root) == "" {
return pipeline.NoopCheckpointLoader(), nil
target, err := fileio.SafePath(root, relative)
if err != nil {
return nil, err
}
return &WorkspaceLoader{root: root, identityDigest: identity.Digest}, nil
return &FilesystemLoader{root: target, identityDigest: identity.Digest}, nil
}
func (l *WorkspaceLoader) Enabled() bool {
func (l *FilesystemLoader) Enabled() bool {
return l != nil && strings.TrimSpace(l.root) != ""
}
func (l *WorkspaceLoader) Source(moduleKey string) (pipeline.SourceCheckpoint, pipeline.CheckpointDecision) {
var manifest coreworkspace.SourceManifest
func (l *FilesystemLoader) Source(moduleKey string) (pipeline.SourceCheckpoint, pipeline.CheckpointDecision) {
var manifest SourceManifest
if decision := l.readJSON("source/manifest.json", &manifest); !decision.Reused {
return pipeline.SourceCheckpoint{}, decision
}
if decision := l.validateManifest(manifest.StageManifest, coreworkspace.StageSource, "", moduleKey, coreworkspace.StatusSucceeded, nil); !decision.Reused {
if decision := l.validateManifest(manifest.StageManifest, StageSource, "", moduleKey, StatusSucceeded, nil); !decision.Reused {
return pipeline.SourceCheckpoint{}, decision
}
var payload sourceDocumentEnvelope
@@ -52,18 +57,18 @@ func (l *WorkspaceLoader) Source(moduleKey string) (pipeline.SourceCheckpoint, p
if strings.TrimSpace(manifest.SourceID) != "" && manifest.SourceID != doc.ID {
return pipeline.SourceCheckpoint{}, invalidDecision("source checkpoint source id does not match payload")
}
if !fingerprintsEqual(coreworkspaceToPipelineFingerprints(manifest.OutputDigests), digestFingerprints("source_document", doc.Digest)) {
if !fingerprintsEqual(checkpointToPipelineFingerprints(manifest.OutputDigests), digestFingerprints("source_document", doc.Digest)) {
return pipeline.SourceCheckpoint{}, invalidDecision("source checkpoint output digest does not match payload")
}
return pipeline.SourceCheckpoint{Document: &doc}, reusedDecision()
}
func (l *WorkspaceLoader) Extract(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint) (pipeline.ExtractCheckpoint, pipeline.CheckpointDecision) {
var manifest coreworkspace.ExtractLaneManifest
func (l *FilesystemLoader) Extract(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint) (pipeline.ExtractCheckpoint, pipeline.CheckpointDecision) {
var manifest ExtractLaneManifest
if d := l.readJSON(laneManifestPath("extract", laneID), &manifest); !d.Reused {
return pipeline.ExtractCheckpoint{}, d
}
if d := l.validateLaneManifest(manifest.StageManifest, coreworkspace.StageExtract, laneID, moduleKey, dependencies, coreworkspace.StatusSucceeded, coreworkspace.StatusSucceededWithRejections); !d.Reused {
if d := l.validateLaneManifest(manifest.StageManifest, StageExtract, laneID, moduleKey, dependencies, StatusSucceeded, StatusSucceededWithRejections); !d.Reused {
return pipeline.ExtractCheckpoint{}, d
}
var payload artifactExtractEnvelope
@@ -74,18 +79,18 @@ func (l *WorkspaceLoader) Extract(laneID, moduleKey string, dependencies []pipel
if err != nil {
return pipeline.ExtractCheckpoint{}, invalidDecision("extract artifact checkpoint payload is invalid: %v", err)
}
if !fingerprintsEqual(coreworkspaceToPipelineFingerprints(manifest.OutputDigests), artifactOutputDigests(outputs)) {
if !fingerprintsEqual(checkpointToPipelineFingerprints(manifest.OutputDigests), artifactOutputDigests(outputs)) {
return pipeline.ExtractCheckpoint{}, invalidDecision("extract artifact checkpoint output digests do not match payload")
}
return pipeline.ExtractCheckpoint{Outputs: outputs, Rejected: cloneRejectedOutputs(payload.Rejected), Warnings: cloneWarnings(payload.Warnings)}, reusedDecision()
}
func (l *WorkspaceLoader) Merge(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint) (pipeline.MergeCheckpoint, pipeline.CheckpointDecision) {
var manifest coreworkspace.MergeLaneManifest
func (l *FilesystemLoader) Merge(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint) (pipeline.MergeCheckpoint, pipeline.CheckpointDecision) {
var manifest MergeLaneManifest
if d := l.readJSON(laneManifestPath("merge", laneID), &manifest); !d.Reused {
return pipeline.MergeCheckpoint{}, d
}
if d := l.validateLaneManifest(manifest.StageManifest, coreworkspace.StageMerge, laneID, moduleKey, dependencies, coreworkspace.StatusSucceeded); !d.Reused {
if d := l.validateLaneManifest(manifest.StageManifest, StageMerge, laneID, moduleKey, dependencies, StatusSucceeded); !d.Reused {
return pipeline.MergeCheckpoint{}, d
}
var payload artifactSingleEnvelope
@@ -96,18 +101,18 @@ func (l *WorkspaceLoader) Merge(laneID, moduleKey string, dependencies []pipelin
if err != nil {
return pipeline.MergeCheckpoint{}, invalidDecision("merge artifact checkpoint payload is invalid: %v", err)
}
if !fingerprintsEqual(coreworkspaceToPipelineFingerprints(manifest.OutputDigests), artifactOutputDigests(values)) {
if !fingerprintsEqual(checkpointToPipelineFingerprints(manifest.OutputDigests), artifactOutputDigests(values)) {
return pipeline.MergeCheckpoint{}, invalidDecision("merge artifact checkpoint output digest does not match payload")
}
return pipeline.MergeCheckpoint{Output: values[0], Warnings: cloneWarnings(payload.Warnings)}, reusedDecision()
}
func (l *WorkspaceLoader) Normalize(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint) (pipeline.NormalizeCheckpoint, pipeline.CheckpointDecision) {
var manifest coreworkspace.NormalizeLaneManifest
func (l *FilesystemLoader) Normalize(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint) (pipeline.NormalizeCheckpoint, pipeline.CheckpointDecision) {
var manifest NormalizeLaneManifest
if d := l.readJSON(laneManifestPath("normalize", laneID), &manifest); !d.Reused {
return pipeline.NormalizeCheckpoint{}, d
}
if d := l.validateLaneManifest(manifest.StageManifest, coreworkspace.StageNormalize, laneID, moduleKey, dependencies, coreworkspace.StatusSucceeded); !d.Reused {
if d := l.validateLaneManifest(manifest.StageManifest, StageNormalize, laneID, moduleKey, dependencies, StatusSucceeded); !d.Reused {
return pipeline.NormalizeCheckpoint{}, d
}
var payload artifactSingleEnvelope
@@ -118,7 +123,7 @@ func (l *WorkspaceLoader) Normalize(laneID, moduleKey string, dependencies []pip
if err != nil {
return pipeline.NormalizeCheckpoint{}, invalidDecision("normalize artifact checkpoint payload is invalid: %v", err)
}
if !fingerprintsEqual(coreworkspaceToPipelineFingerprints(manifest.OutputDigests), artifactOutputDigests(values)) {
if !fingerprintsEqual(checkpointToPipelineFingerprints(manifest.OutputDigests), artifactOutputDigests(values)) {
return pipeline.NormalizeCheckpoint{}, invalidDecision("normalize artifact checkpoint output digest does not match payload")
}
return pipeline.NormalizeCheckpoint{Output: values[0], Warnings: cloneWarnings(payload.Warnings)}, reusedDecision()
@@ -142,11 +147,11 @@ func artifactCheckpointOutputs(values []artifactCheckpointEnvelope) ([]pipeline.
return out, nil
}
func (l *WorkspaceLoader) readJSON(name string, out any) pipeline.CheckpointDecision {
func (l *FilesystemLoader) readJSON(name string, out any) pipeline.CheckpointDecision {
if !l.Enabled() {
return pipeline.CheckpointDecision{Reason: "checkpoint loading disabled"}
}
target, err := coreworkspace.SafePath(l.root, name)
target, err := fileio.SafePath(l.root, name)
if err != nil {
return invalidDecision("checkpoint path is invalid: %v", err)
}
@@ -163,15 +168,15 @@ func (l *WorkspaceLoader) readJSON(name string, out any) pipeline.CheckpointDeci
return reusedDecision()
}
func (l *WorkspaceLoader) validateManifest(manifest coreworkspace.StageManifest, stage coreworkspace.StageName, laneID string, moduleKey string, status coreworkspace.StageStatus, dependencies []pipeline.CheckpointFingerprint) pipeline.CheckpointDecision {
func (l *FilesystemLoader) validateManifest(manifest StageManifest, stage StageName, laneID string, moduleKey string, status StageStatus, dependencies []pipeline.CheckpointFingerprint) pipeline.CheckpointDecision {
return l.validateLaneManifest(manifest, stage, laneID, moduleKey, dependencies, status)
}
func (l *WorkspaceLoader) validateLaneManifest(manifest coreworkspace.StageManifest, stage coreworkspace.StageName, laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, statuses ...coreworkspace.StageStatus) pipeline.CheckpointDecision {
if manifest.WorkspaceSchemaVersion == coreworkspace.WorkspaceSchemaVersionV1 {
return invalidDecision("checkpoint workspace schema version %q is incompatible with %q and must be recomputed", manifest.WorkspaceSchemaVersion, coreworkspace.WorkspaceSchemaVersion)
func (l *FilesystemLoader) validateLaneManifest(manifest StageManifest, stage StageName, laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, statuses ...StageStatus) pipeline.CheckpointDecision {
if manifest.WorkspaceSchemaVersion == WorkspaceSchemaVersionV1 {
return invalidDecision("checkpoint workspace schema version %q is incompatible with %q and must be recomputed", manifest.WorkspaceSchemaVersion, WorkspaceSchemaVersion)
}
if manifest.WorkspaceSchemaVersion != coreworkspace.WorkspaceSchemaVersion {
if manifest.WorkspaceSchemaVersion != WorkspaceSchemaVersion {
return invalidDecision("checkpoint workspace schema version %q is not supported", manifest.WorkspaceSchemaVersion)
}
if strings.TrimSpace(l.identityDigest) != "" && manifest.Metadata["checkpoint_identity_digest"] != l.identityDigest {
@@ -196,7 +201,7 @@ func (l *WorkspaceLoader) validateLaneManifest(manifest coreworkspace.StageManif
if !statusOK {
return invalidDecision("checkpoint status %q cannot be reused", manifest.Status)
}
if !fingerprintsEqual(coreworkspaceToPipelineFingerprints(manifest.DependencyFingerprints), dependencies) {
if !fingerprintsEqual(checkpointToPipelineFingerprints(manifest.DependencyFingerprints), dependencies) {
return invalidDecision("checkpoint dependency fingerprints do not match")
}
return reusedDecision()
@@ -213,7 +218,7 @@ func contentFromEnvelope(value binaryEnvelope) ([]byte, error) {
return content, nil
}
func coreworkspaceToPipelineFingerprints(values []coreworkspace.Fingerprint) []pipeline.CheckpointFingerprint {
func checkpointToPipelineFingerprints(values []Fingerprint) []pipeline.CheckpointFingerprint {
if len(values) == 0 {
return nil
}

15
internal/core/workspace/manifest.go → internal/framework/checkpoint/manifest.go
View File

@@ -1,8 +1,10 @@
package workspace
package checkpoint
import "time"
const (
// These names and values are frozen checkpoint wire-compatibility
// identifiers. They intentionally retain the former terminology.
WorkspaceSchemaVersion = "notarius.workspace.v2"
WorkspaceSchemaVersionV1 = "notarius.workspace.v1"
)
@@ -41,39 +43,30 @@ type StageManifest struct {
CompletedAt *time.Time `json:"completed_at,omitempty"`
Metadata map[string]string `json:"metadata,omitempty"`
}
type RejectionSummary struct {
ValidatorName string `json:"validator_name,omitempty"`
ReasonCode string `json:"reason_code,omitempty"`
Message string `json:"message,omitempty"`
Count int `json:"count,omitempty"`
}
type SourceManifest struct {
StageManifest
SourceID string `json:"source_id,omitempty"`
}
type ExtractLaneManifest struct {
StageManifest
ChunkCount int `json:"chunk_count,omitempty"`
OutputCount int `json:"output_count,omitempty"`
}
type MergeLaneManifest struct {
StageManifest
InputCount int `json:"input_count,omitempty"`
}
type NormalizeLaneManifest struct {
StageManifest
InputCount int `json:"input_count,omitempty"`
}
func NewStageManifest(stage StageName, status StageStatus) StageManifest {
return StageManifest{
WorkspaceSchemaVersion: WorkspaceSchemaVersion,
Stage: stage,
Status: status,
}
return StageManifest{WorkspaceSchemaVersion: WorkspaceSchemaVersion, Stage: stage, Status: status}
}

147
internal/framework/checkpoint/recorder.go
View File

@@ -10,186 +10,191 @@ import (
"strings"
"time"
"gitea.maximumdirect.net/eric/notarius/internal/core/fileio"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
coreworkspace "gitea.maximumdirect.net/eric/notarius/internal/core/workspace"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
type WorkspaceRecorder struct {
type FilesystemRecorder struct {
root string
identityDigest string
now func() time.Time
}
func NewWorkspaceRecorder(settings coreworkspace.Settings, identity coreworkspace.CheckpointIdentity) (pipeline.CheckpointRecorder, error) {
root, err := settings.CheckpointDirectory(identity)
func NewFilesystemRecorder(root string, identity Identity) (pipeline.CheckpointRecorder, error) {
root = strings.TrimSpace(root)
if root == "" {
return pipeline.NoopCheckpointRecorder(), nil
}
relative, err := identity.RelativePath()
if err != nil {
return nil, err
}
if strings.TrimSpace(root) == "" {
return pipeline.NoopCheckpointRecorder(), nil
target, err := fileio.SafePath(root, relative)
if err != nil {
return nil, err
}
return &WorkspaceRecorder{root: root, identityDigest: identity.Digest, now: time.Now}, nil
return &FilesystemRecorder{root: target, identityDigest: identity.Digest, now: time.Now}, nil
}
func (r *WorkspaceRecorder) SourceRunning(moduleKey string) error {
manifest := r.newStageManifest(coreworkspace.StageSource, coreworkspace.StatusRunning)
func (r *FilesystemRecorder) SourceRunning(moduleKey string) error {
manifest := r.newStageManifest(StageSource, StatusRunning)
manifest.ModuleKey = moduleKey
manifest.StartedAt = timePtr(r.timestamp())
return r.writeManifest("source/manifest.json", coreworkspace.SourceManifest{StageManifest: manifest})
return r.writeManifest("source/manifest.json", SourceManifest{StageManifest: manifest})
}
func (r *WorkspaceRecorder) SourceSucceeded(moduleKey string, doc *source.SourceDocument) error {
func (r *FilesystemRecorder) SourceSucceeded(moduleKey string, doc *source.SourceDocument) error {
if doc == nil {
return fmt.Errorf("checkpoint source document must not be nil")
}
if err := r.writePayload("source/source-document.json", sourceDocumentEnvelope{Document: cloneSourceDocument(*doc)}); err != nil {
return err
}
manifest := r.newStageManifest(coreworkspace.StageSource, coreworkspace.StatusSucceeded)
manifest := r.newStageManifest(StageSource, StatusSucceeded)
manifest.ModuleKey = moduleKey
manifest.OutputDigests = workspaceFingerprints(digestFingerprints("source_document", doc.Digest))
manifest.OutputDigests = checkpointFingerprints(digestFingerprints("source_document", doc.Digest))
manifest.CompletedAt = timePtr(r.timestamp())
return r.writeManifest("source/manifest.json", coreworkspace.SourceManifest{
return r.writeManifest("source/manifest.json", SourceManifest{
StageManifest: manifest,
SourceID: doc.ID,
})
}
func (r *WorkspaceRecorder) SourceFailed(moduleKey string, err error) error {
manifest := r.newStageManifest(coreworkspace.StageSource, coreworkspace.StatusFailed)
func (r *FilesystemRecorder) SourceFailed(moduleKey string, err error) error {
manifest := r.newStageManifest(StageSource, StatusFailed)
manifest.ModuleKey = moduleKey
manifest.CompletedAt = timePtr(r.timestamp())
manifest.Metadata = errorMetadata(err)
return r.writeManifest("source/manifest.json", coreworkspace.SourceManifest{StageManifest: manifest})
return r.writeManifest("source/manifest.json", SourceManifest{StageManifest: manifest})
}
func (r *WorkspaceRecorder) ExtractRunning(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint) error {
manifest := r.laneManifest(coreworkspace.StageExtract, coreworkspace.StatusRunning, laneID, moduleKey, dependencies)
func (r *FilesystemRecorder) ExtractRunning(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint) error {
manifest := r.laneManifest(StageExtract, StatusRunning, laneID, moduleKey, dependencies)
manifest.StartedAt = timePtr(r.timestamp())
return r.writeManifest(laneManifestPath("extract", laneID), coreworkspace.ExtractLaneManifest{StageManifest: manifest})
return r.writeManifest(laneManifestPath("extract", laneID), ExtractLaneManifest{StageManifest: manifest})
}
func (r *WorkspaceRecorder) ExtractSucceeded(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint, outputs []pipeline.CheckpointArtifact, rejected []contracts.RejectedOutput, warnings []contracts.Warning) error {
func (r *FilesystemRecorder) ExtractSucceeded(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint, outputs []pipeline.CheckpointArtifact, rejected []contracts.RejectedOutput, warnings []contracts.Warning) error {
payload := artifactExtractEnvelope{Outputs: artifactCheckpointEnvelopes(outputs), Rejected: cloneRejectedOutputs(rejected), Warnings: cloneWarnings(warnings)}
if err := r.writePayload(lanePayloadPath("extract", laneID, "outputs.json"), payload); err != nil {
return err
}
manifest := r.laneManifest(coreworkspace.StageExtract, statusForRejected(rejected), laneID, moduleKey, dependencies)
manifest.OutputDigests = workspaceFingerprints(artifactOutputDigests(outputs))
manifest := r.laneManifest(StageExtract, statusForRejected(rejected), laneID, moduleKey, dependencies)
manifest.OutputDigests = checkpointFingerprints(artifactOutputDigests(outputs))
manifest.ValidationStatus = validationStatusString(warnings, rejected)
manifest.Rejections = rejectionSummaries(rejected)
manifest.CompletedAt = timePtr(r.timestamp())
return r.writeManifest(laneManifestPath("extract", laneID), coreworkspace.ExtractLaneManifest{StageManifest: manifest, ChunkCount: len(outputs) + len(rejected), OutputCount: len(outputs)})
return r.writeManifest(laneManifestPath("extract", laneID), ExtractLaneManifest{StageManifest: manifest, ChunkCount: len(outputs) + len(rejected), OutputCount: len(outputs)})
}
func (r *WorkspaceRecorder) ExtractFailed(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, err error) error {
manifest := r.laneManifest(coreworkspace.StageExtract, coreworkspace.StatusFailed, laneID, moduleKey, dependencies)
func (r *FilesystemRecorder) ExtractFailed(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, err error) error {
manifest := r.laneManifest(StageExtract, StatusFailed, laneID, moduleKey, dependencies)
manifest.CompletedAt = timePtr(r.timestamp())
manifest.Metadata = errorMetadata(err)
return r.writeManifest(laneManifestPath("extract", laneID), coreworkspace.ExtractLaneManifest{StageManifest: manifest})
return r.writeManifest(laneManifestPath("extract", laneID), ExtractLaneManifest{StageManifest: manifest})
}
func (r *WorkspaceRecorder) MergeRunning(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint) error {
manifest := r.laneManifest(coreworkspace.StageMerge, coreworkspace.StatusRunning, laneID, moduleKey, dependencies)
func (r *FilesystemRecorder) MergeRunning(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint) error {
manifest := r.laneManifest(StageMerge, StatusRunning, laneID, moduleKey, dependencies)
manifest.StartedAt = timePtr(r.timestamp())
return r.writeManifest(laneManifestPath("merge", laneID), coreworkspace.MergeLaneManifest{StageManifest: manifest})
return r.writeManifest(laneManifestPath("merge", laneID), MergeLaneManifest{StageManifest: manifest})
}
func (r *WorkspaceRecorder) MergeSucceeded(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint, output pipeline.CheckpointArtifact, warnings []contracts.Warning) error {
func (r *FilesystemRecorder) MergeSucceeded(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint, output pipeline.CheckpointArtifact, warnings []contracts.Warning) error {
if err := r.writePayload(lanePayloadPath("merge", laneID, "output.json"), artifactSingleEnvelope{Output: artifactCheckpointEnvelopeFromOutput(output), Warnings: cloneWarnings(warnings)}); err != nil {
return err
}
manifest := r.laneManifest(coreworkspace.StageMerge, coreworkspace.StatusSucceeded, laneID, moduleKey, dependencies)
manifest.OutputDigests = workspaceFingerprints(artifactOutputDigests([]pipeline.CheckpointArtifact{output}))
manifest := r.laneManifest(StageMerge, StatusSucceeded, laneID, moduleKey, dependencies)
manifest.OutputDigests = checkpointFingerprints(artifactOutputDigests([]pipeline.CheckpointArtifact{output}))
manifest.ValidationStatus = validationStatusString(warnings, nil)
manifest.CompletedAt = timePtr(r.timestamp())
return r.writeManifest(laneManifestPath("merge", laneID), coreworkspace.MergeLaneManifest{StageManifest: manifest, InputCount: len(dependencies)})
return r.writeManifest(laneManifestPath("merge", laneID), MergeLaneManifest{StageManifest: manifest, InputCount: len(dependencies)})
}
func (r *WorkspaceRecorder) MergeRejected(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, rejected contracts.RejectedOutput) error {
manifest := r.laneManifest(coreworkspace.StageMerge, coreworkspace.StatusSucceededWithRejections, laneID, moduleKey, dependencies)
func (r *FilesystemRecorder) MergeRejected(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, rejected contracts.RejectedOutput) error {
manifest := r.laneManifest(StageMerge, StatusSucceededWithRejections, laneID, moduleKey, dependencies)
manifest.ValidationStatus = "rejected"
manifest.Rejections = rejectionSummaries([]contracts.RejectedOutput{rejected})
manifest.CompletedAt = timePtr(r.timestamp())
return r.writeManifest(laneManifestPath("merge", laneID), coreworkspace.MergeLaneManifest{StageManifest: manifest, InputCount: len(dependencies)})
return r.writeManifest(laneManifestPath("merge", laneID), MergeLaneManifest{StageManifest: manifest, InputCount: len(dependencies)})
}
func (r *WorkspaceRecorder) MergeFailed(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, err error) error {
manifest := r.laneManifest(coreworkspace.StageMerge, coreworkspace.StatusFailed, laneID, moduleKey, dependencies)
func (r *FilesystemRecorder) MergeFailed(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, err error) error {
manifest := r.laneManifest(StageMerge, StatusFailed, laneID, moduleKey, dependencies)
manifest.CompletedAt = timePtr(r.timestamp())
manifest.Metadata = errorMetadata(err)
return r.writeManifest(laneManifestPath("merge", laneID), coreworkspace.MergeLaneManifest{StageManifest: manifest})
return r.writeManifest(laneManifestPath("merge", laneID), MergeLaneManifest{StageManifest: manifest})
}
func (r *WorkspaceRecorder) NormalizeRunning(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint) error {
manifest := r.laneManifest(coreworkspace.StageNormalize, coreworkspace.StatusRunning, laneID, moduleKey, dependencies)
func (r *FilesystemRecorder) NormalizeRunning(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint) error {
manifest := r.laneManifest(StageNormalize, StatusRunning, laneID, moduleKey, dependencies)
manifest.StartedAt = timePtr(r.timestamp())
return r.writeManifest(laneManifestPath("normalize", laneID), coreworkspace.NormalizeLaneManifest{StageManifest: manifest})
return r.writeManifest(laneManifestPath("normalize", laneID), NormalizeLaneManifest{StageManifest: manifest})
}
func (r *WorkspaceRecorder) NormalizeSucceeded(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint, output pipeline.CheckpointArtifact, warnings []contracts.Warning) error {
func (r *FilesystemRecorder) NormalizeSucceeded(laneID, moduleKey string, dependencies []pipeline.CheckpointFingerprint, output pipeline.CheckpointArtifact, warnings []contracts.Warning) error {
if err := r.writePayload(lanePayloadPath("normalize", laneID, "output.json"), artifactSingleEnvelope{Output: artifactCheckpointEnvelopeFromOutput(output), Warnings: cloneWarnings(warnings)}); err != nil {
return err
}
manifest := r.laneManifest(coreworkspace.StageNormalize, coreworkspace.StatusSucceeded, laneID, moduleKey, dependencies)
manifest.OutputDigests = workspaceFingerprints(artifactOutputDigests([]pipeline.CheckpointArtifact{output}))
manifest := r.laneManifest(StageNormalize, StatusSucceeded, laneID, moduleKey, dependencies)
manifest.OutputDigests = checkpointFingerprints(artifactOutputDigests([]pipeline.CheckpointArtifact{output}))
manifest.ValidationStatus = validationStatusString(warnings, nil)
manifest.CompletedAt = timePtr(r.timestamp())
return r.writeManifest(laneManifestPath("normalize", laneID), coreworkspace.NormalizeLaneManifest{StageManifest: manifest, InputCount: len(dependencies)})
return r.writeManifest(laneManifestPath("normalize", laneID), NormalizeLaneManifest{StageManifest: manifest, InputCount: len(dependencies)})
}
func (r *WorkspaceRecorder) NormalizeRejected(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, rejected contracts.RejectedOutput) error {
manifest := r.laneManifest(coreworkspace.StageNormalize, coreworkspace.StatusSucceededWithRejections, laneID, moduleKey, dependencies)
func (r *FilesystemRecorder) NormalizeRejected(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, rejected contracts.RejectedOutput) error {
manifest := r.laneManifest(StageNormalize, StatusSucceededWithRejections, laneID, moduleKey, dependencies)
manifest.ValidationStatus = "rejected"
manifest.Rejections = rejectionSummaries([]contracts.RejectedOutput{rejected})
manifest.CompletedAt = timePtr(r.timestamp())
return r.writeManifest(laneManifestPath("normalize", laneID), coreworkspace.NormalizeLaneManifest{StageManifest: manifest, InputCount: len(dependencies)})
return r.writeManifest(laneManifestPath("normalize", laneID), NormalizeLaneManifest{StageManifest: manifest, InputCount: len(dependencies)})
}
func (r *WorkspaceRecorder) NormalizeFailed(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, err error) error {
manifest := r.laneManifest(coreworkspace.StageNormalize, coreworkspace.StatusFailed, laneID, moduleKey, dependencies)
func (r *FilesystemRecorder) NormalizeFailed(laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint, err error) error {
manifest := r.laneManifest(StageNormalize, StatusFailed, laneID, moduleKey, dependencies)
manifest.CompletedAt = timePtr(r.timestamp())
manifest.Metadata = errorMetadata(err)
return r.writeManifest(laneManifestPath("normalize", laneID), coreworkspace.NormalizeLaneManifest{StageManifest: manifest})
return r.writeManifest(laneManifestPath("normalize", laneID), NormalizeLaneManifest{StageManifest: manifest})
}
func (r *WorkspaceRecorder) writeManifest(name string, payload any) error {
func (r *FilesystemRecorder) writeManifest(name string, payload any) error {
return r.writeJSON(name, payload)
}
func (r *WorkspaceRecorder) writePayload(name string, payload any) error {
func (r *FilesystemRecorder) writePayload(name string, payload any) error {
return r.writeJSON(name, payload)
}
func (r *WorkspaceRecorder) writeJSON(name string, payload any) error {
func (r *FilesystemRecorder) writeJSON(name string, payload any) error {
if r == nil || strings.TrimSpace(r.root) == "" {
return nil
}
return coreworkspace.WriteJSON(r.root, name, payload)
return fileio.WriteJSON(r.root, name, payload, 0o700, 0o600)
}
func (r *WorkspaceRecorder) timestamp() time.Time {
func (r *FilesystemRecorder) timestamp() time.Time {
if r == nil || r.now == nil {
return time.Now().UTC()
}
return r.now().UTC()
}
func (r *WorkspaceRecorder) newStageManifest(stage coreworkspace.StageName, status coreworkspace.StageStatus) coreworkspace.StageManifest {
manifest := coreworkspace.NewStageManifest(stage, status)
func (r *FilesystemRecorder) newStageManifest(stage StageName, status StageStatus) StageManifest {
manifest := NewStageManifest(stage, status)
if strings.TrimSpace(r.identityDigest) != "" {
manifest.Metadata = map[string]string{"checkpoint_identity_digest": r.identityDigest}
}
return manifest
}
func (r *WorkspaceRecorder) laneManifest(stage coreworkspace.StageName, status coreworkspace.StageStatus, laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint) coreworkspace.StageManifest {
func (r *FilesystemRecorder) laneManifest(stage StageName, status StageStatus, laneID string, moduleKey string, dependencies []pipeline.CheckpointFingerprint) StageManifest {
manifest := r.newStageManifest(stage, status)
manifest.LaneID = laneID
manifest.ModuleKey = moduleKey
manifest.DependencyFingerprints = workspaceFingerprints(dependencies)
manifest.DependencyFingerprints = checkpointFingerprints(dependencies)
return manifest
}
@@ -316,14 +321,14 @@ func digestFingerprints(name string, digest string) []pipeline.CheckpointFingerp
return []pipeline.CheckpointFingerprint{{Name: name, Value: digest}}
}
func workspaceFingerprints(values []pipeline.CheckpointFingerprint) []coreworkspace.Fingerprint {
func checkpointFingerprints(values []pipeline.CheckpointFingerprint) []Fingerprint {
normalized := normalizeFingerprints(values)
if len(normalized) == 0 {
return nil
}
out := make([]coreworkspace.Fingerprint, 0, len(normalized))
out := make([]Fingerprint, 0, len(normalized))
for _, value := range normalized {
out = append(out, coreworkspace.Fingerprint{Name: value.Name, Value: value.Value})
out = append(out, Fingerprint{Name: value.Name, Value: value.Value})
}
return out
}
@@ -356,7 +361,7 @@ func normalizeFingerprints(values []pipeline.CheckpointFingerprint) []pipeline.C
return out
}
func rejectionSummaries(rejected []contracts.RejectedOutput) []coreworkspace.RejectionSummary {
func rejectionSummaries(rejected []contracts.RejectedOutput) []RejectionSummary {
if len(rejected) == 0 {
return nil
}
@@ -383,9 +388,9 @@ func rejectionSummaries(rejected []contracts.RejectedOutput) []coreworkspace.Rej
}
return keys[i].message < keys[j].message
})
out := make([]coreworkspace.RejectionSummary, 0, len(keys))
out := make([]RejectionSummary, 0, len(keys))
for _, k := range keys {
out = append(out, coreworkspace.RejectionSummary{
out = append(out, RejectionSummary{
ValidatorName: k.validatorName,
ReasonCode: k.reasonCode,
Message: k.message,
@@ -395,11 +400,11 @@ func rejectionSummaries(rejected []contracts.RejectedOutput) []coreworkspace.Rej
return out
}
func statusForRejected(rejected []contracts.RejectedOutput) coreworkspace.StageStatus {
func statusForRejected(rejected []contracts.RejectedOutput) StageStatus {
if len(rejected) > 0 {
return coreworkspace.StatusSucceededWithRejections
return StatusSucceededWithRejections
}
return coreworkspace.StatusSucceeded
return StatusSucceeded
}
func validationStatusString(warnings []contracts.Warning, rejected []contracts.RejectedOutput) string {

289
internal/framework/checkpoint/recorder_test.go
View File

@@ -1,270 +1,51 @@
package checkpoint
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
coreworkspace "gitea.maximumdirect.net/eric/notarius/internal/core/workspace"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestWorkspaceRecorderWritesSuccessfulCheckpointFiles(t *testing.T) {
func TestRootBasedRecorderOutputIsReusable(t *testing.T) {
root := t.TempDir()
recorder := newTestRecorder(t, root)
doc := &source.SourceDocument{
ID: "source-1",
Kind: "document",
Format: "text/plain",
Digest: "sha256:source",
Units: []source.SourceUnit{{ID: 1, Kind: "line", Text: "hello", Ref: source.SourceRef{SourceID: "source-1", StartUnitID: 1, EndUnitID: 1}}},
}
if err := recorder.SourceRunning("seriatim"); err != nil {
t.Fatalf("SourceRunning: %v", err)
}
assertManifestStatus(t, filepath.Join(root, "source", "manifest.json"), coreworkspace.StatusRunning)
if err := recorder.SourceSucceeded("seriatim", doc); err != nil {
t.Fatalf("SourceSucceeded: %v", err)
}
assertManifestStatus(t, filepath.Join(root, "source", "manifest.json"), coreworkspace.StatusSucceeded)
if _, err := os.Stat(filepath.Join(root, "source", "source-document.json")); err != nil {
t.Fatalf("expected source checkpoint payload: %v", err)
}
}
func TestWorkspaceArtifactCheckpointsRoundTripCodecIdentityAndBytes(t *testing.T) {
root := t.TempDir()
recorder := newTestRecorder(t, root)
loader := &WorkspaceLoader{root: root}
schema := contracts.ArtifactSchema{ID: "dnd.spell_response", Name: "spell response", Version: "v1", JSONSchema: []byte(`{"type":"object"}`)}
artifact := contracts.SerializedArtifact{Kind: "dnd.spells", Schema: schema, MediaType: "application/json", Content: []byte(`{"spell_casts":[]}`), Metadata: map[string]any{"spell_cast_count": float64(0)}}
stored := pipeline.CheckpointArtifact{LaneID: "spells", ModuleKey: "dnd/spells", SourceID: "source-1", ChunkID: "chunk-1", ChunkIndex: 2, ChunkRef: source.SourceRef{SourceID: "source-1", StartUnitID: 4, EndUnitID: 8}, Artifact: artifact, SchemaDigest: contracts.DigestArtifactSchema(schema)}
extractDeps := []pipeline.CheckpointFingerprint{{Name: "chunks", Value: "sha256:chunks"}}
if err := recorder.ExtractSucceeded("spells", "dnd/spells", extractDeps, []pipeline.CheckpointArtifact{stored}, nil, nil); err != nil {
t.Fatalf("ExtractSucceeded: %v", err)
}
extracted, decision := loader.Extract("spells", "dnd/spells", extractDeps)
if !decision.Reused || len(extracted.Outputs) != 1 {
t.Fatalf("extract decision=%#v checkpoint=%#v, want reused", decision, extracted)
}
got := extracted.Outputs[0]
if got.Artifact.Kind != artifact.Kind || got.Artifact.Schema.ID != schema.ID || got.Artifact.Schema.Version != schema.Version || got.SchemaDigest != stored.SchemaDigest || string(got.Artifact.Content) != string(artifact.Content) || got.ChunkRef != stored.ChunkRef {
t.Fatalf("artifact checkpoint = %#v, want codec identity, bytes, and provenance", got)
}
mergeDeps := artifactOutputDigests([]pipeline.CheckpointArtifact{stored})
if err := recorder.MergeSucceeded("spells", "merge", mergeDeps, stored, nil); err != nil {
t.Fatalf("MergeSucceeded: %v", err)
}
merged, decision := loader.Merge("spells", "merge", mergeDeps)
if !decision.Reused || string(merged.Output.Artifact.Content) != string(artifact.Content) {
t.Fatalf("merge decision=%#v checkpoint=%#v, want reused", decision, merged)
}
normalizeDeps := artifactOutputDigests([]pipeline.CheckpointArtifact{stored})
if err := recorder.NormalizeSucceeded("spells", "normalize", normalizeDeps, stored, nil); err != nil {
t.Fatalf("NormalizeSucceeded: %v", err)
}
normalized, decision := loader.Normalize("spells", "normalize", normalizeDeps)
if !decision.Reused || normalized.Output.SchemaDigest != stored.SchemaDigest {
t.Fatalf("normalize decision=%#v checkpoint=%#v, want reused", decision, normalized)
}
}
func TestWorkspaceLoaderInvalidatesMissingCorruptAndMismatchedCheckpoints(t *testing.T) {
t.Run("missing", func(t *testing.T) {
loader := &WorkspaceLoader{root: t.TempDir()}
if _, decision := loader.Source("seriatim"); decision.Reused || !strings.Contains(decision.Reason, "missing") {
t.Fatalf("decision = %#v, want missing invalidation", decision)
}
})
t.Run("incompatible workspace schema remains untouched", func(t *testing.T) {
root := t.TempDir()
recorder := newTestRecorder(t, root)
doc := &source.SourceDocument{
ID: "source-1", Kind: "document", Format: "text/plain", Digest: "sha256:source",
Units: []source.SourceUnit{{ID: 1, Kind: "line", Text: "hello", Ref: source.SourceRef{SourceID: "source-1", StartUnitID: 1, EndUnitID: 1}}},
}
if err := recorder.SourceSucceeded("seriatim", doc); err != nil {
t.Fatalf("SourceSucceeded: %v", err)
}
manifestPath := filepath.Join(root, "source", "manifest.json")
manifest := strings.Replace(string(readFile(t, manifestPath)), coreworkspace.WorkspaceSchemaVersion, coreworkspace.WorkspaceSchemaVersionV1, 1)
if err := os.WriteFile(manifestPath, []byte(manifest), 0o644); err != nil {
t.Fatalf("write legacy manifest: %v", err)
}
beforeManifest := readFile(t, manifestPath)
payloadPath := filepath.Join(root, "source", "source-document.json")
beforePayload := readFile(t, payloadPath)
loader := &WorkspaceLoader{root: root}
if _, decision := loader.Source("seriatim"); decision.Reused || !strings.Contains(decision.Reason, "incompatible") || !strings.Contains(decision.Reason, coreworkspace.WorkspaceSchemaVersionV1) {
t.Fatalf("decision = %#v, want incompatible legacy schema invalidation", decision)
}
if got := readFile(t, manifestPath); string(got) != string(beforeManifest) {
t.Fatal("legacy manifest changed during reuse decision")
}
if got := readFile(t, payloadPath); string(got) != string(beforePayload) {
t.Fatal("legacy payload changed during reuse decision")
}
})
}
func TestWorkspaceCheckpointsIgnoreLegacyChunkFiles(t *testing.T) {
root := t.TempDir()
legacyManifest := []byte(`{"legacy":"manifest"}`)
legacyPayload := []byte(`{"legacy":"chunks"}`)
legacyDir := filepath.Join(root, "chunk")
if err := os.MkdirAll(legacyDir, 0o700); err != nil {
t.Fatal(err)
}
manifestPath := filepath.Join(legacyDir, "manifest.json")
payloadPath := filepath.Join(legacyDir, "chunks.json")
if err := os.WriteFile(manifestPath, legacyManifest, 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(payloadPath, legacyPayload, 0o600); err != nil {
t.Fatal(err)
}
doc := &source.SourceDocument{ID: "source-1", Kind: "document", Format: "text/plain", Units: []source.SourceUnit{{ID: 1, Kind: "line", Text: "hello", Ref: source.SourceRef{SourceID: "source-1", StartUnitID: 1, EndUnitID: 1}}}}
doc.Digest, _ = source.DigestDocument(doc)
recorder := newTestRecorder(t, root)
if err := recorder.SourceSucceeded("seriatim", doc); err != nil {
t.Fatal(err)
}
loaded, decision := (&WorkspaceLoader{root: root}).Source("seriatim")
if !decision.Reused || loaded.Document == nil || loaded.Document.Digest != doc.Digest {
t.Fatalf("source checkpoint = %#v decision = %#v", loaded, decision)
}
if got := readFile(t, manifestPath); string(got) != string(legacyManifest) {
t.Fatalf("legacy manifest changed: %s", got)
}
if got := readFile(t, payloadPath); string(got) != string(legacyPayload) {
t.Fatalf("legacy payload changed: %s", got)
}
}
func TestWorkspaceRecorderRecordsRejectedExtractOutputs(t *testing.T) {
root := t.TempDir()
recorder := newTestRecorder(t, root)
rejected := []contracts.RejectedOutput{
{
Stage: string(pipeline.StageExtract),
LaneID: "spells",
ModuleKey: "dnd/spells",
ChunkID: "chunk-1",
ValidatorName: "shape",
ReasonCode: "invalid_shape",
Message: "bad shape",
},
}
if err := recorder.ExtractRunning("spells", "dnd/spells", []pipeline.CheckpointFingerprint{{Name: "chunks", Value: "sha256:chunks"}}); err != nil {
t.Fatalf("ExtractRunning: %v", err)
}
if err := recorder.ExtractSucceeded("spells", "dnd/spells", nil, nil, rejected, nil); err != nil {
t.Fatalf("ExtractSucceeded: %v", err)
}
var manifest coreworkspace.ExtractLaneManifest
readJSON(t, filepath.Join(root, "extract", "spells", "manifest.json"), &manifest)
if manifest.Status != coreworkspace.StatusSucceededWithRejections || manifest.ValidationStatus != "rejected" {
t.Fatalf("extract manifest status = %q validation=%q", manifest.Status, manifest.ValidationStatus)
}
if len(manifest.Rejections) != 1 || manifest.Rejections[0].Count != 1 || manifest.Rejections[0].ReasonCode != "invalid_shape" {
t.Fatalf("rejections = %#v", manifest.Rejections)
}
var payload struct {
Rejected []contracts.RejectedOutput `json:"rejected"`
}
readJSON(t, filepath.Join(root, "extract", "spells", "outputs.json"), &payload)
if len(payload.Rejected) != 1 || payload.Rejected[0].ChunkID != "chunk-1" {
t.Fatalf("checkpoint rejected payload = %#v", payload.Rejected)
}
}
func TestWorkspaceRecorderRecordsFailedStages(t *testing.T) {
root := t.TempDir()
recorder := newTestRecorder(t, root)
if err := recorder.MergeRunning("spells", "appendorder", nil); err != nil {
t.Fatalf("MergeRunning: %v", err)
}
if err := recorder.MergeFailed("spells", "appendorder", nil, assertErr("merge failed")); err != nil {
t.Fatalf("MergeFailed: %v", err)
}
var manifest coreworkspace.MergeLaneManifest
readJSON(t, filepath.Join(root, "merge", "spells", "manifest.json"), &manifest)
if manifest.Status != coreworkspace.StatusFailed {
t.Fatalf("status = %q, want failed", manifest.Status)
}
if !strings.Contains(manifest.Metadata["error"], "merge failed") {
t.Fatalf("metadata = %#v, want error", manifest.Metadata)
}
}
func TestWorkspaceRecorderRecordsWarningOnlyValidation(t *testing.T) {
root := t.TempDir()
recorder := newTestRecorder(t, root)
schema := contracts.ArtifactSchema{ID: "test.artifact", Name: "test_artifact", Version: "v1", JSONSchema: []byte(`{"type":"object"}`)}
output := pipeline.CheckpointArtifact{
LaneID: "events", ModuleKey: "noop", SourceID: "source-1",
Artifact: contracts.SerializedArtifact{Kind: "test/artifact", Schema: schema, MediaType: "application/json", Content: []byte(`{"ok":true}`)},
SchemaDigest: contracts.DigestArtifactSchema(schema),
}
warnings := []contracts.Warning{{ReasonCode: "note", Message: "warning"}}
if err := recorder.NormalizeSucceeded("events", "noop", nil, output, warnings); err != nil {
t.Fatalf("NormalizeSucceeded: %v", err)
}
var manifest coreworkspace.NormalizeLaneManifest
readJSON(t, filepath.Join(root, "normalize", "events", "manifest.json"), &manifest)
if manifest.Status != coreworkspace.StatusSucceeded || manifest.ValidationStatus != "approved_with_warnings" {
t.Fatalf("normalize manifest status = %q validation=%q", manifest.Status, manifest.ValidationStatus)
}
}
func newTestRecorder(t *testing.T, root string) *WorkspaceRecorder {
t.Helper()
return &WorkspaceRecorder{root: root}
}
func assertManifestStatus(t *testing.T, path string, want coreworkspace.StageStatus) {
t.Helper()
var manifest coreworkspace.StageManifest
readJSON(t, path, &manifest)
if manifest.Status != want {
t.Fatalf("%s status = %q, want %q", path, manifest.Status, want)
}
}
func readJSON(t *testing.T, path string, out any) {
t.Helper()
data := readFile(t, path)
if err := json.Unmarshal(data, out); err != nil {
t.Fatalf("decode %q: %v", path, err)
}
}
func readFile(t *testing.T, path string) []byte {
t.Helper()
data, err := os.ReadFile(path)
identity := testIdentity(t)
recorder, err := NewFilesystemRecorder(root, identity)
if err != nil {
t.Fatalf("read %q: %v", path, err)
t.Fatal(err)
}
if err := recorder.ExtractSucceeded("lane", "module", nil, nil, nil, nil); err != nil {
t.Fatal(err)
}
loader, err := NewFilesystemLoader(root, identity)
if err != nil {
t.Fatal(err)
}
result, decision := loader.Extract("lane", "module", nil)
if !decision.Reused || len(result.Outputs) != 0 {
t.Fatalf("load result=%#v decision=%#v", result, decision)
}
relative, err := identity.RelativePath()
if err != nil {
t.Fatal(err)
}
if _, err := os.Stat(filepath.Join(root, relative, "extract", "lane", "manifest.json")); err != nil {
t.Fatal(err)
}
return data
}
type assertErr string
func TestCheckpointSchemaCompatibilityIsUnchanged(t *testing.T) {
if WorkspaceSchemaVersion != "notarius.workspace.v2" || WorkspaceSchemaVersionV1 != "notarius.workspace.v1" {
t.Fatal("checkpoint schema identifiers changed")
}
}
func (e assertErr) Error() string { return string(e) }
func testIdentity(t *testing.T) Identity {
t.Helper()
identity, err := NewIdentity(IdentityInput{Pipeline: pipeline.ResolvedPipeline{ID: "pipeline", Digest: "sha256:aaaaaaaaaaaaaaaa", Input: pipeline.Binding("input")}, RawInputDigest: "sha256:bbbbbbbbbbbbbbbb"})
if err != nil {
t.Fatal(err)
}
return identity
}

23
internal/framework/debug/recorder.go
View File

@@ -3,39 +3,36 @@ package debug
import (
"strings"
coreworkspace "gitea.maximumdirect.net/eric/notarius/internal/core/workspace"
"gitea.maximumdirect.net/eric/notarius/internal/core/fileio"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
type WorkspaceRecorder struct {
type FilesystemRecorder struct {
root string
}
func NewWorkspaceRecorder(settings coreworkspace.Settings, runID string) (pipeline.DebugRecorder, error) {
root, err := settings.DebugRunDirectory(runID)
if err != nil {
return nil, err
}
func NewFilesystemRecorder(root string) (pipeline.DebugRecorder, error) {
root = strings.TrimSpace(root)
if strings.TrimSpace(root) == "" {
return pipeline.NoopDebugRecorder(), nil
}
return &WorkspaceRecorder{root: root}, nil
return &FilesystemRecorder{root: root}, nil
}
func (r *WorkspaceRecorder) Enabled() bool {
func (r *FilesystemRecorder) Enabled() bool {
return r != nil && strings.TrimSpace(r.root) != ""
}
func (r *WorkspaceRecorder) WriteJSON(name string, payload any) error {
func (r *FilesystemRecorder) WriteJSON(name string, payload any) error {
if !r.Enabled() {
return nil
}
return coreworkspace.WriteJSON(r.root, name, payload)
return fileio.WriteJSON(r.root, name, payload, 0o700, 0o600)
}
func (r *WorkspaceRecorder) WriteBytes(name string, data []byte) error {
func (r *FilesystemRecorder) WriteBytes(name string, data []byte) error {
if !r.Enabled() {
return nil
}
return coreworkspace.WriteBytes(r.root, name, data)
return fileio.WriteBytes(r.root, name, data, 0o700, 0o600)
}

28
internal/framework/debug/recorder_test.go Normal file
View File

@@ -0,0 +1,28 @@
package debug
import (
"os"
"path/filepath"
"testing"
)
func TestFilesystemRecorderWritesWithinTraceRoot(t *testing.T) {
root := t.TempDir()
recorder, err := NewFilesystemRecorder(root)
if err != nil {
t.Fatal(err)
}
if err := recorder.WriteBytes("attempt/data", []byte("payload")); err != nil {
t.Fatal(err)
}
info, err := os.Stat(filepath.Join(root, "attempt", "data"))
if err != nil {
t.Fatal(err)
}
if info.Mode().Perm() != 0o600 {
t.Fatalf("mode=%#o", info.Mode().Perm())
}
if err := recorder.WriteBytes("../outside", nil); err == nil {
t.Fatal("accepted traversal")
}
}

13
internal/framework/pipeline/debug.go
View File

@@ -491,10 +491,19 @@ func debugSourceChunkEnvelope(chunk source.Chunk) debugSourceChunk {
}
func cloneSourceUnitsForDebug(units []source.SourceUnit) []source.SourceUnit {
cloned, err := cloneSourceUnits(units)
if err != nil {
if len(units) == 0 {
return nil
}
cloned := make([]source.SourceUnit, len(units))
for i, unit := range units {
cloned[i] = source.SourceUnit{
ID: unit.ID,
Kind: unit.Kind,
Text: string(redactSecretBytes([]byte(unit.Text))),
Ref: unit.Ref,
Metadata: redactSensitiveMap(unit.Metadata),
}
}
return cloned
}

30
internal/framework/pipeline/debug_test.go
View File

@@ -2,6 +2,7 @@ package pipeline
import (
"encoding/json"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
@@ -29,6 +30,35 @@ func TestDebugSourceDocumentPreservesUnitReferences(t *testing.T) {
}
}
func TestDebugSourceUnitsRedactSecrets(t *testing.T) {
units := []source.SourceUnit{{
ID: 1,
Kind: "paragraph",
Text: "application text Bearer secretvalue sk-secretvalue",
Ref: source.SourceRef{SourceID: "source-1", StartUnitID: 1, EndUnitID: 1},
Metadata: map[string]any{"api_key": "sk-secretvalue"},
}}
got := cloneSourceUnitsForDebug(units)
if len(got) != 1 {
t.Fatalf("debug unit count = %d, want 1", len(got))
}
if !strings.Contains(got[0].Text, "application text") {
t.Fatalf("debug unit text = %q, want application content retained", got[0].Text)
}
for _, forbidden := range []string{"secretvalue", "sk-secretvalue"} {
if strings.Contains(got[0].Text, forbidden) {
t.Fatalf("debug unit text contains %q: %q", forbidden, got[0].Text)
}
}
if got, want := got[0].Metadata["api_key"], "[REDACTED]"; got != want {
t.Fatalf("debug unit metadata api_key = %#v, want %q", got, want)
}
if units[0].Text != "application text Bearer secretvalue sk-secretvalue" {
t.Fatalf("source unit text was mutated: %q", units[0].Text)
}
}
func TestDebugSourceChunkPreservesReference(t *testing.T) {
doc := validSourceDocument()
chunk := source.Chunk{

10
internal/modules/integration/testdata/pipeline.yml vendored
View File

@@ -1,4 +1,12 @@
version: 2
version: 3
output:
directory: ./notarius-output
cache:
chunk_plans:
mode: bypass
checkpoints: {}
debug:
directory: ./notarius-debug
pipelines:
dnd-spells-fixture:
input: seriatim

10
internal/modules/seriatim/input/transcript/testdata/pipeline.yml vendored
View File

@@ -1,4 +1,12 @@
version: 2
version: 3
output:
directory: ./notarius-output
cache:
chunk_plans:
mode: bypass
checkpoints: {}
debug:
directory: ./notarius-debug
pipelines:
seriatim-fixture:
input: seriatim