eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document completed workspace behavior

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-08 03:18:50 +00:00
parent a5bbfea9b9
commit ae65b95374
9 changed files with 178 additions and 623 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/cli.md
View File

@@ -42,7 +42,7 @@ Flags:
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are
comma-separated and must be non-empty.
- `--resume`: reuse valid workspace checkpoints for this invocation. Requires
`workspace.resume.enabled: true`.
an effective workspace directory and `workspace.resume.enabled: true`.
- `--output-dir path`: output root. The run writes to `<path>/<run-id>/`.
Defaults to `./notarius-output`.
- `--diagnostics-dir path`: diagnostics work directory override for this
@@ -152,6 +152,11 @@ go run ./cmd/notarius run dnd-session \
--resume
```
Plain `run` does not skip completed work. It executes the pipeline normally and
refreshes checkpoints when checkpointing is enabled. `--resume` verifies each
checkpoint before reuse and executes any missing, corrupt, or incompatible step
normally.
For durable output, diagnostics, retention, and failure inspection, see
[Operations](operations.md).

47
docs/config.md
View File

@@ -67,6 +67,10 @@ workspace:
enabled: false
```
`workspace.directory` is unset by default. Without a workspace directory,
diagnostics continue to use `/tmp/notarius`, and checkpoint and debug workspace
features have no storage root.
No pipelines are built in. A run requires a configured pipeline.
If `scriptorium` is omitted, Notarius uses Scriptorium's built-in profile
@@ -352,12 +356,40 @@ casts still must be present in the source transcript.
- `resume.enabled`: boolean resume checkpointing setting. Default: `false`.
- `debug.enabled`: boolean debug artifact setting. Default: `false`.
When `workspace.resume.enabled` is true, runs write stage-owned checkpoint
artifacts under `<workspace.directory>/checkpoints/`. `notarius run --resume`
can reuse valid checkpoints from a compatible invocation.
Use `/var/lib/notarius` as the standard production workspace directory. For
local development, prefer a project-local ignored path such as
`./.notarius/workspace`.
When `workspace.debug.enabled` is true, runs write per-invocation debug
artifacts under `<workspace.directory>/debug/<run-id>/`.
```yaml
workspace:
directory: /var/lib/notarius
diagnostics:
enabled: true
retention: auto
resume:
enabled: false
debug:
enabled: false
```
When `workspace.directory` is set, diagnostics are written under
`<workspace.directory>/diagnostics/`.
When both `workspace.directory` and `workspace.resume.enabled` are set, runs
write stage-owned checkpoint artifacts under
`<workspace.directory>/checkpoints/`. `notarius run --resume` can reuse valid
checkpoints from a compatible invocation. Checkpoints may contain source text,
intermediate raw outputs, rejected outputs, metadata, and warnings. Protect the
workspace as sensitive local state.
When both `workspace.directory` and `workspace.debug.enabled` are set, runs
write per-invocation debug artifacts under
`<workspace.directory>/debug/<run-id>/`. Debug artifacts may contain source
material, reference material, prompt inputs, model outputs, validation payloads,
and other sensitive content. Debug is disabled by default.
`workspace.resume.enabled` and `workspace.debug.enabled` are independent.
Enabling one does not enable the other.
## Diagnostics
@@ -380,6 +412,11 @@ set.
- `retention`: deprecated compatibility retention mode. `auto`, `always`, or
`never`. Empty uses `auto`.
Existing `diagnostics.work_dir`, `diagnostics.retention`, `NOTARIUS_WORK_DIR`,
and `NOTARIUS_DIAGNOSTICS_RETENTION` inputs remain supported for compatibility.
New configuration should use `workspace.directory` and
`workspace.diagnostics.retention` instead.
`auto` retains diagnostics for failed runs and successful runs with warnings.
`always` retains diagnostics for every run. `never` removes diagnostics for
successful runs without regard to warnings; failed runs are retained.

7
docs/internal/diagnostics.md
View File

@@ -22,6 +22,11 @@ Diagnostics must not expose secrets.
If `workDir` is empty, it defaults to `/tmp/notarius`. Empty retention defaults
to `auto`.
The CLI passes the effective diagnostics root from workspace configuration.
When `workspace.directory` is set and diagnostics are enabled, that root is
`<workspace.directory>/diagnostics`. The legacy diagnostics work directory and
`--diagnostics-dir` still pass a diagnostics-only root to this constructor.
The writer makes the work directory if needed, then attempts to create a unique
run directory. It retries run ID creation a bounded number of times if a
collision occurs.
@@ -93,3 +98,5 @@ manifest before logging the failure.
information needed for recovery.
- Durable output file contracts belong to output modules and integration docs,
not to diagnostics.
- Checkpoint and debug workspace files are separate framework-owned artifacts,
not diagnostics artifacts.

10
docs/internal/pipeline.md
View File

@@ -80,13 +80,17 @@ workspace paths and do not write checkpoint files directly.
For `run --resume`, the CLI also passes a checkpoint loader. The runner consults
the loader in workflow order and reuses only checkpoints whose manifest schema,
status, identity digest, dependency fingerprints, payload files, and payload
digests validate for the current invocation. Missing or invalid checkpoints fall
back to normal execution and are refreshed by the recorder.
digests validate for the current invocation. The identity includes the resolved
pipeline, selected lanes, source/input digest, runtime overrides that affect
execution, and materialized reference digests. Missing or invalid checkpoints
fall back to normal execution and are refreshed by the recorder.
When workspace debug output is enabled, the CLI passes a debug recorder for the
current run ID. The runner writes framework-boundary inputs, outputs,
structured LLM calls, validator calls, timing, and retry attempt metadata
through that interface. Concrete modules still do not receive workspace paths.
through that interface. Debug output is not used for resume and can contain
sensitive source, reference, prompt, and model-output material. Concrete modules
still do not receive workspace paths.
## Registries And Module Specs

20
docs/operations.md
View File

@@ -19,6 +19,10 @@ go run ./cmd/notarius run dnd-session \
The command prints a success line with the pipeline ID, normalized output count,
rejected output count, and the output path.
For production, configure a workspace such as `/var/lib/notarius` and ensure the
Notarius process can create files below it. For local development, prefer an
ignored project-local workspace such as `./.notarius/workspace`.
## Output Directory
Durable output is written to:
@@ -103,7 +107,9 @@ checkpoints and executes any missing, invalid, or incompatible step normally.
Checkpoint payloads preserve byte content with base64 envelopes, media type,
metadata, warnings, and content digests where applicable. Checkpoints do not
include raw prompts, raw reference contents, raw LLM request payloads, or debug
traces.
traces. They can still contain source text, intermediate extracted content,
rejected outputs, metadata, and warnings. Treat checkpoint directories as
sensitive local state.
A checkpoint is reused only when its workspace schema version, checkpoint
identity digest, step status, dependency fingerprints, payload files, and
@@ -111,6 +117,10 @@ payload digests match the current invocation. Changes to input bytes, resolved
pipeline digest, selected lanes, runtime LLM profile override, or materialized
reference digests invalidate reuse.
Plain `notarius run` does not reuse checkpoints. It executes the workflow and
refreshes checkpoint files when checkpointing is enabled. `notarius run
--resume` is the explicit reuse path.
## Debug
When `workspace.debug.enabled: true` and `workspace.directory` is set, runs
@@ -181,6 +191,14 @@ rm -rf /tmp/notarius/run-1234567890
rm -rf ./notarius-output/run-1234567890
```
Workspace checkpoint and debug directories can also be removed when no longer
needed. Remove exact identity or run directories, for example:
```sh
rm -rf /var/lib/notarius/checkpoints/dnd-session/seriatim-abcdef123456/7890abcd1234
rm -rf /var/lib/notarius/debug/run-1234567890
```
Use exact run-directory paths. Avoid broad cleanup commands against parent
directories unless they are part of your own operational policy.

420
docs/roadmap/implementation.md
View File

@@ -1,394 +1,26 @@
# Workspace Implementation Plan
This plan implements the workspace target described in
[Workspace Roadmap](workspace.md). Follow the stages in order. Keep code changes
focused on the current stage, and update tests and canonical docs in the same
stage when behavior changes.
The intended end state is:
- one configurable workspace root for Notarius-owned local state;
- diagnostics written under `workspace/diagnostics/<run-id>/` when workspace
diagnostics are enabled;
- resumability checkpoints written under deterministic `workspace/checkpoints/`
paths only when resume checkpointing is enabled;
- debug artifacts written under `workspace/debug/<run-id>/` only when debug is
enabled;
- stage-owned checkpoint manifests only, with no root-level checkpoint summary;
- explicit resume behavior through `run --resume`, not implicit skipping during
ordinary `run`.
## Stage 1: Add Workspace Configuration
Add workspace configuration while preserving legacy diagnostics configuration
long enough for backward compatibility.
Implement in `internal/core/config`:
- Add `WorkspaceConfig` to `Config`.
- Add nested structs:
- `WorkspaceDiagnosticsConfig`
- `WorkspaceResumeConfig`
- `WorkspaceDebugConfig`
- Add file config parsing for:
```yaml
workspace:
directory: /var/lib/notarius
diagnostics:
enabled: true
retention: auto
resume:
enabled: false
debug:
enabled: false
```
- Preserve existing `diagnostics.work_dir` and `diagnostics.retention` parsing as
deprecated compatibility input.
- Resolve effective diagnostics behavior as follows:
- if `workspace.directory` is set, diagnostics root is
`<workspace.directory>/diagnostics`;
- if `workspace.directory` is unset and legacy `diagnostics.work_dir` is set,
use legacy diagnostics behavior unchanged;
- if neither is set, preserve the current default diagnostics behavior for
migration compatibility;
- `workspace.diagnostics.retention` overrides legacy diagnostics retention
when set;
- legacy diagnostics retention remains accepted when workspace diagnostics
retention is unset.
- If `workspace.diagnostics.enabled` is explicitly false, do not create a
diagnostics run directory and do not write diagnostics artifacts. CLI failure
handling must still print concise errors to stderr without assuming a
diagnostics directory exists.
- Add environment support:
- `NOTARIUS_WORKSPACE_DIR` sets `workspace.directory`;
- `NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED` parses a boolean;
- `NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION` sets workspace diagnostics
retention;
- `NOTARIUS_WORKSPACE_RESUME_ENABLED` parses a boolean;
- `NOTARIUS_WORKSPACE_DEBUG_ENABLED` parses a boolean;
- keep `NOTARIUS_WORK_DIR` and `NOTARIUS_DIAGNOSTICS_RETENTION` as deprecated
compatibility overrides for legacy diagnostics config.
- Keep `--diagnostics-dir` as a backwards-compatible per-invocation diagnostics
root override. It should affect diagnostics only and should not change
checkpoint or debug roots.
- Update redaction/effective-config diagnostics so workspace config is included
and no secrets are introduced.
Tests:
- Add config default tests for workspace defaults.
- Add file config parsing tests for nested workspace fields.
- Add env override tests for all new env vars.
- Add precedence tests covering workspace config, legacy diagnostics config,
env overrides, and `--diagnostics-dir`.
- Update redacted/effective config tests.
## Stage 2: Introduce Workspace Filesystem Helpers
Create reusable filesystem helpers for workspace state. Keep concrete module
packages out of this layer.
Implement a new package, recommended path `internal/core/workspace`, with:
- `Config` or `Settings` describing effective workspace roots:
- root directory;
- diagnostics root;
- checkpoints root;
- debug root;
- enabled flags.
- path construction helpers for:
- diagnostics run directories;
- checkpoint identity directories;
- debug run directories.
- safe path helpers that reject absolute artifact names, `..`, backslashes, and
paths that escape their intended root.
- atomic JSON and byte-file writes using the existing diagnostics atomic-write
behavior as the model.
- optional shared internal helper for atomic file writes so diagnostics and
workspace writers do not duplicate low-level write logic.
Do not add resume behavior in this stage.
Tests:
- Unit-test path construction and path-safety failures.
- Unit-test atomic JSON/byte writes.
- Unit-test disabled workspace settings returning no-op or empty roots as
appropriate.
- Confirm no helper permits writes outside the configured root.
## Stage 3: Move Diagnostics Under Workspace
Update diagnostics creation to use the effective diagnostics root from workspace
configuration when workspace is configured.
Implementation requirements:
- Keep diagnostics artifacts and retention behavior unchanged.
- Honor disabled diagnostics by using a no-op diagnostics writer or nil-safe
diagnostics path through the CLI failure and success paths.
- Preserve the existing `diagnostics.RunDirectory` contract unless a narrow
constructor addition is cleaner.
- Route normal workspace diagnostics to:
```text
<workspace.directory>/diagnostics/<run-id>/
```
- Preserve legacy behavior when only legacy diagnostics config is present.
- Preserve `--diagnostics-dir` behavior as diagnostics-only override.
- Keep diagnostics run IDs run-based and non-deterministic.
- Do not write checkpoint or debug output in this stage.
Docs to update after behavior exists:
- `docs/config.md`
- `docs/operations.md`
- `docs/internal/diagnostics.md`
- `docs/cli.md` if CLI help text changes.
Tests:
- Update diagnostics run directory tests for workspace diagnostics roots.
- Update CLI tests that inspect diagnostics paths.
- Add migration tests proving legacy diagnostics config still works.
- Run focused checks:
- `go test ./internal/core/config`
- `go test ./internal/core/diagnostics`
- `go test ./internal/cli`
## Stage 4: Define Checkpoint Identity And Stage Manifest Types
Add checkpoint identity and manifest data structures before writing stage
payloads.
Implement in `internal/core/workspace` or a closely related core package:
- `CheckpointIdentity`, derived from:
- resolved pipeline ID;
- resolved pipeline digest;
- input adapter key;
- raw input digest or source digest;
- selected lanes;
- runtime overrides that affect execution;
- materialized reference digests;
- prompt/schema/profile provenance not already represented by the pipeline
digest.
- A deterministic, filesystem-safe checkpoint path:
```text
<workspace.directory>/checkpoints/<pipeline-id>/<input-key>-<source-digest-prefix>/<pipeline-digest-prefix>/
```
- Stage manifest structs for:
- source;
- chunk;
- extract lane;
- merge lane;
- normalize lane.
- Shared manifest fields:
- workspace schema version;
- stage name;
- lane ID when applicable;
- module key;
- dependency fingerprints;
- status;
- output digests;
- validation status and rejection summaries where applicable;
- started/completed timestamps where useful.
- Status values:
- `pending`;
- `running`;
- `succeeded`;
- `succeeded_with_rejections`;
- `failed`;
- `invalidated`.
Do not add a root-level checkpoint manifest.
Tests:
- Unit-test deterministic identity generation.
- Unit-test identity changes when pipeline digest, input digest, selected lanes,
or reference digests change.
- Unit-test manifest JSON round trips.
- Unit-test filesystem-safe path generation.
## Stage 5: Write Checkpoints Without Resuming
Add write-only checkpoint support behind `workspace.resume.enabled`.
Implementation requirements:
- Add a framework-owned checkpoint recorder to pipeline execution. Recommended
shape:
- CLI constructs the effective workspace/checkpoint recorder after pipeline
resolution and reference materialization.
- `pipeline.RunInput` receives a recorder interface or no-op recorder.
- concrete modules do not receive workspace paths and do not write directly to
workspace.
- Write checkpoint artifacts only when `workspace.resume.enabled` is true.
- Use stage-owned manifests and no root summary.
- Use JSON envelope types for checkpointed payloads that preserve byte content,
such as `content_base64`, media type, metadata, warnings, and content digest.
Do not rely on existing runtime structs whose byte fields are tagged
`json:"-"`.
- Recommended checkpoint files:
```text
checkpoints/<identity>/source/manifest.json
checkpoints/<identity>/source/source-document.json
checkpoints/<identity>/chunk/manifest.json
checkpoints/<identity>/chunk/chunks.json
checkpoints/<identity>/extract/<lane-id>/manifest.json
checkpoints/<identity>/extract/<lane-id>/outputs.json
checkpoints/<identity>/merge/<lane-id>/manifest.json
checkpoints/<identity>/merge/<lane-id>/output.json
checkpoints/<identity>/normalize/<lane-id>/manifest.json
checkpoints/<identity>/normalize/<lane-id>/output.json
```
- Record rejected extract outputs as checkpointed stage outcomes. Under current
pipeline policy they do not pass downstream, but they are not framework errors.
- Mark a stage `running` before writing its payloads, then atomically replace the
manifest with a final success/failure status after payload writes complete.
- Ensure interrupted or partial writes cannot be mistaken for successful
checkpoints.
- Do not write raw prompts, raw references, raw LLM request payloads, or debug
traces in checkpoint output.
Tests:
- Add runner/CLI tests proving no checkpoint files are written when resume is
disabled.
- Add tests proving checkpoint files are written when resume is enabled.
- Add tests for successful runs, rejected extract outputs, failed stages, and
warning-only validation.
- Add tests proving checkpoint payload content digests match written content.
- Add tests proving modules do not receive filesystem paths.
## Stage 6: Add Explicit Resume Reads
Add explicit resume behavior after write-only checkpoints are stable.
CLI behavior:
- Add `run --resume`.
- `--resume` should require `workspace.resume.enabled: true`; otherwise return a
clear configuration error.
- Plain `run` should continue to execute stages normally and should not silently
skip completed stages.
Resume behavior:
- Load and validate checkpoint stage manifests in workflow order.
- Reuse a checkpoint only when:
- workspace schema version is supported;
- stage status is successful for the current purpose;
- dependency fingerprints match the current invocation;
- referenced checkpoint payload files exist;
- payload digests match manifest digests;
- selected lanes and runtime overrides are compatible.
- If a checkpoint is missing or invalid, execute that stage normally and write a
fresh checkpoint if resume checkpointing remains enabled.
- If a source or chunk checkpoint is reused, downstream dependency fingerprints
must still be validated before downstream reuse.
- If an extract checkpoint includes rejected outputs, preserve those rejection
records and continue to omit rejected outputs from merge input.
- If merge is reused, skip merge execution only for the matching lane.
- If normalize is reused, pass the reused normalized output to the output stage.
- Always create fresh diagnostics for the current invocation, including records
showing which stages were reused versus executed.
Tests:
- Add CLI tests for `--resume` without workspace resume enabled.
- Add tests for reusing source, chunk, extract, merge, and normalize
checkpoints.
- Add tests for invalidation when input, pipeline digest, references, selected
lanes, or runtime LLM profile override changes.
- Add tests for corrupt or missing checkpoint payloads.
- Add tests proving fresh diagnostics are written for resumed invocations.
## Stage 7: Add Debug Output
Add debug output behind `workspace.debug.enabled`.
Implementation requirements:
- Write debug artifacts under:
```text
<workspace.directory>/debug/<run-id>/
```
- Debug is per-invocation and should not be used for resume.
- Debug and resume are independent:
- debug enabled does not imply checkpoint writing;
- resume enabled does not imply debug writing.
- Start with artifacts available from Notarius framework boundaries:
- source/checkpoint-like stage inputs and outputs;
- structured LLM request inputs and response content from Notarius contracts;
- validator requests and results;
- stage timing and attempt metadata.
- Do not depend on Scriptorium internals for rendered upstream provider
requests. If Scriptorium later exposes rendered prompt traces safely, add them
in a separate pass.
- Never write API keys, bearer tokens, or raw provider credentials.
- Document clearly that debug output may contain source material, references,
prompts, model outputs, and other sensitive content.
Tests:
- Add tests proving no debug files are written when debug is disabled.
- Add tests proving debug files are written under `debug/<run-id>/` when enabled.
- Add tests proving debug and resume can be enabled independently.
- Add tests proving obvious secrets are not written.
- Add diagnostics/debug path tests to ensure path roots do not overlap
accidentally.
## Stage 8: Documentation, Examples, And Cleanup
After behavior is implemented, update canonical current-behavior docs.
Update:
- `docs/config.md`
- `docs/cli.md`
- `docs/operations.md`
- `docs/troubleshooting.md`
- `docs/internal/diagnostics.md`
- `docs/internal/pipeline.md`
- maintained examples under `examples/` when practical.
Documentation requirements:
- Document workspace config and defaults.
- Document legacy diagnostics config compatibility and any deprecation language.
- Document `/var/lib/notarius` as the production recommendation.
- Document local development recommendations.
- Document checkpoint sensitivity and debug sensitivity.
- Document explicit resume behavior and invalidation rules.
- Keep future or deferred behavior only in roadmap files.
Cleanup:
- Update `docs/roadmap/workspace.md` to a completed-status note after the
implementation lands.
- Remove or replace this implementation plan with a completed note after the
implementation lands.
Validation:
```sh
go test ./internal/core/config
go test ./internal/core/diagnostics
go test ./internal/core/workspace
go test ./internal/framework/pipeline
go test ./internal/cli
go test ./...
go vet ./...
go build ./cmd/notarius
```
# Workspace Implementation Status
The workspace implementation described by this roadmap has landed. Current
behavior is documented in the canonical current-behavior docs:
- [Configuration](../config.md)
- [CLI Reference](../cli.md)
- [Operations](../operations.md)
- [Troubleshooting](../troubleshooting.md)
- [Diagnostics Internals](../internal/diagnostics.md)
- [Pipeline Internals](../internal/pipeline.md)
Implemented behavior includes:
- `workspace.directory` as the root for Notarius-owned local state;
- workspace diagnostics under `<workspace.directory>/diagnostics/<run-id>/`;
- compatibility for legacy `diagnostics.work_dir`, `diagnostics.retention`,
`NOTARIUS_WORK_DIR`, and `NOTARIUS_DIAGNOSTICS_RETENTION`;
- checkpoint writes under `<workspace.directory>/checkpoints/` when resume
checkpointing is enabled;
- explicit checkpoint reuse through `notarius run --resume`;
- debug artifacts under `<workspace.directory>/debug/<run-id>/` when debug
output is enabled;
- independent resume and debug settings.
Deferred workspace ideas remain in [Workspace Roadmap](workspace.md).

231
docs/roadmap/workspace.md
View File

@@ -1,55 +1,11 @@
# Workspace Roadmap
# Workspace Roadmap Status
Notarius should gain a single configurable workspace root for application-owned
local state. The workspace exists only to support enabled workspace features;
ordinary runs should not write workspace files by default.
The local workspace feature has been implemented. Current behavior is documented
in [Configuration](../config.md), [CLI Reference](../cli.md),
[Operations](../operations.md), and the relevant internal docs.
The workspace is distinct from durable output:
- durable output is the user-requested final product of a run;
- the workspace is application-owned local state for diagnostics, resumable
checkpoints, and development/debug workflows.
Within the workspace, diagnostics, checkpoints, and debug artifacts should use
separate subdirectories because they have different identity and lifecycle
models.
## Target Configuration
Workspace behavior should be configured under a dedicated top-level object:
```yaml
workspace:
directory: /var/lib/notarius
diagnostics:
enabled: true
retention: auto
resume:
enabled: false
debug:
enabled: false
```
Policy:
- `workspace.diagnostics.enabled` defaults to the current diagnostics behavior
during migration.
- `workspace.resume.enabled` defaults to `false`.
- `workspace.debug.enabled` defaults to `false`.
- no checkpoint or debug files are written when those features are disabled.
- `/var/lib/notarius` should be the standard production recommendation in docs
and examples.
- local development docs may recommend a project-local path such as
`./.notarius/workspace`.
`workspace.directory` should become the single configured root for Notarius local
state. Existing `diagnostics.work_dir` behavior should be migrated carefully for
backward compatibility, but the long-term configuration model should avoid two
separate roots for application-owned local state.
## Workspace Layout
The workspace root should contain separate subtrees:
The implemented workspace provides one configurable root for Notarius-owned
local state:
```text
<workspace.directory>/
@@ -58,176 +14,13 @@ The workspace root should contain separate subtrees:
debug/
```
Diagnostics remain run-id based. They are invocation history and should preserve
the existing diagnostics retention model:
```text
<workspace.directory>/
diagnostics/
<run-id>/
invocation.json
effective-config.json
resolved-pipeline.json
resolved-references.json
run-manifest.json
warnings.json
run-report.json
error.log
```
Checkpoints should be deterministic and should include enough identity material
to prevent unsafe reuse across incompatible runs:
```text
<workspace.directory>/
checkpoints/
<pipeline-id>/
<input-key>-<source-digest-prefix>/
<pipeline-digest-prefix>/
source/
chunk/
extract/<lane-id>/
merge/<lane-id>/
normalize/<lane-id>/
```
Checkpoint identity should account for:
- resolved pipeline ID;
- resolved pipeline digest;
- input adapter key;
- source/input digest;
- selected lanes and runtime overrides that affect execution;
- reference digests;
- prompt, schema, and profile provenance when not already captured by the
resolved pipeline digest.
Debug output should be run-id based like diagnostics, because debug traces are
development artifacts from a specific invocation rather than resumable state:
```text
<workspace.directory>/
debug/
<run-id>/
```
Human-readable path segments are useful, but content hashes should be
authoritative for correctness. The exact path shape may change during
implementation, but checkpoint identity must be stable, deterministic, and safe
to compare across runs. Diagnostics and debug output do not need deterministic
resume identity because they are per-invocation artifacts.
## Stage Ownership
Stage manifests should be stage-owned. There should not be a root-level manifest
that summarizes all stages.
Rationale:
- stage manifests are the authoritative state for the stage that wrote them;
- avoiding a root summary prevents duplicate state from drifting;
- stage-local manifests make partial writes, failure recovery, and future stage
invalidation easier to reason about.
Each stage manifest should record enough information to determine whether its
outputs can be trusted for resume:
- workspace schema/version;
- stage name and lane ID when applicable;
- module key and relevant module provenance;
- dependency fingerprints and input digests;
- status such as pending, running, succeeded, succeeded with rejections, failed,
or invalidated;
- output content digests;
- validation status and rejection summaries where relevant;
- timing metadata when useful and non-sensitive.
## Checkpoint Output
When `workspace.resume.enabled` is enabled, Notarius should write under the
`checkpoints/` subtree. Checkpoints should contain only artifacts required to
resume safely.
Checkpoint artifacts may include:
- source document checkpoint;
- chunk collection checkpoint;
- accepted extract outputs;
- rejected extract records;
- merge output;
- normalize output;
- stage manifests with dependency fingerprints and output digests.
Checkpoint output should avoid raw prompts, raw references, raw LLM request
payloads, and other sensitive development artifacts unless they are strictly
required for safe resume. Prefer digests and provenance over copying sensitive
inputs.
Checkpoint writes should be atomic at the file level. Partial or interrupted
writes must not be mistaken for successful stage completion.
## Debug Output
When `workspace.debug.enabled` is enabled, Notarius should write under the
`debug/<run-id>/` subtree. Debug artifacts are useful for inspection but not
required for resume.
Debug artifacts may include:
- rendered prompt inputs;
- structured LLM request and response traces;
- raw model responses;
- copied reference content;
- copied source snippets;
- intermediate raw payloads;
- validator request/response details.
Debug output is explicitly sensitive. Documentation should warn that debug
workspace data may contain source material, references, prompts, and model
outputs. Debug output should remain disabled by default.
`workspace.debug.enabled` and `workspace.resume.enabled` should be independent.
Enabling debug should not imply resume, and enabling resume should not imply
debug.
## Resume Semantics
Resume should be explicit at first, for example through a future `resume` command
or a `run --resume` flag. Plain `run` should not silently skip completed stages
in the initial workspace feature.
Before reusing a checkpoint, Notarius should verify that:
- the workspace schema/version is supported;
- the checkpoint stage manifest is complete and successful;
- dependency fingerprints match the current invocation;
- referenced checkpoint files exist and match recorded digests;
- selected lanes and runtime overrides are compatible with the checkpoint.
Rejected extract outputs should be treated as recorded stage outcomes, not
framework failures. Under current pipeline policy, rejected outputs do not pass
to downstream stages.
## Boundaries
Workspace writing should be framework-owned. Concrete modules should not write
directly to workspace paths.
If modules need to expose debug material later, they should return logical
artifacts through framework contracts and let framework or CLI code perform path
validation, redaction policy, and writes.
Workspace state should preserve existing Notarius boundaries:
- provider plumbing remains behind LLM runtime contracts;
- prompt ownership remains with modules and prompt asset helpers;
- diagnostics remain per-run invocation artifacts under the workspace root;
- output modules continue to own final durable output encoding;
- secrets and sensitive payloads are not written unless an explicit debug policy
enables them.
Implemented behavior includes workspace-backed diagnostics, checkpoint writing,
explicit checkpoint reuse through `notarius run --resume`, workspace debug
artifacts, safe workspace-relative writes, and compatibility for legacy
diagnostics configuration.
## Deferred Work
Default-idempotent `run` behavior with a `--force` override, remote workspace
Default-idempotent `run` behavior with a force override, remote workspace
storage, workspace garbage collection, archival policy, and cross-machine resume
are deferred.
remain deferred.

44
docs/troubleshooting.md
View File

@@ -280,6 +280,50 @@ Fix:
- Pass `--session-id <id>` to `notarius run`.
- Use a stable, non-secret identifier from the external orchestrator.
## Resume Or Checkpoint Reuse Failure
Symptoms include:
- `--resume requires workspace.resume.enabled: true`
- `checkpoint artifact is missing`
- `checkpoint workspace schema version`
- `checkpoint dependency fingerprints do not match`
- a resumed run executes work instead of reusing a checkpoint
Fix:
- Set both `workspace.directory` and `workspace.resume.enabled: true`.
- Use `--resume`; plain `notarius run` executes normally and refreshes
checkpoints.
- Confirm the current run uses the same input bytes, resolved pipeline, selected
lanes, runtime LLM profile override, and materialized references as the run
that wrote the checkpoint.
- Inspect retained diagnostics `checkpoint-events.json` to see which workflow
steps were reused or executed.
- If a checkpoint payload is missing or corrupt, rerun without relying on that
checkpoint. Notarius executes invalidated steps normally and writes fresh
checkpoints when checkpointing remains enabled.
Checkpoint files can contain source text, intermediate outputs, rejected
outputs, metadata, and warnings. Protect the workspace directory accordingly.
## Debug Output Missing Or Too Verbose
Symptoms:
- no files appear under `<workspace.directory>/debug/<run-id>/`;
- debug files contain more source, reference, prompt, or model-output material
than expected.
Fix:
- Set both `workspace.directory` and `workspace.debug.enabled: true`.
- Confirm you are inspecting the current run ID. Debug output is per invocation
and is not used for resume.
- Disable `workspace.debug.enabled` after the inspection run. Debug output may
contain sensitive source material, reference material, prompt inputs, model
outputs, and validation payloads.
## Output Write Failure
Symptoms include:

15
examples/dnd-spells.config.yml
View File

@@ -1,4 +1,19 @@
version: 2
# For production runs, use a writable application-owned workspace such as:
#
# workspace:
# directory: /var/lib/notarius
# diagnostics:
# retention: auto
# resume:
# enabled: false
# debug:
# enabled: false
#
# For local development, use a project-local ignored path such as:
#
# workspace:
# directory: ./.notarius/workspace
pipelines:
dnd-session:
input: seriatim