Add a planning roadmap and a staged implementation plan for the workspace configuration
This commit is contained in:
394
docs/roadmap/implementation.md
Normal file
394
docs/roadmap/implementation.md
Normal file
@@ -0,0 +1,394 @@
|
|||||||
|
# Workspace Implementation Plan
|
||||||
|
|
||||||
|
This plan implements the workspace target described in
|
||||||
|
[Workspace Roadmap](workspace.md). Follow the stages in order. Keep code changes
|
||||||
|
focused on the current stage, and update tests and canonical docs in the same
|
||||||
|
stage when behavior changes.
|
||||||
|
|
||||||
|
The intended end state is:
|
||||||
|
|
||||||
|
- one configurable workspace root for Notarius-owned local state;
|
||||||
|
- diagnostics written under `workspace/diagnostics/<run-id>/` when workspace
|
||||||
|
diagnostics are enabled;
|
||||||
|
- resumability checkpoints written under deterministic `workspace/checkpoints/`
|
||||||
|
paths only when resume checkpointing is enabled;
|
||||||
|
- debug artifacts written under `workspace/debug/<run-id>/` only when debug is
|
||||||
|
enabled;
|
||||||
|
- stage-owned checkpoint manifests only, with no root-level checkpoint summary;
|
||||||
|
- explicit resume behavior through `run --resume`, not implicit skipping during
|
||||||
|
ordinary `run`.
|
||||||
|
|
||||||
|
## Stage 1: Add Workspace Configuration
|
||||||
|
|
||||||
|
Add workspace configuration while preserving legacy diagnostics configuration
|
||||||
|
long enough for backward compatibility.
|
||||||
|
|
||||||
|
Implement in `internal/core/config`:
|
||||||
|
|
||||||
|
- Add `WorkspaceConfig` to `Config`.
|
||||||
|
- Add nested structs:
|
||||||
|
- `WorkspaceDiagnosticsConfig`
|
||||||
|
- `WorkspaceResumeConfig`
|
||||||
|
- `WorkspaceDebugConfig`
|
||||||
|
- Add file config parsing for:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
workspace:
|
||||||
|
directory: /var/lib/notarius
|
||||||
|
diagnostics:
|
||||||
|
enabled: true
|
||||||
|
retention: auto
|
||||||
|
resume:
|
||||||
|
enabled: false
|
||||||
|
debug:
|
||||||
|
enabled: false
|
||||||
|
```
|
||||||
|
|
||||||
|
- Preserve existing `diagnostics.work_dir` and `diagnostics.retention` parsing as
|
||||||
|
deprecated compatibility input.
|
||||||
|
- Resolve effective diagnostics behavior as follows:
|
||||||
|
- if `workspace.directory` is set, diagnostics root is
|
||||||
|
`<workspace.directory>/diagnostics`;
|
||||||
|
- if `workspace.directory` is unset and legacy `diagnostics.work_dir` is set,
|
||||||
|
use legacy diagnostics behavior unchanged;
|
||||||
|
- if neither is set, preserve the current default diagnostics behavior for
|
||||||
|
migration compatibility;
|
||||||
|
- `workspace.diagnostics.retention` overrides legacy diagnostics retention
|
||||||
|
when set;
|
||||||
|
- legacy diagnostics retention remains accepted when workspace diagnostics
|
||||||
|
retention is unset.
|
||||||
|
- If `workspace.diagnostics.enabled` is explicitly false, do not create a
|
||||||
|
diagnostics run directory and do not write diagnostics artifacts. CLI failure
|
||||||
|
handling must still print concise errors to stderr without assuming a
|
||||||
|
diagnostics directory exists.
|
||||||
|
- Add environment support:
|
||||||
|
- `NOTARIUS_WORKSPACE_DIR` sets `workspace.directory`;
|
||||||
|
- `NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED` parses a boolean;
|
||||||
|
- `NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION` sets workspace diagnostics
|
||||||
|
retention;
|
||||||
|
- `NOTARIUS_WORKSPACE_RESUME_ENABLED` parses a boolean;
|
||||||
|
- `NOTARIUS_WORKSPACE_DEBUG_ENABLED` parses a boolean;
|
||||||
|
- keep `NOTARIUS_WORK_DIR` and `NOTARIUS_DIAGNOSTICS_RETENTION` as deprecated
|
||||||
|
compatibility overrides for legacy diagnostics config.
|
||||||
|
- Keep `--diagnostics-dir` as a backwards-compatible per-invocation diagnostics
|
||||||
|
root override. It should affect diagnostics only and should not change
|
||||||
|
checkpoint or debug roots.
|
||||||
|
- Update redaction/effective-config diagnostics so workspace config is included
|
||||||
|
and no secrets are introduced.
|
||||||
|
|
||||||
|
Tests:
|
||||||
|
|
||||||
|
- Add config default tests for workspace defaults.
|
||||||
|
- Add file config parsing tests for nested workspace fields.
|
||||||
|
- Add env override tests for all new env vars.
|
||||||
|
- Add precedence tests covering workspace config, legacy diagnostics config,
|
||||||
|
env overrides, and `--diagnostics-dir`.
|
||||||
|
- Update redacted/effective config tests.
|
||||||
|
|
||||||
|
## Stage 2: Introduce Workspace Filesystem Helpers
|
||||||
|
|
||||||
|
Create reusable filesystem helpers for workspace state. Keep concrete module
|
||||||
|
packages out of this layer.
|
||||||
|
|
||||||
|
Implement a new package, recommended path `internal/core/workspace`, with:
|
||||||
|
|
||||||
|
- `Config` or `Settings` describing effective workspace roots:
|
||||||
|
- root directory;
|
||||||
|
- diagnostics root;
|
||||||
|
- checkpoints root;
|
||||||
|
- debug root;
|
||||||
|
- enabled flags.
|
||||||
|
- path construction helpers for:
|
||||||
|
- diagnostics run directories;
|
||||||
|
- checkpoint identity directories;
|
||||||
|
- debug run directories.
|
||||||
|
- safe path helpers that reject absolute artifact names, `..`, backslashes, and
|
||||||
|
paths that escape their intended root.
|
||||||
|
- atomic JSON and byte-file writes using the existing diagnostics atomic-write
|
||||||
|
behavior as the model.
|
||||||
|
- optional shared internal helper for atomic file writes so diagnostics and
|
||||||
|
workspace writers do not duplicate low-level write logic.
|
||||||
|
|
||||||
|
Do not add resume behavior in this stage.
|
||||||
|
|
||||||
|
Tests:
|
||||||
|
|
||||||
|
- Unit-test path construction and path-safety failures.
|
||||||
|
- Unit-test atomic JSON/byte writes.
|
||||||
|
- Unit-test disabled workspace settings returning no-op or empty roots as
|
||||||
|
appropriate.
|
||||||
|
- Confirm no helper permits writes outside the configured root.
|
||||||
|
|
||||||
|
## Stage 3: Move Diagnostics Under Workspace
|
||||||
|
|
||||||
|
Update diagnostics creation to use the effective diagnostics root from workspace
|
||||||
|
configuration when workspace is configured.
|
||||||
|
|
||||||
|
Implementation requirements:
|
||||||
|
|
||||||
|
- Keep diagnostics artifacts and retention behavior unchanged.
|
||||||
|
- Honor disabled diagnostics by using a no-op diagnostics writer or nil-safe
|
||||||
|
diagnostics path through the CLI failure and success paths.
|
||||||
|
- Preserve the existing `diagnostics.RunDirectory` contract unless a narrow
|
||||||
|
constructor addition is cleaner.
|
||||||
|
- Route normal workspace diagnostics to:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<workspace.directory>/diagnostics/<run-id>/
|
||||||
|
```
|
||||||
|
|
||||||
|
- Preserve legacy behavior when only legacy diagnostics config is present.
|
||||||
|
- Preserve `--diagnostics-dir` behavior as diagnostics-only override.
|
||||||
|
- Keep diagnostics run IDs run-based and non-deterministic.
|
||||||
|
- Do not write checkpoint or debug output in this stage.
|
||||||
|
|
||||||
|
Docs to update after behavior exists:
|
||||||
|
|
||||||
|
- `docs/config.md`
|
||||||
|
- `docs/operations.md`
|
||||||
|
- `docs/internal/diagnostics.md`
|
||||||
|
- `docs/cli.md` if CLI help text changes.
|
||||||
|
|
||||||
|
Tests:
|
||||||
|
|
||||||
|
- Update diagnostics run directory tests for workspace diagnostics roots.
|
||||||
|
- Update CLI tests that inspect diagnostics paths.
|
||||||
|
- Add migration tests proving legacy diagnostics config still works.
|
||||||
|
- Run focused checks:
|
||||||
|
- `go test ./internal/core/config`
|
||||||
|
- `go test ./internal/core/diagnostics`
|
||||||
|
- `go test ./internal/cli`
|
||||||
|
|
||||||
|
## Stage 4: Define Checkpoint Identity And Stage Manifest Types
|
||||||
|
|
||||||
|
Add checkpoint identity and manifest data structures before writing stage
|
||||||
|
payloads.
|
||||||
|
|
||||||
|
Implement in `internal/core/workspace` or a closely related core package:
|
||||||
|
|
||||||
|
- `CheckpointIdentity`, derived from:
|
||||||
|
- resolved pipeline ID;
|
||||||
|
- resolved pipeline digest;
|
||||||
|
- input adapter key;
|
||||||
|
- raw input digest or source digest;
|
||||||
|
- selected lanes;
|
||||||
|
- runtime overrides that affect execution;
|
||||||
|
- materialized reference digests;
|
||||||
|
- prompt/schema/profile provenance not already represented by the pipeline
|
||||||
|
digest.
|
||||||
|
- A deterministic, filesystem-safe checkpoint path:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<workspace.directory>/checkpoints/<pipeline-id>/<input-key>-<source-digest-prefix>/<pipeline-digest-prefix>/
|
||||||
|
```
|
||||||
|
|
||||||
|
- Stage manifest structs for:
|
||||||
|
- source;
|
||||||
|
- chunk;
|
||||||
|
- extract lane;
|
||||||
|
- merge lane;
|
||||||
|
- normalize lane.
|
||||||
|
- Shared manifest fields:
|
||||||
|
- workspace schema version;
|
||||||
|
- stage name;
|
||||||
|
- lane ID when applicable;
|
||||||
|
- module key;
|
||||||
|
- dependency fingerprints;
|
||||||
|
- status;
|
||||||
|
- output digests;
|
||||||
|
- validation status and rejection summaries where applicable;
|
||||||
|
- started/completed timestamps where useful.
|
||||||
|
- Status values:
|
||||||
|
- `pending`;
|
||||||
|
- `running`;
|
||||||
|
- `succeeded`;
|
||||||
|
- `succeeded_with_rejections`;
|
||||||
|
- `failed`;
|
||||||
|
- `invalidated`.
|
||||||
|
|
||||||
|
Do not add a root-level checkpoint manifest.
|
||||||
|
|
||||||
|
Tests:
|
||||||
|
|
||||||
|
- Unit-test deterministic identity generation.
|
||||||
|
- Unit-test identity changes when pipeline digest, input digest, selected lanes,
|
||||||
|
or reference digests change.
|
||||||
|
- Unit-test manifest JSON round trips.
|
||||||
|
- Unit-test filesystem-safe path generation.
|
||||||
|
|
||||||
|
## Stage 5: Write Checkpoints Without Resuming
|
||||||
|
|
||||||
|
Add write-only checkpoint support behind `workspace.resume.enabled`.
|
||||||
|
|
||||||
|
Implementation requirements:
|
||||||
|
|
||||||
|
- Add a framework-owned checkpoint recorder to pipeline execution. Recommended
|
||||||
|
shape:
|
||||||
|
- CLI constructs the effective workspace/checkpoint recorder after pipeline
|
||||||
|
resolution and reference materialization.
|
||||||
|
- `pipeline.RunInput` receives a recorder interface or no-op recorder.
|
||||||
|
- concrete modules do not receive workspace paths and do not write directly to
|
||||||
|
workspace.
|
||||||
|
- Write checkpoint artifacts only when `workspace.resume.enabled` is true.
|
||||||
|
- Use stage-owned manifests and no root summary.
|
||||||
|
- Use JSON envelope types for checkpointed payloads that preserve byte content,
|
||||||
|
such as `content_base64`, media type, metadata, warnings, and content digest.
|
||||||
|
Do not rely on existing runtime structs whose byte fields are tagged
|
||||||
|
`json:"-"`.
|
||||||
|
- Recommended checkpoint files:
|
||||||
|
|
||||||
|
```text
|
||||||
|
checkpoints/<identity>/source/manifest.json
|
||||||
|
checkpoints/<identity>/source/source-document.json
|
||||||
|
checkpoints/<identity>/chunk/manifest.json
|
||||||
|
checkpoints/<identity>/chunk/chunks.json
|
||||||
|
checkpoints/<identity>/extract/<lane-id>/manifest.json
|
||||||
|
checkpoints/<identity>/extract/<lane-id>/outputs.json
|
||||||
|
checkpoints/<identity>/merge/<lane-id>/manifest.json
|
||||||
|
checkpoints/<identity>/merge/<lane-id>/output.json
|
||||||
|
checkpoints/<identity>/normalize/<lane-id>/manifest.json
|
||||||
|
checkpoints/<identity>/normalize/<lane-id>/output.json
|
||||||
|
```
|
||||||
|
|
||||||
|
- Record rejected extract outputs as checkpointed stage outcomes. Under current
|
||||||
|
pipeline policy they do not pass downstream, but they are not framework errors.
|
||||||
|
- Mark a stage `running` before writing its payloads, then atomically replace the
|
||||||
|
manifest with a final success/failure status after payload writes complete.
|
||||||
|
- Ensure interrupted or partial writes cannot be mistaken for successful
|
||||||
|
checkpoints.
|
||||||
|
- Do not write raw prompts, raw references, raw LLM request payloads, or debug
|
||||||
|
traces in checkpoint output.
|
||||||
|
|
||||||
|
Tests:
|
||||||
|
|
||||||
|
- Add runner/CLI tests proving no checkpoint files are written when resume is
|
||||||
|
disabled.
|
||||||
|
- Add tests proving checkpoint files are written when resume is enabled.
|
||||||
|
- Add tests for successful runs, rejected extract outputs, failed stages, and
|
||||||
|
warning-only validation.
|
||||||
|
- Add tests proving checkpoint payload content digests match written content.
|
||||||
|
- Add tests proving modules do not receive filesystem paths.
|
||||||
|
|
||||||
|
## Stage 6: Add Explicit Resume Reads
|
||||||
|
|
||||||
|
Add explicit resume behavior after write-only checkpoints are stable.
|
||||||
|
|
||||||
|
CLI behavior:
|
||||||
|
|
||||||
|
- Add `run --resume`.
|
||||||
|
- `--resume` should require `workspace.resume.enabled: true`; otherwise return a
|
||||||
|
clear configuration error.
|
||||||
|
- Plain `run` should continue to execute stages normally and should not silently
|
||||||
|
skip completed stages.
|
||||||
|
|
||||||
|
Resume behavior:
|
||||||
|
|
||||||
|
- Load and validate checkpoint stage manifests in workflow order.
|
||||||
|
- Reuse a checkpoint only when:
|
||||||
|
- workspace schema version is supported;
|
||||||
|
- stage status is successful for the current purpose;
|
||||||
|
- dependency fingerprints match the current invocation;
|
||||||
|
- referenced checkpoint payload files exist;
|
||||||
|
- payload digests match manifest digests;
|
||||||
|
- selected lanes and runtime overrides are compatible.
|
||||||
|
- If a checkpoint is missing or invalid, execute that stage normally and write a
|
||||||
|
fresh checkpoint if resume checkpointing remains enabled.
|
||||||
|
- If a source or chunk checkpoint is reused, downstream dependency fingerprints
|
||||||
|
must still be validated before downstream reuse.
|
||||||
|
- If an extract checkpoint includes rejected outputs, preserve those rejection
|
||||||
|
records and continue to omit rejected outputs from merge input.
|
||||||
|
- If merge is reused, skip merge execution only for the matching lane.
|
||||||
|
- If normalize is reused, pass the reused normalized output to the output stage.
|
||||||
|
- Always create fresh diagnostics for the current invocation, including records
|
||||||
|
showing which stages were reused versus executed.
|
||||||
|
|
||||||
|
Tests:
|
||||||
|
|
||||||
|
- Add CLI tests for `--resume` without workspace resume enabled.
|
||||||
|
- Add tests for reusing source, chunk, extract, merge, and normalize
|
||||||
|
checkpoints.
|
||||||
|
- Add tests for invalidation when input, pipeline digest, references, selected
|
||||||
|
lanes, or runtime LLM profile override changes.
|
||||||
|
- Add tests for corrupt or missing checkpoint payloads.
|
||||||
|
- Add tests proving fresh diagnostics are written for resumed invocations.
|
||||||
|
|
||||||
|
## Stage 7: Add Debug Output
|
||||||
|
|
||||||
|
Add debug output behind `workspace.debug.enabled`.
|
||||||
|
|
||||||
|
Implementation requirements:
|
||||||
|
|
||||||
|
- Write debug artifacts under:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<workspace.directory>/debug/<run-id>/
|
||||||
|
```
|
||||||
|
|
||||||
|
- Debug is per-invocation and should not be used for resume.
|
||||||
|
- Debug and resume are independent:
|
||||||
|
- debug enabled does not imply checkpoint writing;
|
||||||
|
- resume enabled does not imply debug writing.
|
||||||
|
- Start with artifacts available from Notarius framework boundaries:
|
||||||
|
- source/checkpoint-like stage inputs and outputs;
|
||||||
|
- structured LLM request inputs and response content from Notarius contracts;
|
||||||
|
- validator requests and results;
|
||||||
|
- stage timing and attempt metadata.
|
||||||
|
- Do not depend on Scriptorium internals for rendered upstream provider
|
||||||
|
requests. If Scriptorium later exposes rendered prompt traces safely, add them
|
||||||
|
in a separate pass.
|
||||||
|
- Never write API keys, bearer tokens, or raw provider credentials.
|
||||||
|
- Document clearly that debug output may contain source material, references,
|
||||||
|
prompts, model outputs, and other sensitive content.
|
||||||
|
|
||||||
|
Tests:
|
||||||
|
|
||||||
|
- Add tests proving no debug files are written when debug is disabled.
|
||||||
|
- Add tests proving debug files are written under `debug/<run-id>/` when enabled.
|
||||||
|
- Add tests proving debug and resume can be enabled independently.
|
||||||
|
- Add tests proving obvious secrets are not written.
|
||||||
|
- Add diagnostics/debug path tests to ensure path roots do not overlap
|
||||||
|
accidentally.
|
||||||
|
|
||||||
|
## Stage 8: Documentation, Examples, And Cleanup
|
||||||
|
|
||||||
|
After behavior is implemented, update canonical current-behavior docs.
|
||||||
|
|
||||||
|
Update:
|
||||||
|
|
||||||
|
- `docs/config.md`
|
||||||
|
- `docs/cli.md`
|
||||||
|
- `docs/operations.md`
|
||||||
|
- `docs/troubleshooting.md`
|
||||||
|
- `docs/internal/diagnostics.md`
|
||||||
|
- `docs/internal/pipeline.md`
|
||||||
|
- maintained examples under `examples/` when practical.
|
||||||
|
|
||||||
|
Documentation requirements:
|
||||||
|
|
||||||
|
- Document workspace config and defaults.
|
||||||
|
- Document legacy diagnostics config compatibility and any deprecation language.
|
||||||
|
- Document `/var/lib/notarius` as the production recommendation.
|
||||||
|
- Document local development recommendations.
|
||||||
|
- Document checkpoint sensitivity and debug sensitivity.
|
||||||
|
- Document explicit resume behavior and invalidation rules.
|
||||||
|
- Keep future or deferred behavior only in roadmap files.
|
||||||
|
|
||||||
|
Cleanup:
|
||||||
|
|
||||||
|
- Update `docs/roadmap/workspace.md` to a completed-status note after the
|
||||||
|
implementation lands.
|
||||||
|
- Remove or replace this implementation plan with a completed note after the
|
||||||
|
implementation lands.
|
||||||
|
|
||||||
|
Validation:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test ./internal/core/config
|
||||||
|
go test ./internal/core/diagnostics
|
||||||
|
go test ./internal/core/workspace
|
||||||
|
go test ./internal/framework/pipeline
|
||||||
|
go test ./internal/cli
|
||||||
|
go test ./...
|
||||||
|
go vet ./...
|
||||||
|
go build ./cmd/notarius
|
||||||
|
```
|
||||||
233
docs/roadmap/workspace.md
Normal file
233
docs/roadmap/workspace.md
Normal file
@@ -0,0 +1,233 @@
|
|||||||
|
# Workspace Roadmap
|
||||||
|
|
||||||
|
Notarius should gain a single configurable workspace root for application-owned
|
||||||
|
local state. The workspace exists only to support enabled workspace features;
|
||||||
|
ordinary runs should not write workspace files by default.
|
||||||
|
|
||||||
|
The workspace is distinct from durable output:
|
||||||
|
|
||||||
|
- durable output is the user-requested final product of a run;
|
||||||
|
- the workspace is application-owned local state for diagnostics, resumable
|
||||||
|
checkpoints, and development/debug workflows.
|
||||||
|
|
||||||
|
Within the workspace, diagnostics, checkpoints, and debug artifacts should use
|
||||||
|
separate subdirectories because they have different identity and lifecycle
|
||||||
|
models.
|
||||||
|
|
||||||
|
## Target Configuration
|
||||||
|
|
||||||
|
Workspace behavior should be configured under a dedicated top-level object:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
workspace:
|
||||||
|
directory: /var/lib/notarius
|
||||||
|
diagnostics:
|
||||||
|
enabled: true
|
||||||
|
retention: auto
|
||||||
|
resume:
|
||||||
|
enabled: false
|
||||||
|
debug:
|
||||||
|
enabled: false
|
||||||
|
```
|
||||||
|
|
||||||
|
Policy:
|
||||||
|
|
||||||
|
- `workspace.diagnostics.enabled` defaults to the current diagnostics behavior
|
||||||
|
during migration.
|
||||||
|
- `workspace.resume.enabled` defaults to `false`.
|
||||||
|
- `workspace.debug.enabled` defaults to `false`.
|
||||||
|
- no checkpoint or debug files are written when those features are disabled.
|
||||||
|
- `/var/lib/notarius` should be the standard production recommendation in docs
|
||||||
|
and examples.
|
||||||
|
- local development docs may recommend a project-local path such as
|
||||||
|
`./.notarius/workspace`.
|
||||||
|
|
||||||
|
`workspace.directory` should become the single configured root for Notarius local
|
||||||
|
state. Existing `diagnostics.work_dir` behavior should be migrated carefully for
|
||||||
|
backward compatibility, but the long-term configuration model should avoid two
|
||||||
|
separate roots for application-owned local state.
|
||||||
|
|
||||||
|
## Workspace Layout
|
||||||
|
|
||||||
|
The workspace root should contain separate subtrees:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<workspace.directory>/
|
||||||
|
diagnostics/
|
||||||
|
checkpoints/
|
||||||
|
debug/
|
||||||
|
```
|
||||||
|
|
||||||
|
Diagnostics remain run-id based. They are invocation history and should preserve
|
||||||
|
the existing diagnostics retention model:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<workspace.directory>/
|
||||||
|
diagnostics/
|
||||||
|
<run-id>/
|
||||||
|
invocation.json
|
||||||
|
effective-config.json
|
||||||
|
resolved-pipeline.json
|
||||||
|
resolved-references.json
|
||||||
|
run-manifest.json
|
||||||
|
warnings.json
|
||||||
|
run-report.json
|
||||||
|
error.log
|
||||||
|
```
|
||||||
|
|
||||||
|
Checkpoints should be deterministic and should include enough identity material
|
||||||
|
to prevent unsafe reuse across incompatible runs:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<workspace.directory>/
|
||||||
|
checkpoints/
|
||||||
|
<pipeline-id>/
|
||||||
|
<input-key>-<source-digest-prefix>/
|
||||||
|
<pipeline-digest-prefix>/
|
||||||
|
source/
|
||||||
|
chunk/
|
||||||
|
extract/<lane-id>/
|
||||||
|
merge/<lane-id>/
|
||||||
|
normalize/<lane-id>/
|
||||||
|
```
|
||||||
|
|
||||||
|
Checkpoint identity should account for:
|
||||||
|
|
||||||
|
- resolved pipeline ID;
|
||||||
|
- resolved pipeline digest;
|
||||||
|
- input adapter key;
|
||||||
|
- source/input digest;
|
||||||
|
- selected lanes and runtime overrides that affect execution;
|
||||||
|
- reference digests;
|
||||||
|
- prompt, schema, and profile provenance when not already captured by the
|
||||||
|
resolved pipeline digest.
|
||||||
|
|
||||||
|
Debug output should be run-id based like diagnostics, because debug traces are
|
||||||
|
development artifacts from a specific invocation rather than resumable state:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<workspace.directory>/
|
||||||
|
debug/
|
||||||
|
<run-id>/
|
||||||
|
```
|
||||||
|
|
||||||
|
Human-readable path segments are useful, but content hashes should be
|
||||||
|
authoritative for correctness. The exact path shape may change during
|
||||||
|
implementation, but checkpoint identity must be stable, deterministic, and safe
|
||||||
|
to compare across runs. Diagnostics and debug output do not need deterministic
|
||||||
|
resume identity because they are per-invocation artifacts.
|
||||||
|
|
||||||
|
## Stage Ownership
|
||||||
|
|
||||||
|
Stage manifests should be stage-owned. There should not be a root-level manifest
|
||||||
|
that summarizes all stages.
|
||||||
|
|
||||||
|
Rationale:
|
||||||
|
|
||||||
|
- stage manifests are the authoritative state for the stage that wrote them;
|
||||||
|
- avoiding a root summary prevents duplicate state from drifting;
|
||||||
|
- stage-local manifests make partial writes, failure recovery, and future stage
|
||||||
|
invalidation easier to reason about.
|
||||||
|
|
||||||
|
Each stage manifest should record enough information to determine whether its
|
||||||
|
outputs can be trusted for resume:
|
||||||
|
|
||||||
|
- workspace schema/version;
|
||||||
|
- stage name and lane ID when applicable;
|
||||||
|
- module key and relevant module provenance;
|
||||||
|
- dependency fingerprints and input digests;
|
||||||
|
- status such as pending, running, succeeded, succeeded with rejections, failed,
|
||||||
|
or invalidated;
|
||||||
|
- output content digests;
|
||||||
|
- validation status and rejection summaries where relevant;
|
||||||
|
- timing metadata when useful and non-sensitive.
|
||||||
|
|
||||||
|
## Checkpoint Output
|
||||||
|
|
||||||
|
When `workspace.resume.enabled` is enabled, Notarius should write under the
|
||||||
|
`checkpoints/` subtree. Checkpoints should contain only artifacts required to
|
||||||
|
resume safely.
|
||||||
|
|
||||||
|
Checkpoint artifacts may include:
|
||||||
|
|
||||||
|
- source document checkpoint;
|
||||||
|
- chunk collection checkpoint;
|
||||||
|
- accepted extract outputs;
|
||||||
|
- rejected extract records;
|
||||||
|
- merge output;
|
||||||
|
- normalize output;
|
||||||
|
- stage manifests with dependency fingerprints and output digests.
|
||||||
|
|
||||||
|
Checkpoint output should avoid raw prompts, raw references, raw LLM request
|
||||||
|
payloads, and other sensitive development artifacts unless they are strictly
|
||||||
|
required for safe resume. Prefer digests and provenance over copying sensitive
|
||||||
|
inputs.
|
||||||
|
|
||||||
|
Checkpoint writes should be atomic at the file level. Partial or interrupted
|
||||||
|
writes must not be mistaken for successful stage completion.
|
||||||
|
|
||||||
|
## Debug Output
|
||||||
|
|
||||||
|
When `workspace.debug.enabled` is enabled, Notarius should write under the
|
||||||
|
`debug/<run-id>/` subtree. Debug artifacts are useful for inspection but not
|
||||||
|
required for resume.
|
||||||
|
|
||||||
|
Debug artifacts may include:
|
||||||
|
|
||||||
|
- rendered prompt inputs;
|
||||||
|
- structured LLM request and response traces;
|
||||||
|
- raw model responses;
|
||||||
|
- copied reference content;
|
||||||
|
- copied source snippets;
|
||||||
|
- intermediate raw payloads;
|
||||||
|
- validator request/response details.
|
||||||
|
|
||||||
|
Debug output is explicitly sensitive. Documentation should warn that debug
|
||||||
|
workspace data may contain source material, references, prompts, and model
|
||||||
|
outputs. Debug output should remain disabled by default.
|
||||||
|
|
||||||
|
`workspace.debug.enabled` and `workspace.resume.enabled` should be independent.
|
||||||
|
Enabling debug should not imply resume, and enabling resume should not imply
|
||||||
|
debug.
|
||||||
|
|
||||||
|
## Resume Semantics
|
||||||
|
|
||||||
|
Resume should be explicit at first, for example through a future `resume` command
|
||||||
|
or a `run --resume` flag. Plain `run` should not silently skip completed stages
|
||||||
|
in the initial workspace feature.
|
||||||
|
|
||||||
|
Before reusing a checkpoint, Notarius should verify that:
|
||||||
|
|
||||||
|
- the workspace schema/version is supported;
|
||||||
|
- the checkpoint stage manifest is complete and successful;
|
||||||
|
- dependency fingerprints match the current invocation;
|
||||||
|
- referenced checkpoint files exist and match recorded digests;
|
||||||
|
- selected lanes and runtime overrides are compatible with the checkpoint.
|
||||||
|
|
||||||
|
Rejected extract outputs should be treated as recorded stage outcomes, not
|
||||||
|
framework failures. Under current pipeline policy, rejected outputs do not pass
|
||||||
|
to downstream stages.
|
||||||
|
|
||||||
|
## Boundaries
|
||||||
|
|
||||||
|
Workspace writing should be framework-owned. Concrete modules should not write
|
||||||
|
directly to workspace paths.
|
||||||
|
|
||||||
|
If modules need to expose debug material later, they should return logical
|
||||||
|
artifacts through framework contracts and let framework or CLI code perform path
|
||||||
|
validation, redaction policy, and writes.
|
||||||
|
|
||||||
|
Workspace state should preserve existing Notarius boundaries:
|
||||||
|
|
||||||
|
- provider plumbing remains behind LLM runtime contracts;
|
||||||
|
- prompt ownership remains with modules and prompt asset helpers;
|
||||||
|
- diagnostics remain per-run invocation artifacts under the workspace root;
|
||||||
|
- output modules continue to own final durable output encoding;
|
||||||
|
- secrets and sensitive payloads are not written unless an explicit debug policy
|
||||||
|
enables them.
|
||||||
|
|
||||||
|
## Deferred Work
|
||||||
|
|
||||||
|
Default-idempotent `run` behavior with a `--force` override, remote workspace
|
||||||
|
storage, workspace garbage collection, archival policy, and cross-machine resume
|
||||||
|
are deferred.
|
||||||
Reference in New Issue
Block a user