395 lines
14 KiB
Markdown
395 lines
14 KiB
Markdown
# Workspace Implementation Plan
|
|
|
|
This plan implements the workspace target described in
|
|
[Workspace Roadmap](workspace.md). Follow the stages in order. Keep code changes
|
|
focused on the current stage, and update tests and canonical docs in the same
|
|
stage when behavior changes.
|
|
|
|
The intended end state is:
|
|
|
|
- one configurable workspace root for Notarius-owned local state;
|
|
- diagnostics written under `workspace/diagnostics/<run-id>/` when workspace
|
|
diagnostics are enabled;
|
|
- resumability checkpoints written under deterministic `workspace/checkpoints/`
|
|
paths only when resume checkpointing is enabled;
|
|
- debug artifacts written under `workspace/debug/<run-id>/` only when debug is
|
|
enabled;
|
|
- stage-owned checkpoint manifests only, with no root-level checkpoint summary;
|
|
- explicit resume behavior through `run --resume`, not implicit skipping during
|
|
ordinary `run`.
|
|
|
|
## Stage 1: Add Workspace Configuration
|
|
|
|
Add workspace configuration while preserving legacy diagnostics configuration
|
|
long enough for backward compatibility.
|
|
|
|
Implement in `internal/core/config`:
|
|
|
|
- Add `WorkspaceConfig` to `Config`.
|
|
- Add nested structs:
|
|
- `WorkspaceDiagnosticsConfig`
|
|
- `WorkspaceResumeConfig`
|
|
- `WorkspaceDebugConfig`
|
|
- Add file config parsing for:
|
|
|
|
```yaml
|
|
workspace:
|
|
directory: /var/lib/notarius
|
|
diagnostics:
|
|
enabled: true
|
|
retention: auto
|
|
resume:
|
|
enabled: false
|
|
debug:
|
|
enabled: false
|
|
```
|
|
|
|
- Preserve existing `diagnostics.work_dir` and `diagnostics.retention` parsing as
|
|
deprecated compatibility input.
|
|
- Resolve effective diagnostics behavior as follows:
|
|
- if `workspace.directory` is set, diagnostics root is
|
|
`<workspace.directory>/diagnostics`;
|
|
- if `workspace.directory` is unset and legacy `diagnostics.work_dir` is set,
|
|
use legacy diagnostics behavior unchanged;
|
|
- if neither is set, preserve the current default diagnostics behavior for
|
|
migration compatibility;
|
|
- `workspace.diagnostics.retention` overrides legacy diagnostics retention
|
|
when set;
|
|
- legacy diagnostics retention remains accepted when workspace diagnostics
|
|
retention is unset.
|
|
- If `workspace.diagnostics.enabled` is explicitly false, do not create a
|
|
diagnostics run directory and do not write diagnostics artifacts. CLI failure
|
|
handling must still print concise errors to stderr without assuming a
|
|
diagnostics directory exists.
|
|
- Add environment support:
|
|
- `NOTARIUS_WORKSPACE_DIR` sets `workspace.directory`;
|
|
- `NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED` parses a boolean;
|
|
- `NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION` sets workspace diagnostics
|
|
retention;
|
|
- `NOTARIUS_WORKSPACE_RESUME_ENABLED` parses a boolean;
|
|
- `NOTARIUS_WORKSPACE_DEBUG_ENABLED` parses a boolean;
|
|
- keep `NOTARIUS_WORK_DIR` and `NOTARIUS_DIAGNOSTICS_RETENTION` as deprecated
|
|
compatibility overrides for legacy diagnostics config.
|
|
- Keep `--diagnostics-dir` as a backwards-compatible per-invocation diagnostics
|
|
root override. It should affect diagnostics only and should not change
|
|
checkpoint or debug roots.
|
|
- Update redaction/effective-config diagnostics so workspace config is included
|
|
and no secrets are introduced.
|
|
|
|
Tests:
|
|
|
|
- Add config default tests for workspace defaults.
|
|
- Add file config parsing tests for nested workspace fields.
|
|
- Add env override tests for all new env vars.
|
|
- Add precedence tests covering workspace config, legacy diagnostics config,
|
|
env overrides, and `--diagnostics-dir`.
|
|
- Update redacted/effective config tests.
|
|
|
|
## Stage 2: Introduce Workspace Filesystem Helpers
|
|
|
|
Create reusable filesystem helpers for workspace state. Keep concrete module
|
|
packages out of this layer.
|
|
|
|
Implement a new package, recommended path `internal/core/workspace`, with:
|
|
|
|
- `Config` or `Settings` describing effective workspace roots:
|
|
- root directory;
|
|
- diagnostics root;
|
|
- checkpoints root;
|
|
- debug root;
|
|
- enabled flags.
|
|
- path construction helpers for:
|
|
- diagnostics run directories;
|
|
- checkpoint identity directories;
|
|
- debug run directories.
|
|
- safe path helpers that reject absolute artifact names, `..`, backslashes, and
|
|
paths that escape their intended root.
|
|
- atomic JSON and byte-file writes using the existing diagnostics atomic-write
|
|
behavior as the model.
|
|
- optional shared internal helper for atomic file writes so diagnostics and
|
|
workspace writers do not duplicate low-level write logic.
|
|
|
|
Do not add resume behavior in this stage.
|
|
|
|
Tests:
|
|
|
|
- Unit-test path construction and path-safety failures.
|
|
- Unit-test atomic JSON/byte writes.
|
|
- Unit-test disabled workspace settings returning no-op or empty roots as
|
|
appropriate.
|
|
- Confirm no helper permits writes outside the configured root.
|
|
|
|
## Stage 3: Move Diagnostics Under Workspace
|
|
|
|
Update diagnostics creation to use the effective diagnostics root from workspace
|
|
configuration when workspace is configured.
|
|
|
|
Implementation requirements:
|
|
|
|
- Keep diagnostics artifacts and retention behavior unchanged.
|
|
- Honor disabled diagnostics by using a no-op diagnostics writer or nil-safe
|
|
diagnostics path through the CLI failure and success paths.
|
|
- Preserve the existing `diagnostics.RunDirectory` contract unless a narrow
|
|
constructor addition is cleaner.
|
|
- Route normal workspace diagnostics to:
|
|
|
|
```text
|
|
<workspace.directory>/diagnostics/<run-id>/
|
|
```
|
|
|
|
- Preserve legacy behavior when only legacy diagnostics config is present.
|
|
- Preserve `--diagnostics-dir` behavior as diagnostics-only override.
|
|
- Keep diagnostics run IDs run-based and non-deterministic.
|
|
- Do not write checkpoint or debug output in this stage.
|
|
|
|
Docs to update after behavior exists:
|
|
|
|
- `docs/config.md`
|
|
- `docs/operations.md`
|
|
- `docs/internal/diagnostics.md`
|
|
- `docs/cli.md` if CLI help text changes.
|
|
|
|
Tests:
|
|
|
|
- Update diagnostics run directory tests for workspace diagnostics roots.
|
|
- Update CLI tests that inspect diagnostics paths.
|
|
- Add migration tests proving legacy diagnostics config still works.
|
|
- Run focused checks:
|
|
- `go test ./internal/core/config`
|
|
- `go test ./internal/core/diagnostics`
|
|
- `go test ./internal/cli`
|
|
|
|
## Stage 4: Define Checkpoint Identity And Stage Manifest Types
|
|
|
|
Add checkpoint identity and manifest data structures before writing stage
|
|
payloads.
|
|
|
|
Implement in `internal/core/workspace` or a closely related core package:
|
|
|
|
- `CheckpointIdentity`, derived from:
|
|
- resolved pipeline ID;
|
|
- resolved pipeline digest;
|
|
- input adapter key;
|
|
- raw input digest or source digest;
|
|
- selected lanes;
|
|
- runtime overrides that affect execution;
|
|
- materialized reference digests;
|
|
- prompt/schema/profile provenance not already represented by the pipeline
|
|
digest.
|
|
- A deterministic, filesystem-safe checkpoint path:
|
|
|
|
```text
|
|
<workspace.directory>/checkpoints/<pipeline-id>/<input-key>-<source-digest-prefix>/<pipeline-digest-prefix>/
|
|
```
|
|
|
|
- Stage manifest structs for:
|
|
- source;
|
|
- chunk;
|
|
- extract lane;
|
|
- merge lane;
|
|
- normalize lane.
|
|
- Shared manifest fields:
|
|
- workspace schema version;
|
|
- stage name;
|
|
- lane ID when applicable;
|
|
- module key;
|
|
- dependency fingerprints;
|
|
- status;
|
|
- output digests;
|
|
- validation status and rejection summaries where applicable;
|
|
- started/completed timestamps where useful.
|
|
- Status values:
|
|
- `pending`;
|
|
- `running`;
|
|
- `succeeded`;
|
|
- `succeeded_with_rejections`;
|
|
- `failed`;
|
|
- `invalidated`.
|
|
|
|
Do not add a root-level checkpoint manifest.
|
|
|
|
Tests:
|
|
|
|
- Unit-test deterministic identity generation.
|
|
- Unit-test identity changes when pipeline digest, input digest, selected lanes,
|
|
or reference digests change.
|
|
- Unit-test manifest JSON round trips.
|
|
- Unit-test filesystem-safe path generation.
|
|
|
|
## Stage 5: Write Checkpoints Without Resuming
|
|
|
|
Add write-only checkpoint support behind `workspace.resume.enabled`.
|
|
|
|
Implementation requirements:
|
|
|
|
- Add a framework-owned checkpoint recorder to pipeline execution. Recommended
|
|
shape:
|
|
- CLI constructs the effective workspace/checkpoint recorder after pipeline
|
|
resolution and reference materialization.
|
|
- `pipeline.RunInput` receives a recorder interface or no-op recorder.
|
|
- concrete modules do not receive workspace paths and do not write directly to
|
|
workspace.
|
|
- Write checkpoint artifacts only when `workspace.resume.enabled` is true.
|
|
- Use stage-owned manifests and no root summary.
|
|
- Use JSON envelope types for checkpointed payloads that preserve byte content,
|
|
such as `content_base64`, media type, metadata, warnings, and content digest.
|
|
Do not rely on existing runtime structs whose byte fields are tagged
|
|
`json:"-"`.
|
|
- Recommended checkpoint files:
|
|
|
|
```text
|
|
checkpoints/<identity>/source/manifest.json
|
|
checkpoints/<identity>/source/source-document.json
|
|
checkpoints/<identity>/chunk/manifest.json
|
|
checkpoints/<identity>/chunk/chunks.json
|
|
checkpoints/<identity>/extract/<lane-id>/manifest.json
|
|
checkpoints/<identity>/extract/<lane-id>/outputs.json
|
|
checkpoints/<identity>/merge/<lane-id>/manifest.json
|
|
checkpoints/<identity>/merge/<lane-id>/output.json
|
|
checkpoints/<identity>/normalize/<lane-id>/manifest.json
|
|
checkpoints/<identity>/normalize/<lane-id>/output.json
|
|
```
|
|
|
|
- Record rejected extract outputs as checkpointed stage outcomes. Under current
|
|
pipeline policy they do not pass downstream, but they are not framework errors.
|
|
- Mark a stage `running` before writing its payloads, then atomically replace the
|
|
manifest with a final success/failure status after payload writes complete.
|
|
- Ensure interrupted or partial writes cannot be mistaken for successful
|
|
checkpoints.
|
|
- Do not write raw prompts, raw references, raw LLM request payloads, or debug
|
|
traces in checkpoint output.
|
|
|
|
Tests:
|
|
|
|
- Add runner/CLI tests proving no checkpoint files are written when resume is
|
|
disabled.
|
|
- Add tests proving checkpoint files are written when resume is enabled.
|
|
- Add tests for successful runs, rejected extract outputs, failed stages, and
|
|
warning-only validation.
|
|
- Add tests proving checkpoint payload content digests match written content.
|
|
- Add tests proving modules do not receive filesystem paths.
|
|
|
|
## Stage 6: Add Explicit Resume Reads
|
|
|
|
Add explicit resume behavior after write-only checkpoints are stable.
|
|
|
|
CLI behavior:
|
|
|
|
- Add `run --resume`.
|
|
- `--resume` should require `workspace.resume.enabled: true`; otherwise return a
|
|
clear configuration error.
|
|
- Plain `run` should continue to execute stages normally and should not silently
|
|
skip completed stages.
|
|
|
|
Resume behavior:
|
|
|
|
- Load and validate checkpoint stage manifests in workflow order.
|
|
- Reuse a checkpoint only when:
|
|
- workspace schema version is supported;
|
|
- stage status is successful for the current purpose;
|
|
- dependency fingerprints match the current invocation;
|
|
- referenced checkpoint payload files exist;
|
|
- payload digests match manifest digests;
|
|
- selected lanes and runtime overrides are compatible.
|
|
- If a checkpoint is missing or invalid, execute that stage normally and write a
|
|
fresh checkpoint if resume checkpointing remains enabled.
|
|
- If a source or chunk checkpoint is reused, downstream dependency fingerprints
|
|
must still be validated before downstream reuse.
|
|
- If an extract checkpoint includes rejected outputs, preserve those rejection
|
|
records and continue to omit rejected outputs from merge input.
|
|
- If merge is reused, skip merge execution only for the matching lane.
|
|
- If normalize is reused, pass the reused normalized output to the output stage.
|
|
- Always create fresh diagnostics for the current invocation, including records
|
|
showing which stages were reused versus executed.
|
|
|
|
Tests:
|
|
|
|
- Add CLI tests for `--resume` without workspace resume enabled.
|
|
- Add tests for reusing source, chunk, extract, merge, and normalize
|
|
checkpoints.
|
|
- Add tests for invalidation when input, pipeline digest, references, selected
|
|
lanes, or runtime LLM profile override changes.
|
|
- Add tests for corrupt or missing checkpoint payloads.
|
|
- Add tests proving fresh diagnostics are written for resumed invocations.
|
|
|
|
## Stage 7: Add Debug Output
|
|
|
|
Add debug output behind `workspace.debug.enabled`.
|
|
|
|
Implementation requirements:
|
|
|
|
- Write debug artifacts under:
|
|
|
|
```text
|
|
<workspace.directory>/debug/<run-id>/
|
|
```
|
|
|
|
- Debug is per-invocation and should not be used for resume.
|
|
- Debug and resume are independent:
|
|
- debug enabled does not imply checkpoint writing;
|
|
- resume enabled does not imply debug writing.
|
|
- Start with artifacts available from Notarius framework boundaries:
|
|
- source/checkpoint-like stage inputs and outputs;
|
|
- structured LLM request inputs and response content from Notarius contracts;
|
|
- validator requests and results;
|
|
- stage timing and attempt metadata.
|
|
- Do not depend on Scriptorium internals for rendered upstream provider
|
|
requests. If Scriptorium later exposes rendered prompt traces safely, add them
|
|
in a separate pass.
|
|
- Never write API keys, bearer tokens, or raw provider credentials.
|
|
- Document clearly that debug output may contain source material, references,
|
|
prompts, model outputs, and other sensitive content.
|
|
|
|
Tests:
|
|
|
|
- Add tests proving no debug files are written when debug is disabled.
|
|
- Add tests proving debug files are written under `debug/<run-id>/` when enabled.
|
|
- Add tests proving debug and resume can be enabled independently.
|
|
- Add tests proving obvious secrets are not written.
|
|
- Add diagnostics/debug path tests to ensure path roots do not overlap
|
|
accidentally.
|
|
|
|
## Stage 8: Documentation, Examples, And Cleanup
|
|
|
|
After behavior is implemented, update canonical current-behavior docs.
|
|
|
|
Update:
|
|
|
|
- `docs/config.md`
|
|
- `docs/cli.md`
|
|
- `docs/operations.md`
|
|
- `docs/troubleshooting.md`
|
|
- `docs/internal/diagnostics.md`
|
|
- `docs/internal/pipeline.md`
|
|
- maintained examples under `examples/` when practical.
|
|
|
|
Documentation requirements:
|
|
|
|
- Document workspace config and defaults.
|
|
- Document legacy diagnostics config compatibility and any deprecation language.
|
|
- Document `/var/lib/notarius` as the production recommendation.
|
|
- Document local development recommendations.
|
|
- Document checkpoint sensitivity and debug sensitivity.
|
|
- Document explicit resume behavior and invalidation rules.
|
|
- Keep future or deferred behavior only in roadmap files.
|
|
|
|
Cleanup:
|
|
|
|
- Update `docs/roadmap/workspace.md` to a completed-status note after the
|
|
implementation lands.
|
|
- Remove or replace this implementation plan with a completed note after the
|
|
implementation lands.
|
|
|
|
Validation:
|
|
|
|
```sh
|
|
go test ./internal/core/config
|
|
go test ./internal/core/diagnostics
|
|
go test ./internal/core/workspace
|
|
go test ./internal/framework/pipeline
|
|
go test ./internal/cli
|
|
go test ./...
|
|
go vet ./...
|
|
go build ./cmd/notarius
|
|
```
|