Rewrite internal documentation for current stage and state contracts

This commit is contained in:
2026-05-23 12:57:59 +00:00
parent d723384888
commit 0299b128cf
15 changed files with 450 additions and 993 deletions

View File

@@ -1,78 +1,57 @@
# Workspace internals
# Internal: Workspace
## Purpose
Define the local durable and run-local workspace model used by stages, manifests, resume, and publish.
Define local session layout, run-local stage layout, and cleanup guardrails.
## Inputs and Outputs
Inputs:
- `pipeline.workspace.root`
- `session.campaign`
- `session.session_id`
- generated `run_id`
## Canonical Session Layout
Session root:
- `{workspace.root}/work/{campaign}/{session_id}`
Outputs:
- Session manifest at `{workspace.root}/work/{campaign}/{session_id}/manifest.json`
- Run manifest at `{workspace.root}/work/{campaign}/{session_id}/runs/{run_id}/manifest.json`
- Canonical durable session directories and run-local stage trees
Core directories/files:
- `inputs/`
- `audio/`
- `transcripts/`
- `artifacts/`
- `reports/`
- `logs/`
- `config/`
- `current/`
- `runs/`
- `previous/`
- `manifest.json`
- `.lock`
## Boundaries
Owns:
- Session-level path layout (`inputs/`, `audio/`, `transcripts/`, `artifacts/`, `reports/`, `logs/`, `config/`, `current/`, `runs/`, `previous/`)
- `previous/manifest.json` and `previous/artifacts/**` are reserved for previous-session cache state materialized by `prepare` or `restore`
- Run-local stage sandbox layout under `runs/{run_id}/{stage}/`
- Session lock acquisition/release (`.lock`)
`previous/` reserved files:
- `previous/manifest.json`
- `previous/artifacts/**`
Does not own:
- Stage business logic
- Remote publish semantics (documented in `stage-publish.md`)
- CLI argument parsing
## Run-Local Stage Layout
When run context is available, stages use:
- `runs/{run_id}/{stage}/outputs/`
- `runs/{run_id}/{stage}/logs/`
- `runs/{run_id}/{stage}/reports/`
- `runs/{run_id}/{stage}/config/`
- `runs/{run_id}/{stage}/scratch/`
## Config Fields Used
- `pipeline.workspace.root`
- `pipeline.workspace.cleanup_after_publish`
- `pipeline.spool.root`
- `pipeline.spool.delete_audio_after_publish`
- `pipeline.cache.root`
- `pipeline.cache.s3_audio`
- `session.campaign`
- `session.session_id`
Run-local outputs are materialized back into canonical session paths before stage success.
`previous/**` writes are never redirected to run-local output paths.
## External Adapters Used
None directly in this subsystem. Stages may use object storage adapters and then write local outputs into this layout.
## Locking
`artifacts.LocalStore` enforces single-writer session lock via `.lock` file (`ErrLockConflict` on contention).
## State and Manifest Behavior
- Session state is persisted in the session manifest (`manifest.Manifest`).
- Invocation history is persisted per run in run manifests under `runs/{run_id}/manifest.json`.
- During each run, stage outputs are often written run-local first (`runs/{run_id}/{stage}/outputs/...`) and then materialized to canonical session paths after stage success.
- `manifest.Artifacts` entries record `ProducerRunID` for durable outputs.
- For S3 audio sessions, `prepare` records work/cache paths, S3 provenance, and spool path when the invocation downloaded the object.
- `previous/**` is reconstructed from configured previous-session requirements; restore uses the previous session's committed current publish state rather than treating current-session stored `previous/**` as authoritative.
- Durable cache state under `pipeline.cache.root` is not workspace state and is preserved by default by `narratio clean`.
- `narratio clean <id>` removes the session work root and session spool root.
- `narratio clean --all` removes all local session work under `workspace.root/work` and spool children under `spool.root`.
- `narratio clean --clear-cache` is the explicit opt-in for deleting matching S3 audio cache entries.
## Cleanup Semantics
Automatic post-publish cleanup (`runPostArchiveCleanup`):
- only runs when publish actually executed and succeeded;
- requires `uploaded=true` and `current_pointer_written=true` metadata;
- respects `pipeline.spool.delete_audio_after_publish` and `pipeline.workspace.cleanup_after_publish`;
- refuses unsafe deletes (root delete, out-of-root delete, symlink paths).
## Skip and Resume Behavior
- Skip/resume decisions are made in `internal/app` (`run_control.go`, `resume.go`) using stage status in the session manifest.
- `--force` reruns selected stages and marks downstream previously-succeeded stages as `stale`.
- Workspace layout is idempotent (`EnsureLayoutFor`) and reused across runs.
Manual clean command:
- `clean <session_id>` removes session work and spool subtree.
- `clean --all` removes all workspace work and spool children.
- durable cache is preserved unless `--clear-cache` is requested.
## Failure Behavior
- Failures preserve manifests and run-local files for inspection.
- Lock conflicts fail fast via `ErrLockConflict`.
- Cleanup can fail post-publish; failure is recorded in publish stage metadata and returned by the run.
## Tests to Inspect Before Changing
- `internal/artifacts/local_test.go`
- `internal/stage/run_local_test.go`
- `internal/app/run_control_test.go`
- `internal/app/resume_run_stage_test.go`
- `internal/app/post_archive_cleanup_test.go`
## Architectural Invariants
- Session root is campaign-aware: `{workspace.root}/work/{campaign}/{session_id}`.
- Run roots are always nested: `runs/{run_id}` under the session root.
- Run-local output materialization must end in canonical session paths.
- `previous/**` is session-durable state and must not be treated as run-local output scratch state.
- Automatic post-publish cleanup only targets run-scoped directories and must never delete configured root directories.
- Manual `clean` may delete session-scoped directories or the `workspace.root/work` directory, but it must preserve configured root directories and reject unsafe targets.
## Invariants
- campaign-aware session root is mandatory.
- manifest-driven stage state is durable across runs.
- cleanup guardrails prevent destructive root/out-of-scope deletion.