diff --git a/docs/cli.md b/docs/cli.md index 44d5251..22564cd 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -6,7 +6,7 @@ narratio run --session-id 2026-04-04 ``` -This command uses default config discovery for `pipeline.yml` and `session.yml`; both files must be discoverable unless you pass explicit `--config` and `--session` paths. +This command uses default discovery for `pipeline.yml` and `session.yml`; both files must be discoverable unless you pass explicit `--config` and `--session` paths. ## Command Overview @@ -17,6 +17,7 @@ Implemented commands: - `resume`: continue from first non-succeeded stage unless forced. - `status`: read and print stage statuses from an existing manifest. - `run-stage`: execute exactly one stage. +- `restore`: restore durable local session state from the committed remote archive state. Unknown commands print usage and exit non-zero. @@ -68,6 +69,15 @@ Valid stage names: - `archive` - `notify` +### `restore` + +- `--config ` +- `--session ` +- `--session-id ` +- `--dry-run`: plan restore actions without writing local files. +- `--force`: overwrite local conflicting files with remote archive files. +- `--include-audio`: include durable archived `audio/**` files in restore scope. + ### `status` - `--manifest `: required manifest path. @@ -178,6 +188,39 @@ Common failure cases: - unknown stage name. - using `--artifacts` with any non-`analyze` stage. +### `restore` + +Purpose: +- Restore durable session state (`manifest.json`, `transcripts/**`, `artifacts/**`, and optional `audio/**`) from the committed remote archive current state. + +Syntax: + +```bash +narratio restore [--config ] [--session ] [--session-id ] [--dry-run] [--force] [--include-audio] +``` + +Success output (dry-run): +- `Restore plan for /` +- `Remote run: ` +- `Would download: ` +- `Would skip unchanged: ` +- `Conflicts: ` + +Success output (non-dry-run): +- `Restored session archive for /` +- `Remote run: ` +- `Downloaded: ` +- `Skipped unchanged: ` +- `Conflicts: ` + +Common failure cases: +- storage backend is not configured. +- remote `current/run_id.txt` missing/empty. +- remote `current/manifest.json` missing or invalid. +- remote manifest session/campaign mismatch. +- local conflicts without `--force`. +- session lock conflict. + ## Common Workflows Default-discovery run: @@ -204,6 +247,19 @@ Run only analyze stage with selected artifacts: narratio run-stage --session-id 2026-04-04 --artifacts player_handout analyze ``` +Preview restore actions without writes: + +```bash +narratio restore --session-id 2026-04-04 --dry-run +``` + +Restore and then force analyze: + +```bash +narratio restore --session-id 2026-04-04 +narratio run-stage --session-id 2026-04-04 --force analyze +``` + ## Diagnostic / Recovery Commands Inspect stage status: @@ -219,4 +275,4 @@ Get manifest path from previous output: - `--artifacts` filters which configured artifacts are executable when analyze runs. - `--artifacts` does not imply `--force`. -- If analyze is already `succeeded` and `--force` is not set, runner-level skip still applies. +- if analyze is already `succeeded` and `--force` is not set, runner-level skip still applies. diff --git a/docs/config.md b/docs/config.md index c3095b6..9a8eb7d 100644 --- a/docs/config.md +++ b/docs/config.md @@ -13,6 +13,7 @@ These commands load and validate both files before running: - `narratio plan` - `narratio resume` - `narratio run-stage` +- `narratio restore` Behavior: @@ -23,28 +24,28 @@ Behavior: ## 2. Config file discovery -Pipeline config lookup for `run`, `plan`, `resume`, and `run-stage`: +Pipeline config lookup for `run`, `plan`, `resume`, `run-stage`, and `restore`: -- If `--config ` is provided, that path is used. -- If omitted, Narratio searches in order: +- if `--config ` is provided, that path is used. +- if omitted, Narratio searches in order: 1. `/usr/local/etc/narratio/pipeline.yml` 2. `/etc/narratio/pipeline.yml` -- First existing file wins. +- first existing file wins. ## 3. Session file discovery and templating -Session config lookup for `run`, `plan`, `resume`, and `run-stage`: +Session config lookup for `run`, `plan`, `resume`, `run-stage`, and `restore`: -- If `--session ` is provided, that path is used. -- If omitted, Narratio searches in order: +- if `--session ` is provided, that path is used. +- if omitted, Narratio searches in order: 1. `./session.yml` 2. `/usr/local/etc/narratio/session.yml` 3. `/etc/narratio/session.yml` -- First existing file wins. +- first existing file wins. Template behavior: -- Supported placeholders: +- supported placeholders: - `{{session_id}}` - `{{ session_id }}` - `--session-id ` supplies the placeholder value. @@ -133,6 +134,7 @@ Operational notes: - archive promotion is explicit and source-based via `archive.promote_artifacts`. - `source` is required; `dest` is optional and derived when omitted. - Narratio does not auto-promote all generated analyze artifacts. +- `restore` reads the same config/session inputs and restore scope is bounded by committed archive current state. ## 7. Full pipeline reference @@ -267,6 +269,11 @@ Archive promotion destination rules: - configured sources derive from `pipeline.scriptorium.artifacts..output_path`; - derivation failure is a config validation error. +Restore-related implications: + +- restore remote identity requires archive S3 identity to resolve (`pipeline.storage.s3.bucket` and session prefix derivation inputs). +- restore scope considers only committed current state and durable paths (`manifest.json`, `transcripts/**`, `artifacts/**`, optional `audio/**`). + ## 8. Full session reference | Path | Type | Required | Default | diff --git a/docs/internal/README.md b/docs/internal/README.md index dcbe04f..965e77d 100644 --- a/docs/internal/README.md +++ b/docs/internal/README.md @@ -4,7 +4,7 @@ Developers and LLM coding agents changing Narratio internals. ## Scope -Implementation-accurate contracts for workspace/state, manifests, stages, artifact resolution, and adapter boundaries. +Implementation-accurate contracts for workspace/state, manifests, stages, artifact resolution, adapter boundaries, and restore command behavior. ## Component Docs - `adapters.md`: external adapter map, runtime wiring, and boundary ownership. @@ -12,6 +12,7 @@ Implementation-accurate contracts for workspace/state, manifests, stages, artifa - `manifest.md`: session/run manifest schemas, lifecycle transitions, and persistence semantics. - `artifacts.md`: built-in artifact registry, runtime artifact catalog, and source-resolution behavior. - `workspace.md`: local state model, manifests, run-local layout, promotion, and cleanup invariants. +- `command-restore.md`: restore command discovery/planning/execution/reporting contract. - `stage-prepare.md`: input materialization and provenance capture. - `stage-transcribe.md`: WhisperX transcript generation. - `stage-merge.md`: Seriatim normalization + merge. diff --git a/docs/internal/command-restore.md b/docs/internal/command-restore.md new file mode 100644 index 0000000..44be344 --- /dev/null +++ b/docs/internal/command-restore.md @@ -0,0 +1,86 @@ +# Internal: Command Restore + +## Purpose +Define the implemented `narratio restore` command contract: committed remote-state discovery, deterministic planning, safe file installation, conflict policy, and restore reporting. + +## Inputs and outputs +Inputs: +- CLI flags: `--config`, `--session`, `--session-id`, `--dry-run`, `--force`, `--include-audio`. +- Resolved/validated `pipeline.yml` and `session.yml`. +- Configured remote object store. +- Remote committed current-state markers (`current/run_id.txt`, `current/manifest.json`). + +Outputs: +- Dry-run summary to stdout (plan + counts). +- Non-dry-run completion summary to stdout. +- Local durable session files restored under canonical session root. +- Non-dry-run restore report at `reports/restore-latest.json`. + +## Boundaries +Owns: +- Restore command flag parsing and command wiring. +- Remote current-state discovery and identity validation. +- Restore plan construction and conflict classification. +- Restore execution for planned downloads. +- Restore report model and persistence. + +Does not own: +- Stage execution orchestration (`run`, `resume`, `run-stage`). +- Archive publish behavior (owned by archive stage). +- Storage transport implementation details (owned by storage adapters). + +## Config fields used +- Config/session discovery and templating fields consumed by all commands. +- `pipeline.workspace.root` (local restore target root). +- `pipeline.storage.*` (remote backend + archive identity derivation). +- `pipeline.storage.s3.*` identity components used by archive prefix helpers. +- `session.session_id` +- `session.campaign` + +## External adapters used +- `storage.ObjectStore` for `Exists`, `List`, `Download`. +- `artifacts.Store` (`LocalStore`) for layout and session lock management. +- `manifest.LocalStore` for manifest decode/validation and identity checks. + +## State and manifest behavior +- Restore is not a pipeline run and does not create a run manifest. +- Restore uses committed remote current state only: + - `current/run_id.txt` must exist and be non-empty. + - `current/manifest.json` must decode and match requested session/campaign. +- Non-dry-run writes restore files to canonical session paths. +- Manifest install behavior: + - validated before replacement. + - installed last among download actions. + - existing local manifest is preserved if restored manifest validation/install fails. +- Non-dry-run report persists summary/action status metadata in `reports/restore-latest.json`. + +## Skip and resume behavior +- Restore does not participate in stage skip/resume decisions. +- Restore provides durable local state so subsequent stage commands can resume or rerun based on restored manifest state. +- Dry-run is read-only and returns plan output only. + +## Failure behavior +- Fails when storage backend is unavailable or archive identity cannot be resolved. +- Fails when remote current pointer/manifest is missing or invalid. +- Fails when remote manifest identity mismatches requested campaign/session. +- Fails on local conflicts unless `--force` is set. +- Fails fast on session lock acquisition conflict for non-dry-run execution. +- On execution failure, previously installed files remain; no rollback is performed. + +## Tests to inspect before changing +- `internal/app/restore_test.go` +- `internal/app/restore_discovery_test.go` +- `internal/app/restore_plan_test.go` +- `internal/app/restore_execution_test.go` +- `internal/app/restore_workflow_test.go` +- `internal/artifacts/archive_identity_test.go` + +## Architectural invariants +- Restore relies on centralized archive identity/key helpers (`internal/artifacts`) rather than ad hoc key building. +- `current/run_id.txt` is the remote commit marker; restore must not infer committed state from incidental files. +- Local path mapping is traversal-safe and constrained to session root. +- Restore scope is deterministic and path-classified: + - include `manifest.json`, `transcripts/**`, `artifacts/**` + - include `audio/**` only with `--include-audio` + - exclude `runs/**`, `logs/**`, `reports/**`, `config/**`, `inputs/**` +- Command remains standalone; no implicit `run --restore` behavior. diff --git a/docs/operations.md b/docs/operations.md index 84ce816..b6dda00 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -6,7 +6,7 @@ For field-level configuration, see [docs/config.md](./config.md). For full comma ## Normal workflow (S3-first path) -1. Upload session `.flac` files to object storage under the session audio prefix. +1. Upload session `.flac` files to object storage under the configured session audio prefix. 2. Run Narratio: ```bash @@ -21,6 +21,37 @@ Notes: - default config/session discovery applies unless `--config` and `--session` are passed. - S3 audio mode requires `session.inputs.audio_s3.prefix` and valid object-store access. +## Restore workflow + +Use restore when local durable session state is missing or stale and archive current state is authoritative. + +Dry-run (no local writes): + +```bash +narratio restore --session-id 2026-04-04 --dry-run +``` + +Execution: + +```bash +narratio restore --session-id 2026-04-04 +``` + +Post-restore analyze rerun pattern: + +```bash +narratio run-stage --session-id 2026-04-04 --force analyze +``` + +Restore source-of-truth: +- remote commit marker: `current/run_id.txt` +- remote current manifest: `current/manifest.json` + +Restore default scope: +- includes `manifest.json`, `transcripts/**`, `artifacts/**` +- includes `audio/**` only with `--include-audio` +- excludes `runs/**`, `logs/**`, `reports/**`, `config/**`, `inputs/**`, and `current/**` (except remote `current/manifest.json` as source) + ## Local filesystem layout and state artifacts Session root: @@ -29,7 +60,7 @@ Session root: Primary state: - `manifest.json`: session-level stage state. - `runs/{run_id}/manifest.json`: invocation-level state. -- `.lock`: session lock while a run is active. +- `.lock`: session lock while a modifying command is active. Canonical session directories: - `inputs/` @@ -48,6 +79,7 @@ Run-local stage directories: Behavior: - directory creation is idempotent. - stage outputs are generally generated run-local first, then promoted to canonical paths on success. +- restore installs downloaded files to canonical session paths and does not recreate historical run sandboxes. ## Analyze artifact execution lifecycle @@ -91,7 +123,7 @@ Archive promotion is explicit and source-based: - missing optional promotion sources are skipped. - invalid resolved artifacts fail archive stage. -## Resume, retry, and safe rerun behavior +## Resume, retry, restore, and safe rerun behavior Default skip: - `run` and `run-stage` skip already-succeeded stages unless `--force` is set. @@ -100,6 +132,11 @@ Resume: - `resume` starts at first non-succeeded stage. - `resume --force` runs full stage order. +Restore conflict policy: +- restore classifies local differences as conflicts. +- without `--force`, restore fails when conflicts exist. +- with `--force`, conflicting local files are overwritten by remote archive files. + Forced reruns: - force-rerunning an upstream succeeded stage marks downstream succeeded stages as `stale`. @@ -125,13 +162,18 @@ No cleanup for failed/incomplete/unarchived/archive-skipped runs. ## Failure and recovery playbooks -After failure, Narratio keeps: +After run failure, Narratio keeps: - session manifest - run manifest - run-local artifacts/logs/config/reports Failed or incomplete runs remain local-only. +After restore failure: +- already-installed restore files remain in place. +- restore does not roll back prior successful installs. +- existing local manifest is preserved if restored manifest validation/install fails. + Recommended recovery: 1. inspect state: @@ -140,8 +182,27 @@ Recommended recovery: narratio status --manifest ``` -2. fix root cause (config/input/credentials/service availability). -3. continue with `resume`, or targeted `run-stage --force` followed by `resume`. +2. for restore-specific checks, run: + +```bash +narratio restore --session-id 2026-04-04 --dry-run +``` + +3. fix root cause (config/input/credentials/storage/service availability). +4. continue with `resume`, or targeted `run-stage --force` followed by `resume`. + +## Restore report + +Non-dry-run restore writes a durable report at: +- `reports/restore-latest.json` + +Report content includes: +- identity (`campaign`, `session_id`, `run_id`) +- mode flags (`dry_run`, `force`, `include_audio`) +- plan counts and execution counts +- per-action status + +Dry-run does not write restore report files. ## Operational caveats @@ -149,3 +210,4 @@ narratio status --manifest - local and S3 audio input modes are mutually exclusive. - archive publish requires upstream stages through `analyze` to be `succeeded`. - required promotion rules can fail when selected analyze artifacts did not generate a required file path. +- restore requires configured remote object storage and committed remote current state. diff --git a/docs/roadmap/restore.md b/docs/roadmap/restore.md index 0807d15..606d88a 100644 --- a/docs/roadmap/restore.md +++ b/docs/roadmap/restore.md @@ -2,7 +2,7 @@ ## Status -Planned. This document is implementation guidance for 5.3-Codex. +Implemented through Step 8. This document remains as roadmap and design history for the restore feature, and as the home for future restore-related ideas (for example `run --restore`). ## Summary @@ -546,7 +546,7 @@ Do not document future `narratio run --restore` behavior outside `docs/roadmap/` ## Implementation phases -### Phase 1: Audit existing archive and path helpers +### Phase 1: Audit existing archive and path helpers (completed) Before coding behavior, inspect: @@ -574,7 +574,7 @@ Deliverable: - small code comments or internal helper selection; - no large behavior change yet unless required by tests. -### Phase 2: Add CLI surface and command wiring +### Phase 2: Add CLI surface and command wiring (completed) Add `narratio restore` command parsing. @@ -599,7 +599,7 @@ Deliverable: - command returns a clear “not yet implemented” or calls an empty planner if phased commits are desired; - CLI tests pass. -### Phase 3: Implement remote current-state discovery +### Phase 3: Implement remote current-state discovery (completed) Add restore code that: @@ -615,7 +615,7 @@ Deliverable: - fake-storage tests for current-state discovery; - no local file writes beyond temporary files. -### Phase 4: Implement restore planning +### Phase 4: Implement restore planning (completed) Build deterministic restore plans for default scope and `--include-audio`. @@ -627,7 +627,7 @@ Deliverable: - dry-run output works; - no real file overwrite yet except temp comparisons as needed. -### Phase 5: Implement restore execution +### Phase 5: Implement restore execution (completed) Execute the plan safely: @@ -644,7 +644,7 @@ Deliverable: - restore works end-to-end against fake storage; - failures are clear and do not corrupt existing local manifest. -### Phase 6: Add restore report and operator summary +### Phase 6: Add restore report and operator summary (completed) Add concise stdout summary and optional JSON restore report if consistent with project diagnostics. @@ -654,7 +654,7 @@ Deliverable: - durable diagnostic report if implemented; - no content leakage. -### Phase 7: Workflow integration test +### Phase 7: Workflow integration test (completed) Add a test for restoring a previous session and then forcing `analyze`. @@ -664,7 +664,7 @@ Deliverable: - no upstream stages rerun; - no reliance on live subprocesses or S3. -### Phase 8: Documentation update +### Phase 8: Documentation update (completed) Once implemented, update current-behavior docs and internal command docs. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 7e69715..a69f578 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -6,7 +6,7 @@ Canonical operator troubleshooting guide for recurring implemented Narratio fail ## Config file discovery failure Symptom: -- `run`, `plan`, `resume`, or `run-stage` fails with config/session not found. +- `run`, `plan`, `resume`, `run-stage`, or `restore` fails with config/session not found. Likely Cause: - `pipeline.yml` or `session.yml` is missing from discovery paths. @@ -192,7 +192,7 @@ Links: ## Session lock conflict (`.lock`) Symptom: -- run fails with lock conflict for session workdir. +- `run`, `resume`, `run-stage`, or `restore` fails with lock conflict for session workdir. Likely Cause: - another Narratio process is running same session. @@ -207,13 +207,104 @@ ps aux | grep narratio ``` Safe Fix: -- wait for active run to finish. +- wait for active process to finish. - if no process is active, remove only stale session `.lock` file. Links: - [docs/operations.md](./operations.md) - [docs/internal/workspace.md](./internal/workspace.md) +## Restore remote current pointer or manifest missing + +Symptom: +- `restore` fails with remote current pointer or current manifest errors. + +Likely Cause: +- `current/run_id.txt` was never published. +- `current/manifest.json` is missing for the session prefix. +- archive commit did not complete. + +Diagnostics: + +```bash +narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run +``` + +Safe Fix: +- verify archive stage succeeded for the target session. +- rerun/archive from a healthy source workspace so current pointers are published. + +Links: +- [docs/operations.md](./operations.md) +- [docs/internal/stage-archive.md](./internal/stage-archive.md) + +## Restore manifest identity mismatch + +Symptom: +- `restore` fails because remote manifest session or campaign does not match requested values. + +Likely Cause: +- wrong `--session-id` or wrong session config selected. +- archive prefix points to a different campaign/session. + +Diagnostics: + +```bash +narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run +``` + +Safe Fix: +- use the correct session config and `--session-id`. +- verify campaign/session identity in local config before restore. + +Links: +- [docs/config.md](./config.md) +- [docs/operations.md](./operations.md) + +## Restore conflict without `--force` + +Symptom: +- `restore` fails with `restore conflict` and conflict counts. + +Likely Cause: +- local durable file differs from remote file for one or more planned restore paths. + +Diagnostics: + +```bash +narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run +``` + +Safe Fix: +- review planned conflicts. +- rerun with `--force` only when remote state should overwrite local state. + +Links: +- [docs/cli.md](./cli.md) +- [docs/operations.md](./operations.md) + +## Restore report expectations + +Symptom: +- operator expects restore report file but does not find one. + +Likely Cause: +- restore was executed in `--dry-run` mode. +- restore failed before report persistence path (for example lock acquisition failure). + +Diagnostics: + +```bash +ls -l {workspace.root}/work/{campaign}/{session_id}/reports/restore-latest.json +``` + +Safe Fix: +- run non-dry-run restore for durable report output. +- resolve lock or early preflight failures and retry. + +Links: +- [docs/operations.md](./operations.md) + ## Secrets env-dir or credential-env failure Symptom: