Implemented operations helper commands for validation, locking, and status

This commit is contained in:
2026-05-21 11:50:20 -05:00
parent a813bd5a50
commit 228c348e42
19 changed files with 1653 additions and 408 deletions

View File

@@ -17,9 +17,15 @@ Implemented commands:
- `run`: execute pipeline stages and persist manifest state.
- `plan`: validate config, prepare workspace layout, and print stage run/skip decisions.
- `resume`: continue from first non-succeeded stage unless forced.
- `status`: read and print stage statuses from an existing manifest.
- `status`: read an existing manifest or inspect local/remote state for a session.
- `run-stage`: execute exactly one stage.
- `restore`: restore durable local session state from the committed remote archive state.
- `session validate`: run read-only preflight checks for a session.
- `session init`: create local or remote `session.yml`.
- `artifacts list`: list effective artifact source IDs.
- `locks`: list effective archive promotion locks.
- `lock`: add or update a remote session lock.
- `unlock`: remove a remote session lock.
Unknown commands print usage and exit non-zero.
@@ -92,7 +98,55 @@ Valid stage names:
### `status`
- `--manifest <path>`: required manifest path.
- `--manifest <path>`: inspect one manifest file.
- `--config <path>`
- `--campaign <path>`
- `--session <path>`
- `--session-id <value>`
- `--previous-session-id <value>`
### `session validate`
- `--config <path>`
- `--campaign <path>`
- `--session <path>`
- `--session-id <value>`
- `--previous-session-id <value>`
### `session init`
- `--config <path>`: required.
- `--campaign <path>`: required.
- `--session-id <value>`: required.
- `--output <path>`: local `session.yml` target; mutually exclusive with `--remote`.
- `--remote`: write remote `session.yml` to the canonical session prefix; mutually exclusive with `--output`.
- `--previous-session-id <value>`
- `--date <value>`
- `--title <value>`
- `--audio-s3-prefix <prefix>`: defaults to `audio/` when neither audio flag is provided.
- `--audio-dir <path>`: local audio directory; mutually exclusive with `--audio-s3-prefix`.
- `--force`: overwrite existing local or remote target.
### `artifacts list`
- `--config <path>`
- `--campaign <path>`
- `--session <path>`
- `--session-id <value>`
- `--previous-session-id <value>`
- `--remote`: check promoted remote object availability.
### `locks`, `lock`, `unlock`
- `--config <path>`
- `--campaign <path>`
- `--session <path>`
- `--session-id <value>`
- `--previous-session-id <value>`
- `lock <source>` positional source ID.
- `lock --reason <text>` optional remote lock reason.
- `lock --force` updates an existing remote lock.
- `unlock <source>` positional source ID.
## Command Reference
@@ -161,22 +215,101 @@ Common failure cases:
### `status`
Purpose:
- Inspect one manifest file without executing stages.
- Inspect one manifest file, or inspect configured local/remote state for a session.
Syntax:
```bash
narratio status --manifest <manifest.json>
narratio status [--config <pipeline.yml>] [--campaign <campaign.yml>] [--session <session.yml>] [--session-id <id>] [--previous-session-id <id>]
```
Success output includes:
Manifest output includes:
- `session_id: <id>`
- `updated_at: <timestamp>`
- `stages:` entries (`- <stage>: <status>`)
Session output includes:
- session ID, campaign, workspace, session config source.
- local manifest state when present.
- remote current archive state when storage is configured.
- effective archive locks and conservative next actions.
Common failure cases:
- missing `--manifest`.
- missing `--manifest` when no config/session flags are provided.
- unreadable or invalid manifest path.
- invalid config or remote session fallback failure in session mode.
### `session validate`
Purpose:
- Run read-only preflight checks for a session.
Syntax:
```bash
narratio session validate [--config <pipeline.yml>] [--campaign <campaign.yml>] [--session <session.yml>] [--session-id <id>] [--previous-session-id <id>]
```
Checks include:
- effective config and session source.
- stable input files.
- local or remote audio availability.
- previous-session requirements.
- archive promotions and effective locks.
Warnings do not fail the command. Any `ERROR` finding exits non-zero.
### `session init`
Purpose:
- Create a strict-decoded session skeleton locally or in object storage.
Syntax:
```bash
narratio session init --config <pipeline.yml> --campaign <campaign.yml> --session-id <id> --output ./session.yml
narratio session init --config <pipeline.yml> --campaign <campaign.yml> --session-id <id> --remote
```
Behavior:
- exactly one of `--output` or `--remote` is required.
- remote writes target `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`.
- existing local or remote targets fail unless `--force` is passed.
- remote writes use existence checks, not compare-and-swap.
### `artifacts list`
Purpose:
- List built-in, configured, previous-session, promoted, and locked artifact sources.
Syntax:
```bash
narratio artifacts list [--config <pipeline.yml>] [--campaign <campaign.yml>] [--session <session.yml>] [--session-id <id>] [--previous-session-id <id>] [--remote]
```
`--remote` checks promoted top-level object availability through the storage adapter.
### `locks`, `lock`, and `unlock`
Purpose:
- Inspect and mutate source-based archive promotion locks.
Syntax:
```bash
narratio locks [--config <pipeline.yml>] [--campaign <campaign.yml>] [--session <session.yml>] [--session-id <id>]
narratio lock [flags] <source>
narratio unlock [flags] <source>
```
Behavior:
- static locks from `pipeline.archive.locks` and remote locks from `{session_prefix}/locks.yml` are merged.
- static locks win when sources duplicate remote locks.
- `lock` writes or updates only remote locks.
- `unlock` removes only remote locks and cannot remove static pipeline locks.
- `lock --force` is required to update an existing remote lock reason.
### `run-stage`

View File

@@ -237,7 +237,8 @@ Operational notes:
- archive promotion is explicit and source-based via `archive.promote_artifacts`.
- `source` is required; `dest` is optional and derived when omitted.
- `archive.locks` skips top-level promotion overwrites for locked sources while preserving run-local uploads.
- `archive.locks` skips top-level promotion overwrites for static locked sources while preserving run-local uploads.
- operator-created mutable locks are stored at `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/locks.yml` and are merged with static locks.
- Narratio does not auto-promote all generated analyze artifacts.
- `restore` reads the same config/campaign/session inputs and restore scope is bounded by committed archive current state.
@@ -384,10 +385,18 @@ Archive lock rules:
- locks are source-based and do not accept `dest`.
- duplicate lock sources are rejected.
- static `pipeline.archive.locks` win over remote mutable locks for the same source.
- locked promotions are recorded as intentional skips in archive metadata.
- locked required promotions do not fail archive by default.
- ordinary `--force` reruns do not override locks.
Remote mutable lock store:
- path: `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/locks.yml`.
- strict YAML shape: top-level `locks`, each with `source` and optional `reason`.
- `narratio lock` and `narratio unlock` mutate only the remote lock store.
- writes use existence checks plus `--force` for updates; they are not compare-and-swap atomic.
Restore-related implications:
- restore remote identity requires archive S3 identity to resolve (`pipeline.storage.s3.bucket` and session prefix derivation inputs).

View File

@@ -16,7 +16,7 @@ Outputs:
- resolved artifact path + provenance (`ResolvedSessionArtifact`);
- runtime catalog entries for built-ins and configured artifacts;
- requirement sets for canonical previous-session inputs.
- canonical S3 session, run, current, session config, audio, and promoted artifact keys.
- canonical S3 session, run, current, session config, session locks, audio, and promoted artifact keys.
## Boundaries
Owns:
@@ -47,6 +47,7 @@ Does not own:
## S3 key helpers
- session prefix: `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/`
- session config: `{session_prefix}/session.yml`
- session lock store: `{session_prefix}/locks.yml`
- run prefix: `{session_prefix}/runs/{run_id}/`
- audio prefix: `{session_prefix}/{session.inputs.audio_s3.prefix}`
- current manifest: `{session_prefix}/current/manifest.json`

View File

@@ -8,7 +8,7 @@ Inputs:
- session manifest and prerequisite stage records
- run root contents under `runs/{run_id}/`
- promotion rules with artifact `source` IDs and archive `dest` paths (`archive.promote_artifacts`)
- source-based promotion locks (`archive.locks`)
- effective source-based promotion locks from static config and remote session lock store
- session-level `previous/**` cache files when present
Outputs:
@@ -37,6 +37,7 @@ Does not own:
- `pipeline.archive.upload_run`
- `pipeline.archive.promote_artifacts`
- `pipeline.archive.locks`
- `{session_prefix}/locks.yml` loaded by app orchestration before archive execution
- `pipeline.storage.s3.bucket`
- `pipeline.storage.s3.root_prefix`
- `pipeline.workspace.root`
@@ -50,7 +51,8 @@ Does not own:
- Requires `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, and `analyze` status `succeeded`.
- Resolves bucket/prefix from manifest identity first, then config fallback.
- Uploads session `previous/**` files as durable session state when the local `previous/` directory exists.
- Skips top-level promotion uploads for sources listed in `pipeline.archive.locks`; run-local uploads still publish.
- Skips top-level promotion uploads for effective locked sources; run-local uploads still publish.
- Effective locks are the union of `pipeline.archive.locks` and remote `{session_prefix}/locks.yml`; static pipeline locks win on duplicate sources.
- Writes metadata including:
- upload counts/paths
- `previous_files_uploaded` and `previous_uploaded_paths`
@@ -77,5 +79,6 @@ Does not own:
- Run upload excludes `audio/` subtree.
- Session `previous/**` is archiveable durable input/provenance state, not run-local output.
- Ordinary `--force` does not override archive locks.
- Malformed or unreadable remote lock store fails archive-capable execution before promotion.
- `current/manifest.json` uploads before `current/run_id.txt`.
- `current/run_id.txt` is the remote publish commit marker.

View File

@@ -6,7 +6,7 @@ For field-level configuration, see [docs/config.md](./config.md). For full comma
## Normal workflow (S3-first path)
1. Upload `session.yml` to the configured session prefix, or pass a local `session.yml` explicitly.
1. Create or upload `session.yml`, or pass a local `session.yml` explicitly.
2. Upload session `.flac` files to object storage under the configured session audio prefix.
3. Run Narratio:
@@ -23,6 +23,20 @@ Notes:
- when local `session.yml` discovery misses, `--session-id` loads remote `session.yml` from `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`.
- S3 audio mode requires `session.inputs.audio_s3.prefix` and valid object-store access.
Initialize a remote session skeleton:
```bash
narratio session init --config /etc/narratio/pipeline.yml --campaign /etc/narratio/campaign.yml --session-id 2026-04-04 --remote
```
Remote init writes `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`. It fails if the object already exists unless `--force` is passed.
Validate before running:
```bash
narratio session validate --session-id 2026-04-04
```
## Restore workflow
Use restore when local durable session state is missing or stale and archive current state is authoritative.
@@ -119,6 +133,7 @@ When archive is enabled and run upload is enabled, archive publishes under:
Archive uploads:
- run record files from run root (excluding `audio/`).
- promoted files from explicit `archive.promote_artifacts` rules.
- mutable session locks from helper commands live at `{session_prefix}/locks.yml`.
Publish order:
1. upload `current/manifest.json`
@@ -133,8 +148,17 @@ Archive promotion is explicit and source-based:
- missing optional promotion sources are skipped.
- invalid resolved artifacts fail archive stage.
- `archive.locks` skips top-level promotion overwrites for locked sources while run-local uploads still publish.
- remote locks from `{session_prefix}/locks.yml` are merged with static `archive.locks`; static locks win on duplicate sources.
- locked required promotions are treated as intentional successful skips and are recorded in archive metadata.
Lock helper behavior:
- `narratio locks --session-id <id>` lists effective static and remote locks.
- `narratio lock <source> --session-id <id> --reason <text>` writes or updates a remote lock.
- `narratio unlock <source> --session-id <id>` removes only a remote lock.
- `lock --force` is required to update an existing remote lock reason.
- `unlock` cannot remove static pipeline locks.
- remote lock writes check whether the lock store exists, but are not compare-and-swap atomic.
## Resume, retry, restore, and safe rerun behavior
Default skip:
@@ -191,18 +215,24 @@ Recommended recovery:
1. inspect state:
```bash
narratio status --session-id 2026-04-04
```
2. for one manifest file, run:
```bash
narratio status --manifest <manifest-path>
```
2. for restore-specific checks, run:
3. for restore-specific checks, run:
```bash
narratio restore --session-id 2026-04-04 --dry-run
```
3. fix root cause (config/input/credentials/storage/service availability).
4. continue with `resume`, or targeted `run-stage --force` followed by `resume`.
4. fix root cause (config/input/credentials/storage/service availability).
5. continue with `resume`, or targeted `run-stage --force` followed by `resume`.
## Restore report
@@ -219,7 +249,8 @@ Dry-run does not write restore report files.
## Operational caveats
- `status` requires explicit `--manifest`; there is no session-id lookup command.
- `status` with no config/session flags still requires explicit `--manifest`.
- `status --session-id <id>` uses normal config/session loading, including remote session fallback.
- local and S3 audio input modes are mutually exclusive.
- archive publish requires upstream stages through `analyze` to be `succeeded`.
- required promotion rules can fail when selected analyze artifacts did not generate a required file path.

View File

@@ -0,0 +1,53 @@
# Roadmap: Operator Helper Commands
## Status
Implemented.
The operator helper command set is no longer conceptual. Current behavior is documented in:
- `docs/cli.md`
- `docs/operations.md`
- `docs/config.md`
- `docs/internal/artifacts.md`
- `docs/internal/stage-archive.md`
## Implemented Commands
- `narratio session validate`
- `narratio status --manifest <path>`
- `narratio status --session-id <id>`
- `narratio session init --output <path>`
- `narratio session init --remote`
- `narratio artifacts list`
- `narratio artifacts list --remote`
- `narratio locks`
- `narratio lock <source>`
- `narratio unlock <source>`
## Implemented Decisions
- Helper output is text-only. No JSON schema exists yet.
- `status` remains a top-level command.
- `session validate`, `session init`, and `artifacts list` are nested helper commands.
- `lock`, `unlock`, and `locks` are top-level commands.
- Remote session initialization requires explicit `--remote`.
- Local session initialization requires `--output`.
- Remote artifact availability is opt-in with `artifacts list --remote`.
- Mutable locks are source-based and stored at `{session_prefix}/locks.yml`.
- The remote lock store uses strict YAML with top-level `locks`.
- Static `pipeline.archive.locks` and remote locks are merged; static locks win on duplicate sources.
- `unlock` removes only remote locks.
- Ordinary execution `--force` does not override locks.
- Remote lock writes use existence checks and `--force` for updates; there is no compare-and-swap protection.
## Remaining Future Enhancements
These are intentionally not implemented:
- `--json` output for helper commands.
- Optimistic concurrency or ETag compare-and-swap for remote lock mutations.
- Rich remote artifact availability across historical run-local objects.
- Session-lock acquisition for remote mutation helpers.
- Broader campaign helper commands such as `campaign validate` or `campaign publish`.

View File

@@ -1,317 +0,0 @@
# Roadmap: Campaign State, Remote Sessions, and Archive Locks
## Purpose
This roadmap tracks implementation work for three related feature areas:
1. `campaign.yml` configuration for stable campaign-level inputs.
2. Remote `session.yml` loading from the existing S3 object-store backend.
3. Logical archive locks that prevent selected top-level transcript/artifact promotions from overwriting curated archive state while still preserving run-local outputs.
Treat future sections of this document as an implementation plan, not as current behavior. Keep planned behavior under `docs/roadmap/` until each item is implemented and canonical docs are updated.
## Current Code Facts
The current codebase already settles several design choices:
- CLI commands use short noun flags: `--config`, `--session`, `--session-id`, `--previous-session-id`, `--force`, and `--artifacts`.
- `run-stage` uses flags before the positional stage name, for example:
narratio run-stage --config ./pipeline.yml --campaign ./campaign.yml --session ./session.yml prepare
- `session.yml` is represented by `config.SessionConfig` and currently owns `session_id`, `previous_session_id`, `campaign`, `date`, `title`, and `inputs`.
- Strict YAML decoding is already implemented with `yaml.Decoder.KnownFields(true)`.
- Default local config discovery uses system paths under `/usr/local/etc/narratio/` and `/etc/narratio/`; working-directory files are used only when passed explicitly.
- The canonical S3 session prefix is already:
{root_prefix}/campaigns/{campaign}/sessions/{session_id}/
- Archive promotion is already source-based through `archive.promote_artifacts[].source`, with destination derivation and validation in `internal/config`.
- Storage adapters receive bucket-relative keys and do not infer campaign, session, run, or root-prefix semantics.
## Guardrails
Keep Narratio explicit and stage-driven. Do not introduce a generic workflow engine, broad config language, or stage behavior that reaches through adapter boundaries.
Implementation must preserve these constraints:
- Keep storage details behind `internal/adapters/storage`.
- Compute session, campaign, archive, and remote config keys in app/artifact/path helpers, not inside storage implementations.
- Use centralized path helpers in `internal/artifacts` or the established local path model.
- Preserve manifest-driven resume and stage status semantics.
- Keep strict YAML decoding for `pipeline.yml`, `campaign.yml`, and `session.yml`.
- Keep raw secrets out of configs, manifests, logs, generated configs, archive metadata, and roadmap examples.
- Update canonical user-facing docs only after behavior is implemented.
## Phase 1: Add `campaign.yml` (implemented)
Add campaign-level configuration for stable campaign identity and stable input files. Do not add remote campaign loading in this phase.
### CLI and Discovery
Add `--campaign <path>` to `run`, `plan`, `resume`, `run-stage`, and `restore`.
Examples:
narratio run --campaign ./campaign.yml --session ./session.yml
narratio plan --campaign ./campaign.yml --session ./session.yml
narratio resume --campaign ./campaign.yml --session ./session.yml
narratio run-stage --campaign ./campaign.yml --session ./session.yml prepare
narratio restore --campaign ./campaign.yml --session ./session.yml
Campaign config discovery order:
1. explicit `--campaign <path>`;
2. `/usr/local/etc/narratio/campaign.yml`;
3. `/etc/narratio/campaign.yml`.
Implement this in the same style as `resolvePipelineConfigPath` and `resolveSessionConfigPath`. Add default path constants and a search-path variable in `internal/config/defaults.go`.
### Config Shape
Initial `campaign.yml` fields:
campaign: icewind-dale
inputs:
speakers_file: ./speakers.yml
autocorrect_file: ./autocorrect.yml
glossary_file: ./glossary.yml
Do not add speculative campaign artifact defaults, prompt defaults, or title conventions in the first implementation.
### Merge Behavior
Add `CampaignConfig` and keep the final stage-facing config explicit.
Required behavior:
- `pipeline.yml` remains host/runtime configuration.
- `campaign.yml` supplies campaign identity and stable input file defaults.
- `session.yml` remains the source for `session_id`, `previous_session_id`, `date`, `title`, and audio input.
- Campaign-level `speakers_file`, `autocorrect_file`, and `glossary_file` fill missing session-level stable input fields.
- Session-level stable input fields override campaign-level stable input fields.
- If both `campaign.yml` and `session.yml` specify `campaign`, the values must match.
- The resolved session must satisfy the existing session validation rules before stages run.
- Unknown fields in `campaign.yml` fail strict decode.
Path resolution must preserve source-file locality:
- campaign-provided stable input paths resolve relative to `campaign.yml`;
- session-provided stable input overrides resolve relative to `session.yml`;
- absolute paths keep existing behavior.
Track enough provenance in the resolved config or prepare inputs so `prepare` can copy the correct source files without guessing which file supplied each path.
### Prepare Behavior
Update `prepare` to materialize the resolved campaign/session inputs into canonical session input paths:
inputs/campaign.yml
inputs/session.yml
inputs/pipeline.resolved.yml
inputs/speakers.yml
inputs/autocorrect.yml
inputs/glossary.yml
Continue recording deterministic `manifest.Inputs` records with checksums. If a prepared input came from `campaign.yml`, record source/provenance using the existing manifest input fields where practical; add narrow metadata only if the existing fields cannot describe it.
## Phase 2: Load Remote `session.yml` (implemented)
Support running with no local session file when a remote session file exists under the canonical session prefix.
### Preconditions
Build this phase after `campaign.yml`, because campaign identity is required to compute the remote session key. Do not infer campaign identity from object-store listing.
### Loading Precedence
Session loading order:
1. If `--session <path>` is supplied, load that local file.
2. If `--session` is omitted, use existing local discovery: `/usr/local/etc/narratio/session.yml`, `/etc/narratio/session.yml`.
3. If no local session file is found, `--session-id` is present, storage is configured, and campaign identity is resolved, load remote `session.yml`.
4. If no local or remote session can be loaded, fail with a message that lists the local search paths and the remote key that was attempted when applicable.
Do not make remote loading mask local discovery. Existing local discovery remains the local fallback before remote is attempted. Once remote loading is attempted, a missing remote object, storage init error, or malformed remote YAML fails clearly because no local session was available.
### Remote Key Layout
Use the existing canonical S3 layout:
session prefix: {root_prefix}/campaigns/{campaign}/sessions/{session_id}/
session file: {session_prefix}/session.yml
audio prefix: {session_prefix}/{session.inputs.audio_s3.prefix}
Add a centralized helper near `internal/artifacts/s3_keys.go`:
S3SessionConfigKey(sessionPrefix string) string
The helper should return `{session_prefix}/session.yml` using the same key normalization style as `S3CurrentManifestKey`, `S3CurrentRunPointerKey`, and `S3PromotedArtifactKey`.
### Decode, Template, and Provenance
Remote `session.yml` uses the same template variables and mismatch checks as local sessions:
- `{{session_id}}`
- `{{ session_id }}`
- `{{previous_session_id}}`
- `{{ previous_session_id }}`
Decode remote session YAML with strict known-field validation. Reuse the current session template/render/decode path by adding a byte/string-based loader rather than duplicating YAML decode logic.
When `prepare` materializes a remote session into `inputs/session.yml`, record that it came from S3. Preserve useful non-secret provenance when available:
- bucket;
- key;
- ETag;
- size;
- local checksum;
- downloaded temp/materialized path.
## Phase 3: Add Logical Archive Locks (implemented)
Narratio supports source-based archive locks under `pipeline.archive.locks`. The promotion system is source-based; destination-based locks are not supported.
### Config Shape
Lock entries:
archive:
locks:
- source: narratio.transcript.polished
reason: Human-reviewed transcript; do not overwrite automatically.
- source: narratio.artifact.session_recap
reason: Final recap was manually edited.
Validation rules:
- `source` is required.
- `source` must be a built-in source ID or configured `narratio.artifact.<key>` accepted by the same source validation used for `promote_artifacts`.
- `reason` is optional and non-secret.
- duplicate lock sources fail validation.
- lock entries do not support `dest` in the first implementation; unknown fields already fail strict decode.
### Archive Behavior
Archive continues uploading complete run-local outputs under `runs/{run_id}/`.
Promotion behavior:
1. Resolve promotion source and destination using existing source-based promotion logic.
2. If the promotion source is unlocked, upload the top-level promoted object normally.
3. If the promotion source is locked, skip only the top-level promotion overwrite.
4. Treat locked required promotions as intentional successful skips by default.
5. Continue archive commit when all run-local uploads and all non-locked required promotions succeed.
6. Upload `current/manifest.json` and `current/run_id.txt` in the existing order, with `current/run_id.txt` last.
Ordinary `--force` does not override locks. A lock-break override remains out of scope.
### Metadata
Archive records locked promotion skips in archive metadata/reporting so operators can distinguish missing optional promotions from lock-protected promotions.
Include:
- source ID;
- destination relative path and remote key;
- reason;
- local resolved path;
- resolved provenance;
- whether the original promotion rule was required.
Keep existing metadata such as `promoted_paths`, `skipped_optional_promotions`, `current_manifest_key`, `current_run_id_key`, and `current_pointer_written`.
## Phase 4: Future Operator Helpers
These commands are future work only. Do not implement them with the first campaign, remote-session, or lock changes.
Potential helper shapes:
narratio session validate --session-id 2026-06-07
narratio session init --session-id 2026-06-07 --title "The Black Cabin"
narratio status --session-id 2026-06-07
narratio locks --session-id 2026-06-07
narratio lock narratio.artifact.session_recap --session-id 2026-06-07
narratio unlock narratio.artifact.session_recap --session-id 2026-06-07
Potential behavior:
- validate remote session config;
- check audio object availability;
- inspect committed remote current state;
- list promoted transcripts/artifacts;
- list archive lock status;
- initialize a remote session skeleton;
- publish or sync campaign assets.
## Implementation Sequence
Use small, reviewable commits.
1. Campaign config types and discovery:
add `CampaignConfig`, strict loading, defaults/search paths, `--campaign` flags, and config/app tests.
2. Campaign/session merge:
implement resolved stable input merge, path provenance, validation, and prepare materialization.
3. Campaign docs after implementation:
update canonical docs and examples only for implemented behavior.
4. Remote session key and loader:
add `S3SessionConfigKey`, byte/string session loading, remote download through `ObjectStore`, and app-level precedence tests.
5. Remote session prepare provenance:
materialize downloaded session config and record S3 provenance.
6. Remote session docs after implementation:
update canonical docs and examples only after behavior exists.
7. Archive lock config:
add lock config structs, strict decode coverage, source validation, and duplicate detection.
8. Archive lock enforcement:
skip locked top-level promotions, preserve run-local uploads, record lock metadata, and protect commit ordering.
9. Final sweep:
run focused tests, then `go test ./...`; verify planned behavior remains only in roadmap docs until implemented.
## Test Plan
Add focused coverage in these packages:
- `internal/config`: campaign load, strict decode, discovery constants, merge validation, campaign/session mismatch, lock validation, duplicate lock rejection.
- `internal/app`: `--campaign` parsing on `run`, `plan`, `resume`, `run-stage`, and `restore`; campaign discovery; explicit `--session` precedence; local discovery before remote; remote session fallback when local discovery misses.
- `internal/artifacts`: `S3SessionConfigKey`; canonical session prefix compatibility; source ID validation for lock sources.
- `internal/stage/prepare`: campaign/session stable input materialization; campaign-relative and session-relative path resolution; remote session provenance in `manifest.Inputs`.
- `internal/stage/archive`: locked required promotion succeeds as skipped; unlocked promotion uploads; run-local outputs upload when top-level promotion is locked; `--force` does not break locks; `current/run_id.txt` remains the last upload.
- `internal/adapters/storage`: fake object key normalization and remote session download expectations.
Run at least:
go test ./internal/config -v
go test ./internal/app -run TestExecute -v
go test ./internal/artifacts -v
go test ./internal/stage -run 'Prepare|Archive' -v
go test ./internal/adapters/storage -v
go test ./...
Use fake storage for remote-session and archive-lock behavior. Ordinary tests must not require live S3.
## Documentation Updates After Implementation
After each phase is implemented, update only docs for behavior that exists.
Likely files:
- `docs/config.md`
- `docs/cli.md`
- `docs/operations.md`
- `docs/internal/stage-prepare.md`
- `docs/internal/stage-archive.md`
- `docs/internal/storage.md`
- `docs/internal/artifacts.md`
- relevant examples under `examples/`
Do not document remote campaign loading, helper commands, or lock override flags as current behavior until implemented.
## Remaining Open Decisions
The codebase resolves the campaign flag name, campaign discovery order, remote session layout, local-vs-remote session precedence, source-based archive lock model, and locked required promotion policy.
Remaining decisions:
1. Whether `campaign.yml` should eventually be loadable from S3. Do not implement remote campaign loading in the first phase.
2. Whether a future explicit lock override command or flag is needed. Do not make ordinary `--force` break locks.
3. Whether future helper commands should be top-level commands or subcommands. Keep them out of the first implementation.