Compare commits
49 Commits
v0.10.0
...
f050b9dd54
| Author | SHA1 | Date | |
|---|---|---|---|
| f050b9dd54 | |||
| 9c9cb54339 | |||
| 7657ec3ad6 | |||
| cee52aa092 | |||
| e920f3a8d5 | |||
| 591c529a09 | |||
| 7324c5a686 | |||
| d0936fb022 | |||
| 2aa074c5cf | |||
| 782d0cf3b9 | |||
| 083c01cfa0 | |||
| 2937696024 | |||
| b817a5b772 | |||
| 3022f20beb | |||
| ca1ded1821 | |||
| 3752f3ed28 | |||
| 870c2d69d5 | |||
| 135407ba7c | |||
| 228c348e42 | |||
| a813bd5a50 | |||
| d8f58dce31 | |||
| 7111edeca4 | |||
| 3aae4bbb12 | |||
| b29d8eeb50 | |||
| dffb432537 | |||
| 2dd38c7913 | |||
| bc2ade38d9 | |||
| 5be831eb13 | |||
| cae4d99a89 | |||
| e09dc0512d | |||
| ae82bc1ce0 | |||
| 01eb7aa1aa | |||
| 2ca700195c | |||
| 2b08c34539 | |||
| 79f1fc1e09 | |||
| 9c753270bd | |||
| b907cb01aa | |||
| 7824afd4a5 | |||
| 2a4e1e912c | |||
| dd03c09d75 | |||
| 5bc8e8683f | |||
| 648001a8fe | |||
| 6684774f52 | |||
| f3b63bd5e5 | |||
| 23d6470b0f | |||
| 128449040f | |||
| 02ab106ade | |||
| c128970f58 | |||
| d001baa660 |
@@ -5,7 +5,7 @@ Narratio is a Go orchestration application that turns D&D session audio into pol
|
||||
It coordinates transcription, merge/polish/normalize/trim processing, artifact generation, archive publishing, and resumable run state in one operator workflow.
|
||||
|
||||
```bash
|
||||
narratio run --session-id 2026-04-04
|
||||
narratio run 2026-04-04
|
||||
```
|
||||
|
||||
This command requires discoverable `pipeline.yml` and `session.yml` files (or explicit `--config` and `--session` flags).
|
||||
|
||||
415
docs/cli.md
415
docs/cli.md
@@ -3,58 +3,95 @@
|
||||
## Shortest Useful Command
|
||||
|
||||
```bash
|
||||
narratio run --session-id 2026-04-04
|
||||
narratio run 2026-04-04
|
||||
```
|
||||
|
||||
This command uses default config discovery for `pipeline.yml` and `session.yml`; both files must be discoverable unless you pass explicit `--config` and `--session` paths.
|
||||
This command uses default system discovery for `pipeline.yml`, the pipeline default campaign ID, and local `session.yml`. If local session discovery misses and S3 storage is configured, the positional session ID loads remote `session.yml` from the canonical session prefix.
|
||||
|
||||
Default pipeline and session discovery checks system config locations only. Pass `--config`, `--campaign-file`, and `--session` to use files from the current working directory. Pass `--campaign <id>` to select a campaign from `pipeline.campaigns.root`.
|
||||
|
||||
Ordinary local and remote `session.yml` files must be concrete YAML. Templates belong to `narratio session init`, which renders a configured campaign template before writing the concrete file.
|
||||
|
||||
## Command Overview
|
||||
|
||||
Implemented commands:
|
||||
Top-level commands:
|
||||
|
||||
- `run`: execute pipeline stages and persist manifest state.
|
||||
- `plan`: validate config, prepare workspace layout, and print stage run/skip decisions.
|
||||
- `resume`: continue from first non-succeeded stage unless forced.
|
||||
- `status`: read and print stage statuses from an existing manifest.
|
||||
- `run-stage`: execute exactly one stage.
|
||||
- `run <session_id>`: execute pipeline stages and persist manifest state.
|
||||
- `run-stage <stage> <session_id>`: execute exactly one stage.
|
||||
- `resume <session_id>`: continue from first non-succeeded stage unless forced.
|
||||
- `analyze <session_id>`: force-rerun the analyze stage.
|
||||
- `publish <session_id>`: force-rerun the archive stage.
|
||||
- `clean <session_id>|--all`: remove local workspace/spool state.
|
||||
- `session <subcommand>`: session-scoped helper commands.
|
||||
|
||||
Session subcommands:
|
||||
|
||||
- `session init <session_id>`: create local or remote `session.yml`.
|
||||
- `session validate <session_id>`: run read-only preflight checks.
|
||||
- `session status <session_id>`: inspect local/remote session state.
|
||||
- `session plan <session_id>`: validate config, prepare workspace layout, and print stage run/skip decisions.
|
||||
- `session restore <session_id>`: restore durable local state from committed remote archive state.
|
||||
- `session artifacts <session_id>`: list effective artifact source IDs.
|
||||
- `session locks <session_id>`: list archive promotion locks.
|
||||
- `session locks add <session_id> <source>`: add or update a remote lock.
|
||||
- `session locks remove <session_id> <source>`: remove a remote lock.
|
||||
|
||||
Unknown commands print usage and exit non-zero.
|
||||
|
||||
For config semantics, see [docs/config.md](./config.md). For operator lifecycle and recovery, see [docs/operations.md](./operations.md).
|
||||
|
||||
## Complete Flag Reference
|
||||
## Common Flags
|
||||
|
||||
Most session-aware commands accept:
|
||||
|
||||
- `--config <path>`: optional explicit `pipeline.yml` path.
|
||||
- `--campaign <id>`: optional campaign ID selector.
|
||||
- `--campaign-file <path>`: optional explicit `campaign.yml` path.
|
||||
- `--session <path>`: optional explicit concrete `session.yml` path.
|
||||
- `--previous-session-id <value>`: expected previous session identifier.
|
||||
|
||||
The positional `<session_id>` is required even when `--session` is provided. It is used as the expected session identity and as the remote session lookup value when local session discovery misses.
|
||||
|
||||
## Command Reference
|
||||
|
||||
### `run`
|
||||
|
||||
- `--config <path>`: optional explicit `pipeline.yml` path.
|
||||
- `--session <path>`: optional explicit `session.yml` path.
|
||||
- `--session-id <value>`: session template variable value.
|
||||
- `--force`: force stage execution.
|
||||
- `--artifacts <names>`: analyze artifact keys to execute (repeatable or comma-separated).
|
||||
```bash
|
||||
narratio run <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--force] [--artifacts <name[,name...]>]
|
||||
```
|
||||
|
||||
### `plan`
|
||||
Purpose:
|
||||
- Execute configured stages in canonical order.
|
||||
|
||||
- `--config <path>`
|
||||
- `--session <path>`
|
||||
- `--session-id <value>`
|
||||
- `--force`
|
||||
Success output:
|
||||
- `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
|
||||
|
||||
Common failure cases:
|
||||
- missing system default config/session paths when flags are omitted.
|
||||
- missing selected campaign under `pipeline.campaigns.root`.
|
||||
- missing local session plus missing/unavailable remote `session.yml`.
|
||||
- templated `session.yml`; run `narratio session init` to generate concrete YAML.
|
||||
- concrete session identity mismatch.
|
||||
- unknown configured artifact key in `--artifacts`.
|
||||
|
||||
### `resume`
|
||||
|
||||
- `--config <path>`
|
||||
- `--session <path>`
|
||||
- `--session-id <value>`
|
||||
- `--force`
|
||||
- `--artifacts <names>`: analyze artifact keys to execute (repeatable or comma-separated).
|
||||
```bash
|
||||
narratio resume <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--force] [--artifacts <name[,name...]>]
|
||||
```
|
||||
|
||||
Purpose:
|
||||
- Continue from session-manifest stage status.
|
||||
|
||||
Success output:
|
||||
- `narratio resume: session <session_id> has no remaining stages`
|
||||
- or `narratio resume: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
|
||||
|
||||
### `run-stage`
|
||||
|
||||
- `--config <path>`
|
||||
- `--session <path>`
|
||||
- `--session-id <value>`
|
||||
- `--force`
|
||||
- `--artifacts <names>`: analyze artifact keys to execute (repeatable or comma-separated).
|
||||
- positional `<stage>`: required stage name.
|
||||
```bash
|
||||
narratio run-stage <stage> <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--force] [--artifacts <name[,name...]>]
|
||||
```
|
||||
|
||||
Valid stage names:
|
||||
|
||||
@@ -68,155 +105,237 @@ Valid stage names:
|
||||
- `archive`
|
||||
- `notify`
|
||||
|
||||
### `status`
|
||||
|
||||
- `--manifest <path>`: required manifest path.
|
||||
|
||||
## Command Reference
|
||||
|
||||
### `run`
|
||||
|
||||
Purpose:
|
||||
- Execute configured stages in canonical order.
|
||||
|
||||
Syntax:
|
||||
|
||||
```bash
|
||||
narratio run [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] [--artifacts <name[,name...]>]
|
||||
```
|
||||
|
||||
Success output:
|
||||
- `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
|
||||
|
||||
Common failure cases:
|
||||
- missing default config/session paths when flags omitted.
|
||||
- invalid template/rendered session mismatch.
|
||||
- unknown/invalid `--artifacts` value.
|
||||
- `--artifacts` with unknown configured artifact key.
|
||||
|
||||
### `plan`
|
||||
|
||||
Purpose:
|
||||
- Validate config, load secrets (if configured), prepare workdir, and print stage run/skip decisions.
|
||||
|
||||
Syntax:
|
||||
|
||||
```bash
|
||||
narratio plan [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force]
|
||||
```
|
||||
|
||||
Success output includes:
|
||||
- `narratio plan: workdir prepared at <path>`
|
||||
- one line per stage (`<stage>: run|skip`)
|
||||
- `totals: run=<n> skip=<n>`
|
||||
|
||||
Common failure cases:
|
||||
- same config/session discovery and validation failures as `run`.
|
||||
- secrets directory read failures when `pipeline.secrets.env_dir` is configured.
|
||||
|
||||
### `resume`
|
||||
|
||||
Purpose:
|
||||
- Continue from session-manifest stage status.
|
||||
|
||||
Syntax:
|
||||
|
||||
```bash
|
||||
narratio resume [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] [--artifacts <name[,name...]>]
|
||||
```
|
||||
|
||||
Success output:
|
||||
- `narratio resume: session <session_id> has no remaining stages`
|
||||
- or `narratio resume: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
|
||||
|
||||
Common failure cases:
|
||||
- same discovery/template/validation failures as `run`.
|
||||
- manifest load errors when existing manifest is unreadable.
|
||||
- invalid or unknown artifact selections.
|
||||
|
||||
### `status`
|
||||
|
||||
Purpose:
|
||||
- Inspect one manifest file without executing stages.
|
||||
|
||||
Syntax:
|
||||
|
||||
```bash
|
||||
narratio status --manifest <manifest.json>
|
||||
```
|
||||
|
||||
Success output includes:
|
||||
- `session_id: <id>`
|
||||
- `updated_at: <timestamp>`
|
||||
- `stages:` entries (`- <stage>: <status>`)
|
||||
|
||||
Common failure cases:
|
||||
- missing `--manifest`.
|
||||
- unreadable or invalid manifest path.
|
||||
|
||||
### `run-stage`
|
||||
|
||||
Purpose:
|
||||
- Execute exactly one stage.
|
||||
|
||||
Syntax:
|
||||
|
||||
```bash
|
||||
narratio run-stage [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] [--artifacts <name[,name...]>] <stage>
|
||||
```
|
||||
|
||||
Success output:
|
||||
- `narratio run-stage: stage=<name> executed=<n> skipped=<n> force=<true|false>; manifest=<path>`
|
||||
|
||||
`--artifacts` behavior:
|
||||
- accepted only when `<stage>` is `analyze`.
|
||||
- names are normalized (trimmed, deduplicated, sorted).
|
||||
- unknown configured artifact keys fail.
|
||||
`--artifacts` is accepted only for `analyze` and `archive`.
|
||||
|
||||
Common failure cases:
|
||||
- missing stage positional arg.
|
||||
- unknown stage name.
|
||||
- using `--artifacts` with any non-`analyze` stage.
|
||||
### `analyze`
|
||||
|
||||
```bash
|
||||
narratio analyze <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--artifacts <name[,name...]>]
|
||||
```
|
||||
|
||||
Purpose:
|
||||
- Force-rerun the analyze stage.
|
||||
- Shorter equivalent for `narratio run-stage analyze <session_id> --force`.
|
||||
|
||||
`analyze` is force-by-design and does not accept `--force`.
|
||||
|
||||
### `publish`
|
||||
|
||||
```bash
|
||||
narratio publish <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--artifacts <name[,name...]>]
|
||||
```
|
||||
|
||||
Purpose:
|
||||
- Force-rerun the archive stage.
|
||||
- Shorter equivalent for `narratio run-stage archive <session_id> --force`.
|
||||
|
||||
`publish` is force-by-design and does not accept `--force` or a stage positional argument.
|
||||
|
||||
### `clean`
|
||||
|
||||
```bash
|
||||
narratio clean <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--dry-run] [--clear-cache]
|
||||
narratio clean --all [--config <pipeline.yml>] [--dry-run] [--clear-cache]
|
||||
```
|
||||
|
||||
Session cleanup deletes:
|
||||
- `{workspace.root}/work/{campaign}/{session_id}`
|
||||
- `{spool.root}/{campaign}/{session_id}`
|
||||
|
||||
All-session cleanup deletes:
|
||||
- `{workspace.root}/work`
|
||||
- the contents of `{spool.root}`, while preserving the spool root directory itself.
|
||||
|
||||
Cache behavior:
|
||||
- cache is preserved by default.
|
||||
- `--clear-cache` in session mode removes cached S3 audio files for the resolved session.
|
||||
- `--all --clear-cache` removes the configured Narratio S3 audio cache namespace for the configured bucket/root prefix.
|
||||
- `--clear-cache` does not delete arbitrary files under `pipeline.cache.root`.
|
||||
|
||||
### `session plan`
|
||||
|
||||
```bash
|
||||
narratio session plan <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--force]
|
||||
```
|
||||
|
||||
Purpose:
|
||||
- Validate config, load secrets if configured, prepare workdir, and print stage run/skip decisions.
|
||||
|
||||
Success output includes:
|
||||
- `narratio session plan: workdir prepared at <path>`
|
||||
- one line per stage (`<stage>: run|skip`)
|
||||
- `totals: run=<n> skip=<n>`
|
||||
|
||||
### `session status`
|
||||
|
||||
```bash
|
||||
narratio session status <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>]
|
||||
```
|
||||
|
||||
Output includes:
|
||||
- session ID, campaign, workspace, and session config source.
|
||||
- local manifest state when present.
|
||||
- remote current archive state when storage is configured.
|
||||
- catalog-based promoted output availability for expected transcript and artifact sources.
|
||||
- effective archive locks and conservative next actions.
|
||||
|
||||
### `session validate`
|
||||
|
||||
```bash
|
||||
narratio session validate <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>]
|
||||
```
|
||||
|
||||
Checks include:
|
||||
- effective config and session source.
|
||||
- stable input files.
|
||||
- local or remote audio availability.
|
||||
- previous-session requirements.
|
||||
- archive promotions and effective locks.
|
||||
|
||||
Warnings do not fail the command. Any `ERROR` finding exits non-zero.
|
||||
|
||||
### `session init`
|
||||
|
||||
```bash
|
||||
narratio session init <session_id> --output ./session.yml
|
||||
narratio session init <session_id> --remote
|
||||
narratio session init <session_id> --config <pipeline.yml> --campaign icewind --remote
|
||||
narratio session init <session_id> --config <pipeline.yml> --campaign-file ./campaign.yml --remote
|
||||
```
|
||||
|
||||
Additional flags:
|
||||
|
||||
- `--previous-session-id <value>`
|
||||
- `--date <value>`
|
||||
- `--title <value>`
|
||||
- `--audio-s3-prefix <prefix>`: defaults to `audio/` when neither audio flag is provided.
|
||||
- `--audio-dir <path>`: local audio directory; mutually exclusive with `--audio-s3-prefix`.
|
||||
- `--force`: overwrite existing local or remote target.
|
||||
|
||||
Behavior:
|
||||
- exactly one of `--output` or `--remote` is required.
|
||||
- `--config`, `--campaign`, and `--campaign-file` are optional overrides; omitted campaign selection uses `pipeline.campaigns.default_campaign_id`.
|
||||
- `--campaign <id>` selects a campaign under `pipeline.campaigns.root`.
|
||||
- `--campaign-file <path>` loads an explicit campaign file.
|
||||
- if `campaign.yml` sets `session_template_file`, the template path is resolved relative to `campaign.yml` and rendered from init flags.
|
||||
- if no session template is configured, a minimal concrete session file is generated directly.
|
||||
- template variables must be supplied by matching flags, and supplied template-related flags must be used by the template.
|
||||
- remote writes target `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`.
|
||||
- existing local or remote targets fail unless `--force` is passed.
|
||||
- remote writes use existence checks, not compare-and-swap.
|
||||
|
||||
### `session restore`
|
||||
|
||||
```bash
|
||||
narratio session restore <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--dry-run] [--force] [--include-audio]
|
||||
```
|
||||
|
||||
Purpose:
|
||||
- Restore durable session state from the committed remote archive current state.
|
||||
- Default restore installs `manifest.json`, `transcripts/**`, and `artifacts/**` from the current session archive.
|
||||
- When configured previous-session inputs require it, restore reconstructs `previous/**` from the previous session's committed current archive.
|
||||
- `audio/**` is restored only with `--include-audio`.
|
||||
|
||||
Dry-run output may include planned previous-cache downloads. Existing differing files under `previous/**` follow the normal restore conflict policy and require `--force` to overwrite.
|
||||
|
||||
When `--include-audio` is set, S3 audio files are restored through the shared audio cache. Cache hits avoid re-downloading large audio objects.
|
||||
|
||||
### `session artifacts`
|
||||
|
||||
```bash
|
||||
narratio session artifacts <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--remote]
|
||||
```
|
||||
|
||||
Purpose:
|
||||
- List built-in, configured, previous-session, promoted, and locked artifact sources.
|
||||
|
||||
`--remote` checks promoted top-level object availability through the storage adapter. Remote markers appear only in the `Promoted` section, which reports each configured archive promotion destination and includes `dest=<path>` when that destination differs from the source's canonical path.
|
||||
|
||||
### `session locks`
|
||||
|
||||
```bash
|
||||
narratio session locks <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>]
|
||||
narratio session locks add <session_id> <source> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--reason <text>] [--force]
|
||||
narratio session locks remove <session_id> <source> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>]
|
||||
```
|
||||
|
||||
Behavior:
|
||||
- list mode prints effective locks from static `pipeline.archive.locks` and remote `{session_prefix}/locks.yml`.
|
||||
- `locks add` writes only the remote lock store and fails if the source is already locked by pipeline config.
|
||||
- `locks remove` removes only remote locks and cannot remove static pipeline locks.
|
||||
- `locks add --force` is required to update an existing remote lock reason.
|
||||
|
||||
## Common Workflows
|
||||
|
||||
Default-discovery run:
|
||||
|
||||
```bash
|
||||
narratio run --session-id 2026-04-04
|
||||
narratio run 2026-04-04
|
||||
```
|
||||
|
||||
Run only selected analyze artifacts:
|
||||
|
||||
```bash
|
||||
narratio run --session-id 2026-04-04 --artifacts session_recap,player_handout
|
||||
narratio run 2026-04-04 --artifacts session_recap,player_handout
|
||||
```
|
||||
|
||||
Resume with selected analyze artifacts:
|
||||
|
||||
```bash
|
||||
narratio resume --session-id 2026-04-04 --artifacts player_handout
|
||||
narratio resume 2026-04-04 --artifacts player_handout
|
||||
```
|
||||
|
||||
Run only analyze stage with selected artifacts:
|
||||
Force-rerun analyze with selected artifacts:
|
||||
|
||||
```bash
|
||||
narratio run-stage --session-id 2026-04-04 --artifacts player_handout analyze
|
||||
narratio analyze 2026-04-04 --artifacts player_handout
|
||||
```
|
||||
|
||||
## Diagnostic / Recovery Commands
|
||||
|
||||
Inspect stage status:
|
||||
Force-rerun archive publishing:
|
||||
|
||||
```bash
|
||||
narratio status --manifest <manifest.json>
|
||||
narratio publish 2026-04-04
|
||||
```
|
||||
|
||||
Get manifest path from previous output:
|
||||
- `run`, `resume`, and `run-stage` print `manifest=<path>` on success.
|
||||
Preview restore actions without writes:
|
||||
|
||||
```bash
|
||||
narratio session restore 2026-04-04 --dry-run
|
||||
```
|
||||
|
||||
Restore and then force analyze:
|
||||
|
||||
```bash
|
||||
narratio session restore 2026-04-04
|
||||
narratio analyze 2026-04-04
|
||||
```
|
||||
|
||||
Rehydrate canonical previous-session inputs after artifact-input changes:
|
||||
|
||||
```bash
|
||||
narratio run-stage prepare 2026-04-04 --force
|
||||
```
|
||||
|
||||
Reset local state before testing restore:
|
||||
|
||||
```bash
|
||||
narratio clean 2026-04-04 --dry-run
|
||||
narratio clean 2026-04-04
|
||||
narratio session restore 2026-04-04 --include-audio
|
||||
```
|
||||
|
||||
Clean all local sessions while keeping cached S3 audio:
|
||||
|
||||
```bash
|
||||
narratio clean --all
|
||||
```
|
||||
|
||||
## `--artifacts` and `--force`
|
||||
|
||||
- `--artifacts` filters which configured artifacts are executable when analyze runs.
|
||||
- `--artifacts` filters which configured artifacts are executable when analyze runs and which configured artifact promotions archive publishes.
|
||||
- `--artifacts` does not imply `--force`.
|
||||
- If analyze is already `succeeded` and `--force` is not set, runner-level skip still applies.
|
||||
- if analyze is already `succeeded` and `--force` is not set, runner-level skip still applies.
|
||||
- `--artifacts` does not suppress built-in transcript or bounds promotions.
|
||||
|
||||
339
docs/config.md
339
docs/config.md
@@ -2,58 +2,120 @@
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Narratio loads two YAML files:
|
||||
Narratio loads three YAML files:
|
||||
|
||||
- `pipeline.yml`: pipeline-level runtime configuration.
|
||||
- `session.yml`: per-session metadata and input selection.
|
||||
- `campaign.yml`: stable campaign identity and campaign-level input defaults.
|
||||
- `session.yml`: per-session metadata and input selection, loaded locally or from the configured S3 backend.
|
||||
|
||||
These commands load and validate both files before running:
|
||||
These commands load and validate all three files before running:
|
||||
|
||||
- `narratio run`
|
||||
- `narratio plan`
|
||||
- `narratio resume`
|
||||
- `narratio run-stage`
|
||||
- `narratio analyze`
|
||||
- `narratio publish`
|
||||
- `narratio session plan`
|
||||
- `narratio session status`
|
||||
- `narratio session validate`
|
||||
- `narratio session restore`
|
||||
- `narratio session artifacts`
|
||||
- `narratio session locks`
|
||||
- `narratio clean <session_id>`
|
||||
|
||||
Behavior:
|
||||
|
||||
- strict YAML decode is enabled (`KnownFields(true)`): unknown fields fail.
|
||||
- session templates render before session YAML decode.
|
||||
- ordinary local and remote `session.yml` files must be concrete YAML; template placeholders are rejected.
|
||||
- defaults are applied for optional pipeline fields.
|
||||
- campaign identity is selected by ID from the pipeline campaign registry unless `--campaign-file` is used.
|
||||
- campaign-level stable input paths fill missing session input paths.
|
||||
- session-level stable input paths override campaign-level input paths.
|
||||
- campaign config may point `session init` to a session template.
|
||||
- validation enforces required fields, value formats, and cross-field constraints.
|
||||
|
||||
## 2. Config file discovery
|
||||
|
||||
Pipeline config lookup for `run`, `plan`, `resume`, and `run-stage`:
|
||||
These commands use the same config discovery behavior:
|
||||
|
||||
- If `--config <path>` is provided, that path is used.
|
||||
- If omitted, Narratio searches in order:
|
||||
- `narratio run`
|
||||
- `narratio resume`
|
||||
- `narratio run-stage`
|
||||
- `narratio analyze`
|
||||
- `narratio publish`
|
||||
- `narratio session plan`
|
||||
- `narratio session status`
|
||||
- `narratio session validate`
|
||||
- `narratio session restore`
|
||||
- `narratio session artifacts`
|
||||
- `narratio session locks`
|
||||
- `narratio clean <session_id>`
|
||||
|
||||
Pipeline config lookup:
|
||||
|
||||
- if `--config <path>` is provided, that path is used.
|
||||
- if omitted, Narratio searches in order:
|
||||
1. `/usr/local/etc/narratio/pipeline.yml`
|
||||
2. `/etc/narratio/pipeline.yml`
|
||||
- First existing file wins.
|
||||
- first existing file wins.
|
||||
|
||||
## 3. Session file discovery and templating
|
||||
Campaign config lookup:
|
||||
|
||||
Session config lookup for `run`, `plan`, `resume`, and `run-stage`:
|
||||
- pipeline config is loaded first.
|
||||
- if `--campaign-file <path>` is provided, that path is used.
|
||||
- otherwise, if `--campaign <id>` is provided, Narratio loads:
|
||||
- `{pipeline.campaigns.root}/{id}/campaign.yml`
|
||||
- otherwise, Narratio uses `pipeline.campaigns.default_campaign_id` and loads:
|
||||
- `{pipeline.campaigns.root}/{default_campaign_id}/campaign.yml`
|
||||
- `--campaign` and `--campaign-file` are mutually exclusive.
|
||||
- campaign IDs must be single path segments, not paths.
|
||||
|
||||
- If `--session <path>` is provided, that path is used.
|
||||
- If omitted, Narratio searches in order:
|
||||
1. `./session.yml`
|
||||
2. `/usr/local/etc/narratio/session.yml`
|
||||
3. `/etc/narratio/session.yml`
|
||||
- First existing file wins.
|
||||
Session config lookup:
|
||||
|
||||
Template behavior:
|
||||
- if `--session <path>` is provided, that path is used.
|
||||
- if `--session` is omitted, Narratio searches locally in order:
|
||||
1. `/usr/local/etc/narratio/session.yml`
|
||||
2. `/etc/narratio/session.yml`
|
||||
- first existing local file wins.
|
||||
- if no local session file is found, a positional `<session_id>` is present, storage is configured, and campaign identity is resolved, Narratio loads remote `session.yml` from:
|
||||
- `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`
|
||||
- local discovery always runs before remote fallback.
|
||||
- local files in the current working directory are used only when passed explicitly, for example `--config ./pipeline.yml --campaign-file ./campaign.yml --session ./session.yml`.
|
||||
|
||||
- Supported placeholders:
|
||||
- `{{session_id}}`
|
||||
## 3. Session templating
|
||||
|
||||
Template behavior for local and remote `session.yml` loaded by downstream commands:
|
||||
|
||||
- downstream commands do not render templates.
|
||||
- local and remote `session.yml` must be concrete.
|
||||
- any `{{ ... }}` placeholder in loaded `session.yml` fails with guidance to run `narratio session init`.
|
||||
- if concrete `session_id` mismatches the positional `<session_id>`, load fails.
|
||||
- if concrete `previous_session_id` mismatches `--previous-session-id`, load fails.
|
||||
|
||||
Template behavior for `narratio session init`:
|
||||
|
||||
- `campaign.yml` may set `session_template_file`.
|
||||
- relative template paths resolve relative to `campaign.yml`.
|
||||
- supported init template variables:
|
||||
- `{{ session_id }}`
|
||||
- `--session-id <value>` supplies the placeholder value.
|
||||
- unresolved placeholders fail load.
|
||||
- if rendered `session_id` mismatches `--session-id`, load fails.
|
||||
- `{{ previous_session_id }}`
|
||||
- `{{ date }}`
|
||||
- `{{ title }}`
|
||||
- `{{ audio_s3_prefix }}`
|
||||
- `{{ audio_dir }}`
|
||||
- each template variable must be supplied by the matching `session init` flag.
|
||||
- template-related flags such as `--date`, `--title`, `--audio-s3-prefix`, `--audio-dir`, and `--previous-session-id` fail if the configured template does not use them.
|
||||
- rendered output is strict-decoded and validated before it is written locally or remotely.
|
||||
- if `session_template_file` is omitted, `session init` generates the minimal concrete session YAML directly.
|
||||
|
||||
## 4. Minimal pipeline config
|
||||
## 4. Minimal config set
|
||||
|
||||
### `pipeline.yml`
|
||||
|
||||
```yaml
|
||||
campaigns:
|
||||
root: /usr/local/share/narratio/campaigns
|
||||
default_campaign_id: sample-campaign
|
||||
whisperx:
|
||||
transcribe_url: "https://transcription.example.com/transcribe"
|
||||
```
|
||||
@@ -61,28 +123,64 @@ whisperx:
|
||||
Why this is sufficient:
|
||||
|
||||
- `whisperx.transcribe_url` is required.
|
||||
- `campaigns.default_campaign_id` selects the default campaign when `--campaign` is omitted.
|
||||
- `workspace.root` defaults to `/var/lib/narratio`.
|
||||
- optional sections (`seriatim`, `audita`, `archive`, `scriptorium`, `trim`, `normalize`, etc.) receive defaults or stay inactive.
|
||||
|
||||
## 5. Minimal session template
|
||||
### `campaign.yml`
|
||||
|
||||
```yaml
|
||||
session_id: "{{ session_id }}"
|
||||
campaign: sample-campaign
|
||||
campaign_id: sample-campaign
|
||||
session_template_file: ./session.template.yml
|
||||
inputs:
|
||||
speakers_file: ./speakers.yml
|
||||
autocorrect_file: ./autocorrect.yml
|
||||
glossary_file: ./glossary.yml
|
||||
```
|
||||
|
||||
Why this is sufficient:
|
||||
|
||||
- `campaign_id` supplies the stable campaign identity.
|
||||
- stable input files are required and resolve relative to `campaign.yml` when copied during `prepare`.
|
||||
|
||||
### `session.yml`
|
||||
|
||||
```yaml
|
||||
session_id: 2026-05-03
|
||||
inputs:
|
||||
audio_dir: ./audio
|
||||
speakers_file: ./examples/speakers.yml
|
||||
autocorrect_file: ./examples/autocorrect.yml
|
||||
glossary_file: ./examples/glossary.yml
|
||||
```
|
||||
|
||||
Usage:
|
||||
Why this is sufficient:
|
||||
|
||||
- `session_id` is required.
|
||||
- `campaign` can be omitted because it is supplied by `campaign.yml`.
|
||||
- stable input paths can be omitted because `campaign.yml` supplies defaults.
|
||||
- local `audio_dir` resolves relative to `session.yml`.
|
||||
|
||||
Minimal local-file usage:
|
||||
|
||||
```bash
|
||||
narratio run --config /path/to/pipeline.yml --session ./session.yml --session-id 2026-05-03
|
||||
narratio run 2026-05-03 --config /path/to/pipeline.yml --campaign sample-campaign --session ./session.yml
|
||||
narratio run 2026-05-03 --config /path/to/pipeline.yml --campaign-file ./campaign.yml --session ./session.yml
|
||||
```
|
||||
|
||||
## 6. Production-oriented config
|
||||
Previous-session-enabled variant:
|
||||
|
||||
```yaml
|
||||
session_id: 2026-05-03
|
||||
previous_session_id: 2026-04-26
|
||||
inputs:
|
||||
audio_dir: ./audio
|
||||
```
|
||||
|
||||
```bash
|
||||
narratio run 2026-05-03 --config /path/to/pipeline.yml --campaign sample-campaign --session ./session.yml --previous-session-id 2026-04-26
|
||||
```
|
||||
|
||||
## 5. Production-oriented config set
|
||||
|
||||
### `pipeline.yml`
|
||||
|
||||
```yaml
|
||||
workspace:
|
||||
@@ -98,20 +196,31 @@ storage:
|
||||
access_key_id_env: OBJECT_STORAGE_KEY_ID
|
||||
secret_access_key_env: OBJECT_STORAGE_KEY
|
||||
|
||||
campaigns:
|
||||
root: /srv/narratio/campaigns
|
||||
default_campaign_id: forsaken
|
||||
|
||||
spool:
|
||||
root: /var/spool/narratio
|
||||
delete_audio_after_archive: true
|
||||
|
||||
cache:
|
||||
root: /var/cache/narratio
|
||||
s3_audio: true
|
||||
|
||||
archive:
|
||||
enabled: true
|
||||
upload_run: true
|
||||
promote_artifacts:
|
||||
- from: transcripts/trimmed.json
|
||||
to: transcripts/trimmed.json
|
||||
- source: narratio.transcript.final_trimmed
|
||||
dest: transcripts/final.trimmed.json
|
||||
required: true
|
||||
- from: artifacts/session_recap.md
|
||||
to: artifacts/session_recap.md
|
||||
- source: narratio.artifact.session_recap
|
||||
dest: artifacts/session_recap.md
|
||||
required: true
|
||||
locks:
|
||||
- source: narratio.artifact.session_recap
|
||||
reason: Final recap was manually edited.
|
||||
|
||||
whisperx:
|
||||
transcribe_url: "https://transcription.example.com/transcribe"
|
||||
@@ -124,25 +233,69 @@ scriptorium:
|
||||
output_path: artifacts/session_recap.md
|
||||
inputs:
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
source: narratio.transcript.final_trimmed
|
||||
required: true
|
||||
previous_recap:
|
||||
source: narratio.previous_session.artifact.session_recap
|
||||
required: false
|
||||
```
|
||||
|
||||
### `campaign.yml`
|
||||
|
||||
```yaml
|
||||
campaign_id: forsaken
|
||||
inputs:
|
||||
speakers_file: /srv/narratio/campaigns/forsaken/speakers.yml
|
||||
autocorrect_file: /srv/narratio/campaigns/forsaken/autocorrect.yml
|
||||
glossary_file: /srv/narratio/campaigns/forsaken/glossary.yml
|
||||
```
|
||||
|
||||
### Local `session.yml`
|
||||
|
||||
```yaml
|
||||
session_id: 2026-05-03
|
||||
previous_session_id: 2026-04-26
|
||||
date: 2026-05-03
|
||||
title: The Black Cabin
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
```
|
||||
|
||||
### S3-first session config
|
||||
|
||||
For S3-first operation, upload the same `session.yml` content to:
|
||||
|
||||
```text
|
||||
{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml
|
||||
```
|
||||
|
||||
Then run with explicit or discovered pipeline/campaign config and no `--session`:
|
||||
|
||||
```bash
|
||||
narratio run 2026-05-03 --config /usr/local/etc/narratio/pipeline.yml --campaign forsaken --previous-session-id 2026-04-26
|
||||
```
|
||||
|
||||
Operational notes:
|
||||
|
||||
- archive promotion is explicit and path-based via `archive.promote_artifacts`.
|
||||
- archive promotion is explicit and source-based via `archive.promote_artifacts`.
|
||||
- `source` is required; `dest` is optional and derived when omitted.
|
||||
- `archive.locks` skips top-level promotion overwrites for static locked sources while preserving run-local uploads.
|
||||
- operator-created mutable locks are stored at `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/locks.yml` and are merged with static locks.
|
||||
- Narratio does not auto-promote all generated analyze artifacts.
|
||||
- `restore` reads the same config/campaign/session inputs and restore scope is bounded by committed archive current state.
|
||||
- `clean` removes workspace/spool state by default and preserves `pipeline.cache.root` unless `--clear-cache` is passed.
|
||||
|
||||
## 7. Full pipeline reference
|
||||
## 6. Full pipeline reference
|
||||
|
||||
| Path | Type | Required | Default |
|
||||
| --- | --- | --- | --- |
|
||||
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
|
||||
| `pipeline.workspace.cleanup_after_archive` | bool | No | `false` |
|
||||
| `pipeline.campaigns.root` | string | No | `/usr/local/share/narratio/campaigns` |
|
||||
| `pipeline.campaigns.default_campaign_id` | string | No | empty |
|
||||
| `pipeline.secrets.env_dir` | string | Conditional | none |
|
||||
| `pipeline.storage.backend` | string | No | empty |
|
||||
| `pipeline.storage.bucket` | string | No | empty |
|
||||
| `pipeline.storage.prefix` | string | No | empty |
|
||||
| `pipeline.storage.s3.bucket` | string | Conditional | empty |
|
||||
| `pipeline.storage.s3.root_prefix` | string | No | `dnd` |
|
||||
| `pipeline.storage.s3.region` | string | No | empty |
|
||||
@@ -152,12 +305,17 @@ Operational notes:
|
||||
| `pipeline.storage.s3.secret_access_key_env` | string | No | `OBJECT_STORAGE_KEY` |
|
||||
| `pipeline.spool.root` | string | No | `/var/spool/narratio` |
|
||||
| `pipeline.spool.delete_audio_after_archive` | bool | No | `false` |
|
||||
| `pipeline.cache.root` | string | No | `/var/cache/narratio` |
|
||||
| `pipeline.cache.s3_audio` | bool | No | `true` |
|
||||
| `pipeline.archive.enabled` | bool | No | `true` |
|
||||
| `pipeline.archive.upload_run` | bool | No | `true` |
|
||||
| `pipeline.archive.promote_artifacts[]` | list | No | trimmed + session_recap rules |
|
||||
| `pipeline.archive.promote_artifacts[].from` | string | Yes (per rule) | none |
|
||||
| `pipeline.archive.promote_artifacts[].to` | string | Yes (per rule) | none |
|
||||
| `pipeline.archive.promote_artifacts[]` | list | No | final-trimmed transcript rule |
|
||||
| `pipeline.archive.promote_artifacts[].source` | string | Yes (per rule) | none |
|
||||
| `pipeline.archive.promote_artifacts[].dest` | string | No | derived from source |
|
||||
| `pipeline.archive.promote_artifacts[].required` | bool | No | `true` |
|
||||
| `pipeline.archive.locks[]` | list | No | empty |
|
||||
| `pipeline.archive.locks[].source` | string | Yes (per lock) | none |
|
||||
| `pipeline.archive.locks[].reason` | string | No | empty |
|
||||
| `pipeline.whisperx.transcribe_url` | string | Yes | none |
|
||||
| `pipeline.whisperx.language` | string | No | `en` |
|
||||
| `pipeline.whisperx.timeout` | duration string | No | `30m` |
|
||||
@@ -188,7 +346,7 @@ Operational notes:
|
||||
| `pipeline.audita.output_schema` | string | No | empty |
|
||||
| `pipeline.audita.work_dir_retention` | string | No | empty |
|
||||
| `pipeline.audita.report` | bool | No | `true` |
|
||||
| `pipeline.normalize.output_path` | string | No | `transcripts/normalized.json` |
|
||||
| `pipeline.normalize.output_path` | string | No | `transcripts/final.json` |
|
||||
| `pipeline.normalize.output_schema` | string | No | `seriatim-intermediate` |
|
||||
| `pipeline.normalize.report` | bool | No | `true` |
|
||||
| `pipeline.trim.enabled` | bool | No | `false` |
|
||||
@@ -218,10 +376,6 @@ Operational notes:
|
||||
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.path` | string | No | empty |
|
||||
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.required` | bool | No | `false` |
|
||||
| `pipeline.scriptorium.artifacts.<name>.vars.<key>` | map value | No | empty |
|
||||
| `pipeline.analyzer.binary_path` | string | No | empty |
|
||||
| `pipeline.analyzer.timeout` | duration string | No | empty |
|
||||
| `pipeline.analyzer.artifacts.output_dir` | string | No | empty |
|
||||
| `pipeline.analyzer.artifacts.types[]` | list[string] | No | empty |
|
||||
| `pipeline.notification.backend` | string | No | empty |
|
||||
| `pipeline.notification.recipient` | string | No | empty |
|
||||
| `pipeline.notification.timeout` | duration string | No | empty |
|
||||
@@ -240,28 +394,86 @@ Scriptorium artifact-key and dependency rules:
|
||||
|
||||
Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:
|
||||
|
||||
- `previous_session_artifact`
|
||||
- `narratio.transcript.merged`
|
||||
- `narratio.previous_session.artifact.<configured_artifact_key>`
|
||||
- `narratio.transcript.base`
|
||||
- `narratio.transcript.polished`
|
||||
- `narratio.transcript.full`
|
||||
- `narratio.transcript.trimmed`
|
||||
- `narratio.transcript.final`
|
||||
- `narratio.transcript.final_trimmed`
|
||||
- `narratio.bounds.session`
|
||||
- `narratio.artifact.<configured_artifact_key>`
|
||||
|
||||
`pipeline.archive.promote_artifacts[].source` values:
|
||||
|
||||
- `narratio.transcript.base`
|
||||
- `narratio.transcript.polished`
|
||||
- `narratio.transcript.final`
|
||||
- `narratio.transcript.final_trimmed`
|
||||
- `narratio.bounds.session`
|
||||
- `narratio.artifact.<configured_artifact_key>`
|
||||
|
||||
`pipeline.archive.locks[].source` accepts the same source values as `pipeline.archive.promote_artifacts[].source`.
|
||||
|
||||
Archive promotion destination rules:
|
||||
|
||||
- `dest` must be a clean relative path (not absolute, no traversal).
|
||||
- duplicate `dest` values are rejected.
|
||||
- if `dest` is omitted:
|
||||
- built-in sources derive their canonical destination path;
|
||||
- configured sources derive from `pipeline.scriptorium.artifacts.<name>.output_path`;
|
||||
- derivation failure is a config validation error.
|
||||
|
||||
Archive lock rules:
|
||||
|
||||
- locks are source-based and do not accept `dest`.
|
||||
- duplicate lock sources are rejected.
|
||||
- static `pipeline.archive.locks` win over remote mutable locks for the same source.
|
||||
- locked promotions are recorded as intentional skips in archive metadata.
|
||||
- locked required promotions do not fail archive by default.
|
||||
- ordinary `--force` reruns do not override locks.
|
||||
|
||||
Remote mutable lock store:
|
||||
|
||||
- path: `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/locks.yml`.
|
||||
- strict YAML shape: top-level `locks`, each with `source` and optional `reason`.
|
||||
- `narratio session locks add` and `narratio session locks remove` mutate only the remote lock store.
|
||||
- writes use existence checks plus `--force` for updates; they are not compare-and-swap atomic.
|
||||
|
||||
Restore-related implications:
|
||||
|
||||
- restore remote identity requires archive S3 identity to resolve (`pipeline.storage.s3.bucket` and session prefix derivation inputs).
|
||||
- restore scope considers committed current state and durable paths (`manifest.json`, `transcripts/**`, `artifacts/**`, `previous/**`, optional `audio/**`).
|
||||
- S3 audio downloads use `pipeline.spool.root` for active downloads and `pipeline.cache.root` for reusable cached audio when `pipeline.cache.s3_audio` is true.
|
||||
- `pipeline.cache.root` is durable local cache state. It is not workspace state and is preserved by default by `narratio clean`.
|
||||
|
||||
## 7. Full campaign reference
|
||||
|
||||
| Path | Type | Required | Default |
|
||||
| --- | --- | --- | --- |
|
||||
| `campaign.campaign_id` | string | Yes | none |
|
||||
| `campaign.session_template_file` | string | No | none |
|
||||
| `campaign.inputs.speakers_file` | string | Yes | none |
|
||||
| `campaign.inputs.autocorrect_file` | string | Yes | none |
|
||||
| `campaign.inputs.glossary_file` | string | Yes | none |
|
||||
|
||||
Campaign input paths and `campaign.session_template_file` may be absolute or relative. Relative paths resolve from the directory containing `campaign.yml`.
|
||||
|
||||
## 8. Full session reference
|
||||
|
||||
| Path | Type | Required | Default |
|
||||
| --- | --- | --- | --- |
|
||||
| `session.session_id` | string | Yes | none |
|
||||
| `session.campaign` | string | Yes | none |
|
||||
| `session.previous_session_id` | string | No | empty |
|
||||
| `session.campaign` | string | No | `campaign.campaign_id` |
|
||||
| `session.date` | string | No | empty |
|
||||
| `session.title` | string | No | empty |
|
||||
| `session.inputs.audio_dir` | string | Conditional | empty |
|
||||
| `session.inputs.audio_files[]` | list[string] | Conditional | empty |
|
||||
| `session.inputs.audio_s3.prefix` | string | Conditional | none |
|
||||
| `session.inputs.speakers_file` | string | Yes | none |
|
||||
| `session.inputs.autocorrect_file` | string | Yes | none |
|
||||
| `session.inputs.glossary_file` | string | Yes | none |
|
||||
| `session.inputs.speakers_file` | string | No | `campaign.inputs.speakers_file` |
|
||||
| `session.inputs.autocorrect_file` | string | No | `campaign.inputs.autocorrect_file` |
|
||||
| `session.inputs.glossary_file` | string | No | `campaign.inputs.glossary_file` |
|
||||
|
||||
Session input paths may be absolute or relative. Relative audio paths and session-level stable input overrides resolve from the directory containing `session.yml`. If both `campaign.yml` and `session.yml` specify campaign identity, the values must match.
|
||||
|
||||
Audio-source rule:
|
||||
|
||||
@@ -271,6 +483,11 @@ Audio-source rule:
|
||||
- `audio_s3.prefix`
|
||||
- `audio_s3` cannot be combined with local audio fields.
|
||||
|
||||
Previous-session rule:
|
||||
|
||||
- if `session.previous_session_id` is set, it must not equal `session.session_id`.
|
||||
- canonical previous-session sources (`narratio.previous_session.artifact.<name>`) are hydrated during `prepare` from archive current state when required by enabled configured artifacts.
|
||||
|
||||
## 9. Secrets
|
||||
|
||||
Narratio supports filesystem-based secret injection via `pipeline.secrets.env_dir`.
|
||||
@@ -297,7 +514,11 @@ Maintained examples:
|
||||
- `examples/pipeline.minimal.yml`
|
||||
- `examples/pipeline.production.yml`
|
||||
- `examples/pipeline.full.annotated.yml`
|
||||
- `examples/session.template.yml`
|
||||
- `examples/campaigns/sample-campaign/campaign.yml`
|
||||
- `examples/campaigns/sample-campaign/speakers.yml`
|
||||
- `examples/campaigns/sample-campaign/autocorrect.yml`
|
||||
- `examples/campaigns/sample-campaign/glossary.yml`
|
||||
- `examples/campaigns/sample-campaign/session.template.yml`
|
||||
- `examples/session.local-audio.yml`
|
||||
- `examples/session.s3-audio.yml`
|
||||
|
||||
|
||||
@@ -69,6 +69,8 @@ For design principles and invariants, see [docs/architecture.md](./architecture.
|
||||
2. Add or update command tests (`TestExecute` and command-specific tests).
|
||||
3. Update [docs/cli.md](./cli.md) and, if operator workflow changes, [docs/operations.md](./operations.md).
|
||||
|
||||
Remote-storage commands must obtain object storage through the app-level command object-store helper. Do not call `storage.NewObjectStoreFromConfig` directly from command handlers; the helper loads configured filesystem secrets before constructing the storage adapter.
|
||||
|
||||
### Add or modify stages/adapters
|
||||
|
||||
1. Implement stage behavior in `internal/stage` with clear input/output boundaries.
|
||||
|
||||
@@ -5,9 +5,9 @@ Define Narratio's adapter contract for transcript polishing via Audita CLI subpr
|
||||
|
||||
## Inputs and Outputs
|
||||
Inputs (`audita.PolishRequest`):
|
||||
- merged transcript path
|
||||
- base transcript path
|
||||
- glossary path
|
||||
- output processed transcript path
|
||||
- output polished transcript path
|
||||
- optional report path (required when report enabled)
|
||||
- work dir
|
||||
- generated config path
|
||||
@@ -15,7 +15,7 @@ Inputs (`audita.PolishRequest`):
|
||||
- optional module/model/base URL and concurrency knobs
|
||||
|
||||
Outputs (`audita.PolishResult`):
|
||||
- processed transcript path
|
||||
- polished transcript path
|
||||
- optional report path
|
||||
- generated config path
|
||||
- stdout/stderr log paths
|
||||
@@ -27,7 +27,7 @@ Owns:
|
||||
- Deterministic CLI argument construction for `audita process`
|
||||
- Environment bridging for API credentials
|
||||
- Invocation config emission
|
||||
- Output validation for processed transcript and report
|
||||
- Output validation for polished transcript and report
|
||||
|
||||
Does not own:
|
||||
- Upstream/downstream stage orchestration
|
||||
@@ -52,7 +52,7 @@ Via `pipeline.audita.*` mapped in app/stage wiring:
|
||||
|
||||
## Failure Behavior
|
||||
- Constructor validation fails on invalid binary/timeout/schema/concurrency/URL values.
|
||||
- Run fails on missing required paths, missing required credential env var, subprocess errors, invalid processed JSON shape, or invalid report JSON.
|
||||
- Run fails on missing required paths, missing required credential env var, subprocess errors, invalid polished JSON shape, or invalid report JSON.
|
||||
- Failures preserve stdout/stderr paths in returned result metadata.
|
||||
|
||||
## Tests to Inspect Before Changing
|
||||
@@ -61,6 +61,6 @@ Via `pipeline.audita.*` mapped in app/stage wiring:
|
||||
- `internal/stage/polish_test.go`
|
||||
|
||||
## Architectural Invariants
|
||||
- Processed output must be valid JSON with top-level `segments` array.
|
||||
- Polished output must be valid JSON with top-level `segments` array.
|
||||
- When report is enabled, report output must be valid JSON.
|
||||
- If `llm_api_key_env` is configured, credential must be present in environment.
|
||||
|
||||
@@ -5,7 +5,7 @@ Define Narratio's adapter contract for merge, normalize, and trim subprocess inv
|
||||
|
||||
## Inputs and Outputs
|
||||
Inputs:
|
||||
- `MergeRequest`: raw/normalized transcript inputs, output path, optional report, speaker/autocorrect paths, logs/config
|
||||
- `MergeRequest`: raw/per-speaker normalized transcript inputs, base output path, optional report, speaker/autocorrect paths, logs/config
|
||||
- `NormalizeRequest`: input transcript, output path, schema, optional report, timeout/log/config
|
||||
- `TrimRequest`: input transcript, output path, keep selector, timeout/log/config
|
||||
|
||||
@@ -44,8 +44,8 @@ Via `pipeline.seriatim.*` mapped in app/stage wiring:
|
||||
## Failure Behavior
|
||||
- Constructor fails for invalid binary/timeout/output-schema/coalesce-gap.
|
||||
- Merge fails on missing output path/inputs/report path (if enabled), subprocess errors, invalid merged output JSON, invalid report JSON.
|
||||
- Normalize fails on missing input/output, invalid schema, subprocess errors, invalid normalized output JSON shape, invalid report JSON.
|
||||
- Trim fails on missing input/output/keep selector, subprocess errors, invalid trimmed output JSON shape.
|
||||
- Normalize fails on missing input/output, invalid schema, subprocess errors, invalid final output JSON shape, invalid report JSON.
|
||||
- Trim fails on missing input/output/keep selector, subprocess errors, invalid final-trimmed output JSON shape.
|
||||
|
||||
## Tests to Inspect Before Changing
|
||||
- `internal/adapters/seriatim/subprocess_test.go`
|
||||
@@ -56,5 +56,5 @@ Via `pipeline.seriatim.*` mapped in app/stage wiring:
|
||||
|
||||
## Architectural Invariants
|
||||
- Supported output schemas are limited to `seriatim-minimal`, `seriatim-intermediate`, `seriatim-full`.
|
||||
- Normalize/trim outputs must include `segments` arrays.
|
||||
- Final and final-trimmed outputs must include `segments` arrays.
|
||||
- Merge/normalize/trim all route through deterministic subprocess invocation.
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
Developers and LLM coding agents changing Narratio internals.
|
||||
|
||||
## Scope
|
||||
Implementation-accurate contracts for workspace/state, manifests, stages, artifact resolution, and adapter boundaries.
|
||||
Implementation-accurate contracts for workspace/state, manifests, stages, artifact resolution, adapter boundaries, and restore command behavior.
|
||||
|
||||
## Component Docs
|
||||
- `adapters.md`: external adapter map, runtime wiring, and boundary ownership.
|
||||
@@ -12,6 +12,7 @@ Implementation-accurate contracts for workspace/state, manifests, stages, artifa
|
||||
- `manifest.md`: session/run manifest schemas, lifecycle transitions, and persistence semantics.
|
||||
- `artifacts.md`: built-in artifact registry, runtime artifact catalog, and source-resolution behavior.
|
||||
- `workspace.md`: local state model, manifests, run-local layout, promotion, and cleanup invariants.
|
||||
- `command-restore.md`: restore command discovery/planning/execution/reporting contract.
|
||||
- `stage-prepare.md`: input materialization and provenance capture.
|
||||
- `stage-transcribe.md`: WhisperX transcript generation.
|
||||
- `stage-merge.md`: Seriatim normalization + merge.
|
||||
|
||||
@@ -39,14 +39,14 @@ Runtime env boundary fields (`internal/stage.Env`):
|
||||
- `scriptorium.Runner`
|
||||
- `storage.ObjectStore`
|
||||
- `notify.Sender`
|
||||
- `analyzer.Runner`
|
||||
|
||||
Current execution usage:
|
||||
- Actively used by implemented stages: `WhisperX`, `Seriatim`, `Audita`, `Scriptorium`, `ObjectStore`, `Notifier`.
|
||||
- Present but not used by implemented stage set: `Analyzer`, legacy `storage.Backend`.
|
||||
- Present but not used by implemented stage set: legacy `storage.Backend`.
|
||||
|
||||
Default construction in app runner:
|
||||
- Auto-constructed when not injected: WhisperX HTTP client, Seriatim subprocess runner, Audita subprocess runner, Scriptorium subprocess runner, object store (only when needed), and `notify.NoopSender`.
|
||||
- Object-store construction goes through app command orchestration so configured filesystem secrets are loaded before the storage adapter is initialized.
|
||||
- Callers can inject test/fake implementations through `app.RunOptions.Env`.
|
||||
|
||||
## State and manifest behavior
|
||||
@@ -70,7 +70,6 @@ Default construction in app runner:
|
||||
- `internal/adapters/scriptorium/subprocess_test.go`
|
||||
- `internal/adapters/storage/*_test.go`
|
||||
- `internal/adapters/notify/fake_test.go`
|
||||
- `internal/adapters/analyzer/fake_test.go`
|
||||
- `internal/app/runner_test.go`
|
||||
|
||||
## Architectural invariants
|
||||
|
||||
@@ -1,87 +1,106 @@
|
||||
# Internal: Artifacts
|
||||
|
||||
## Purpose
|
||||
Define Narratio's artifact identity and resolution model for built-in transcript/bounds artifacts and runtime-configured analyze artifacts.
|
||||
Define Narratio artifact identity, catalog, and source-resolution behavior for:
|
||||
- built-in session artifacts;
|
||||
- configured analyze artifacts;
|
||||
- canonical previous-session artifact sources.
|
||||
|
||||
## Inputs and outputs
|
||||
Inputs:
|
||||
- artifact sources from config/runtime (`pipeline.scriptorium.artifacts.*.inputs.*.source`)
|
||||
- session paths and optional session manifest stage outputs
|
||||
- runtime artifact catalog state for configured artifact sources
|
||||
- configured input sources (`pipeline.scriptorium.artifacts.*.inputs.*.source`);
|
||||
- session paths and manifest inputs/outputs;
|
||||
- runtime catalog state.
|
||||
|
||||
Outputs:
|
||||
- resolved local artifact path and provenance (`ResolvedSessionArtifact`)
|
||||
- runtime catalog entries for planned/executable/available artifacts
|
||||
- validation errors for unsupported, missing, or invalid artifact sources
|
||||
- resolved artifact path + provenance (`ResolvedSessionArtifact`);
|
||||
- runtime catalog entries for built-ins and configured artifacts;
|
||||
- requirement sets for canonical previous-session inputs.
|
||||
- canonical S3 session, run, current, session config, session locks, audio, and promoted artifact keys.
|
||||
|
||||
## Boundaries
|
||||
Owns:
|
||||
- built-in artifact registry and content validation rules
|
||||
- runtime artifact catalog for configured artifact source IDs
|
||||
- source resolution behavior for built-in and configured artifact sources
|
||||
- built-in source registry and validation;
|
||||
- configured artifact catalog identity (`narratio.artifact.<name>`);
|
||||
- canonical previous-session source parsing and resolution;
|
||||
- previous-session requirement collection (`CollectPreviousArtifactRequirements`).
|
||||
|
||||
Does not own:
|
||||
- artifact generation (stages produce files)
|
||||
- manifest transition policy
|
||||
- archive promotion behavior
|
||||
|
||||
## Config fields used
|
||||
- `pipeline.scriptorium.artifacts.<name>.enabled`
|
||||
- `pipeline.scriptorium.artifacts.<name>.output_path`
|
||||
- `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source`
|
||||
|
||||
## External adapters used
|
||||
- none
|
||||
|
||||
## State and manifest behavior
|
||||
Built-in registry entries:
|
||||
- prepare-stage remote hydration;
|
||||
- stage success/skip transitions;
|
||||
- archive upload orchestration.
|
||||
|
||||
## Built-in IDs
|
||||
| Artifact ID | Canonical file | Producer stage | Output kind |
|
||||
| --- | --- | --- | --- |
|
||||
| `narratio.transcript.merged` | `transcripts/merged.json` | `merge` | `transcript_merged` |
|
||||
| `narratio.transcript.polished` | `transcripts/processed.json` | `polish` | `transcript_processed` |
|
||||
| `narratio.transcript.full` | `transcripts/normalized.json` | `normalize` | `transcript_normalized` |
|
||||
| `narratio.transcript.trimmed` | `transcripts/trimmed.json` | `trim` | `transcript_trimmed` |
|
||||
| `narratio.transcript.base` | `transcripts/base.json` | `merge` | `transcript_base` |
|
||||
| `narratio.transcript.polished` | `transcripts/polished.json` | `polish` | `transcript_polished` |
|
||||
| `narratio.transcript.final` | `transcripts/final.json` | `normalize` | `transcript_final` |
|
||||
| `narratio.transcript.final_trimmed` | `transcripts/final.trimmed.json` | `trim` | `transcript_final_trimmed` |
|
||||
| `narratio.bounds.session` | `artifacts/session_bounds.json` | `trim` | `session_bounds` |
|
||||
|
||||
Runtime catalog entries include built-ins and configured `narratio.artifact.<name>` sources.
|
||||
## Source families
|
||||
- built-in: `narratio.transcript.*`, `narratio.bounds.session`
|
||||
- configured artifact: `narratio.artifact.<artifact_key>`
|
||||
- canonical previous-session artifact: `narratio.previous_session.artifact.<artifact_key>`
|
||||
|
||||
Catalog states:
|
||||
- `planned`: source is registered and known for this run
|
||||
- `executable`: configured artifact is selected for analyze execution
|
||||
- `available`: artifact has a usable file path (generated this run or reused from disk)
|
||||
## S3 key helpers
|
||||
- session prefix: `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/`
|
||||
- session config: `{session_prefix}/session.yml`
|
||||
- session lock store: `{session_prefix}/locks.yml`
|
||||
- run prefix: `{session_prefix}/runs/{run_id}/`
|
||||
- audio prefix: `{session_prefix}/{session.inputs.audio_s3.prefix}`
|
||||
- current manifest: `{session_prefix}/current/manifest.json`
|
||||
- current run pointer: `{session_prefix}/current/run_id.txt`
|
||||
|
||||
Resolution behavior:
|
||||
- built-in sources resolve via manifest producer outputs first, then canonical fallback path
|
||||
- configured `narratio.artifact.<name>` sources resolve through runtime catalog availability
|
||||
- configured source lookup requires catalog context
|
||||
## Runtime catalog model
|
||||
Catalog entries track:
|
||||
- `planned`: source is registered for this run;
|
||||
- `executable`: configured artifact is selected for analyze execution;
|
||||
- `available`: usable local file exists (generated this run or reused from disk).
|
||||
|
||||
Configured artifact provenance values:
|
||||
Configured artifact provenance values include:
|
||||
- `generated.current_analyze_run`
|
||||
- `filesystem.disabled_artifact_output`
|
||||
|
||||
Content validation:
|
||||
- transcript built-ins: JSON with top-level `segments` array
|
||||
- bounds built-in: valid JSON
|
||||
- configured artifacts: non-empty text file
|
||||
Previous-session canonical provenance values include:
|
||||
- `manifest.inputs.previous_cache`
|
||||
- `current_session.previous_cache`
|
||||
|
||||
## Skip and resume behavior
|
||||
- resolver and catalog have no direct skip/resume decisions
|
||||
- stage/runner skip-resume behavior consumes catalog/resolver results
|
||||
## Resolution behavior
|
||||
- Built-ins resolve via manifest producer outputs first, then canonical fallback paths.
|
||||
- Configured `narratio.artifact.<name>` sources resolve through catalog availability.
|
||||
- Canonical previous-session sources resolve to current-session `previous/` cache candidates derived from configured artifact canonical output paths.
|
||||
- Archive-relative configured artifact paths under `artifacts/` are cached without a redundant nested `artifacts/` segment.
|
||||
- Previous-session canonical resolution prefers manifest-recorded input paths when present, then filesystem fallback under `previous/artifacts/**`.
|
||||
|
||||
## Previous-session requirement scanning
|
||||
`CollectPreviousArtifactRequirements`:
|
||||
- scans enabled configured artifacts only;
|
||||
- includes canonical previous-session sources only;
|
||||
- deduplicates by artifact key;
|
||||
- merges required/optional references (`required` wins);
|
||||
- records deterministic sorted source locations for diagnostics.
|
||||
|
||||
## Validation behavior
|
||||
- transcript built-ins: JSON with top-level `segments` array;
|
||||
- bounds built-in: valid JSON;
|
||||
- configured and previous-session artifact files: non-empty text content.
|
||||
|
||||
## Failure behavior
|
||||
- unsupported source -> source validation error
|
||||
- known source unavailable -> `ErrSessionArtifactNotFound`
|
||||
- configured source without catalog -> resolution error
|
||||
- resolved file with invalid content -> validation error
|
||||
- unsupported source or malformed canonical previous source: validation/resolution error;
|
||||
- known source unavailable: `ErrSessionArtifactNotFound`;
|
||||
- configured/previous canonical source without catalog: error;
|
||||
- resolved invalid file content: validation error.
|
||||
|
||||
## Tests to inspect before changing
|
||||
- `internal/artifacts/artifact_resolver_test.go`
|
||||
- `internal/artifacts/catalog_test.go`
|
||||
- `internal/artifacts/previous_requirements_test.go`
|
||||
- `internal/stage/prepare_previous_test.go`
|
||||
- `internal/stage/analyze_test.go`
|
||||
- `internal/config/scriptorium_test.go`
|
||||
|
||||
## Architectural invariants
|
||||
- built-in IDs are static and registry-backed
|
||||
- configured artifact IDs are runtime-derived (`narratio.artifact.<name>`) and catalog-backed
|
||||
- built-in/source resolution remains deterministic and validation-gated
|
||||
- Built-in source IDs are static.
|
||||
- Configured and previous-session source IDs are artifact-key based and validation-gated.
|
||||
- Resolution behavior remains deterministic and manifest-aware.
|
||||
|
||||
106
docs/internal/command-restore.md
Normal file
106
docs/internal/command-restore.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# Internal: Command Restore
|
||||
|
||||
## Purpose
|
||||
Define the implemented `narratio session restore` contract: committed remote-state discovery, deterministic plan classification, safe file install semantics, and restore reporting.
|
||||
|
||||
## Inputs and outputs
|
||||
Inputs:
|
||||
- CLI syntax: `narratio session restore <session_id>`.
|
||||
- CLI flags: `--config`, `--campaign`, `--campaign-file`, `--session`, `--previous-session-id`, `--dry-run`, `--force`, `--include-audio`.
|
||||
- Resolved/validated `pipeline.yml` and `session.yml`.
|
||||
- Configured remote object store.
|
||||
- Remote committed current-state markers (`current/run_id.txt`, `current/manifest.json`).
|
||||
|
||||
Outputs:
|
||||
- Dry-run summary to stdout (plan + counts).
|
||||
- Non-dry-run completion summary to stdout.
|
||||
- Local durable session files restored under canonical session root.
|
||||
- Non-dry-run restore report at `reports/restore-latest.json`.
|
||||
|
||||
## Boundaries
|
||||
Owns:
|
||||
- Restore command flag parsing and command wiring.
|
||||
- Remote current-state discovery and identity validation.
|
||||
- Restore plan construction and conflict classification.
|
||||
- Restore execution for planned downloads.
|
||||
- Restore report model and persistence.
|
||||
|
||||
Does not own:
|
||||
- Stage execution orchestration (`run`, `resume`, `run-stage`).
|
||||
- Archive publish behavior (owned by archive stage).
|
||||
- Storage transport implementation details (owned by storage adapters).
|
||||
|
||||
## Config fields used
|
||||
- Config/session discovery and templating fields consumed by all commands.
|
||||
- `pipeline.workspace.root` (local restore target root).
|
||||
- `pipeline.storage.*` (remote backend + archive identity derivation).
|
||||
- `pipeline.storage.s3.*` identity components used by archive prefix helpers.
|
||||
- `pipeline.spool.root` for active audio downloads.
|
||||
- `pipeline.cache.root` and `pipeline.cache.s3_audio` for reusable S3 audio cache.
|
||||
- `session.session_id`
|
||||
- `session.campaign`
|
||||
|
||||
## External adapters used
|
||||
- `storage.ObjectStore` for `Exists`, `List`, `Download`.
|
||||
- `artifacts.Store` (`LocalStore`) for layout and session lock management.
|
||||
- `manifest.LocalStore` for manifest decode/validation and identity checks.
|
||||
|
||||
## State and manifest behavior
|
||||
- Restore is not a pipeline run and does not create a run manifest.
|
||||
- Restore uses committed remote current state only:
|
||||
- `current/run_id.txt` must exist and be non-empty.
|
||||
- `current/manifest.json` must decode and match requested session/campaign.
|
||||
- Non-dry-run writes restore files to canonical session paths.
|
||||
- With `--include-audio`, restore uses the shared S3 audio cache for `audio/**` objects. Cache hits avoid object downloads; cache misses download through spool, install the work file, and populate cache.
|
||||
- Manifest install behavior:
|
||||
- validated before replacement.
|
||||
- installed last among download actions.
|
||||
- existing local manifest is preserved if restored manifest validation/install fails.
|
||||
- Non-dry-run report persists summary/action status metadata in `reports/restore-latest.json`.
|
||||
|
||||
Restore path scope:
|
||||
- includes:
|
||||
- `manifest.json`
|
||||
- `transcripts/**`
|
||||
- `artifacts/**`
|
||||
- `previous/**`
|
||||
- `audio/**` only when `--include-audio` is set
|
||||
- excludes:
|
||||
- `runs/**`
|
||||
- `logs/**`
|
||||
- `reports/**`
|
||||
- `config/**`
|
||||
- `inputs/**`
|
||||
- remote `current/**` pointer files as local restore targets
|
||||
|
||||
## Skip and resume behavior
|
||||
- Restore does not participate in stage skip/resume decisions.
|
||||
- Restore provides durable local state so subsequent stage commands can resume or rerun based on restored manifest state.
|
||||
- Audio cache is outside the workspace and is reused across restore and prepare invocations.
|
||||
- Dry-run is read-only and returns plan output only.
|
||||
|
||||
## Failure behavior
|
||||
- Fails when storage backend is unavailable or archive identity cannot be resolved.
|
||||
- Fails when remote current pointer/manifest is missing or invalid.
|
||||
- Fails when remote manifest identity mismatches requested campaign/session.
|
||||
- Fails on local conflicts unless `--force` is set.
|
||||
- Fails fast on session lock acquisition conflict for non-dry-run execution.
|
||||
- On execution failure, previously installed files remain; no rollback is performed.
|
||||
|
||||
## Tests to inspect before changing
|
||||
- `internal/app/restore_test.go`
|
||||
- `internal/app/restore_discovery_test.go`
|
||||
- `internal/app/restore_plan_test.go`
|
||||
- `internal/app/restore_execution_test.go`
|
||||
- `internal/app/restore_workflow_test.go`
|
||||
- `internal/artifacts/archive_identity_test.go`
|
||||
|
||||
## Architectural invariants
|
||||
- Restore relies on centralized archive identity/key helpers (`internal/artifacts`) rather than ad hoc key building.
|
||||
- `current/run_id.txt` is the remote commit marker; restore must not infer committed state from incidental files.
|
||||
- Local path mapping is traversal-safe and constrained to session root.
|
||||
- Restore scope is deterministic and path-classified:
|
||||
- include `manifest.json`, `transcripts/**`, `artifacts/**`, `previous/**`
|
||||
- include `audio/**` only with `--include-audio`
|
||||
- exclude `runs/**`, `logs/**`, `reports/**`, `config/**`, `inputs/**`
|
||||
- Command remains standalone; no implicit `run --restore` behavior.
|
||||
@@ -3,31 +3,35 @@
|
||||
## Purpose
|
||||
Execute selected configured Scriptorium artifacts in deterministic dependency order and promote successful outputs to canonical session artifact paths.
|
||||
|
||||
## Inputs and Outputs
|
||||
## Inputs and outputs
|
||||
Inputs:
|
||||
- configured artifact definitions from `pipeline.scriptorium.artifacts`
|
||||
- selected artifact filter from runtime (`--artifacts`) when provided
|
||||
- resolved artifact input sources declared per artifact (`inputs.*.source`)
|
||||
- optional previous-session file inputs (`previous_session_artifact`)
|
||||
- configured artifact definitions from `pipeline.scriptorium.artifacts`;
|
||||
- selected artifact filter (`--artifacts`) when provided;
|
||||
- resolved artifact sources from resolver/catalog.
|
||||
|
||||
Source types used by analyze:
|
||||
- built-ins: `narratio.transcript.*`, `narratio.bounds.session`;
|
||||
- configured artifacts: `narratio.artifact.<artifact_key>`;
|
||||
- canonical previous-session artifacts: `narratio.previous_session.artifact.<artifact_key>`.
|
||||
|
||||
Outputs:
|
||||
- one promoted output file per executed configured artifact at that artifact's configured `output_path`
|
||||
- stage metadata containing generated artifact entries and reused disabled-artifact entries
|
||||
- promoted configured artifact files at each configured `output_path`;
|
||||
- stage metadata (`generated_artifacts`, `reused_artifacts`, selected/order info).
|
||||
|
||||
## Boundaries
|
||||
Owns:
|
||||
- runtime artifact catalog construction for analyze execution
|
||||
- selected-artifact planning and dependency ordering
|
||||
- per-artifact input resolution, var resolution, timeout/render-debug resolution
|
||||
- Scriptorium run/render invocation for each selected artifact
|
||||
- run-local output generation and canonical promotion
|
||||
- runtime artifact catalog construction;
|
||||
- selected-artifact planning and dependency ordering;
|
||||
- per-input resolution and required/optional handling;
|
||||
- Scriptorium render/run invocation;
|
||||
- run-local output generation and canonical promotion.
|
||||
|
||||
Does not own:
|
||||
- transcript generation/processing stages
|
||||
- archive promotion policy
|
||||
- per-artifact resume semantics
|
||||
- prepare-time previous-session hydration;
|
||||
- object-store access for previous-session sources;
|
||||
- archive promotion policy.
|
||||
|
||||
## Config Fields Used
|
||||
## Config fields used
|
||||
- `session.session_id`
|
||||
- `session.campaign`
|
||||
- `pipeline.workspace.root`
|
||||
@@ -36,49 +40,41 @@ Does not own:
|
||||
- `pipeline.scriptorium.timeout`
|
||||
- `pipeline.scriptorium.render_debug`
|
||||
- `pipeline.scriptorium.artifacts.<name>.*`
|
||||
- `enabled`
|
||||
- `depends_on`
|
||||
- `prompt_id`
|
||||
- `profile_id`
|
||||
- `timeout`
|
||||
- `output_path`
|
||||
- `render_debug`
|
||||
- `inputs`
|
||||
- `vars`
|
||||
|
||||
## External Adapters Used
|
||||
## External adapters used
|
||||
- Scriptorium adapter:
|
||||
- optional `RenderArtifact` (render debug)
|
||||
- `RunArtifact` (artifact generation)
|
||||
- optional `RenderArtifact` when render-debug is enabled;
|
||||
- `RunArtifact` for artifact generation.
|
||||
|
||||
## State and Manifest Behavior
|
||||
- If `pipeline.scriptorium` is absent, stage returns success metadata with `skipped=true`.
|
||||
- If no artifacts are configured, stage returns success metadata with `skipped=true`.
|
||||
- If zero artifacts are executable after `enabled` + `--artifacts` filtering, stage returns success metadata with `skipped=true`.
|
||||
- Builds runtime catalog with built-ins and configured artifacts.
|
||||
- Non-executable configured artifacts are marked available only when their configured output file exists and is valid on disk.
|
||||
- Executes selected configured artifacts in topological order with deterministic tie-breaking.
|
||||
- For each generated artifact, records metadata fields including `name`, `source_id`, `output_kind`, `path`, `prompt_id`, `profile_id`, and `provenance`.
|
||||
- Reused disabled artifacts are recorded separately in `reused_artifacts` with provenance `filesystem.disabled_artifact_output`.
|
||||
## State and manifest behavior
|
||||
- If Scriptorium config is absent, or no artifacts are executable after filtering, analyze returns success metadata with `skipped=true`.
|
||||
- Builds runtime catalog with built-ins and configured `narratio.artifact.<name>` entries.
|
||||
- Non-executable configured artifacts may still be marked available from existing canonical output files.
|
||||
- Resolves canonical previous-session sources from local prepared `previous/` cache:
|
||||
- prefers manifest-backed previous input paths when present;
|
||||
- may fall back to current-session `previous/` filesystem paths.
|
||||
- Analyze does not call object storage for canonical previous-session source resolution.
|
||||
- Required canonical previous-session input missing:
|
||||
- fails with guidance to run `narratio run-stage --force prepare`.
|
||||
- Optional missing sources are omitted from adapter input paths.
|
||||
|
||||
## Skip and Resume Behavior
|
||||
## Skip and resume behavior
|
||||
- Runner-level skip applies when analyze is already `succeeded` and `--force` is not set.
|
||||
- Analyze remains stage-scoped for resume/skip; there is no per-artifact resume state.
|
||||
- `--artifacts` filters which configured artifacts are executable when analyze runs; it does not imply `--force`.
|
||||
- Analyze is stage-scoped for resume; no per-artifact manifest resume state.
|
||||
- `--artifacts` filters executable artifacts but does not imply force rerun.
|
||||
|
||||
## Failure Behavior
|
||||
- Fails on invalid dependency ordering, unavailable required configured inputs, invalid built-in input prerequisites, render/run adapter failures, validation-failed adapter results, or missing/empty outputs.
|
||||
- Required configured dependency missing from catalog availability fails clearly before invocation.
|
||||
- Optional missing inputs are omitted.
|
||||
## Failure behavior
|
||||
- Fails on dependency-order violations, missing required inputs, resolver validation failures, adapter errors, and missing/empty generated outputs.
|
||||
- Required unavailable configured artifact source (`narratio.artifact.<name>`) fails before invocation.
|
||||
- Required canonical previous-session source fails with prepare-rerun guidance.
|
||||
|
||||
## Tests to Inspect Before Changing
|
||||
## Tests to inspect before changing
|
||||
- `internal/stage/analyze_test.go`
|
||||
- `internal/artifacts/catalog_test.go`
|
||||
- `internal/artifacts/artifact_resolver_test.go`
|
||||
- `internal/adapters/scriptorium/subprocess_test.go`
|
||||
- `internal/app/restore_workflow_test.go`
|
||||
|
||||
## Architectural Invariants
|
||||
- Configured artifacts are identified by `narratio.artifact.<name>` source IDs.
|
||||
- Artifact-to-artifact references rely on explicit `depends_on` declarations validated in config.
|
||||
- Generated analyze outputs are treated uniformly as Scriptorium artifacts.
|
||||
- Successful outputs must exist and be non-empty before promotion.
|
||||
## Architectural invariants
|
||||
- Canonical previous-session behavior is local-cache only during analyze.
|
||||
- Generated outputs are validated and promoted before stage success is recorded.
|
||||
- Resolver/catalog decisions stay deterministic and validation-gated.
|
||||
|
||||
@@ -1,17 +1,20 @@
|
||||
# Stage: archive
|
||||
|
||||
## Purpose
|
||||
Publish run records and promoted session artifacts to object storage, then atomically advance the remote current pointer.
|
||||
Publish durable run/session state to object storage, then atomically advance remote current state.
|
||||
|
||||
## Inputs and Outputs
|
||||
Inputs:
|
||||
- session manifest and prerequisite stage records
|
||||
- run root contents under `runs/{run_id}/`
|
||||
- promotion sources from session root (`archive.promote_artifacts`)
|
||||
- promotion rules with artifact `source` IDs and archive `dest` paths (`archive.promote_artifacts`)
|
||||
- effective source-based promotion locks from static config and remote session lock store
|
||||
- session-level `previous/**` cache files when present
|
||||
|
||||
Outputs:
|
||||
- uploaded run files under `{session_prefix}/runs/{run_id}/...`
|
||||
- uploaded promoted artifacts under `{session_prefix}/...`
|
||||
- uploaded session previous-cache files under `{session_prefix}/previous/...` when present
|
||||
- `{session_prefix}/current/manifest.json`
|
||||
- `{session_prefix}/current/run_id.txt` written last
|
||||
|
||||
@@ -21,6 +24,8 @@ Owns:
|
||||
- Prerequisite stage success enforcement
|
||||
- Run file collection and upload (excluding `audio/`)
|
||||
- Promotion rule resolution and upload
|
||||
- Promotion lock enforcement
|
||||
- Session previous-cache file collection/upload
|
||||
- Commit pointer publish order
|
||||
|
||||
Does not own:
|
||||
@@ -31,6 +36,8 @@ Does not own:
|
||||
- `pipeline.archive.enabled`
|
||||
- `pipeline.archive.upload_run`
|
||||
- `pipeline.archive.promote_artifacts`
|
||||
- `pipeline.archive.locks`
|
||||
- `{session_prefix}/locks.yml` loaded by app orchestration before archive execution
|
||||
- `pipeline.storage.s3.bucket`
|
||||
- `pipeline.storage.s3.root_prefix`
|
||||
- `pipeline.workspace.root`
|
||||
@@ -43,8 +50,15 @@ Does not own:
|
||||
## State and Manifest Behavior
|
||||
- Requires `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, and `analyze` status `succeeded`.
|
||||
- Resolves bucket/prefix from manifest identity first, then config fallback.
|
||||
- Uploads session `previous/**` files as durable session state when the local `previous/` directory exists.
|
||||
- Skips top-level promotion uploads for effective locked sources; run-local uploads still publish.
|
||||
- When selected configured artifact keys are supplied, skips promotion rules for unselected `narratio.artifact.<key>` sources; built-in transcript and bounds promotions still publish.
|
||||
- Effective locks are the union of `pipeline.archive.locks` and remote `{session_prefix}/locks.yml`; static pipeline locks win on duplicate sources.
|
||||
- Writes metadata including:
|
||||
- upload counts/paths
|
||||
- `previous_files_uploaded` and `previous_uploaded_paths`
|
||||
- `skipped_unselected_promotions`
|
||||
- `locked_promotion_count` and `locked_promotions`
|
||||
- `current_manifest_key`
|
||||
- `current_run_id_key`
|
||||
- `current_pointer_written`
|
||||
@@ -55,7 +69,8 @@ Does not own:
|
||||
- Runner-level skip also applies for previously succeeded stage unless forced.
|
||||
|
||||
## Failure Behavior
|
||||
- Fails on missing prerequisite success, missing object store when required, missing run root, missing required promotion source, upload failures, or pointer write failures.
|
||||
- Fails on missing prerequisite success, missing object store when required, missing run root, missing unlocked required promotion source, upload failures, or pointer write failures.
|
||||
- Locked required promotions are intentional skips and do not fail archive.
|
||||
- Pointer semantics are fail-safe: `current/run_id.txt` is not written if prior required uploads fail.
|
||||
|
||||
## Tests to Inspect Before Changing
|
||||
@@ -64,5 +79,8 @@ Does not own:
|
||||
|
||||
## Architectural Invariants
|
||||
- Run upload excludes `audio/` subtree.
|
||||
- Session `previous/**` is archiveable durable input/provenance state, not run-local output.
|
||||
- Ordinary `--force` does not override archive locks.
|
||||
- Malformed or unreadable remote lock store fails archive-capable execution before promotion.
|
||||
- `current/manifest.json` uploads before `current/run_id.txt`.
|
||||
- `current/run_id.txt` is the remote publish commit marker.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Stage: merge
|
||||
|
||||
## Purpose
|
||||
Normalize per-speaker raw transcripts and merge them into one merged transcript via Seriatim.
|
||||
Normalize per-speaker raw transcripts and merge them into the base transcript via Seriatim.
|
||||
|
||||
## Inputs and Outputs
|
||||
Inputs:
|
||||
@@ -10,7 +10,7 @@ Inputs:
|
||||
- `inputs/autocorrect.yml`
|
||||
|
||||
Outputs:
|
||||
- `transcripts/merged.json`
|
||||
- `transcripts/base.json`
|
||||
- optional `artifacts/seriatim.report.json` (when report enabled)
|
||||
|
||||
## Boundaries
|
||||
@@ -19,7 +19,7 @@ Owns:
|
||||
- Per-input normalize calls to Seriatim
|
||||
- Final merge call to Seriatim
|
||||
- Run-local log/config/report path wiring
|
||||
- Promotion of merged/report outputs to canonical paths
|
||||
- Promotion of base/report outputs to canonical paths
|
||||
|
||||
Does not own:
|
||||
- Transcript polishing or downstream artifact generation
|
||||
@@ -43,7 +43,7 @@ Does not own:
|
||||
## State and Manifest Behavior
|
||||
- Reads transcript inputs from transcribe stage outputs in manifest when present; falls back to canonical raw directory.
|
||||
- Writes run-local outputs/logs/config under `runs/{run_id}/merge/...` when enabled.
|
||||
- Promotes canonical merged transcript and optional report.
|
||||
- Promotes canonical base transcript and optional report.
|
||||
- Records normalized-input provenance and adapter metadata in stage metadata.
|
||||
|
||||
## Skip and Resume Behavior
|
||||
@@ -51,7 +51,7 @@ Does not own:
|
||||
- Forced rerun of this or upstream stages can stale downstream succeeded stages via runner invalidation.
|
||||
|
||||
## Failure Behavior
|
||||
- Fails on missing/invalid raw transcripts, missing speakers/autocorrect files, normalize failure, merge failure, invalid merged output JSON, or invalid report JSON when enabled.
|
||||
- Fails on missing/invalid raw transcripts, missing speakers/autocorrect files, normalize failure, merge failure, invalid base output JSON, or invalid report JSON when enabled.
|
||||
|
||||
## Tests to Inspect Before Changing
|
||||
- `internal/stage/merge_test.go`
|
||||
@@ -59,5 +59,5 @@ Does not own:
|
||||
|
||||
## Architectural Invariants
|
||||
- Merge consumes normalized forms of each raw transcript.
|
||||
- Merged transcript must validate before promotion.
|
||||
- Base transcript must validate before promotion.
|
||||
- Report output is optional and gated by config.
|
||||
|
||||
@@ -1,22 +1,22 @@
|
||||
# Stage: normalize
|
||||
|
||||
## Purpose
|
||||
Normalize the processed transcript into a deterministic intermediate schema for trim and optionally emit a normalize report.
|
||||
Normalize the polished transcript into the full final transcript and optionally emit a normalize report.
|
||||
|
||||
## Inputs and Outputs
|
||||
Inputs:
|
||||
- `transcripts/processed.json`
|
||||
- `transcripts/polished.json`
|
||||
|
||||
Outputs:
|
||||
- `transcripts/normalized.json` (or configured normalize output path)
|
||||
- `transcripts/final.json` (or configured normalize output path)
|
||||
- optional `artifacts/seriatim.normalize.report.json`
|
||||
|
||||
## Boundaries
|
||||
Owns:
|
||||
- Processed transcript discovery/validation
|
||||
- Polished transcript discovery/validation
|
||||
- Normalize request construction and invocation
|
||||
- Optional normalize report wiring
|
||||
- Promotion of normalized transcript and optional report
|
||||
- Promotion of final transcript and optional report
|
||||
|
||||
Does not own:
|
||||
- Bounds detection or segment trimming
|
||||
@@ -35,9 +35,9 @@ Does not own:
|
||||
- Seriatim adapter (`Normalize`).
|
||||
|
||||
## State and Manifest Behavior
|
||||
- Reads processed transcript from polish outputs in manifest when present; falls back to canonical path.
|
||||
- Reads polished transcript from polish outputs in manifest when present; falls back to canonical path.
|
||||
- Uses run-local output/report/log/config paths when run layout is enabled.
|
||||
- Promotes canonical normalized transcript and optional normalize report.
|
||||
- Promotes canonical final transcript and optional normalize report.
|
||||
- Records adapter/result metadata including source path selection.
|
||||
|
||||
## Skip and Resume Behavior
|
||||
@@ -45,12 +45,12 @@ Does not own:
|
||||
- Forced reruns can stale downstream succeeded stages.
|
||||
|
||||
## Failure Behavior
|
||||
- Fails on missing/invalid processed transcript, adapter error, invalid normalized output, or invalid report output when report enabled.
|
||||
- Fails on missing/invalid polished transcript, adapter error, invalid final output, or invalid report output when report enabled.
|
||||
|
||||
## Tests to Inspect Before Changing
|
||||
- `internal/stage/normalize_test.go`
|
||||
- `internal/adapters/seriatim/subprocess_test.go`
|
||||
|
||||
## Architectural Invariants
|
||||
- Normalized output must validate as processed-transcript-compatible JSON (`segments` array required).
|
||||
- Final output must validate as transcript-compatible JSON (`segments` array required).
|
||||
- Default normalize config is applied when `pipeline.normalize` is unset.
|
||||
|
||||
@@ -1,23 +1,23 @@
|
||||
# Stage: polish
|
||||
|
||||
## Purpose
|
||||
Polish merged transcript with Audita and produce a processed transcript for downstream normalization/analyze.
|
||||
Polish the base transcript with Audita and produce a polished transcript for downstream normalization/analyze.
|
||||
|
||||
## Inputs and Outputs
|
||||
Inputs:
|
||||
- `transcripts/merged.json`
|
||||
- `transcripts/base.json`
|
||||
- `inputs/glossary.yml`
|
||||
|
||||
Outputs:
|
||||
- `transcripts/processed.json`
|
||||
- `transcripts/polished.json`
|
||||
- optional `artifacts/audita.report.json` (when report enabled)
|
||||
|
||||
## Boundaries
|
||||
Owns:
|
||||
- Merged transcript discovery/validation
|
||||
- Base transcript discovery/validation
|
||||
- Audita invocation request construction
|
||||
- Run-local logs/config/work-dir/report wiring
|
||||
- Promotion of processed transcript and optional report
|
||||
- Promotion of polished transcript and optional report
|
||||
|
||||
Does not own:
|
||||
- Upstream merge normalization
|
||||
@@ -47,9 +47,9 @@ Does not own:
|
||||
- Audita adapter (`env.Audita.Run`).
|
||||
|
||||
## State and Manifest Behavior
|
||||
- Reads merged transcript from merge manifest outputs when available; falls back to canonical merged path.
|
||||
- Reads base transcript from merge manifest outputs when available; falls back to canonical base path.
|
||||
- Uses run-local output/report/log/config/scratch paths when run layout is enabled.
|
||||
- Promotes canonical `transcripts/processed.json` and optional report.
|
||||
- Promotes canonical `transcripts/polished.json` and optional report.
|
||||
- Records adapter invocation metadata, credential presence signal, and output provenance in stage metadata.
|
||||
|
||||
## Skip and Resume Behavior
|
||||
@@ -57,13 +57,13 @@ Does not own:
|
||||
- Forced rerun can stale downstream succeeded stages via runner invalidation.
|
||||
|
||||
## Failure Behavior
|
||||
- Fails on missing/invalid merged transcript, missing glossary, adapter error, invalid processed output shape (`segments` array required), or invalid report JSON when enabled.
|
||||
- Fails on missing/invalid base transcript, missing glossary, adapter error, invalid polished output shape (`segments` array required), or invalid report JSON when enabled.
|
||||
|
||||
## Tests to Inspect Before Changing
|
||||
- `internal/stage/polish_test.go`
|
||||
- `internal/adapters/audita/subprocess_test.go`
|
||||
|
||||
## Architectural Invariants
|
||||
- Processed transcript must contain a top-level `segments` array.
|
||||
- Polished transcript must contain a top-level `segments` array.
|
||||
- Report behavior is strictly config-gated.
|
||||
- Stage output canonicalization always ends at `transcripts/processed.json`.
|
||||
- Stage output canonicalization always ends at `transcripts/polished.json`.
|
||||
|
||||
@@ -1,40 +1,51 @@
|
||||
# Stage: prepare
|
||||
|
||||
## Purpose
|
||||
Materialize all required session inputs into canonical local workspace paths and record input provenance in the session manifest.
|
||||
Materialize canonical current-session input state and provenance before downstream stages run.
|
||||
|
||||
## Inputs and Outputs
|
||||
Prepare owns:
|
||||
- local input file materialization (`inputs/**`);
|
||||
- audio input materialization (`audio/**`);
|
||||
- previous-session cache hydration (`previous/**`) for canonical previous-session artifact sources.
|
||||
|
||||
## Inputs and outputs
|
||||
Inputs:
|
||||
- `session.yml` (resolved session config)
|
||||
- `pipeline.resolved.yml` (materialized from resolved pipeline config)
|
||||
- `speakers.yml`
|
||||
- `autocorrect.yml`
|
||||
- `glossary.yml`
|
||||
- resolved config/campaign/session (`pipeline.yml`, `campaign.yml`, `session.yml`);
|
||||
- remote session provenance when `session.yml` was loaded from S3;
|
||||
- campaign or session input files (`speakers`, `autocorrect`, `glossary`);
|
||||
- audio source:
|
||||
- local (`session.inputs.audio_dir` or `session.inputs.audio_files`), or
|
||||
- S3 (`session.inputs.audio_s3.prefix`)
|
||||
- local: `session.inputs.audio_dir` or `session.inputs.audio_files`;
|
||||
- S3: `session.inputs.audio_s3.prefix`;
|
||||
- configured enabled Scriptorium artifact inputs (for previous-session requirement scanning);
|
||||
- remote previous-session current archive state when previous hydration is required.
|
||||
|
||||
Outputs:
|
||||
- `inputs/session.yml`
|
||||
- `inputs/pipeline.resolved.yml`
|
||||
- `inputs/speakers.yml`
|
||||
- `inputs/autocorrect.yml`
|
||||
- `inputs/glossary.yml`
|
||||
- `audio/*.flac` in session workdir
|
||||
- `manifest.Inputs` records with checksums and source metadata
|
||||
- `inputs/campaign.yml`;
|
||||
- `inputs/session.yml`;
|
||||
- `inputs/pipeline.resolved.yml`;
|
||||
- `inputs/speakers.yml`;
|
||||
- `inputs/autocorrect.yml`;
|
||||
- `inputs/glossary.yml`;
|
||||
- `audio/*.flac` in canonical session `audio/`;
|
||||
- optional `previous/manifest.json`;
|
||||
- optional `previous/artifacts/**`;
|
||||
- deterministic `manifest.Inputs` records with checksums and provenance metadata.
|
||||
|
||||
## Boundaries
|
||||
Owns:
|
||||
- Input path resolution and validation
|
||||
- Local copy/materialization of configs and audio files
|
||||
- S3 audio download to run-scoped spool, then copy into work audio dir
|
||||
- input path resolution and materialization;
|
||||
- S3 audio list/download/copy flow;
|
||||
- previous-session artifact requirement collection from enabled configured artifacts;
|
||||
- previous cache lifecycle when requirements exist (clear and rehydrate managed `previous/` state).
|
||||
|
||||
Does not own:
|
||||
- Transcript generation/processing
|
||||
- Archive publish behavior
|
||||
- transcript or artifact generation;
|
||||
- analyze-stage source resolution;
|
||||
- archive commit behavior.
|
||||
|
||||
## Config Fields Used
|
||||
## Config fields used
|
||||
- `session.session_id`
|
||||
- `session.previous_session_id`
|
||||
- `session.campaign`
|
||||
- `session.inputs.speakers_file`
|
||||
- `session.inputs.autocorrect_file`
|
||||
@@ -44,31 +55,70 @@ Does not own:
|
||||
- `session.inputs.audio_s3.prefix`
|
||||
- `pipeline.workspace.root`
|
||||
- `pipeline.spool.root`
|
||||
- `pipeline.cache.root`
|
||||
- `pipeline.cache.s3_audio`
|
||||
- `pipeline.storage.s3.bucket`
|
||||
- `pipeline.storage.s3.root_prefix`
|
||||
- `pipeline.scriptorium.artifacts.<name>.enabled`
|
||||
- `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source`
|
||||
- `pipeline.scriptorium.artifacts.<name>.inputs.<key>.required`
|
||||
- `campaign.campaign_id`
|
||||
- `campaign.inputs.speakers_file`
|
||||
- `campaign.inputs.autocorrect_file`
|
||||
- `campaign.inputs.glossary_file`
|
||||
|
||||
## External Adapters Used
|
||||
- Object storage backend (`env.ObjectStore`) for S3 audio list/download when `audio_s3` is configured.
|
||||
## External adapters used
|
||||
- `storage.ObjectStore` for:
|
||||
- S3 audio listing/downloads;
|
||||
- previous-session current pointer/manifest/artifact object checks and downloads.
|
||||
|
||||
## State and Manifest Behavior
|
||||
## State and manifest behavior
|
||||
- Ensures workspace layout exists.
|
||||
- Writes resolved config and input files to canonical `inputs/` paths.
|
||||
- Records all prepared inputs into `manifest.Inputs` (sorted deterministically by kind/path).
|
||||
- For S3 audio, records `S3Bucket`, `S3Key`, `S3Size`, `S3ETag`, and `SpoolPath` in each audio input record.
|
||||
- Materializes canonical input files and audio files.
|
||||
- For S3 audio, uses run-scoped spool for active downloads and durable cache for reusable audio files; cache hits copy directly to work audio without downloading the object again.
|
||||
- Records `inputs/session.yml` provenance as local `session_config` or remote `session_config.s3`.
|
||||
- Resolves campaign-provided stable input paths relative to `campaign.yml`.
|
||||
- Resolves session-provided stable input overrides relative to `session.yml`.
|
||||
- Scans enabled configured artifact inputs for canonical sources:
|
||||
- `narratio.previous_session.artifact.<artifact_key>`
|
||||
- If one or more canonical previous-session requirements exist:
|
||||
- clears managed `previous/` state;
|
||||
- hydrates required/optional previous artifacts from the configured previous session’s committed archive current state;
|
||||
- writes `previous/manifest.json` and hydrated `previous/artifacts/**`;
|
||||
- stores archive-relative artifact paths such as `artifacts/session_recap.md` as `previous/artifacts/session_recap.md`, not `previous/artifacts/artifacts/session_recap.md`;
|
||||
- records hydrated previous inputs in `manifest.Inputs` with source `previous_session_archive.current`.
|
||||
- If no canonical previous-session requirements exist, prepare does not manage `previous/`.
|
||||
- `manifest.Inputs` is sorted deterministically by `(kind, path)`.
|
||||
- S3 audio `manifest.Inputs` retain S3 provenance and include `cache_path`; `spool_path` is present only when the current prepare invocation downloaded the file.
|
||||
|
||||
## Skip and Resume Behavior
|
||||
- Runner-level skip applies when stage already `succeeded` and `--force` is not set.
|
||||
- Stage itself is deterministic/idempotent for unchanged inputs (`copyFileIfChanged`, `writeBytesIfChanged`).
|
||||
## Required and optional previous-session behavior
|
||||
- `previous_session_id` unset:
|
||||
- if any referenced previous artifact is required: fail;
|
||||
- if all referenced previous artifacts are optional: continue and omit them.
|
||||
- Previous session archive current pointer or manifest missing:
|
||||
- if any referenced previous artifact is required: fail;
|
||||
- if all referenced previous artifacts are optional: continue and omit missing ones.
|
||||
- Missing required previous artifact object: fail.
|
||||
- Missing optional previous artifact object: omit.
|
||||
- Downloaded previous artifacts must validate as non-empty files.
|
||||
|
||||
## Failure Behavior
|
||||
- Fails on missing required files, invalid audio source combinations, no discoverable `.flac` files, duplicate audio basenames, missing object store for S3 mode, or S3 list/download failures.
|
||||
## Skip and resume behavior
|
||||
- Runner-level skip remains authoritative:
|
||||
- if `prepare` already succeeded and run is not forced, `prepare` does not run and no hydration/download occurs.
|
||||
- If `prepare` runs (including with `--force`), it owns managed `previous/` state for canonical previous-session inputs.
|
||||
|
||||
## Tests to Inspect Before Changing
|
||||
## Failure behavior
|
||||
- Fails on missing required input files, invalid audio-source combinations, empty/duplicate audio inputs, missing object store for S3 modes, and remote access/download/validation errors.
|
||||
- For required canonical previous-session inputs, analyze-time missing-input guidance is to rerun:
|
||||
- `narratio run-stage --force prepare`
|
||||
|
||||
## Tests to inspect before changing
|
||||
- `internal/stage/prepare_test.go`
|
||||
- `internal/app/session_cli_test.go`
|
||||
- `internal/config/load_validate_test.go`
|
||||
- `internal/stage/prepare_previous_test.go`
|
||||
- `internal/artifacts/previous_requirements_test.go`
|
||||
- `internal/app/runner_test.go`
|
||||
|
||||
## Architectural Invariants
|
||||
## Architectural invariants
|
||||
- `audio_dir`/`audio_files` and `audio_s3` are mutually exclusive.
|
||||
- Audio files must be `.flac`.
|
||||
- Canonical `inputs/*` and `audio/*` paths are the durable source for downstream stages.
|
||||
- Storage keys are computed by callers using archive/path helpers; storage adapter receives explicit keys.
|
||||
- `prepare` is the only stage that hydrates canonical previous-session cache state.
|
||||
|
||||
@@ -1,21 +1,21 @@
|
||||
# Stage: trim
|
||||
|
||||
## Purpose
|
||||
Optionally trim the normalized transcript to session bounds; always produce a durable trimmed transcript.
|
||||
Optionally trim the final transcript to session bounds; always produce a durable final-trimmed transcript.
|
||||
|
||||
## Inputs and Outputs
|
||||
Inputs:
|
||||
- `transcripts/normalized.json`
|
||||
- `transcripts/final.json`
|
||||
|
||||
Outputs:
|
||||
- `transcripts/trimmed.json` (or configured trim output path)
|
||||
- `transcripts/final.trimmed.json` (or configured trim output path)
|
||||
- when trim enabled: `artifacts/session_bounds.json`
|
||||
|
||||
## Boundaries
|
||||
Owns:
|
||||
- Trim-enabled switch behavior
|
||||
- Bounds generation via Scriptorium artifact run
|
||||
- Bounds validation against normalized transcript
|
||||
- Bounds validation against final transcript
|
||||
- Keep-selector derivation and Seriatim trim invocation
|
||||
- Copy-through behavior when disabled or bounds indicate unchanged transcript
|
||||
|
||||
@@ -50,19 +50,19 @@ Does not own:
|
||||
- `Trim` when bounds indicate trimming is required
|
||||
|
||||
## State and Manifest Behavior
|
||||
- Reads normalized transcript from normalize manifest outputs when available; falls back to canonical path.
|
||||
- Reads final transcript from normalize manifest outputs when available; falls back to canonical path.
|
||||
- Uses run-local outputs/logs/reports/config/scratch paths when run layout is enabled.
|
||||
- Promotes canonical trimmed transcript; promotes session bounds when trim enabled.
|
||||
- Promotes canonical final-trimmed transcript; promotes session bounds when trim enabled.
|
||||
- Records bounds diagnostics, trim action, keep selector, and adapter metadata.
|
||||
|
||||
## Skip and Resume Behavior
|
||||
- Runner-level skip applies when already succeeded and not forced.
|
||||
- Forced reruns can stale downstream succeeded stages.
|
||||
- When `trim.enabled=false`, stage still succeeds by copying normalized to trimmed output.
|
||||
- When `trim.enabled=false`, stage still succeeds by copying final to final-trimmed output.
|
||||
|
||||
## Failure Behavior
|
||||
- Fails on missing/invalid normalized transcript.
|
||||
- With trim enabled, fails on missing adapters/config, bounds generation/validation errors, invalid bounds JSON, invalid range/segment ids, trim adapter failures, or invalid trimmed output.
|
||||
- Fails on missing/invalid final transcript.
|
||||
- With trim enabled, fails on missing adapters/config, bounds generation/validation errors, invalid bounds JSON, invalid range/segment ids, trim adapter failures, or invalid final-trimmed output.
|
||||
|
||||
## Tests to Inspect Before Changing
|
||||
- `internal/stage/trim_test.go`
|
||||
@@ -70,6 +70,6 @@ Does not own:
|
||||
- `internal/adapters/seriatim/subprocess_test.go`
|
||||
|
||||
## Architectural Invariants
|
||||
- Trim never falls back to processed transcript; normalized transcript is required input.
|
||||
- Trim never falls back to polished transcript; final transcript is required input.
|
||||
- `session_bounds` output exists only for enabled trim path.
|
||||
- Render-debug artifacts are diagnostics and not declared stage outputs.
|
||||
|
||||
@@ -6,7 +6,8 @@ Document Narratio's remote storage backend contracts and implementations under `
|
||||
## Inputs and outputs
|
||||
Inputs:
|
||||
- Resolved storage config (`pipeline.storage.*`).
|
||||
- Bucket-relative object keys and local file paths from stage/app orchestration.
|
||||
- Already-loaded environment variables for configured S3 credentials.
|
||||
- Bucket-relative object keys and local file paths from app/stage orchestration.
|
||||
|
||||
Outputs:
|
||||
- Listed/downloaded/uploaded object metadata (`ObjectInfo`).
|
||||
@@ -22,6 +23,7 @@ Does not own:
|
||||
- Session/run prefix semantics.
|
||||
- Archive commit order semantics.
|
||||
- Manifest updates.
|
||||
- Filesystem secret loading from `pipeline.secrets.env_dir`.
|
||||
|
||||
## Config fields used
|
||||
- `pipeline.storage.backend`
|
||||
@@ -55,8 +57,10 @@ Implementations:
|
||||
## Failure behavior
|
||||
- `NewObjectStoreFromConfig` fails when no remote backend is configured or required S3 config is missing.
|
||||
- `S3Backend` constructor fails when required bucket is missing or AWS client setup fails.
|
||||
- App command orchestration loads configured filesystem secrets before calling the object-store factory.
|
||||
- CRUD operations return contextual errors (including not-found behavior via `Exists`).
|
||||
- Key normalization is applied before operations (`\\` to `/`, leading slash trimmed).
|
||||
- Remote session loading uses `List` to find the exact `session.yml` key and `Download` to materialize it to a local temp file.
|
||||
|
||||
## Tests to inspect before changing
|
||||
- `internal/adapters/storage/factory_test.go`
|
||||
|
||||
@@ -17,7 +17,8 @@ Outputs:
|
||||
|
||||
## Boundaries
|
||||
Owns:
|
||||
- Session-level path layout (`inputs/`, `audio/`, `transcripts/`, `artifacts/`, `reports/`, `logs/`, `config/`, `current/`, `runs/`)
|
||||
- Session-level path layout (`inputs/`, `audio/`, `transcripts/`, `artifacts/`, `reports/`, `logs/`, `config/`, `current/`, `runs/`, `previous/`)
|
||||
- `previous/manifest.json` and `previous/artifacts/**` are reserved for previous-session cache state materialized by `prepare` or `restore`
|
||||
- Run-local stage sandbox layout under `runs/{run_id}/{stage}/`
|
||||
- Session lock acquisition/release (`.lock`)
|
||||
|
||||
@@ -31,6 +32,8 @@ Does not own:
|
||||
- `pipeline.workspace.cleanup_after_archive`
|
||||
- `pipeline.spool.root`
|
||||
- `pipeline.spool.delete_audio_after_archive`
|
||||
- `pipeline.cache.root`
|
||||
- `pipeline.cache.s3_audio`
|
||||
- `session.campaign`
|
||||
- `session.session_id`
|
||||
|
||||
@@ -42,7 +45,12 @@ None directly in this subsystem. Stages may use object storage adapters and then
|
||||
- Invocation history is persisted per run in run manifests under `runs/{run_id}/manifest.json`.
|
||||
- During each run, stage outputs are often written run-local first (`runs/{run_id}/{stage}/outputs/...`) and promoted to canonical session paths after stage success.
|
||||
- `manifest.Artifacts` entries record `ProducerRunID` for durable outputs.
|
||||
- For S3 audio sessions, `prepare` records spool/work paths and S3 provenance in `manifest.Inputs`.
|
||||
- For S3 audio sessions, `prepare` records work/cache paths, S3 provenance, and spool path when the invocation downloaded the object.
|
||||
- `previous/**` is reconstructed from configured previous-session requirements; restore uses the previous session's committed current archive rather than treating current-session archived `previous/**` as authoritative.
|
||||
- Durable cache state under `pipeline.cache.root` is not workspace state and is preserved by default by `narratio clean`.
|
||||
- `narratio clean <id>` removes the session work root and session spool root.
|
||||
- `narratio clean --all` removes all local session work under `workspace.root/work` and spool children under `spool.root`.
|
||||
- `narratio clean --clear-cache` is the explicit opt-in for deleting matching S3 audio cache entries.
|
||||
|
||||
## Skip and Resume Behavior
|
||||
- Skip/resume decisions are made in `internal/app` (`run_control.go`, `resume.go`) using stage status in the session manifest.
|
||||
@@ -65,4 +73,6 @@ None directly in this subsystem. Stages may use object storage adapters and then
|
||||
- Session root is campaign-aware: `{workspace.root}/work/{campaign}/{session_id}`.
|
||||
- Run roots are always nested: `runs/{run_id}` under the session root.
|
||||
- Run-local output promotion must end in canonical session paths.
|
||||
- Cleanup only targets run-scoped directories and must never delete configured root directories.
|
||||
- `previous/**` is session-durable state and must not be treated as run-local output scratch state.
|
||||
- Automatic post-archive cleanup only targets run-scoped directories and must never delete configured root directories.
|
||||
- Manual `clean` may delete session-scoped directories or the `workspace.root/work` directory, but it must preserve configured root directories and reject unsafe targets.
|
||||
|
||||
@@ -6,21 +6,80 @@ For field-level configuration, see [docs/config.md](./config.md). For full comma
|
||||
|
||||
## Normal workflow (S3-first path)
|
||||
|
||||
1. Upload session `.flac` files to object storage under the session audio prefix.
|
||||
2. Run Narratio:
|
||||
1. Create or upload `session.yml`, or pass a local `session.yml` explicitly.
|
||||
2. Upload session `.flac` files to object storage under the configured session audio prefix.
|
||||
3. Run Narratio:
|
||||
|
||||
```bash
|
||||
narratio run --session-id 2026-04-04
|
||||
narratio run 2026-04-04
|
||||
```
|
||||
|
||||
3. Read success output:
|
||||
4. Read success output:
|
||||
- `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
|
||||
- use `manifest=<path>` with `status` for inspection.
|
||||
- use `narratio session status <session_id>` for inspection.
|
||||
|
||||
Notes:
|
||||
- default config/session discovery applies unless `--config` and `--session` are passed.
|
||||
- default pipeline/session discovery checks system config locations; campaign selection uses `pipeline.campaigns.default_campaign_id` unless `--campaign <id>` or `--campaign-file <path>` is passed.
|
||||
- when local `session.yml` discovery misses, positional `<session_id>` loads remote `session.yml` from `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`.
|
||||
- S3 audio mode requires `session.inputs.audio_s3.prefix` and valid object-store access.
|
||||
|
||||
Initialize a remote session skeleton:
|
||||
|
||||
```bash
|
||||
narratio session init 2026-04-04 --remote
|
||||
```
|
||||
|
||||
Remote init uses normal default config discovery and writes `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`. If `campaign.yml` sets `session_template_file`, init renders that template from the supplied flags and writes concrete YAML. Pass `--config`, `--campaign <id>`, or `--campaign-file <path>` when testing non-system config files. It fails if the object already exists unless `--force` is passed.
|
||||
|
||||
Validate before running:
|
||||
|
||||
```bash
|
||||
narratio session validate 2026-04-04
|
||||
```
|
||||
|
||||
## Restore workflow
|
||||
|
||||
Use restore when local durable session state is missing or stale and archive current state is authoritative.
|
||||
|
||||
Dry-run (no local writes):
|
||||
|
||||
```bash
|
||||
narratio session restore 2026-04-04 --dry-run
|
||||
```
|
||||
|
||||
Execution:
|
||||
|
||||
```bash
|
||||
narratio session restore 2026-04-04
|
||||
```
|
||||
|
||||
Post-restore analyze rerun pattern:
|
||||
|
||||
```bash
|
||||
narratio analyze 2026-04-04
|
||||
```
|
||||
|
||||
Restore source-of-truth:
|
||||
- remote commit marker: `current/run_id.txt`
|
||||
- remote current manifest: `current/manifest.json`
|
||||
- configured previous-session requirements are reconstructed from the previous session's remote `current/` state, not from archived `previous/**` objects in the current session.
|
||||
|
||||
Restore default scope:
|
||||
- includes `manifest.json`, `transcripts/**`, and `artifacts/**` from the current session archive.
|
||||
- includes `previous/**` only when configured previous-session artifact inputs require it; restore hydrates those files the same way `prepare` would.
|
||||
- includes `audio/**` only with `--include-audio`
|
||||
- excludes `runs/**`, `logs/**`, `reports/**`, `config/**`, `inputs/**`, and `current/**` (except remote `current/manifest.json` as source)
|
||||
|
||||
Reset local state before restore testing:
|
||||
|
||||
```bash
|
||||
narratio clean 2026-04-04 --dry-run
|
||||
narratio clean 2026-04-04
|
||||
narratio session restore 2026-04-04 --include-audio
|
||||
```
|
||||
|
||||
`clean` removes the local session work directory and session spool directory. It preserves the durable S3 audio cache by default, so repeated restore or forced prepare tests do not re-download large audio files.
|
||||
|
||||
## Local filesystem layout and state artifacts
|
||||
|
||||
Session root:
|
||||
@@ -29,13 +88,15 @@ Session root:
|
||||
Primary state:
|
||||
- `manifest.json`: session-level stage state.
|
||||
- `runs/{run_id}/manifest.json`: invocation-level state.
|
||||
- `.lock`: session lock while a run is active.
|
||||
- `.lock`: session lock while a modifying command is active.
|
||||
- `inputs/campaign.yml`, `inputs/session.yml`, and `inputs/pipeline.resolved.yml`: materialized config inputs for the run.
|
||||
|
||||
Canonical session directories:
|
||||
- `inputs/`
|
||||
- `audio/`
|
||||
- `transcripts/`
|
||||
- `artifacts/`
|
||||
- `previous/`
|
||||
- `reports/`
|
||||
- `logs/`
|
||||
- `config/`
|
||||
@@ -48,6 +109,7 @@ Run-local stage directories:
|
||||
Behavior:
|
||||
- directory creation is idempotent.
|
||||
- stage outputs are generally generated run-local first, then promoted to canonical paths on success.
|
||||
- restore installs downloaded files to canonical session paths and does not recreate historical run sandboxes.
|
||||
|
||||
## Analyze artifact execution lifecycle
|
||||
|
||||
@@ -64,11 +126,29 @@ Configured artifact source reuse:
|
||||
- reused configured artifact provenance is `filesystem.disabled_artifact_output`.
|
||||
|
||||
`--artifacts` behavior:
|
||||
- accepted on `run`, `resume`, and `run-stage analyze`.
|
||||
- filters analyze execution only; does not force stage rerun.
|
||||
- accepted on `run`, `resume`, `run-stage analyze`, `run-stage archive`, `analyze`, and `publish`.
|
||||
- filters analyze execution and configured artifact promotions.
|
||||
- built-in transcript and bounds promotions are not filtered.
|
||||
- does not imply force on `run`, `resume`, or `run-stage`; `narratio analyze` is force-by-design.
|
||||
- `publish` is force-by-design and accepts `--artifacts` for configured artifact promotions.
|
||||
|
||||
Canonical previous-session input behavior:
|
||||
- canonical sources use `narratio.previous_session.artifact.<artifact_key>`.
|
||||
- these inputs are hydrated by `prepare` and by `restore`; `analyze` expects the local previous cache to already exist.
|
||||
- if analyze fails due to missing canonical previous cache, rerun:
|
||||
- `narratio run-stage prepare <id> --force`
|
||||
- or `narratio session restore <id>` when remote archive current state is authoritative.
|
||||
|
||||
## Remote archive layout and publish contract
|
||||
|
||||
Preferred manual publish command:
|
||||
|
||||
```bash
|
||||
narratio publish <id>
|
||||
```
|
||||
|
||||
`publish` is equivalent to `narratio run-stage archive <id> --force`; use `run-stage` when you need the general single-stage command form.
|
||||
|
||||
When archive is enabled and run upload is enabled, archive publishes under:
|
||||
|
||||
- session prefix: `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/`
|
||||
@@ -77,6 +157,7 @@ When archive is enabled and run upload is enabled, archive publishes under:
|
||||
Archive uploads:
|
||||
- run record files from run root (excluding `audio/`).
|
||||
- promoted files from explicit `archive.promote_artifacts` rules.
|
||||
- mutable session locks from helper commands live at `{session_prefix}/locks.yml`.
|
||||
|
||||
Publish order:
|
||||
1. upload `current/manifest.json`
|
||||
@@ -84,12 +165,25 @@ Publish order:
|
||||
|
||||
`current/run_id.txt` is the remote commit marker.
|
||||
|
||||
Archive promotion is explicit and path-based:
|
||||
Archive promotion is explicit and source-based:
|
||||
- Narratio does not auto-promote all generated analyze artifacts.
|
||||
- each rule resolves `source` through the artifact resolver/catalog model, then uploads to `dest`.
|
||||
- missing required promotion sources fail archive stage.
|
||||
- missing optional promotion sources are skipped.
|
||||
- invalid resolved artifacts fail archive stage.
|
||||
- `archive.locks` skips top-level promotion overwrites for locked sources while run-local uploads still publish.
|
||||
- remote locks from `{session_prefix}/locks.yml` are merged with static `archive.locks`; static locks win on duplicate sources.
|
||||
- locked required promotions are treated as intentional successful skips and are recorded in archive metadata.
|
||||
|
||||
## Resume, retry, and safe rerun behavior
|
||||
Lock helper behavior:
|
||||
- `narratio session locks <id>` lists effective static and remote locks.
|
||||
- `narratio session locks add <id> <source> --reason <text>` writes a remote lock.
|
||||
- `narratio session locks add <id> <source> --force --reason <text>` updates an existing remote lock reason.
|
||||
- `narratio session locks remove <id> <source>` removes only a remote lock.
|
||||
- `locks remove` cannot remove static pipeline locks.
|
||||
- remote lock writes check whether the lock store exists, but are not compare-and-swap atomic.
|
||||
|
||||
## Resume, retry, restore, and safe rerun behavior
|
||||
|
||||
Default skip:
|
||||
- `run` and `run-stage` skip already-succeeded stages unless `--force` is set.
|
||||
@@ -98,8 +192,14 @@ Resume:
|
||||
- `resume` starts at first non-succeeded stage.
|
||||
- `resume --force` runs full stage order.
|
||||
|
||||
Restore conflict policy:
|
||||
- restore classifies local differences as conflicts.
|
||||
- without `--force`, restore fails when conflicts exist.
|
||||
- with `--force`, conflicting local files are overwritten by remote archive files.
|
||||
|
||||
Forced reruns:
|
||||
- force-rerunning an upstream succeeded stage marks downstream succeeded stages as `stale`.
|
||||
- ordinary `--force` does not override archive locks.
|
||||
|
||||
Safe rerun pattern:
|
||||
1. rerun the changed stage with `--force`.
|
||||
@@ -107,12 +207,20 @@ Safe rerun pattern:
|
||||
|
||||
## Cleanup behavior
|
||||
|
||||
Cleanup is considered only when archive stage executed and succeeded.
|
||||
Automatic post-archive cleanup is considered only when archive stage executed and succeeded.
|
||||
|
||||
Cleanup toggles:
|
||||
Automatic cleanup toggles:
|
||||
- `pipeline.spool.delete_audio_after_archive=true` deletes run-scoped spool audio.
|
||||
- `pipeline.workspace.cleanup_after_archive=true` deletes run-scoped local run directory.
|
||||
|
||||
Manual cleanup:
|
||||
- `narratio clean <id>` deletes `{workspace.root}/work/{campaign}/{session_id}` and `{spool.root}/{campaign}/{session_id}`.
|
||||
- `narratio clean --all` deletes all local session work under `{workspace.root}/work` and all spool children under `{spool.root}`.
|
||||
- `--dry-run` prints targets without deleting.
|
||||
- `--clear-cache` also removes matching S3 audio cache files. Without it, cache is preserved.
|
||||
|
||||
The S3 audio cache under `pipeline.cache.root` is durable input cache state, not workspace or spool state. Automatic cleanup and default manual cleanup do not delete it.
|
||||
|
||||
Cleanup eligibility gates:
|
||||
- archive enabled
|
||||
- archive run upload enabled
|
||||
@@ -123,27 +231,55 @@ No cleanup for failed/incomplete/unarchived/archive-skipped runs.
|
||||
|
||||
## Failure and recovery playbooks
|
||||
|
||||
After failure, Narratio keeps:
|
||||
After run failure, Narratio keeps:
|
||||
- session manifest
|
||||
- run manifest
|
||||
- run-local artifacts/logs/config/reports
|
||||
|
||||
Failed or incomplete runs remain local-only.
|
||||
|
||||
After restore failure:
|
||||
- already-installed restore files remain in place.
|
||||
- restore does not roll back prior successful installs.
|
||||
- existing local manifest is preserved if restored manifest validation/install fails.
|
||||
|
||||
Recommended recovery:
|
||||
|
||||
1. inspect state:
|
||||
|
||||
```bash
|
||||
narratio status --manifest <manifest-path>
|
||||
narratio session status 2026-04-04
|
||||
```
|
||||
|
||||
2. fix root cause (config/input/credentials/service availability).
|
||||
3. continue with `resume`, or targeted `run-stage --force` followed by `resume`.
|
||||
This reports local manifest state, committed remote current state, expected remote transcript/artifact availability, and archive locks.
|
||||
|
||||
2. for restore-specific checks, run:
|
||||
|
||||
```bash
|
||||
narratio session restore 2026-04-04 --dry-run
|
||||
```
|
||||
|
||||
3. fix root cause (config/input/credentials/storage/service availability).
|
||||
4. continue with `resume`, or targeted `run-stage <stage> <id> --force` followed by `resume`.
|
||||
|
||||
## Restore report
|
||||
|
||||
Non-dry-run restore writes a durable report at:
|
||||
- `reports/restore-latest.json`
|
||||
|
||||
Report content includes:
|
||||
- identity (`campaign`, `session_id`, `run_id`)
|
||||
- mode flags (`dry_run`, `force`, `include_audio`)
|
||||
- plan counts and execution counts
|
||||
- per-action status
|
||||
|
||||
Dry-run does not write restore report files.
|
||||
|
||||
## Operational caveats
|
||||
|
||||
- `status` requires explicit `--manifest`; there is no session-id lookup command.
|
||||
- `session status <session_id>` uses normal config/session loading, including remote session fallback.
|
||||
- `session status <session_id>` includes the same promoted remote output availability view as `session artifacts <session_id> --remote` when storage is configured.
|
||||
- local and S3 audio input modes are mutually exclusive.
|
||||
- archive publish requires upstream stages through `analyze` to be `succeeded`.
|
||||
- required promotion rules can fail when selected analyze artifacts did not generate a required file path.
|
||||
- required configured artifact promotions for unselected `--artifacts` keys are skipped intentionally; selected required promotions still fail if their files are missing.
|
||||
- restore requires configured remote object storage and committed remote current state.
|
||||
|
||||
231
docs/roadmap/campaign.md
Normal file
231
docs/roadmap/campaign.md
Normal file
@@ -0,0 +1,231 @@
|
||||
# Roadmap: Campaign Registry
|
||||
|
||||
Status: Implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Narratio currently treats campaign configuration as one selected
|
||||
`campaign.yml` file:
|
||||
|
||||
- command flags use `--campaign <path>`;
|
||||
- default discovery searches fixed system file locations;
|
||||
- `campaign.yml` uses `campaign:` as the identity field.
|
||||
|
||||
That model works for a single campaign, but it is awkward for installations
|
||||
that manage multiple campaigns. Operators need to pass file paths or maintain a
|
||||
single global campaign config, while the newer session-oriented CLI already
|
||||
uses concise positional session IDs and remote session lookup.
|
||||
|
||||
The campaign selection model should become ID-based and pipeline-owned.
|
||||
Pipeline config should describe where campaigns live, commands should select a
|
||||
campaign by ID, and each campaign directory should contain its stable campaign
|
||||
materials.
|
||||
|
||||
## Target Model
|
||||
|
||||
`pipeline.yml` owns the campaign registry:
|
||||
|
||||
campaigns:
|
||||
root: /usr/local/share/narratio/campaigns
|
||||
default_campaign_id: dilfs
|
||||
|
||||
Campaign files live at the conventional path:
|
||||
|
||||
{campaigns.root}/{campaign_id}/campaign.yml
|
||||
|
||||
The first implementation should use only the conventional path. Recursive
|
||||
discovery of every `campaign.yml` under `campaigns.root` is deferred to a
|
||||
future stage.
|
||||
|
||||
Each campaign file uses `campaign_id` as the canonical identity field:
|
||||
|
||||
campaign_id: dilfs
|
||||
session_template_file: ./session.template.yml
|
||||
inputs:
|
||||
speakers_file: ./speakers.yml
|
||||
autocorrect_file: ./autocorrect.yml
|
||||
glossary_file: ./glossary.yml
|
||||
|
||||
Campaign-relative files continue to resolve relative to the selected
|
||||
`campaign.yml`, including stable input files and `session_template_file`.
|
||||
|
||||
The public CLI changes from path-based campaign selection to ID-based campaign
|
||||
selection:
|
||||
|
||||
- `--campaign <id>` selects a campaign ID.
|
||||
- `--campaign-file <path>` explicitly loads one campaign file for
|
||||
development, tests, and unusual local workflows.
|
||||
- `--campaign` and `--campaign-file` are mutually exclusive.
|
||||
|
||||
If neither `--campaign` nor `--campaign-file` is passed, Narratio uses
|
||||
`pipeline.campaigns.default_campaign_id`. If no campaign can be selected,
|
||||
commands fail clearly before session loading or stage execution.
|
||||
|
||||
Resolved campaign ID remains the campaign segment used for:
|
||||
|
||||
- workspace paths;
|
||||
- spool paths;
|
||||
- S3 session prefixes;
|
||||
- remote `session.yml` lookup;
|
||||
- archive locks and promoted output keys;
|
||||
- session/campaign mismatch validation;
|
||||
- status, plan, restore, and helper output.
|
||||
|
||||
## Compatibility Policy
|
||||
|
||||
This is a breaking public/config contract change.
|
||||
|
||||
After the cutover:
|
||||
|
||||
- `--campaign` no longer accepts a filesystem path;
|
||||
- default fixed campaign file discovery is removed;
|
||||
- `campaign:` is no longer accepted in `campaign.yml`;
|
||||
- `campaign_id:` is required.
|
||||
|
||||
Keep `--campaign-file` as the only explicit file override. Do not retain hidden
|
||||
aliases for the old `--campaign <path>` behavior.
|
||||
|
||||
## Implementation Stages
|
||||
|
||||
### Stage 1: Add Campaign Registry Selection
|
||||
|
||||
Status: Implemented
|
||||
|
||||
Add the registry model and switch command loading to resolve campaigns through
|
||||
pipeline config.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Add `pipeline.campaigns.root`.
|
||||
- Add `pipeline.campaigns.default_campaign_id`.
|
||||
- Add `campaign_id` to campaign config and make it the canonical identity.
|
||||
- Resolve pipeline config first, then campaign selection.
|
||||
- Use this selection order:
|
||||
1. explicit `--campaign-file <path>`;
|
||||
2. explicit `--campaign <id>`;
|
||||
3. `pipeline.campaigns.default_campaign_id`;
|
||||
4. fail clearly.
|
||||
- For ID selection, load `{campaigns.root}/{campaign_id}/campaign.yml`.
|
||||
- Validate that the loaded `campaign_id` matches the selected ID.
|
||||
- Reject `--campaign` with `--campaign-file`.
|
||||
- Preserve strict YAML decoding.
|
||||
- Preserve campaign-relative stable input and session template resolution.
|
||||
- Keep storage details behind the existing storage adapter and object-store
|
||||
helper.
|
||||
- Keep remote session lookup and archive key construction based on the
|
||||
resolved campaign ID.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- Commands can run with only a pipeline config and the pipeline default
|
||||
campaign ID.
|
||||
- Commands can select another campaign with `--campaign <id>`.
|
||||
- Commands can load a specific file with `--campaign-file <path>`.
|
||||
- Existing session loading, remote session fallback, prepare materialization,
|
||||
restore, archive, locks, clean, analyze, and publish behavior continue to use
|
||||
the same resolved campaign identity.
|
||||
- No generic config registry framework is introduced.
|
||||
|
||||
### Stage 2: Remove Old Single-File Campaign Behavior
|
||||
|
||||
Status: Implemented
|
||||
|
||||
Remove the old public campaign file model after registry selection is in
|
||||
place.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Remove fixed default campaign config discovery from command loading.
|
||||
- Remove `DefaultCampaignConfigSearchPaths` and related path-only resolution if
|
||||
no current tests or helpers still need them.
|
||||
- Remove support for `campaign:` from `campaign.yml`.
|
||||
- Update validation errors to refer to `campaign_id`.
|
||||
- Update examples to use campaign directories and `campaign_id`.
|
||||
- Update current-behavior docs to document:
|
||||
- `pipeline.campaigns.root`;
|
||||
- `pipeline.campaigns.default_campaign_id`;
|
||||
- `campaign_id`;
|
||||
- `--campaign <id>`;
|
||||
- `--campaign-file <path>`.
|
||||
- Update troubleshooting examples that currently pass `--campaign <path>`.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- `campaign.yml` files with `campaign:` fail strict decoding.
|
||||
- `--campaign /path/to/campaign.yml` is treated as a campaign ID and fails
|
||||
unless that ID exists under `campaigns.root`.
|
||||
- `--campaign-file /path/to/campaign.yml` is the supported file override.
|
||||
- User-facing docs no longer describe fixed campaign config discovery.
|
||||
|
||||
## Test Guidance
|
||||
|
||||
Focused tests:
|
||||
|
||||
- `go test ./internal/config -v`
|
||||
- `go test ./internal/app -v`
|
||||
- `go test ./internal/stage -run Prepare -v`
|
||||
|
||||
Full validation:
|
||||
|
||||
- `go test ./...`
|
||||
|
||||
Config tests to add or update:
|
||||
|
||||
- strict decode accepts `pipeline.campaigns.root`;
|
||||
- strict decode accepts `pipeline.campaigns.default_campaign_id`;
|
||||
- strict decode accepts `campaign_id`;
|
||||
- selected campaign ID mismatch fails;
|
||||
- missing campaign root fails when ID selection is needed;
|
||||
- missing default campaign ID fails when no explicit campaign selector is
|
||||
passed;
|
||||
- old `campaign:` fails after Stage 2.
|
||||
|
||||
App tests to add or update:
|
||||
|
||||
- `--campaign <id>` resolves `{campaigns.root}/{id}/campaign.yml`;
|
||||
- omitted `--campaign` uses `pipeline.campaigns.default_campaign_id`;
|
||||
- `--campaign-file` loads an explicit campaign file;
|
||||
- `--campaign` plus `--campaign-file` fails;
|
||||
- remote session fallback uses the resolved campaign ID;
|
||||
- `session init`, `run`, `run-stage`, `resume`, `analyze`, `publish`, `clean`,
|
||||
and `session` subcommands all use the same campaign selection path;
|
||||
- path-based `--campaign` examples and tests are removed after Stage 2.
|
||||
|
||||
## Documentation Guidance
|
||||
|
||||
Update current-behavior docs only after implementation lands:
|
||||
|
||||
- `docs/config.md`
|
||||
- `docs/cli.md`
|
||||
- `docs/operations.md`
|
||||
- `docs/troubleshooting.md`
|
||||
- relevant files under `docs/internal/`
|
||||
- `examples/`
|
||||
|
||||
Planned campaign registry behavior belongs only in this roadmap until the code,
|
||||
tests, examples, and current-behavior docs are updated.
|
||||
|
||||
## Architecture Guardrails
|
||||
|
||||
- Keep Narratio explicit and stage-driven.
|
||||
- Do not introduce a generic configuration registry or workflow framework.
|
||||
- Keep YAML decoding strict.
|
||||
- Keep defaults centralized and testable.
|
||||
- Keep campaign-relative path resolution centralized.
|
||||
- Use centralized S3 and workspace path helpers.
|
||||
- Keep storage details behind `storage.ObjectStore`.
|
||||
- Keep secret-backed object-store construction in `internal/app`.
|
||||
- Preserve manifest-driven resume and restore behavior.
|
||||
- Do not store raw secrets in campaign configs, manifests, logs, generated
|
||||
configs, or archive metadata.
|
||||
|
||||
## Assumptions
|
||||
|
||||
- The canonical pipeline schema is grouped under `campaigns`.
|
||||
- The canonical campaign identity field is `campaign_id`.
|
||||
- `--campaign` means campaign ID.
|
||||
- `--campaign-file` is retained as an explicit override.
|
||||
- Recursive discovery is planned but not part of the first implementation.
|
||||
- Existing production configs can be migrated from `campaign:` to
|
||||
`campaign_id:` and from `--campaign <path>` to `--campaign <id>` or
|
||||
`--campaign-file <path>`.
|
||||
159
docs/roadmap/cleanup.md
Normal file
159
docs/roadmap/cleanup.md
Normal file
@@ -0,0 +1,159 @@
|
||||
# Roadmap: Legacy Config Cleanup
|
||||
|
||||
Status: Implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Narratio's current pipeline config schema still accepts fields that predate the current storage, artifact, and previous-session models:
|
||||
|
||||
- `pipeline.storage.bucket`
|
||||
- `pipeline.storage.prefix`
|
||||
- `pipeline.analyzer.*`
|
||||
- `previous_session_artifact`
|
||||
|
||||
These names make the config reference harder to trust because they suggest supported behavior that operators should no longer use. The modern interface is:
|
||||
|
||||
- `pipeline.storage.s3.*` for remote storage.
|
||||
- Scriptorium configured artifacts under `pipeline.scriptorium.artifacts`.
|
||||
- Canonical artifact source IDs such as `narratio.artifact.<configured_artifact_key>`.
|
||||
- Canonical previous-session artifact sources such as `narratio.previous_session.artifact.<configured_artifact_key>`.
|
||||
|
||||
Strict YAML decoding should reject removed legacy fields once this cleanup lands.
|
||||
|
||||
## Current State
|
||||
|
||||
`pipeline.storage.bucket` and `pipeline.storage.prefix` were inert compatibility fields and have been removed:
|
||||
|
||||
- They are no longer present on `config.StorageConfig`.
|
||||
- Strict decoding rejects them.
|
||||
- Runtime S3 behavior uses `pipeline.storage.s3.bucket` and `pipeline.storage.s3.root_prefix`.
|
||||
- No current code reads the top-level storage bucket or prefix fields.
|
||||
|
||||
`pipeline.analyzer.*` was legacy code surface and has been removed:
|
||||
|
||||
- `config.PipelineConfig` no longer includes analyzer config.
|
||||
- Strict decoding rejects `pipeline.analyzer`.
|
||||
- `stage.Env` no longer exposes an analyzer runner, and `internal/adapters/analyzer` has been deleted.
|
||||
- Modern analyze execution is Scriptorium-backed; the analyzer adapter is not used by current stage execution.
|
||||
|
||||
`previous_session_artifact` was a live legacy behavior and has been removed:
|
||||
|
||||
- Config validation rejects it as an unsupported Scriptorium input source.
|
||||
- The analyze stage no longer has path-based previous-artifact resolution through `inputs.<name>.path`.
|
||||
- Tests cover canonical previous-session sources and the rejection of the legacy source.
|
||||
- The canonical replacement is `narratio.previous_session.artifact.<configured_artifact_key>`, resolved through the previous-session cache/catalog model.
|
||||
|
||||
## Target Model
|
||||
|
||||
The pipeline config schema should expose only current behavior:
|
||||
|
||||
- Remote storage is configured only through `pipeline.storage.s3.*`.
|
||||
- Generated artifacts are configured only through `pipeline.scriptorium.artifacts`.
|
||||
- Scriptorium artifact inputs use canonical source IDs.
|
||||
- Previous-session artifact inputs use `narratio.previous_session.artifact.<configured_artifact_key>`.
|
||||
- Unknown legacy fields fail strict YAML decoding.
|
||||
|
||||
No compatibility aliases should remain unless a future migration requirement explicitly reintroduces them.
|
||||
|
||||
## Cleanup Order
|
||||
|
||||
### Stage 1: Remove Inert Storage Compatibility Fields
|
||||
|
||||
Status: Implemented
|
||||
|
||||
Remove `pipeline.storage.bucket` and `pipeline.storage.prefix`.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Delete `StorageConfig.Bucket` and `StorageConfig.Prefix`.
|
||||
- Keep `StorageConfig.Backend` and `StorageConfig.S3`.
|
||||
- Confirm all runtime storage paths continue to use `storage.s3.bucket` and `storage.s3.root_prefix`.
|
||||
- Update examples and docs to remove top-level storage `bucket` and `prefix`.
|
||||
- Add or update strict-decode tests proving `pipeline.storage.bucket` and `pipeline.storage.prefix` are rejected.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- Existing S3 workflows still pass with `pipeline.storage.s3.bucket`.
|
||||
- Pipeline configs containing top-level `storage.bucket` or `storage.prefix` fail to load.
|
||||
- No docs or examples present those fields as available.
|
||||
|
||||
### Stage 2: Remove Legacy Analyzer Schema and Adapter Surface
|
||||
|
||||
Status: Implemented
|
||||
|
||||
Remove the unused analyzer configuration and adapter contract.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Delete `PipelineConfig.Analyzer`.
|
||||
- Delete `AnalyzerConfig` and `ArtifactSettings`.
|
||||
- Remove analyzer timeout validation.
|
||||
- Remove `stage.Env.Analyzer`.
|
||||
- Delete `internal/adapters/analyzer` if no remaining code imports it.
|
||||
- Remove `pipeline.analyzer.*` from tests, examples, and docs.
|
||||
- Add or update strict-decode tests proving `pipeline.analyzer` is rejected.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- Analyze behavior remains fully Scriptorium-backed.
|
||||
- No runtime code imports `internal/adapters/analyzer`.
|
||||
- Pipeline configs containing `pipeline.analyzer` fail to load.
|
||||
- Contributor and internal adapter docs no longer list the analyzer adapter.
|
||||
|
||||
### Stage 3: Remove Path-Based Previous Session Artifact Source
|
||||
|
||||
Status: Implemented
|
||||
|
||||
Remove `previous_session_artifact` and require canonical previous-session artifact sources.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Remove `previous_session_artifact` from supported Scriptorium input sources.
|
||||
- Remove analyze-stage special-case handling that resolves `inputs.<name>.path` for previous artifacts.
|
||||
- Keep canonical handling for `narratio.previous_session.artifact.<configured_artifact_key>`.
|
||||
- Rewrite tests that use `previous_session_artifact` to use canonical sources and prepared previous-cache fixtures.
|
||||
- Add validation tests proving `previous_session_artifact` is rejected.
|
||||
- Update docs to remove the legacy path-based source and document only canonical previous-session sources.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- `pipeline.scriptorium.artifacts.*.inputs.*.source: previous_session_artifact` fails validation.
|
||||
- Canonical previous-session sources continue to work for required and optional inputs.
|
||||
- Prepare/restore previous-cache behavior remains unchanged.
|
||||
- No docs or examples mention `previous_session_artifact` as supported.
|
||||
|
||||
## Test Guidance
|
||||
|
||||
Run focused tests after each stage:
|
||||
|
||||
- `go test ./internal/config -v`
|
||||
- `go test ./internal/stage -run Analyze -v`
|
||||
- `go test ./internal/app -v`
|
||||
- `go test ./...`
|
||||
|
||||
For Stage 1, focus on config load/strict-decode and S3 workflow regression tests.
|
||||
|
||||
For Stage 2, focus on compile-time removal, config strict-decode tests, and full app/stage tests to catch stale adapter references.
|
||||
|
||||
For Stage 3, focus on Scriptorium config validation, analyze-stage input resolution, previous-cache behavior, and restore/analyze workflows.
|
||||
|
||||
## Documentation Updates
|
||||
|
||||
Update current-behavior docs only after the corresponding code removal lands:
|
||||
|
||||
- `docs/config.md`
|
||||
- `docs/cli.md`, only if command behavior text references removed fields.
|
||||
- `docs/operations.md`, only if operator workflow text references removed fields.
|
||||
- `docs/internal/stage-analyze.md`
|
||||
- `docs/internal/adapters.md`
|
||||
- `examples/pipeline.full.annotated.yml`
|
||||
- `examples/pipeline.production.yml`
|
||||
|
||||
Do not preserve removed fields in examples as compatibility notes. The goal is to make strict config behavior and documentation line up.
|
||||
|
||||
## Assumptions
|
||||
|
||||
- This is a hard cleanup; no backward-compatible aliases are retained.
|
||||
- Current production configs can be migrated to `storage.s3.*`, Scriptorium artifacts, and canonical previous-session sources before this lands.
|
||||
- Removing the unused analyzer adapter does not block any active stage behavior.
|
||||
- The cleanup should be implemented in the listed order so inert schema removal is separated from behavior removal.
|
||||
255
docs/roadmap/cli.md
Normal file
255
docs/roadmap/cli.md
Normal file
@@ -0,0 +1,255 @@
|
||||
# Roadmap: Session-Oriented CLI Cleanup
|
||||
|
||||
Status: Implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Narratio's public CLI has accumulated too many top-level commands. Several
|
||||
commands are session-scoped operator helpers, but they currently appear as
|
||||
independent top-level verbs:
|
||||
|
||||
- `plan`
|
||||
- `status`
|
||||
- `restore`
|
||||
- `artifacts list`
|
||||
- `locks`
|
||||
- `session validate`
|
||||
- `session init`
|
||||
|
||||
This makes the command surface harder to learn because the CLI does not clearly
|
||||
separate primary workflow actions from session inspection, initialization,
|
||||
restore, and helper operations.
|
||||
|
||||
## Target Model
|
||||
|
||||
Keep primary workflow commands at top level:
|
||||
|
||||
- `run`
|
||||
- `run-stage`
|
||||
- `resume`
|
||||
- `analyze`
|
||||
- `publish`
|
||||
- `clean`
|
||||
- `session`
|
||||
|
||||
Keep `clean` top-level because it can operate on one session or all local
|
||||
sessions and is a workspace maintenance command, not only a session helper.
|
||||
|
||||
Move session-scoped helper commands under `narratio session` and use positional
|
||||
session identifiers:
|
||||
|
||||
- `narratio session init <session_id> [--remote|--output <path>] [--flags]`
|
||||
- `narratio session validate <session_id> [--flags]`
|
||||
- `narratio session status <session_id> [--flags]`
|
||||
- `narratio session plan <session_id> [--flags]`
|
||||
- `narratio session restore <session_id> [--flags]`
|
||||
- `narratio session artifacts <session_id> [--remote] [--flags]`
|
||||
- `narratio session locks <session_id> [--flags]`
|
||||
- `narratio session locks add <session_id> <source> [--reason <text>] [--force] [--flags]`
|
||||
- `narratio session locks remove <session_id> <source> [--flags]`
|
||||
|
||||
Update top-level workflow commands to use positional session identifiers:
|
||||
|
||||
- `narratio run <session_id> [--flags]`
|
||||
- `narratio resume <session_id> [--flags]`
|
||||
- `narratio analyze <session_id> [--flags]`
|
||||
- `narratio publish <session_id> [--flags]`
|
||||
- `narratio run-stage <stage> <session_id> [--flags]`
|
||||
|
||||
The positional session ID replaces `--session-id` as the primary public
|
||||
interface. Existing `--config`, `--campaign`, `--session`, and
|
||||
`--previous-session-id` flags remain available where they are meaningful.
|
||||
|
||||
## Command Mapping
|
||||
|
||||
| Current command | Target command |
|
||||
| --- | --- |
|
||||
| `narratio run --session-id <id>` | `narratio run <id>` |
|
||||
| `narratio resume --session-id <id>` | `narratio resume <id>` |
|
||||
| `narratio analyze --session-id <id>` | `narratio analyze <id>` |
|
||||
| `narratio publish --session-id <id>` | `narratio publish <id>` |
|
||||
| `narratio run-stage [flags] <stage> --session-id <id>` | `narratio run-stage <stage> <id> [flags]` |
|
||||
| `narratio plan --session-id <id>` | `narratio session plan <id>` |
|
||||
| `narratio status --session-id <id>` | `narratio session status <id>` |
|
||||
| `narratio restore --session-id <id>` | `narratio session restore <id>` |
|
||||
| `narratio artifacts list --session-id <id>` | `narratio session artifacts <id>` |
|
||||
| `narratio locks --session-id <id>` | `narratio session locks <id>` |
|
||||
| `narratio locks add --session-id <id> <source>` | `narratio session locks add <id> <source>` |
|
||||
| `narratio locks remove --session-id <id> <source>` | `narratio session locks remove <id> <source>` |
|
||||
| `narratio session validate --session-id <id>` | `narratio session validate <id>` |
|
||||
| `narratio session init --session-id <id>` | `narratio session init <id>` |
|
||||
| `narratio clean --session-id <id>` | `narratio clean <id>` |
|
||||
| `narratio clean --all` | unchanged |
|
||||
|
||||
`clean` remains top-level, but its session-scoped form should also move from
|
||||
`--session-id` to positional `<session_id>` for consistency.
|
||||
|
||||
## Compatibility Policy
|
||||
|
||||
This is a hard public CLI cleanup after the migration step lands.
|
||||
|
||||
During Step 1, old forms may remain as compatibility aliases to keep the
|
||||
implementation reviewable. During Step 2, remove the old forms from command
|
||||
dispatch, tests, docs, and examples:
|
||||
|
||||
- remove top-level `plan`;
|
||||
- remove top-level `status`;
|
||||
- remove top-level `restore`;
|
||||
- remove top-level `artifacts`;
|
||||
- remove top-level `locks`;
|
||||
- remove `--session-id` from the public command syntax for session-aware
|
||||
commands.
|
||||
|
||||
Do not keep long-term deprecated aliases unless a later roadmap explicitly
|
||||
chooses a compatibility window.
|
||||
|
||||
`status --manifest` does not fit the session-oriented command shape. Remove it
|
||||
from the public CLI in this cleanup. If direct manifest inspection is needed
|
||||
later, add a separate diagnostic command in a future roadmap rather than keeping
|
||||
it as a special case in `session status`.
|
||||
|
||||
## Implementation Step 1: Add New Session-Oriented Interface
|
||||
|
||||
Status: Implemented
|
||||
|
||||
Add the target command forms while preserving current behavior internally.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Add positional session ID parsing helpers in `internal/app`.
|
||||
- Keep the existing `loadCommandConfig` behavior and populate
|
||||
`config.SessionLoadOptions.SessionID` from the positional ID.
|
||||
- Add or update command wrappers:
|
||||
- `Run(ctx, args, out)` parses `run <session_id>`.
|
||||
- `Resume(ctx, args, out)` parses `resume <session_id>`.
|
||||
- `Analyze(ctx, args, out)` parses `analyze <session_id>`.
|
||||
- `Publish(ctx, args, out)` parses `publish <session_id>`.
|
||||
- `RunStage(ctx, args, out)` parses `run-stage <stage> <session_id>`.
|
||||
- `Clean(ctx, args, out)` parses `clean <session_id>` and keeps
|
||||
`clean --all`.
|
||||
- Extend `Session(ctx, args, out)` dispatch to support:
|
||||
- `init <session_id>`
|
||||
- `validate <session_id>`
|
||||
- `status <session_id>`
|
||||
- `plan <session_id>`
|
||||
- `restore <session_id>`
|
||||
- `artifacts <session_id>`
|
||||
- `locks <session_id>`
|
||||
- `locks add <session_id> <source>`
|
||||
- `locks remove <session_id> <source>`
|
||||
- Keep storage access through the existing app-level object-store helper.
|
||||
- Keep AWS SDK details behind storage adapters.
|
||||
- Keep the runner, stages, manifest behavior, archive behavior, restore
|
||||
planning, lock semantics, and artifact catalog behavior unchanged.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- New forms execute the same code paths and produce equivalent results.
|
||||
- Positional session ID mismatch with concrete local or remote `session.yml`
|
||||
fails through existing session identity checks.
|
||||
- Remote session fallback still uses the positional session ID as the lookup
|
||||
value.
|
||||
- Current command tests cover the new forms before old forms are removed.
|
||||
|
||||
## Implementation Step 2: Remove Old Public Forms
|
||||
|
||||
Status: Implemented
|
||||
|
||||
Remove compatibility aliases and make the session-oriented interface the only
|
||||
documented and supported public CLI.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Remove top-level dispatch for:
|
||||
- `plan`
|
||||
- `status`
|
||||
- `restore`
|
||||
- `artifacts`
|
||||
- `locks`
|
||||
- Remove `--session-id` flags from public session-aware commands.
|
||||
- Keep `--previous-session-id` as an expected previous-session identity flag.
|
||||
- Keep explicit `--session <path>` for loading a local concrete session file,
|
||||
but still require the positional session ID for commands that operate on a
|
||||
session.
|
||||
- Remove `status --manifest`.
|
||||
- Update usage text and invalid-command errors.
|
||||
- Update `docs/cli.md` and `docs/operations.md` to use only the new forms.
|
||||
- Update any roadmap docs that mention old helper command names.
|
||||
- Update tests to expect old top-level helper commands and `--session-id` forms
|
||||
to fail.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- Top-level command list is exactly:
|
||||
- `run`
|
||||
- `run-stage`
|
||||
- `resume`
|
||||
- `analyze`
|
||||
- `publish`
|
||||
- `clean`
|
||||
- `session`
|
||||
- All session-oriented commands use `narratio session <subcommand>
|
||||
<session_id> [--flags]`, except nested lock mutation forms, which use
|
||||
`narratio session locks add|remove <session_id> <source> [--flags]`.
|
||||
- `clean <session_id>` and `clean --all` remain top-level.
|
||||
- Current-behavior docs and tests no longer advertise `--session-id`.
|
||||
|
||||
## Test Guidance
|
||||
|
||||
Focused tests:
|
||||
|
||||
- `go test ./internal/app -run TestExecute -v`
|
||||
- `go test ./internal/app -run 'Session|Status|Restore|Clean|Locks|Artifacts|Plan|RunStage|Analyze|Publish' -v`
|
||||
- `go test ./internal/config -v`
|
||||
|
||||
Full validation:
|
||||
|
||||
- `go test ./...`
|
||||
|
||||
Test cases to add or update:
|
||||
|
||||
- `run <session_id>` loads local and remote sessions through the existing
|
||||
config path.
|
||||
- `resume <session_id>`, `analyze <session_id>`, and `publish <session_id>`
|
||||
preserve current behavior.
|
||||
- `run-stage <stage> <session_id>` preserves current run-stage output and
|
||||
force/artifact-selection behavior.
|
||||
- `session plan <session_id>` replaces top-level `plan`.
|
||||
- `session status <session_id>` replaces top-level session status.
|
||||
- `session validate <session_id>` replaces `session validate --session-id`.
|
||||
- `session init <session_id>` writes the same local or remote concrete
|
||||
`session.yml`.
|
||||
- `session restore <session_id>` preserves restore planning/execution.
|
||||
- `session artifacts <session_id> --remote` preserves promoted-output
|
||||
availability reporting.
|
||||
- `session locks <session_id>`, `session locks add <session_id> <source>`, and
|
||||
`session locks remove <session_id> <source>` preserve static/remote lock
|
||||
semantics.
|
||||
- `clean <session_id>` preserves session cleanup behavior, while `clean --all`
|
||||
remains unchanged.
|
||||
- Old top-level helper commands fail after Step 2.
|
||||
- `--session-id` fails after Step 2.
|
||||
- `status --manifest` fails after Step 2.
|
||||
|
||||
## Documentation Guidance
|
||||
|
||||
Update only after implementation lands:
|
||||
|
||||
- `docs/cli.md`
|
||||
- `docs/operations.md`
|
||||
- any internal docs that list command names or examples
|
||||
|
||||
Keep planned behavior only in this roadmap until the command refactor is
|
||||
implemented.
|
||||
|
||||
## Architecture Guardrails
|
||||
|
||||
- Keep Narratio explicit and stage-driven.
|
||||
- Do not introduce a generic workflow or command framework abstraction.
|
||||
- Reuse existing app command helpers where practical.
|
||||
- Keep config loading strict and centralized.
|
||||
- Keep storage details behind `storage.ObjectStore`.
|
||||
- Keep secret-backed object-store construction in `internal/app`.
|
||||
- Preserve manifest-driven resume and restore behavior.
|
||||
- Treat command renaming as a public CLI contract change, not a runtime stage
|
||||
behavior change.
|
||||
287
docs/roadmap/publish.md
Normal file
287
docs/roadmap/publish.md
Normal file
@@ -0,0 +1,287 @@
|
||||
# Roadmap: Publish Contract
|
||||
|
||||
Status: Planned
|
||||
|
||||
## Problem
|
||||
|
||||
Narratio currently uses several terms for one operator-facing concept:
|
||||
|
||||
- `archive` is the stage that uploads run state and commits remote current
|
||||
state.
|
||||
- `publish` is the convenience command that force-runs the archive stage.
|
||||
- `promote`, `promoted`, and `promote_artifacts` describe configured top-level
|
||||
remote output writes.
|
||||
|
||||
This mixed vocabulary makes the public contract harder to explain. Operators
|
||||
should not need to distinguish "archive the run", "publish the run", and
|
||||
"promote artifacts" when these are all part of the same publish action.
|
||||
|
||||
The public model should use:
|
||||
|
||||
- `publish` for the stage, command, config section, and action;
|
||||
- `published` for an expected remote output that exists at its top-level
|
||||
current destination;
|
||||
- `publish rules` for the configured source-to-destination output rules;
|
||||
- `locked` for sources whose top-level published destination must not be
|
||||
overwritten;
|
||||
- `run history` for immutable per-run records under `runs/<run_id>/`.
|
||||
|
||||
## Target Model
|
||||
|
||||
The public stage is `publish`.
|
||||
|
||||
The convenience command:
|
||||
|
||||
narratio publish <session_id>
|
||||
|
||||
is equivalent to:
|
||||
|
||||
narratio run-stage publish <session_id> --force
|
||||
|
||||
Pipeline configuration uses `publish`:
|
||||
|
||||
publish:
|
||||
enabled: true
|
||||
upload_run: true
|
||||
outputs:
|
||||
- source: narratio.transcript.final_trimmed
|
||||
- source: narratio.artifact.session_recap
|
||||
locks:
|
||||
- source: narratio.artifact.session_recap
|
||||
reason: Final recap was manually edited.
|
||||
|
||||
Publish output rules are source-based. Each rule writes one artifact source to
|
||||
a top-level remote destination. If `dest` is omitted, Narratio derives the
|
||||
destination from the artifact registry or configured artifact output path.
|
||||
|
||||
The mutable remote lock store remains:
|
||||
|
||||
{session_prefix}/locks.yml
|
||||
|
||||
Remote availability output uses `published`:
|
||||
|
||||
Published:
|
||||
- narratio.transcript.final_trimmed remote=published
|
||||
- narratio.artifact.session_recap locked remote=published
|
||||
|
||||
The remote key layout is otherwise unchanged:
|
||||
|
||||
- immutable run history stays under `{session_prefix}/runs/{run_id}/`;
|
||||
- current state stays under `{session_prefix}/current/manifest.json`;
|
||||
- the final commit marker stays `{session_prefix}/current/run_id.txt`;
|
||||
- `current/run_id.txt` is still written last.
|
||||
|
||||
## Compatibility Policy
|
||||
|
||||
This is a hard cutover.
|
||||
|
||||
After implementation:
|
||||
|
||||
- `pipeline.archive` is rejected by strict YAML decoding.
|
||||
- `pipeline.archive.promote_artifacts` is rejected.
|
||||
- `pipeline.workspace.cleanup_after_archive` is rejected.
|
||||
- `pipeline.spool.delete_audio_after_archive` is rejected.
|
||||
- `narratio run-stage archive <session_id>` is an unknown stage.
|
||||
- manifests that record an `archive` stage are not migrated.
|
||||
- old archive/promotion metadata keys are not read as compatibility fallbacks.
|
||||
|
||||
Existing remote objects are not moved or renamed. Remote layout remains stable;
|
||||
the rename changes configuration, stage names, status output, metadata, helper
|
||||
names, tests, examples, and documentation.
|
||||
|
||||
## Implementation Stages
|
||||
|
||||
### Stage 1: Public Schema and Stage Cutover
|
||||
|
||||
Status: Planned
|
||||
|
||||
Switch the public config and stage contract to publish terminology.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Replace `pipeline.archive` with `pipeline.publish`.
|
||||
- Replace `archive.promote_artifacts` with `publish.outputs`.
|
||||
- Keep output rule fields:
|
||||
- `source`
|
||||
- `dest`
|
||||
- `required`
|
||||
- Replace `pipeline.archive.locks` with `pipeline.publish.locks`.
|
||||
- Rename post-publish cleanup fields:
|
||||
- `pipeline.workspace.cleanup_after_publish`
|
||||
- `pipeline.spool.delete_audio_after_publish`
|
||||
- Rename the registered stage from `archive` to `publish`.
|
||||
- Update stage order so `publish` runs after `analyze` and before `notify`.
|
||||
- Update top-level `narratio publish` to target stage `publish`.
|
||||
- Keep `run-stage --artifacts <names> publish` support.
|
||||
- Reject `run-stage --artifacts <names>` for stages other than `analyze` and
|
||||
`publish`.
|
||||
- Preserve the remote commit ordering and storage adapter boundaries.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- `narratio run-stage publish <session_id>` executes the publish stage.
|
||||
- `narratio publish <session_id>` force-runs the publish stage.
|
||||
- `narratio run-stage archive <session_id>` fails clearly as an unknown stage.
|
||||
- Old archive config fields fail strict decoding.
|
||||
- New publish config fields load, default, and validate.
|
||||
|
||||
### Stage 2: Runtime Terminology and Metadata Cutover
|
||||
|
||||
Status: Planned
|
||||
|
||||
Rename implementation concepts and runtime output to publish terminology.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Rename archive/promotion config and runtime types conceptually to
|
||||
publish/output terms.
|
||||
- Rename the remote key helper intent from promoted artifact to published
|
||||
output while keeping generated keys unchanged.
|
||||
- Change helper output:
|
||||
- `Promoted:` becomes `Published:`
|
||||
- `remote=promoted` becomes `remote=published`
|
||||
- lock output uses `published` / `not-published`
|
||||
- Rename publish-stage metadata, including:
|
||||
- `promoted_paths` to `published_paths`
|
||||
- `promoted_files_uploaded` to `published_files_uploaded`
|
||||
- `skipped_optional_promotions` to `skipped_optional_outputs`
|
||||
- `skipped_unselected_promotions` to `skipped_unselected_outputs`
|
||||
- `locked_promotion_count` to `locked_output_count`
|
||||
- `locked_promotions` to `locked_outputs`
|
||||
- Update previous-cache and restore logic to use the `publish` stage and
|
||||
`published_paths` metadata only.
|
||||
- Keep run-local stage output materialization separate from remote publish
|
||||
terminology. If local helper names are confusing, rename them to
|
||||
materialization-oriented names rather than publish names.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- Status and artifact helper output use `Published:` and `remote=published`.
|
||||
- Publish metadata contains only publish/output terminology.
|
||||
- Previous-cache and restore behavior works with publish metadata and does not
|
||||
depend on old archive metadata.
|
||||
- Storage adapters still receive explicit keys and no AWS SDK details leak into
|
||||
app or stage logic.
|
||||
|
||||
### Stage 3: Documentation, Examples, and Final Cleanup
|
||||
|
||||
Status: Planned
|
||||
|
||||
Update implemented-behavior docs and remove stale public terminology after the
|
||||
runtime cutover lands.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Update current-behavior docs:
|
||||
- `docs/config.md`
|
||||
- `docs/cli.md`
|
||||
- `docs/operations.md`
|
||||
- `docs/troubleshooting.md`
|
||||
- `docs/architecture.md`
|
||||
- relevant files under `docs/internal/`
|
||||
- Rename `docs/internal/stage-archive.md` to
|
||||
`docs/internal/stage-publish.md`.
|
||||
- Update internal documentation links and references.
|
||||
- Update examples to use:
|
||||
- `publish.outputs`
|
||||
- `publish.locks`
|
||||
- `cleanup_after_publish`
|
||||
- `delete_audio_after_publish`
|
||||
- Update tests and final searches so old terminology remains only in this
|
||||
roadmap as historical context.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- Maintained examples load and validate.
|
||||
- Current-behavior docs describe only implemented publish terminology.
|
||||
- Internal docs describe run history, published outputs, locks, and current
|
||||
commit ordering clearly.
|
||||
- Old user-facing archive/promote wording is removed except where discussing
|
||||
historical behavior in this roadmap.
|
||||
|
||||
## Test Guidance
|
||||
|
||||
Focused tests:
|
||||
|
||||
- `go test ./internal/config -v`
|
||||
- `go test ./internal/app -v`
|
||||
- `go test ./internal/stage -v`
|
||||
- `go test ./internal/artifacts -v`
|
||||
|
||||
Full validation:
|
||||
|
||||
- `go test ./...`
|
||||
|
||||
Config tests to add or update:
|
||||
|
||||
- `publish.outputs` defaults and validates.
|
||||
- `publish.outputs[].dest` derives from the artifact registry when omitted.
|
||||
- `publish.locks` validates with the same source rules as publish outputs.
|
||||
- old `archive` fails strict decode.
|
||||
- old `promote_artifacts` fails strict decode.
|
||||
- old cleanup fields fail strict decode.
|
||||
|
||||
App and stage tests to add or update:
|
||||
|
||||
- stage order uses `publish` before `notify`.
|
||||
- `run-stage publish` succeeds.
|
||||
- `run-stage archive` fails clearly.
|
||||
- `narratio publish` force-runs the `publish` stage.
|
||||
- `--artifacts` is accepted for `run-stage publish`.
|
||||
- `--artifacts` error text names `analyze` and `publish`.
|
||||
- status and artifact list output show `Published:` and `remote=published`.
|
||||
- lock output says `published` or `not-published`.
|
||||
- previous-cache and restore use `publish` stage metadata.
|
||||
|
||||
Final searches:
|
||||
|
||||
- Config/stage names:
|
||||
- `pipeline.archive`
|
||||
- `archive:`
|
||||
- `promote_artifacts`
|
||||
- `cleanup_after_archive`
|
||||
- `delete_audio_after_archive`
|
||||
- User-facing output:
|
||||
- `Promoted:`
|
||||
- `remote=promoted`
|
||||
- `not-promoted`
|
||||
- Runtime symbols and metadata:
|
||||
- `ArchiveConfig`
|
||||
- `ArchivePromotionRule`
|
||||
- `S3PromotedArtifactKey`
|
||||
- `promoted_paths`
|
||||
- `promoted_files_uploaded`
|
||||
- `locked_promotions`
|
||||
|
||||
Expected remaining matches should be limited to this roadmap and narrowly
|
||||
justified historical references until the roadmap is fully retired.
|
||||
|
||||
## Architecture Guardrails
|
||||
|
||||
- Keep Narratio explicit and stage-driven.
|
||||
- Do not introduce a generic workflow or DAG abstraction.
|
||||
- Keep strict YAML decoding.
|
||||
- Keep remote path construction centralized.
|
||||
- Keep storage details behind `storage.ObjectStore`.
|
||||
- Keep AWS SDK types inside storage adapters.
|
||||
- Preserve manifest-driven resume and restore behavior.
|
||||
- Preserve current-state commit ordering with `current/run_id.txt` written
|
||||
last.
|
||||
- Keep raw secrets out of configs, manifests, logs, generated configs, and
|
||||
publish metadata.
|
||||
- Keep planned behavior only in this roadmap until implementation lands.
|
||||
|
||||
## Assumptions
|
||||
|
||||
- This is a breaking public/config/stage contract change.
|
||||
- No compatibility aliases are retained.
|
||||
- No migration logic is needed for in-progress local manifests.
|
||||
- No migration logic is needed for old remote manifests.
|
||||
- Existing remote objects are not moved or renamed.
|
||||
- `publish` means uploading run history, writing configured published outputs,
|
||||
and committing current state.
|
||||
- `run history` is the preferred term for immutable per-run records under
|
||||
`runs/<run_id>/`.
|
||||
- `archive` remains acceptable only as a generic English concept in historical
|
||||
roadmap context, not as a public Narratio command, config field, stage name,
|
||||
or metadata term after implementation.
|
||||
@@ -1,762 +0,0 @@
|
||||
# Roadmap: Runtime-Defined Scriptorium Artifacts
|
||||
|
||||
## Status
|
||||
|
||||
Implementation roadmap for a pre-release hard cutover.
|
||||
|
||||
## Purpose
|
||||
|
||||
Narratio currently treats artifact generation as a narrow `analyze` stage that supports a hard-coded `session_recap` artifact. This roadmap describes how to generalize artifact generation so operators can define Scriptorium-backed output artifacts at runtime through `pipeline.yml`.
|
||||
|
||||
The goal is to keep Narratio as a fixed pipeline orchestrator while making the artifact generation step configurable, composable, deterministic, and easy to regenerate selectively.
|
||||
|
||||
## Desired Outcome
|
||||
|
||||
Operators should be able to define artifacts such as session recaps, player handouts, NPC summaries, quest logs, entity maps, or other campaign-specific outputs without changing Narratio code.
|
||||
|
||||
A configured artifact is declared under:
|
||||
|
||||
```text
|
||||
pipeline.scriptorium.artifacts.<name>
|
||||
```
|
||||
|
||||
Each configured artifact becomes a canonical runtime artifact source ID:
|
||||
|
||||
```text
|
||||
narratio.artifact.<name>
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
artifacts:
|
||||
session_recap:
|
||||
enabled: true
|
||||
prompt_id: dnd_session.session_recap
|
||||
output_path: artifacts/session_recap.md
|
||||
inputs:
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
required: true
|
||||
```
|
||||
|
||||
This artifact is addressable by later artifacts as:
|
||||
|
||||
```text
|
||||
narratio.artifact.session_recap
|
||||
```
|
||||
|
||||
A dependent artifact can then consume it explicitly:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
artifacts:
|
||||
player_handout:
|
||||
enabled: true
|
||||
depends_on:
|
||||
- session_recap
|
||||
prompt_id: dnd_session.player_handout
|
||||
output_path: artifacts/player_handout.md
|
||||
inputs:
|
||||
recap:
|
||||
source: narratio.artifact.session_recap
|
||||
required: true
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
required: true
|
||||
```
|
||||
|
||||
## Resolved Design Decisions
|
||||
|
||||
The following decisions are settled for the initial implementation:
|
||||
|
||||
1. Configured artifact outputs must live under Narratio's internal artifact output directory, initially `artifacts/`.
|
||||
2. The artifact output directory should be defined as an internal default in `internal/config/defaults.go`, but no public configuration knob should be exposed yet.
|
||||
3. Artifact `output_path` should remain explicit in the initial implementation to avoid guessing file extensions or output formats.
|
||||
4. A disabled artifact may still be referenced as an input if its declared output already exists on disk and passes basic validation.
|
||||
5. A disabled artifact is not executable during the current analyze run.
|
||||
6. Artifact-to-artifact references require an explicit `depends_on` entry. Narratio should fail fast if the dependency declaration is missing.
|
||||
7. The manifest remains stage-oriented: `analyze` succeeds or fails as a full stage.
|
||||
8. Analyze-stage metadata may record per-artifact output details for provenance and later resolution, but not for intra-stage resume semantics.
|
||||
9. `--artifacts` should be added as a CLI filter for selective artifact generation.
|
||||
10. `--artifacts` does not imply `--force`; it only changes which configured artifacts are treated as executable when `analyze` actually runs.
|
||||
11. Because Narratio is still pre-release, the hard-coded `session_recap` behavior should be removed immediately rather than deprecated gradually.
|
||||
|
||||
## Scope
|
||||
|
||||
This roadmap covers:
|
||||
|
||||
- introducing a runtime artifact catalog;
|
||||
- generalizing configured Scriptorium artifact execution;
|
||||
- supporting `narratio.artifact.<name>` source IDs;
|
||||
- adding explicit artifact dependencies;
|
||||
- supporting disabled-but-resolvable artifact inputs;
|
||||
- adding selective artifact execution via `--artifacts`;
|
||||
- recording generated artifacts in analyze-stage metadata and/or manifest outputs;
|
||||
- removing hard-coded `session_recap` behavior;
|
||||
- updating tests and documentation.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This feature should not turn Narratio into a general workflow engine.
|
||||
|
||||
The initial implementation should not add:
|
||||
|
||||
- arbitrary shell-command artifacts;
|
||||
- arbitrary user-defined stages;
|
||||
- loops or conditional branching;
|
||||
- automatic archive promotion of generated artifacts;
|
||||
- semantic knowledge of particular artifact types;
|
||||
- per-artifact resume semantics within a successful or failed analyze stage;
|
||||
- automatic dependency inference without `depends_on`.
|
||||
|
||||
Narratio should continue to orchestrate a fixed pipeline. The configurable part is the set of Scriptorium artifact invocations performed during the `analyze` stage.
|
||||
|
||||
## Current State
|
||||
|
||||
Narratio already has several relevant pieces in place:
|
||||
|
||||
- `pipeline.scriptorium.artifacts` is modeled as a map of artifact definitions.
|
||||
- The Scriptorium adapter already accepts generic run/render requests.
|
||||
- The artifact resolver already understands canonical artifact source IDs.
|
||||
- The `analyze` stage already resolves inputs, optionally runs render-debug, invokes Scriptorium, verifies output, and records metadata.
|
||||
|
||||
The main limitation is that `analyze` currently treats `session_recap` as the only executable artifact and rejects other enabled artifact definitions.
|
||||
|
||||
## Target Architecture
|
||||
|
||||
### Runtime Artifact Catalog
|
||||
|
||||
Introduce a per-run artifact catalog that tracks built-in artifacts and configured artifacts.
|
||||
|
||||
Conceptually:
|
||||
|
||||
```text
|
||||
ArtifactCatalog
|
||||
├── built-in artifacts
|
||||
│ ├── narratio.transcript.merged
|
||||
│ ├── narratio.transcript.polished
|
||||
│ ├── narratio.transcript.full
|
||||
│ ├── narratio.transcript.trimmed
|
||||
│ └── narratio.bounds.session
|
||||
│
|
||||
└── configured artifacts
|
||||
├── narratio.artifact.session_recap
|
||||
├── narratio.artifact.player_handout
|
||||
└── narratio.artifact.npc_summary
|
||||
```
|
||||
|
||||
The catalog should distinguish between three states:
|
||||
|
||||
```text
|
||||
planned valid configured or built-in artifact known to Narratio
|
||||
available artifact has been produced or otherwise resolved
|
||||
executable configured artifact selected for execution in this analyze run
|
||||
```
|
||||
|
||||
Configured artifacts can be planned without being executable. This distinction is important for disabled artifacts and for `--artifacts` filtering.
|
||||
|
||||
### Configured Artifact Source IDs
|
||||
|
||||
Configured artifact keys map directly to source IDs:
|
||||
|
||||
```text
|
||||
pipeline.scriptorium.artifacts.<name>
|
||||
→ narratio.artifact.<name>
|
||||
```
|
||||
|
||||
`session_recap` should no longer be a special built-in analyze artifact. Instead, it is just a conventional configured artifact key:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
artifacts:
|
||||
session_recap:
|
||||
enabled: true
|
||||
prompt_id: dnd_session.session_recap
|
||||
output_path: artifacts/session_recap.md
|
||||
```
|
||||
|
||||
`narratio.artifact.session_recap` remains valid only because `session_recap` is configured.
|
||||
|
||||
### Artifact Output Directory
|
||||
|
||||
Add an internal default artifact output directory, initially:
|
||||
|
||||
```text
|
||||
artifacts
|
||||
```
|
||||
|
||||
This default should live in `internal/config/defaults.go` or the existing equivalent defaults location.
|
||||
|
||||
For the initial implementation:
|
||||
|
||||
- expose no public config knob for the artifact output directory;
|
||||
- require each configured artifact to provide an explicit `output_path`;
|
||||
- validate that each configured artifact `output_path` is run-relative;
|
||||
- validate that each configured artifact `output_path` is under the internal artifact output directory;
|
||||
- reject output paths that escape the run workspace or use path traversal.
|
||||
|
||||
This preserves future configurability without forcing Narratio to guess output extensions or formats now.
|
||||
|
||||
### Enabled, Disabled, and Selected Artifacts
|
||||
|
||||
Configured artifacts should have three distinct execution states:
|
||||
|
||||
```text
|
||||
enabled by config artifact has enabled: true
|
||||
selected for execution artifact remains executable after --artifacts filtering
|
||||
disabled for execution artifact is not executable, but may be resolvable from disk
|
||||
```
|
||||
|
||||
Without `--artifacts`, all configured artifacts with `enabled: true` are selected for execution.
|
||||
|
||||
With `--artifacts`, only the named artifacts are selected for execution. All other configured artifacts are treated as disabled for the current analyze invocation, regardless of their configured `enabled` value.
|
||||
|
||||
Disabled artifacts may still be resolved as inputs if their configured `output_path` exists on disk and passes validation.
|
||||
|
||||
### Disabled Artifact Resolution
|
||||
|
||||
If artifact `B` references artifact `A`, and `A` is disabled for execution, Narratio should attempt to resolve `A` from disk.
|
||||
|
||||
This should succeed only when:
|
||||
|
||||
1. `A` is defined in `pipeline.scriptorium.artifacts`;
|
||||
2. `A` has a valid `output_path`;
|
||||
3. the output path exists in the current run workspace;
|
||||
4. the output is non-empty, or otherwise passes any available artifact-specific validation.
|
||||
|
||||
The resolved provenance should make the source clear, for example:
|
||||
|
||||
```text
|
||||
filesystem.disabled_artifact_output
|
||||
```
|
||||
|
||||
If the file does not exist or fails validation, the dependent artifact should fail before invoking Scriptorium.
|
||||
|
||||
Example error wording:
|
||||
|
||||
```text
|
||||
artifact player_handout requires narratio.artifact.session_recap, but session_recap is disabled for execution and artifacts/session_recap.md does not exist
|
||||
```
|
||||
|
||||
### Explicit Dependencies
|
||||
|
||||
Artifact-to-artifact references require explicit `depends_on` entries.
|
||||
|
||||
If artifact `B` has an input source of `narratio.artifact.A`, then `B.depends_on` must include `A`.
|
||||
|
||||
This should fail:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
artifacts:
|
||||
player_handout:
|
||||
enabled: true
|
||||
prompt_id: dnd_session.player_handout
|
||||
output_path: artifacts/player_handout.md
|
||||
inputs:
|
||||
recap:
|
||||
source: narratio.artifact.session_recap
|
||||
required: true
|
||||
```
|
||||
|
||||
This should pass:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
artifacts:
|
||||
player_handout:
|
||||
enabled: true
|
||||
depends_on:
|
||||
- session_recap
|
||||
prompt_id: dnd_session.player_handout
|
||||
output_path: artifacts/player_handout.md
|
||||
inputs:
|
||||
recap:
|
||||
source: narratio.artifact.session_recap
|
||||
required: true
|
||||
```
|
||||
|
||||
`depends_on` values refer to configured artifact keys, not full source IDs.
|
||||
|
||||
Dependency validation should fail on:
|
||||
|
||||
- references to unknown artifact keys;
|
||||
- missing `depends_on` entries for artifact-to-artifact input references;
|
||||
- self-dependencies;
|
||||
- dependency cycles among executable artifacts.
|
||||
|
||||
Dependencies on disabled artifacts are permitted, but the disabled dependency must resolve from disk before the dependent artifact runs.
|
||||
|
||||
### Execution Order
|
||||
|
||||
The analyze stage should execute selected artifacts in dependency order.
|
||||
|
||||
Rules:
|
||||
|
||||
- selected artifacts are executable;
|
||||
- disabled artifacts are never executed;
|
||||
- selected artifacts may depend on other selected artifacts;
|
||||
- selected artifacts may depend on disabled artifacts if those disabled artifacts resolve from disk;
|
||||
- independent selected artifacts run in deterministic sorted-name order.
|
||||
|
||||
Use topological sorting over selected artifacts, while validating dependency references across the full configured artifact set.
|
||||
|
||||
### Input Resolution
|
||||
|
||||
Input resolution should use the artifact catalog and existing artifact resolver behavior.
|
||||
|
||||
For each configured artifact input:
|
||||
|
||||
- built-in sources resolve through existing resolver behavior;
|
||||
- `previous_session_artifact` preserves existing behavior;
|
||||
- `narratio.artifact.<name>` resolves through the runtime artifact catalog;
|
||||
- selected dependencies resolve after being produced earlier in the same analyze execution;
|
||||
- disabled dependencies resolve from their configured output path on disk;
|
||||
- optional missing inputs are omitted;
|
||||
- required missing inputs fail before Scriptorium is invoked.
|
||||
|
||||
### Analyze Stage Generalization
|
||||
|
||||
The `analyze` stage should become the generic Scriptorium artifact stage.
|
||||
|
||||
High-level flow:
|
||||
|
||||
1. Load configured Scriptorium artifacts.
|
||||
2. Apply the `--artifacts` filter, if present.
|
||||
3. If no artifacts are selected for execution, return success metadata with `skipped=true`.
|
||||
4. Build the runtime artifact catalog.
|
||||
5. Validate artifact names, output paths, source IDs, dependencies, selected artifacts, and required fields.
|
||||
6. Resolve any disabled dependencies that are required by selected artifacts.
|
||||
7. Sort selected artifacts by dependency order.
|
||||
8. For each selected artifact:
|
||||
- resolve configured inputs;
|
||||
- build the Scriptorium run request;
|
||||
- optionally run Scriptorium render-debug;
|
||||
- run Scriptorium;
|
||||
- fail on validation-failed result;
|
||||
- verify the output exists and is non-empty;
|
||||
- record artifact output metadata;
|
||||
- register `narratio.artifact.<name>` as available in the catalog.
|
||||
9. Return aggregate analyze-stage metadata containing all generated and reused artifacts relevant to the run.
|
||||
|
||||
The Scriptorium adapter should remain generic. It should not decide which artifacts run, how dependencies work, or how artifacts are registered.
|
||||
|
||||
### Manifest and Metadata
|
||||
|
||||
The manifest should remain stage-oriented.
|
||||
|
||||
This means:
|
||||
|
||||
- `analyze` succeeds or fails as a full stage;
|
||||
- if `analyze` has already succeeded and the user does not force it, the runner skips it as a full stage;
|
||||
- Narratio should not implement per-artifact resume in the first version.
|
||||
|
||||
However, analyze-stage metadata should still record artifact outputs for provenance and future resolution.
|
||||
|
||||
Recommended metadata shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"skipped": false,
|
||||
"artifacts": [
|
||||
{
|
||||
"name": "session_recap",
|
||||
"source_id": "narratio.artifact.session_recap",
|
||||
"output_kind": "scriptorium_artifact",
|
||||
"path": "artifacts/session_recap.md",
|
||||
"prompt_id": "dnd_session.session_recap",
|
||||
"profile_id": "local-gemma-31b",
|
||||
"provenance": "generated.current_analyze_run"
|
||||
},
|
||||
{
|
||||
"name": "player_handout",
|
||||
"source_id": "narratio.artifact.player_handout",
|
||||
"output_kind": "scriptorium_artifact",
|
||||
"path": "artifacts/player_handout.md",
|
||||
"prompt_id": "dnd_session.player_handout",
|
||||
"profile_id": "local-gemma-31b",
|
||||
"provenance": "generated.current_analyze_run"
|
||||
}
|
||||
],
|
||||
"reused_artifacts": [
|
||||
{
|
||||
"name": "session_recap",
|
||||
"source_id": "narratio.artifact.session_recap",
|
||||
"path": "artifacts/session_recap.md",
|
||||
"provenance": "filesystem.disabled_artifact_output"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The exact struct can differ from this example, but it should preserve:
|
||||
|
||||
- artifact name;
|
||||
- canonical source ID;
|
||||
- output path;
|
||||
- prompt/profile provenance for generated artifacts;
|
||||
- reused-vs-generated provenance.
|
||||
|
||||
### Resume and Force Behavior
|
||||
|
||||
Keep resume behavior stage-level.
|
||||
|
||||
Recommended semantics:
|
||||
|
||||
```text
|
||||
No --force, analyze already succeeded:
|
||||
runner skips analyze, regardless of --artifacts.
|
||||
|
||||
--force, no --artifacts:
|
||||
analyze regenerates all configured artifacts with enabled: true.
|
||||
|
||||
--force --artifacts player_handout:
|
||||
analyze treats only player_handout as executable.
|
||||
all other configured artifacts are disabled for execution.
|
||||
disabled dependencies may be reused from disk.
|
||||
|
||||
--artifacts player_handout on a not-yet-completed analyze stage:
|
||||
analyze runs only player_handout.
|
||||
disabled dependencies may be reused from disk.
|
||||
```
|
||||
|
||||
`--artifacts` should not imply `--force`. It is an execution filter, not a resume override.
|
||||
|
||||
### `--artifacts` CLI Flag
|
||||
|
||||
Add an `--artifacts` flag to commands that can execute or resume the analyze stage.
|
||||
|
||||
The flag should accept one or more configured artifact names. Internally, normalize values to a set of artifact keys.
|
||||
|
||||
Recommended behavior:
|
||||
|
||||
- validate all requested artifact names against `pipeline.scriptorium.artifacts`;
|
||||
- reject unknown artifact names before running stages;
|
||||
- treat requested artifacts as the only executable artifacts for the analyze stage;
|
||||
- treat all other configured artifacts as disabled for execution;
|
||||
- allow disabled artifacts to satisfy dependencies from disk as described above;
|
||||
- if `--artifacts` is used while executing a stage other than `analyze`, either reject it or ignore it with a clear validation error. Prefer rejection.
|
||||
|
||||
The exact CLI parsing style can follow Narratio's existing conventions. Both comma-separated and repeatable values are acceptable if the CLI package supports them cleanly, but the internal representation should be a set of artifact keys.
|
||||
|
||||
### Archive Behavior
|
||||
|
||||
Do not automatically archive every generated artifact.
|
||||
|
||||
Artifact generation and archive promotion should remain separate concerns. Operators should continue to use `archive.promote_artifacts` to decide which generated files should be promoted or uploaded.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
archive:
|
||||
promote_artifacts:
|
||||
- from: artifacts/session_recap.md
|
||||
to: artifacts/session_recap.md
|
||||
required: true
|
||||
- from: artifacts/player_handout.md
|
||||
to: artifacts/player_handout.md
|
||||
required: false
|
||||
```
|
||||
|
||||
A later enhancement may add opt-in automatic promotion of configured artifacts, but explicit promotion should remain the default.
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Config Model and Defaults
|
||||
|
||||
Add or update the configured artifact model to include:
|
||||
|
||||
- `enabled`;
|
||||
- `depends_on`;
|
||||
- `prompt_id`;
|
||||
- `profile_id`;
|
||||
- `output_path`;
|
||||
- `timeout`;
|
||||
- `render_debug`;
|
||||
- `inputs`;
|
||||
- `vars`.
|
||||
|
||||
Add an internal default artifact output directory in `internal/config/defaults.go`, initially set to `artifacts`.
|
||||
|
||||
Validation rules:
|
||||
|
||||
- artifact names must match a conservative identifier pattern such as `^[a-z][a-z0-9_]*$`;
|
||||
- selected/executable artifacts require `prompt_id` and `output_path`;
|
||||
- configured artifacts that may be referenced while disabled require `output_path`;
|
||||
- configured artifact output paths must be run-relative;
|
||||
- configured artifact output paths must live under the internal artifact output directory;
|
||||
- configured artifact output paths must not escape the run workspace;
|
||||
- `narratio.artifact.<name>` input sources must refer to configured artifact keys;
|
||||
- any `narratio.artifact.<name>` input source must have a matching `depends_on` entry;
|
||||
- `depends_on` entries must refer to configured artifact keys;
|
||||
- dependencies must not contain self-references or executable cycles;
|
||||
- input names and var names must remain compatible with the Scriptorium adapter's validation rules;
|
||||
- unknown YAML fields must continue to fail strict decode.
|
||||
|
||||
Tests:
|
||||
|
||||
- valid single configured artifact;
|
||||
- valid multiple independent artifacts;
|
||||
- valid artifact-to-artifact dependency;
|
||||
- valid dependency on disabled artifact with output path;
|
||||
- invalid artifact name;
|
||||
- missing required fields;
|
||||
- output path outside `artifacts/`;
|
||||
- dependency on missing artifact;
|
||||
- missing `depends_on` for artifact input source;
|
||||
- self-dependency;
|
||||
- cycle detection;
|
||||
- typo in `narratio.artifact.<name>` source;
|
||||
- unknown YAML fields still fail strict decode.
|
||||
|
||||
### Phase 2: CLI Filtering
|
||||
|
||||
Add the `--artifacts` flag and carry the selected artifact set into the run execution options.
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- parse values according to existing CLI conventions;
|
||||
- normalize to artifact key strings;
|
||||
- validate against configured artifact definitions after config load;
|
||||
- make the selected set available to the analyze stage;
|
||||
- reject use with commands or stages where analyze cannot run.
|
||||
|
||||
Tests:
|
||||
|
||||
- no `--artifacts` means all enabled artifacts are selected;
|
||||
- one requested artifact is selected;
|
||||
- multiple requested artifacts are selected;
|
||||
- unknown requested artifact fails;
|
||||
- `--artifacts` does not imply `--force`;
|
||||
- `--artifacts` with already-succeeded analyze stage is skipped unless forced;
|
||||
- `--artifacts` on unsupported stage command fails clearly.
|
||||
|
||||
### Phase 3: Runtime Artifact Catalog
|
||||
|
||||
Introduce an internal artifact catalog abstraction.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- register built-in artifact definitions;
|
||||
- register configured artifact definitions;
|
||||
- map configured artifact keys to `narratio.artifact.<name>` IDs;
|
||||
- track planned, available, and executable artifact states;
|
||||
- expose lookup by canonical source ID;
|
||||
- record generated provenance;
|
||||
- record disabled-from-disk provenance.
|
||||
|
||||
Keep the catalog narrow. It should not execute Scriptorium and should not understand prompt semantics.
|
||||
|
||||
Tests:
|
||||
|
||||
- built-in source lookup;
|
||||
- configured source registration;
|
||||
- duplicate/conflicting source handling;
|
||||
- planned but unavailable artifact lookup;
|
||||
- selected artifact state;
|
||||
- disabled artifact state;
|
||||
- registering an artifact as available after generation;
|
||||
- registering a disabled artifact as available from disk;
|
||||
- resolving a configured artifact from analyze metadata if that behavior is implemented.
|
||||
|
||||
### Phase 4: Resolver Integration
|
||||
|
||||
Update artifact resolution so configured artifact IDs are resolved through the runtime catalog.
|
||||
|
||||
Resolution behavior:
|
||||
|
||||
- built-in sources continue using existing resolver behavior;
|
||||
- configured artifact sources resolve from catalog availability/provenance;
|
||||
- selected configured artifacts become available after generation;
|
||||
- disabled configured artifacts may become available from disk;
|
||||
- missing optional configured artifact inputs are omitted;
|
||||
- missing required configured artifact inputs fail clearly.
|
||||
|
||||
Tests:
|
||||
|
||||
- configured artifact consumes a built-in transcript source;
|
||||
- configured artifact consumes another configured artifact produced earlier in the same analyze run;
|
||||
- configured artifact consumes a disabled artifact resolved from disk;
|
||||
- required disabled artifact missing on disk fails;
|
||||
- required configured artifact missing fails;
|
||||
- optional missing configured artifact is omitted;
|
||||
- reused artifact provenance is recorded distinctly from generated artifact provenance.
|
||||
|
||||
### Phase 5: Analyze Stage Generalization
|
||||
|
||||
Refactor `analyze` to execute selected configured artifacts.
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- remove the hard-coded `session_recap` selection path;
|
||||
- remove the hard-coded rejection of non-`session_recap` artifacts;
|
||||
- preserve skip behavior when Scriptorium config is absent or no artifacts are selected;
|
||||
- build the runtime artifact catalog;
|
||||
- apply `--artifacts` filtering;
|
||||
- validate selected artifacts and their dependencies;
|
||||
- pre-resolve disabled dependencies from disk where required;
|
||||
- compute deterministic dependency order;
|
||||
- execute selected artifacts one at a time in dependency order;
|
||||
- keep render-debug behavior at global and artifact levels;
|
||||
- keep Scriptorium adapter invocation generic;
|
||||
- after each successful run, register the artifact as available in the catalog;
|
||||
- aggregate generated and reused artifact metadata.
|
||||
|
||||
Tests:
|
||||
|
||||
- no Scriptorium config skips;
|
||||
- empty artifact map skips;
|
||||
- no selected artifacts skips;
|
||||
- disabled artifacts do not run;
|
||||
- one selected artifact runs;
|
||||
- multiple independent artifacts run in deterministic order;
|
||||
- dependent selected artifact receives prior selected artifact as input;
|
||||
- dependent selected artifact receives disabled-from-disk artifact as input;
|
||||
- render-debug works for configured artifacts;
|
||||
- Scriptorium validation failure fails the stage;
|
||||
- missing required input fails the stage;
|
||||
- successful outputs are non-empty and recorded;
|
||||
- artifact filter executes only requested artifacts.
|
||||
|
||||
### Phase 6: Manifest and Stage Metadata
|
||||
|
||||
Update analyze-stage metadata and manifest output recording to support dynamic configured artifacts.
|
||||
|
||||
Recommended behavior:
|
||||
|
||||
- every generated configured artifact gets `source_id: narratio.artifact.<name>`;
|
||||
- every generated configured artifact gets a generic output kind such as `scriptorium_artifact`;
|
||||
- reused disabled artifacts are recorded separately from generated artifacts;
|
||||
- metadata is sufficient for debugging, provenance, and future resolver support;
|
||||
- metadata does not create per-artifact resume semantics.
|
||||
|
||||
Because this is a pre-release hard cutover, do not preserve a special legacy `session_recap` output kind unless a current internal test or archive path still requires it temporarily. Prefer updating tests and examples to treat `session_recap` as an ordinary configured artifact.
|
||||
|
||||
Tests:
|
||||
|
||||
- metadata records one generated configured artifact;
|
||||
- metadata records multiple generated configured artifacts;
|
||||
- metadata records reused disabled artifact provenance;
|
||||
- `session_recap` is recorded as a normal configured artifact;
|
||||
- manifest still treats `analyze` as a single succeeded or failed stage;
|
||||
- runner skip behavior remains stage-level.
|
||||
|
||||
### Phase 7: Archive and Promotion Review
|
||||
|
||||
Review archive behavior after dynamic artifacts are recorded.
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- do not automatically promote every configured artifact;
|
||||
- keep `archive.promote_artifacts` explicit;
|
||||
- update default or example promotion rules to use configured `session_recap` output path;
|
||||
- ensure required promotion rules fail clearly when selected artifact generation did not produce a required file.
|
||||
|
||||
Tests:
|
||||
|
||||
- generated artifact can be promoted by explicit archive rule;
|
||||
- required archive promotion fails if selected artifact was not generated and no file exists;
|
||||
- optional archive promotion skips cleanly if file is absent;
|
||||
- hard cutover does not rely on hard-coded `session_recap` generation.
|
||||
|
||||
### Phase 8: Documentation and Examples
|
||||
|
||||
Status: complete.
|
||||
|
||||
Update documentation after the implementation is complete.
|
||||
|
||||
Recommended documentation changes:
|
||||
|
||||
- update `docs/config.md` with the generalized artifact configuration model;
|
||||
- update `docs/internal/artifacts.md` to describe the runtime artifact catalog;
|
||||
- update `docs/stages/analyze.md` to describe generic Scriptorium artifact generation;
|
||||
- update Scriptorium integration docs only if the adapter contract changes;
|
||||
- update full annotated pipeline examples;
|
||||
- add at least one example with multiple artifacts and one dependency;
|
||||
- document `--artifacts` behavior and its relationship to `--force`;
|
||||
- remove documentation stating that only `session_recap` is supported.
|
||||
|
||||
Documentation should make clear that:
|
||||
|
||||
- configured artifact source IDs use `narratio.artifact.<name>`;
|
||||
- `depends_on` uses artifact keys, not full source IDs;
|
||||
- artifact-to-artifact source references require explicit `depends_on`;
|
||||
- disabled artifacts can be reused from disk when required by selected artifacts;
|
||||
- `--artifacts` filters execution but does not imply `--force`;
|
||||
- archive promotion remains explicit;
|
||||
- per-artifact resume is not part of the initial implementation.
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
Because Narratio is pre-release, perform a hard cutover.
|
||||
|
||||
Required changes:
|
||||
|
||||
1. Remove the hard-coded `session_recap` analyze behavior.
|
||||
2. Require `session_recap` to be declared under `pipeline.scriptorium.artifacts.session_recap` if the operator wants a session recap.
|
||||
3. Treat `narratio.artifact.session_recap` as valid only when `session_recap` is a configured artifact key.
|
||||
4. Update config examples to show `session_recap` as a normal configured artifact.
|
||||
5. Update tests to stop assuming that `session_recap` is a built-in analyze artifact.
|
||||
6. Keep archive promotion explicit and path-based.
|
||||
|
||||
Example replacement config:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
binary: scriptorium
|
||||
config_path: /etc/scriptorium/config.yml
|
||||
timeout: 10m
|
||||
render_debug: false
|
||||
artifacts:
|
||||
session_recap:
|
||||
enabled: true
|
||||
prompt_id: dnd_session.session_recap
|
||||
profile_id: local-gemma-31b
|
||||
output_path: artifacts/session_recap.md
|
||||
timeout: 20m
|
||||
inputs:
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
required: true
|
||||
prior_recap:
|
||||
source: previous_session_artifact
|
||||
artifact: artifacts/session_recap.md
|
||||
required: false
|
||||
vars:
|
||||
artifact_title: Session Recap
|
||||
```
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
The feature is complete when:
|
||||
|
||||
- operators can define more than one enabled Scriptorium artifact in `pipeline.yml`;
|
||||
- Narratio runs selected artifacts in deterministic dependency order;
|
||||
- configured artifacts are addressable as `narratio.artifact.<name>`;
|
||||
- one configured artifact can consume another configured artifact as an input;
|
||||
- artifact-to-artifact input references require explicit `depends_on`;
|
||||
- disabled artifacts can satisfy dependencies from existing on-disk outputs;
|
||||
- missing required disabled artifacts fail clearly;
|
||||
- optional missing inputs are omitted;
|
||||
- `--artifacts` can selectively execute valid configured artifact names;
|
||||
- `--artifacts` does not imply `--force`;
|
||||
- render-debug behavior works for all configured artifacts;
|
||||
- generated and reused artifacts are recorded in analyze-stage metadata;
|
||||
- `session_recap` is no longer hard-coded and works as a normal configured artifact;
|
||||
- archive promotion remains explicit;
|
||||
- tests cover config validation, dependency sorting, disabled artifact resolution, resolver behavior, CLI filtering, analyze execution, archive interactions, and metadata.
|
||||
|
||||
## Suggested Implementation Order
|
||||
|
||||
1. Config model, defaults, and validation.
|
||||
2. CLI parsing and propagation of `--artifacts` selection.
|
||||
3. Runtime artifact catalog.
|
||||
4. Resolver integration for configured artifacts.
|
||||
5. Analyze stage generalization.
|
||||
6. Stage metadata and manifest output recording.
|
||||
7. Archive behavior review.
|
||||
8. Documentation and examples.
|
||||
|
||||
This order keeps the most static pieces first, then moves into execution behavior once the configuration contract is explicit and well tested.
|
||||
210
docs/roadmap/transcripts.md
Normal file
210
docs/roadmap/transcripts.md
Normal file
@@ -0,0 +1,210 @@
|
||||
# Roadmap: Transcript Artifact Naming
|
||||
|
||||
Status: Implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Narratio's built-in transcript artifact names and canonical paths currently mix
|
||||
operator-facing artifact meaning with historical stage and tool terminology:
|
||||
|
||||
- `narratio.transcript.merged` maps to `transcripts/merged.json`.
|
||||
- `narratio.transcript.polished` maps to `transcripts/processed.json`.
|
||||
- `narratio.transcript.full` maps to `transcripts/normalized.json`.
|
||||
- `narratio.transcript.trimmed` maps to `transcripts/trimmed.json`.
|
||||
|
||||
This makes the public artifact surface harder to reason about. Operators see
|
||||
`full`, `normalized`, `processed`, `polished`, `merged`, and `trimmed` used in
|
||||
different places for the same transcript lineage.
|
||||
|
||||
The transcript source IDs, canonical paths, and manifest output kinds should
|
||||
use one vocabulary based on each transcript's role in the session artifact
|
||||
model.
|
||||
|
||||
## Target Model
|
||||
|
||||
Built-in transcript artifacts should use these public source IDs, canonical
|
||||
paths, and manifest output kinds:
|
||||
|
||||
| Source ID | Canonical path | Output kind | Meaning |
|
||||
| --- | --- | --- | --- |
|
||||
| `narratio.transcript.base` | `transcripts/base.json` | `transcript_base` | First unified transcript produced by merging per-speaker raw transcripts. |
|
||||
| `narratio.transcript.polished` | `transcripts/polished.json` | `transcript_polished` | Audita-polished transcript. |
|
||||
| `narratio.transcript.final` | `transcripts/final.json` | `transcript_final` | Full final transcript after normalization. |
|
||||
| `narratio.transcript.final_trimmed` | `transcripts/final.trimmed.json` | `transcript_final_trimmed` | Trimmed version of the final transcript. |
|
||||
|
||||
Stage names remain process-oriented and unchanged:
|
||||
|
||||
- `merge`
|
||||
- `polish`
|
||||
- `normalize`
|
||||
- `trim`
|
||||
|
||||
Downstream adapter contracts also remain process-oriented. The rename changes
|
||||
Narratio's artifact model, canonical paths, config examples, archive promotion
|
||||
sources, lock sources, status output, and documentation. It should not rename
|
||||
the stages themselves or move external integration details into stage logic.
|
||||
|
||||
## Compatibility Policy
|
||||
|
||||
This is a hard cutover.
|
||||
|
||||
After implementation, these old source IDs should be rejected:
|
||||
|
||||
- `narratio.transcript.merged`
|
||||
- `narratio.transcript.full`
|
||||
- `narratio.transcript.trimmed`
|
||||
|
||||
These old canonical paths should not be compatibility fallbacks:
|
||||
|
||||
- `transcripts/merged.json`
|
||||
- `transcripts/processed.json`
|
||||
- `transcripts/normalized.json`
|
||||
- `transcripts/trimmed.json`
|
||||
|
||||
Existing remote archives are not migrated automatically. Operators who want
|
||||
new promoted keys for old sessions should republish those sessions after
|
||||
updating configuration.
|
||||
|
||||
## Implementation Stages
|
||||
|
||||
### Stage 1: Centralize Transcript Artifact Naming
|
||||
|
||||
Status: Implemented
|
||||
|
||||
Consolidate transcript artifact source IDs, canonical paths, and output kinds
|
||||
in the artifact/path layer before changing runtime behavior.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Add or consolidate constants/helpers for built-in transcript source IDs.
|
||||
- Add or consolidate constants/helpers for canonical transcript paths.
|
||||
- Add or consolidate constants/helpers for transcript manifest output kinds.
|
||||
- Keep source ID, path, and output-kind mappings in one registry or one
|
||||
obviously shared artifact model.
|
||||
- Update artifact registry tests to prove the target mapping.
|
||||
- Avoid changing stage output behavior in this stage unless the implementation
|
||||
is simpler and still reviewable.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- There is one clear source of truth for built-in transcript artifact names,
|
||||
paths, and output kinds.
|
||||
- Tests prove the new target mapping in the artifact layer.
|
||||
- No generic workflow abstraction is introduced.
|
||||
|
||||
### Stage 2: Rename Runtime Outputs and Defaults
|
||||
|
||||
Status: Implemented
|
||||
|
||||
Switch runtime behavior to the new transcript artifact model.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Update `merge` to write and record `transcripts/base.json` with
|
||||
`transcript_base`.
|
||||
- Update `polish` to write and record `transcripts/polished.json` with
|
||||
`transcript_polished`.
|
||||
- Update `normalize` to write and record `transcripts/final.json` with
|
||||
`transcript_final`.
|
||||
- Update `trim` to write and record `transcripts/final.trimmed.json` with
|
||||
`transcript_final_trimmed`.
|
||||
- Update normalize and trim defaults to:
|
||||
- `pipeline.normalize.output_path: transcripts/final.json`
|
||||
- `pipeline.trim.output_path: transcripts/final.trimmed.json`
|
||||
- Update built-in artifact resolution, archive promotion destination
|
||||
derivation, archive locks, status output, artifact catalog output,
|
||||
previous-cache resolution, restore planning, and restore execution to use
|
||||
the new registry values.
|
||||
- Ensure old source IDs fail config validation.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- New runs produce the target canonical transcript files.
|
||||
- Manifest outputs use the target output kinds.
|
||||
- Archive promotion and lock validation accept new source IDs and reject old
|
||||
source IDs.
|
||||
- Status and artifact listing display new source IDs.
|
||||
- Restore uses the new canonical paths and does not restore old transcript
|
||||
paths as canonical outputs.
|
||||
|
||||
### Stage 3: Update Tests, Examples, and Current Documentation
|
||||
|
||||
Status: Implemented
|
||||
|
||||
Update all implemented-behavior references after the runtime cutover lands.
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- Update examples to use `narratio.transcript.final_trimmed` and
|
||||
`transcripts/final.trimmed.json` where trimmed final transcript is intended.
|
||||
- Update examples that refer to full final transcripts to use
|
||||
`narratio.transcript.final` and `transcripts/final.json`.
|
||||
- Update `docs/config.md`, `docs/internal/artifacts.md`, stage docs,
|
||||
CLI examples, operations examples, archive examples, lock examples, and
|
||||
status/artifact-list examples.
|
||||
- Add strict validation tests proving old source IDs are rejected.
|
||||
- Mark roadmap stages implemented only after code, tests, examples, and
|
||||
current-behavior docs agree.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- Maintained examples load and validate.
|
||||
- Current-behavior docs describe only implemented new names.
|
||||
- Old names remain only in this roadmap as historical/planning context until
|
||||
this roadmap is retired or archived.
|
||||
|
||||
## Test Guidance
|
||||
|
||||
Run focused tests while implementing:
|
||||
|
||||
- `go test ./internal/artifacts -v`
|
||||
- `go test ./internal/config -v`
|
||||
- `go test ./internal/stage -v`
|
||||
- `go test ./internal/app -v`
|
||||
|
||||
Run full validation before finishing:
|
||||
|
||||
- `go test ./...`
|
||||
|
||||
Run final searches:
|
||||
|
||||
- Old source IDs:
|
||||
- `narratio.transcript.merged`
|
||||
- `narratio.transcript.full`
|
||||
- `narratio.transcript.trimmed`
|
||||
- Old paths:
|
||||
- `transcripts/merged.json`
|
||||
- `transcripts/processed.json`
|
||||
- `transcripts/normalized.json`
|
||||
- `transcripts/trimmed.json`
|
||||
- Old output kinds:
|
||||
- `transcript_merged`
|
||||
- `transcript_processed`
|
||||
- `transcript_normalized`
|
||||
- `transcript_trimmed`
|
||||
|
||||
Expected remaining matches should be limited to this roadmap's
|
||||
historical/planning references until the roadmap is fully completed.
|
||||
|
||||
## Architecture Guardrails
|
||||
|
||||
- Keep Narratio explicit and stage-driven; do not introduce a generic workflow
|
||||
or DAG abstraction.
|
||||
- Keep path and artifact naming in centralized helpers rather than scattered
|
||||
string concatenation.
|
||||
- Preserve manifest-driven resume behavior.
|
||||
- Keep storage details behind storage adapters.
|
||||
- Do not move Seriatim, Audita, or Scriptorium command details out of their
|
||||
adapter boundaries.
|
||||
- Keep current-behavior documentation in sync only after implementation lands;
|
||||
planned behavior belongs in this roadmap until then.
|
||||
|
||||
## Assumptions
|
||||
|
||||
- The cutover is intentionally not backward-compatible.
|
||||
- Existing remote archive objects are not renamed or migrated automatically.
|
||||
- Stage names and downstream adapter request field names remain unchanged.
|
||||
- The term `base` is preferred over `merged` for the first unified transcript.
|
||||
- The term `final` is preferred over `full` or `normalized` for the full final
|
||||
transcript.
|
||||
- The trimmed final path is `transcripts/final.trimmed.json`.
|
||||
@@ -6,46 +6,46 @@ Canonical operator troubleshooting guide for recurring implemented Narratio fail
|
||||
## Config file discovery failure
|
||||
|
||||
Symptom:
|
||||
- `run`, `plan`, `resume`, or `run-stage` fails with config/session not found.
|
||||
- `run`, `resume`, `run-stage`, `session plan`, or `session restore` fails with config/session not found.
|
||||
|
||||
Likely Cause:
|
||||
- `pipeline.yml` or `session.yml` is missing from discovery paths.
|
||||
- wrong working directory when relying on `./session.yml`.
|
||||
- `pipeline.yml` or `session.yml` is missing from system discovery paths.
|
||||
- the selected campaign ID does not exist under `pipeline.campaigns.root`.
|
||||
- a local working-directory config file was not passed explicitly.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
pwd
|
||||
ls -l ./session.yml
|
||||
ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml
|
||||
ls -l /usr/local/etc/narratio/session.yml /etc/narratio/session.yml
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- pass explicit `--config` and `--session`.
|
||||
- or place files in documented discovery paths.
|
||||
- pass explicit `--config`, `--campaign <id>`, `--campaign-file <path>`, and `--session` as appropriate.
|
||||
- or place files in documented discovery paths and set `pipeline.campaigns.default_campaign_id`.
|
||||
|
||||
Links:
|
||||
- [docs/config.md](./config.md)
|
||||
- [docs/cli.md](./cli.md)
|
||||
|
||||
## Session template rendering failure
|
||||
## Templated session file rejected
|
||||
|
||||
Symptom:
|
||||
- load fails with unresolved placeholder or `session_id` mismatch.
|
||||
- load fails with a message that `session.yml must be concrete`.
|
||||
|
||||
Likely Cause:
|
||||
- templated `session.yml` used without `--session-id`.
|
||||
- rendered `session_id` differs from passed `--session-id`.
|
||||
- a template authoring file such as `session.template.yml` was passed to `--session` or uploaded as remote `session.yml`.
|
||||
- `session.yml` still contains `{{ ... }}` placeholders.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio plan --session ./session.yml --session-id 2026-04-04
|
||||
narratio session plan 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session ./session.yml
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- pass `--session-id` when template placeholders are present.
|
||||
- ensure rendered `session_id` matches intended run session id.
|
||||
- generate concrete YAML with `narratio session init`.
|
||||
- pass the generated concrete `session.yml` to downstream commands or upload it through `session init --remote`.
|
||||
|
||||
Links:
|
||||
- [docs/config.md](./config.md)
|
||||
@@ -62,7 +62,7 @@ Likely Cause:
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04
|
||||
narratio session plan 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
@@ -84,7 +84,7 @@ Likely Cause:
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio run --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout
|
||||
narratio run 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --artifacts player_handout
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
@@ -95,22 +95,22 @@ Links:
|
||||
- [docs/cli.md](./cli.md)
|
||||
- [docs/config.md](./config.md)
|
||||
|
||||
## `run-stage --artifacts` on non-analyze stage
|
||||
## `run-stage --artifacts` on unsupported stage
|
||||
|
||||
Symptom:
|
||||
- `run-stage` fails with `--artifacts is only supported for stage "analyze"`.
|
||||
- `run-stage` fails because `--artifacts` is only supported for `analyze` and `archive`.
|
||||
|
||||
Likely Cause:
|
||||
- `--artifacts` was used with a non-`analyze` stage.
|
||||
- `--artifacts` was used with a stage other than `analyze` or `archive`.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts session_recap polish
|
||||
narratio run-stage polish 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --artifacts session_recap
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- use `--artifacts` only with `run-stage ... analyze`.
|
||||
- use `--artifacts` only with `run-stage analyze ...` or `run-stage archive ...`.
|
||||
|
||||
Links:
|
||||
- [docs/cli.md](./cli.md)
|
||||
@@ -129,7 +129,7 @@ Likely Cause:
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04
|
||||
narratio session plan 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
@@ -153,8 +153,8 @@ Likely Cause:
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio status --manifest /path/to/manifest.json
|
||||
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout analyze
|
||||
narratio session status 2026-04-04
|
||||
narratio run-stage analyze 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --artifacts player_handout
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
@@ -168,22 +168,21 @@ Links:
|
||||
## Manifest/status path failure
|
||||
|
||||
Symptom:
|
||||
- `status` fails because manifest path is missing, unreadable, or invalid.
|
||||
- `session status` fails because config/session state is missing, unreadable, or invalid.
|
||||
|
||||
Likely Cause:
|
||||
- wrong manifest path.
|
||||
- wrong session ID.
|
||||
- wrong config/campaign/session file selected.
|
||||
- manifest removed after cleanup.
|
||||
- `--manifest` omitted.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio status --manifest /path/to/manifest.json
|
||||
ls -l /path/to/manifest.json
|
||||
narratio session status 2026-04-04
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- use manifest path printed by `run`, `resume`, or `run-stage`.
|
||||
- use the same session ID and config files that will be used for `run`, `resume`, or `run-stage`.
|
||||
|
||||
Links:
|
||||
- [docs/cli.md](./cli.md)
|
||||
@@ -192,7 +191,7 @@ Links:
|
||||
## Session lock conflict (`.lock`)
|
||||
|
||||
Symptom:
|
||||
- run fails with lock conflict for session workdir.
|
||||
- `run`, `resume`, `run-stage`, or `session restore` fails with lock conflict for session workdir.
|
||||
|
||||
Likely Cause:
|
||||
- another Narratio process is running same session.
|
||||
@@ -207,13 +206,104 @@ ps aux | grep narratio
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- wait for active run to finish.
|
||||
- wait for active process to finish.
|
||||
- if no process is active, remove only stale session `.lock` file.
|
||||
|
||||
Links:
|
||||
- [docs/operations.md](./operations.md)
|
||||
- [docs/internal/workspace.md](./internal/workspace.md)
|
||||
|
||||
## Restore remote current pointer or manifest missing
|
||||
|
||||
Symptom:
|
||||
- `session restore` fails with remote current pointer or current manifest errors.
|
||||
|
||||
Likely Cause:
|
||||
- `current/run_id.txt` was never published.
|
||||
- `current/manifest.json` is missing for the session prefix.
|
||||
- archive commit did not complete.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio session restore 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --dry-run
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- verify archive stage succeeded for the target session.
|
||||
- rerun/archive from a healthy source workspace so current pointers are published.
|
||||
|
||||
Links:
|
||||
- [docs/operations.md](./operations.md)
|
||||
- [docs/internal/stage-archive.md](./internal/stage-archive.md)
|
||||
|
||||
## Restore manifest identity mismatch
|
||||
|
||||
Symptom:
|
||||
- `session restore` fails because remote manifest session or campaign does not match requested values.
|
||||
|
||||
Likely Cause:
|
||||
- wrong positional session ID or wrong session config selected.
|
||||
- archive prefix points to a different campaign/session.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio session restore 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --dry-run
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- use the correct session config and positional session ID.
|
||||
- verify campaign/session identity in local config before restore.
|
||||
|
||||
Links:
|
||||
- [docs/config.md](./config.md)
|
||||
- [docs/operations.md](./operations.md)
|
||||
|
||||
## Restore conflict without `--force`
|
||||
|
||||
Symptom:
|
||||
- `session restore` fails with `restore conflict` and conflict counts.
|
||||
|
||||
Likely Cause:
|
||||
- local durable file differs from remote file for one or more planned restore paths.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio session restore 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --dry-run
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- review planned conflicts.
|
||||
- rerun with `--force` only when remote state should overwrite local state.
|
||||
|
||||
Links:
|
||||
- [docs/cli.md](./cli.md)
|
||||
- [docs/operations.md](./operations.md)
|
||||
|
||||
## Restore report expectations
|
||||
|
||||
Symptom:
|
||||
- operator expects restore report file but does not find one.
|
||||
|
||||
Likely Cause:
|
||||
- restore was executed in `--dry-run` mode.
|
||||
- restore failed before report persistence path (for example lock acquisition failure).
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
ls -l {workspace.root}/work/{campaign}/{session_id}/reports/restore-latest.json
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- run non-dry-run restore for durable report output.
|
||||
- resolve lock or early preflight failures and retry.
|
||||
|
||||
Links:
|
||||
- [docs/operations.md](./operations.md)
|
||||
|
||||
## Secrets env-dir or credential-env failure
|
||||
|
||||
Symptom:
|
||||
@@ -251,7 +341,7 @@ Likely Cause:
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 prepare
|
||||
narratio run-stage prepare 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
@@ -274,13 +364,13 @@ Likely Cause:
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio status --manifest /path/to/manifest.json
|
||||
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 archive
|
||||
narratio session status 2026-04-04
|
||||
narratio run-stage archive 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- rerun or resume upstream stages to generate required files.
|
||||
- adjust promotion rules to match files that must exist.
|
||||
- adjust promotion `source`/`dest` rules to match artifacts that must exist.
|
||||
- retry after storage issue is resolved.
|
||||
|
||||
Links:
|
||||
|
||||
1
examples/campaigns/sample-campaign/autocorrect.yml
Normal file
1
examples/campaigns/sample-campaign/autocorrect.yml
Normal file
@@ -0,0 +1 @@
|
||||
[]
|
||||
6
examples/campaigns/sample-campaign/campaign.yml
Normal file
6
examples/campaigns/sample-campaign/campaign.yml
Normal file
@@ -0,0 +1,6 @@
|
||||
campaign_id: sample-campaign
|
||||
session_template_file: ./session.template.yml
|
||||
inputs:
|
||||
speakers_file: ./speakers.yml
|
||||
autocorrect_file: ./autocorrect.yml
|
||||
glossary_file: ./glossary.yml
|
||||
1
examples/campaigns/sample-campaign/glossary.yml
Normal file
1
examples/campaigns/sample-campaign/glossary.yml
Normal file
@@ -0,0 +1 @@
|
||||
[]
|
||||
3
examples/campaigns/sample-campaign/session.template.yml
Normal file
3
examples/campaigns/sample-campaign/session.template.yml
Normal file
@@ -0,0 +1,3 @@
|
||||
session_id: "{{ session_id }}"
|
||||
inputs:
|
||||
audio_dir: ./audio
|
||||
5
examples/campaigns/sample-campaign/speakers.yml
Normal file
5
examples/campaigns/sample-campaign/speakers.yml
Normal file
@@ -0,0 +1,5 @@
|
||||
match:
|
||||
- speaker: "Eric Rakestraw"
|
||||
match:
|
||||
- "Eric_Rakestraw"
|
||||
- "Eric"
|
||||
@@ -14,9 +14,6 @@ workspace:
|
||||
storage:
|
||||
# Optional storage backend selector; use "s3" for archive + S3 audio workflows.
|
||||
backend: s3
|
||||
# Compatibility fields retained in schema.
|
||||
bucket: ""
|
||||
prefix: ""
|
||||
s3:
|
||||
# Required when using S3 audio or S3 archive uploads.
|
||||
bucket: my-dnd-archive
|
||||
@@ -30,6 +27,12 @@ storage:
|
||||
access_key_id_env: OBJECT_STORAGE_KEY_ID
|
||||
secret_access_key_env: OBJECT_STORAGE_KEY
|
||||
|
||||
campaigns:
|
||||
# Optional; defaults to /usr/local/share/narratio/campaigns.
|
||||
root: /usr/local/share/narratio/campaigns
|
||||
# Optional command default when --campaign is omitted.
|
||||
default_campaign_id: sample-campaign
|
||||
|
||||
spool:
|
||||
# Optional; defaults to /var/spool/narratio.
|
||||
root: /var/spool/narratio
|
||||
@@ -40,16 +43,16 @@ archive:
|
||||
# Optional booleans; defaults are true.
|
||||
enabled: true
|
||||
upload_run: true
|
||||
# Optional promotion rules; required files fail archive if missing.
|
||||
# Optional promotion rules; sources use Narratio artifact source IDs.
|
||||
promote_artifacts:
|
||||
- from: transcripts/trimmed.json
|
||||
to: transcripts/trimmed.json
|
||||
- source: narratio.transcript.final_trimmed
|
||||
dest: transcripts/final.trimmed.json
|
||||
required: true
|
||||
- from: artifacts/session_recap.md
|
||||
to: artifacts/session_recap.md
|
||||
- source: narratio.artifact.session_recap
|
||||
dest: artifacts/session_recap.md
|
||||
required: true
|
||||
- from: artifacts/player_handout.md
|
||||
to: artifacts/player_handout.md
|
||||
- source: narratio.artifact.player_handout
|
||||
dest: artifacts/player_handout.md
|
||||
required: false
|
||||
|
||||
whisperx:
|
||||
@@ -96,14 +99,14 @@ audita:
|
||||
|
||||
normalize:
|
||||
# Optional; defaults shown explicitly.
|
||||
output_path: transcripts/normalized.json
|
||||
output_path: transcripts/final.json
|
||||
output_schema: seriatim-intermediate
|
||||
report: true
|
||||
|
||||
trim:
|
||||
# Keep disabled unless bounds prompt integration is configured.
|
||||
enabled: false
|
||||
output_path: transcripts/trimmed.json
|
||||
output_path: transcripts/final.trimmed.json
|
||||
bounds:
|
||||
prompt_id: dnd.session_bounds
|
||||
profile_id: local-fast
|
||||
@@ -130,12 +133,10 @@ scriptorium:
|
||||
timeout: 10m
|
||||
inputs:
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
source: narratio.transcript.final_trimmed
|
||||
required: true
|
||||
previous_recap:
|
||||
source: previous_session_artifact
|
||||
artifact: session_recap
|
||||
path: ""
|
||||
source: narratio.previous_session.artifact.session_recap
|
||||
required: false
|
||||
vars:
|
||||
session_id: true
|
||||
@@ -160,21 +161,13 @@ scriptorium:
|
||||
source: narratio.artifact.session_recap
|
||||
required: true
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
source: narratio.transcript.final_trimmed
|
||||
required: true
|
||||
vars:
|
||||
session_id: true
|
||||
campaign_name: true
|
||||
output_kind: player_handout
|
||||
|
||||
analyzer:
|
||||
# Optional adapter settings.
|
||||
binary_path: ""
|
||||
timeout: 2m
|
||||
artifacts:
|
||||
output_dir: ""
|
||||
types: []
|
||||
|
||||
notification:
|
||||
# Optional notification settings.
|
||||
backend: ""
|
||||
|
||||
@@ -1,2 +1,6 @@
|
||||
campaigns:
|
||||
root: /usr/local/share/narratio/campaigns
|
||||
default_campaign_id: sample-campaign
|
||||
|
||||
whisperx:
|
||||
transcribe_url: "https://transcription.example.com/transcribe"
|
||||
|
||||
@@ -11,6 +11,10 @@ storage:
|
||||
access_key_id_env: OBJECT_STORAGE_KEY_ID
|
||||
secret_access_key_env: OBJECT_STORAGE_KEY
|
||||
|
||||
campaigns:
|
||||
root: /usr/local/share/narratio/campaigns
|
||||
default_campaign_id: sample-campaign
|
||||
|
||||
spool:
|
||||
root: /var/spool/narratio
|
||||
delete_audio_after_archive: true
|
||||
@@ -19,14 +23,14 @@ archive:
|
||||
enabled: true
|
||||
upload_run: true
|
||||
promote_artifacts:
|
||||
- from: transcripts/trimmed.json
|
||||
to: transcripts/trimmed.json
|
||||
- source: narratio.transcript.final_trimmed
|
||||
dest: transcripts/final.trimmed.json
|
||||
required: true
|
||||
- from: artifacts/session_recap.md
|
||||
to: artifacts/session_recap.md
|
||||
- source: narratio.artifact.session_recap
|
||||
dest: artifacts/session_recap.md
|
||||
required: true
|
||||
- from: artifacts/player_handout.md
|
||||
to: artifacts/player_handout.md
|
||||
- source: narratio.artifact.player_handout
|
||||
dest: artifacts/player_handout.md
|
||||
required: false
|
||||
|
||||
whisperx:
|
||||
@@ -57,7 +61,7 @@ audita:
|
||||
report: true
|
||||
|
||||
normalize:
|
||||
output_path: transcripts/normalized.json
|
||||
output_path: transcripts/final.json
|
||||
output_schema: seriatim-intermediate
|
||||
report: true
|
||||
|
||||
@@ -78,11 +82,10 @@ scriptorium:
|
||||
timeout: 10m
|
||||
inputs:
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
source: narratio.transcript.final_trimmed
|
||||
required: true
|
||||
previous_recap:
|
||||
source: previous_session_artifact
|
||||
artifact: session_recap
|
||||
source: narratio.previous_session.artifact.session_recap
|
||||
required: false
|
||||
vars:
|
||||
session_id: true
|
||||
@@ -103,14 +106,11 @@ scriptorium:
|
||||
source: narratio.artifact.session_recap
|
||||
required: true
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
source: narratio.transcript.final_trimmed
|
||||
required: true
|
||||
vars:
|
||||
session_id: true
|
||||
output_kind: player_handout
|
||||
|
||||
analyzer:
|
||||
timeout: 2m
|
||||
|
||||
notification:
|
||||
timeout: 30s
|
||||
|
||||
@@ -1,9 +1,5 @@
|
||||
session_id: 2026-05-03
|
||||
campaign: sample-campaign
|
||||
date: 2026-05-03
|
||||
title: Sample Session
|
||||
inputs:
|
||||
audio_dir: ./audio
|
||||
speakers_file: ./examples/speakers.yml
|
||||
autocorrect_file: ./examples/autocorrect.yml
|
||||
glossary_file: ./examples/glossary.yml
|
||||
|
||||
@@ -1,10 +1,6 @@
|
||||
session_id: 2026-05-03
|
||||
campaign: sample-campaign
|
||||
date: 2026-05-03
|
||||
title: Sample Session
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
speakers_file: ./examples/speakers.yml
|
||||
autocorrect_file: ./examples/autocorrect.yml
|
||||
glossary_file: ./examples/glossary.yml
|
||||
|
||||
@@ -1,7 +1,3 @@
|
||||
session_id: "{{ session_id }}"
|
||||
campaign: sample-campaign
|
||||
inputs:
|
||||
audio_dir: ./audio
|
||||
speakers_file: ./examples/speakers.yml
|
||||
autocorrect_file: ./examples/autocorrect.yml
|
||||
glossary_file: ./examples/glossary.yml
|
||||
|
||||
@@ -1,40 +0,0 @@
|
||||
package analyzer
|
||||
|
||||
import "context"
|
||||
|
||||
// NoopRunner is a deterministic no-op analyzer adapter.
|
||||
type NoopRunner struct{}
|
||||
|
||||
// Run returns the requested output path with placeholder metadata.
|
||||
func (n *NoopRunner) Run(ctx context.Context, req AnalyzeRequest) (AnalyzeResult, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return AnalyzeResult{}, err
|
||||
}
|
||||
return AnalyzeResult{ArtifactPath: req.OutputPath, Metadata: map[string]any{"placeholder": true}}, nil
|
||||
}
|
||||
|
||||
// FakeRunner captures analyze requests and returns deterministic responses.
|
||||
type FakeRunner struct {
|
||||
Requests []AnalyzeRequest
|
||||
Err error
|
||||
Result AnalyzeResult
|
||||
}
|
||||
|
||||
// Run records request and returns configured response.
|
||||
func (f *FakeRunner) Run(ctx context.Context, req AnalyzeRequest) (AnalyzeResult, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return AnalyzeResult{}, err
|
||||
}
|
||||
f.Requests = append(f.Requests, req)
|
||||
if f.Err != nil {
|
||||
return AnalyzeResult{}, f.Err
|
||||
}
|
||||
res := f.Result
|
||||
if res.ArtifactPath == "" {
|
||||
res.ArtifactPath = req.OutputPath
|
||||
}
|
||||
if res.Metadata == nil {
|
||||
res.Metadata = map[string]any{"fake": true}
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
package analyzer
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestFakeRunnerCapturesRequestAndReturnsPath(t *testing.T) {
|
||||
fake := &FakeRunner{}
|
||||
req := AnalyzeRequest{ArtifactType: "session-log", OutputPath: "artifacts/session-log.md"}
|
||||
|
||||
res, err := fake.Run(context.Background(), req)
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
if len(fake.Requests) != 1 || fake.Requests[0].ArtifactType != "session-log" {
|
||||
t.Fatalf("requests = %#v, want captured request", fake.Requests)
|
||||
}
|
||||
if res.ArtifactPath != req.OutputPath {
|
||||
t.Fatalf("artifact path = %q, want %q", res.ArtifactPath, req.OutputPath)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFakeRunnerError(t *testing.T) {
|
||||
fake := &FakeRunner{Err: errors.New("boom")}
|
||||
_, err := fake.Run(context.Background(), AnalyzeRequest{})
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
}
|
||||
@@ -1,28 +0,0 @@
|
||||
// Package analyzer declares the adapter contract for artifact analysis generation.
|
||||
package analyzer
|
||||
|
||||
import "context"
|
||||
|
||||
// TODO: implement analyzer integration once the analyzer contract is finalized.
|
||||
|
||||
// Runner is the adapter boundary for analyzer invocations.
|
||||
type Runner interface {
|
||||
Run(ctx context.Context, req AnalyzeRequest) (AnalyzeResult, error)
|
||||
}
|
||||
|
||||
// AnalyzeRequest describes one analyzer artifact generation request.
|
||||
type AnalyzeRequest struct {
|
||||
ArtifactType string
|
||||
ProcessedTranscriptPath string
|
||||
ContextReferences []string
|
||||
OutputPath string
|
||||
GeneratedConfigPath string
|
||||
StdoutLogPath string
|
||||
StderrLogPath string
|
||||
}
|
||||
|
||||
// AnalyzeResult describes analyzer output.
|
||||
type AnalyzeResult struct {
|
||||
ArtifactPath string
|
||||
Metadata map[string]any
|
||||
}
|
||||
@@ -14,7 +14,7 @@ func TestFakeRunnerCapturesRequestAndReturnsPath(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
req := PolishRequest{
|
||||
GeneratedConfigPath: filepath.Join(dir, "config", "audita.yml"),
|
||||
OutputProcessedPath: filepath.Join(dir, "transcripts", "processed.json"),
|
||||
OutputProcessedPath: filepath.Join(dir, "transcripts", "polished.json"),
|
||||
StdoutLogPath: filepath.Join(dir, "logs", "audita.stdout.log"),
|
||||
StderrLogPath: filepath.Join(dir, "logs", "audita.stderr.log"),
|
||||
}
|
||||
|
||||
@@ -52,9 +52,9 @@ func TestSubprocessRunnerSuccessArgsEnvAndValidation(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
req := PolishRequest{
|
||||
GeneratedConfigPath: filepath.Join(dir, "audita.generated.yml"),
|
||||
MergedTranscriptPath: filepath.Join(dir, "merged.json"),
|
||||
MergedTranscriptPath: filepath.Join(dir, "base.json"),
|
||||
GlossaryPath: filepath.Join(dir, "glossary.yml"),
|
||||
OutputProcessedPath: filepath.Join(dir, "processed.json"),
|
||||
OutputProcessedPath: filepath.Join(dir, "polished.json"),
|
||||
ReportPath: filepath.Join(dir, "audita.report.json"),
|
||||
WorkDir: filepath.Join(dir, "artifacts", "audita-work"),
|
||||
StdoutLogPath: filepath.Join(dir, "audita.stdout.log"),
|
||||
@@ -571,7 +571,7 @@ func mustAuditaRunner(t *testing.T, cfg SubprocessRunnerConfig) *SubprocessRunne
|
||||
func auditaReqForTest(t *testing.T, withReport bool) PolishRequest {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
merged := filepath.Join(dir, "merged.json")
|
||||
merged := filepath.Join(dir, "base.json")
|
||||
glossary := filepath.Join(dir, "glossary.yml")
|
||||
writeAuditaTestFile(t, merged, `{"segments":[]}`)
|
||||
writeAuditaTestFile(t, glossary, "terms: []\n")
|
||||
@@ -579,7 +579,7 @@ func auditaReqForTest(t *testing.T, withReport bool) PolishRequest {
|
||||
GeneratedConfigPath: filepath.Join(dir, "audita.generated.yml"),
|
||||
MergedTranscriptPath: merged,
|
||||
GlossaryPath: glossary,
|
||||
OutputProcessedPath: filepath.Join(dir, "processed.json"),
|
||||
OutputProcessedPath: filepath.Join(dir, "polished.json"),
|
||||
WorkDir: filepath.Join(dir, "artifacts", "audita-work"),
|
||||
StdoutLogPath: filepath.Join(dir, "audita.stdout.log"),
|
||||
StderrLogPath: filepath.Join(dir, "audita.stderr.log"),
|
||||
|
||||
@@ -31,7 +31,7 @@ func TestSubprocessRunnerRunSuccessBuildsDeterministicArgsAndCapturesLogs(t *tes
|
||||
ConfigPath: "/etc/scriptorium/config.yml",
|
||||
PromptID: "dnd.session_recap",
|
||||
ProfileID: "local-quality",
|
||||
InputPaths: map[string]string{"transcript": filepath.Join(dir, "processed.json"), "other": filepath.Join(dir, "other.md")},
|
||||
InputPaths: map[string]string{"transcript": filepath.Join(dir, "polished.json"), "other": filepath.Join(dir, "other.md")},
|
||||
Vars: map[string]string{"session_id": "2026-05-03", "campaign_name": "Icewind Dale"},
|
||||
OutputPath: filepath.Join(dir, "artifacts", "session_recap.md"),
|
||||
StdoutLogPath: filepath.Join(dir, "logs", "scriptorium.run.stdout.log"),
|
||||
@@ -180,7 +180,7 @@ func TestSubprocessRunnerRenderSuccess(t *testing.T) {
|
||||
req := RenderArtifactRequest{
|
||||
Binary: wrapper,
|
||||
PromptID: "dnd.session_recap",
|
||||
InputPaths: map[string]string{"transcript": filepath.Join(dir, "processed.json")},
|
||||
InputPaths: map[string]string{"transcript": filepath.Join(dir, "polished.json")},
|
||||
OutputPath: filepath.Join(dir, "artifacts", "session_recap.render.json"),
|
||||
StdoutLogPath: filepath.Join(dir, "logs", "scriptorium.render.stdout.log"),
|
||||
StderrLogPath: filepath.Join(dir, "logs", "scriptorium.render.stderr.log"),
|
||||
@@ -285,7 +285,7 @@ type scriptoriumHelperRecord struct {
|
||||
func runReqForTest(t *testing.T, binary string) RunArtifactRequest {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
transcriptPath := filepath.Join(dir, "processed.json")
|
||||
transcriptPath := filepath.Join(dir, "polished.json")
|
||||
writeScriptoriumFile(t, transcriptPath, `{"segments":[]}`)
|
||||
return RunArtifactRequest{
|
||||
Binary: binary,
|
||||
|
||||
@@ -14,7 +14,7 @@ func TestFakeRunnerCapturesRequestAndReturnsPath(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
req := MergeRequest{
|
||||
GeneratedConfigPath: filepath.Join(dir, "config", "seriatim.yml"),
|
||||
OutputMergedTranscriptPath: filepath.Join(dir, "transcripts", "merged.json"),
|
||||
OutputMergedTranscriptPath: filepath.Join(dir, "transcripts", "base.json"),
|
||||
StdoutLogPath: filepath.Join(dir, "logs", "seriatim.stdout.log"),
|
||||
StderrLogPath: filepath.Join(dir, "logs", "seriatim.stderr.log"),
|
||||
}
|
||||
@@ -57,8 +57,8 @@ func TestFakeRunnerTrimCapturesRequestAndReturnsPath(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
req := TrimRequest{
|
||||
GeneratedConfigPath: filepath.Join(dir, "config", "seriatim.trim.yml"),
|
||||
InputTranscriptPath: filepath.Join(dir, "transcripts", "processed.json"),
|
||||
OutputTrimmedPath: filepath.Join(dir, "transcripts", "trimmed.json"),
|
||||
InputTranscriptPath: filepath.Join(dir, "transcripts", "polished.json"),
|
||||
OutputTrimmedPath: filepath.Join(dir, "transcripts", "final.trimmed.json"),
|
||||
KeepSelector: "1-10",
|
||||
StdoutLogPath: filepath.Join(dir, "logs", "seriatim.trim.stdout.log"),
|
||||
StderrLogPath: filepath.Join(dir, "logs", "seriatim.trim.stderr.log"),
|
||||
@@ -105,8 +105,8 @@ func TestFakeRunnerNormalizeCapturesRequestAndReturnsPath(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
req := NormalizeRequest{
|
||||
GeneratedConfigPath: filepath.Join(dir, "config", "seriatim.normalize.yml"),
|
||||
InputTranscriptPath: filepath.Join(dir, "transcripts", "processed.json"),
|
||||
OutputNormalizedPath: filepath.Join(dir, "transcripts", "normalized.json"),
|
||||
InputTranscriptPath: filepath.Join(dir, "transcripts", "polished.json"),
|
||||
OutputNormalizedPath: filepath.Join(dir, "transcripts", "final.json"),
|
||||
OutputSchema: "seriatim-intermediate",
|
||||
ReportPath: filepath.Join(dir, "artifacts", "seriatim.normalize.report.json"),
|
||||
StdoutLogPath: filepath.Join(dir, "logs", "seriatim.normalize.stdout.log"),
|
||||
|
||||
@@ -50,7 +50,7 @@ func TestSubprocessRunnerSuccessWithReportArgsAndEnv(t *testing.T) {
|
||||
req := MergeRequest{
|
||||
GeneratedConfigPath: filepath.Join(dir, "seriatim.generated.yml"),
|
||||
InputTranscriptPaths: []string{filepath.Join(dir, "a.json"), filepath.Join(dir, "b.json")},
|
||||
OutputMergedTranscriptPath: filepath.Join(dir, "merged.json"),
|
||||
OutputMergedTranscriptPath: filepath.Join(dir, "base.json"),
|
||||
ReportPath: filepath.Join(dir, "seriatim.report.json"),
|
||||
SpeakersPath: filepath.Join(dir, "speakers.yml"),
|
||||
AutocorrectPath: filepath.Join(dir, "autocorrect.yml"),
|
||||
@@ -732,7 +732,7 @@ func mergeReqForTest(t *testing.T, withReport bool) MergeRequest {
|
||||
req := MergeRequest{
|
||||
GeneratedConfigPath: filepath.Join(dir, "seriatim.generated.yml"),
|
||||
InputTranscriptPaths: []string{in1, in2},
|
||||
OutputMergedTranscriptPath: filepath.Join(dir, "merged.json"),
|
||||
OutputMergedTranscriptPath: filepath.Join(dir, "base.json"),
|
||||
StdoutLogPath: filepath.Join(dir, "seriatim.stdout.log"),
|
||||
StderrLogPath: filepath.Join(dir, "seriatim.stderr.log"),
|
||||
}
|
||||
@@ -745,11 +745,11 @@ func mergeReqForTest(t *testing.T, withReport bool) MergeRequest {
|
||||
func trimReqForTest(t *testing.T) TrimRequest {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
input := filepath.Join(dir, "processed.json")
|
||||
input := filepath.Join(dir, "polished.json")
|
||||
writeSeriatimFile(t, input, `{"schema":"seriatim.intermediate.v1","segments":[]}`)
|
||||
return TrimRequest{
|
||||
InputTranscriptPath: input,
|
||||
OutputTrimmedPath: filepath.Join(dir, "trimmed.json"),
|
||||
OutputTrimmedPath: filepath.Join(dir, "final.trimmed.json"),
|
||||
KeepSelector: "5-12",
|
||||
GeneratedConfigPath: filepath.Join(dir, "seriatim.trim.generated.yml"),
|
||||
StdoutLogPath: filepath.Join(dir, "seriatim.trim.stdout.log"),
|
||||
@@ -760,12 +760,12 @@ func trimReqForTest(t *testing.T) TrimRequest {
|
||||
func normalizeReqForTest(t *testing.T, withReport bool) NormalizeRequest {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
input := filepath.Join(dir, "processed.json")
|
||||
input := filepath.Join(dir, "polished.json")
|
||||
writeSeriatimFile(t, input, `{"schema":"audita.processed.v1","segments":[]}`)
|
||||
|
||||
req := NormalizeRequest{
|
||||
InputTranscriptPath: input,
|
||||
OutputNormalizedPath: filepath.Join(dir, "normalized.json"),
|
||||
OutputNormalizedPath: filepath.Join(dir, "final.json"),
|
||||
OutputSchema: "seriatim-intermediate",
|
||||
GeneratedConfigPath: filepath.Join(dir, "seriatim.normalize.generated.yml"),
|
||||
StdoutLogPath: filepath.Join(dir, "seriatim.normalize.stdout.log"),
|
||||
|
||||
@@ -27,8 +27,9 @@ type FakeBackend struct {
|
||||
Err error
|
||||
Result ArchiveResult
|
||||
|
||||
Objects map[string]FakeObject
|
||||
Uploads []FakeUploadCall
|
||||
Objects map[string]FakeObject
|
||||
Uploads []FakeUploadCall
|
||||
Downloads []FakeDownloadCall
|
||||
|
||||
ListErr error
|
||||
DownloadErr error
|
||||
@@ -43,6 +44,12 @@ type FakeUploadCall struct {
|
||||
Options UploadOptions
|
||||
}
|
||||
|
||||
// FakeDownloadCall captures one download invocation in call order.
|
||||
type FakeDownloadCall struct {
|
||||
Key string
|
||||
LocalPath string
|
||||
}
|
||||
|
||||
// Archive records request and returns configured response.
|
||||
func (f *FakeBackend) Archive(ctx context.Context, req ArchiveRequest) (ArchiveResult, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
@@ -130,6 +137,10 @@ func (f *FakeBackend) Download(ctx context.Context, key, localPath string) error
|
||||
if !ok {
|
||||
return fmt.Errorf("download object %q: %w", key, os.ErrNotExist)
|
||||
}
|
||||
f.Downloads = append(f.Downloads, FakeDownloadCall{
|
||||
Key: normalizeObjectKey(key),
|
||||
LocalPath: localPath,
|
||||
})
|
||||
|
||||
if err := os.MkdirAll(filepath.Dir(localPath), 0o755); err != nil {
|
||||
return fmt.Errorf("download object %q: create parent directory: %w", key, err)
|
||||
|
||||
@@ -56,7 +56,7 @@ func TestFakeBackendDownload(t *testing.T) {
|
||||
fake.SeedObject(FakeObject{Key: "audio/a.flac", Data: []byte("audio-a")})
|
||||
|
||||
dst := filepath.Join(t.TempDir(), "nested", "a.flac")
|
||||
if err := fake.Download(context.Background(), "audio/a.flac", dst); err != nil {
|
||||
if err := fake.Download(context.Background(), `audio\a.flac`, dst); err != nil {
|
||||
t.Fatalf("Download() error = %v", err)
|
||||
}
|
||||
data, err := os.ReadFile(dst)
|
||||
|
||||
@@ -46,7 +46,7 @@ func (f *artifactSelectionFlag) Normalize() ([]string, error) {
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func validateSelectedAnalyzeArtifacts(cfg *config.Config, selected []string) error {
|
||||
func validateSelectedArtifacts(cfg *config.Config, selected []string) error {
|
||||
if len(selected) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -9,36 +9,80 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/stage"
|
||||
)
|
||||
|
||||
func TestExecuteRunStageArtifactsNonAnalyzeFails(t *testing.T) {
|
||||
func TestExecuteRunStageArtifactsUnsupportedStageFails(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(
|
||||
[]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "--artifacts", "session_recap", "polish"},
|
||||
[]string{"run-stage", "polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), `run-stage: --artifacts is only supported for stage "analyze"`) {
|
||||
if !strings.Contains(stderr.String(), `run-stage: --artifacts is only supported for stages "analyze" and "archive"`) {
|
||||
t.Fatalf("stderr = %q, want stage-gating error", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRunStageArchivePropagatesSelectedArtifacts(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
|
||||
var capturedStages []string
|
||||
var capturedArtifacts []string
|
||||
origExecuteStagesFn := executeStagesFn
|
||||
t.Cleanup(func() {
|
||||
executeStagesFn = origExecuteStagesFn
|
||||
})
|
||||
executeStagesFn = func(_ context.Context, _ *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||
for _, s := range stages {
|
||||
capturedStages = append(capturedStages, s.Name())
|
||||
}
|
||||
capturedArtifacts = append([]string(nil), opts.SelectedArtifacts...)
|
||||
return &RunSummary{ManifestPath: filepath.Join(workspaceRoot, "manifest.json"), Executed: []string{"archive"}}, nil
|
||||
}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(
|
||||
[]string{
|
||||
"run-stage", "archive", "2026-05-03",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--artifacts", "session_recap",
|
||||
},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if len(capturedStages) != 1 || capturedStages[0] != "archive" {
|
||||
t.Fatalf("captured stages = %#v, want [archive]", capturedStages)
|
||||
}
|
||||
if strings.Join(capturedArtifacts, ",") != "session_recap" {
|
||||
t.Fatalf("captured artifacts = %#v, want [session_recap]", capturedArtifacts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteUnknownArtifactsFailValidation(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(
|
||||
[]string{"run", "--config", pipelinePath, "--session", sessionPath, "--artifacts", "unknown_artifact"},
|
||||
[]string{"run", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "unknown_artifact"},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
@@ -52,7 +96,7 @@ func TestExecuteUnknownArtifactsFailValidation(t *testing.T) {
|
||||
|
||||
func TestRunStageArtifactsDoesNotImplyForce(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||
|
||||
store := &manifest.LocalStore{}
|
||||
@@ -65,7 +109,7 @@ func TestRunStageArtifactsDoesNotImplyForce(t *testing.T) {
|
||||
var out bytes.Buffer
|
||||
err := RunStage(
|
||||
context.Background(),
|
||||
[]string{"--config", pipelinePath, "--session", sessionPath, "--artifacts", "session_recap,session_recap", "analyze"},
|
||||
[]string{"analyze", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap,session_recap"},
|
||||
&out,
|
||||
)
|
||||
if err != nil {
|
||||
@@ -78,7 +122,7 @@ func TestRunStageArtifactsDoesNotImplyForce(t *testing.T) {
|
||||
|
||||
func TestResumeArtifactsWithSucceededAnalyzeSkipsUnlessForced(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||
|
||||
store := &manifest.LocalStore{}
|
||||
@@ -93,7 +137,7 @@ func TestResumeArtifactsWithSucceededAnalyzeSkipsUnlessForced(t *testing.T) {
|
||||
var out bytes.Buffer
|
||||
err := Resume(
|
||||
context.Background(),
|
||||
[]string{"--config", pipelinePath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||
[]string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||
&out,
|
||||
)
|
||||
if err != nil {
|
||||
@@ -104,10 +148,260 @@ func TestResumeArtifactsWithSucceededAnalyzeSkipsUnlessForced(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func writeValidConfigFilesWithScriptoriumArtifacts(t *testing.T, workspaceRoot string) (string, string) {
|
||||
func TestExecuteAnalyzeForceRunsAnalyze(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
|
||||
var capturedStages []string
|
||||
var capturedForce bool
|
||||
origExecuteStagesFn := executeStagesFn
|
||||
t.Cleanup(func() {
|
||||
executeStagesFn = origExecuteStagesFn
|
||||
})
|
||||
executeStagesFn = func(_ context.Context, _ *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||
for _, s := range stages {
|
||||
capturedStages = append(capturedStages, s.Name())
|
||||
}
|
||||
capturedForce = opts.Force
|
||||
return &RunSummary{
|
||||
ManifestPath: filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json"),
|
||||
Executed: []string{"analyze"},
|
||||
}, nil
|
||||
}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(
|
||||
[]string{"analyze", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if len(capturedStages) != 1 || capturedStages[0] != "analyze" {
|
||||
t.Fatalf("captured stages = %#v, want [analyze]", capturedStages)
|
||||
}
|
||||
if !capturedForce {
|
||||
t.Fatal("captured force = false, want true")
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "narratio analyze: executed=1 skipped=0 force=true; manifest=") {
|
||||
t.Fatalf("stdout = %q, want analyze summary", stdout.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteAnalyzePropagatesSelectedArtifacts(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
|
||||
var capturedArtifacts []string
|
||||
origExecuteStagesFn := executeStagesFn
|
||||
t.Cleanup(func() {
|
||||
executeStagesFn = origExecuteStagesFn
|
||||
})
|
||||
executeStagesFn = func(_ context.Context, _ *config.Config, _ []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||
capturedArtifacts = append([]string(nil), opts.SelectedArtifacts...)
|
||||
return &RunSummary{ManifestPath: filepath.Join(workspaceRoot, "manifest.json"), Executed: []string{"analyze"}}, nil
|
||||
}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(
|
||||
[]string{
|
||||
"analyze",
|
||||
"2026-05-03",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--artifacts", "player_handout,session_recap",
|
||||
},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if strings.Join(capturedArtifacts, ",") != "player_handout,session_recap" {
|
||||
t.Fatalf("captured artifacts = %#v, want sorted selected artifacts", capturedArtifacts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteAnalyzeUnknownArtifactFailsValidation(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(
|
||||
[]string{"analyze", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "unknown_artifact"},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), `analyze: --artifacts includes unknown artifact "unknown_artifact"`) {
|
||||
t.Fatalf("stderr = %q, want unknown-artifact validation error", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteAnalyzeRejectsPositionalArgsAndForceFlag(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
args []string
|
||||
want string
|
||||
}{
|
||||
{name: "extra positional", args: []string{"analyze", "2026-05-03", "extra"}, want: "analyze: unexpected positional arguments"},
|
||||
{name: "force flag", args: []string{"analyze", "--force"}, want: "analyze: invalid flags: flag provided but not defined: -force"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(tc.args, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), tc.want) {
|
||||
t.Fatalf("stderr = %q, want %q", stderr.String(), tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteAnalyzeMissingConfigUsesRunStageLoadingPath(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"analyze", "2026-05-03"}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "analyze: no pipeline config path provided and no default pipeline config found; searched:") {
|
||||
t.Fatalf("stderr = %q, want pipeline discovery error", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecutePublishForceRunsArchive(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
|
||||
var capturedStages []string
|
||||
var capturedForce bool
|
||||
var capturedArtifacts []string
|
||||
origExecuteStagesFn := executeStagesFn
|
||||
t.Cleanup(func() {
|
||||
executeStagesFn = origExecuteStagesFn
|
||||
})
|
||||
executeStagesFn = func(_ context.Context, _ *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||
for _, s := range stages {
|
||||
capturedStages = append(capturedStages, s.Name())
|
||||
}
|
||||
capturedForce = opts.Force
|
||||
capturedArtifacts = append([]string(nil), opts.SelectedArtifacts...)
|
||||
return &RunSummary{
|
||||
ManifestPath: filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json"),
|
||||
Executed: []string{"archive"},
|
||||
}, nil
|
||||
}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(
|
||||
[]string{"publish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if len(capturedStages) != 1 || capturedStages[0] != "archive" {
|
||||
t.Fatalf("captured stages = %#v, want [archive]", capturedStages)
|
||||
}
|
||||
if !capturedForce {
|
||||
t.Fatal("captured force = false, want true")
|
||||
}
|
||||
if strings.Join(capturedArtifacts, ",") != "session_recap" {
|
||||
t.Fatalf("captured artifacts = %#v, want [session_recap]", capturedArtifacts)
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "narratio publish: executed=1 skipped=0 force=true; manifest=") {
|
||||
t.Fatalf("stdout = %q, want publish summary", stdout.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecutePublishRejectsUnsupportedArgsAndFlags(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
args []string
|
||||
want string
|
||||
}{
|
||||
{name: "extra positional", args: []string{"publish", "2026-05-03", "extra"}, want: "publish: unexpected positional arguments"},
|
||||
{name: "force flag", args: []string{"publish", "--force"}, want: "publish: invalid flags: flag provided but not defined: -force"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(tc.args, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), tc.want) {
|
||||
t.Fatalf("stderr = %q, want %q", stderr.String(), tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecutePublishUnknownArtifactFailsValidation(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(
|
||||
[]string{"publish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "unknown_artifact"},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), `publish: --artifacts includes unknown artifact "unknown_artifact"`) {
|
||||
t.Fatalf("stderr = %q, want unknown-artifact validation error", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecutePublishMissingConfigUsesRunStageLoadingPath(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"publish", "2026-05-03"}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "publish: no pipeline config path provided and no default pipeline config found; searched:") {
|
||||
t.Fatalf("stderr = %q, want pipeline discovery error", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteUsageIncludesAnalyzeAndPublish(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(nil, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "analyze") {
|
||||
t.Fatalf("stderr = %q, want usage to include analyze", stderr.String())
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "publish") {
|
||||
t.Fatalf("stderr = %q, want usage to include publish", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func writeValidConfigFilesWithScriptoriumArtifacts(t *testing.T, workspaceRoot string) (string, string, string) {
|
||||
t.Helper()
|
||||
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
f, err := os.OpenFile(pipelinePath, os.O_APPEND|os.O_WRONLY, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("open pipeline config for append: %v", err)
|
||||
@@ -136,5 +430,5 @@ scriptorium:
|
||||
if _, err := f.WriteString(extra); err != nil {
|
||||
t.Fatalf("append scriptorium config: %v", err)
|
||||
}
|
||||
return pipelinePath, sessionPath
|
||||
return pipelinePath, campaignPath, sessionPath
|
||||
}
|
||||
|
||||
@@ -64,7 +64,7 @@ func TestArtifactSelectionFlagNormalize(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateSelectedAnalyzeArtifacts(t *testing.T) {
|
||||
func TestValidateSelectedArtifacts(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
cfg *config.Config
|
||||
@@ -114,7 +114,7 @@ func TestValidateSelectedAnalyzeArtifacts(t *testing.T) {
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
err := validateSelectedAnalyzeArtifacts(tt.cfg, tt.selected)
|
||||
err := validateSelectedArtifacts(tt.cfg, tt.selected)
|
||||
if tt.wantErr != "" {
|
||||
if err == nil {
|
||||
t.Fatalf("error = nil, want %q", tt.wantErr)
|
||||
|
||||
44
internal/app/campaign_config_path.go
Normal file
44
internal/app/campaign_config_path.go
Normal file
@@ -0,0 +1,44 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
func resolveCampaignConfigPath(pipelineCfg *config.PipelineConfig, campaignIDFlag, campaignFileFlag string) (string, error) {
|
||||
campaignID := strings.TrimSpace(campaignIDFlag)
|
||||
campaignFile := strings.TrimSpace(campaignFileFlag)
|
||||
if campaignID != "" && campaignFile != "" {
|
||||
return "", fmt.Errorf("--campaign and --campaign-file are mutually exclusive")
|
||||
}
|
||||
if campaignFile != "" {
|
||||
return filepath.Clean(campaignFile), nil
|
||||
}
|
||||
if campaignID == "" && pipelineCfg != nil {
|
||||
campaignID = strings.TrimSpace(pipelineCfg.Campaigns.DefaultCampaignID)
|
||||
}
|
||||
if campaignID == "" {
|
||||
return "", fmt.Errorf("no campaign selected; pass --campaign <id> or set pipeline.campaigns.default_campaign_id")
|
||||
}
|
||||
if err := validateCampaignIDToken(campaignID); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if pipelineCfg == nil || strings.TrimSpace(pipelineCfg.Campaigns.Root) == "" {
|
||||
return "", fmt.Errorf("pipeline.campaigns.root is required to select campaign %q", campaignID)
|
||||
}
|
||||
return filepath.Clean(filepath.Join(pipelineCfg.Campaigns.Root, campaignID, "campaign.yml")), nil
|
||||
}
|
||||
|
||||
func validateCampaignIDToken(campaignID string) error {
|
||||
if filepath.IsAbs(campaignID) ||
|
||||
strings.Contains(campaignID, "/") ||
|
||||
strings.Contains(campaignID, `\`) ||
|
||||
campaignID == "." ||
|
||||
campaignID == ".." {
|
||||
return fmt.Errorf("campaign id %q must be a single path segment", campaignID)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
84
internal/app/campaign_config_path_test.go
Normal file
84
internal/app/campaign_config_path_test.go
Normal file
@@ -0,0 +1,84 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
func TestResolveCampaignConfigPathCampaignFileWins(t *testing.T) {
|
||||
explicit := filepath.Join(t.TempDir(), "custom-campaign.yml")
|
||||
got, err := resolveCampaignConfigPath(&config.PipelineConfig{}, "", explicit)
|
||||
if err != nil {
|
||||
t.Fatalf("resolveCampaignConfigPath() error = %v", err)
|
||||
}
|
||||
if got != explicit {
|
||||
t.Fatalf("path = %q, want explicit path %q", got, explicit)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveCampaignConfigPathUsesSelectedCampaignID(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
pipelineCfg := &config.PipelineConfig{}
|
||||
pipelineCfg.Campaigns.Root = dir
|
||||
|
||||
got, err := resolveCampaignConfigPath(pipelineCfg, "icewind", "")
|
||||
if err != nil {
|
||||
t.Fatalf("resolveCampaignConfigPath() error = %v", err)
|
||||
}
|
||||
want := filepath.Join(dir, "icewind", "campaign.yml")
|
||||
if got != filepath.Clean(want) {
|
||||
t.Fatalf("path = %q, want %q", got, filepath.Clean(want))
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveCampaignConfigPathUsesDefaultCampaignID(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
pipelineCfg := &config.PipelineConfig{}
|
||||
pipelineCfg.Campaigns.Root = dir
|
||||
pipelineCfg.Campaigns.DefaultCampaignID = "dilfs"
|
||||
|
||||
got, err := resolveCampaignConfigPath(pipelineCfg, "", "")
|
||||
if err != nil {
|
||||
t.Fatalf("resolveCampaignConfigPath() error = %v", err)
|
||||
}
|
||||
want := filepath.Join(dir, "dilfs", "campaign.yml")
|
||||
if got != filepath.Clean(want) {
|
||||
t.Fatalf("path = %q, want %q", got, filepath.Clean(want))
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveCampaignConfigPathRejectsCampaignIDAndFile(t *testing.T) {
|
||||
_, err := resolveCampaignConfigPath(&config.PipelineConfig{}, "dilfs", filepath.Join(t.TempDir(), "campaign.yml"))
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "mutually exclusive") {
|
||||
t.Fatalf("error = %q, want mutual exclusion", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveCampaignConfigPathRequiresCampaignSelection(t *testing.T) {
|
||||
_, err := resolveCampaignConfigPath(&config.PipelineConfig{}, "", "")
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "no campaign selected") {
|
||||
t.Fatalf("error = %q, want missing selection guidance", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveCampaignConfigPathRejectsPathLikeCampaignID(t *testing.T) {
|
||||
pipelineCfg := &config.PipelineConfig{}
|
||||
pipelineCfg.Campaigns.Root = t.TempDir()
|
||||
|
||||
_, err := resolveCampaignConfigPath(pipelineCfg, "../icewind", "")
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "single path segment") {
|
||||
t.Fatalf("error = %q, want path segment guidance", err.Error())
|
||||
}
|
||||
}
|
||||
347
internal/app/clean.go
Normal file
347
internal/app/clean.go
Normal file
@@ -0,0 +1,347 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
// Clean removes local workspace/spool state while preserving durable cache
|
||||
// state unless cache cleanup is explicitly requested.
|
||||
func Clean(ctx context.Context, args []string, out io.Writer) error {
|
||||
positionalSessionID, args := pullLeadingSessionID(args)
|
||||
fs := flag.NewFlagSet("clean", flag.ContinueOnError)
|
||||
fs.SetOutput(io.Discard)
|
||||
var flags commonConfigFlags
|
||||
var all bool
|
||||
var dryRun bool
|
||||
var clearCache bool
|
||||
addCommonConfigFlags(fs, &flags)
|
||||
fs.BoolVar(&all, "all", false, "clean all local session work/spool state")
|
||||
fs.BoolVar(&dryRun, "dry-run", false, "print cleanup targets without deleting")
|
||||
fs.BoolVar(&clearCache, "clear-cache", false, "also clear durable S3 audio cache entries")
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return fmt.Errorf("clean: invalid flags: %w", err)
|
||||
}
|
||||
if positionalSessionID == "" {
|
||||
if err := applyParsedSessionIDArg("clean", fs, &flags.sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
if fs.NArg() != 0 {
|
||||
return fmt.Errorf("clean: unexpected positional arguments")
|
||||
}
|
||||
if err := applyPositionalSessionID("clean", positionalSessionID, &flags.sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if all {
|
||||
return cleanAllLocal(flags, dryRun, clearCache, out)
|
||||
}
|
||||
return cleanSession(ctx, flags, dryRun, clearCache, out)
|
||||
}
|
||||
|
||||
func cleanSession(ctx context.Context, flags commonConfigFlags, dryRun, clearCache bool, out io.Writer) error {
|
||||
if strings.TrimSpace(flags.sessionID) == "" {
|
||||
return fmt.Errorf("clean: session_id is required unless --all is set")
|
||||
}
|
||||
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
|
||||
if err != nil {
|
||||
return fmt.Errorf("clean: %w", err)
|
||||
}
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Session == nil {
|
||||
return fmt.Errorf("clean: resolved pipeline and session config are required")
|
||||
}
|
||||
campaign := strings.TrimSpace(cfg.Session.Campaign)
|
||||
sessionID := strings.TrimSpace(cfg.Session.SessionID)
|
||||
if campaign == "" || sessionID == "" {
|
||||
return fmt.Errorf("clean: campaign and session_id are required")
|
||||
}
|
||||
|
||||
if dryRun {
|
||||
fmt.Fprintf(out, "Clean plan for %s/%s\n", campaign, sessionID)
|
||||
} else {
|
||||
fmt.Fprintf(out, "Cleaned %s/%s\n", campaign, sessionID)
|
||||
}
|
||||
|
||||
workDir := artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, campaign, sessionID)
|
||||
spoolDir := artifacts.SessionSpoolDir(cfg.Pipeline.Spool.Root, campaign, sessionID)
|
||||
if err := reportCleanScopedDir(out, cfg.Pipeline.Workspace.Root, workDir, "clean.workspace.session", dryRun); err != nil {
|
||||
return fmt.Errorf("clean: %w", err)
|
||||
}
|
||||
if err := reportCleanScopedDir(out, cfg.Pipeline.Spool.Root, spoolDir, "clean.spool.session", dryRun); err != nil {
|
||||
return fmt.Errorf("clean: %w", err)
|
||||
}
|
||||
|
||||
if clearCache {
|
||||
if err := cleanSessionAudioCache(ctx, cfg, dryRun, out); err != nil {
|
||||
return fmt.Errorf("clean: %w", err)
|
||||
}
|
||||
} else {
|
||||
fmt.Fprintln(out, "Cache: preserved")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func cleanAllLocal(flags commonConfigFlags, dryRun, clearCache bool, out io.Writer) error {
|
||||
if strings.TrimSpace(flags.campaignPath) != "" ||
|
||||
strings.TrimSpace(flags.campaignFilePath) != "" ||
|
||||
strings.TrimSpace(flags.sessionPath) != "" ||
|
||||
strings.TrimSpace(flags.sessionID) != "" ||
|
||||
strings.TrimSpace(flags.previousSessionID) != "" {
|
||||
return fmt.Errorf("clean: --all cannot be combined with --campaign, --campaign-file, --session, a session_id, or --previous-session-id")
|
||||
}
|
||||
resolvedPipelinePath, err := resolvePipelineConfigPath(flags.pipelinePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("clean: %w", err)
|
||||
}
|
||||
pipelineCfg, err := config.LoadPipeline(resolvedPipelinePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("clean: %w", err)
|
||||
}
|
||||
|
||||
if dryRun {
|
||||
fmt.Fprintln(out, "Clean plan for all local sessions")
|
||||
} else {
|
||||
fmt.Fprintln(out, "Cleaned all local sessions")
|
||||
}
|
||||
|
||||
workRoot := filepath.Join(pipelineCfg.Workspace.Root, config.PathWorkDirSegment)
|
||||
if err := reportCleanScopedDir(out, pipelineCfg.Workspace.Root, workRoot, "clean.workspace.all", dryRun); err != nil {
|
||||
return fmt.Errorf("clean: %w", err)
|
||||
}
|
||||
if err := reportCleanRootChildren(out, pipelineCfg.Spool.Root, "clean.spool.all", dryRun); err != nil {
|
||||
return fmt.Errorf("clean: %w", err)
|
||||
}
|
||||
|
||||
if clearCache {
|
||||
if err := cleanAllAudioCache(pipelineCfg, dryRun, out); err != nil {
|
||||
return fmt.Errorf("clean: %w", err)
|
||||
}
|
||||
} else {
|
||||
fmt.Fprintln(out, "Cache: preserved")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func reportCleanScopedDir(out io.Writer, root, target, policy string, dryRun bool) error {
|
||||
dir, err := validateScopedDir(root, target, policy)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if dryRun {
|
||||
if dir.Exists {
|
||||
fmt.Fprintf(out, "Would delete: %s\n", dir.TargetAbs)
|
||||
} else {
|
||||
fmt.Fprintf(out, "Would skip missing: %s\n", dir.TargetAbs)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if !dir.Exists {
|
||||
fmt.Fprintf(out, "Missing: %s\n", dir.TargetAbs)
|
||||
return nil
|
||||
}
|
||||
if err := os.RemoveAll(dir.TargetAbs); err != nil {
|
||||
return fmt.Errorf("cleanup policy %s: remove %q: %w", policy, dir.TargetAbs, err)
|
||||
}
|
||||
fmt.Fprintf(out, "Deleted: %s\n", dir.TargetAbs)
|
||||
return nil
|
||||
}
|
||||
|
||||
func reportCleanRootChildren(out io.Writer, root, policy string, dryRun bool) error {
|
||||
rootAbs, entries, err := cleanableRootChildren(root, policy)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(entries) == 0 {
|
||||
if dryRun {
|
||||
fmt.Fprintf(out, "Would skip empty: %s\n", rootAbs)
|
||||
} else {
|
||||
fmt.Fprintf(out, "Empty: %s\n", rootAbs)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
for _, entry := range entries {
|
||||
if dryRun {
|
||||
fmt.Fprintf(out, "Would delete: %s\n", entry)
|
||||
continue
|
||||
}
|
||||
if err := os.RemoveAll(entry); err != nil {
|
||||
return fmt.Errorf("cleanup policy %s: remove %q: %w", policy, entry, err)
|
||||
}
|
||||
fmt.Fprintf(out, "Deleted: %s\n", entry)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func cleanableRootChildren(root, policy string) (string, []string, error) {
|
||||
cleanRoot := strings.TrimSpace(root)
|
||||
if cleanRoot == "" {
|
||||
return "", nil, fmt.Errorf("cleanup policy %s: root path is required", policy)
|
||||
}
|
||||
rootAbs, err := filepath.Abs(cleanRoot)
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
||||
}
|
||||
info, err := os.Lstat(rootAbs)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return rootAbs, nil, nil
|
||||
}
|
||||
return "", nil, fmt.Errorf("cleanup policy %s: stat root %q: %w", policy, rootAbs, err)
|
||||
}
|
||||
if info.Mode()&os.ModeSymlink != 0 {
|
||||
return "", nil, fmt.Errorf("cleanup policy %s: refusing to clean symlink root %q", policy, rootAbs)
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return "", nil, fmt.Errorf("cleanup policy %s: root %q is not a directory", policy, rootAbs)
|
||||
}
|
||||
entries, err := os.ReadDir(rootAbs)
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("cleanup policy %s: read root %q: %w", policy, rootAbs, err)
|
||||
}
|
||||
out := make([]string, 0, len(entries))
|
||||
for _, entry := range entries {
|
||||
path := filepath.Join(rootAbs, entry.Name())
|
||||
info, err := os.Lstat(path)
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("cleanup policy %s: stat child %q: %w", policy, path, err)
|
||||
}
|
||||
if info.Mode()&os.ModeSymlink != 0 {
|
||||
return "", nil, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, path)
|
||||
}
|
||||
out = append(out, path)
|
||||
}
|
||||
return rootAbs, out, nil
|
||||
}
|
||||
|
||||
func cleanSessionAudioCache(ctx context.Context, cfg *config.Config, dryRun bool, out io.Writer) error {
|
||||
if cfg.Session.Inputs.AudioS3 == nil {
|
||||
fmt.Fprintln(out, "Cache: skipped (session does not use audio_s3)")
|
||||
return nil
|
||||
}
|
||||
if cfg.Pipeline.Storage.S3 == nil || strings.TrimSpace(cfg.Pipeline.Storage.S3.Bucket) == "" {
|
||||
return fmt.Errorf("clear cache requires pipeline.storage.s3.bucket")
|
||||
}
|
||||
store, err := newCommandObjectStore(ctx, cfg, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("initialize object store for cache cleanup: %w", err)
|
||||
}
|
||||
sessionPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
audioPrefix := artifacts.S3AudioPrefix(sessionPrefix, cfg.Session.Inputs.AudioS3.Prefix)
|
||||
objects, err := store.List(ctx, audioPrefix)
|
||||
if err != nil {
|
||||
return fmt.Errorf("list s3 audio objects under %q: %w", audioPrefix, err)
|
||||
}
|
||||
count := 0
|
||||
for _, obj := range objects {
|
||||
key := strings.TrimSpace(obj.Key)
|
||||
if key == "" || strings.HasSuffix(key, "/") || !cleanIsFlac(key) {
|
||||
continue
|
||||
}
|
||||
cachePath, err := artifacts.S3AudioCachePath(cfg.Pipeline.Cache.Root, cfg.Pipeline.Storage.S3.Bucket, key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
deleted, err := reportCleanScopedFile(out, cfg.Pipeline.Cache.Root, cachePath, "clean.cache.session", dryRun)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if deleted {
|
||||
count++
|
||||
}
|
||||
}
|
||||
if count == 0 {
|
||||
fmt.Fprintf(out, "Cache: no cached S3 audio files found for %s\n", audioPrefix)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func cleanAllAudioCache(cfg *config.PipelineConfig, dryRun bool, out io.Writer) error {
|
||||
if cfg.Storage.S3 == nil || strings.TrimSpace(cfg.Storage.S3.Bucket) == "" {
|
||||
return fmt.Errorf("clear cache requires pipeline.storage.s3.bucket")
|
||||
}
|
||||
namespaceDir, err := artifacts.S3AudioCacheNamespaceDir(cfg.Cache.Root, cfg.Storage.S3.Bucket, cfg.Storage.S3.RootPrefix)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return reportCleanScopedDir(out, cfg.Cache.Root, namespaceDir, "clean.cache.all", dryRun)
|
||||
}
|
||||
|
||||
func reportCleanScopedFile(out io.Writer, root, target, policy string, dryRun bool) (bool, error) {
|
||||
file, err := validateScopedFile(root, target, policy)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if dryRun {
|
||||
if file.Exists {
|
||||
fmt.Fprintf(out, "Would delete cache file: %s\n", file.TargetAbs)
|
||||
return true, nil
|
||||
}
|
||||
fmt.Fprintf(out, "Would skip missing cache file: %s\n", file.TargetAbs)
|
||||
return false, nil
|
||||
}
|
||||
if !file.Exists {
|
||||
fmt.Fprintf(out, "Missing cache file: %s\n", file.TargetAbs)
|
||||
return false, nil
|
||||
}
|
||||
if err := os.Remove(file.TargetAbs); err != nil {
|
||||
return false, fmt.Errorf("cleanup policy %s: remove %q: %w", policy, file.TargetAbs, err)
|
||||
}
|
||||
fmt.Fprintf(out, "Deleted cache file: %s\n", file.TargetAbs)
|
||||
return true, nil
|
||||
}
|
||||
|
||||
func validateScopedFile(root, target, policy string) (scopedDir, error) {
|
||||
cleanRoot := strings.TrimSpace(root)
|
||||
cleanTarget := strings.TrimSpace(target)
|
||||
if cleanRoot == "" {
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: root path is required", policy)
|
||||
}
|
||||
if cleanTarget == "" {
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: target path is required", policy)
|
||||
}
|
||||
rootAbs, err := filepath.Abs(cleanRoot)
|
||||
if err != nil {
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
||||
}
|
||||
targetAbs, err := filepath.Abs(cleanTarget)
|
||||
if err != nil {
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
|
||||
}
|
||||
rel, err := filepath.Rel(rootAbs, targetAbs)
|
||||
if err != nil {
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
|
||||
}
|
||||
if rel == "." {
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
|
||||
}
|
||||
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
|
||||
}
|
||||
info, err := os.Lstat(targetAbs)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: false}, nil
|
||||
}
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
|
||||
}
|
||||
if info.Mode()&os.ModeSymlink != 0 {
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
|
||||
}
|
||||
if info.IsDir() {
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is a directory", policy, targetAbs)
|
||||
}
|
||||
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: true}, nil
|
||||
}
|
||||
|
||||
func cleanIsFlac(path string) bool {
|
||||
return strings.EqualFold(filepath.Ext(path), ".flac")
|
||||
}
|
||||
255
internal/app/clean_test.go
Normal file
255
internal/app/clean_test.go
Normal file
@@ -0,0 +1,255 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
)
|
||||
|
||||
func TestExecuteCleanSessionDeletesWorkAndSpoolButPreservesCache(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
workDir := artifacts.SessionWorkDirForCampaign(workspaceRoot, "sample-campaign", "2026-05-03")
|
||||
spoolDir := artifacts.SessionSpoolDir(filepath.Join(workspaceRoot, "spool"), "sample-campaign", "2026-05-03")
|
||||
cachePath, err := artifacts.S3AudioCachePath(filepath.Join(workspaceRoot, "cache"), "test-bucket", "dnd/campaigns/sample-campaign/sessions/2026-05-03/audio/alice.flac")
|
||||
if err != nil {
|
||||
t.Fatalf("S3AudioCachePath() error = %v", err)
|
||||
}
|
||||
mustWriteTestFile(t, filepath.Join(workDir, "manifest.json"), "{}")
|
||||
mustWriteTestFile(t, filepath.Join(spoolDir, "run-1", "audio", "alice.flac"), "audio")
|
||||
mustWriteTestFile(t, cachePath, "cached-audio")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
cleanAssertMissing(t, workDir)
|
||||
cleanAssertMissing(t, spoolDir)
|
||||
cleanAssertExists(t, cachePath)
|
||||
if !strings.Contains(stdout.String(), "Cache: preserved") {
|
||||
t.Fatalf("stdout = %q, want cache preserved", stdout.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteCleanSessionDryRunDeletesNothing(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
workDir := artifacts.SessionWorkDirForCampaign(workspaceRoot, "sample-campaign", "2026-05-03")
|
||||
spoolDir := artifacts.SessionSpoolDir(filepath.Join(workspaceRoot, "spool"), "sample-campaign", "2026-05-03")
|
||||
mustWriteTestFile(t, filepath.Join(workDir, "manifest.json"), "{}")
|
||||
mustWriteTestFile(t, filepath.Join(spoolDir, "run-1", "audio", "alice.flac"), "audio")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--dry-run"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
cleanAssertExists(t, workDir)
|
||||
cleanAssertExists(t, spoolDir)
|
||||
if !strings.Contains(stdout.String(), "Would delete:") {
|
||||
t.Fatalf("stdout = %q, want dry-run delete plan", stdout.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteCleanMissingSessionPathsSucceeds(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "Missing:") {
|
||||
t.Fatalf("stdout = %q, want missing path output", stdout.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteCleanSessionClearCacheRemovesOnlyS3AudioCache(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
if err := os.WriteFile(sessionPath, []byte(`session_id: 2026-05-03
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
`), 0o644); err != nil {
|
||||
t.Fatalf("write session: %v", err)
|
||||
}
|
||||
|
||||
audioKey := "dnd/campaigns/sample-campaign/sessions/2026-05-03/audio/alice.flac"
|
||||
fake := &storage.FakeBackend{}
|
||||
fake.SeedObject(storage.FakeObject{Key: audioKey, Data: []byte("audio")})
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
cacheRoot := filepath.Join(workspaceRoot, "cache")
|
||||
cachePath, err := artifacts.S3AudioCachePath(cacheRoot, "test-bucket", audioKey)
|
||||
if err != nil {
|
||||
t.Fatalf("S3AudioCachePath() error = %v", err)
|
||||
}
|
||||
otherCachePath, err := artifacts.S3AudioCachePath(cacheRoot, "test-bucket", "dnd/campaigns/other/sessions/2026-05-03/audio/bob.flac")
|
||||
if err != nil {
|
||||
t.Fatalf("S3AudioCachePath() error = %v", err)
|
||||
}
|
||||
mustWriteTestFile(t, cachePath, "cached-audio")
|
||||
mustWriteTestFile(t, otherCachePath, "other-audio")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--clear-cache"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
cleanAssertMissing(t, cachePath)
|
||||
cleanAssertExists(t, otherCachePath)
|
||||
if storeInitCalls != 1 {
|
||||
t.Fatalf("object store init calls = %d, want 1", storeInitCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteCleanLocalAudioClearCacheIsNoop(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--clear-cache"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "Cache: skipped (session does not use audio_s3)") {
|
||||
t.Fatalf("stdout = %q, want local audio cache no-op", stdout.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteCleanAllDeletesWorkAndSpoolContentsButPreservesCache(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, _, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
workRoot := filepath.Join(workspaceRoot, "work")
|
||||
spoolRoot := filepath.Join(workspaceRoot, "spool")
|
||||
cachePath := filepath.Join(workspaceRoot, "cache", "keep.txt")
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "sample-campaign", "2026-05-03", "manifest.json"), "{}")
|
||||
mustWriteTestFile(t, filepath.Join(spoolRoot, "sample-campaign", "2026-05-03", "run-1", "audio", "alice.flac"), "audio")
|
||||
mustWriteTestFile(t, cachePath, "cache")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"clean", "--config", pipelinePath, "--all"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
cleanAssertMissing(t, workRoot)
|
||||
cleanAssertExists(t, spoolRoot)
|
||||
cleanAssertMissing(t, filepath.Join(spoolRoot, "sample-campaign"))
|
||||
cleanAssertExists(t, cachePath)
|
||||
}
|
||||
|
||||
func TestExecuteCleanAllClearCacheRemovesS3AudioNamespaceOnly(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, _, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
cacheRoot := filepath.Join(workspaceRoot, "cache")
|
||||
audioCachePath, err := artifacts.S3AudioCachePath(cacheRoot, "test-bucket", "dnd/campaigns/sample-campaign/sessions/2026-05-03/audio/alice.flac")
|
||||
if err != nil {
|
||||
t.Fatalf("S3AudioCachePath() error = %v", err)
|
||||
}
|
||||
otherCachePath, err := artifacts.S3AudioCachePath(cacheRoot, "test-bucket", "other-root/campaigns/sample-campaign/sessions/2026-05-03/audio/alice.flac")
|
||||
if err != nil {
|
||||
t.Fatalf("S3AudioCachePath() error = %v", err)
|
||||
}
|
||||
mustWriteTestFile(t, audioCachePath, "cached-audio")
|
||||
mustWriteTestFile(t, otherCachePath, "other-cache")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"clean", "--config", pipelinePath, "--all", "--clear-cache"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
cleanAssertMissing(t, audioCachePath)
|
||||
cleanAssertExists(t, otherCachePath)
|
||||
}
|
||||
|
||||
func TestExecuteCleanAllRejectsSessionScopedFlags(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--all"}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "--all cannot be combined") {
|
||||
t.Fatalf("stderr = %q, want --all conflict", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestCleanRequiresSessionID(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"clean"}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "session_id is required unless --all is set") {
|
||||
t.Fatalf("stderr = %q, want missing session-id", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestCleanRejectsUnsafeTargets(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
outside := t.TempDir()
|
||||
if err := reportCleanScopedDir(&bytes.Buffer{}, root, filepath.Join(outside, "target"), "test.outside", false); err == nil {
|
||||
t.Fatal("outside target error = nil, want error")
|
||||
}
|
||||
if err := reportCleanScopedDir(&bytes.Buffer{}, root, root, "test.root", false); err == nil {
|
||||
t.Fatal("root target error = nil, want error")
|
||||
}
|
||||
filePath := filepath.Join(root, "file.txt")
|
||||
mustWriteTestFile(t, filePath, "file")
|
||||
if err := reportCleanScopedDir(&bytes.Buffer{}, root, filePath, "test.file", false); err == nil {
|
||||
t.Fatal("file target error = nil, want error")
|
||||
}
|
||||
symlinkPath := filepath.Join(root, "link")
|
||||
if err := os.Symlink(filepath.Join(root, "missing"), symlinkPath); err != nil {
|
||||
t.Fatalf("Symlink() error = %v", err)
|
||||
}
|
||||
if err := reportCleanScopedDir(&bytes.Buffer{}, root, symlinkPath, "test.symlink", false); err == nil {
|
||||
t.Fatal("symlink target error = nil, want error")
|
||||
}
|
||||
}
|
||||
|
||||
func TestClearIsNotCommandAlias(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"clear"}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), `unknown command: "clear"`) {
|
||||
t.Fatalf("stderr = %q, want unknown clear command", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func cleanAssertExists(t *testing.T, path string) {
|
||||
t.Helper()
|
||||
if _, err := os.Stat(path); err != nil {
|
||||
t.Fatalf("expected %q to exist: %v", path, err)
|
||||
}
|
||||
}
|
||||
|
||||
func cleanAssertMissing(t *testing.T, path string) {
|
||||
t.Helper()
|
||||
if _, err := os.Stat(path); !os.IsNotExist(err) {
|
||||
t.Fatalf("expected %q to be missing, stat err=%v", path, err)
|
||||
}
|
||||
}
|
||||
@@ -7,7 +7,7 @@ import (
|
||||
"strings"
|
||||
)
|
||||
|
||||
var supportedCommands = []string{"run", "plan", "status", "resume", "run-stage"}
|
||||
var supportedCommands = []string{"run", "run-stage", "resume", "analyze", "publish", "clean", "session"}
|
||||
|
||||
// Execute dispatches CLI commands and returns a process exit code.
|
||||
func Execute(args []string, stdout, stderr io.Writer) int {
|
||||
@@ -24,14 +24,18 @@ func Execute(args []string, stdout, stderr io.Writer) int {
|
||||
switch cmd {
|
||||
case "run":
|
||||
err = Run(ctx, cmdArgs, stdout)
|
||||
case "plan":
|
||||
err = Plan(ctx, cmdArgs, stdout)
|
||||
case "status":
|
||||
err = Status(ctx, cmdArgs, stdout)
|
||||
case "resume":
|
||||
err = Resume(ctx, cmdArgs, stdout)
|
||||
case "run-stage":
|
||||
err = RunStage(ctx, cmdArgs, stdout)
|
||||
case "analyze":
|
||||
err = Analyze(ctx, cmdArgs, stdout)
|
||||
case "publish":
|
||||
err = Publish(ctx, cmdArgs, stdout)
|
||||
case "session":
|
||||
err = Session(ctx, cmdArgs, stdout)
|
||||
case "clean":
|
||||
err = Clean(ctx, cmdArgs, stdout)
|
||||
default:
|
||||
fmt.Fprintf(stderr, "unknown command: %q\n\n", cmd)
|
||||
printUsage(stderr)
|
||||
|
||||
@@ -24,19 +24,18 @@ func TestExecuteValidCommands(t *testing.T) {
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||
manifestPath := writeManifestPathForExecute(t)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
args []string
|
||||
wantOut string
|
||||
}{
|
||||
{name: "run", args: []string{"run", "--config", pipelinePath, "--session", sessionPath}, wantOut: "narratio run: session 2026-05-03; executed=9 skipped=0; manifest="},
|
||||
{name: "plan", args: []string{"plan", "--config", pipelinePath, "--session", sessionPath}, wantOut: "prepare: skip\ntranscribe: skip\nmerge: skip\npolish: skip\nnormalize: skip\ntrim: skip\nanalyze: skip\narchive: skip\nnotify: skip"},
|
||||
{name: "status", args: []string{"status", "--manifest", manifestPath}, wantOut: "session_id: 2026-05-03"},
|
||||
{name: "resume", args: []string{"resume", "--config", pipelinePath, "--session", sessionPath}, wantOut: "narratio resume: session 2026-05-03 has no remaining stages"},
|
||||
{name: "run-stage", args: []string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "polish"}, wantOut: "narratio run-stage: stage=polish executed=0 skipped=1 force=false; manifest="},
|
||||
{name: "run", args: []string{"run", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "narratio run: session 2026-05-03; executed=9 skipped=0; manifest="},
|
||||
{name: "session plan", args: []string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "prepare: skip\ntranscribe: skip\nmerge: skip\npolish: skip\nnormalize: skip\ntrim: skip\nanalyze: skip\narchive: skip\nnotify: skip"},
|
||||
{name: "session status", args: []string{"session", "status", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "Session: 2026-05-03"},
|
||||
{name: "resume", args: []string{"resume", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "narratio resume: session 2026-05-03 has no remaining stages"},
|
||||
{name: "run-stage", args: []string{"run-stage", "polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "narratio run-stage: stage=polish executed=0 skipped=1 force=false; manifest="},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
@@ -64,13 +63,13 @@ func TestExecuteMissingRequiredFlags(t *testing.T) {
|
||||
args []string
|
||||
want string
|
||||
}{
|
||||
{name: "run missing flags", args: []string{"run"}, want: "run: no pipeline config path provided and no default pipeline config found; searched:"},
|
||||
{name: "plan missing flags", args: []string{"plan"}, want: "plan: no pipeline config path provided and no default pipeline config found; searched:"},
|
||||
{name: "status missing flags", args: []string{"status"}, want: "status: --manifest is required"},
|
||||
{name: "resume missing flags", args: []string{"resume"}, want: "resume: no pipeline config path provided and no default pipeline config found; searched:"},
|
||||
{name: "run-stage missing name", args: []string{"run-stage", "--config", "a", "--session", "b"}, want: "run-stage: expected exactly one stage name"},
|
||||
{name: "run-stage missing config flags", args: []string{"run-stage", "polish"}, want: "run-stage: no pipeline config path provided and no default pipeline config found; searched:"},
|
||||
{name: "run missing config uses defaults", args: []string{"run", "--session", "session.yml"}, want: "run: no pipeline config path provided and no default pipeline config found; searched:"},
|
||||
{name: "run missing session", args: []string{"run"}, want: "run: session_id is required"},
|
||||
{name: "plan old top-level removed", args: []string{"plan"}, want: `unknown command: "plan"`},
|
||||
{name: "status old top-level removed", args: []string{"status"}, want: `unknown command: "status"`},
|
||||
{name: "resume missing session", args: []string{"resume"}, want: "resume: session_id is required"},
|
||||
{name: "run-stage missing name", args: []string{"run-stage", "--config", "a", "--session", "b"}, want: "run-stage: expected stage name and session_id"},
|
||||
{name: "run-stage missing session", args: []string{"run-stage", "polish"}, want: "run-stage: expected stage name and session_id"},
|
||||
{name: "run missing config uses defaults", args: []string{"run", "2026-05-03", "--session", "session.yml"}, want: "run: no pipeline config path provided and no default pipeline config found; searched:"},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
@@ -94,12 +93,12 @@ func TestExecuteMissingRequiredFlags(t *testing.T) {
|
||||
|
||||
func TestExecuteRunStageUnknownFails(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, "https://example.com/transcribe")
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, "https://example.com/transcribe")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
|
||||
code := Execute([]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "unknown"}, &stdout, &stderr)
|
||||
code := Execute([]string{"run-stage", "unknown", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
@@ -110,14 +109,14 @@ func TestExecuteRunStageUnknownFails(t *testing.T) {
|
||||
|
||||
func TestExecuteRunStageNormalizeIsAccepted(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, "https://example.com/transcribe")
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, "https://example.com/transcribe")
|
||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "processed.json"), `{"segments":[{"id":1}]}`)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "polished.json"), `{"segments":[{"id":1}]}`)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
|
||||
code := Execute([]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "normalize"}, &stdout, &stderr)
|
||||
code := Execute([]string{"run-stage", "normalize", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
@@ -136,19 +135,19 @@ func TestExecuteRunStageTranscribeUsesConfiguredWhisperXServer(t *testing.T) {
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
|
||||
code := Execute([]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "prepare"}, &stdout, &stderr)
|
||||
code := Execute([]string{"run-stage", "prepare", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("prepare exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
|
||||
code = Execute([]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "--force", "transcribe"}, &stdout, &stderr)
|
||||
code = Execute([]string{"run-stage", "transcribe", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("transcribe exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
@@ -188,6 +187,7 @@ func TestExecuteRunStagePolishLoadsCredentialFromSecretsDir(t *testing.T) {
|
||||
t.Setenv("GO_WANT_APP_AUDITA_HELPER", "1")
|
||||
|
||||
pipelinePath := filepath.Join(configDir, "pipeline.yml")
|
||||
campaignPath := writeAppTestCampaignConfig(t, configDir)
|
||||
sessionPath := filepath.Join(configDir, "session.yml")
|
||||
pipelineYAML := `workspace:
|
||||
root: ` + workspaceRoot + `
|
||||
@@ -202,8 +202,6 @@ seriatim:
|
||||
audita:
|
||||
binary: ` + auditaBinary + `
|
||||
llm_api_key_env: OPENROUTER_API_KEY
|
||||
analyzer:
|
||||
timeout: 20m
|
||||
notification:
|
||||
timeout: 10s
|
||||
`
|
||||
@@ -234,12 +232,12 @@ inputs:
|
||||
})
|
||||
|
||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", sessionID)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "merged.json"), `{"schema":"seriatim-intermediate","segments":[]}`)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "base.json"), `{"schema":"seriatim-intermediate","segments":[]}`)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "[]\n")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "--force", "polish"}, &stdout, &stderr)
|
||||
code := Execute([]string{"run-stage", "polish", sessionID, "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
@@ -252,6 +250,7 @@ func TestExecuteRunFailsWhenConfiguredSecretsDirMissing(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
configDir := t.TempDir()
|
||||
pipelinePath := filepath.Join(configDir, "pipeline.yml")
|
||||
campaignPath := writeAppTestCampaignConfig(t, configDir)
|
||||
sessionPath := filepath.Join(configDir, "session.yml")
|
||||
|
||||
pipelineYAML := `workspace:
|
||||
@@ -266,8 +265,6 @@ seriatim:
|
||||
binary: seriatim
|
||||
audita:
|
||||
binary: audita
|
||||
analyzer:
|
||||
timeout: 20m
|
||||
notification:
|
||||
timeout: 10s
|
||||
`
|
||||
@@ -288,7 +285,7 @@ inputs:
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"run", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
||||
code := Execute([]string{"run", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
@@ -305,16 +302,17 @@ func TestExecuteUsesDefaultPipelineConfigPathWhenConfigFlagOmitted(t *testing.T)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||
originalDefaults := append([]string(nil), config.DefaultPipelineConfigSearchPaths...)
|
||||
config.DefaultPipelineConfigSearchPaths = []string{pipelinePath}
|
||||
defer func() {
|
||||
config.DefaultPipelineConfigSearchPaths = originalDefaults
|
||||
}()
|
||||
_ = campaignPath
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"run", "--session", sessionPath}, &stdout, &stderr)
|
||||
code := Execute([]string{"run", "2026-05-03", "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
@@ -323,6 +321,86 @@ func TestExecuteUsesDefaultPipelineConfigPathWhenConfigFlagOmitted(t *testing.T)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteMissingCampaignConfigReportsRegistryPath(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
if err := os.Remove(campaignPath); err != nil {
|
||||
t.Fatalf("remove campaign config: %v", err)
|
||||
}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"run", "2026-05-03", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if stdout.Len() != 0 {
|
||||
t.Fatalf("stdout = %q, want empty", stdout.String())
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "load campaign config") {
|
||||
t.Fatalf("stderr = %q, want campaign discovery failure", stderr.String())
|
||||
}
|
||||
if !strings.Contains(stderr.String(), filepath.ToSlash(filepath.Join("campaigns", "sample-campaign", "campaign.yml"))) {
|
||||
t.Fatalf("stderr = %q, want campaign registry path", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteUsesPipelineDefaultCampaignID(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, _, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "status", "2026-05-03", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "Campaign: sample-campaign") {
|
||||
t.Fatalf("stdout = %q, want default campaign", stdout.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteCampaignIDSelectsRegistryCampaign(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
campaignRoot := filepath.Dir(filepath.Dir(campaignPath))
|
||||
otherDir := filepath.Join(campaignRoot, "icewind")
|
||||
mustWriteTestFile(t, filepath.Join(otherDir, "campaign.yml"), `campaign_id: icewind
|
||||
inputs:
|
||||
speakers_file: ./speakers.yml
|
||||
autocorrect_file: ./autocorrect.yml
|
||||
glossary_file: ./glossary.yml
|
||||
`)
|
||||
mustWriteTestFile(t, filepath.Join(otherDir, "speakers.yml"), "match:\n - speaker: Alice\n match: [\"alice\"]\n")
|
||||
mustWriteTestFile(t, filepath.Join(otherDir, "autocorrect.yml"), "[]\n")
|
||||
mustWriteTestFile(t, filepath.Join(otherDir, "glossary.yml"), "[]\n")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "status", "2026-05-03", "--config", pipelinePath, "--campaign", "icewind", "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "Campaign: icewind") {
|
||||
t.Fatalf("stdout = %q, want selected campaign", stdout.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRejectsCampaignIDAndCampaignFile(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "status", "2026-05-03", "--config", pipelinePath, "--campaign", "sample-campaign", "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "mutually exclusive") {
|
||||
t.Fatalf("stderr = %q, want mutually exclusive error", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteInvalidCommand(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
@@ -359,11 +437,14 @@ func TestExecuteMissingCommand(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func writeValidConfigFiles(t *testing.T, workspaceRoot string, transcribeURL ...string) (string, string) {
|
||||
func writeValidConfigFiles(t *testing.T, workspaceRoot string, transcribeURL ...string) (string, string, string) {
|
||||
t.Helper()
|
||||
|
||||
dir := t.TempDir()
|
||||
pipelinePath := filepath.Join(dir, "pipeline.yml")
|
||||
campaignRoot := filepath.Join(dir, "campaigns")
|
||||
campaignDir := filepath.Join(campaignRoot, "sample-campaign")
|
||||
campaignPath := filepath.Join(campaignDir, "campaign.yml")
|
||||
sessionPath := filepath.Join(dir, "session.yml")
|
||||
url := "https://example.com/transcribe"
|
||||
if len(transcribeURL) > 0 && strings.TrimSpace(transcribeURL[0]) != "" {
|
||||
@@ -377,6 +458,13 @@ func writeValidConfigFiles(t *testing.T, workspaceRoot string, transcribeURL ...
|
||||
|
||||
pipelineYAML := `workspace:
|
||||
root: ` + workspaceRoot + `
|
||||
campaigns:
|
||||
root: ` + campaignRoot + `
|
||||
default_campaign_id: sample-campaign
|
||||
cache:
|
||||
root: ` + filepath.Join(workspaceRoot, "cache") + `
|
||||
spool:
|
||||
root: ` + filepath.Join(workspaceRoot, "spool") + `
|
||||
storage:
|
||||
backend: s3
|
||||
s3:
|
||||
@@ -398,18 +486,16 @@ seriatim:
|
||||
report: true
|
||||
audita:
|
||||
binary: ` + auditaBinary + `
|
||||
analyzer:
|
||||
timeout: 20m
|
||||
artifacts:
|
||||
output_dir: artifacts
|
||||
notification:
|
||||
timeout: 10s
|
||||
`
|
||||
|
||||
sessionYAML := `session_id: 2026-05-03
|
||||
campaign: sample-campaign
|
||||
inputs:
|
||||
audio_dir: ./audio
|
||||
`
|
||||
campaignYAML := `campaign_id: sample-campaign
|
||||
inputs:
|
||||
speakers_file: ./speakers.yml
|
||||
autocorrect_file: ./autocorrect.yml
|
||||
glossary_file: ./glossary.yml
|
||||
@@ -418,16 +504,37 @@ inputs:
|
||||
if err := os.WriteFile(pipelinePath, []byte(pipelineYAML), 0o644); err != nil {
|
||||
t.Fatalf("write pipeline config: %v", err)
|
||||
}
|
||||
if err := os.MkdirAll(campaignDir, 0o755); err != nil {
|
||||
t.Fatalf("create campaign dir: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(campaignPath, []byte(campaignYAML), 0o644); err != nil {
|
||||
t.Fatalf("write campaign config: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(sessionPath, []byte(sessionYAML), 0o644); err != nil {
|
||||
t.Fatalf("write session config: %v", err)
|
||||
}
|
||||
|
||||
mustWriteTestFile(t, filepath.Join(dir, "speakers.yml"), "match:\n - speaker: Alice\n match: [\"alice\"]\n")
|
||||
mustWriteTestFile(t, filepath.Join(dir, "autocorrect.yml"), "[]\n")
|
||||
mustWriteTestFile(t, filepath.Join(dir, "glossary.yml"), "[]\n")
|
||||
mustWriteTestFile(t, filepath.Join(campaignDir, "speakers.yml"), "match:\n - speaker: Alice\n match: [\"alice\"]\n")
|
||||
mustWriteTestFile(t, filepath.Join(campaignDir, "autocorrect.yml"), "[]\n")
|
||||
mustWriteTestFile(t, filepath.Join(campaignDir, "glossary.yml"), "[]\n")
|
||||
mustWriteTestFile(t, filepath.Join(dir, "audio", "alice.flac"), "audio-bytes")
|
||||
|
||||
return pipelinePath, sessionPath
|
||||
return pipelinePath, campaignPath, sessionPath
|
||||
}
|
||||
|
||||
func writeAppTestCampaignConfig(t *testing.T, dir string) string {
|
||||
t.Helper()
|
||||
campaignPath := filepath.Join(dir, "campaign.yml")
|
||||
campaignYAML := `campaign_id: sample-campaign
|
||||
inputs:
|
||||
speakers_file: ./speakers.yml
|
||||
autocorrect_file: ./autocorrect.yml
|
||||
glossary_file: ./glossary.yml
|
||||
`
|
||||
if err := os.WriteFile(campaignPath, []byte(campaignYAML), 0o644); err != nil {
|
||||
t.Fatalf("write campaign.yml: %v", err)
|
||||
}
|
||||
return campaignPath
|
||||
}
|
||||
|
||||
func writeManifestPathForExecute(t *testing.T) string {
|
||||
|
||||
157
internal/app/config_loader.go
Normal file
157
internal/app/config_loader.go
Normal file
@@ -0,0 +1,157 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
type pipelineCampaignConfig struct {
|
||||
PipelinePath string
|
||||
CampaignPath string
|
||||
Pipeline *config.PipelineConfig
|
||||
Campaign *config.CampaignConfig
|
||||
}
|
||||
|
||||
func loadCommandConfig(ctx context.Context, pipelineFlag, campaignFlag, campaignFileFlag, sessionFlag string, sessionOpts config.SessionLoadOptions) (*config.Config, error) {
|
||||
base, err := loadPipelineCampaignConfig(pipelineFlag, campaignFlag, campaignFileFlag)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if explicitSession := strings.TrimSpace(sessionFlag); explicitSession != "" {
|
||||
return config.LoadWithSessionOptions(base.PipelinePath, base.CampaignPath, explicitSession, sessionOpts)
|
||||
}
|
||||
|
||||
discoveredSession, err := discoverSessionConfigPathWithCandidates(config.DefaultSessionConfigSearchPaths)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if discoveredSession.Path != "" {
|
||||
return config.LoadWithSessionOptions(base.PipelinePath, base.CampaignPath, discoveredSession.Path, sessionOpts)
|
||||
}
|
||||
|
||||
sessionID := strings.TrimSpace(sessionOpts.SessionID)
|
||||
if sessionID == "" {
|
||||
return nil, missingSessionConfigError(discoveredSession.Searched, "remote session loading requires a session_id")
|
||||
}
|
||||
|
||||
sessionPrefix := artifacts.S3SessionPrefix(base.Pipeline.Storage.S3.RootPrefix, config.CampaignID(base.Campaign), sessionID)
|
||||
remoteKey := artifacts.S3SessionConfigKey(sessionPrefix)
|
||||
partialCfg := &config.Config{
|
||||
Pipeline: base.Pipeline,
|
||||
Campaign: base.Campaign,
|
||||
PipelinePath: base.PipelinePath,
|
||||
CampaignPath: base.CampaignPath,
|
||||
}
|
||||
store, err := newCommandObjectStore(ctx, partialCfg, nil)
|
||||
if err != nil {
|
||||
return nil, missingSessionConfigError(discoveredSession.Searched, fmt.Sprintf("remote session %q unavailable: %v", remoteKey, err))
|
||||
}
|
||||
|
||||
sessionInfo, err := findRemoteSessionConfig(ctx, store, sessionPrefix, remoteKey)
|
||||
if err != nil {
|
||||
return nil, missingSessionConfigError(discoveredSession.Searched, err.Error())
|
||||
}
|
||||
sessionTempPath, err := downloadRemoteSessionConfig(ctx, store, remoteKey)
|
||||
if err != nil {
|
||||
return nil, missingSessionConfigError(discoveredSession.Searched, fmt.Sprintf("remote session %q download failed: %v", remoteKey, err))
|
||||
}
|
||||
sessionBytes, err := os.ReadFile(sessionTempPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read downloaded remote session %q: %w", sessionTempPath, err)
|
||||
}
|
||||
sessionCfg, err := config.LoadSessionBytesWithOptions("s3://"+s3BucketName(base.Pipeline)+"/"+remoteKey, sessionBytes, sessionOpts)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return config.Resolve(
|
||||
base.PipelinePath,
|
||||
base.Pipeline,
|
||||
base.CampaignPath,
|
||||
base.Campaign,
|
||||
sessionTempPath,
|
||||
sessionCfg,
|
||||
config.SessionSource{
|
||||
Source: "session_config.s3",
|
||||
LocalPath: sessionTempPath,
|
||||
S3Bucket: s3BucketName(base.Pipeline),
|
||||
S3Key: remoteKey,
|
||||
S3Size: sessionInfo.Size,
|
||||
S3ETag: sessionInfo.ETag,
|
||||
SpoolPath: sessionTempPath,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
func loadPipelineCampaignConfig(pipelineFlag, campaignFlag, campaignFileFlag string) (*pipelineCampaignConfig, error) {
|
||||
resolvedPipelinePath, err := resolvePipelineConfigPath(pipelineFlag)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
pipelineCfg, err := config.LoadPipeline(resolvedPipelinePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resolvedCampaignPath, err := resolveCampaignConfigPath(pipelineCfg, campaignFlag, campaignFileFlag)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
campaignCfg, err := config.LoadCampaign(resolvedCampaignPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if selectedID := strings.TrimSpace(campaignFlag); selectedID != "" && strings.TrimSpace(campaignFileFlag) == "" {
|
||||
if got := config.CampaignID(campaignCfg); got != selectedID {
|
||||
return nil, fmt.Errorf("campaign config %q invalid: campaign_id %q does not match selected campaign %q", resolvedCampaignPath, got, selectedID)
|
||||
}
|
||||
}
|
||||
return &pipelineCampaignConfig{
|
||||
PipelinePath: resolvedPipelinePath,
|
||||
CampaignPath: resolvedCampaignPath,
|
||||
Pipeline: pipelineCfg,
|
||||
Campaign: campaignCfg,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func findRemoteSessionConfig(ctx context.Context, store storage.ObjectStore, sessionPrefix, remoteKey string) (storage.ObjectInfo, error) {
|
||||
objects, err := store.List(ctx, sessionPrefix)
|
||||
if err != nil {
|
||||
return storage.ObjectInfo{}, fmt.Errorf("remote session %q list failed: %w", remoteKey, err)
|
||||
}
|
||||
for _, obj := range objects {
|
||||
if obj.Key == remoteKey {
|
||||
return obj, nil
|
||||
}
|
||||
}
|
||||
return storage.ObjectInfo{}, fmt.Errorf("remote session %q not found", remoteKey)
|
||||
}
|
||||
|
||||
func downloadRemoteSessionConfig(ctx context.Context, store storage.ObjectStore, remoteKey string) (string, error) {
|
||||
f, err := os.CreateTemp("", "narratio-session-*.yml")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create temp file: %w", err)
|
||||
}
|
||||
path := f.Name()
|
||||
if err := f.Close(); err != nil {
|
||||
return "", fmt.Errorf("close temp file %q: %w", path, err)
|
||||
}
|
||||
if err := store.Download(ctx, remoteKey, path); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return filepath.Clean(path), nil
|
||||
}
|
||||
|
||||
func s3BucketName(cfg *config.PipelineConfig) string {
|
||||
if cfg == nil || cfg.Storage.S3 == nil {
|
||||
return ""
|
||||
}
|
||||
return strings.TrimSpace(cfg.Storage.S3.Bucket)
|
||||
}
|
||||
21
internal/app/object_store.go
Normal file
21
internal/app/object_store.go
Normal file
@@ -0,0 +1,21 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
func newCommandObjectStore(ctx context.Context, cfg *config.Config, logger *slog.Logger) (storage.ObjectStore, error) {
|
||||
if _, err := loadSecretsFromConfig(cfg, logger); err != nil {
|
||||
return nil, fmt.Errorf("load secrets from files: %w", err)
|
||||
}
|
||||
store, err := newObjectStoreFromConfigFn(ctx, cfg)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initialize object store backend: %w", err)
|
||||
}
|
||||
return store, nil
|
||||
}
|
||||
165
internal/app/object_store_test.go
Normal file
165
internal/app/object_store_test.go
Normal file
@@ -0,0 +1,165 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
func TestNewCommandObjectStoreLoadsSecretsBeforeFactory(t *testing.T) {
|
||||
accessKeyEnv := "NARRATIO_TEST_COMMAND_STORE_KEY_ID"
|
||||
secretKeyEnv := "NARRATIO_TEST_COMMAND_STORE_SECRET"
|
||||
restoreEnvAfterTest(t, accessKeyEnv, secretKeyEnv)
|
||||
|
||||
secretsDir := t.TempDir()
|
||||
mustWriteSecretFile(t, filepath.Join(secretsDir, accessKeyEnv), "loaded-key-id\n")
|
||||
mustWriteSecretFile(t, filepath.Join(secretsDir, secretKeyEnv), "loaded-secret\n")
|
||||
|
||||
cfg := commandObjectStoreTestConfig(secretsDir)
|
||||
fake := &storage.FakeBackend{}
|
||||
called := false
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
called = true
|
||||
if got := os.Getenv(accessKeyEnv); got != "loaded-key-id" {
|
||||
return nil, errors.New("access key was not loaded before object store init")
|
||||
}
|
||||
if got := os.Getenv(secretKeyEnv); got != "loaded-secret" {
|
||||
return nil, errors.New("secret key was not loaded before object store init")
|
||||
}
|
||||
return fake, nil
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
})
|
||||
|
||||
store, err := newCommandObjectStore(context.Background(), cfg, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("newCommandObjectStore() error = %v", err)
|
||||
}
|
||||
if store != fake {
|
||||
t.Fatalf("store = %#v, want fake backend", store)
|
||||
}
|
||||
if !called {
|
||||
t.Fatal("object store factory was not called")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewCommandObjectStorePreservesExistingEnv(t *testing.T) {
|
||||
accessKeyEnv := "NARRATIO_TEST_COMMAND_STORE_EXISTING_KEY_ID"
|
||||
secretKeyEnv := "NARRATIO_TEST_COMMAND_STORE_EXISTING_SECRET"
|
||||
t.Setenv(accessKeyEnv, "existing-key-id")
|
||||
t.Setenv(secretKeyEnv, "existing-secret")
|
||||
|
||||
secretsDir := t.TempDir()
|
||||
mustWriteSecretFile(t, filepath.Join(secretsDir, accessKeyEnv), "file-key-id\n")
|
||||
mustWriteSecretFile(t, filepath.Join(secretsDir, secretKeyEnv), "file-secret\n")
|
||||
|
||||
cfg := commandObjectStoreTestConfig(secretsDir)
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
if got := os.Getenv(accessKeyEnv); got != "existing-key-id" {
|
||||
return nil, errors.New("existing access key was overwritten")
|
||||
}
|
||||
if got := os.Getenv(secretKeyEnv); got != "existing-secret" {
|
||||
return nil, errors.New("existing secret key was overwritten")
|
||||
}
|
||||
return &storage.FakeBackend{}, nil
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
})
|
||||
|
||||
if _, err := newCommandObjectStore(context.Background(), cfg, nil); err != nil {
|
||||
t.Fatalf("newCommandObjectStore() error = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewCommandObjectStoreSecretErrorStopsFactory(t *testing.T) {
|
||||
cfg := commandObjectStoreTestConfig(filepath.Join(t.TempDir(), "missing"))
|
||||
called := false
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
called = true
|
||||
return &storage.FakeBackend{}, nil
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
})
|
||||
|
||||
_, err := newCommandObjectStore(context.Background(), cfg, nil)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if called {
|
||||
t.Fatal("object store factory was called after secret load failure")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "load secrets from files") {
|
||||
t.Fatalf("error = %q, want secret loading context", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewCommandObjectStoreFactoryErrorIsContextual(t *testing.T) {
|
||||
cfg := commandObjectStoreTestConfig("")
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
return nil, errors.New("factory boom")
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
})
|
||||
|
||||
_, err := newCommandObjectStore(context.Background(), cfg, nil)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "initialize object store backend") || !strings.Contains(err.Error(), "factory boom") {
|
||||
t.Fatalf("error = %q, want factory context", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func commandObjectStoreTestConfig(secretsDir string) *config.Config {
|
||||
cfg := &config.Config{
|
||||
Pipeline: &config.PipelineConfig{
|
||||
Storage: config.StorageConfig{
|
||||
Backend: "s3",
|
||||
S3: &config.StorageS3Config{
|
||||
Bucket: "test-bucket",
|
||||
AccessKeyIDEnv: "NARRATIO_TEST_COMMAND_STORE_KEY_ID",
|
||||
SecretKeyEnv: "NARRATIO_TEST_COMMAND_STORE_SECRET",
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
if strings.TrimSpace(secretsDir) != "" {
|
||||
cfg.Pipeline.Secrets = &config.SecretsConfig{EnvDir: secretsDir}
|
||||
}
|
||||
return cfg
|
||||
}
|
||||
|
||||
func restoreEnvAfterTest(t *testing.T, names ...string) {
|
||||
t.Helper()
|
||||
originals := make(map[string]string, len(names))
|
||||
present := make(map[string]bool, len(names))
|
||||
for _, name := range names {
|
||||
value, ok := os.LookupEnv(name)
|
||||
originals[name] = value
|
||||
present[name] = ok
|
||||
_ = os.Unsetenv(name)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
for _, name := range names {
|
||||
if present[name] {
|
||||
_ = os.Setenv(name, originals[name])
|
||||
} else {
|
||||
_ = os.Unsetenv(name)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
1145
internal/app/operator_helpers.go
Normal file
1145
internal/app/operator_helpers.go
Normal file
File diff suppressed because it is too large
Load Diff
962
internal/app/operator_helpers_test.go
Normal file
962
internal/app/operator_helpers_test.go
Normal file
@@ -0,0 +1,962 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
)
|
||||
|
||||
func TestExecuteSessionInitRemoteWritesCanonicalSessionConfig(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "init", "2026-06-07",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--title", "The Black Cabin",
|
||||
"--remote",
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
key := artifacts.S3SessionConfigKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-06-07"))
|
||||
obj, ok := fake.Objects[key]
|
||||
if !ok {
|
||||
t.Fatalf("remote session key %q not uploaded; objects=%v", key, fake.Objects)
|
||||
}
|
||||
if !strings.Contains(string(obj.Data), `session_id: "2026-06-07"`) || !strings.Contains(string(obj.Data), "prefix: audio/") {
|
||||
t.Fatalf("remote session data = %q", string(obj.Data))
|
||||
}
|
||||
if storeInitCalls != 1 {
|
||||
t.Fatalf("object store init calls = %d, want 1", storeInitCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitRemoteUsesDefaultConfigDiscovery(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
withDefaultPipelineCampaignConfigs(t, pipelinePath, campaignPath)
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "init", "2026-06-07",
|
||||
"--remote",
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
key := artifacts.S3SessionConfigKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-06-07"))
|
||||
if _, ok := fake.Objects[key]; !ok {
|
||||
t.Fatalf("remote session key %q not uploaded; objects=%v", key, fake.Objects)
|
||||
}
|
||||
if storeInitCalls != 1 {
|
||||
t.Fatalf("object store init calls = %d, want 1", storeInitCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitLocalUsesDefaultConfigDiscovery(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
withDefaultPipelineCampaignConfigs(t, pipelinePath, campaignPath)
|
||||
outputPath := filepath.Join(t.TempDir(), "session.yml")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "init", "2026-06-07",
|
||||
"--output", outputPath,
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
data, err := os.ReadFile(outputPath)
|
||||
if err != nil {
|
||||
t.Fatalf("read generated session: %v", err)
|
||||
}
|
||||
if !strings.Contains(string(data), `session_id: "2026-06-07"`) || !strings.Contains(string(data), "prefix: audio/") {
|
||||
t.Fatalf("generated session = %q", string(data))
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitExplicitConfigWinsOverDefaults(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
defaultPipeline, defaultCampaign, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
withDefaultPipelineCampaignConfigs(t, defaultPipeline, defaultCampaign)
|
||||
|
||||
explicitDir := t.TempDir()
|
||||
explicitCampaign := filepath.Join(explicitDir, "campaign.yml")
|
||||
if err := os.WriteFile(explicitCampaign, []byte(`campaign_id: explicit-campaign
|
||||
inputs:
|
||||
speakers_file: ./speakers.yml
|
||||
autocorrect_file: ./autocorrect.yml
|
||||
glossary_file: ./glossary.yml
|
||||
`), 0o644); err != nil {
|
||||
t.Fatalf("write explicit campaign: %v", err)
|
||||
}
|
||||
mustWriteTestFile(t, filepath.Join(explicitDir, "speakers.yml"), "match:\n - speaker: Alice\n match: [\"alice\"]\n")
|
||||
mustWriteTestFile(t, filepath.Join(explicitDir, "autocorrect.yml"), "[]\n")
|
||||
mustWriteTestFile(t, filepath.Join(explicitDir, "glossary.yml"), "[]\n")
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "init", "2026-06-07",
|
||||
"--config", defaultPipeline,
|
||||
"--campaign-file", explicitCampaign,
|
||||
"--remote",
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
explicitKey := artifacts.S3SessionConfigKey(artifacts.S3SessionPrefix("dnd", "explicit-campaign", "2026-06-07"))
|
||||
if _, ok := fake.Objects[explicitKey]; !ok {
|
||||
t.Fatalf("explicit campaign remote key %q not uploaded; objects=%v", explicitKey, fake.Objects)
|
||||
}
|
||||
defaultKey := artifacts.S3SessionConfigKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-06-07"))
|
||||
if _, ok := fake.Objects[defaultKey]; ok {
|
||||
t.Fatalf("default campaign key %q uploaded despite explicit campaign override", defaultKey)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitRequiresSessionID(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
withDefaultPipelineCampaignConfigs(t, pipelinePath, campaignPath)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "init", "--remote"}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "session init: session_id is required") {
|
||||
t.Fatalf("stderr = %q, want session-id required error", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitMissingDefaultConfigReportsSearchedPaths(t *testing.T) {
|
||||
origPipelineDefaults := append([]string(nil), config.DefaultPipelineConfigSearchPaths...)
|
||||
config.DefaultPipelineConfigSearchPaths = []string{filepath.Join(t.TempDir(), "missing-pipeline.yml")}
|
||||
t.Cleanup(func() {
|
||||
config.DefaultPipelineConfigSearchPaths = origPipelineDefaults
|
||||
})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "init", "2026-06-07", "--remote"}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "session init: no pipeline config path provided and no default pipeline config found; searched:") {
|
||||
t.Fatalf("stderr = %q, want default pipeline searched-path error", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitRemoteLoadsSecretsBeforeObjectStoreInit(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
withDefaultPipelineCampaignConfigs(t, pipelinePath, campaignPath)
|
||||
accessKeyEnv := "NARRATIO_TEST_SESSION_INIT_OBJECT_KEY_ID"
|
||||
secretKeyEnv := "NARRATIO_TEST_SESSION_INIT_OBJECT_SECRET"
|
||||
restoreEnvAfterTest(t, accessKeyEnv, secretKeyEnv)
|
||||
secretsDir := t.TempDir()
|
||||
mustWriteTestFile(t, filepath.Join(secretsDir, accessKeyEnv), "test-key-id\n")
|
||||
mustWriteTestFile(t, filepath.Join(secretsDir, secretKeyEnv), "test-secret\n")
|
||||
addSecretsToPipelineConfig(t, pipelinePath, secretsDir, accessKeyEnv, secretKeyEnv)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
if os.Getenv(accessKeyEnv) != "test-key-id" || os.Getenv(secretKeyEnv) != "test-secret" {
|
||||
return nil, fmt.Errorf("secrets were not loaded before object store init")
|
||||
}
|
||||
return fake, nil
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "init", "2026-06-07", "--remote"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitLocalRendersCampaignTemplate(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
writeSessionInitTemplate(t, campaignPath, `session_id: "{{ session_id }}"
|
||||
previous_session_id: "{{ previous_session_id }}"
|
||||
date: "{{ date }}"
|
||||
title: "{{ title }}"
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: "{{ audio_s3_prefix }}"
|
||||
`)
|
||||
outputPath := filepath.Join(t.TempDir(), "session.yml")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "init", "2026-06-07",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--previous-session-id", "2026-05-31",
|
||||
"--date", "2026-06-07",
|
||||
"--title", "The Black Cabin",
|
||||
"--audio-s3-prefix", "audio/",
|
||||
"--output", outputPath,
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
data, err := os.ReadFile(outputPath)
|
||||
if err != nil {
|
||||
t.Fatalf("read generated session: %v", err)
|
||||
}
|
||||
got := string(data)
|
||||
for _, want := range []string{
|
||||
`session_id: "2026-06-07"`,
|
||||
`previous_session_id: "2026-05-31"`,
|
||||
`date: "2026-06-07"`,
|
||||
`title: "The Black Cabin"`,
|
||||
`prefix: "audio/"`,
|
||||
} {
|
||||
if !strings.Contains(got, want) {
|
||||
t.Fatalf("generated session = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
if strings.Contains(got, "{{") {
|
||||
t.Fatalf("generated session still contains template placeholder: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitRemoteRendersCampaignTemplate(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
writeSessionInitTemplate(t, campaignPath, `session_id: "{{ session_id }}"
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
`)
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "init", "2026-06-07",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--remote",
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
key := artifacts.S3SessionConfigKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-06-07"))
|
||||
obj, ok := fake.Objects[key]
|
||||
if !ok {
|
||||
t.Fatalf("remote session key %q not uploaded; objects=%v", key, fake.Objects)
|
||||
}
|
||||
if strings.Contains(string(obj.Data), "{{") || !strings.Contains(string(obj.Data), `session_id: "2026-06-07"`) {
|
||||
t.Fatalf("remote session data = %q, want rendered concrete session", string(obj.Data))
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitTemplatePathIsCampaignRelative(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
templateDir := filepath.Join(filepath.Dir(campaignPath), "templates")
|
||||
if err := os.MkdirAll(templateDir, 0o755); err != nil {
|
||||
t.Fatalf("mkdir template dir: %v", err)
|
||||
}
|
||||
templatePath := filepath.Join(templateDir, "session.template.yml")
|
||||
if err := os.WriteFile(templatePath, []byte(`session_id: "{{ session_id }}"
|
||||
inputs:
|
||||
audio_dir: ./audio
|
||||
`), 0o644); err != nil {
|
||||
t.Fatalf("write session template: %v", err)
|
||||
}
|
||||
addSessionTemplateToCampaign(t, campaignPath, "./templates/session.template.yml")
|
||||
outputPath := filepath.Join(t.TempDir(), "session.yml")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "init", "2026-06-07",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--output", outputPath,
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
data, err := os.ReadFile(outputPath)
|
||||
if err != nil {
|
||||
t.Fatalf("read generated session: %v", err)
|
||||
}
|
||||
if !strings.Contains(string(data), `session_id: "2026-06-07"`) {
|
||||
t.Fatalf("generated session = %q, want campaign-relative template output", string(data))
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitTemplateMissingVariableFails(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
writeSessionInitTemplate(t, campaignPath, `session_id: "{{ session_id }}"
|
||||
date: "{{ date }}"
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
`)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "init", "2026-06-07",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--remote",
|
||||
}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "missing required template variable value(s): date") {
|
||||
t.Fatalf("stderr = %q, want missing date variable", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitTemplateUnusedFlagFails(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
writeSessionInitTemplate(t, campaignPath, `session_id: "{{ session_id }}"
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
`)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "init", "2026-06-07",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--title", "Unused Title",
|
||||
"--remote",
|
||||
}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "unused template variable value(s): title") {
|
||||
t.Fatalf("stderr = %q, want unused title variable", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitTemplateStrictDecodeFailure(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
writeSessionInitTemplate(t, campaignPath, `session_id: "{{ session_id }}"
|
||||
unknown: true
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
`)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "init", "2026-06-07",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--remote",
|
||||
}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "strict decode failed") {
|
||||
t.Fatalf("stderr = %q, want strict decode error", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionValidateLoadsSecretsBeforeObjectStoreInit(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
accessKeyEnv := "NARRATIO_TEST_VALIDATE_OBJECT_KEY_ID"
|
||||
secretKeyEnv := "NARRATIO_TEST_VALIDATE_OBJECT_SECRET"
|
||||
restoreEnvAfterTest(t, accessKeyEnv, secretKeyEnv)
|
||||
secretsDir := t.TempDir()
|
||||
mustWriteTestFile(t, filepath.Join(secretsDir, accessKeyEnv), "test-key-id\n")
|
||||
mustWriteTestFile(t, filepath.Join(secretsDir, secretKeyEnv), "test-secret\n")
|
||||
addSecretsToPipelineConfig(t, pipelinePath, secretsDir, accessKeyEnv, secretKeyEnv)
|
||||
if err := os.WriteFile(sessionPath, []byte(`session_id: 2026-05-03
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
`), 0o644); err != nil {
|
||||
t.Fatalf("write session: %v", err)
|
||||
}
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
audioKey := artifacts.S3PromotedArtifactKey(artifacts.S3AudioPrefix(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"), "audio/"), "alice.flac")
|
||||
fake.SeedObject(storage.FakeObject{Key: audioKey, Data: []byte("audio")})
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
if os.Getenv(accessKeyEnv) != "test-key-id" || os.Getenv(secretKeyEnv) != "test-secret" {
|
||||
return nil, fmt.Errorf("secrets were not loaded before object store init")
|
||||
}
|
||||
return fake, nil
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "validate", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stdout=%q stderr=%q", code, stdout.String(), stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "OK audio") {
|
||||
t.Fatalf("stdout = %q, want OK audio", stdout.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteLocksAddListAndRemoveUseRemoteLockStore(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--reason", "manual edit",
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("locks add exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
key := artifacts.S3SessionLocksKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"))
|
||||
obj, ok := fake.Objects[key]
|
||||
if !ok {
|
||||
t.Fatalf("remote locks key %q not uploaded", key)
|
||||
}
|
||||
if !strings.Contains(string(obj.Data), "source: narratio.transcript.final_trimmed") || !strings.Contains(string(obj.Data), "reason: manual edit") {
|
||||
t.Fatalf("lock store data = %q", string(obj.Data))
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
code = Execute([]string{
|
||||
"session", "locks", "2026-05-03",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("locks list exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "- narratio.transcript.final_trimmed origin=remote") {
|
||||
t.Fatalf("stdout = %q, want remote lock", stdout.String())
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
code = Execute([]string{
|
||||
"session", "locks", "remove", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("locks remove exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
store, err := config.LoadArchiveLockStoreBytes("locks.yml", fake.Objects[key].Data, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadArchiveLockStoreBytes() error = %v", err)
|
||||
}
|
||||
if len(store.Locks) != 0 {
|
||||
t.Fatalf("locks after remove = %#v, want empty", store.Locks)
|
||||
}
|
||||
if storeInitCalls != 3 {
|
||||
t.Fatalf("object store init calls = %d, want 3", storeInitCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteLocksAddDuplicateRequiresForce(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--reason", "first",
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("initial locks add exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
code = Execute([]string{
|
||||
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--reason", "second",
|
||||
}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("duplicate locks add exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "pass --force to update") {
|
||||
t.Fatalf("stderr = %q, want force guidance", stderr.String())
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
code = Execute([]string{
|
||||
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--reason", "second",
|
||||
"--force",
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("forced locks add exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
key := artifacts.S3SessionLocksKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"))
|
||||
if !strings.Contains(string(fake.Objects[key].Data), "reason: second") {
|
||||
t.Fatalf("lock store data = %q, want updated reason", string(fake.Objects[key].Data))
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteLocksRequireSessionID(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
args []string
|
||||
want string
|
||||
}{
|
||||
{"list", []string{"session", "locks"}, "locks: session_id is required"},
|
||||
{"add", []string{"session", "locks", "add", "narratio.transcript.final_trimmed"}, "locks add: expected session_id and source id"},
|
||||
{"remove", []string{"session", "locks", "remove", "narratio.transcript.final_trimmed"}, "locks remove: expected session_id and source id"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(tt.args, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), tt.want) {
|
||||
t.Fatalf("stderr = %q, want %q", stderr.String(), tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteLocksCannotModifyStaticLocks(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
addStaticArchiveLockToPipelineConfig(t, pipelinePath, "narratio.transcript.final_trimmed")
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("locks add static lock exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "locked by pipeline config") {
|
||||
t.Fatalf("stderr = %q, want static lock error", stderr.String())
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
code = Execute([]string{
|
||||
"session", "locks", "remove", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("locks remove static lock exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "locked by pipeline config") {
|
||||
t.Fatalf("stderr = %q, want static lock error", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteTopLevelLockAndUnlockAreRemoved(t *testing.T) {
|
||||
tests := []string{"lock", "unlock"}
|
||||
for _, cmd := range tests {
|
||||
t.Run(cmd, func(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{cmd, "narratio.transcript.final_trimmed"}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), `unknown command: "`+cmd+`"`) {
|
||||
t.Fatalf("stderr = %q, want unknown command", stderr.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func withDefaultPipelineCampaignConfigs(t *testing.T, pipelinePath, campaignPath string) {
|
||||
t.Helper()
|
||||
origPipelineDefaults := append([]string(nil), config.DefaultPipelineConfigSearchPaths...)
|
||||
config.DefaultPipelineConfigSearchPaths = []string{pipelinePath}
|
||||
t.Cleanup(func() {
|
||||
config.DefaultPipelineConfigSearchPaths = origPipelineDefaults
|
||||
})
|
||||
_ = campaignPath
|
||||
}
|
||||
|
||||
func writeSessionInitTemplate(t *testing.T, campaignPath, templateYAML string) {
|
||||
t.Helper()
|
||||
templatePath := filepath.Join(filepath.Dir(campaignPath), "session.template.yml")
|
||||
if err := os.WriteFile(templatePath, []byte(templateYAML), 0o644); err != nil {
|
||||
t.Fatalf("write session template: %v", err)
|
||||
}
|
||||
addSessionTemplateToCampaign(t, campaignPath, "./session.template.yml")
|
||||
}
|
||||
|
||||
func addSessionTemplateToCampaign(t *testing.T, campaignPath, templateFile string) {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(campaignPath)
|
||||
if err != nil {
|
||||
t.Fatalf("read campaign config: %v", err)
|
||||
}
|
||||
if strings.Contains(string(data), "session_template_file:") {
|
||||
t.Fatalf("campaign config already has session_template_file: %q", string(data))
|
||||
}
|
||||
updated := "session_template_file: " + templateFile + "\n" + string(data)
|
||||
if err := os.WriteFile(campaignPath, []byte(updated), 0o644); err != nil {
|
||||
t.Fatalf("write campaign config: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteArtifactsListRemoteReportsPromotedAvailability(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
addArchivePromotionsToPipeline(t, pipelinePath, `
|
||||
promote_artifacts:
|
||||
- source: narratio.transcript.final_trimmed
|
||||
dest: transcripts/final.trimmed.json
|
||||
required: true
|
||||
`)
|
||||
fake := &storage.FakeBackend{}
|
||||
trimmedKey := artifacts.S3PromotedArtifactKey(
|
||||
artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"),
|
||||
"transcripts/final.trimmed.json",
|
||||
)
|
||||
fake.SeedObject(storage.FakeObject{Key: trimmedKey, Data: []byte(`{"segments":[]}`)})
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "artifacts", "2026-05-03",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--remote",
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "narratio.transcript.final_trimmed remote=promoted") {
|
||||
t.Fatalf("stdout = %q, want promoted remote availability", stdout.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteArtifactsListRemoteUsesPromotionDestinations(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
addArchivePromotionsToPipeline(t, pipelinePath, `
|
||||
promote_artifacts:
|
||||
- source: narratio.transcript.final
|
||||
dest: transcripts/full.json
|
||||
required: true
|
||||
- source: narratio.bounds.session
|
||||
dest: transcripts/bounds.json
|
||||
required: true
|
||||
`)
|
||||
fake := &storage.FakeBackend{}
|
||||
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
|
||||
fake.SeedObject(storage.FakeObject{Key: artifacts.S3PromotedArtifactKey(sessionPrefix, "transcripts/full.json"), Data: []byte(`{"segments":[]}`)})
|
||||
fake.SeedObject(storage.FakeObject{Key: artifacts.S3PromotedArtifactKey(sessionPrefix, "transcripts/bounds.json"), Data: []byte(`{}`)})
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "artifacts", "2026-05-03",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--remote",
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
out := stdout.String()
|
||||
for _, unwanted := range []string{
|
||||
"narratio.transcript.final remote=missing",
|
||||
"narratio.bounds.session remote=missing",
|
||||
} {
|
||||
if strings.Contains(out, unwanted) {
|
||||
t.Fatalf("stdout = %q, did not want catalog remote marker %q", out, unwanted)
|
||||
}
|
||||
}
|
||||
for _, want := range []string{
|
||||
"narratio.transcript.final dest=transcripts/full.json remote=promoted",
|
||||
"narratio.bounds.session dest=transcripts/bounds.json remote=promoted",
|
||||
} {
|
||||
if !strings.Contains(out, want) {
|
||||
t.Fatalf("stdout = %q, want %q", out, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteStatusReportsRemoteArtifactCatalog(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
addArchivePromotionsToPipeline(t, pipelinePath, `
|
||||
promote_artifacts:
|
||||
- source: narratio.transcript.final_trimmed
|
||||
dest: transcripts/final.trimmed.json
|
||||
required: true
|
||||
- source: narratio.transcript.final
|
||||
dest: transcripts/full.json
|
||||
required: true
|
||||
`)
|
||||
fake := &storage.FakeBackend{}
|
||||
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
|
||||
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
||||
trimmedKey := artifacts.S3PromotedArtifactKey(sessionPrefix, "transcripts/final.trimmed.json")
|
||||
fullKey := artifacts.S3PromotedArtifactKey(sessionPrefix, "transcripts/full.json")
|
||||
lockKey := artifacts.S3SessionLocksKey(sessionPrefix)
|
||||
fake.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||
fake.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, "2026-05-03", "sample-campaign")})
|
||||
fake.SeedObject(storage.FakeObject{Key: trimmedKey, Data: []byte(`{"segments":[]}`)})
|
||||
fake.SeedObject(storage.FakeObject{Key: fullKey, Data: []byte(`{"segments":[]}`)})
|
||||
fake.SeedObject(storage.FakeObject{Key: lockKey, Data: []byte("locks:\n - source: narratio.transcript.final_trimmed\n reason: remote review\n")})
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "status", "2026-05-03",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
out := stdout.String()
|
||||
for _, want := range []string{
|
||||
"Remote outputs:",
|
||||
"Built-in:",
|
||||
"Configured:",
|
||||
"Previous-session:",
|
||||
"Promoted:",
|
||||
"narratio.transcript.final_trimmed locked",
|
||||
"narratio.transcript.final_trimmed locked remote=promoted",
|
||||
"narratio.transcript.final dest=transcripts/full.json remote=promoted",
|
||||
} {
|
||||
if !strings.Contains(out, want) {
|
||||
t.Fatalf("stdout = %q, want %q", out, want)
|
||||
}
|
||||
}
|
||||
if strings.Contains(out, "narratio.transcript.base remote=missing") {
|
||||
t.Fatalf("stdout = %q, did not want catalog remote marker", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteStatusReportsRemoteArtifactCatalogErrorsWithoutFailing(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
addArchivePromotionsToPipeline(t, pipelinePath, `
|
||||
promote_artifacts:
|
||||
- source: narratio.transcript.final_trimmed
|
||||
dest: transcripts/final.trimmed.json
|
||||
required: true
|
||||
`)
|
||||
fake := &storage.FakeBackend{ExistsErr: fmt.Errorf("exists failed")}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "status", "2026-05-03",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
out := stdout.String()
|
||||
if !strings.Contains(out, "Remote archive: missing or unavailable:") {
|
||||
t.Fatalf("stdout = %q, want remote archive unavailable state", out)
|
||||
}
|
||||
if !strings.Contains(out, "Remote outputs:") || !strings.Contains(out, "narratio.transcript.final_trimmed remote=error") {
|
||||
t.Fatalf("stdout = %q, want remote output error state", out)
|
||||
}
|
||||
if !strings.Contains(out, "Archive locks: error:") {
|
||||
t.Fatalf("stdout = %q, want archive locks error", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteArchiveLoadsRemoteLocks(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidArchiveConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
lockKey := artifacts.S3SessionLocksKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"))
|
||||
fake.SeedObject(storage.FakeObject{Key: lockKey, Data: []byte("locks:\n - source: narratio.transcript.final_trimmed\n reason: remote review\n")})
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze"} {
|
||||
// The archive stage only checks the manifest statuses and source files.
|
||||
_ = stageName
|
||||
}
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.trimmed.json"), `{"segments":[]}`)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"run-stage", "archive", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
promotedKey := artifacts.S3PromotedArtifactKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"), "transcripts/final.trimmed.json")
|
||||
if _, ok := fake.Objects[promotedKey]; ok {
|
||||
t.Fatalf("locked promoted key %q was uploaded", promotedKey)
|
||||
}
|
||||
}
|
||||
|
||||
func addArchivePromotionsToPipeline(t *testing.T, pipelinePath, archiveYAML string) {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(pipelinePath)
|
||||
if err != nil {
|
||||
t.Fatalf("read pipeline: %v", err)
|
||||
}
|
||||
updated := strings.Replace(string(data), " upload_run: false\n", " upload_run: false\n"+archiveYAML, 1)
|
||||
if updated == string(data) {
|
||||
t.Fatalf("pipeline %q did not contain archive upload_run marker", pipelinePath)
|
||||
}
|
||||
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
|
||||
t.Fatalf("write pipeline: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func writeValidArchiveConfigFiles(t *testing.T, workspaceRoot string) (string, string, string) {
|
||||
t.Helper()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
data, err := os.ReadFile(pipelinePath)
|
||||
if err != nil {
|
||||
t.Fatalf("read pipeline: %v", err)
|
||||
}
|
||||
updated := strings.Replace(string(data), "upload_run: false", "upload_run: true", 1)
|
||||
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
|
||||
t.Fatalf("write pipeline: %v", err)
|
||||
}
|
||||
ctx := context.Background()
|
||||
cfg, err := config.LoadWithSessionOptions(pipelinePath, campaignPath, sessionPath, config.SessionLoadOptions{})
|
||||
if err != nil {
|
||||
t.Fatalf("LoadWithSessionOptions() error = %v", err)
|
||||
}
|
||||
store := &manifest.LocalStore{}
|
||||
m := manifest.New("2026-05-03", nowUTC())
|
||||
m.Campaign = "sample-campaign"
|
||||
m.RunID = "20260521T160000Z-test"
|
||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze"} {
|
||||
m.MarkStageSucceeded(name, nowUTC(), nil)
|
||||
}
|
||||
path := artifacts.SessionManifestPathForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
if err := store.Save(ctx, path, m); err != nil {
|
||||
t.Fatalf("save manifest: %v", err)
|
||||
}
|
||||
runManifestPath := artifacts.SessionRunManifestPathForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID, m.RunID)
|
||||
if err := os.MkdirAll(filepath.Dir(runManifestPath), 0o755); err != nil {
|
||||
t.Fatalf("mkdir run manifest: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(runManifestPath, []byte("{}\n"), 0o644); err != nil {
|
||||
t.Fatalf("write run manifest: %v", err)
|
||||
}
|
||||
return pipelinePath, campaignPath, sessionPath
|
||||
}
|
||||
|
||||
func addStaticArchiveLockToPipelineConfig(t *testing.T, pipelinePath, source string) {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(pipelinePath)
|
||||
if err != nil {
|
||||
t.Fatalf("read pipeline: %v", err)
|
||||
}
|
||||
updated := strings.Replace(
|
||||
string(data),
|
||||
"archive:\n enabled: true\n upload_run: false\n",
|
||||
"archive:\n enabled: true\n upload_run: false\n locks:\n - source: "+source+"\n reason: static review\n",
|
||||
1,
|
||||
)
|
||||
if updated == string(data) {
|
||||
t.Fatalf("archive section not found in pipeline config")
|
||||
}
|
||||
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
|
||||
t.Fatalf("write pipeline: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"io"
|
||||
"log/slog"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
@@ -16,35 +17,45 @@ import (
|
||||
|
||||
// Plan validates configuration, prepares the local workdir, and prints stage order.
|
||||
func Plan(ctx context.Context, args []string, out io.Writer) error {
|
||||
positionalSessionID, args := pullLeadingSessionID(args)
|
||||
fs := flag.NewFlagSet("plan", flag.ContinueOnError)
|
||||
fs.SetOutput(io.Discard)
|
||||
|
||||
var pipelinePath string
|
||||
var campaignPath string
|
||||
var campaignFilePath string
|
||||
var sessionPath string
|
||||
var sessionID string
|
||||
var previousSessionID string
|
||||
var force bool
|
||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||
fs.StringVar(&sessionID, "session-id", "", "session identifier for session.yml templates")
|
||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
||||
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return fmt.Errorf("plan: invalid flags: %w", err)
|
||||
}
|
||||
if fs.NArg() != 0 {
|
||||
return fmt.Errorf("plan: unexpected positional arguments")
|
||||
if positionalSessionID == "" {
|
||||
if err := applyParsedSessionIDArg("plan", fs, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
if fs.NArg() != 0 {
|
||||
return fmt.Errorf("plan: unexpected positional arguments")
|
||||
}
|
||||
if err := applyPositionalSessionID("plan", positionalSessionID, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
resolvedPipelinePath, err := resolvePipelineConfigPath(pipelinePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("plan: %w", err)
|
||||
if strings.TrimSpace(sessionID) == "" {
|
||||
return fmt.Errorf("plan: session_id is required")
|
||||
}
|
||||
resolvedSessionPath, err := resolveSessionConfigPath(sessionPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("plan: %w", err)
|
||||
}
|
||||
|
||||
cfg, err := config.LoadWithSessionOptions(resolvedPipelinePath, resolvedSessionPath, config.SessionLoadOptions{
|
||||
SessionID: sessionID,
|
||||
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
||||
SessionID: sessionID,
|
||||
PreviousSessionID: previousSessionID,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("plan: %w", err)
|
||||
@@ -72,7 +83,7 @@ func Plan(ctx context.Context, args []string, out io.Writer) error {
|
||||
|
||||
runCount := 0
|
||||
skipCount := 0
|
||||
if _, err := fmt.Fprintf(out, "narratio plan: workdir prepared at %s\n", paths.Root); err != nil {
|
||||
if _, err := fmt.Fprintf(out, "narratio session plan: workdir prepared at %s\n", paths.Root); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, d := range decisions {
|
||||
|
||||
@@ -15,16 +15,16 @@ import (
|
||||
|
||||
func TestPlanCreatesAndReusesWorkdir(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var out bytes.Buffer
|
||||
args := []string{"--config", pipelinePath, "--session", sessionPath}
|
||||
args := []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}
|
||||
|
||||
if err := Plan(context.Background(), args, &out); err != nil {
|
||||
t.Fatalf("first Plan() error = %v", err)
|
||||
}
|
||||
got := out.String()
|
||||
if !strings.Contains(got, "narratio plan: workdir prepared at") {
|
||||
if !strings.Contains(got, "narratio session plan: workdir prepared at") {
|
||||
t.Fatalf("first output = %q, want workdir prepared", got)
|
||||
}
|
||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "archive", "notify"} {
|
||||
@@ -55,14 +55,14 @@ func TestPlanCreatesAndReusesWorkdir(t *testing.T) {
|
||||
if err := Plan(context.Background(), args, &out); err != nil {
|
||||
t.Fatalf("second Plan() error = %v", err)
|
||||
}
|
||||
if !strings.Contains(out.String(), "narratio plan: workdir prepared at") {
|
||||
if !strings.Contains(out.String(), "narratio session plan: workdir prepared at") {
|
||||
t.Fatalf("second output = %q, want workdir prepared", out.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestPlanShowsRunAndSkipFromManifest(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||
|
||||
store := &manifest.LocalStore{}
|
||||
@@ -74,7 +74,7 @@ func TestPlanShowsRunAndSkipFromManifest(t *testing.T) {
|
||||
}
|
||||
|
||||
var out bytes.Buffer
|
||||
if err := Plan(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath}, &out); err != nil {
|
||||
if err := Plan(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out); err != nil {
|
||||
t.Fatalf("Plan() error = %v", err)
|
||||
}
|
||||
got := out.String()
|
||||
@@ -93,6 +93,7 @@ func TestPlanFailsWhenConfiguredSecretsDirMissing(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
configDir := t.TempDir()
|
||||
pipelinePath := filepath.Join(configDir, "pipeline.yml")
|
||||
campaignPath := writeAppTestCampaignConfig(t, configDir)
|
||||
sessionPath := filepath.Join(configDir, "session.yml")
|
||||
|
||||
pipelineYAML := `workspace:
|
||||
@@ -107,8 +108,6 @@ seriatim:
|
||||
binary: seriatim
|
||||
audita:
|
||||
binary: audita
|
||||
analyzer:
|
||||
timeout: 20m
|
||||
notification:
|
||||
timeout: 10s
|
||||
`
|
||||
@@ -128,7 +127,7 @@ inputs:
|
||||
}
|
||||
|
||||
var out bytes.Buffer
|
||||
err := Plan(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath}, &out)
|
||||
err := Plan(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
|
||||
@@ -153,53 +153,70 @@ func archiveCleanupEligible(cfg *config.Config, sr *manifest.StageRecord) (bool,
|
||||
return true, ""
|
||||
}
|
||||
|
||||
type scopedDir struct {
|
||||
RootAbs string
|
||||
TargetAbs string
|
||||
Exists bool
|
||||
}
|
||||
|
||||
func removeRunScopedDir(root, target, policy string) error {
|
||||
dir, err := validateScopedDir(root, target, policy)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !dir.Exists {
|
||||
return nil
|
||||
}
|
||||
if err := os.RemoveAll(dir.TargetAbs); err != nil {
|
||||
return fmt.Errorf("cleanup policy %s: remove %q: %w", policy, dir.TargetAbs, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func validateScopedDir(root, target, policy string) (scopedDir, error) {
|
||||
cleanRoot := strings.TrimSpace(root)
|
||||
cleanTarget := strings.TrimSpace(target)
|
||||
if cleanRoot == "" {
|
||||
return fmt.Errorf("cleanup policy %s: root path is required", policy)
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: root path is required", policy)
|
||||
}
|
||||
if cleanTarget == "" {
|
||||
return fmt.Errorf("cleanup policy %s: target path is required", policy)
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: target path is required", policy)
|
||||
}
|
||||
|
||||
rootAbs, err := filepath.Abs(cleanRoot)
|
||||
if err != nil {
|
||||
return fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
||||
}
|
||||
targetAbs, err := filepath.Abs(cleanTarget)
|
||||
if err != nil {
|
||||
return fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
|
||||
}
|
||||
|
||||
rel, err := filepath.Rel(rootAbs, targetAbs)
|
||||
if err != nil {
|
||||
return fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
|
||||
}
|
||||
if rel == "." {
|
||||
return fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
|
||||
}
|
||||
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
return fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
|
||||
}
|
||||
|
||||
info, err := os.Lstat(targetAbs)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil
|
||||
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: false}, nil
|
||||
}
|
||||
return fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
|
||||
}
|
||||
if info.Mode()&os.ModeSymlink != 0 {
|
||||
return fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return fmt.Errorf("cleanup policy %s: target %q is not a directory", policy, targetAbs)
|
||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is not a directory", policy, targetAbs)
|
||||
}
|
||||
if err := os.RemoveAll(targetAbs); err != nil {
|
||||
return fmt.Errorf("cleanup policy %s: remove %q: %w", policy, targetAbs, err)
|
||||
}
|
||||
return nil
|
||||
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: true}, nil
|
||||
}
|
||||
|
||||
func asString(v any) string {
|
||||
|
||||
@@ -82,6 +82,7 @@ func TestPostArchiveCleanupWorkdirOnly(t *testing.T) {
|
||||
|
||||
assertExists(t, cfg.Pipeline.Workspace.Root)
|
||||
assertExists(t, seed.otherRunDir)
|
||||
assertExists(t, seed.previousCachePath)
|
||||
assertMissing(t, seed.runWorkDir)
|
||||
assertExists(t, seed.spoolAudioDir)
|
||||
}
|
||||
@@ -98,6 +99,7 @@ func TestPostArchiveCleanupBothPolicies(t *testing.T) {
|
||||
assertMissing(t, seed.spoolAudioDir)
|
||||
assertMissing(t, seed.runWorkDir)
|
||||
assertExists(t, seed.otherRunDir)
|
||||
assertExists(t, seed.previousCachePath)
|
||||
}
|
||||
|
||||
func TestPostArchiveCleanupNotRunWhenArchiveFails(t *testing.T) {
|
||||
@@ -195,7 +197,7 @@ func TestPostArchiveCleanupNotRunWhenPromotionIsMissing(t *testing.T) {
|
||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
||||
cfg.Pipeline.Workspace.CleanupAfterArchive = true
|
||||
cfg.Pipeline.Archive.PromoteArtifacts = []config.ArchivePromotionRule{
|
||||
{From: "artifacts/missing.md", To: "artifacts/missing.md", Required: boolPtr(true)},
|
||||
{Source: "narratio.transcript.base", Dest: "transcripts/base.json", Required: boolPtr(true)},
|
||||
}
|
||||
|
||||
archiveStageImpl, err := stage.Select("archive")
|
||||
@@ -203,7 +205,7 @@ func TestPostArchiveCleanupNotRunWhenPromotionIsMissing(t *testing.T) {
|
||||
t.Fatalf("Select(archive) error = %v", err)
|
||||
}
|
||||
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
||||
if err == nil || !strings.Contains(err.Error(), "required promotion source missing") {
|
||||
if err == nil || !strings.Contains(err.Error(), "required promotion source unavailable") {
|
||||
t.Fatalf("executeStages() error = %v, want promotion-missing failure", err)
|
||||
}
|
||||
|
||||
@@ -256,11 +258,12 @@ func TestPostArchiveCleanupNotRunWhenCurrentPointerUploadFails(t *testing.T) {
|
||||
}
|
||||
|
||||
type cleanupSeed struct {
|
||||
runWorkDir string
|
||||
otherRunDir string
|
||||
spoolAudioDir string
|
||||
localSourceAudio string
|
||||
sessionPrefix string
|
||||
runWorkDir string
|
||||
otherRunDir string
|
||||
spoolAudioDir string
|
||||
localSourceAudio string
|
||||
previousCachePath string
|
||||
sessionPrefix string
|
||||
}
|
||||
|
||||
func cleanupFixtureConfig(t *testing.T) (*config.Config, cleanupSeed) {
|
||||
@@ -274,11 +277,18 @@ func cleanupFixtureConfig(t *testing.T) (*config.Config, cleanupSeed) {
|
||||
runWorkDir := artifacts.SessionRunRootForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID, runID)
|
||||
otherRunDir := artifacts.SessionRunRootForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID, "20260516T010204Z-5e6f7a8b")
|
||||
spoolAudioDir := artifacts.SessionSpoolAudioDir(cfg.Pipeline.Spool.Root, cfg.Session.Campaign, cfg.Session.SessionID, runID)
|
||||
previousCachePath := artifacts.SessionPreviousArtifactPathForCampaign(
|
||||
cfg.Pipeline.Workspace.Root,
|
||||
cfg.Session.Campaign,
|
||||
cfg.Session.SessionID,
|
||||
"session_recap.md",
|
||||
)
|
||||
|
||||
mustWriteFile(t, filepath.Join(runWorkDir, "manifest.json"), "{}\n")
|
||||
mustWriteFile(t, filepath.Join(runWorkDir, "logs", "stage.log"), "log\n")
|
||||
mustWriteFile(t, filepath.Join(otherRunDir, "logs", "stage.log"), "other\n")
|
||||
mustWriteFile(t, filepath.Join(spoolAudioDir, "speaker.flac"), "flac\n")
|
||||
mustWriteFile(t, previousCachePath, "# previous recap\n")
|
||||
|
||||
localSourceAudio := filepath.Join(filepath.Dir(cfg.SessionPath), "audio", "alice.flac")
|
||||
mustWriteFile(t, localSourceAudio, "source\n")
|
||||
@@ -301,11 +311,12 @@ func cleanupFixtureConfig(t *testing.T) (*config.Config, cleanupSeed) {
|
||||
}
|
||||
|
||||
return cfg, cleanupSeed{
|
||||
runWorkDir: runWorkDir,
|
||||
otherRunDir: otherRunDir,
|
||||
spoolAudioDir: spoolAudioDir,
|
||||
localSourceAudio: localSourceAudio,
|
||||
sessionPrefix: seed.S3SessionPrefix,
|
||||
runWorkDir: runWorkDir,
|
||||
otherRunDir: otherRunDir,
|
||||
spoolAudioDir: spoolAudioDir,
|
||||
localSourceAudio: localSourceAudio,
|
||||
previousCachePath: previousCachePath,
|
||||
sessionPrefix: seed.S3SessionPrefix,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -322,8 +333,15 @@ func archiveStageCleanupFixture(t *testing.T) (*config.Config, cleanupSeed, stri
|
||||
Enabled: boolPtr(true),
|
||||
UploadRun: boolPtr(true),
|
||||
PromoteArtifacts: []config.ArchivePromotionRule{
|
||||
{From: "transcripts/trimmed.json", To: "transcripts/trimmed.json", Required: boolPtr(true)},
|
||||
{From: "artifacts/session_recap.md", To: "artifacts/session_recap.md", Required: boolPtr(true)},
|
||||
{Source: "narratio.transcript.final_trimmed", Dest: "transcripts/final.trimmed.json", Required: boolPtr(true)},
|
||||
{Source: "narratio.artifact.session_recap", Dest: "artifacts/session_recap.md", Required: boolPtr(true)},
|
||||
},
|
||||
}
|
||||
cfg.Pipeline.Scriptorium = &config.ScriptoriumConfig{
|
||||
Artifacts: map[string]config.ScriptoriumArtifactConfig{
|
||||
"session_recap": {
|
||||
OutputPath: "artifacts/session_recap.md",
|
||||
},
|
||||
},
|
||||
}
|
||||
writeArchiveFixtureRunFiles(
|
||||
@@ -353,14 +371,14 @@ func writeArchiveFixtureRunFiles(t *testing.T, runWorkDir, sessionRoot string) {
|
||||
t.Helper()
|
||||
mustWriteFile(t, filepath.Join(runWorkDir, "prepare", "inputs", "session.yml"), "session_id: 2026-05-03\n")
|
||||
mustWriteFile(t, filepath.Join(runWorkDir, "transcribe", "outputs", "transcripts", "raw", "speaker.json"), "{}\n")
|
||||
mustWriteFile(t, filepath.Join(runWorkDir, "trim", "outputs", "transcripts", "trimmed.json"), "{}\n")
|
||||
mustWriteFile(t, filepath.Join(runWorkDir, "trim", "outputs", "transcripts", "final.trimmed.json"), "{\"segments\":[]}\n")
|
||||
mustWriteFile(t, filepath.Join(runWorkDir, "analyze", "outputs", "artifacts", "session_recap.md"), "# recap\n")
|
||||
mustWriteFile(t, filepath.Join(runWorkDir, "polish", "reports", "audita.report.json"), "{}\n")
|
||||
mustWriteFile(t, filepath.Join(runWorkDir, "merge", "config", "seriatim.generated.yml"), "key: value\n")
|
||||
mustWriteFile(t, filepath.Join(runWorkDir, "logs", "audita.stderr.log"), "stderr\n")
|
||||
mustWriteFile(t, filepath.Join(runWorkDir, "manifest.json"), "{}\n")
|
||||
|
||||
mustWriteFile(t, filepath.Join(sessionRoot, "transcripts", "trimmed.json"), "{}\n")
|
||||
mustWriteFile(t, filepath.Join(sessionRoot, "transcripts", "final.trimmed.json"), "{\"segments\":[]}\n")
|
||||
mustWriteFile(t, filepath.Join(sessionRoot, "artifacts", "session_recap.md"), "# recap\n")
|
||||
}
|
||||
|
||||
|
||||
157
internal/app/remote_locks.go
Normal file
157
internal/app/remote_locks.go
Normal file
@@ -0,0 +1,157 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
type effectiveLocks struct {
|
||||
Static []config.ArchiveLockRule
|
||||
Remote []config.ArchiveLockRule
|
||||
All []config.ArchiveLockRule
|
||||
Key string
|
||||
}
|
||||
|
||||
func remoteLocksKey(cfg *config.Config) (string, error) {
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Session == nil {
|
||||
return "", fmt.Errorf("resolved config is required")
|
||||
}
|
||||
if cfg.Pipeline.Storage.S3 == nil {
|
||||
return "", fmt.Errorf("pipeline.storage.s3 configuration is required")
|
||||
}
|
||||
sessionPrefix := artifacts.S3SessionPrefix(
|
||||
cfg.Pipeline.Storage.S3.RootPrefix,
|
||||
cfg.Session.Campaign,
|
||||
cfg.Session.SessionID,
|
||||
)
|
||||
return artifacts.S3SessionLocksKey(sessionPrefix), nil
|
||||
}
|
||||
|
||||
func loadRemoteLockStore(ctx context.Context, cfg *config.Config, store storage.ObjectStore) (*config.ArchiveLockStore, string, error) {
|
||||
key, err := remoteLocksKey(cfg)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
exists, err := store.Exists(ctx, key)
|
||||
if err != nil {
|
||||
return nil, key, fmt.Errorf("check remote locks %q: %w", key, err)
|
||||
}
|
||||
if !exists {
|
||||
return &config.ArchiveLockStore{}, key, nil
|
||||
}
|
||||
tmp, err := downloadObjectToTemp(ctx, store, key, "narratio-locks-*.yml")
|
||||
if err != nil {
|
||||
return nil, key, fmt.Errorf("download remote locks %q: %w", key, err)
|
||||
}
|
||||
defer func() { _ = os.Remove(tmp) }()
|
||||
data, err := os.ReadFile(tmp)
|
||||
if err != nil {
|
||||
return nil, key, fmt.Errorf("read remote locks %q: %w", key, err)
|
||||
}
|
||||
lockStore, err := config.LoadArchiveLockStoreBytes("s3://"+s3BucketName(cfg.Pipeline)+"/"+key, data, cfg.Pipeline.Scriptorium)
|
||||
if err != nil {
|
||||
return nil, key, err
|
||||
}
|
||||
return lockStore, key, nil
|
||||
}
|
||||
|
||||
func loadEffectiveLocks(ctx context.Context, cfg *config.Config, store storage.ObjectStore) (*effectiveLocks, error) {
|
||||
staticLocks := staticArchiveLocks(cfg)
|
||||
if store == nil {
|
||||
return &effectiveLocks{
|
||||
Static: staticLocks,
|
||||
All: append([]config.ArchiveLockRule(nil), staticLocks...),
|
||||
}, nil
|
||||
}
|
||||
lockStore, key, err := loadRemoteLockStore(ctx, cfg, store)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
remoteLocks := append([]config.ArchiveLockRule(nil), lockStore.Locks...)
|
||||
return &effectiveLocks{
|
||||
Static: staticLocks,
|
||||
Remote: remoteLocks,
|
||||
All: config.MergeArchiveLockRules(staticLocks, remoteLocks),
|
||||
Key: key,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func staticArchiveLocks(cfg *config.Config) []config.ArchiveLockRule {
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Archive == nil {
|
||||
return nil
|
||||
}
|
||||
return append([]config.ArchiveLockRule(nil), cfg.Pipeline.Archive.Locks...)
|
||||
}
|
||||
|
||||
func applyEffectiveLocks(cfg *config.Config, locks []config.ArchiveLockRule) {
|
||||
if cfg == nil || cfg.Pipeline == nil {
|
||||
return
|
||||
}
|
||||
if cfg.Pipeline.Archive == nil {
|
||||
cfg.Pipeline.Archive = &config.ArchiveConfig{}
|
||||
}
|
||||
cfg.Pipeline.Archive.Locks = append([]config.ArchiveLockRule(nil), locks...)
|
||||
}
|
||||
|
||||
func uploadRemoteLockStore(ctx context.Context, store storage.ObjectStore, key string, lockStore *config.ArchiveLockStore) error {
|
||||
data, err := config.MarshalArchiveLockStore(lockStore)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tmp, err := os.CreateTemp("", "narratio-locks-upload-*.yml")
|
||||
if err != nil {
|
||||
return fmt.Errorf("create lock store temp file: %w", err)
|
||||
}
|
||||
tmpPath := tmp.Name()
|
||||
defer func() { _ = os.Remove(tmpPath) }()
|
||||
if _, err := tmp.Write(data); err != nil {
|
||||
_ = tmp.Close()
|
||||
return fmt.Errorf("write lock store temp file: %w", err)
|
||||
}
|
||||
if err := tmp.Close(); err != nil {
|
||||
return fmt.Errorf("close lock store temp file: %w", err)
|
||||
}
|
||||
if _, err := store.Upload(ctx, tmpPath, key, storage.UploadOptions{ContentType: "application/x-yaml; charset=utf-8"}); err != nil {
|
||||
return fmt.Errorf("upload remote locks %q: %w", key, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func lockSourceSet(locks []config.ArchiveLockRule) map[string]config.ArchiveLockRule {
|
||||
out := make(map[string]config.ArchiveLockRule, len(locks))
|
||||
for _, lock := range locks {
|
||||
source := strings.TrimSpace(lock.Source)
|
||||
if source == "" {
|
||||
continue
|
||||
}
|
||||
lock.Source = source
|
||||
lock.Reason = strings.TrimSpace(lock.Reason)
|
||||
out[source] = lock
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func writeLocalFile(path string, data []byte, force bool) error {
|
||||
cleaned := filepath.Clean(strings.TrimSpace(path))
|
||||
if cleaned == "" || cleaned == "." {
|
||||
return fmt.Errorf("output path is required")
|
||||
}
|
||||
if !force {
|
||||
if _, err := os.Stat(cleaned); err == nil {
|
||||
return fmt.Errorf("output file %q already exists; pass --force to overwrite", cleaned)
|
||||
} else if err != nil && !os.IsNotExist(err) {
|
||||
return fmt.Errorf("check output file %q: %w", cleaned, err)
|
||||
}
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(cleaned), 0o755); err != nil {
|
||||
return fmt.Errorf("create output directory: %w", err)
|
||||
}
|
||||
return os.WriteFile(cleaned, data, 0o644)
|
||||
}
|
||||
294
internal/app/remote_session_test.go
Normal file
294
internal/app/remote_session_test.go
Normal file
@@ -0,0 +1,294 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
func TestExecuteRemoteSessionFallbackLoadsFromObjectStore(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
remoteKey := seedRemoteSessionConfig(t, fake, "2026-05-03", `session_id: 2026-05-03
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
`)
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if storeInitCalls != 1 {
|
||||
t.Fatalf("object store init calls = %d, want 1", storeInitCalls)
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "narratio session plan: workdir prepared") {
|
||||
t.Fatalf("stdout = %q, want plan output", stdout.String())
|
||||
}
|
||||
if _, ok := fake.Objects[remoteKey]; !ok {
|
||||
t.Fatalf("remote session key %q was not seeded", remoteKey)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRemoteSessionFallbackLoadsSecretsBeforeObjectStoreInit(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
accessKeyEnv := "NARRATIO_TEST_REMOTE_SESSION_KEY_ID"
|
||||
secretKeyEnv := "NARRATIO_TEST_REMOTE_SESSION_SECRET"
|
||||
restoreEnvAfterTest(t, accessKeyEnv, secretKeyEnv)
|
||||
secretsDir := t.TempDir()
|
||||
mustWriteTestFile(t, filepath.Join(secretsDir, accessKeyEnv), "remote-session-key-id\n")
|
||||
mustWriteTestFile(t, filepath.Join(secretsDir, secretKeyEnv), "remote-session-secret\n")
|
||||
addSecretsToPipelineConfig(t, pipelinePath, secretsDir, accessKeyEnv, secretKeyEnv)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
seedRemoteSessionConfig(t, fake, "2026-05-03", `session_id: 2026-05-03
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
`)
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
origSessionDefaults := append([]string(nil), config.DefaultSessionConfigSearchPaths...)
|
||||
config.DefaultSessionConfigSearchPaths = []string{filepath.Join(t.TempDir(), "session.yml")}
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
if os.Getenv(accessKeyEnv) != "remote-session-key-id" || os.Getenv(secretKeyEnv) != "remote-session-secret" {
|
||||
return nil, fmt.Errorf("secrets were not loaded before remote session object store init")
|
||||
}
|
||||
return fake, nil
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
config.DefaultSessionConfigSearchPaths = origSessionDefaults
|
||||
})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stdout=%q stderr=%q", code, stdout.String(), stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteExplicitLocalSessionPrecedenceSkipsRemote(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if storeInitCalls != 0 {
|
||||
t.Fatalf("object store init calls = %d, want 0", storeInitCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteLocalSessionDiscoveryPrecedenceSkipsRemote(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if storeInitCalls != 0 {
|
||||
t.Fatalf("object store init calls = %d, want 0", storeInitCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRemoteSessionMissingObjectFailsClearly(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
missingSessionPath := filepath.Join(t.TempDir(), "session.yml")
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{missingSessionPath})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "remote session") || !strings.Contains(stderr.String(), "session.yml") || !strings.Contains(stderr.String(), "not found") {
|
||||
t.Fatalf("stderr = %q, want remote session not found context", stderr.String())
|
||||
}
|
||||
if !strings.Contains(stderr.String(), missingSessionPath) {
|
||||
t.Fatalf("stderr = %q, want local searched path", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRemoteSessionRequiresSessionID(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, &storage.FakeBackend{}, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "plan", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "plan: session_id is required") {
|
||||
t.Fatalf("stderr = %q, want session_id guidance", stderr.String())
|
||||
}
|
||||
if storeInitCalls != 0 {
|
||||
t.Fatalf("object store init calls = %d, want 0", storeInitCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRemoteSessionStorageInitErrorFailsClearly(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
origSessionDefaults := append([]string(nil), config.DefaultSessionConfigSearchPaths...)
|
||||
config.DefaultSessionConfigSearchPaths = []string{filepath.Join(t.TempDir(), "session.yml")}
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
return nil, errors.New("storage unavailable")
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
config.DefaultSessionConfigSearchPaths = origSessionDefaults
|
||||
})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "storage unavailable") || !strings.Contains(stderr.String(), "remote session") {
|
||||
t.Fatalf("stderr = %q, want remote storage context", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRemoteSessionMalformedYAMLFailsStrictDecode(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
seedRemoteSessionConfig(t, fake, "2026-05-03", "session_id: 2026-05-03\nunknown: true\n")
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "strict decode failed") {
|
||||
t.Fatalf("stderr = %q, want strict decode context", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRemoteSessionTemplateFailsConcreteSessionCheck(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
seedRemoteSessionConfig(t, fake, "2026-05-03", `session_id: "{{ session_id }}"
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
`)
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "session.yml must be concrete") || !strings.Contains(stderr.String(), "run narratio session init") {
|
||||
t.Fatalf("stderr = %q, want concrete session guidance", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRemoteSessionMismatchFails(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
seedRemoteSessionConfig(t, fake, "2026-05-03", "session_id: 2026-05-04\ninputs:\n audio_s3:\n prefix: audio/\n")
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "session_id mismatch") {
|
||||
t.Fatalf("stderr = %q, want session_id mismatch", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func restoreAppConfigTestGlobals(t *testing.T, fake *storage.FakeBackend, storeInitCalls *int, sessionDefaults []string) {
|
||||
t.Helper()
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
origSessionDefaults := append([]string(nil), config.DefaultSessionConfigSearchPaths...)
|
||||
config.DefaultSessionConfigSearchPaths = append([]string(nil), sessionDefaults...)
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
if storeInitCalls != nil {
|
||||
(*storeInitCalls)++
|
||||
}
|
||||
return fake, nil
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
config.DefaultSessionConfigSearchPaths = origSessionDefaults
|
||||
})
|
||||
}
|
||||
|
||||
func seedRemoteSessionConfig(t *testing.T, fake *storage.FakeBackend, sessionID, content string) string {
|
||||
t.Helper()
|
||||
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", sessionID)
|
||||
remoteKey := artifacts.S3SessionConfigKey(sessionPrefix)
|
||||
fake.SeedObject(storage.FakeObject{
|
||||
Key: remoteKey,
|
||||
Data: []byte(content),
|
||||
ETag: "remote-session-etag",
|
||||
})
|
||||
return remoteKey
|
||||
}
|
||||
|
||||
func addSecretsToPipelineConfig(t *testing.T, pipelinePath, secretsDir, accessKeyEnv, secretKeyEnv string) {
|
||||
t.Helper()
|
||||
pipelineData, err := os.ReadFile(pipelinePath)
|
||||
if err != nil {
|
||||
t.Fatalf("read pipeline: %v", err)
|
||||
}
|
||||
pipelineYAML := strings.Replace(
|
||||
string(pipelineData),
|
||||
"storage:\n backend: s3\n s3:\n bucket: test-bucket\n",
|
||||
"storage:\n backend: s3\n s3:\n bucket: test-bucket\n access_key_id_env: "+accessKeyEnv+"\n secret_access_key_env: "+secretKeyEnv+"\nsecrets:\n env_dir: "+secretsDir+"\n",
|
||||
1,
|
||||
)
|
||||
if err := os.WriteFile(pipelinePath, []byte(pipelineYAML), 0o644); err != nil {
|
||||
t.Fatalf("write pipeline: %v", err)
|
||||
}
|
||||
}
|
||||
159
internal/app/restore.go
Normal file
159
internal/app/restore.go
Normal file
@@ -0,0 +1,159 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/logging"
|
||||
)
|
||||
|
||||
var newObjectStoreFromConfigFn = storage.NewObjectStoreFromConfig
|
||||
var discoverRemoteCurrentStateFn = discoverRemoteCurrentState
|
||||
var buildRestorePlanFn = buildRestorePlan
|
||||
var executeRestorePlanFn = executeRestorePlan
|
||||
|
||||
// Restore validates restore CLI/config inputs and storage preflight for future restore phases.
|
||||
func Restore(ctx context.Context, args []string, out io.Writer) error {
|
||||
positionalSessionID, args := pullLeadingSessionID(args)
|
||||
fs := flag.NewFlagSet("restore", flag.ContinueOnError)
|
||||
fs.SetOutput(out)
|
||||
|
||||
var pipelinePath string
|
||||
var campaignPath string
|
||||
var campaignFilePath string
|
||||
var sessionPath string
|
||||
var sessionID string
|
||||
var previousSessionID string
|
||||
var dryRun bool
|
||||
var force bool
|
||||
var includeAudio bool
|
||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||
fs.BoolVar(&dryRun, "dry-run", false, "plan restore actions without writing local files")
|
||||
fs.BoolVar(&force, "force", false, "overwrite local conflicts with remote state")
|
||||
fs.BoolVar(&includeAudio, "include-audio", false, "include archived session-level audio objects")
|
||||
fs.Usage = func() {
|
||||
_, _ = fmt.Fprintln(out, "Usage: narratio session restore <session_id> [--config <path>] [--campaign <id>] [--campaign-file <path>] [--session <path>] [--previous-session-id <value>] [--dry-run] [--force] [--include-audio]")
|
||||
_, _ = fmt.Fprintln(out)
|
||||
_, _ = fmt.Fprintln(out, "Flags:")
|
||||
fs.PrintDefaults()
|
||||
}
|
||||
|
||||
if err := fs.Parse(args); err != nil {
|
||||
if errors.Is(err, flag.ErrHelp) {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("restore: invalid flags: %w", err)
|
||||
}
|
||||
if positionalSessionID == "" {
|
||||
if err := applyParsedSessionIDArg("restore", fs, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
if fs.NArg() != 0 {
|
||||
return fmt.Errorf("restore: unexpected positional arguments")
|
||||
}
|
||||
if err := applyPositionalSessionID("restore", positionalSessionID, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if strings.TrimSpace(sessionID) == "" {
|
||||
return fmt.Errorf("restore: session_id is required")
|
||||
}
|
||||
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
||||
SessionID: sessionID,
|
||||
PreviousSessionID: previousSessionID,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("restore: %w", err)
|
||||
}
|
||||
if err := config.Validate(cfg); err != nil {
|
||||
return fmt.Errorf("restore: %w", err)
|
||||
}
|
||||
objectStore, err := newCommandObjectStore(ctx, cfg, logging.NewLogger(os.Stderr, slog.LevelInfo))
|
||||
if err != nil {
|
||||
return fmt.Errorf("restore: %w", err)
|
||||
}
|
||||
current, err := discoverRemoteCurrentStateFn(ctx, cfg, objectStore)
|
||||
if err != nil {
|
||||
return fmt.Errorf("restore: %w", err)
|
||||
}
|
||||
plan, err := buildRestorePlanFn(ctx, cfg, current, objectStore, RestorePlanOptions{
|
||||
IncludeAudio: includeAudio,
|
||||
Force: force,
|
||||
DryRun: dryRun,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("restore: %w", err)
|
||||
}
|
||||
report, err := newRestoreReport(current, plan, RestorePlanOptions{
|
||||
IncludeAudio: includeAudio,
|
||||
Force: force,
|
||||
DryRun: dryRun,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("restore: %w", err)
|
||||
}
|
||||
if dryRun {
|
||||
if err := writeRestoreDryRunSummary(out, report); err != nil {
|
||||
return fmt.Errorf("restore: write plan output: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
artifactStore := artifacts.NewLocalStore(cfg.Pipeline.Workspace.Root)
|
||||
if _, err := artifactStore.EnsureLayoutFor(cfg.Session.Campaign, cfg.Session.SessionID); err != nil {
|
||||
return fmt.Errorf("restore: prepare workdir: %w", err)
|
||||
}
|
||||
lock, err := artifactStore.AcquireSessionLockFor(cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("restore: acquire session lock: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
_ = artifactStore.ReleaseSessionLock(lock)
|
||||
}()
|
||||
|
||||
if plan.ConflictCount > 0 && !force {
|
||||
report.setFailed(fmt.Errorf("conflict: %d conflicting path(s)", plan.ConflictCount))
|
||||
if _, reportErr := persistRestoreReport(artifactStore, cfg, report); reportErr != nil {
|
||||
return fmt.Errorf("restore: report failure: %w", reportErr)
|
||||
}
|
||||
return fmt.Errorf(
|
||||
"restore conflict: %d conflicting path(s); rerun with --force to overwrite (download=%d skip_same=%d conflicts=%d)",
|
||||
plan.ConflictCount,
|
||||
plan.DownloadCount,
|
||||
plan.SkipSameCount,
|
||||
plan.ConflictCount,
|
||||
)
|
||||
}
|
||||
|
||||
result, err := executeRestorePlanFn(ctx, cfg, current, plan, report, objectStore)
|
||||
if err != nil {
|
||||
report.setFailed(err)
|
||||
if _, reportErr := persistRestoreReport(artifactStore, cfg, report); reportErr != nil {
|
||||
return fmt.Errorf("restore: execute plan failed (%v) and report write failed (%v)", err, reportErr)
|
||||
}
|
||||
return fmt.Errorf("restore: execute plan: %w", err)
|
||||
}
|
||||
report.Execution.Downloaded = result.DownloadedCount
|
||||
report.setSucceeded()
|
||||
if _, err := persistRestoreReport(artifactStore, cfg, report); err != nil {
|
||||
return fmt.Errorf("restore: write report: %w", err)
|
||||
}
|
||||
if err := writeRestoreSuccessSummary(out, report); err != nil {
|
||||
return fmt.Errorf("restore: write summary: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
139
internal/app/restore_discovery.go
Normal file
139
internal/app/restore_discovery.go
Normal file
@@ -0,0 +1,139 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
)
|
||||
|
||||
// RemoteCurrentState captures discovered committed remote archive state for one session.
|
||||
type RemoteCurrentState struct {
|
||||
Bucket string
|
||||
SessionPrefix string
|
||||
CurrentRunIDKey string
|
||||
CurrentManifestKey string
|
||||
RunID string
|
||||
SessionID string
|
||||
Campaign string
|
||||
Manifest *manifest.Manifest
|
||||
}
|
||||
|
||||
func discoverRemoteCurrentState(ctx context.Context, cfg *config.Config, store storage.ObjectStore) (*RemoteCurrentState, error) {
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Session == nil {
|
||||
return nil, fmt.Errorf("resolved config with pipeline/session is required")
|
||||
}
|
||||
if store == nil {
|
||||
return nil, fmt.Errorf("remote object store is required")
|
||||
}
|
||||
|
||||
bucket := artifacts.ResolveArchiveBucket(cfg, nil)
|
||||
if strings.TrimSpace(bucket) == "" {
|
||||
return nil, fmt.Errorf("archive bucket is required")
|
||||
}
|
||||
sessionPrefix, err := artifacts.ResolveArchiveSessionPrefix(cfg, nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("resolve archive session prefix: %w", err)
|
||||
}
|
||||
currentManifestKey, currentRunIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
||||
|
||||
exists, err := store.Exists(ctx, currentRunIDKey)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("check remote current run pointer %q: %w", currentRunIDKey, err)
|
||||
}
|
||||
if !exists {
|
||||
return nil, fmt.Errorf("remote current run pointer missing: %q", currentRunIDKey)
|
||||
}
|
||||
|
||||
runIDPath, err := downloadObjectToTemp(ctx, store, currentRunIDKey, "narratio-restore-current-run-id-*.txt")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("download remote current run pointer %q: %w", currentRunIDKey, err)
|
||||
}
|
||||
defer func() { _ = os.Remove(runIDPath) }()
|
||||
|
||||
runIDData, err := os.ReadFile(runIDPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read downloaded run pointer %q: %w", currentRunIDKey, err)
|
||||
}
|
||||
runID := strings.TrimSpace(string(runIDData))
|
||||
if runID == "" {
|
||||
return nil, fmt.Errorf("remote current run pointer %q is empty", currentRunIDKey)
|
||||
}
|
||||
|
||||
exists, err = store.Exists(ctx, currentManifestKey)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("check remote current manifest %q: %w", currentManifestKey, err)
|
||||
}
|
||||
if !exists {
|
||||
return nil, fmt.Errorf("remote current manifest missing: %q", currentManifestKey)
|
||||
}
|
||||
|
||||
manifestPath, err := downloadObjectToTemp(ctx, store, currentManifestKey, "narratio-restore-current-manifest-*.json")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("download remote current manifest %q: %w", currentManifestKey, err)
|
||||
}
|
||||
defer func() { _ = os.Remove(manifestPath) }()
|
||||
|
||||
manifestStore := &manifest.LocalStore{}
|
||||
remoteManifest, err := manifestStore.Load(ctx, manifestPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("remote current manifest decode failed: %w", err)
|
||||
}
|
||||
|
||||
requestedSession := strings.TrimSpace(cfg.Session.SessionID)
|
||||
requestedCampaign := strings.TrimSpace(cfg.Session.Campaign)
|
||||
manifestSession := strings.TrimSpace(remoteManifest.SessionID)
|
||||
manifestCampaign := strings.TrimSpace(remoteManifest.Campaign)
|
||||
|
||||
if manifestSession != requestedSession {
|
||||
return nil, fmt.Errorf(
|
||||
"remote current manifest session_id %q does not match requested session_id %q",
|
||||
manifestSession,
|
||||
requestedSession,
|
||||
)
|
||||
}
|
||||
if manifestCampaign == "" {
|
||||
return nil, fmt.Errorf("remote current manifest campaign is required")
|
||||
}
|
||||
if manifestCampaign != requestedCampaign {
|
||||
return nil, fmt.Errorf(
|
||||
"remote current manifest campaign %q does not match requested campaign %q",
|
||||
manifestCampaign,
|
||||
requestedCampaign,
|
||||
)
|
||||
}
|
||||
|
||||
return &RemoteCurrentState{
|
||||
Bucket: bucket,
|
||||
SessionPrefix: sessionPrefix,
|
||||
CurrentRunIDKey: currentRunIDKey,
|
||||
CurrentManifestKey: currentManifestKey,
|
||||
RunID: runID,
|
||||
SessionID: manifestSession,
|
||||
Campaign: manifestCampaign,
|
||||
Manifest: remoteManifest,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func downloadObjectToTemp(ctx context.Context, store storage.ObjectStore, key, pattern string) (string, error) {
|
||||
tmp, err := os.CreateTemp("", pattern)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create temp file: %w", err)
|
||||
}
|
||||
path := tmp.Name()
|
||||
if err := tmp.Close(); err != nil {
|
||||
_ = os.Remove(path)
|
||||
return "", fmt.Errorf("close temp file: %w", err)
|
||||
}
|
||||
|
||||
if err := store.Download(ctx, key, path); err != nil {
|
||||
_ = os.Remove(path)
|
||||
return "", err
|
||||
}
|
||||
return path, nil
|
||||
}
|
||||
239
internal/app/restore_discovery_test.go
Normal file
239
internal/app/restore_discovery_test.go
Normal file
@@ -0,0 +1,239 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
func TestDiscoverRemoteCurrentStateSuccess(t *testing.T) {
|
||||
cfg := restoreDiscoveryConfig()
|
||||
store := &storage.FakeBackend{}
|
||||
sessionPrefix, manifestKey, runIDKey := restoreDiscoveryKeys(cfg)
|
||||
|
||||
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign)})
|
||||
|
||||
state, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
||||
if err != nil {
|
||||
t.Fatalf("discoverRemoteCurrentState() error = %v", err)
|
||||
}
|
||||
if state.RunID != "20260519T010203Z-a1b2c3d4" {
|
||||
t.Fatalf("run id = %q, want 20260519T010203Z-a1b2c3d4", state.RunID)
|
||||
}
|
||||
if state.SessionPrefix != sessionPrefix {
|
||||
t.Fatalf("session prefix = %q, want %q", state.SessionPrefix, sessionPrefix)
|
||||
}
|
||||
if state.CurrentRunIDKey != runIDKey {
|
||||
t.Fatalf("current run id key = %q, want %q", state.CurrentRunIDKey, runIDKey)
|
||||
}
|
||||
if state.CurrentManifestKey != manifestKey {
|
||||
t.Fatalf("current manifest key = %q, want %q", state.CurrentManifestKey, manifestKey)
|
||||
}
|
||||
if state.Manifest == nil {
|
||||
t.Fatal("manifest is nil")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiscoverRemoteCurrentStateMissingRunPointerFails(t *testing.T) {
|
||||
cfg := restoreDiscoveryConfig()
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
||||
if err == nil || !strings.Contains(err.Error(), "remote current run pointer missing") {
|
||||
t.Fatalf("error = %v, want missing run pointer failure", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiscoverRemoteCurrentStateEmptyRunPointerFails(t *testing.T) {
|
||||
cfg := restoreDiscoveryConfig()
|
||||
store := &storage.FakeBackend{}
|
||||
_, manifestKey, runIDKey := restoreDiscoveryKeys(cfg)
|
||||
|
||||
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte(" \n\t")})
|
||||
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign)})
|
||||
|
||||
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
||||
if err == nil || !strings.Contains(err.Error(), "is empty") {
|
||||
t.Fatalf("error = %v, want empty run pointer failure", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiscoverRemoteCurrentStateMissingManifestFails(t *testing.T) {
|
||||
cfg := restoreDiscoveryConfig()
|
||||
store := &storage.FakeBackend{}
|
||||
_, _, runIDKey := restoreDiscoveryKeys(cfg)
|
||||
|
||||
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||
|
||||
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
||||
if err == nil || !strings.Contains(err.Error(), "remote current manifest missing") {
|
||||
t.Fatalf("error = %v, want missing manifest failure", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiscoverRemoteCurrentStateInvalidManifestFails(t *testing.T) {
|
||||
cfg := restoreDiscoveryConfig()
|
||||
store := &storage.FakeBackend{}
|
||||
_, manifestKey, runIDKey := restoreDiscoveryKeys(cfg)
|
||||
|
||||
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: []byte("{invalid json")})
|
||||
|
||||
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
||||
if err == nil || !strings.Contains(err.Error(), "remote current manifest decode failed") {
|
||||
t.Fatalf("error = %v, want manifest decode failure", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiscoverRemoteCurrentStateSessionMismatchFails(t *testing.T) {
|
||||
cfg := restoreDiscoveryConfig()
|
||||
store := &storage.FakeBackend{}
|
||||
_, manifestKey, runIDKey := restoreDiscoveryKeys(cfg)
|
||||
|
||||
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, "wrong-session", cfg.Session.Campaign)})
|
||||
|
||||
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
||||
if err == nil || !strings.Contains(err.Error(), "does not match requested session_id") {
|
||||
t.Fatalf("error = %v, want session mismatch failure", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiscoverRemoteCurrentStateCampaignMismatchFails(t *testing.T) {
|
||||
cfg := restoreDiscoveryConfig()
|
||||
store := &storage.FakeBackend{}
|
||||
_, manifestKey, runIDKey := restoreDiscoveryKeys(cfg)
|
||||
|
||||
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, cfg.Session.SessionID, "wrong-campaign")})
|
||||
|
||||
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
||||
if err == nil || !strings.Contains(err.Error(), "does not match requested campaign") {
|
||||
t.Fatalf("error = %v, want campaign mismatch failure", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiscoverRemoteCurrentStateEmptyCampaignFails(t *testing.T) {
|
||||
cfg := restoreDiscoveryConfig()
|
||||
store := &storage.FakeBackend{}
|
||||
_, manifestKey, runIDKey := restoreDiscoveryKeys(cfg)
|
||||
|
||||
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, cfg.Session.SessionID, "")})
|
||||
|
||||
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
||||
if err == nil || !strings.Contains(err.Error(), "campaign is required") {
|
||||
t.Fatalf("error = %v, want empty campaign failure", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiscoverRemoteCurrentStateUsesCurrentKeysUnderSessionPrefix(t *testing.T) {
|
||||
cfg := restoreDiscoveryConfig()
|
||||
sessionPrefix, manifestKey, runIDKey := restoreDiscoveryKeys(cfg)
|
||||
base := &storage.FakeBackend{}
|
||||
store := &captureObjectStore{delegate: base}
|
||||
|
||||
base.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||
base.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign)})
|
||||
|
||||
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
||||
if err != nil {
|
||||
t.Fatalf("discoverRemoteCurrentState() error = %v", err)
|
||||
}
|
||||
|
||||
expectedRunKey := fmt.Sprintf("%scurrent/run_id.txt", sessionPrefix)
|
||||
expectedManifestKey := fmt.Sprintf("%scurrent/manifest.json", sessionPrefix)
|
||||
if !containsString(store.existsKeys, expectedRunKey) {
|
||||
t.Fatalf("exists keys = %#v, want run pointer key %q", store.existsKeys, expectedRunKey)
|
||||
}
|
||||
if !containsString(store.existsKeys, expectedManifestKey) {
|
||||
t.Fatalf("exists keys = %#v, want manifest key %q", store.existsKeys, expectedManifestKey)
|
||||
}
|
||||
if !containsString(store.downloadKeys, expectedRunKey) {
|
||||
t.Fatalf("download keys = %#v, want run pointer key %q", store.downloadKeys, expectedRunKey)
|
||||
}
|
||||
if !containsString(store.downloadKeys, expectedManifestKey) {
|
||||
t.Fatalf("download keys = %#v, want manifest key %q", store.downloadKeys, expectedManifestKey)
|
||||
}
|
||||
}
|
||||
|
||||
type captureObjectStore struct {
|
||||
delegate storage.ObjectStore
|
||||
existsKeys []string
|
||||
downloadKeys []string
|
||||
}
|
||||
|
||||
func (s *captureObjectStore) List(ctx context.Context, prefix string) ([]storage.ObjectInfo, error) {
|
||||
return s.delegate.List(ctx, prefix)
|
||||
}
|
||||
|
||||
func (s *captureObjectStore) Download(ctx context.Context, key, localPath string) error {
|
||||
s.downloadKeys = append(s.downloadKeys, key)
|
||||
return s.delegate.Download(ctx, key, localPath)
|
||||
}
|
||||
|
||||
func (s *captureObjectStore) Upload(ctx context.Context, localPath, key string, opts storage.UploadOptions) (storage.ObjectInfo, error) {
|
||||
return s.delegate.Upload(ctx, localPath, key, opts)
|
||||
}
|
||||
|
||||
func (s *captureObjectStore) Exists(ctx context.Context, key string) (bool, error) {
|
||||
s.existsKeys = append(s.existsKeys, key)
|
||||
return s.delegate.Exists(ctx, key)
|
||||
}
|
||||
|
||||
func restoreDiscoveryConfig() *config.Config {
|
||||
return &config.Config{
|
||||
Pipeline: &config.PipelineConfig{
|
||||
Storage: config.StorageConfig{
|
||||
S3: &config.StorageS3Config{
|
||||
Bucket: "my-dnd-archive",
|
||||
RootPrefix: "dnd",
|
||||
},
|
||||
},
|
||||
},
|
||||
Session: &config.SessionConfig{
|
||||
SessionID: "2026-05-03",
|
||||
Campaign: "sample-campaign",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func restoreDiscoveryKeys(cfg *config.Config) (sessionPrefix, manifestKey, runIDKey string) {
|
||||
sessionPrefix = artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
manifestKey, runIDKey = artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
||||
return sessionPrefix, manifestKey, runIDKey
|
||||
}
|
||||
|
||||
func restoreManifestJSON(t *testing.T, sessionID, campaign string) []byte {
|
||||
t.Helper()
|
||||
now := time.Date(2026, 5, 19, 23, 0, 0, 0, time.UTC).Format(time.RFC3339Nano)
|
||||
payload := map[string]any{
|
||||
"session_id": sessionID,
|
||||
"campaign": campaign,
|
||||
"created_at": now,
|
||||
"updated_at": now,
|
||||
"stages": map[string]any{},
|
||||
}
|
||||
data, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal manifest payload: %v", err)
|
||||
}
|
||||
return append(data, '\n')
|
||||
}
|
||||
|
||||
func containsString(values []string, target string) bool {
|
||||
for _, value := range values {
|
||||
if value == target {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
217
internal/app/restore_execute.go
Normal file
217
internal/app/restore_execute.go
Normal file
@@ -0,0 +1,217 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/audio"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
)
|
||||
|
||||
// RestoreExecutionResult captures concrete file-install results for one restore execution.
|
||||
type RestoreExecutionResult struct {
|
||||
DownloadedCount int
|
||||
}
|
||||
|
||||
func executeRestorePlan(
|
||||
ctx context.Context,
|
||||
cfg *config.Config,
|
||||
current *RemoteCurrentState,
|
||||
plan *RestorePlan,
|
||||
report *RestoreReport,
|
||||
store storage.ObjectStore,
|
||||
) (*RestoreExecutionResult, error) {
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Session == nil {
|
||||
return nil, fmt.Errorf("resolved config with pipeline/session is required")
|
||||
}
|
||||
if current == nil {
|
||||
return nil, fmt.Errorf("remote current state is required")
|
||||
}
|
||||
if plan == nil {
|
||||
return nil, fmt.Errorf("restore plan is required")
|
||||
}
|
||||
if store == nil {
|
||||
return nil, fmt.Errorf("remote object store is required")
|
||||
}
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
manifestActions := make([]RestoreAction, 0, 1)
|
||||
actions := make([]RestoreAction, 0, len(plan.Actions))
|
||||
for _, action := range plan.Actions {
|
||||
if action.Kind != RestoreActionDownload {
|
||||
continue
|
||||
}
|
||||
if action.LocalRelativePath == config.PathManifestFile {
|
||||
manifestActions = append(manifestActions, action)
|
||||
continue
|
||||
}
|
||||
actions = append(actions, action)
|
||||
}
|
||||
if len(manifestActions) > 1 {
|
||||
return nil, fmt.Errorf("restore plan includes multiple manifest download actions")
|
||||
}
|
||||
if len(manifestActions) == 1 {
|
||||
actions = append(actions, manifestActions[0])
|
||||
}
|
||||
|
||||
result := &RestoreExecutionResult{}
|
||||
for _, action := range actions {
|
||||
if err := executeRestoreDownloadAction(ctx, cfg, sessionRoot, current, action, store); err != nil {
|
||||
if report != nil {
|
||||
report.markFailed(action, err)
|
||||
}
|
||||
return nil, fmt.Errorf("install %q from %q: %w", action.LocalRelativePath, action.RemoteKey, err)
|
||||
}
|
||||
if report != nil {
|
||||
report.markDownloaded(action)
|
||||
}
|
||||
result.DownloadedCount++
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func executeRestoreDownloadAction(
|
||||
ctx context.Context,
|
||||
cfg *config.Config,
|
||||
sessionRoot string,
|
||||
current *RemoteCurrentState,
|
||||
action RestoreAction,
|
||||
store storage.ObjectStore,
|
||||
) error {
|
||||
safeLocalPath, err := joinWithinSessionRoot(sessionRoot, action.LocalRelativePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("resolve safe local path: %w", err)
|
||||
}
|
||||
if strings.TrimSpace(action.LocalPath) != "" && filepath.Clean(action.LocalPath) != safeLocalPath {
|
||||
return fmt.Errorf("restore plan local path mismatch for %q", action.LocalRelativePath)
|
||||
}
|
||||
|
||||
if restoreActionIsAudio(action) {
|
||||
return executeRestoreAudioAction(ctx, cfg, safeLocalPath, action, store)
|
||||
}
|
||||
|
||||
tmpPath, err := downloadObjectToSiblingTemp(ctx, store, action.RemoteKey, safeLocalPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("download to temp file: %w", err)
|
||||
}
|
||||
removeTmp := true
|
||||
defer func() {
|
||||
if removeTmp {
|
||||
_ = os.Remove(tmpPath)
|
||||
}
|
||||
}()
|
||||
|
||||
if action.LocalRelativePath == config.PathManifestFile {
|
||||
if err := validateRestoredManifest(ctx, cfg, current, tmpPath); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if err := os.Chmod(tmpPath, 0o644); err != nil {
|
||||
return fmt.Errorf("set file permissions: %w", err)
|
||||
}
|
||||
if err := os.Rename(tmpPath, safeLocalPath); err != nil {
|
||||
return fmt.Errorf("install file atomically: %w", err)
|
||||
}
|
||||
removeTmp = false
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func executeRestoreAudioAction(
|
||||
ctx context.Context,
|
||||
cfg *config.Config,
|
||||
safeLocalPath string,
|
||||
action RestoreAction,
|
||||
store storage.ObjectStore,
|
||||
) error {
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Storage.S3 == nil || cfg.Session == nil {
|
||||
return fmt.Errorf("resolved s3 config and session are required")
|
||||
}
|
||||
spoolDir := artifacts.SessionSpoolRestoreAudioDir(cfg.Pipeline.Spool.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
spoolPath := filepath.Join(spoolDir, filepath.Base(safeLocalPath))
|
||||
cacheEnabled := cfg.Pipeline.Cache.S3Audio == nil || *cfg.Pipeline.Cache.S3Audio
|
||||
_, err := audio.MaterializeS3Audio(ctx, audio.S3MaterializeRequest{
|
||||
Store: store,
|
||||
Object: storage.ObjectInfo{
|
||||
Key: action.RemoteKey,
|
||||
Size: action.Size,
|
||||
ETag: action.ETag,
|
||||
},
|
||||
Bucket: strings.TrimSpace(cfg.Pipeline.Storage.S3.Bucket),
|
||||
CacheRoot: strings.TrimSpace(cfg.Pipeline.Cache.Root),
|
||||
CacheEnabled: cacheEnabled,
|
||||
SpoolPath: spoolPath,
|
||||
DestPath: safeLocalPath,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("materialize audio: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func downloadObjectToSiblingTemp(ctx context.Context, store storage.ObjectStore, remoteKey, destPath string) (string, error) {
|
||||
if strings.TrimSpace(destPath) == "" {
|
||||
return "", fmt.Errorf("destination path is required")
|
||||
}
|
||||
dir := filepath.Dir(destPath)
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
return "", fmt.Errorf("create destination directory: %w", err)
|
||||
}
|
||||
base := filepath.Base(destPath)
|
||||
tmp, err := os.CreateTemp(dir, "."+base+".restore-*.tmp")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create temp file: %w", err)
|
||||
}
|
||||
tmpPath := tmp.Name()
|
||||
if err := tmp.Close(); err != nil {
|
||||
_ = os.Remove(tmpPath)
|
||||
return "", fmt.Errorf("close temp file: %w", err)
|
||||
}
|
||||
|
||||
if err := store.Download(ctx, remoteKey, tmpPath); err != nil {
|
||||
_ = os.Remove(tmpPath)
|
||||
return "", err
|
||||
}
|
||||
|
||||
return tmpPath, nil
|
||||
}
|
||||
|
||||
func validateRestoredManifest(ctx context.Context, cfg *config.Config, current *RemoteCurrentState, path string) error {
|
||||
manifestStore := &manifest.LocalStore{}
|
||||
m, err := manifestStore.Load(ctx, path)
|
||||
if err != nil {
|
||||
return fmt.Errorf("validate manifest decode: %w", err)
|
||||
}
|
||||
|
||||
requestedSession := strings.TrimSpace(cfg.Session.SessionID)
|
||||
requestedCampaign := strings.TrimSpace(cfg.Session.Campaign)
|
||||
manifestSession := strings.TrimSpace(m.SessionID)
|
||||
manifestCampaign := strings.TrimSpace(m.Campaign)
|
||||
if manifestSession != requestedSession {
|
||||
return fmt.Errorf("manifest session_id %q does not match requested session_id %q", manifestSession, requestedSession)
|
||||
}
|
||||
if manifestCampaign == "" {
|
||||
return fmt.Errorf("manifest campaign is required")
|
||||
}
|
||||
if manifestCampaign != requestedCampaign {
|
||||
return fmt.Errorf("manifest campaign %q does not match requested campaign %q", manifestCampaign, requestedCampaign)
|
||||
}
|
||||
if current != nil {
|
||||
if expected := strings.TrimSpace(current.SessionID); expected != "" && manifestSession != expected {
|
||||
return fmt.Errorf("manifest session_id %q does not match discovered session_id %q", manifestSession, expected)
|
||||
}
|
||||
if expected := strings.TrimSpace(current.Campaign); expected != "" && manifestCampaign != expected {
|
||||
return fmt.Errorf("manifest campaign %q does not match discovered campaign %q", manifestCampaign, expected)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
539
internal/app/restore_execution_test.go
Normal file
539
internal/app/restore_execution_test.go
Normal file
@@ -0,0 +1,539 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
)
|
||||
|
||||
func TestExecuteRestoreNonDryRunRestoresDurableFiles(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
cfg, sessionPrefix, manifestKey, runIDKey := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte(`{"segments":[1,2,3]}`))
|
||||
seedRestoreObject(fake, sessionPrefix+"artifacts/session_recap.md", []byte("# recap\n"))
|
||||
seedRestoreObject(fake, sessionPrefix+"audio/alice.flac", []byte("remote-audio"))
|
||||
seedRestoreObject(fake, runIDKey, []byte("20260519T010203Z-a1b2c3d4\n"))
|
||||
seedRestoreObject(fake, manifestKey, restoreManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign))
|
||||
|
||||
restoreWithStoreAndRealPhases(t, fake)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if stderr.Len() != 0 {
|
||||
t.Fatalf("stderr = %q, want empty", stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "Restored session archive for sample-campaign/2026-05-03") {
|
||||
t.Fatalf("stdout = %q, want completion summary", stdout.String())
|
||||
}
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "transcripts", "full.json"), `{"segments":[1,2,3]}`)
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "artifacts", "session_recap.md"), "# recap\n")
|
||||
reportPath := filepath.Join(sessionRoot, "reports", "restore-latest.json")
|
||||
report := mustReadRestoreReport(t, reportPath)
|
||||
if report.Status != "succeeded" {
|
||||
t.Fatalf("report status = %q, want succeeded", report.Status)
|
||||
}
|
||||
if report.Execution.Downloaded != 3 {
|
||||
t.Fatalf("report execution.downloaded = %d, want 3", report.Execution.Downloaded)
|
||||
}
|
||||
if len(report.Actions) == 0 {
|
||||
t.Fatal("report actions is empty")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(sessionRoot, "audio", "alice.flac")); !os.IsNotExist(err) {
|
||||
t.Fatalf("audio should not be restored by default; stat err=%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreIncludeAudioRestoresAudio(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||
seedRestoreObject(fake, sessionPrefix+"audio/alice.flac", []byte("remote-audio"))
|
||||
|
||||
restoreWithStoreAndRealPhases(t, fake)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--include-audio"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "audio", "alice.flac"), "remote-audio")
|
||||
report := mustReadRestoreReport(t, filepath.Join(sessionRoot, "reports", "restore-latest.json"))
|
||||
if !report.IncludeAudio {
|
||||
t.Fatalf("report include_audio = %v, want true", report.IncludeAudio)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreIncludeAudioUsesCacheAfterWorkspaceDeletion(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||
audioKey := sessionPrefix + "audio/alice.flac"
|
||||
seedRestoreObject(fake, audioKey, []byte("remote-audio"))
|
||||
|
||||
restoreWithStoreAndRealPhases(t, fake)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--include-audio"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("first restore exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if got := fakeDownloadCount(fake, audioKey); got != 1 {
|
||||
t.Fatalf("audio downloads after first restore = %d, want 1", got)
|
||||
}
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "audio", "alice.flac"), "remote-audio")
|
||||
|
||||
if err := os.RemoveAll(sessionRoot); err != nil {
|
||||
t.Fatalf("remove session root: %v", err)
|
||||
}
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
code = Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--include-audio"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("second restore exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if got := fakeDownloadCount(fake, audioKey); got != 1 {
|
||||
t.Fatalf("audio downloads after cached restore = %d, want still 1", got)
|
||||
}
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "audio", "alice.flac"), "remote-audio")
|
||||
}
|
||||
|
||||
func TestExecuteRestoreRestoresPreviousCacheWhenPresent(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
appendRestoreWorkflowPreviousInputConfig(t, pipelinePath, sessionPath)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
cfg, _, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||
seedRestorePreviousCurrent(t, fake, cfg, "# previous recap\n")
|
||||
|
||||
restoreWithStoreAndRealPhases(t, fake)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
previousManifestBytes, err := os.ReadFile(filepath.Join(sessionRoot, "previous", "manifest.json"))
|
||||
if err != nil {
|
||||
t.Fatalf("read restored previous manifest: %v", err)
|
||||
}
|
||||
if !strings.Contains(string(previousManifestBytes), `"session_id":"2026-04-26"`) {
|
||||
t.Fatalf("restored previous manifest = %q, want previous session id", string(previousManifestBytes))
|
||||
}
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md"), "# previous recap\n")
|
||||
report := mustReadRestoreReport(t, filepath.Join(sessionRoot, "reports", "restore-latest.json"))
|
||||
if report.Execution.Downloaded != 3 {
|
||||
t.Fatalf("report execution.downloaded = %d, want 3", report.Execution.Downloaded)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreDryRunReportsPreviousCacheWithoutWriting(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
appendRestoreWorkflowPreviousInputConfig(t, pipelinePath, sessionPath)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
cfg, _, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||
seedRestorePreviousCurrent(t, fake, cfg, "# previous recap\n")
|
||||
|
||||
restoreWithStoreAndRealPhases(t, fake)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--dry-run"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "previous/artifacts/session_recap.md") {
|
||||
t.Fatalf("stdout = %q, want planned previous-cache artifact", stdout.String())
|
||||
}
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
if _, err := os.Stat(filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md")); !os.IsNotExist(err) {
|
||||
t.Fatalf("previous artifact should not be written during dry-run; stat err=%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreConflictWithoutForceDoesNotOverwrite(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustWriteTestFile(t, filepath.Join(sessionRoot, "transcripts", "full.json"), "local-transcript")
|
||||
|
||||
restoreWithStoreAndRealPhases(t, fake)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "conflicting path") {
|
||||
t.Fatalf("stderr = %q, want conflict failure", stderr.String())
|
||||
}
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "transcripts", "full.json"), "local-transcript")
|
||||
report := mustReadRestoreReport(t, filepath.Join(sessionRoot, "reports", "restore-latest.json"))
|
||||
if report.Status != "failed" {
|
||||
t.Fatalf("report status = %q, want failed", report.Status)
|
||||
}
|
||||
if report.Plan.Conflicts != 1 {
|
||||
t.Fatalf("report plan.conflicts = %d, want 1", report.Plan.Conflicts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreForceOverwritesDifferingFile(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustWriteTestFile(t, filepath.Join(sessionRoot, "transcripts", "full.json"), "local-transcript")
|
||||
|
||||
restoreWithStoreAndRealPhases(t, fake)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "transcripts", "full.json"), "remote-transcript")
|
||||
report := mustReadRestoreReport(t, filepath.Join(sessionRoot, "reports", "restore-latest.json"))
|
||||
if !report.Force {
|
||||
t.Fatalf("report force = %v, want true", report.Force)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreForceOverwritesDifferingPreviousCacheFile(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
appendRestoreWorkflowPreviousInputConfig(t, pipelinePath, sessionPath)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
cfg, _, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||
seedRestorePreviousCurrent(t, fake, cfg, "# remote previous recap\n")
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustWriteTestFile(t, filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md"), "# local previous recap\n")
|
||||
|
||||
restoreWithStoreAndRealPhases(t, fake)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md"), "# remote previous recap\n")
|
||||
}
|
||||
|
||||
func TestExecuteRestoreLockConflictFailsAndWritesNothing(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
||||
|
||||
store := artifacts.NewLocalStore(workspaceRoot)
|
||||
lock, err := store.AcquireSessionLockFor(cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
if err != nil {
|
||||
t.Fatalf("AcquireSessionLockFor() error = %v", err)
|
||||
}
|
||||
defer func() { _ = store.ReleaseSessionLock(lock) }()
|
||||
|
||||
restoreWithStoreAndRealPhases(t, fake)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "acquire session lock") {
|
||||
t.Fatalf("stderr = %q, want lock failure", stderr.String())
|
||||
}
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
if _, err := os.Stat(filepath.Join(sessionRoot, "transcripts", "full.json")); !os.IsNotExist(err) {
|
||||
t.Fatalf("transcript should not be restored when lock acquisition fails; stat err=%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreInvalidManifestDoesNotCorruptExistingManifest(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
base := &storage.FakeBackend{}
|
||||
cfg, sessionPrefix, manifestKey, _ := seedRestoreCommittedState(t, base, pipelinePath, campaignPath, sessionPath)
|
||||
seedRestoreObject(base, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
||||
|
||||
toggled := &stagedManifestDownloadStore{
|
||||
delegate: base,
|
||||
manifestKey: manifestKey,
|
||||
firstManifest: restoreManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign),
|
||||
secondManifest: []byte("{invalid json"),
|
||||
manifestReads: 0,
|
||||
}
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
existing := manifest.New(cfg.Session.SessionID, nowUTC())
|
||||
existing.Campaign = cfg.Session.Campaign
|
||||
existingPath := filepath.Join(sessionRoot, "manifest.json")
|
||||
manifestStore := &manifest.LocalStore{}
|
||||
if err := manifestStore.Save(context.Background(), existingPath, existing); err != nil {
|
||||
t.Fatalf("save existing local manifest: %v", err)
|
||||
}
|
||||
existingData, err := os.ReadFile(existingPath)
|
||||
if err != nil {
|
||||
t.Fatalf("read existing local manifest: %v", err)
|
||||
}
|
||||
|
||||
restoreWithStoreAndRealPhases(t, toggled)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "validate manifest decode") {
|
||||
t.Fatalf("stderr = %q, want manifest validation failure", stderr.String())
|
||||
}
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "transcripts", "full.json"), "remote-transcript")
|
||||
report := mustReadRestoreReport(t, filepath.Join(sessionRoot, "reports", "restore-latest.json"))
|
||||
if report.Status != "failed" {
|
||||
t.Fatalf("report status = %q, want failed", report.Status)
|
||||
}
|
||||
if strings.TrimSpace(report.Error) == "" {
|
||||
t.Fatal("report error is empty, want failure context")
|
||||
}
|
||||
afterData, err := os.ReadFile(existingPath)
|
||||
if err != nil {
|
||||
t.Fatalf("read local manifest after failure: %v", err)
|
||||
}
|
||||
if string(afterData) != string(existingData) {
|
||||
t.Fatalf("local manifest changed after failed restore; before=%q after=%q", string(existingData), string(afterData))
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestorePlanPathMismatchFails(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
seedRestoreObject(store, current.SessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
||||
|
||||
plan := &RestorePlan{Actions: []RestoreAction{{
|
||||
Kind: RestoreActionDownload,
|
||||
RemoteKey: current.SessionPrefix + "transcripts/full.json",
|
||||
LocalRelativePath: "transcripts/full.json",
|
||||
LocalPath: "/tmp/escape.txt",
|
||||
}}}
|
||||
|
||||
report, err := newRestoreReport(current, plan, RestorePlanOptions{})
|
||||
if err != nil {
|
||||
t.Fatalf("newRestoreReport() error = %v", err)
|
||||
}
|
||||
_, err = executeRestorePlan(context.Background(), cfg, current, plan, report, store)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "local path mismatch") {
|
||||
t.Fatalf("error = %v, want local path mismatch", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustReadRestoreReport(t *testing.T, path string) *RestoreReport {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadFile(%q): %v", path, err)
|
||||
}
|
||||
var report RestoreReport
|
||||
if err := json.Unmarshal(data, &report); err != nil {
|
||||
t.Fatalf("Unmarshal restore report %q: %v", path, err)
|
||||
}
|
||||
return &report
|
||||
}
|
||||
|
||||
func restoreWithStoreAndRealPhases(t *testing.T, objectStore storage.ObjectStore) {
|
||||
t.Helper()
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
origDiscoverFn := discoverRemoteCurrentStateFn
|
||||
origPlanFn := buildRestorePlanFn
|
||||
origExecuteFn := executeRestorePlanFn
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
discoverRemoteCurrentStateFn = origDiscoverFn
|
||||
buildRestorePlanFn = origPlanFn
|
||||
executeRestorePlanFn = origExecuteFn
|
||||
})
|
||||
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
return objectStore, nil
|
||||
}
|
||||
discoverRemoteCurrentStateFn = discoverRemoteCurrentState
|
||||
buildRestorePlanFn = buildRestorePlan
|
||||
executeRestorePlanFn = executeRestorePlan
|
||||
}
|
||||
|
||||
func seedRestoreCommittedState(t *testing.T, fake *storage.FakeBackend, pipelinePath, campaignPath, sessionPath string) (*config.Config, string, string, string) {
|
||||
t.Helper()
|
||||
cfg, err := config.LoadWithSessionOptions(pipelinePath, campaignPath, sessionPath, config.SessionLoadOptions{})
|
||||
if err != nil {
|
||||
t.Fatalf("LoadWithSessionOptions() error = %v", err)
|
||||
}
|
||||
if err := config.Validate(cfg); err != nil {
|
||||
t.Fatalf("Validate() error = %v", err)
|
||||
}
|
||||
|
||||
sessionPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
||||
|
||||
seedRestoreObject(fake, runIDKey, []byte("20260519T010203Z-a1b2c3d4\n"))
|
||||
seedRestoreObject(fake, manifestKey, restoreManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign))
|
||||
|
||||
return cfg, sessionPrefix, manifestKey, runIDKey
|
||||
}
|
||||
|
||||
func appendRestoreWorkflowPreviousInputConfig(t *testing.T, pipelinePath, sessionPath string) {
|
||||
t.Helper()
|
||||
appendRestoreWorkflowScriptoriumConfig(t, pipelinePath, `
|
||||
scriptorium:
|
||||
binary: scriptorium
|
||||
artifacts:
|
||||
session_recap:
|
||||
enabled: true
|
||||
prompt_id: dnd.session_recap
|
||||
output_path: artifacts/session_recap.md
|
||||
inputs:
|
||||
previous_recap:
|
||||
source: narratio.previous_session.artifact.session_recap
|
||||
required: true
|
||||
`)
|
||||
appendRestoreWorkflowScriptoriumConfig(t, sessionPath, `
|
||||
previous_session_id: 2026-04-26
|
||||
`)
|
||||
}
|
||||
|
||||
func seedRestorePreviousCurrent(t *testing.T, fake *storage.FakeBackend, cfg *config.Config, artifactBody string) {
|
||||
t.Helper()
|
||||
seedRestorePreviousCurrentManifestOnly(t, fake, cfg)
|
||||
previousPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.PreviousSessionID)
|
||||
seedRestoreObject(fake, previousPrefix+"artifacts/session_recap.md", []byte(artifactBody))
|
||||
}
|
||||
|
||||
func seedRestorePreviousCurrentManifestOnly(t *testing.T, fake *storage.FakeBackend, cfg *config.Config) {
|
||||
t.Helper()
|
||||
previousPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.PreviousSessionID)
|
||||
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(previousPrefix)
|
||||
previousRunID := "20260426T010203Z-a1b2c3d4"
|
||||
seedRestoreObject(fake, runIDKey, []byte(previousRunID+"\n"))
|
||||
|
||||
m := manifest.New(cfg.Session.PreviousSessionID, nowUTC())
|
||||
m.Campaign = cfg.Session.Campaign
|
||||
m.RunID = previousRunID
|
||||
data, err := json.Marshal(m)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal previous restore manifest: %v", err)
|
||||
}
|
||||
seedRestoreObject(fake, manifestKey, append(data, '\n'))
|
||||
}
|
||||
|
||||
func mustReadEquals(t *testing.T, path, want string) {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadFile(%q): %v", path, err)
|
||||
}
|
||||
if string(data) != want {
|
||||
t.Fatalf("file %q = %q, want %q", path, string(data), want)
|
||||
}
|
||||
}
|
||||
|
||||
func fakeDownloadCount(fake *storage.FakeBackend, key string) int {
|
||||
count := 0
|
||||
for _, call := range fake.Downloads {
|
||||
if call.Key == key {
|
||||
count++
|
||||
}
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
type stagedManifestDownloadStore struct {
|
||||
delegate *storage.FakeBackend
|
||||
manifestKey string
|
||||
firstManifest []byte
|
||||
secondManifest []byte
|
||||
manifestReads int
|
||||
}
|
||||
|
||||
func (s *stagedManifestDownloadStore) List(ctx context.Context, prefix string) ([]storage.ObjectInfo, error) {
|
||||
return s.delegate.List(ctx, prefix)
|
||||
}
|
||||
|
||||
func (s *stagedManifestDownloadStore) Download(ctx context.Context, key, localPath string) error {
|
||||
if strings.TrimSpace(key) == strings.TrimSpace(s.manifestKey) {
|
||||
s.manifestReads++
|
||||
payload := s.secondManifest
|
||||
if s.manifestReads <= 1 {
|
||||
payload = s.firstManifest
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(localPath), 0o755); err != nil {
|
||||
return fmt.Errorf("download staged manifest: create parent: %w", err)
|
||||
}
|
||||
if err := os.WriteFile(localPath, payload, 0o644); err != nil {
|
||||
return fmt.Errorf("download staged manifest: write local file: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return s.delegate.Download(ctx, key, localPath)
|
||||
}
|
||||
|
||||
func (s *stagedManifestDownloadStore) Upload(ctx context.Context, localPath, key string, opts storage.UploadOptions) (storage.ObjectInfo, error) {
|
||||
return s.delegate.Upload(ctx, localPath, key, opts)
|
||||
}
|
||||
|
||||
func (s *stagedManifestDownloadStore) Exists(ctx context.Context, key string) (bool, error) {
|
||||
return s.delegate.Exists(ctx, key)
|
||||
}
|
||||
421
internal/app/restore_plan.go
Normal file
421
internal/app/restore_plan.go
Normal file
@@ -0,0 +1,421 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/previouscache"
|
||||
)
|
||||
|
||||
// RestoreActionKind identifies one restore planner action.
|
||||
type RestoreActionKind string
|
||||
|
||||
const (
|
||||
RestoreActionDownload RestoreActionKind = "download"
|
||||
RestoreActionSkipSame RestoreActionKind = "skip_same"
|
||||
RestoreActionConflict RestoreActionKind = "conflict"
|
||||
)
|
||||
|
||||
// RestoreAction is one deterministic planner action.
|
||||
type RestoreAction struct {
|
||||
Kind RestoreActionKind
|
||||
RemoteKey string
|
||||
LocalRelativePath string
|
||||
LocalPath string
|
||||
Size int64
|
||||
ETag string
|
||||
ExistsLocal bool
|
||||
SameLocal bool
|
||||
Conflict bool
|
||||
Reason string
|
||||
}
|
||||
|
||||
// RestorePlan is the deterministic output of restore planning.
|
||||
type RestorePlan struct {
|
||||
Actions []RestoreAction
|
||||
DownloadCount int
|
||||
SkipSameCount int
|
||||
ConflictCount int
|
||||
}
|
||||
|
||||
// RestorePlanOptions control restore planning scope and classification.
|
||||
type RestorePlanOptions struct {
|
||||
IncludeAudio bool
|
||||
Force bool
|
||||
DryRun bool
|
||||
}
|
||||
|
||||
func buildRestorePlan(ctx context.Context, cfg *config.Config, current *RemoteCurrentState, store storage.ObjectStore, opts RestorePlanOptions) (*RestorePlan, error) {
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Session == nil {
|
||||
return nil, fmt.Errorf("resolved config with pipeline/session is required")
|
||||
}
|
||||
if current == nil {
|
||||
return nil, fmt.Errorf("remote current state is required")
|
||||
}
|
||||
if store == nil {
|
||||
return nil, fmt.Errorf("remote object store is required")
|
||||
}
|
||||
prefix := normalizeRemoteKey(current.SessionPrefix)
|
||||
if strings.TrimSpace(prefix) == "" {
|
||||
return nil, fmt.Errorf("remote session prefix is required")
|
||||
}
|
||||
if !strings.HasSuffix(prefix, "/") {
|
||||
prefix += "/"
|
||||
}
|
||||
|
||||
sessionPaths := artifacts.NewLocalStore(cfg.Pipeline.Workspace.Root).SessionPathsFor(cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
objects, err := store.List(ctx, prefix)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list remote session objects under %q: %w", prefix, err)
|
||||
}
|
||||
|
||||
candidates := make(map[string]storage.ObjectInfo, len(objects)+1)
|
||||
for _, obj := range objects {
|
||||
key := normalizeRemoteKey(obj.Key)
|
||||
if key == "" {
|
||||
continue
|
||||
}
|
||||
obj.Key = key
|
||||
candidates[key] = obj
|
||||
}
|
||||
if strings.TrimSpace(current.CurrentManifestKey) != "" {
|
||||
key := normalizeRemoteKey(current.CurrentManifestKey)
|
||||
if _, ok := candidates[key]; !ok {
|
||||
candidates[key] = storage.ObjectInfo{Key: key}
|
||||
}
|
||||
}
|
||||
|
||||
actions := make([]RestoreAction, 0, len(candidates))
|
||||
for key, obj := range candidates {
|
||||
rel, include, err := restoreLocalRelativePathForKey(prefix, normalizeRemoteKey(current.CurrentManifestKey), key, opts.IncludeAudio)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("map remote key %q: %w", key, err)
|
||||
}
|
||||
if !include {
|
||||
continue
|
||||
}
|
||||
localPath, err := joinWithinSessionRoot(sessionPaths.Root, rel)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("map remote key %q: %w", key, err)
|
||||
}
|
||||
|
||||
action, err := classifyRestoreAction(ctx, store, obj, rel, localPath, opts.Force)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("classify remote key %q: %w", key, err)
|
||||
}
|
||||
actions = append(actions, action)
|
||||
}
|
||||
|
||||
previousActions, err := buildPreviousCacheRestoreActions(ctx, cfg, sessionPaths, store, opts.Force)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
actions = append(actions, previousActions...)
|
||||
|
||||
sort.Slice(actions, func(i, j int) bool {
|
||||
if actions[i].LocalRelativePath == actions[j].LocalRelativePath {
|
||||
return actions[i].RemoteKey < actions[j].RemoteKey
|
||||
}
|
||||
return actions[i].LocalRelativePath < actions[j].LocalRelativePath
|
||||
})
|
||||
|
||||
plan := &RestorePlan{Actions: actions}
|
||||
for _, action := range actions {
|
||||
switch action.Kind {
|
||||
case RestoreActionDownload:
|
||||
plan.DownloadCount++
|
||||
case RestoreActionSkipSame:
|
||||
plan.SkipSameCount++
|
||||
case RestoreActionConflict:
|
||||
plan.ConflictCount++
|
||||
}
|
||||
}
|
||||
|
||||
_ = opts.DryRun
|
||||
return plan, nil
|
||||
}
|
||||
|
||||
func normalizeRemoteKey(v string) string {
|
||||
return strings.Trim(strings.ReplaceAll(strings.TrimSpace(v), "\\", "/"), "/")
|
||||
}
|
||||
|
||||
func restoreLocalRelativePathForKey(sessionPrefix, currentManifestKey, key string, includeAudio bool) (string, bool, error) {
|
||||
if key == "" {
|
||||
return "", false, nil
|
||||
}
|
||||
if key == currentManifestKey {
|
||||
return config.PathManifestFile, true, nil
|
||||
}
|
||||
if !strings.HasPrefix(key, sessionPrefix) {
|
||||
return "", false, fmt.Errorf("key is outside resolved session prefix %q", sessionPrefix)
|
||||
}
|
||||
|
||||
rel := strings.TrimPrefix(key, sessionPrefix)
|
||||
rel = strings.TrimSpace(rel)
|
||||
if rel == "" {
|
||||
return "", false, nil
|
||||
}
|
||||
|
||||
cleanRel := path.Clean(rel)
|
||||
if cleanRel == "." || cleanRel == "" {
|
||||
return "", false, nil
|
||||
}
|
||||
if cleanRel == ".." || strings.HasPrefix(cleanRel, "../") || strings.HasPrefix(cleanRel, "/") {
|
||||
return "", false, fmt.Errorf("key relative path %q escapes session scope", rel)
|
||||
}
|
||||
|
||||
if cleanRel == config.PathManifestFile {
|
||||
return config.PathManifestFile, true, nil
|
||||
}
|
||||
if strings.HasPrefix(cleanRel, config.S3CurrentSegment+"/") {
|
||||
return "", false, nil
|
||||
}
|
||||
if strings.HasPrefix(cleanRel, config.S3RunsSegment+"/") {
|
||||
return "", false, nil
|
||||
}
|
||||
|
||||
excludedRoots := []string{
|
||||
config.PathLogsDirSegment,
|
||||
config.PathReportsDirSegment,
|
||||
config.PathConfigDirSegment,
|
||||
config.PathInputsDirSegment,
|
||||
}
|
||||
for _, root := range excludedRoots {
|
||||
if cleanRel == root || strings.HasPrefix(cleanRel, root+"/") {
|
||||
return "", false, nil
|
||||
}
|
||||
}
|
||||
|
||||
if cleanRel == config.PathTranscriptsSegment || strings.HasPrefix(cleanRel, config.PathTranscriptsSegment+"/") {
|
||||
return cleanRel, true, nil
|
||||
}
|
||||
if cleanRel == config.PathArtifactsDirSegment || strings.HasPrefix(cleanRel, config.PathArtifactsDirSegment+"/") {
|
||||
return cleanRel, true, nil
|
||||
}
|
||||
if cleanRel == config.PathPreviousDirSegment || strings.HasPrefix(cleanRel, config.PathPreviousDirSegment+"/") {
|
||||
return "", false, nil
|
||||
}
|
||||
if includeAudio && (cleanRel == config.PathAudioDirSegment || strings.HasPrefix(cleanRel, config.PathAudioDirSegment+"/")) {
|
||||
return cleanRel, true, nil
|
||||
}
|
||||
|
||||
return "", false, nil
|
||||
}
|
||||
|
||||
func joinWithinSessionRoot(sessionRoot, relative string) (string, error) {
|
||||
if strings.TrimSpace(sessionRoot) == "" {
|
||||
return "", fmt.Errorf("session root is required")
|
||||
}
|
||||
cleanRel := path.Clean(strings.TrimSpace(relative))
|
||||
if cleanRel == "." || cleanRel == "" {
|
||||
return "", fmt.Errorf("relative path is required")
|
||||
}
|
||||
if cleanRel == ".." || strings.HasPrefix(cleanRel, "../") || strings.HasPrefix(cleanRel, "/") {
|
||||
return "", fmt.Errorf("relative path escapes session root")
|
||||
}
|
||||
abs := filepath.Clean(filepath.Join(sessionRoot, filepath.FromSlash(cleanRel)))
|
||||
root := filepath.Clean(sessionRoot)
|
||||
if abs != root && !strings.HasPrefix(abs, root+string(filepath.Separator)) {
|
||||
return "", fmt.Errorf("resolved local path escapes session root")
|
||||
}
|
||||
return abs, nil
|
||||
}
|
||||
|
||||
func buildPreviousCacheRestoreActions(
|
||||
ctx context.Context,
|
||||
cfg *config.Config,
|
||||
sessionPaths artifacts.SessionPaths,
|
||||
store storage.ObjectStore,
|
||||
force bool,
|
||||
) ([]RestoreAction, error) {
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Scriptorium == nil {
|
||||
return nil, nil
|
||||
}
|
||||
requirements := artifacts.CollectPreviousArtifactRequirements(cfg.Pipeline.Scriptorium.Artifacts)
|
||||
if len(requirements) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
plan, err := previouscache.BuildPlan(ctx, cfg, sessionPaths, requirements, store)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("plan previous-session cache restore: %w", err)
|
||||
}
|
||||
actions := make([]RestoreAction, 0, len(plan.Records))
|
||||
for _, record := range plan.Records {
|
||||
action, err := classifyRestoreAction(ctx, store, storage.ObjectInfo{Key: record.RemoteKey}, record.LocalRelativePath, record.LocalPath, force)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("classify previous-session cache object %q: %w", record.RemoteKey, err)
|
||||
}
|
||||
actions = append(actions, action)
|
||||
}
|
||||
return actions, nil
|
||||
}
|
||||
|
||||
func classifyRestoreAction(
|
||||
ctx context.Context,
|
||||
store storage.ObjectStore,
|
||||
object storage.ObjectInfo,
|
||||
localRelPath string,
|
||||
localPath string,
|
||||
force bool,
|
||||
) (RestoreAction, error) {
|
||||
action := RestoreAction{
|
||||
RemoteKey: normalizeRemoteKey(object.Key),
|
||||
LocalRelativePath: localRelPath,
|
||||
LocalPath: localPath,
|
||||
Size: object.Size,
|
||||
ETag: object.ETag,
|
||||
}
|
||||
|
||||
info, err := os.Stat(localPath)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
action.Kind = RestoreActionDownload
|
||||
action.Reason = "local file missing"
|
||||
return action, nil
|
||||
}
|
||||
return RestoreAction{}, fmt.Errorf("stat local file: %w", err)
|
||||
}
|
||||
|
||||
action.ExistsLocal = true
|
||||
if info.IsDir() {
|
||||
action.Kind = RestoreActionConflict
|
||||
action.Conflict = true
|
||||
action.Reason = "local path is a directory"
|
||||
return action, nil
|
||||
}
|
||||
|
||||
if restoreRelativePathIsAudio(localRelPath) {
|
||||
if object.Size > 0 {
|
||||
if info.Size() == object.Size {
|
||||
action.Kind = RestoreActionSkipSame
|
||||
action.SameLocal = true
|
||||
action.Reason = "local audio size matches remote content"
|
||||
return action, nil
|
||||
}
|
||||
if force {
|
||||
action.Kind = RestoreActionDownload
|
||||
action.Reason = "local audio differs (size mismatch); overwrite with --force"
|
||||
return action, nil
|
||||
}
|
||||
action.Kind = RestoreActionConflict
|
||||
action.Conflict = true
|
||||
action.Reason = "local audio differs (size mismatch)"
|
||||
return action, nil
|
||||
}
|
||||
if force {
|
||||
action.Kind = RestoreActionDownload
|
||||
action.Reason = "local audio exists; remote size unavailable; overwrite with --force"
|
||||
return action, nil
|
||||
}
|
||||
action.Kind = RestoreActionConflict
|
||||
action.Conflict = true
|
||||
action.Reason = "local audio exists; remote size unavailable"
|
||||
return action, nil
|
||||
}
|
||||
|
||||
if object.Size > 0 && info.Size() != object.Size {
|
||||
if force {
|
||||
action.Kind = RestoreActionDownload
|
||||
action.Reason = "local file differs (size mismatch); overwrite with --force"
|
||||
return action, nil
|
||||
}
|
||||
action.Kind = RestoreActionConflict
|
||||
action.Conflict = true
|
||||
action.Reason = "local file differs (size mismatch)"
|
||||
return action, nil
|
||||
}
|
||||
|
||||
localDigest, err := artifacts.SHA256File(localPath)
|
||||
if err != nil {
|
||||
return RestoreAction{}, fmt.Errorf("checksum local file: %w", err)
|
||||
}
|
||||
remotePath, err := downloadObjectToTemp(ctx, store, action.RemoteKey, "narratio-restore-plan-remote-*.tmp")
|
||||
if err != nil {
|
||||
return RestoreAction{}, fmt.Errorf("download remote object: %w", err)
|
||||
}
|
||||
defer func() { _ = os.Remove(remotePath) }()
|
||||
|
||||
remoteDigest, err := artifacts.SHA256File(remotePath)
|
||||
if err != nil {
|
||||
return RestoreAction{}, fmt.Errorf("checksum remote object: %w", err)
|
||||
}
|
||||
|
||||
if remoteDigest == localDigest {
|
||||
action.Kind = RestoreActionSkipSame
|
||||
action.SameLocal = true
|
||||
action.Reason = "local file matches remote content"
|
||||
return action, nil
|
||||
}
|
||||
|
||||
if force {
|
||||
action.Kind = RestoreActionDownload
|
||||
action.Reason = "local file differs; overwrite with --force"
|
||||
return action, nil
|
||||
}
|
||||
|
||||
action.Kind = RestoreActionConflict
|
||||
action.Conflict = true
|
||||
action.Reason = "local file differs"
|
||||
return action, nil
|
||||
}
|
||||
|
||||
func restoreActionIsAudio(action RestoreAction) bool {
|
||||
return restoreRelativePathIsAudio(action.LocalRelativePath)
|
||||
}
|
||||
|
||||
func restoreRelativePathIsAudio(rel string) bool {
|
||||
cleanRel := path.Clean(strings.TrimSpace(rel))
|
||||
return cleanRel == config.PathAudioDirSegment || strings.HasPrefix(cleanRel, config.PathAudioDirSegment+"/")
|
||||
}
|
||||
|
||||
func writeRestorePlan(out io.Writer, current *RemoteCurrentState, plan *RestorePlan, opts RestorePlanOptions) error {
|
||||
if out == nil {
|
||||
return fmt.Errorf("output writer is required")
|
||||
}
|
||||
if current == nil {
|
||||
return fmt.Errorf("remote current state is required")
|
||||
}
|
||||
if plan == nil {
|
||||
return fmt.Errorf("restore plan is required")
|
||||
}
|
||||
|
||||
if _, err := fmt.Fprintf(
|
||||
out,
|
||||
"restore plan: session %s/%s run=%s actions=%d download=%d skip_same=%d conflict=%d dry_run=%t force=%t include_audio=%t\n",
|
||||
current.Campaign,
|
||||
current.SessionID,
|
||||
current.RunID,
|
||||
len(plan.Actions),
|
||||
plan.DownloadCount,
|
||||
plan.SkipSameCount,
|
||||
plan.ConflictCount,
|
||||
opts.DryRun,
|
||||
opts.Force,
|
||||
opts.IncludeAudio,
|
||||
); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, action := range plan.Actions {
|
||||
if _, err := fmt.Fprintf(out, "%s %s <- %s", action.Kind, action.LocalRelativePath, action.RemoteKey); err != nil {
|
||||
return err
|
||||
}
|
||||
if strings.TrimSpace(action.Reason) != "" {
|
||||
if _, err := fmt.Fprintf(out, " (%s)", action.Reason); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if _, err := fmt.Fprintln(out); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
354
internal/app/restore_plan_test.go
Normal file
354
internal/app/restore_plan_test.go
Normal file
@@ -0,0 +1,354 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
func TestRestorePlanDefaultScope(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||
seedRestoreObject(store, current.SessionPrefix+"transcripts/full.json", []byte(`{"segments":[1]}`))
|
||||
seedRestoreObject(store, current.SessionPrefix+"artifacts/session_recap.md", []byte("# recap\n"))
|
||||
seedRestoreObject(store, current.SessionPrefix+"audio/alice.flac", []byte("audio"))
|
||||
seedRestoreObject(store, current.SessionPrefix+"runs/20260519T010203Z-a1b2/manifest.json", []byte("{}"))
|
||||
seedRestoreObject(store, current.SessionPrefix+"logs/archive.log", []byte("log"))
|
||||
|
||||
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||
if err != nil {
|
||||
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||
}
|
||||
|
||||
got := actionRelPaths(plan.Actions)
|
||||
want := []string{"artifacts/session_recap.md", "manifest.json", "transcripts/full.json"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("action local paths = %#v, want %#v", got, want)
|
||||
}
|
||||
if plan.DownloadCount != 3 || plan.SkipSameCount != 0 || plan.ConflictCount != 0 {
|
||||
t.Fatalf("counts = download=%d skip_same=%d conflict=%d, want 3/0/0", plan.DownloadCount, plan.SkipSameCount, plan.ConflictCount)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestorePlanIncludeAudio(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||
seedRestoreObject(store, current.SessionPrefix+"audio/alice.flac", []byte("audio"))
|
||||
|
||||
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{IncludeAudio: true})
|
||||
if err != nil {
|
||||
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||
}
|
||||
got := actionRelPaths(plan.Actions)
|
||||
want := []string{"audio/alice.flac", "manifest.json"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("action local paths = %#v, want %#v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestorePlanExistingAudioUsesSizeWithoutRemoteChecksumDownload(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||
seedRestoreObject(store, current.SessionPrefix+"audio/alice.flac", []byte("audio"))
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustWriteTestFile(t, filepath.Join(sessionRoot, "audio", "alice.flac"), "local")
|
||||
|
||||
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{IncludeAudio: true})
|
||||
if err != nil {
|
||||
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||
}
|
||||
if len(store.Downloads) != 0 {
|
||||
t.Fatalf("downloads = %d, want no remote checksum download for audio", len(store.Downloads))
|
||||
}
|
||||
actionByRel := map[string]RestoreAction{}
|
||||
for _, action := range plan.Actions {
|
||||
actionByRel[action.LocalRelativePath] = action
|
||||
}
|
||||
audioAction := actionByRel["audio/alice.flac"]
|
||||
if audioAction.Kind != RestoreActionSkipSame {
|
||||
t.Fatalf("audio action kind = %q, want %q", audioAction.Kind, RestoreActionSkipSame)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestorePlanIncludesPreviousCacheByDefault(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
configureRestorePlanPreviousRequirement(cfg, true)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||
seedRestorePreviousCurrent(t, store, cfg, "# previous recap\n")
|
||||
|
||||
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||
if err != nil {
|
||||
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||
}
|
||||
|
||||
got := actionRelPaths(plan.Actions)
|
||||
want := []string{"manifest.json", "previous/artifacts/session_recap.md", "previous/manifest.json"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("action local paths = %#v, want %#v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestorePlanIgnoresCurrentSessionArchivedPreviousCache(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||
seedRestoreObject(store, current.SessionPrefix+"previous/manifest.json", []byte(`{"session_id":"2026-04-26"}`))
|
||||
seedRestoreObject(store, current.SessionPrefix+"previous/artifacts/session_recap.md", []byte("# previous recap\n"))
|
||||
|
||||
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||
if err != nil {
|
||||
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||
}
|
||||
|
||||
got := actionRelPaths(plan.Actions)
|
||||
want := []string{"manifest.json"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("action local paths = %#v, want %#v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestorePlanMissingOptionalPreviousCacheSkipsArtifact(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
configureRestorePlanPreviousRequirement(cfg, false)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||
seedRestorePreviousCurrentManifestOnly(t, store, cfg)
|
||||
|
||||
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||
if err != nil {
|
||||
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||
}
|
||||
|
||||
got := actionRelPaths(plan.Actions)
|
||||
want := []string{"manifest.json", "previous/manifest.json"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("action local paths = %#v, want %#v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestorePlanMissingRequiredPreviousCacheFails(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
configureRestorePlanPreviousRequirement(cfg, true)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||
seedRestorePreviousCurrentManifestOnly(t, store, cfg)
|
||||
|
||||
_, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||
if err == nil || !strings.Contains(err.Error(), "required previous-session artifact") {
|
||||
t.Fatalf("buildRestorePlan() error = %v, want required previous artifact failure", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestorePlanPreviousCacheConflictRequiresForce(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
configureRestorePlanPreviousRequirement(cfg, true)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||
seedRestorePreviousCurrent(t, store, cfg, "# remote previous recap\n")
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustWriteTestFile(t, filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md"), "# local previous recap\n")
|
||||
|
||||
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||
if err != nil {
|
||||
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||
}
|
||||
if plan.ConflictCount != 1 {
|
||||
t.Fatalf("ConflictCount = %d, want 1", plan.ConflictCount)
|
||||
}
|
||||
|
||||
plan, err = buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{Force: true})
|
||||
if err != nil {
|
||||
t.Fatalf("buildRestorePlan(force) error = %v", err)
|
||||
}
|
||||
if plan.ConflictCount != 0 {
|
||||
t.Fatalf("force ConflictCount = %d, want 0", plan.ConflictCount)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestorePlanClassifiesSameAndConflict(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||
seedRestoreObject(store, current.SessionPrefix+"transcripts/full.json", []byte(`{"segments":[1]}`))
|
||||
seedRestoreObject(store, current.SessionPrefix+"artifacts/session_recap.md", []byte("remote-content\n"))
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustWriteTestFile(t, filepath.Join(sessionRoot, "transcripts", "full.json"), `{"segments":[1]}`)
|
||||
mustWriteTestFile(t, filepath.Join(sessionRoot, "artifacts", "session_recap.md"), "different\n")
|
||||
|
||||
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||
if err != nil {
|
||||
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||
}
|
||||
|
||||
if plan.SkipSameCount != 1 {
|
||||
t.Fatalf("SkipSameCount = %d, want 1", plan.SkipSameCount)
|
||||
}
|
||||
if plan.ConflictCount != 1 {
|
||||
t.Fatalf("ConflictCount = %d, want 1", plan.ConflictCount)
|
||||
}
|
||||
|
||||
actionByRel := map[string]RestoreAction{}
|
||||
for _, action := range plan.Actions {
|
||||
actionByRel[action.LocalRelativePath] = action
|
||||
}
|
||||
if actionByRel["transcripts/full.json"].Kind != RestoreActionSkipSame {
|
||||
t.Fatalf("transcripts/full.json kind = %q, want %q", actionByRel["transcripts/full.json"].Kind, RestoreActionSkipSame)
|
||||
}
|
||||
if actionByRel["artifacts/session_recap.md"].Kind != RestoreActionConflict {
|
||||
t.Fatalf("artifacts/session_recap.md kind = %q, want %q", actionByRel["artifacts/session_recap.md"].Kind, RestoreActionConflict)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestorePlanForceTurnsConflictsIntoDownloads(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||
seedRestoreObject(store, current.SessionPrefix+"artifacts/session_recap.md", []byte("remote-content\n"))
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustWriteTestFile(t, filepath.Join(sessionRoot, "artifacts", "session_recap.md"), "different\n")
|
||||
|
||||
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{Force: true})
|
||||
if err != nil {
|
||||
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||
}
|
||||
|
||||
actionByRel := map[string]RestoreAction{}
|
||||
for _, action := range plan.Actions {
|
||||
actionByRel[action.LocalRelativePath] = action
|
||||
}
|
||||
recap := actionByRel["artifacts/session_recap.md"]
|
||||
if recap.Kind != RestoreActionDownload {
|
||||
t.Fatalf("artifacts/session_recap.md kind = %q, want %q", recap.Kind, RestoreActionDownload)
|
||||
}
|
||||
if plan.ConflictCount != 0 {
|
||||
t.Fatalf("ConflictCount = %d, want 0", plan.ConflictCount)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestorePlanTraversalUnsafeKeyFails(t *testing.T) {
|
||||
cfg := restorePlanConfig(t)
|
||||
current := restorePlanCurrentState(t, cfg)
|
||||
store := &storage.FakeBackend{}
|
||||
|
||||
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||
seedRestoreObject(store, current.SessionPrefix+"artifacts/../../escape.txt", []byte("bad"))
|
||||
|
||||
_, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "escapes session scope") {
|
||||
t.Fatalf("error = %v, want traversal safety failure", err)
|
||||
}
|
||||
}
|
||||
|
||||
func seedRestoreObject(store *storage.FakeBackend, key string, data []byte) {
|
||||
store.SeedObject(storage.FakeObject{Key: key, Data: data})
|
||||
}
|
||||
|
||||
func actionRelPaths(actions []RestoreAction) []string {
|
||||
out := make([]string, 0, len(actions))
|
||||
for _, action := range actions {
|
||||
out = append(out, action.LocalRelativePath)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func restorePlanConfig(t *testing.T) *config.Config {
|
||||
t.Helper()
|
||||
workspaceRoot := t.TempDir()
|
||||
return &config.Config{
|
||||
Pipeline: &config.PipelineConfig{
|
||||
Workspace: config.WorkspaceConfig{Root: workspaceRoot},
|
||||
Storage: config.StorageConfig{S3: &config.StorageS3Config{
|
||||
Bucket: "test-bucket",
|
||||
RootPrefix: "dnd",
|
||||
}},
|
||||
},
|
||||
Session: &config.SessionConfig{
|
||||
SessionID: "2026-05-03",
|
||||
Campaign: "sample-campaign",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func configureRestorePlanPreviousRequirement(cfg *config.Config, required bool) {
|
||||
cfg.Session.PreviousSessionID = "2026-04-26"
|
||||
cfg.Pipeline.Scriptorium = &config.ScriptoriumConfig{
|
||||
Artifacts: map[string]config.ScriptoriumArtifactConfig{
|
||||
"session_recap": {
|
||||
Enabled: true,
|
||||
OutputPath: "artifacts/session_recap.md",
|
||||
Inputs: map[string]config.ScriptoriumInputConfig{
|
||||
"previous_recap": {
|
||||
Source: "narratio.previous_session.artifact.session_recap",
|
||||
Required: required,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func restorePlanCurrentState(t *testing.T, cfg *config.Config) *RemoteCurrentState {
|
||||
t.Helper()
|
||||
sessionPrefix := artifacts.S3SessionPrefix("dnd", cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
||||
return &RemoteCurrentState{
|
||||
Bucket: "test-bucket",
|
||||
SessionPrefix: sessionPrefix,
|
||||
CurrentManifestKey: manifestKey,
|
||||
CurrentRunIDKey: runIDKey,
|
||||
RunID: "20260519T010203Z-a1b2c3d4",
|
||||
SessionID: cfg.Session.SessionID,
|
||||
Campaign: cfg.Session.Campaign,
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteRestorePlan(t *testing.T) {
|
||||
current := &RemoteCurrentState{Campaign: "sample-campaign", SessionID: "2026-05-03", RunID: "r-1"}
|
||||
plan := &RestorePlan{Actions: []RestoreAction{{Kind: RestoreActionDownload, LocalRelativePath: "manifest.json", RemoteKey: "k", Reason: "local file missing"}}, DownloadCount: 1}
|
||||
var out strings.Builder
|
||||
if err := writeRestorePlan(&out, current, plan, RestorePlanOptions{DryRun: true}); err != nil {
|
||||
t.Fatalf("writeRestorePlan() error = %v", err)
|
||||
}
|
||||
text := out.String()
|
||||
if !strings.Contains(text, "restore plan: session sample-campaign/2026-05-03 run=r-1") {
|
||||
t.Fatalf("output = %q, want plan summary", text)
|
||||
}
|
||||
if !strings.Contains(text, "download manifest.json <- k") {
|
||||
t.Fatalf("output = %q, want action line", text)
|
||||
}
|
||||
}
|
||||
243
internal/app/restore_report.go
Normal file
243
internal/app/restore_report.go
Normal file
@@ -0,0 +1,243 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
// RestoreReport is the durable restore diagnostic model.
|
||||
type RestoreReport struct {
|
||||
GeneratedAt string `json:"generated_at"`
|
||||
SessionID string `json:"session_id"`
|
||||
Campaign string `json:"campaign"`
|
||||
RunID string `json:"run_id"`
|
||||
DryRun bool `json:"dry_run"`
|
||||
Force bool `json:"force"`
|
||||
IncludeAudio bool `json:"include_audio"`
|
||||
Status string `json:"status"`
|
||||
Error string `json:"error,omitempty"`
|
||||
Plan RestorePlanSummary `json:"plan"`
|
||||
Execution RestoreExecutionStats `json:"execution"`
|
||||
Actions []RestoreReportAction `json:"actions"`
|
||||
reportPathRel string
|
||||
}
|
||||
|
||||
type RestorePlanSummary struct {
|
||||
Actions int `json:"actions"`
|
||||
Download int `json:"download"`
|
||||
SkipSame int `json:"skip_same"`
|
||||
Conflicts int `json:"conflicts"`
|
||||
}
|
||||
|
||||
type RestoreExecutionStats struct {
|
||||
Downloaded int `json:"downloaded"`
|
||||
Failed int `json:"failed"`
|
||||
}
|
||||
|
||||
type RestoreReportAction struct {
|
||||
Kind string `json:"kind"`
|
||||
LocalRelativePath string `json:"local_relative_path"`
|
||||
RemoteKey string `json:"remote_key"`
|
||||
Reason string `json:"reason,omitempty"`
|
||||
Status string `json:"status"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
func newRestoreReport(current *RemoteCurrentState, plan *RestorePlan, opts RestorePlanOptions) (*RestoreReport, error) {
|
||||
if current == nil {
|
||||
return nil, fmt.Errorf("remote current state is required")
|
||||
}
|
||||
if plan == nil {
|
||||
return nil, fmt.Errorf("restore plan is required")
|
||||
}
|
||||
r := &RestoreReport{
|
||||
GeneratedAt: nowUTC().Format("2006-01-02T15:04:05.999999999Z07:00"),
|
||||
SessionID: current.SessionID,
|
||||
Campaign: current.Campaign,
|
||||
RunID: current.RunID,
|
||||
DryRun: opts.DryRun,
|
||||
Force: opts.Force,
|
||||
IncludeAudio: opts.IncludeAudio,
|
||||
Status: "planned",
|
||||
Plan: RestorePlanSummary{
|
||||
Actions: len(plan.Actions),
|
||||
Download: plan.DownloadCount,
|
||||
SkipSame: plan.SkipSameCount,
|
||||
Conflicts: plan.ConflictCount,
|
||||
},
|
||||
Actions: make([]RestoreReportAction, 0, len(plan.Actions)),
|
||||
reportPathRel: filepath.ToSlash(filepath.Join(config.PathReportsDirSegment, "restore-latest.json")),
|
||||
}
|
||||
for _, action := range plan.Actions {
|
||||
r.Actions = append(r.Actions, RestoreReportAction{
|
||||
Kind: string(action.Kind),
|
||||
LocalRelativePath: action.LocalRelativePath,
|
||||
RemoteKey: action.RemoteKey,
|
||||
Reason: action.Reason,
|
||||
Status: initialRestoreActionStatus(action.Kind),
|
||||
})
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
|
||||
func initialRestoreActionStatus(kind RestoreActionKind) string {
|
||||
switch kind {
|
||||
case RestoreActionDownload:
|
||||
return "planned_download"
|
||||
case RestoreActionSkipSame:
|
||||
return "skipped_same"
|
||||
case RestoreActionConflict:
|
||||
return "conflict"
|
||||
default:
|
||||
return "planned"
|
||||
}
|
||||
}
|
||||
|
||||
func (r *RestoreReport) markDownloaded(action RestoreAction) {
|
||||
if r == nil {
|
||||
return
|
||||
}
|
||||
if idx := r.findAction(action); idx >= 0 {
|
||||
r.Actions[idx].Status = "downloaded"
|
||||
r.Actions[idx].Error = ""
|
||||
}
|
||||
r.Execution.Downloaded++
|
||||
}
|
||||
|
||||
func (r *RestoreReport) markFailed(action RestoreAction, err error) {
|
||||
if r == nil {
|
||||
return
|
||||
}
|
||||
if idx := r.findAction(action); idx >= 0 {
|
||||
r.Actions[idx].Status = "failed"
|
||||
if err != nil {
|
||||
r.Actions[idx].Error = err.Error()
|
||||
}
|
||||
}
|
||||
r.Execution.Failed++
|
||||
}
|
||||
|
||||
func (r *RestoreReport) setFailed(err error) {
|
||||
if r == nil {
|
||||
return
|
||||
}
|
||||
r.Status = "failed"
|
||||
if err != nil {
|
||||
r.Error = err.Error()
|
||||
}
|
||||
}
|
||||
|
||||
func (r *RestoreReport) setSucceeded() {
|
||||
if r == nil {
|
||||
return
|
||||
}
|
||||
r.Status = "succeeded"
|
||||
r.Error = ""
|
||||
}
|
||||
|
||||
func (r *RestoreReport) findAction(action RestoreAction) int {
|
||||
if r == nil {
|
||||
return -1
|
||||
}
|
||||
for i := range r.Actions {
|
||||
if r.Actions[i].LocalRelativePath == action.LocalRelativePath && r.Actions[i].RemoteKey == action.RemoteKey {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
func writeRestoreDryRunSummary(out io.Writer, report *RestoreReport) error {
|
||||
if out == nil {
|
||||
return fmt.Errorf("output writer is required")
|
||||
}
|
||||
if report == nil {
|
||||
return fmt.Errorf("restore report is required")
|
||||
}
|
||||
if _, err := fmt.Fprintf(out, "Restore plan for %s/%s\n", report.Campaign, report.SessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := fmt.Fprintf(out, "Remote run: %s\n", report.RunID); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := fmt.Fprintf(out, "Would download: %d\n", report.Plan.Download); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := fmt.Fprintf(out, "Would skip unchanged: %d\n", report.Plan.SkipSame); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := fmt.Fprintf(out, "Conflicts: %d\n", report.Plan.Conflicts); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, action := range report.Actions {
|
||||
line := ""
|
||||
switch action.Status {
|
||||
case "planned_download":
|
||||
line = "Would download: " + action.LocalRelativePath
|
||||
case "skipped_same":
|
||||
line = "Would skip unchanged: " + action.LocalRelativePath
|
||||
case "conflict":
|
||||
line = "Conflict: " + action.LocalRelativePath
|
||||
default:
|
||||
line = strings.TrimSpace(action.Kind) + ": " + action.LocalRelativePath
|
||||
}
|
||||
if _, err := fmt.Fprintln(out, line); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func writeRestoreSuccessSummary(out io.Writer, report *RestoreReport) error {
|
||||
if out == nil {
|
||||
return fmt.Errorf("output writer is required")
|
||||
}
|
||||
if report == nil {
|
||||
return fmt.Errorf("restore report is required")
|
||||
}
|
||||
if _, err := fmt.Fprintf(out, "Restored session archive for %s/%s\n", report.Campaign, report.SessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := fmt.Fprintf(out, "Remote run: %s\n", report.RunID); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := fmt.Fprintf(out, "Downloaded: %d\n", report.Execution.Downloaded); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := fmt.Fprintf(out, "Skipped unchanged: %d\n", report.Plan.SkipSame); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := fmt.Fprintf(out, "Conflicts: %d\n", report.Plan.Conflicts); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func persistRestoreReport(store artifacts.Store, cfg *config.Config, report *RestoreReport) (string, error) {
|
||||
if store == nil {
|
||||
return "", fmt.Errorf("artifact store is required")
|
||||
}
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Session == nil {
|
||||
return "", fmt.Errorf("resolved config with pipeline/session is required")
|
||||
}
|
||||
if report == nil {
|
||||
return "", fmt.Errorf("restore report is required")
|
||||
}
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
reportPath := filepath.Join(sessionRoot, filepath.FromSlash(report.reportPathRel))
|
||||
payload, err := json.MarshalIndent(report, "", " ")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("marshal restore report: %w", err)
|
||||
}
|
||||
payload = append(payload, '\n')
|
||||
if err := store.WriteFileAtomic(reportPath, payload, 0o644); err != nil {
|
||||
return "", fmt.Errorf("write restore report %q: %w", reportPath, err)
|
||||
}
|
||||
return reportPath, nil
|
||||
}
|
||||
413
internal/app/restore_test.go
Normal file
413
internal/app/restore_test.go
Normal file
@@ -0,0 +1,413 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
func TestExecuteRestoreHelp(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
|
||||
code := Execute([]string{"session", "restore", "--help"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0", code)
|
||||
}
|
||||
if stderr.Len() != 0 {
|
||||
t.Fatalf("stderr = %q, want empty", stderr.String())
|
||||
}
|
||||
out := stdout.String()
|
||||
if !strings.Contains(out, "Usage: narratio session restore <session_id>") {
|
||||
t.Fatalf("stdout = %q, want restore usage", out)
|
||||
}
|
||||
if !strings.Contains(out, "--include-audio") {
|
||||
t.Fatalf("stdout = %q, want --include-audio flag", out)
|
||||
}
|
||||
if !strings.Contains(out, "--campaign") {
|
||||
t.Fatalf("stdout = %q, want --campaign flag", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreRecognizedAndReturnsNYI(t *testing.T) {
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
origDiscoverFn := discoverRemoteCurrentStateFn
|
||||
origPlanFn := buildRestorePlanFn
|
||||
origExecuteFn := executeRestorePlanFn
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
discoverRemoteCurrentStateFn = origDiscoverFn
|
||||
buildRestorePlanFn = origPlanFn
|
||||
executeRestorePlanFn = origExecuteFn
|
||||
})
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
return &storage.FakeBackend{}, nil
|
||||
}
|
||||
discoverRemoteCurrentStateFn = func(context.Context, *config.Config, storage.ObjectStore) (*RemoteCurrentState, error) {
|
||||
return &RemoteCurrentState{
|
||||
SessionID: "2026-05-03",
|
||||
Campaign: "sample-campaign",
|
||||
RunID: "20260519T010203Z-a1b2c3d4",
|
||||
}, nil
|
||||
}
|
||||
buildRestorePlanFn = func(context.Context, *config.Config, *RemoteCurrentState, storage.ObjectStore, RestorePlanOptions) (*RestorePlan, error) {
|
||||
return &RestorePlan{
|
||||
Actions: []RestoreAction{
|
||||
{
|
||||
Kind: RestoreActionDownload,
|
||||
LocalRelativePath: "manifest.json",
|
||||
RemoteKey: "dnd/campaigns/sample-campaign/sessions/2026-05-03/current/manifest.json",
|
||||
Reason: "local file missing",
|
||||
},
|
||||
},
|
||||
DownloadCount: 1,
|
||||
}, nil
|
||||
}
|
||||
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(
|
||||
[]string{
|
||||
"session", "restore", "2026-05-03",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--dry-run",
|
||||
"--force",
|
||||
"--include-audio",
|
||||
},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0 for --dry-run restore planning; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if stderr.Len() != 0 {
|
||||
t.Fatalf("stderr = %q, want empty", stderr.String())
|
||||
}
|
||||
outText := stdout.String()
|
||||
if !strings.Contains(outText, "Restore plan for sample-campaign/2026-05-03") {
|
||||
t.Fatalf("stdout = %q, want restore plan summary", outText)
|
||||
}
|
||||
if !strings.Contains(outText, "Would download: 1") {
|
||||
t.Fatalf("stdout = %q, want plan count output", outText)
|
||||
}
|
||||
if !strings.Contains(outText, "Would download: manifest.json") {
|
||||
t.Fatalf("stdout = %q, want action output", outText)
|
||||
}
|
||||
|
||||
manifestPath := artifacts.SessionManifestPathForCampaign(workspaceRoot, "sample-campaign", "2026-05-03")
|
||||
if _, err := os.Stat(manifestPath); !os.IsNotExist(err) {
|
||||
t.Fatalf("manifest should not be created during phase-4 restore planning; stat err=%v", err)
|
||||
}
|
||||
reportPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "reports", "restore-latest.json")
|
||||
if _, err := os.Stat(reportPath); !os.IsNotExist(err) {
|
||||
t.Fatalf("restore report should not be written during dry-run; stat err=%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreRejectsUnexpectedPositionalArguments(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "extra"}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "restore: unexpected positional arguments") {
|
||||
t.Fatalf("stderr = %q, want positional-args failure", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreFailsWhenStorageBackendNotConfigured(t *testing.T) {
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
origDiscoverFn := discoverRemoteCurrentStateFn
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
discoverRemoteCurrentStateFn = origDiscoverFn
|
||||
})
|
||||
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeRestoreConfigWithoutStorage(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "no remote object store backend is configured") {
|
||||
t.Fatalf("stderr = %q, want storage backend preflight failure", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreDiscoveryErrorSurfaced(t *testing.T) {
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
origDiscoverFn := discoverRemoteCurrentStateFn
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
discoverRemoteCurrentStateFn = origDiscoverFn
|
||||
})
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
return &storage.FakeBackend{}, nil
|
||||
}
|
||||
discoverRemoteCurrentStateFn = func(context.Context, *config.Config, storage.ObjectStore) (*RemoteCurrentState, error) {
|
||||
return nil, fmt.Errorf("remote current run pointer missing: %q", "dnd/campaigns/sample-campaign/sessions/2026-05-03/current/run_id.txt")
|
||||
}
|
||||
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "remote current run pointer missing") {
|
||||
t.Fatalf("stderr = %q, want discovery error context", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreLoadsSecretsBeforeObjectStoreInit(t *testing.T) {
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
origDiscoverFn := discoverRemoteCurrentStateFn
|
||||
origPlanFn := buildRestorePlanFn
|
||||
origExecuteFn := executeRestorePlanFn
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
discoverRemoteCurrentStateFn = origDiscoverFn
|
||||
buildRestorePlanFn = origPlanFn
|
||||
executeRestorePlanFn = origExecuteFn
|
||||
})
|
||||
|
||||
const accessKeyEnv = "OBJECT_STORAGE_KEY_ID"
|
||||
const secretKeyEnv = "OBJECT_STORAGE_KEY"
|
||||
restoreEnv := func(name string) {
|
||||
value, exists := os.LookupEnv(name)
|
||||
_ = os.Unsetenv(name)
|
||||
t.Cleanup(func() {
|
||||
if exists {
|
||||
_ = os.Setenv(name, value)
|
||||
return
|
||||
}
|
||||
_ = os.Unsetenv(name)
|
||||
})
|
||||
}
|
||||
restoreEnv(accessKeyEnv)
|
||||
restoreEnv(secretKeyEnv)
|
||||
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
secretsDir := filepath.Join(t.TempDir(), "secrets")
|
||||
mustWriteTestFile(t, filepath.Join(secretsDir, accessKeyEnv), "test-access-key-id\n")
|
||||
mustWriteTestFile(t, filepath.Join(secretsDir, secretKeyEnv), "test-secret-key\n")
|
||||
f, err := os.OpenFile(pipelinePath, os.O_APPEND|os.O_WRONLY, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("open pipeline config for append: %v", err)
|
||||
}
|
||||
defer f.Close()
|
||||
if _, err := f.WriteString("\nsecrets:\n env_dir: " + secretsDir + "\n"); err != nil {
|
||||
t.Fatalf("append secrets config: %v", err)
|
||||
}
|
||||
|
||||
storeInitCalled := false
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
storeInitCalled = true
|
||||
gotID, okID := os.LookupEnv(accessKeyEnv)
|
||||
if !okID || gotID != "test-access-key-id" {
|
||||
return nil, fmt.Errorf("missing or unexpected %s: %q (set=%t)", accessKeyEnv, gotID, okID)
|
||||
}
|
||||
gotSecret, okSecret := os.LookupEnv(secretKeyEnv)
|
||||
if !okSecret || gotSecret != "test-secret-key" {
|
||||
return nil, fmt.Errorf("missing or unexpected %s: %q (set=%t)", secretKeyEnv, gotSecret, okSecret)
|
||||
}
|
||||
return &storage.FakeBackend{}, nil
|
||||
}
|
||||
discoverRemoteCurrentStateFn = func(context.Context, *config.Config, storage.ObjectStore) (*RemoteCurrentState, error) {
|
||||
return &RemoteCurrentState{
|
||||
SessionID: "2026-05-03",
|
||||
Campaign: "sample-campaign",
|
||||
RunID: "20260519T010203Z-a1b2c3d4",
|
||||
}, nil
|
||||
}
|
||||
buildRestorePlanFn = func(context.Context, *config.Config, *RemoteCurrentState, storage.ObjectStore, RestorePlanOptions) (*RestorePlan, error) {
|
||||
return &RestorePlan{
|
||||
Actions: []RestoreAction{
|
||||
{
|
||||
Kind: RestoreActionDownload,
|
||||
LocalRelativePath: "manifest.json",
|
||||
RemoteKey: "dnd/campaigns/sample-campaign/sessions/2026-05-03/current/manifest.json",
|
||||
Reason: "local file missing",
|
||||
},
|
||||
},
|
||||
DownloadCount: 1,
|
||||
}, nil
|
||||
}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(
|
||||
[]string{
|
||||
"session", "restore", "2026-05-03",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--dry-run",
|
||||
},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !storeInitCalled {
|
||||
t.Fatal("expected object store initialization to be called")
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreNonDryRunConflictFailsBeforeNYI(t *testing.T) {
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
origDiscoverFn := discoverRemoteCurrentStateFn
|
||||
origPlanFn := buildRestorePlanFn
|
||||
origExecuteFn := executeRestorePlanFn
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
discoverRemoteCurrentStateFn = origDiscoverFn
|
||||
buildRestorePlanFn = origPlanFn
|
||||
executeRestorePlanFn = origExecuteFn
|
||||
})
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
return &storage.FakeBackend{}, nil
|
||||
}
|
||||
discoverRemoteCurrentStateFn = func(context.Context, *config.Config, storage.ObjectStore) (*RemoteCurrentState, error) {
|
||||
return &RemoteCurrentState{
|
||||
SessionID: "2026-05-03",
|
||||
Campaign: "sample-campaign",
|
||||
RunID: "20260519T010203Z-a1b2c3d4",
|
||||
}, nil
|
||||
}
|
||||
buildRestorePlanFn = func(context.Context, *config.Config, *RemoteCurrentState, storage.ObjectStore, RestorePlanOptions) (*RestorePlan, error) {
|
||||
return &RestorePlan{
|
||||
Actions: []RestoreAction{
|
||||
{Kind: RestoreActionConflict, LocalRelativePath: "transcripts/full.json", RemoteKey: "k", Reason: "local file differs"},
|
||||
},
|
||||
ConflictCount: 1,
|
||||
}, nil
|
||||
}
|
||||
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if stdout.Len() != 0 {
|
||||
t.Fatalf("stdout = %q, want empty on conflict failure", stdout.String())
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "restore conflict: 1 conflicting path(s); rerun with --force to overwrite") {
|
||||
t.Fatalf("stderr = %q, want conflict failure", stderr.String())
|
||||
}
|
||||
if strings.Contains(stderr.String(), "phase 4: restore execution") {
|
||||
t.Fatalf("stderr = %q, should fail before phase-4 NYI boundary", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRestoreNonDryRunForceExecutesPlan(t *testing.T) {
|
||||
origStoreFn := newObjectStoreFromConfigFn
|
||||
origDiscoverFn := discoverRemoteCurrentStateFn
|
||||
origPlanFn := buildRestorePlanFn
|
||||
origExecuteFn := executeRestorePlanFn
|
||||
t.Cleanup(func() {
|
||||
newObjectStoreFromConfigFn = origStoreFn
|
||||
discoverRemoteCurrentStateFn = origDiscoverFn
|
||||
buildRestorePlanFn = origPlanFn
|
||||
executeRestorePlanFn = origExecuteFn
|
||||
})
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
return &storage.FakeBackend{}, nil
|
||||
}
|
||||
discoverRemoteCurrentStateFn = func(context.Context, *config.Config, storage.ObjectStore) (*RemoteCurrentState, error) {
|
||||
return &RemoteCurrentState{
|
||||
SessionID: "2026-05-03",
|
||||
Campaign: "sample-campaign",
|
||||
RunID: "20260519T010203Z-a1b2c3d4",
|
||||
}, nil
|
||||
}
|
||||
buildRestorePlanFn = func(context.Context, *config.Config, *RemoteCurrentState, storage.ObjectStore, RestorePlanOptions) (*RestorePlan, error) {
|
||||
return &RestorePlan{
|
||||
Actions: []RestoreAction{
|
||||
{Kind: RestoreActionDownload, LocalRelativePath: "transcripts/full.json", RemoteKey: "k", Reason: "local file differs; overwrite with --force"},
|
||||
},
|
||||
DownloadCount: 1,
|
||||
}, nil
|
||||
}
|
||||
executeRestorePlanFn = func(context.Context, *config.Config, *RemoteCurrentState, *RestorePlan, *RestoreReport, storage.ObjectStore) (*RestoreExecutionResult, error) {
|
||||
return &RestoreExecutionResult{DownloadedCount: 1}, nil
|
||||
}
|
||||
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "Restored session archive for sample-campaign/2026-05-03") {
|
||||
t.Fatalf("stdout = %q, want completion summary", stdout.String())
|
||||
}
|
||||
if stderr.Len() != 0 {
|
||||
t.Fatalf("stderr = %q, want empty", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func writeRestoreConfigWithoutStorage(t *testing.T, workspaceRoot string) (string, string, string) {
|
||||
t.Helper()
|
||||
|
||||
dir := t.TempDir()
|
||||
pipelinePath := filepath.Join(dir, "pipeline.yml")
|
||||
campaignPath := writeAppTestCampaignConfig(t, dir)
|
||||
sessionPath := filepath.Join(dir, "session.yml")
|
||||
|
||||
pipelineYAML := `workspace:
|
||||
root: ` + workspaceRoot + `
|
||||
whisperx:
|
||||
transcribe_url: https://example.com/transcribe
|
||||
`
|
||||
sessionYAML := `session_id: 2026-05-03
|
||||
campaign: sample-campaign
|
||||
inputs:
|
||||
audio_dir: ./audio
|
||||
speakers_file: ./speakers.yml
|
||||
autocorrect_file: ./autocorrect.yml
|
||||
glossary_file: ./glossary.yml
|
||||
`
|
||||
if err := os.WriteFile(pipelinePath, []byte(pipelineYAML), 0o644); err != nil {
|
||||
t.Fatalf("write pipeline config: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(sessionPath, []byte(sessionYAML), 0o644); err != nil {
|
||||
t.Fatalf("write session config: %v", err)
|
||||
}
|
||||
mustWriteTestFile(t, filepath.Join(dir, "speakers.yml"), "alice: alice.flac\n")
|
||||
mustWriteTestFile(t, filepath.Join(dir, "autocorrect.yml"), "[]\n")
|
||||
mustWriteTestFile(t, filepath.Join(dir, "glossary.yml"), "[]\n")
|
||||
mustWriteTestFile(t, filepath.Join(dir, "audio", "alice.flac"), "audio-bytes")
|
||||
|
||||
return pipelinePath, campaignPath, sessionPath
|
||||
}
|
||||
306
internal/app/restore_workflow_test.go
Normal file
306
internal/app/restore_workflow_test.go
Normal file
@@ -0,0 +1,306 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/scriptorium"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/stage"
|
||||
)
|
||||
|
||||
func TestRestoreThenRunStageForceAnalyzeUsesRestoredDurableState(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
|
||||
fake := &storage.FakeBackend{}
|
||||
cfg, sessionPrefix, manifestKey, runIDKey := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||
seedRestoreObject(fake, runIDKey, []byte("20260519T010203Z-a1b2c3d4\n"))
|
||||
seedRestoreObject(fake, manifestKey, restoreWorkflowManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign))
|
||||
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte(`{"segments":[1,2,3]}`+"\n"))
|
||||
seedRestoreObject(fake, sessionPrefix+"artifacts/session_recap.md", []byte("# restored recap\n"))
|
||||
|
||||
restoreWithStoreAndRealPhases(t, fake)
|
||||
|
||||
origExecuteStagesFn := executeStagesFn
|
||||
t.Cleanup(func() {
|
||||
executeStagesFn = origExecuteStagesFn
|
||||
})
|
||||
executeStagesFn = func(ctx context.Context, cfg *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||
if opts.Env == nil {
|
||||
opts.Env = &Env{}
|
||||
}
|
||||
opts.Env.Scriptorium = &scriptorium.NoopRunner{}
|
||||
return executeStages(ctx, cfg, stages, opts)
|
||||
}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
restoreCode := Execute(
|
||||
[]string{
|
||||
"session", "restore", cfg.Session.SessionID,
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if restoreCode != 0 {
|
||||
t.Fatalf("restore exit code = %d, want 0; stderr=%q", restoreCode, stderr.String())
|
||||
}
|
||||
if stderr.Len() != 0 {
|
||||
t.Fatalf("restore stderr = %q, want empty", stderr.String())
|
||||
}
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "transcripts", "full.json"), `{"segments":[1,2,3]}`+"\n")
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "artifacts", "session_recap.md"), "# restored recap\n")
|
||||
|
||||
manifestStore := &manifest.LocalStore{}
|
||||
sessionManifestPath := artifacts.SessionManifestPathForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
beforeAnalyze, err := manifestStore.Load(context.Background(), sessionManifestPath)
|
||||
if err != nil {
|
||||
t.Fatalf("load restored session manifest: %v", err)
|
||||
}
|
||||
upstreamCompletedAt := map[string]time.Time{}
|
||||
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim"} {
|
||||
rec := beforeAnalyze.Stages[stageName]
|
||||
if rec == nil || rec.Status != manifest.StatusSucceeded || rec.CompletedAt == nil {
|
||||
t.Fatalf("restored manifest stage %q = %#v, want succeeded with completion timestamp", stageName, rec)
|
||||
}
|
||||
upstreamCompletedAt[stageName] = *rec.CompletedAt
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
runStageCode := Execute(
|
||||
[]string{
|
||||
"run-stage", "analyze", cfg.Session.SessionID,
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--force",
|
||||
"--artifacts", "player_handout",
|
||||
},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if runStageCode != 0 {
|
||||
t.Fatalf("run-stage exit code = %d, want 0; stderr=%q", runStageCode, stderr.String())
|
||||
}
|
||||
if stderr.Len() != 0 {
|
||||
t.Fatalf("run-stage stderr = %q, want empty", stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "stage=analyze executed=1 skipped=0 force=true") {
|
||||
t.Fatalf("run-stage stdout = %q, want analyze execution summary", stdout.String())
|
||||
}
|
||||
|
||||
playerHandoutPath := filepath.Join(sessionRoot, "artifacts", "player_handout.md")
|
||||
if _, err := os.Stat(playerHandoutPath); err != nil {
|
||||
t.Fatalf("restored analyze output %q missing: %v", playerHandoutPath, err)
|
||||
}
|
||||
|
||||
afterAnalyze, err := manifestStore.Load(context.Background(), sessionManifestPath)
|
||||
if err != nil {
|
||||
t.Fatalf("load session manifest after run-stage analyze: %v", err)
|
||||
}
|
||||
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim"} {
|
||||
rec := afterAnalyze.Stages[stageName]
|
||||
if rec == nil || rec.Status != manifest.StatusSucceeded || rec.CompletedAt == nil {
|
||||
t.Fatalf("post-analyze manifest stage %q = %#v, want succeeded with completion timestamp", stageName, rec)
|
||||
}
|
||||
if !rec.CompletedAt.Equal(upstreamCompletedAt[stageName]) {
|
||||
t.Fatalf(
|
||||
"stage %q completion changed: before=%s after=%s",
|
||||
stageName,
|
||||
upstreamCompletedAt[stageName].Format(time.RFC3339Nano),
|
||||
rec.CompletedAt.Format(time.RFC3339Nano),
|
||||
)
|
||||
}
|
||||
}
|
||||
analyzeRec := afterAnalyze.Stages["analyze"]
|
||||
if analyzeRec == nil || analyzeRec.Status != manifest.StatusSucceeded {
|
||||
t.Fatalf("post-analyze stage record = %#v, want succeeded", analyzeRec)
|
||||
}
|
||||
|
||||
runManifestPaths, err := filepath.Glob(filepath.Join(sessionRoot, "runs", "*", "manifest.json"))
|
||||
if err != nil {
|
||||
t.Fatalf("glob run manifests: %v", err)
|
||||
}
|
||||
if len(runManifestPaths) != 1 {
|
||||
t.Fatalf("run manifest count = %d, want 1; paths=%v", len(runManifestPaths), runManifestPaths)
|
||||
}
|
||||
runManifest, err := manifestStore.LoadRun(context.Background(), runManifestPaths[0])
|
||||
if err != nil {
|
||||
t.Fatalf("load run manifest %q: %v", runManifestPaths[0], err)
|
||||
}
|
||||
if len(runManifest.RequestedStages) != 1 || runManifest.RequestedStages[0] != "analyze" {
|
||||
t.Fatalf("run manifest requested_stages = %#v, want [analyze]", runManifest.RequestedStages)
|
||||
}
|
||||
if runManifest.Stages["analyze"] == nil || runManifest.Stages["analyze"].Status != manifest.StatusSucceeded {
|
||||
t.Fatalf("run manifest analyze stage = %#v, want succeeded", runManifest.Stages["analyze"])
|
||||
}
|
||||
if runManifest.Stages["prepare"] != nil {
|
||||
t.Fatalf("run manifest should not include upstream prepare stage, got %#v", runManifest.Stages["prepare"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestoreThenAnalyzeUsesRestoredPreviousCacheWithoutObjectStore(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
appendRestoreWorkflowScriptoriumConfig(t, pipelinePath, `
|
||||
scriptorium:
|
||||
binary: scriptorium
|
||||
artifacts:
|
||||
session_recap:
|
||||
enabled: true
|
||||
prompt_id: dnd.session_recap
|
||||
output_path: artifacts/session_recap.md
|
||||
inputs:
|
||||
transcript:
|
||||
source: narratio.transcript.final_trimmed
|
||||
required: true
|
||||
previous_recap:
|
||||
source: narratio.previous_session.artifact.session_recap
|
||||
required: true
|
||||
`)
|
||||
appendRestoreWorkflowScriptoriumConfig(t, sessionPath, `
|
||||
previous_session_id: 2026-04-26
|
||||
`)
|
||||
|
||||
fakeStore := &storage.FakeBackend{}
|
||||
cfg, sessionPrefix, manifestKey, runIDKey := seedRestoreCommittedState(t, fakeStore, pipelinePath, campaignPath, sessionPath)
|
||||
seedRestoreObject(fakeStore, runIDKey, []byte("20260519T010203Z-a1b2c3d4\n"))
|
||||
seedRestoreObject(fakeStore, manifestKey, restoreWorkflowManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign))
|
||||
seedRestoreObject(fakeStore, sessionPrefix+"transcripts/final.trimmed.json", []byte(`{"segments":[]}`+"\n"))
|
||||
seedRestorePreviousCurrent(t, fakeStore, cfg, "# previous recap\n")
|
||||
|
||||
restoreWithStoreAndRealPhases(t, fakeStore)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
restoreCode := Execute(
|
||||
[]string{
|
||||
"session", "restore", cfg.Session.SessionID,
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if restoreCode != 0 {
|
||||
t.Fatalf("restore exit code = %d, want 0; stderr=%q", restoreCode, stderr.String())
|
||||
}
|
||||
if stderr.Len() != 0 {
|
||||
t.Fatalf("restore stderr = %q, want empty", stderr.String())
|
||||
}
|
||||
|
||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "transcripts", "final.trimmed.json"), `{"segments":[]}`+"\n")
|
||||
previousManifestBytes, err := os.ReadFile(filepath.Join(sessionRoot, "previous", "manifest.json"))
|
||||
if err != nil {
|
||||
t.Fatalf("read restored previous manifest: %v", err)
|
||||
}
|
||||
if !strings.Contains(string(previousManifestBytes), `"session_id":"2026-04-26"`) {
|
||||
t.Fatalf("restored previous manifest = %q, want previous session id", string(previousManifestBytes))
|
||||
}
|
||||
mustReadEquals(t, filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md"), "# previous recap\n")
|
||||
|
||||
scriptoriumFake := &scriptorium.FakeRunner{}
|
||||
origExecuteStagesFn := executeStagesFn
|
||||
origObjectStoreFn := newObjectStoreFromConfigFn
|
||||
objectStoreConstructed := false
|
||||
t.Cleanup(func() {
|
||||
executeStagesFn = origExecuteStagesFn
|
||||
newObjectStoreFromConfigFn = origObjectStoreFn
|
||||
})
|
||||
executeStagesFn = func(ctx context.Context, cfg *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||
if opts.Env == nil {
|
||||
opts.Env = &Env{}
|
||||
}
|
||||
opts.Env.Scriptorium = scriptoriumFake
|
||||
return executeStages(ctx, cfg, stages, opts)
|
||||
}
|
||||
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||
objectStoreConstructed = true
|
||||
return nil, context.Canceled
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
runStageCode := Execute(
|
||||
[]string{
|
||||
"run-stage", "analyze", cfg.Session.SessionID,
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--force",
|
||||
"--artifacts", "session_recap",
|
||||
},
|
||||
&stdout,
|
||||
&stderr,
|
||||
)
|
||||
if runStageCode != 0 {
|
||||
t.Fatalf("run-stage exit code = %d, want 0; stderr=%q", runStageCode, stderr.String())
|
||||
}
|
||||
if stderr.Len() != 0 {
|
||||
t.Fatalf("run-stage stderr = %q, want empty", stderr.String())
|
||||
}
|
||||
if objectStoreConstructed {
|
||||
t.Fatal("analyze run-stage should not construct object store for previous-session input resolution")
|
||||
}
|
||||
if len(scriptoriumFake.RunRequests) != 1 {
|
||||
t.Fatalf("scriptorium run requests = %d, want 1", len(scriptoriumFake.RunRequests))
|
||||
}
|
||||
req := scriptoriumFake.RunRequests[0]
|
||||
if got := req.InputPaths["transcript"]; got != filepath.Join(sessionRoot, "transcripts", "final.trimmed.json") {
|
||||
t.Fatalf("transcript input = %q, want trimmed transcript path", got)
|
||||
}
|
||||
if got := req.InputPaths["previous_recap"]; got != filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md") {
|
||||
t.Fatalf("previous_recap input = %q, want restored previous cache path", got)
|
||||
}
|
||||
}
|
||||
|
||||
func restoreWorkflowManifestJSON(t *testing.T, sessionID, campaign string) []byte {
|
||||
t.Helper()
|
||||
store := &manifest.LocalStore{}
|
||||
now := time.Date(2026, 5, 19, 23, 0, 0, 0, time.UTC)
|
||||
m := manifest.New(sessionID, now)
|
||||
m.Campaign = campaign
|
||||
m.RunID = "20260519T010203Z-a1b2c3d4"
|
||||
stages := []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim"}
|
||||
for i, stageName := range stages {
|
||||
m.MarkStageSucceeded(stageName, now.Add(time.Duration(i+1)*time.Minute), nil)
|
||||
}
|
||||
path := filepath.Join(t.TempDir(), "manifest.json")
|
||||
if err := store.Save(context.Background(), path, m); err != nil {
|
||||
t.Fatalf("save workflow manifest fixture: %v", err)
|
||||
}
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("read workflow manifest fixture: %v", err)
|
||||
}
|
||||
return data
|
||||
}
|
||||
|
||||
func appendRestoreWorkflowScriptoriumConfig(t *testing.T, pipelinePath, extra string) {
|
||||
t.Helper()
|
||||
f, err := os.OpenFile(pipelinePath, os.O_APPEND|os.O_WRONLY, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("open pipeline config for append: %v", err)
|
||||
}
|
||||
defer f.Close()
|
||||
if _, err := f.WriteString(extra); err != nil {
|
||||
t.Fatalf("append pipeline config: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
@@ -13,37 +14,47 @@ import (
|
||||
|
||||
// Resume continues execution from the first non-succeeded stage in the manifest.
|
||||
func Resume(ctx context.Context, args []string, out io.Writer) error {
|
||||
positionalSessionID, args := pullLeadingSessionID(args)
|
||||
fs := flag.NewFlagSet("resume", flag.ContinueOnError)
|
||||
fs.SetOutput(io.Discard)
|
||||
|
||||
var pipelinePath string
|
||||
var campaignPath string
|
||||
var campaignFilePath string
|
||||
var sessionPath string
|
||||
var sessionID string
|
||||
var previousSessionID string
|
||||
var force bool
|
||||
var selectedArtifacts artifactSelectionFlag
|
||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||
fs.StringVar(&sessionID, "session-id", "", "session identifier for session.yml templates")
|
||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||
fs.BoolVar(&force, "force", false, "force stage execution")
|
||||
fs.Var(&selectedArtifacts, "artifacts", "artifact names to execute during analyze (comma-separated or repeatable)")
|
||||
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute and publish (comma-separated or repeatable)")
|
||||
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return fmt.Errorf("resume: invalid flags: %w", err)
|
||||
}
|
||||
if fs.NArg() != 0 {
|
||||
return fmt.Errorf("resume: unexpected positional arguments")
|
||||
if positionalSessionID == "" {
|
||||
if err := applyParsedSessionIDArg("resume", fs, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
if fs.NArg() != 0 {
|
||||
return fmt.Errorf("resume: unexpected positional arguments")
|
||||
}
|
||||
if err := applyPositionalSessionID("resume", positionalSessionID, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
resolvedPipelinePath, err := resolvePipelineConfigPath(pipelinePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("resume: %w", err)
|
||||
if strings.TrimSpace(sessionID) == "" {
|
||||
return fmt.Errorf("resume: session_id is required")
|
||||
}
|
||||
resolvedSessionPath, err := resolveSessionConfigPath(sessionPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("resume: %w", err)
|
||||
}
|
||||
|
||||
cfg, err := config.LoadWithSessionOptions(resolvedPipelinePath, resolvedSessionPath, config.SessionLoadOptions{
|
||||
SessionID: sessionID,
|
||||
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
||||
SessionID: sessionID,
|
||||
PreviousSessionID: previousSessionID,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("resume: %w", err)
|
||||
@@ -55,7 +66,7 @@ func Resume(ctx context.Context, args []string, out io.Writer) error {
|
||||
if err != nil {
|
||||
return fmt.Errorf("resume: invalid --artifacts: %w", err)
|
||||
}
|
||||
if err := validateSelectedAnalyzeArtifacts(cfg, normalizedArtifacts); err != nil {
|
||||
if err := validateSelectedArtifacts(cfg, normalizedArtifacts); err != nil {
|
||||
return fmt.Errorf("resume: %w", err)
|
||||
}
|
||||
|
||||
@@ -76,7 +87,7 @@ func Resume(ctx context.Context, args []string, out io.Writer) error {
|
||||
}
|
||||
}
|
||||
|
||||
summary, err := executeStages(ctx, cfg, selected, RunOptions{
|
||||
summary, err := executeStagesFn(ctx, cfg, selected, RunOptions{
|
||||
Force: force,
|
||||
SelectedArtifacts: normalizedArtifacts,
|
||||
})
|
||||
|
||||
@@ -15,7 +15,7 @@ import (
|
||||
|
||||
func TestResumeStartsAfterCompletedStages(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||
|
||||
store := &manifest.LocalStore{}
|
||||
@@ -32,7 +32,7 @@ func TestResumeStartsAfterCompletedStages(t *testing.T) {
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
||||
|
||||
var out bytes.Buffer
|
||||
err := Resume(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath}, &out)
|
||||
err := Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("Resume() error = %v", err)
|
||||
}
|
||||
@@ -51,7 +51,7 @@ func TestResumeStartsAfterCompletedStages(t *testing.T) {
|
||||
|
||||
func TestResumeNoRemainingStages(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||
|
||||
store := &manifest.LocalStore{}
|
||||
@@ -64,7 +64,7 @@ func TestResumeNoRemainingStages(t *testing.T) {
|
||||
}
|
||||
|
||||
var out bytes.Buffer
|
||||
err := Resume(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath}, &out)
|
||||
err := Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("Resume() error = %v", err)
|
||||
}
|
||||
@@ -80,7 +80,7 @@ func TestResumeForceRerunsSucceeded(t *testing.T) {
|
||||
_, _ = w.Write([]byte(`{"source":"resume-force-test","segments":[{"speaker":"alice"}]}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||
|
||||
store := &manifest.LocalStore{}
|
||||
@@ -93,7 +93,7 @@ func TestResumeForceRerunsSucceeded(t *testing.T) {
|
||||
}
|
||||
|
||||
var out bytes.Buffer
|
||||
err := Resume(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "--force"}, &out)
|
||||
err := Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("Resume() error = %v", err)
|
||||
}
|
||||
@@ -104,14 +104,14 @@ func TestResumeForceRerunsSucceeded(t *testing.T) {
|
||||
|
||||
func TestRunStageExecutesOnlySelectedStage(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "merged.json"), `{"segments":[]}`)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "base.json"), `{"segments":[]}`)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
||||
|
||||
var out bytes.Buffer
|
||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "polish"}, &out)
|
||||
err := RunStage(context.Background(), []string{"polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("RunStage() error = %v", err)
|
||||
}
|
||||
@@ -134,10 +134,10 @@ func TestRunStageExecutesOnlySelectedStage(t *testing.T) {
|
||||
|
||||
func TestRunStageSkipAndForce(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "merged.json"), `{"segments":[]}`)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "base.json"), `{"segments":[]}`)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
||||
|
||||
store := &manifest.LocalStore{}
|
||||
@@ -148,7 +148,7 @@ func TestRunStageSkipAndForce(t *testing.T) {
|
||||
}
|
||||
|
||||
var out bytes.Buffer
|
||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "polish"}, &out)
|
||||
err := RunStage(context.Background(), []string{"polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("RunStage() error = %v", err)
|
||||
}
|
||||
@@ -157,7 +157,7 @@ func TestRunStageSkipAndForce(t *testing.T) {
|
||||
}
|
||||
|
||||
out.Reset()
|
||||
err = RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "--force", "polish"}, &out)
|
||||
err = RunStage(context.Background(), []string{"polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("RunStage(force) error = %v", err)
|
||||
}
|
||||
@@ -168,10 +168,10 @@ func TestRunStageSkipAndForce(t *testing.T) {
|
||||
|
||||
func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "merged.json"), `{"segments":[]}`)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "base.json"), `{"segments":[]}`)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
||||
|
||||
store := &manifest.LocalStore{}
|
||||
@@ -184,7 +184,7 @@ func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing
|
||||
}
|
||||
|
||||
var out bytes.Buffer
|
||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "--force", "polish"}, &out)
|
||||
err := RunStage(context.Background(), []string{"polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("RunStage(force) error = %v", err)
|
||||
}
|
||||
@@ -203,7 +203,7 @@ func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing
|
||||
}
|
||||
|
||||
out.Reset()
|
||||
err = Resume(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath}, &out)
|
||||
err = Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("Resume() error = %v", err)
|
||||
}
|
||||
@@ -214,13 +214,13 @@ func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing
|
||||
|
||||
func TestRunStageTrimExecutes(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "normalized.json"), `{"segments":[{"id":1},{"id":2}]}`)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.json"), `{"segments":[{"id":1},{"id":2}]}`)
|
||||
|
||||
var out bytes.Buffer
|
||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "trim"}, &out)
|
||||
err := RunStage(context.Background(), []string{"trim", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("RunStage(trim) error = %v", err)
|
||||
}
|
||||
@@ -243,13 +243,13 @@ func TestRunStageTrimExecutes(t *testing.T) {
|
||||
|
||||
func TestRunStageNormalizeExecutes(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "processed.json"), `{"segments":[{"id":1},{"id":2}]}`)
|
||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "polished.json"), `{"segments":[{"id":1},{"id":2}]}`)
|
||||
|
||||
var out bytes.Buffer
|
||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "normalize"}, &out)
|
||||
err := RunStage(context.Background(), []string{"normalize", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("RunStage(normalize) error = %v", err)
|
||||
}
|
||||
|
||||
@@ -5,43 +5,54 @@ import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
// Run executes the pipeline plan and persists manifest state.
|
||||
func Run(ctx context.Context, args []string, out io.Writer) error {
|
||||
positionalSessionID, args := pullLeadingSessionID(args)
|
||||
fs := flag.NewFlagSet("run", flag.ContinueOnError)
|
||||
fs.SetOutput(io.Discard)
|
||||
|
||||
var pipelinePath string
|
||||
var campaignPath string
|
||||
var campaignFilePath string
|
||||
var sessionPath string
|
||||
var sessionID string
|
||||
var previousSessionID string
|
||||
var force bool
|
||||
var selectedArtifacts artifactSelectionFlag
|
||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||
fs.StringVar(&sessionID, "session-id", "", "session identifier for session.yml templates")
|
||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
||||
fs.Var(&selectedArtifacts, "artifacts", "artifact names to execute during analyze (comma-separated or repeatable)")
|
||||
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute and publish (comma-separated or repeatable)")
|
||||
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return fmt.Errorf("run: invalid flags: %w", err)
|
||||
}
|
||||
if fs.NArg() != 0 {
|
||||
return fmt.Errorf("run: unexpected positional arguments")
|
||||
if positionalSessionID == "" {
|
||||
if err := applyParsedSessionIDArg("run", fs, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
if fs.NArg() != 0 {
|
||||
return fmt.Errorf("run: unexpected positional arguments")
|
||||
}
|
||||
if err := applyPositionalSessionID("run", positionalSessionID, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
resolvedPipelinePath, err := resolvePipelineConfigPath(pipelinePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("run: %w", err)
|
||||
if strings.TrimSpace(sessionID) == "" {
|
||||
return fmt.Errorf("run: session_id is required")
|
||||
}
|
||||
resolvedSessionPath, err := resolveSessionConfigPath(sessionPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("run: %w", err)
|
||||
}
|
||||
|
||||
cfg, err := config.LoadWithSessionOptions(resolvedPipelinePath, resolvedSessionPath, config.SessionLoadOptions{
|
||||
SessionID: sessionID,
|
||||
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
||||
SessionID: sessionID,
|
||||
PreviousSessionID: previousSessionID,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("run: %w", err)
|
||||
@@ -53,12 +64,12 @@ func Run(ctx context.Context, args []string, out io.Writer) error {
|
||||
if err != nil {
|
||||
return fmt.Errorf("run: invalid --artifacts: %w", err)
|
||||
}
|
||||
if err := validateSelectedAnalyzeArtifacts(cfg, normalizedArtifacts); err != nil {
|
||||
if err := validateSelectedArtifacts(cfg, normalizedArtifacts); err != nil {
|
||||
return fmt.Errorf("run: %w", err)
|
||||
}
|
||||
|
||||
stages := BuildFullPlan()
|
||||
summary, err := executeStages(ctx, cfg, stages, RunOptions{
|
||||
summary, err := executeStagesFn(ctx, cfg, stages, RunOptions{
|
||||
Force: force,
|
||||
SelectedArtifacts: normalizedArtifacts,
|
||||
})
|
||||
|
||||
@@ -5,79 +5,88 @@ import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
// RunStage executes exactly one selected stage.
|
||||
func RunStage(ctx context.Context, args []string, out io.Writer) error {
|
||||
var stageName string
|
||||
var positionalSessionID string
|
||||
if len(args) >= 2 && !isCLIFlagToken(args[0]) && !isCLIFlagToken(args[1]) {
|
||||
stageName = strings.TrimSpace(args[0])
|
||||
positionalSessionID = strings.TrimSpace(args[1])
|
||||
args = append([]string(nil), args[2:]...)
|
||||
}
|
||||
|
||||
fs := flag.NewFlagSet("run-stage", flag.ContinueOnError)
|
||||
fs.SetOutput(io.Discard)
|
||||
|
||||
var pipelinePath string
|
||||
var campaignPath string
|
||||
var campaignFilePath string
|
||||
var sessionPath string
|
||||
var sessionID string
|
||||
var previousSessionID string
|
||||
var force bool
|
||||
var selectedArtifacts artifactSelectionFlag
|
||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||
fs.StringVar(&sessionID, "session-id", "", "session identifier for session.yml templates")
|
||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
||||
fs.Var(&selectedArtifacts, "artifacts", "artifact names to execute during analyze (comma-separated or repeatable)")
|
||||
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute or publish (comma-separated or repeatable)")
|
||||
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return fmt.Errorf("run-stage: invalid flags: %w", err)
|
||||
}
|
||||
if fs.NArg() != 1 {
|
||||
return fmt.Errorf("run-stage: expected exactly one stage name")
|
||||
if stageName == "" {
|
||||
switch fs.NArg() {
|
||||
case 2:
|
||||
stageName = strings.TrimSpace(fs.Arg(0))
|
||||
positionalSessionID = strings.TrimSpace(fs.Arg(1))
|
||||
default:
|
||||
return fmt.Errorf("run-stage: expected stage name and session_id")
|
||||
}
|
||||
} else if fs.NArg() != 0 {
|
||||
return fmt.Errorf("run-stage: unexpected positional arguments")
|
||||
}
|
||||
if err := applyPositionalSessionID("run-stage", positionalSessionID, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
if strings.TrimSpace(sessionID) == "" {
|
||||
return fmt.Errorf("run-stage: session_id is required")
|
||||
}
|
||||
stageName := fs.Arg(0)
|
||||
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
||||
if err != nil {
|
||||
return fmt.Errorf("run-stage: invalid --artifacts: %w", err)
|
||||
}
|
||||
if len(normalizedArtifacts) > 0 && stageName != "analyze" {
|
||||
return fmt.Errorf("run-stage: --artifacts is only supported for stage \"analyze\"")
|
||||
}
|
||||
stages, err := BuildSingleStagePlan(stageName)
|
||||
if err != nil {
|
||||
return fmt.Errorf("run-stage: %w", err)
|
||||
if len(normalizedArtifacts) > 0 && stageName != "analyze" && stageName != "archive" {
|
||||
return fmt.Errorf("run-stage: --artifacts is only supported for stages \"analyze\" and \"archive\"")
|
||||
}
|
||||
|
||||
resolvedPipelinePath, err := resolvePipelineConfigPath(pipelinePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("run-stage: %w", err)
|
||||
}
|
||||
resolvedSessionPath, err := resolveSessionConfigPath(sessionPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("run-stage: %w", err)
|
||||
}
|
||||
|
||||
cfg, err := config.LoadWithSessionOptions(resolvedPipelinePath, resolvedSessionPath, config.SessionLoadOptions{
|
||||
SessionID: sessionID,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("run-stage: %w", err)
|
||||
}
|
||||
if err := config.Validate(cfg); err != nil {
|
||||
return fmt.Errorf("run-stage: %w", err)
|
||||
}
|
||||
if err := validateSelectedAnalyzeArtifacts(cfg, normalizedArtifacts); err != nil {
|
||||
return fmt.Errorf("run-stage: %w", err)
|
||||
}
|
||||
|
||||
summary, err := executeStages(ctx, cfg, stages, RunOptions{
|
||||
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
||||
CommandName: "run-stage",
|
||||
StageName: stageName,
|
||||
PipelinePath: pipelinePath,
|
||||
CampaignPath: campaignPath,
|
||||
CampaignFilePath: campaignFilePath,
|
||||
SessionPath: sessionPath,
|
||||
SessionID: sessionID,
|
||||
PreviousSessionID: previousSessionID,
|
||||
Force: force,
|
||||
SelectedArtifacts: normalizedArtifacts,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("run-stage: %w", err)
|
||||
return err
|
||||
}
|
||||
|
||||
_, err = fmt.Fprintf(
|
||||
out,
|
||||
"narratio run-stage: stage=%s executed=%d skipped=%d force=%t; manifest=%s\n",
|
||||
stages[0].Name(),
|
||||
stageName,
|
||||
len(summary.Executed),
|
||||
len(summary.Skipped),
|
||||
force,
|
||||
@@ -85,3 +94,184 @@ func RunStage(ctx context.Context, args []string, out io.Writer) error {
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
// Analyze force-runs the analyze stage.
|
||||
func Analyze(ctx context.Context, args []string, out io.Writer) error {
|
||||
positionalSessionID, args := pullLeadingSessionID(args)
|
||||
fs := flag.NewFlagSet("analyze", flag.ContinueOnError)
|
||||
fs.SetOutput(io.Discard)
|
||||
|
||||
var pipelinePath string
|
||||
var campaignPath string
|
||||
var campaignFilePath string
|
||||
var sessionPath string
|
||||
var sessionID string
|
||||
var previousSessionID string
|
||||
var selectedArtifacts artifactSelectionFlag
|
||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute during analyze (comma-separated or repeatable)")
|
||||
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return fmt.Errorf("analyze: invalid flags: %w", err)
|
||||
}
|
||||
if positionalSessionID == "" {
|
||||
if err := applyParsedSessionIDArg("analyze", fs, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
if fs.NArg() != 0 {
|
||||
return fmt.Errorf("analyze: unexpected positional arguments")
|
||||
}
|
||||
if err := applyPositionalSessionID("analyze", positionalSessionID, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if strings.TrimSpace(sessionID) == "" {
|
||||
return fmt.Errorf("analyze: session_id is required")
|
||||
}
|
||||
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
||||
if err != nil {
|
||||
return fmt.Errorf("analyze: invalid --artifacts: %w", err)
|
||||
}
|
||||
|
||||
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
||||
CommandName: "analyze",
|
||||
StageName: "analyze",
|
||||
PipelinePath: pipelinePath,
|
||||
CampaignPath: campaignPath,
|
||||
CampaignFilePath: campaignFilePath,
|
||||
SessionPath: sessionPath,
|
||||
SessionID: sessionID,
|
||||
PreviousSessionID: previousSessionID,
|
||||
Force: true,
|
||||
SelectedArtifacts: normalizedArtifacts,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
_, err = fmt.Fprintf(
|
||||
out,
|
||||
"narratio analyze: executed=%d skipped=%d force=true; manifest=%s\n",
|
||||
len(summary.Executed),
|
||||
len(summary.Skipped),
|
||||
summary.ManifestPath,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
// Publish force-runs the archive stage.
|
||||
func Publish(ctx context.Context, args []string, out io.Writer) error {
|
||||
positionalSessionID, args := pullLeadingSessionID(args)
|
||||
fs := flag.NewFlagSet("publish", flag.ContinueOnError)
|
||||
fs.SetOutput(io.Discard)
|
||||
|
||||
var pipelinePath string
|
||||
var campaignPath string
|
||||
var campaignFilePath string
|
||||
var sessionPath string
|
||||
var sessionID string
|
||||
var previousSessionID string
|
||||
var selectedArtifacts artifactSelectionFlag
|
||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to publish (comma-separated or repeatable)")
|
||||
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return fmt.Errorf("publish: invalid flags: %w", err)
|
||||
}
|
||||
if positionalSessionID == "" {
|
||||
if err := applyParsedSessionIDArg("publish", fs, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
if fs.NArg() != 0 {
|
||||
return fmt.Errorf("publish: unexpected positional arguments")
|
||||
}
|
||||
if err := applyPositionalSessionID("publish", positionalSessionID, &sessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if strings.TrimSpace(sessionID) == "" {
|
||||
return fmt.Errorf("publish: session_id is required")
|
||||
}
|
||||
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
||||
if err != nil {
|
||||
return fmt.Errorf("publish: invalid --artifacts: %w", err)
|
||||
}
|
||||
|
||||
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
||||
CommandName: "publish",
|
||||
StageName: "archive",
|
||||
PipelinePath: pipelinePath,
|
||||
CampaignPath: campaignPath,
|
||||
CampaignFilePath: campaignFilePath,
|
||||
SessionPath: sessionPath,
|
||||
SessionID: sessionID,
|
||||
PreviousSessionID: previousSessionID,
|
||||
Force: true,
|
||||
SelectedArtifacts: normalizedArtifacts,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
_, err = fmt.Fprintf(
|
||||
out,
|
||||
"narratio publish: executed=%d skipped=%d force=true; manifest=%s\n",
|
||||
len(summary.Executed),
|
||||
len(summary.Skipped),
|
||||
summary.ManifestPath,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
type singleStageCommand struct {
|
||||
CommandName string
|
||||
StageName string
|
||||
PipelinePath string
|
||||
CampaignPath string
|
||||
CampaignFilePath string
|
||||
SessionPath string
|
||||
SessionID string
|
||||
PreviousSessionID string
|
||||
Force bool
|
||||
SelectedArtifacts []string
|
||||
}
|
||||
|
||||
func runSingleStageCommand(ctx context.Context, req singleStageCommand) (*RunSummary, error) {
|
||||
stages, err := BuildSingleStagePlan(req.StageName)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", req.CommandName, err)
|
||||
}
|
||||
|
||||
cfg, err := loadCommandConfig(ctx, req.PipelinePath, req.CampaignPath, req.CampaignFilePath, req.SessionPath, config.SessionLoadOptions{
|
||||
SessionID: req.SessionID,
|
||||
PreviousSessionID: req.PreviousSessionID,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", req.CommandName, err)
|
||||
}
|
||||
if err := config.Validate(cfg); err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", req.CommandName, err)
|
||||
}
|
||||
if err := validateSelectedArtifacts(cfg, req.SelectedArtifacts); err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", req.CommandName, err)
|
||||
}
|
||||
|
||||
summary, err := executeStagesFn(ctx, cfg, stages, RunOptions{
|
||||
Force: req.Force,
|
||||
SelectedArtifacts: req.SelectedArtifacts,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", req.CommandName, err)
|
||||
}
|
||||
return summary, nil
|
||||
}
|
||||
|
||||
@@ -36,6 +36,8 @@ type RunSummary struct {
|
||||
Skipped []string
|
||||
}
|
||||
|
||||
var executeStagesFn = executeStages
|
||||
|
||||
func executeStages(ctx context.Context, cfg *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||
env := opts.Env
|
||||
if env == nil {
|
||||
@@ -44,7 +46,7 @@ func executeStages(ctx context.Context, cfg *config.Config, stages []stage.Stage
|
||||
if env.Config == nil {
|
||||
env.Config = cfg
|
||||
}
|
||||
env.SelectedAnalyzeArtifacts = append([]string(nil), opts.SelectedArtifacts...)
|
||||
env.SelectedArtifactKeys = append([]string(nil), opts.SelectedArtifacts...)
|
||||
if env.ArtifactStore == nil {
|
||||
env.ArtifactStore = artifacts.NewLocalStore(cfg.Pipeline.Workspace.Root)
|
||||
}
|
||||
@@ -85,12 +87,19 @@ func executeStages(ctx context.Context, cfg *config.Config, stages []stage.Stage
|
||||
env.Storage = &storage.NoopBackend{}
|
||||
}
|
||||
if env.ObjectStore == nil && needsObjectStoreForRun(env.Config, stages) {
|
||||
objectStore, err := storage.NewObjectStoreFromConfig(ctx, env.Config)
|
||||
objectStore, err := newCommandObjectStore(ctx, env.Config, nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initialize object store backend: %w", err)
|
||||
return nil, err
|
||||
}
|
||||
env.ObjectStore = objectStore
|
||||
}
|
||||
if needsRemoteLocksForRun(env.Config, stages) {
|
||||
locks, err := loadEffectiveLocks(ctx, env.Config, env.ObjectStore)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("load remote archive locks: %w", err)
|
||||
}
|
||||
applyEffectiveLocks(env.Config, locks.All)
|
||||
}
|
||||
if env.Notifier == nil {
|
||||
env.Notifier = ¬ify.NoopSender{}
|
||||
}
|
||||
@@ -553,6 +562,12 @@ func needsObjectStoreForRun(cfg *config.Config, stages []stage.Stage) bool {
|
||||
if cfg.Session.Inputs.AudioS3 != nil && stageRequested("prepare") {
|
||||
return true
|
||||
}
|
||||
if stageRequested("prepare") {
|
||||
requirements := artifacts.CollectPreviousArtifactRequirements(configuredScriptoriumArtifacts(cfg))
|
||||
if len(requirements) > 0 && strings.TrimSpace(cfg.Session.PreviousSessionID) != "" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
if !stageRequested("archive") {
|
||||
return false
|
||||
}
|
||||
@@ -567,3 +582,36 @@ func needsObjectStoreForRun(cfg *config.Config, stages []stage.Stage) bool {
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func needsRemoteLocksForRun(cfg *config.Config, stages []stage.Stage) bool {
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Session == nil {
|
||||
return false
|
||||
}
|
||||
archiveRequested := false
|
||||
for _, s := range stages {
|
||||
if s != nil && s.Name() == "archive" {
|
||||
archiveRequested = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !archiveRequested {
|
||||
return false
|
||||
}
|
||||
if cfg.Pipeline.Archive == nil {
|
||||
return false
|
||||
}
|
||||
if cfg.Pipeline.Archive.Enabled != nil && !*cfg.Pipeline.Archive.Enabled {
|
||||
return false
|
||||
}
|
||||
if cfg.Pipeline.Archive.UploadRun != nil && !*cfg.Pipeline.Archive.UploadRun {
|
||||
return false
|
||||
}
|
||||
return cfg.Pipeline.Storage.S3 != nil
|
||||
}
|
||||
|
||||
func configuredScriptoriumArtifacts(cfg *config.Config) map[string]config.ScriptoriumArtifactConfig {
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Scriptorium == nil {
|
||||
return nil
|
||||
}
|
||||
return cfg.Pipeline.Scriptorium.Artifacts
|
||||
}
|
||||
|
||||
@@ -54,7 +54,7 @@ func (s captureSelectedArtifactsStage) Name() string { return s.name }
|
||||
func (s captureSelectedArtifactsStage) Declares() stage.IODecl { return stage.IODecl{} }
|
||||
func (s captureSelectedArtifactsStage) Run(_ context.Context, env *stage.Env, _ *manifest.Manifest) (*stage.StageResult, error) {
|
||||
if s.captured != nil {
|
||||
*s.captured = append((*s.captured)[:0], env.SelectedAnalyzeArtifacts...)
|
||||
*s.captured = append((*s.captured)[:0], env.SelectedArtifactKeys...)
|
||||
}
|
||||
return &stage.StageResult{Metadata: map[string]any{"captured": true}}, nil
|
||||
}
|
||||
@@ -78,12 +78,12 @@ type selectedAnalyzeArtifactStage struct {
|
||||
func (s selectedAnalyzeArtifactStage) Name() string { return "analyze" }
|
||||
func (s selectedAnalyzeArtifactStage) Declares() stage.IODecl { return stage.IODecl{} }
|
||||
func (s selectedAnalyzeArtifactStage) Run(_ context.Context, env *stage.Env, m *manifest.Manifest) (*stage.StageResult, error) {
|
||||
if len(env.SelectedAnalyzeArtifacts) != len(s.expected) {
|
||||
return nil, fmt.Errorf("selected artifacts len = %d, want %d", len(env.SelectedAnalyzeArtifacts), len(s.expected))
|
||||
if len(env.SelectedArtifactKeys) != len(s.expected) {
|
||||
return nil, fmt.Errorf("selected artifacts len = %d, want %d", len(env.SelectedArtifactKeys), len(s.expected))
|
||||
}
|
||||
for i := range s.expected {
|
||||
if env.SelectedAnalyzeArtifacts[i] != s.expected[i] {
|
||||
return nil, fmt.Errorf("selected artifacts[%d] = %q, want %q", i, env.SelectedAnalyzeArtifacts[i], s.expected[i])
|
||||
if env.SelectedArtifactKeys[i] != s.expected[i] {
|
||||
return nil, fmt.Errorf("selected artifacts[%d] = %q, want %q", i, env.SelectedArtifactKeys[i], s.expected[i])
|
||||
}
|
||||
}
|
||||
|
||||
@@ -199,7 +199,56 @@ func TestExecuteStagesAnalyzeOutputsPersistAsScriptoriumArtifacts(t *testing.T)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteStagesArchiveFailsWhenRequiredRecapPromotionMissingForSelectedArtifacts(t *testing.T) {
|
||||
func TestNeedsObjectStoreForRunPrepareWithPreviousRequirements(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
previousSessionID string
|
||||
want bool
|
||||
}{
|
||||
{
|
||||
name: "previous session configured",
|
||||
previousSessionID: "2026-05-10",
|
||||
want: true,
|
||||
},
|
||||
{
|
||||
name: "previous session missing",
|
||||
previousSessionID: "",
|
||||
want: false,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
cfg := &config.Config{
|
||||
Pipeline: &config.PipelineConfig{
|
||||
Scriptorium: &config.ScriptoriumConfig{
|
||||
Artifacts: map[string]config.ScriptoriumArtifactConfig{
|
||||
"session_recap": {
|
||||
Enabled: true,
|
||||
Inputs: map[string]config.ScriptoriumInputConfig{
|
||||
"previous_recap": {
|
||||
Source: "narratio.previous_session.artifact.session_recap",
|
||||
Required: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
Session: &config.SessionConfig{
|
||||
PreviousSessionID: tt.previousSessionID,
|
||||
},
|
||||
}
|
||||
|
||||
got := needsObjectStoreForRun(cfg, []stage.Stage{countingStage{name: "prepare", runs: new(int)}})
|
||||
if got != tt.want {
|
||||
t.Fatalf("needsObjectStoreForRun() = %v, want %v", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testing.T) {
|
||||
cfg := testConfig(t)
|
||||
cfg.Pipeline.Storage.S3 = &config.StorageS3Config{
|
||||
Bucket: "my-dnd-archive",
|
||||
@@ -209,7 +258,14 @@ func TestExecuteStagesArchiveFailsWhenRequiredRecapPromotionMissingForSelectedAr
|
||||
Enabled: boolPtr(true),
|
||||
UploadRun: boolPtr(true),
|
||||
PromoteArtifacts: []config.ArchivePromotionRule{
|
||||
{From: "artifacts/session_recap.md", To: "artifacts/session_recap.md", Required: boolPtr(true)},
|
||||
{Source: "narratio.artifact.session_recap", Dest: "artifacts/session_recap.md", Required: boolPtr(true)},
|
||||
},
|
||||
}
|
||||
cfg.Pipeline.Scriptorium = &config.ScriptoriumConfig{
|
||||
Artifacts: map[string]config.ScriptoriumArtifactConfig{
|
||||
"session_recap": {
|
||||
OutputPath: "artifacts/session_recap.md",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
@@ -232,7 +288,7 @@ func TestExecuteStagesArchiveFailsWhenRequiredRecapPromotionMissingForSelectedAr
|
||||
t.Fatalf("Select(archive) error = %v", err)
|
||||
}
|
||||
|
||||
_, err = executeStages(
|
||||
summary, err := executeStages(
|
||||
context.Background(),
|
||||
cfg,
|
||||
[]stage.Stage{
|
||||
@@ -244,11 +300,28 @@ func TestExecuteStagesArchiveFailsWhenRequiredRecapPromotionMissingForSelectedAr
|
||||
Env: &Env{ObjectStore: &storage.FakeBackend{}},
|
||||
},
|
||||
)
|
||||
if err == nil {
|
||||
t.Fatal("expected archive promotion failure, got nil")
|
||||
if err != nil {
|
||||
t.Fatalf("executeStages() error = %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "required promotion source missing") {
|
||||
t.Fatalf("error = %q, want required promotion source missing", err.Error())
|
||||
if len(summary.Executed) != 2 || summary.Executed[0] != "analyze" || summary.Executed[1] != "archive" {
|
||||
t.Fatalf("executed = %#v, want analyze and archive", summary.Executed)
|
||||
}
|
||||
|
||||
loadedManifest, err := store.Load(context.Background(), summary.ManifestPath)
|
||||
if err != nil {
|
||||
t.Fatalf("Load manifest error = %v", err)
|
||||
}
|
||||
meta := loadedManifest.Stages["archive"].Metadata
|
||||
skipped, ok := meta["skipped_unselected_promotions"].([]any)
|
||||
if !ok || len(skipped) != 1 {
|
||||
t.Fatalf("skipped_unselected_promotions = %#v, want one item", meta["skipped_unselected_promotions"])
|
||||
}
|
||||
item, ok := skipped[0].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("skipped item = %#v, want object", skipped[0])
|
||||
}
|
||||
if item["source"] != "narratio.artifact.session_recap" || item["dest"] != "artifacts/session_recap.md" || item["required"] != true {
|
||||
t.Fatalf("skipped item = %#v, want required session_recap promotion", item)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -688,7 +761,7 @@ func TestExecuteStagesRunLocalArtifactsAndCanonicalPromotion(t *testing.T) {
|
||||
filepath.Join(runRoot, "merge", "logs", "seriatim.stdout.log"),
|
||||
filepath.Join(runRoot, "polish", "config", "audita.generated.yml"),
|
||||
filepath.Join(runRoot, "normalize", "logs", "seriatim.normalize.stdout.log"),
|
||||
filepath.Join(runRoot, "trim", "outputs", "transcripts", "trimmed.json"),
|
||||
filepath.Join(runRoot, "trim", "outputs", "transcripts", "final.trimmed.json"),
|
||||
}
|
||||
for _, p := range runLocalChecks {
|
||||
if _, statErr := os.Stat(p); statErr != nil {
|
||||
@@ -698,10 +771,10 @@ func TestExecuteStagesRunLocalArtifactsAndCanonicalPromotion(t *testing.T) {
|
||||
|
||||
canonicalChecks := []string{
|
||||
filepath.Join(paths.TranscriptsRawDir, "alice.json"),
|
||||
filepath.Join(paths.TranscriptsDir, "merged.json"),
|
||||
filepath.Join(paths.TranscriptsDir, "processed.json"),
|
||||
filepath.Join(paths.TranscriptsDir, "normalized.json"),
|
||||
filepath.Join(paths.TranscriptsDir, "trimmed.json"),
|
||||
filepath.Join(paths.TranscriptsDir, "base.json"),
|
||||
filepath.Join(paths.TranscriptsDir, "polished.json"),
|
||||
filepath.Join(paths.TranscriptsDir, "final.json"),
|
||||
filepath.Join(paths.TranscriptsDir, "final.trimmed.json"),
|
||||
}
|
||||
for _, p := range canonicalChecks {
|
||||
if _, statErr := os.Stat(p); statErr != nil {
|
||||
@@ -846,7 +919,7 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
|
||||
if ensureErr != nil {
|
||||
t.Fatalf("EnsureLayout() error = %v", ensureErr)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(paths.TranscriptsDir, "merged.json"), []byte(`{"segments":[]}`), 0o644); err != nil {
|
||||
if err := os.WriteFile(filepath.Join(paths.TranscriptsDir, "base.json"), []byte(`{"segments":[]}`), 0o644); err != nil {
|
||||
t.Fatalf("write merged transcript: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(paths.InputsDir, "glossary.yml"), []byte("terms: []\n"), 0o644); err != nil {
|
||||
@@ -858,7 +931,7 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
|
||||
if ensureErr != nil {
|
||||
t.Fatalf("EnsureLayout() error = %v", ensureErr)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(paths.TranscriptsDir, "processed.json"), []byte(`{"segments":[]}`), 0o644); err != nil {
|
||||
if err := os.WriteFile(filepath.Join(paths.TranscriptsDir, "polished.json"), []byte(`{"segments":[]}`), 0o644); err != nil {
|
||||
t.Fatalf("write processed transcript: %v", err)
|
||||
}
|
||||
cfg.Pipeline.Scriptorium = &config.ScriptoriumConfig{
|
||||
@@ -938,10 +1011,12 @@ func testConfig(t *testing.T) *config.Config {
|
||||
workspace := t.TempDir()
|
||||
cfgDir := t.TempDir()
|
||||
sessionPath := filepath.Join(cfgDir, "session.yml")
|
||||
campaignPath := filepath.Join(cfgDir, "campaign.yml")
|
||||
pipelinePath := filepath.Join(cfgDir, "pipeline.yml")
|
||||
|
||||
mustWriteFile(t, pipelinePath, "workspace:\n root: "+workspace+"\n")
|
||||
mustWriteFile(t, sessionPath, "session_id: 2026-05-03\ncampaign: sample-campaign\n")
|
||||
mustWriteFile(t, campaignPath, "campaign_id: sample-campaign\ninputs:\n speakers_file: ./speakers.yml\n autocorrect_file: ./autocorrect.yml\n glossary_file: ./glossary.yml\n")
|
||||
mustWriteFile(t, sessionPath, "session_id: 2026-05-03\ncampaign: sample-campaign\ninputs:\n audio_dir: ./audio\n")
|
||||
mustWriteFile(t, filepath.Join(cfgDir, "speakers.yml"), "alice: alice.flac\n")
|
||||
mustWriteFile(t, filepath.Join(cfgDir, "autocorrect.yml"), "[]\n")
|
||||
mustWriteFile(t, filepath.Join(cfgDir, "glossary.yml"), "[]\n")
|
||||
@@ -949,8 +1024,27 @@ func testConfig(t *testing.T) *config.Config {
|
||||
|
||||
return &config.Config{
|
||||
Pipeline: &config.PipelineConfig{Workspace: config.WorkspaceConfig{Root: workspace}},
|
||||
Campaign: &config.CampaignConfig{CampaignID: "sample-campaign"},
|
||||
PipelinePath: pipelinePath,
|
||||
CampaignPath: campaignPath,
|
||||
SessionPath: sessionPath,
|
||||
StableInputs: config.ResolvedStableInputs{
|
||||
SpeakersFile: config.ResolvedInputFile{
|
||||
Path: "./speakers.yml",
|
||||
ConfigPath: campaignPath,
|
||||
Source: "campaign_config",
|
||||
},
|
||||
AutocorrectFile: config.ResolvedInputFile{
|
||||
Path: "./autocorrect.yml",
|
||||
ConfigPath: campaignPath,
|
||||
Source: "campaign_config",
|
||||
},
|
||||
GlossaryFile: config.ResolvedInputFile{
|
||||
Path: "./glossary.yml",
|
||||
ConfigPath: campaignPath,
|
||||
Source: "campaign_config",
|
||||
},
|
||||
},
|
||||
Session: &config.SessionConfig{
|
||||
SessionID: "2026-05-03",
|
||||
Campaign: "sample-campaign",
|
||||
@@ -967,15 +1061,20 @@ func testConfig(t *testing.T) *config.Config {
|
||||
func TestBuildDefaultRunnersWithOmittedToolSections(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
pipelinePath := filepath.Join(dir, "pipeline.yml")
|
||||
campaignPath := filepath.Join(dir, "campaign.yml")
|
||||
sessionPath := filepath.Join(dir, "session.yml")
|
||||
pipelineYAML := `workspace:
|
||||
root: ` + t.TempDir() + `
|
||||
whisperx:
|
||||
transcribe_url: https://example.com/transcribe
|
||||
analyzer:
|
||||
timeout: 20m
|
||||
notification:
|
||||
timeout: 10s
|
||||
`
|
||||
campaignYAML := `campaign_id: sample-campaign
|
||||
inputs:
|
||||
speakers_file: ./speakers.yml
|
||||
autocorrect_file: ./autocorrect.yml
|
||||
glossary_file: ./glossary.yml
|
||||
`
|
||||
sessionYAML := `session_id: 2026-05-03
|
||||
campaign: sample-campaign
|
||||
@@ -986,6 +1085,7 @@ inputs:
|
||||
glossary_file: ./glossary.yml
|
||||
`
|
||||
mustWriteFile(t, pipelinePath, pipelineYAML)
|
||||
mustWriteFile(t, campaignPath, campaignYAML)
|
||||
mustWriteFile(t, sessionPath, sessionYAML)
|
||||
|
||||
cfg, err := config.Load(pipelinePath, sessionPath)
|
||||
|
||||
43
internal/app/session_args.go
Normal file
43
internal/app/session_args.go
Normal file
@@ -0,0 +1,43 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func isCLIFlagToken(arg string) bool {
|
||||
return strings.HasPrefix(arg, "-") && arg != "-"
|
||||
}
|
||||
|
||||
func pullLeadingSessionID(args []string) (string, []string) {
|
||||
if len(args) == 0 || isCLIFlagToken(args[0]) {
|
||||
return "", args
|
||||
}
|
||||
rest := append([]string(nil), args[1:]...)
|
||||
return strings.TrimSpace(args[0]), rest
|
||||
}
|
||||
|
||||
func applyPositionalSessionID(command, positional string, sessionID *string) error {
|
||||
positional = strings.TrimSpace(positional)
|
||||
if positional == "" {
|
||||
return nil
|
||||
}
|
||||
existing := strings.TrimSpace(*sessionID)
|
||||
if existing != "" && existing != positional {
|
||||
return fmt.Errorf("%s: positional session id %q does not match expected session id %q", command, positional, existing)
|
||||
}
|
||||
*sessionID = positional
|
||||
return nil
|
||||
}
|
||||
|
||||
func applyParsedSessionIDArg(command string, fs *flag.FlagSet, sessionID *string) error {
|
||||
switch fs.NArg() {
|
||||
case 0:
|
||||
return nil
|
||||
case 1:
|
||||
return applyPositionalSessionID(command, fs.Arg(0), sessionID)
|
||||
default:
|
||||
return fmt.Errorf("%s: unexpected positional arguments", command)
|
||||
}
|
||||
}
|
||||
@@ -4,16 +4,18 @@ import (
|
||||
"bytes"
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
func TestPlanUsesDiscoveredSessionTemplateWithSessionID(t *testing.T) {
|
||||
func TestPlanRejectsDiscoveredSessionTemplate(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
sessionTemplate := `session_id: "{{ session_id }}"
|
||||
previous_session_id: "{{ previous_session_id }}"
|
||||
campaign: sample-campaign
|
||||
inputs:
|
||||
audio_dir: ./audio
|
||||
@@ -25,31 +27,34 @@ inputs:
|
||||
t.Fatalf("write session template: %v", err)
|
||||
}
|
||||
|
||||
cwd := filepath.Dir(sessionPath)
|
||||
originalWD, err := os.Getwd()
|
||||
if err != nil {
|
||||
t.Fatalf("Getwd(): %v", err)
|
||||
}
|
||||
if err := os.Chdir(cwd); err != nil {
|
||||
t.Fatalf("Chdir(%q): %v", cwd, err)
|
||||
}
|
||||
t.Cleanup(func() { _ = os.Chdir(originalWD) })
|
||||
origSessionDefaults := append([]string(nil), config.DefaultSessionConfigSearchPaths...)
|
||||
config.DefaultSessionConfigSearchPaths = []string{sessionPath}
|
||||
t.Cleanup(func() { config.DefaultSessionConfigSearchPaths = origSessionDefaults })
|
||||
|
||||
var out bytes.Buffer
|
||||
if err := Plan(context.Background(), []string{"--config", pipelinePath, "--session-id", "2026-04-04"}, &out); err != nil {
|
||||
t.Fatalf("Plan() error = %v", err)
|
||||
err := Plan(context.Background(), []string{
|
||||
"2026-04-04",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--previous-session-id", "2026-03-28",
|
||||
}, &out)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !strings.Contains(out.String(), "narratio plan: workdir prepared") {
|
||||
t.Fatalf("output = %q, want plan output", out.String())
|
||||
if !strings.Contains(err.Error(), "session.yml must be concrete") {
|
||||
t.Fatalf("error = %q, want concrete session guidance", err.Error())
|
||||
}
|
||||
if !strings.Contains(err.Error(), "run narratio session init") {
|
||||
t.Fatalf("error = %q, want session init guidance", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func TestPlanFailsWhenSessionIDMismatchesConcreteSession(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var out bytes.Buffer
|
||||
err := Plan(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "--session-id", "2026-04-04"}, &out)
|
||||
err := Plan(context.Background(), []string{"2026-04-04", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
@@ -58,12 +63,45 @@ func TestPlanFailsWhenSessionIDMismatchesConcreteSession(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunStageAcceptsSessionIDFlagAndParsesStageName(t *testing.T) {
|
||||
func TestPlanFailsWhenPreviousSessionIDMismatchesConcreteSession(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
sessionYAML := `session_id: 2026-05-03
|
||||
previous_session_id: 2026-04-26
|
||||
campaign: sample-campaign
|
||||
inputs:
|
||||
audio_dir: ./audio
|
||||
speakers_file: ./speakers.yml
|
||||
autocorrect_file: ./autocorrect.yml
|
||||
glossary_file: ./glossary.yml
|
||||
`
|
||||
if err := os.WriteFile(sessionPath, []byte(sessionYAML), 0o644); err != nil {
|
||||
t.Fatalf("write session.yml: %v", err)
|
||||
}
|
||||
|
||||
var out bytes.Buffer
|
||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "--session-id", "2026-05-03", "prepare"}, &out)
|
||||
err := Plan(context.Background(), []string{
|
||||
"2026-05-03",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--previous-session-id", "2026-04-25",
|
||||
}, &out)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "previous_session_id mismatch") {
|
||||
t.Fatalf("error = %q, want mismatch context", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunStageAcceptsPositionalSessionIDAndParsesStageName(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var out bytes.Buffer
|
||||
err := RunStage(context.Background(), []string{"prepare", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("RunStage() error = %v", err)
|
||||
}
|
||||
@@ -73,7 +111,7 @@ func TestRunStageAcceptsSessionIDFlagAndParsesStageName(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestResolveSessionConfigPathErrorIncludesSearchedPaths(t *testing.T) {
|
||||
_, err := resolveSessionConfigPathWithCandidates("", []string{"./session.yml", "/usr/local/etc/narratio/session.yml", "/etc/narratio/session.yml"})
|
||||
_, err := resolveSessionConfigPathWithCandidates("", []string{"/usr/local/etc/narratio/session.yml", "/etc/narratio/session.yml"})
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
|
||||
@@ -19,6 +19,22 @@ func resolveSessionConfigPathWithCandidates(flagValue string, candidates []strin
|
||||
return explicit, nil
|
||||
}
|
||||
|
||||
resolved, err := discoverSessionConfigPathWithCandidates(candidates)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if resolved.Path != "" {
|
||||
return resolved.Path, nil
|
||||
}
|
||||
return "", missingSessionConfigError(resolved.Searched, "")
|
||||
}
|
||||
|
||||
type sessionConfigDiscovery struct {
|
||||
Path string
|
||||
Searched []string
|
||||
}
|
||||
|
||||
func discoverSessionConfigPathWithCandidates(candidates []string) (sessionConfigDiscovery, error) {
|
||||
ordered := make([]string, 0, len(candidates))
|
||||
for _, raw := range candidates {
|
||||
path := strings.TrimSpace(raw)
|
||||
@@ -31,19 +47,32 @@ func resolveSessionConfigPathWithCandidates(flagValue string, candidates []strin
|
||||
if info.IsDir() {
|
||||
continue
|
||||
}
|
||||
return filepath.Clean(path), nil
|
||||
return sessionConfigDiscovery{Path: filepath.Clean(path), Searched: ordered}, nil
|
||||
}
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
continue
|
||||
}
|
||||
return "", fmt.Errorf("check default session config %q: %w", path, err)
|
||||
return sessionConfigDiscovery{}, fmt.Errorf("check default session config %q: %w", path, err)
|
||||
}
|
||||
|
||||
return sessionConfigDiscovery{Searched: ordered}, nil
|
||||
}
|
||||
|
||||
func missingSessionConfigError(searched []string, remoteDetail string) error {
|
||||
ordered := append([]string(nil), searched...)
|
||||
if len(ordered) == 0 {
|
||||
return "", fmt.Errorf("no session config path provided and no default locations configured")
|
||||
if strings.TrimSpace(remoteDetail) != "" {
|
||||
return fmt.Errorf("no session config path provided and no default locations configured; %s", remoteDetail)
|
||||
}
|
||||
return fmt.Errorf("no session config path provided and no default locations configured")
|
||||
}
|
||||
return "", fmt.Errorf(
|
||||
"no session config path provided and no default session config found; searched: %s; pass --session to use an explicit path",
|
||||
msg := fmt.Sprintf(
|
||||
"no session config path provided and no default session config found; searched: %s",
|
||||
strings.Join(ordered, ", "),
|
||||
)
|
||||
if strings.TrimSpace(remoteDetail) != "" {
|
||||
msg += "; " + strings.TrimSpace(remoteDetail)
|
||||
}
|
||||
msg += "; pass --session to use an explicit path"
|
||||
return fmt.Errorf("%s", msg)
|
||||
}
|
||||
|
||||
353
internal/app/session_oriented_cli_test.go
Normal file
353
internal/app/session_oriented_cli_test.go
Normal file
@@ -0,0 +1,353 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/stage"
|
||||
)
|
||||
|
||||
func TestExecuteRunAcceptsPositionalSessionID(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var capturedSessionID string
|
||||
origExecuteStagesFn := executeStagesFn
|
||||
t.Cleanup(func() { executeStagesFn = origExecuteStagesFn })
|
||||
executeStagesFn = func(_ context.Context, cfg *config.Config, _ []stage.Stage, _ RunOptions) (*RunSummary, error) {
|
||||
capturedSessionID = cfg.Session.SessionID
|
||||
return &RunSummary{
|
||||
SessionID: cfg.Session.SessionID,
|
||||
ManifestPath: filepath.Join(workspaceRoot, "manifest.json"),
|
||||
Executed: []string{"prepare"},
|
||||
}, nil
|
||||
}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"run",
|
||||
"2026-05-03",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "narratio run: session 2026-05-03") {
|
||||
t.Fatalf("stdout = %q, want run summary", stdout.String())
|
||||
}
|
||||
if capturedSessionID != "2026-05-03" {
|
||||
t.Fatalf("captured session = %q, want positional session id", capturedSessionID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecutePositionalSessionIDMismatchFails(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"run",
|
||||
"2026-05-04",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "session_id mismatch") {
|
||||
t.Fatalf("stderr = %q, want session mismatch", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionIDFlagFails(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"session", "status", "2026-05-03", "--session-id", "2026-05-04"}, &stdout, &stderr)
|
||||
if code == 0 {
|
||||
t.Fatal("exit code = 0, want non-zero")
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "flag provided but not defined: -session-id") {
|
||||
t.Fatalf("stderr = %q, want invalid --session-id flag", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteRemoteSessionFallbackUsesPositionalSessionID(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||
remoteKey := seedRemoteSessionConfig(t, fake, "2026-06-07", `session_id: 2026-06-07
|
||||
inputs:
|
||||
audio_s3:
|
||||
prefix: audio/
|
||||
`)
|
||||
origExecuteStagesFn := executeStagesFn
|
||||
t.Cleanup(func() { executeStagesFn = origExecuteStagesFn })
|
||||
executeStagesFn = func(_ context.Context, cfg *config.Config, _ []stage.Stage, _ RunOptions) (*RunSummary, error) {
|
||||
return &RunSummary{
|
||||
SessionID: cfg.Session.SessionID,
|
||||
ManifestPath: filepath.Join(workspaceRoot, "manifest.json"),
|
||||
Executed: []string{"prepare"},
|
||||
}, nil
|
||||
}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"run",
|
||||
"2026-06-07",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
downloaded := false
|
||||
for _, call := range fake.Downloads {
|
||||
if call.Key == remoteKey {
|
||||
downloaded = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !downloaded {
|
||||
t.Fatalf("remote session %q was not downloaded; downloads=%v", remoteKey, fake.Downloads)
|
||||
}
|
||||
if storeInitCalls == 0 {
|
||||
t.Fatal("object store was not initialized")
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteWorkflowCommandsAcceptPositionalSessionID(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
args []string
|
||||
wantStage string
|
||||
wantForce bool
|
||||
}{
|
||||
{
|
||||
name: "resume",
|
||||
args: []string{"resume", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||
wantStage: "prepare",
|
||||
wantForce: false,
|
||||
},
|
||||
{
|
||||
name: "analyze",
|
||||
args: []string{"analyze", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||
wantStage: "analyze",
|
||||
wantForce: true,
|
||||
},
|
||||
{
|
||||
name: "publish",
|
||||
args: []string{"publish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||
wantStage: "archive",
|
||||
wantForce: true,
|
||||
},
|
||||
{
|
||||
name: "run-stage",
|
||||
args: []string{"run-stage", "archive", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||
wantStage: "archive",
|
||||
wantForce: false,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var capturedStages []string
|
||||
var capturedForce bool
|
||||
var capturedArtifacts []string
|
||||
origExecuteStagesFn := executeStagesFn
|
||||
t.Cleanup(func() { executeStagesFn = origExecuteStagesFn })
|
||||
executeStagesFn = func(_ context.Context, _ *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||
for _, s := range stages {
|
||||
capturedStages = append(capturedStages, s.Name())
|
||||
}
|
||||
capturedForce = opts.Force
|
||||
capturedArtifacts = append([]string(nil), opts.SelectedArtifacts...)
|
||||
return &RunSummary{
|
||||
SessionID: "2026-05-03",
|
||||
ManifestPath: filepath.Join(workspaceRoot, "manifest.json"),
|
||||
Executed: []string{tt.wantStage},
|
||||
}, nil
|
||||
}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(tt.args, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
if len(capturedStages) == 0 || capturedStages[0] != tt.wantStage {
|
||||
t.Fatalf("captured stages = %#v, want first %q", capturedStages, tt.wantStage)
|
||||
}
|
||||
if capturedForce != tt.wantForce {
|
||||
t.Fatalf("captured force = %t, want %t", capturedForce, tt.wantForce)
|
||||
}
|
||||
if tt.name == "analyze" || tt.name == "publish" || tt.name == "run-stage" {
|
||||
if strings.Join(capturedArtifacts, ",") != "session_recap" {
|
||||
t.Fatalf("captured artifacts = %#v, want [session_recap]", capturedArtifacts)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionSubcommandsAcceptPositionalSessionID(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
|
||||
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
||||
fake.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||
fake.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, "2026-05-03", "sample-campaign")})
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
args []string
|
||||
want string
|
||||
}{
|
||||
{
|
||||
name: "validate",
|
||||
args: []string{"session", "validate", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||
want: "OK config",
|
||||
},
|
||||
{
|
||||
name: "status",
|
||||
args: []string{"session", "status", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||
want: "Session: 2026-05-03",
|
||||
},
|
||||
{
|
||||
name: "plan",
|
||||
args: []string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||
want: "narratio session plan: workdir prepared",
|
||||
},
|
||||
{
|
||||
name: "artifacts",
|
||||
args: []string{"session", "artifacts", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||
want: "Built-in:",
|
||||
},
|
||||
{
|
||||
name: "locks",
|
||||
args: []string{"session", "locks", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||
want: "Archive locks:",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute(tt.args, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stdout=%q stderr=%q", code, stdout.String(), stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), tt.want) {
|
||||
t.Fatalf("stdout = %q, want %q", stdout.String(), tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionInitAcceptsPositionalSessionID(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||
outputPath := filepath.Join(t.TempDir(), "session.yml")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "init", "2026-06-07",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--output", outputPath,
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
data, err := os.ReadFile(outputPath)
|
||||
if err != nil {
|
||||
t.Fatalf("read generated session: %v", err)
|
||||
}
|
||||
if !strings.Contains(string(data), `session_id: "2026-06-07"`) {
|
||||
t.Fatalf("generated session = %q, want positional session id", string(data))
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteSessionLocksMutationAcceptsPositionalSessionID(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
fake := &storage.FakeBackend{}
|
||||
var storeInitCalls int
|
||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{
|
||||
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
"--reason", "review",
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("locks add exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
key := artifacts.S3SessionLocksKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"))
|
||||
if !strings.Contains(string(fake.Objects[key].Data), "reason: review") {
|
||||
t.Fatalf("lock store data = %q, want reason", string(fake.Objects[key].Data))
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
code = Execute([]string{
|
||||
"session", "locks", "remove", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||
"--config", pipelinePath,
|
||||
"--campaign-file", campaignPath,
|
||||
"--session", sessionPath,
|
||||
}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("locks remove exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
store, err := config.LoadArchiveLockStoreBytes("locks.yml", fake.Objects[key].Data, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadArchiveLockStoreBytes() error = %v", err)
|
||||
}
|
||||
if len(store.Locks) != 0 {
|
||||
t.Fatalf("locks after remove = %#v, want empty", store.Locks)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecuteCleanAcceptsPositionalSessionID(t *testing.T) {
|
||||
workspaceRoot := t.TempDir()
|
||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||
workDir := artifacts.SessionWorkDirForCampaign(workspaceRoot, "sample-campaign", "2026-05-03")
|
||||
spoolDir := artifacts.SessionSpoolDir(filepath.Join(workspaceRoot, "spool"), "sample-campaign", "2026-05-03")
|
||||
mustWriteTestFile(t, filepath.Join(workDir, "manifest.json"), "{}")
|
||||
mustWriteTestFile(t, filepath.Join(spoolDir, "run-1", "audio", "alice.flac"), "audio")
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||
if code != 0 {
|
||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||
}
|
||||
cleanAssertMissing(t, workDir)
|
||||
cleanAssertMissing(t, spoolDir)
|
||||
}
|
||||
@@ -1,67 +0,0 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"sort"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
)
|
||||
|
||||
// Status reads and prints stage statuses from an existing manifest.
|
||||
func Status(ctx context.Context, args []string, out io.Writer) error {
|
||||
fs := flag.NewFlagSet("status", flag.ContinueOnError)
|
||||
fs.SetOutput(io.Discard)
|
||||
|
||||
var manifestPath string
|
||||
fs.StringVar(&manifestPath, "manifest", "", "path to manifest.json")
|
||||
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return fmt.Errorf("status: invalid flags: %w", err)
|
||||
}
|
||||
if fs.NArg() != 0 {
|
||||
return fmt.Errorf("status: unexpected positional arguments")
|
||||
}
|
||||
if manifestPath == "" {
|
||||
return fmt.Errorf("status: --manifest is required")
|
||||
}
|
||||
|
||||
store := &manifest.LocalStore{}
|
||||
m, err := store.Load(ctx, manifestPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("status: %w", err)
|
||||
}
|
||||
|
||||
if _, err := fmt.Fprintf(out, "session_id: %s\n", m.SessionID); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := fmt.Fprintf(out, "updated_at: %s\n", m.UpdatedAt.UTC().Format("2006-01-02T15:04:05Z07:00")); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if len(m.Stages) == 0 {
|
||||
_, err := fmt.Fprintln(out, "stages: no stages recorded")
|
||||
return err
|
||||
}
|
||||
|
||||
if _, err := fmt.Fprintln(out, "stages:"); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
names := make([]string, 0, len(m.Stages))
|
||||
for name := range m.Stages {
|
||||
names = append(names, name)
|
||||
}
|
||||
sort.Strings(names)
|
||||
|
||||
for _, name := range names {
|
||||
status := m.Stages[name].Status
|
||||
if _, err := fmt.Fprintf(out, "- %s: %s\n", name, status); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -3,71 +3,28 @@ package app
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
)
|
||||
|
||||
func TestStatusCommandReadsManifest(t *testing.T) {
|
||||
manifestPath := writeManifestForStatus(t)
|
||||
|
||||
var out bytes.Buffer
|
||||
err := Status(context.Background(), []string{"--manifest", manifestPath}, &out)
|
||||
if err != nil {
|
||||
t.Fatalf("Status() error = %v", err)
|
||||
}
|
||||
|
||||
s := out.String()
|
||||
if !strings.Contains(s, "session_id: 2026-05-03") {
|
||||
t.Fatalf("output = %q, want session_id", s)
|
||||
}
|
||||
if !strings.Contains(s, "- merge: succeeded") {
|
||||
t.Fatalf("output = %q, want stage status", s)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStatusCommandMissingManifestFlag(t *testing.T) {
|
||||
func TestStatusCommandRequiresSessionID(t *testing.T) {
|
||||
var out bytes.Buffer
|
||||
err := Status(context.Background(), nil, &out)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "--manifest is required") {
|
||||
t.Fatalf("error = %q, want missing manifest flag", err.Error())
|
||||
if !strings.Contains(err.Error(), "status: session_id is required") {
|
||||
t.Fatalf("error = %q, want missing session_id error", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func TestStatusCommandBadManifest(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "manifest.json")
|
||||
if err := os.WriteFile(path, []byte("{not-json"), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
|
||||
func TestStatusCommandRejectsManifestFlag(t *testing.T) {
|
||||
var out bytes.Buffer
|
||||
err := Status(context.Background(), []string{"--manifest", path}, &out)
|
||||
err := Status(context.Background(), []string{"2026-05-03", "--manifest", "manifest.json"}, &out)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "decode manifest") {
|
||||
t.Fatalf("error = %q, want decode error", err.Error())
|
||||
if !strings.Contains(err.Error(), "status: invalid flags: flag provided but not defined: -manifest") {
|
||||
t.Fatalf("error = %q, want invalid manifest flag", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func writeManifestForStatus(t *testing.T) string {
|
||||
t.Helper()
|
||||
|
||||
store := &manifest.LocalStore{}
|
||||
m := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
||||
m.MarkStageSucceeded("merge", time.Date(2026, 5, 3, 10, 5, 0, 0, time.UTC), nil)
|
||||
|
||||
path := filepath.Join(t.TempDir(), "manifest.json")
|
||||
if err := store.Save(context.Background(), path, m); err != nil {
|
||||
t.Fatalf("Save() error = %v", err)
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
84
internal/artifactmodel/transcripts.go
Normal file
84
internal/artifactmodel/transcripts.go
Normal file
@@ -0,0 +1,84 @@
|
||||
package artifactmodel
|
||||
|
||||
import "strings"
|
||||
|
||||
const (
|
||||
SourceTranscriptBase = "narratio.transcript.base"
|
||||
SourceTranscriptPolished = "narratio.transcript.polished"
|
||||
SourceTranscriptFinal = "narratio.transcript.final"
|
||||
SourceTranscriptFinalTrimmed = "narratio.transcript.final_trimmed"
|
||||
)
|
||||
|
||||
const (
|
||||
TranscriptPathBase = "transcripts/base.json"
|
||||
TranscriptPathPolished = "transcripts/polished.json"
|
||||
TranscriptPathFinal = "transcripts/final.json"
|
||||
TranscriptPathFinalTrimmed = "transcripts/final.trimmed.json"
|
||||
)
|
||||
|
||||
const (
|
||||
TranscriptOutputKindBase = "transcript_base"
|
||||
TranscriptOutputKindPolished = "transcript_polished"
|
||||
TranscriptOutputKindFinal = "transcript_final"
|
||||
TranscriptOutputKindFinalTrimmed = "transcript_final_trimmed"
|
||||
)
|
||||
|
||||
// TranscriptArtifactSpec describes one built-in transcript artifact mapping.
|
||||
type TranscriptArtifactSpec struct {
|
||||
SourceID string
|
||||
CanonicalRelPath string
|
||||
ProducerStage string
|
||||
OutputKind string
|
||||
}
|
||||
|
||||
var runtimeTranscriptArtifacts = []TranscriptArtifactSpec{
|
||||
{
|
||||
SourceID: SourceTranscriptBase,
|
||||
CanonicalRelPath: TranscriptPathBase,
|
||||
ProducerStage: "merge",
|
||||
OutputKind: TranscriptOutputKindBase,
|
||||
},
|
||||
{
|
||||
SourceID: SourceTranscriptPolished,
|
||||
CanonicalRelPath: TranscriptPathPolished,
|
||||
ProducerStage: "polish",
|
||||
OutputKind: TranscriptOutputKindPolished,
|
||||
},
|
||||
{
|
||||
SourceID: SourceTranscriptFinal,
|
||||
CanonicalRelPath: TranscriptPathFinal,
|
||||
ProducerStage: "normalize",
|
||||
OutputKind: TranscriptOutputKindFinal,
|
||||
},
|
||||
{
|
||||
SourceID: SourceTranscriptFinalTrimmed,
|
||||
CanonicalRelPath: TranscriptPathFinalTrimmed,
|
||||
ProducerStage: "trim",
|
||||
OutputKind: TranscriptOutputKindFinalTrimmed,
|
||||
},
|
||||
}
|
||||
|
||||
// RuntimeTranscriptArtifacts returns transcript mappings in pipeline order.
|
||||
func RuntimeTranscriptArtifacts() []TranscriptArtifactSpec {
|
||||
return cloneTranscriptSpecs(runtimeTranscriptArtifacts)
|
||||
}
|
||||
|
||||
// LookupRuntimeTranscriptArtifact returns runtime transcript metadata by source ID.
|
||||
func LookupRuntimeTranscriptArtifact(sourceID string) (TranscriptArtifactSpec, bool) {
|
||||
trimmed := strings.TrimSpace(sourceID)
|
||||
for _, spec := range runtimeTranscriptArtifacts {
|
||||
if spec.SourceID == trimmed {
|
||||
return spec, true
|
||||
}
|
||||
}
|
||||
return TranscriptArtifactSpec{}, false
|
||||
}
|
||||
|
||||
func cloneTranscriptSpecs(specs []TranscriptArtifactSpec) []TranscriptArtifactSpec {
|
||||
if len(specs) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]TranscriptArtifactSpec, len(specs))
|
||||
copy(out, specs)
|
||||
return out
|
||||
}
|
||||
77
internal/artifacts/archive_identity.go
Normal file
77
internal/artifacts/archive_identity.go
Normal file
@@ -0,0 +1,77 @@
|
||||
package artifacts
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
)
|
||||
|
||||
// ResolveArchiveBucket resolves archive bucket identity with manifest-first precedence.
|
||||
func ResolveArchiveBucket(cfg *config.Config, m *manifest.Manifest) string {
|
||||
if m != nil && strings.TrimSpace(m.S3Bucket) != "" {
|
||||
return strings.TrimSpace(m.S3Bucket)
|
||||
}
|
||||
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Storage.S3 == nil {
|
||||
return ""
|
||||
}
|
||||
return strings.TrimSpace(cfg.Pipeline.Storage.S3.Bucket)
|
||||
}
|
||||
|
||||
// ResolveArchiveSessionPrefix resolves archive session prefix with manifest-first precedence.
|
||||
func ResolveArchiveSessionPrefix(cfg *config.Config, m *manifest.Manifest) (string, error) {
|
||||
if m != nil && strings.TrimSpace(m.S3SessionPrefix) != "" {
|
||||
return strings.TrimSpace(m.S3SessionPrefix), nil
|
||||
}
|
||||
if cfg == nil || cfg.Session == nil || cfg.Pipeline == nil {
|
||||
return "", fmt.Errorf("resolved config is required")
|
||||
}
|
||||
|
||||
sessionID := strings.TrimSpace(cfg.Session.SessionID)
|
||||
if sessionID == "" && m != nil {
|
||||
sessionID = strings.TrimSpace(m.SessionID)
|
||||
}
|
||||
campaign := strings.TrimSpace(cfg.Session.Campaign)
|
||||
if campaign == "" && m != nil {
|
||||
campaign = strings.TrimSpace(m.Campaign)
|
||||
}
|
||||
if cfg.Pipeline.Storage.S3 == nil {
|
||||
return "", fmt.Errorf("pipeline.storage.s3 configuration is required")
|
||||
}
|
||||
|
||||
sessionPrefix := S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, campaign, sessionID)
|
||||
if strings.TrimSpace(sessionPrefix) == "" {
|
||||
return "", fmt.Errorf("session prefix is required")
|
||||
}
|
||||
return sessionPrefix, nil
|
||||
}
|
||||
|
||||
// ResolveArchiveRunPrefix resolves archive run prefix with manifest-first precedence.
|
||||
func ResolveArchiveRunPrefix(cfg *config.Config, m *manifest.Manifest) (string, error) {
|
||||
if m != nil {
|
||||
runPrefix := strings.TrimSpace(m.S3RunPrefix)
|
||||
if runPrefix != "" {
|
||||
return runPrefix, nil
|
||||
}
|
||||
}
|
||||
|
||||
sessionPrefix, err := ResolveArchiveSessionPrefix(cfg, m)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
runID := ""
|
||||
if m != nil {
|
||||
runID = strings.TrimSpace(m.RunID)
|
||||
}
|
||||
if runID == "" {
|
||||
return "", fmt.Errorf("run id is required")
|
||||
}
|
||||
return S3RunPrefix(sessionPrefix, runID), nil
|
||||
}
|
||||
|
||||
// ResolveArchiveCurrentStateKeys returns current pointer keys for a session prefix.
|
||||
func ResolveArchiveCurrentStateKeys(sessionPrefix string) (manifestKey, runIDKey string) {
|
||||
return S3CurrentManifestKey(sessionPrefix), S3CurrentRunPointerKey(sessionPrefix)
|
||||
}
|
||||
130
internal/artifacts/archive_identity_test.go
Normal file
130
internal/artifacts/archive_identity_test.go
Normal file
@@ -0,0 +1,130 @@
|
||||
package artifacts
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
)
|
||||
|
||||
func TestResolveArchiveBucketPrefersManifestThenConfig(t *testing.T) {
|
||||
cfg := &config.Config{
|
||||
Pipeline: &config.PipelineConfig{
|
||||
Storage: config.StorageConfig{
|
||||
S3: &config.StorageS3Config{Bucket: "cfg-bucket"},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
if got := ResolveArchiveBucket(cfg, &manifest.Manifest{S3Bucket: "manifest-bucket"}); got != "manifest-bucket" {
|
||||
t.Fatalf("bucket = %q, want manifest-bucket", got)
|
||||
}
|
||||
if got := ResolveArchiveBucket(cfg, &manifest.Manifest{}); got != "cfg-bucket" {
|
||||
t.Fatalf("bucket = %q, want cfg-bucket", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveArchiveSessionPrefixPrefersManifestThenConfig(t *testing.T) {
|
||||
cfg := &config.Config{
|
||||
Pipeline: &config.PipelineConfig{
|
||||
Storage: config.StorageConfig{
|
||||
S3: &config.StorageS3Config{RootPrefix: "dnd"},
|
||||
},
|
||||
},
|
||||
Session: &config.SessionConfig{
|
||||
SessionID: "2026-04-19",
|
||||
Campaign: "forsaken",
|
||||
},
|
||||
}
|
||||
m := &manifest.Manifest{S3SessionPrefix: "manifest/session/prefix/"}
|
||||
|
||||
got, err := ResolveArchiveSessionPrefix(cfg, m)
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveArchiveSessionPrefix() error = %v", err)
|
||||
}
|
||||
if got != "manifest/session/prefix/" {
|
||||
t.Fatalf("session prefix = %q, want manifest/session/prefix/", got)
|
||||
}
|
||||
|
||||
got, err = ResolveArchiveSessionPrefix(cfg, &manifest.Manifest{})
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveArchiveSessionPrefix() error = %v", err)
|
||||
}
|
||||
want := "dnd/campaigns/forsaken/sessions/2026-04-19/"
|
||||
if got != want {
|
||||
t.Fatalf("session prefix = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveArchiveRunPrefixPrefersManifestThenDerived(t *testing.T) {
|
||||
cfg := &config.Config{
|
||||
Pipeline: &config.PipelineConfig{
|
||||
Storage: config.StorageConfig{
|
||||
S3: &config.StorageS3Config{RootPrefix: "dnd"},
|
||||
},
|
||||
},
|
||||
Session: &config.SessionConfig{
|
||||
SessionID: "2026-04-19",
|
||||
Campaign: "forsaken",
|
||||
},
|
||||
}
|
||||
|
||||
m := &manifest.Manifest{
|
||||
RunID: "20260516T010203Z-1a2b3c4d",
|
||||
S3RunPrefix: "manifest/run/prefix/",
|
||||
}
|
||||
got, err := ResolveArchiveRunPrefix(cfg, m)
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveArchiveRunPrefix() error = %v", err)
|
||||
}
|
||||
if got != "manifest/run/prefix/" {
|
||||
t.Fatalf("run prefix = %q, want manifest/run/prefix/", got)
|
||||
}
|
||||
|
||||
m = &manifest.Manifest{
|
||||
RunID: "20260516T010203Z-1a2b3c4d",
|
||||
}
|
||||
got, err = ResolveArchiveRunPrefix(cfg, m)
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveArchiveRunPrefix() error = %v", err)
|
||||
}
|
||||
want := "dnd/campaigns/forsaken/sessions/2026-04-19/runs/20260516T010203Z-1a2b3c4d/"
|
||||
if got != want {
|
||||
t.Fatalf("run prefix = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveArchiveIdentityErrorsAreDeterministic(t *testing.T) {
|
||||
cfgNoS3 := &config.Config{
|
||||
Pipeline: &config.PipelineConfig{},
|
||||
Session: &config.SessionConfig{SessionID: "2026-04-19", Campaign: "forsaken"},
|
||||
}
|
||||
_, err := ResolveArchiveSessionPrefix(cfgNoS3, &manifest.Manifest{})
|
||||
if err == nil || !strings.Contains(err.Error(), "pipeline.storage.s3 configuration is required") {
|
||||
t.Fatalf("error = %v, want missing storage.s3", err)
|
||||
}
|
||||
|
||||
cfg := &config.Config{
|
||||
Pipeline: &config.PipelineConfig{
|
||||
Storage: config.StorageConfig{
|
||||
S3: &config.StorageS3Config{RootPrefix: "dnd"},
|
||||
},
|
||||
},
|
||||
Session: &config.SessionConfig{SessionID: "2026-04-19", Campaign: "forsaken"},
|
||||
}
|
||||
_, err = ResolveArchiveRunPrefix(cfg, &manifest.Manifest{})
|
||||
if err == nil || !strings.Contains(err.Error(), "run id is required") {
|
||||
t.Fatalf("error = %v, want missing run id", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveArchiveCurrentStateKeys(t *testing.T) {
|
||||
manifestKey, runIDKey := ResolveArchiveCurrentStateKeys("dnd/campaigns/forsaken/sessions/2026-04-19/")
|
||||
if manifestKey != "dnd/campaigns/forsaken/sessions/2026-04-19/current/manifest.json" {
|
||||
t.Fatalf("manifest key = %q", manifestKey)
|
||||
}
|
||||
if runIDKey != "dnd/campaigns/forsaken/sessions/2026-04-19/current/run_id.txt" {
|
||||
t.Fatalf("run id key = %q", runIDKey)
|
||||
}
|
||||
}
|
||||
@@ -9,20 +9,39 @@ import (
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifactmodel"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
)
|
||||
|
||||
const (
|
||||
ArtifactTranscriptMerged = "narratio.transcript.merged"
|
||||
ArtifactTranscriptPolished = "narratio.transcript.polished"
|
||||
ArtifactTranscriptFull = "narratio.transcript.full"
|
||||
ArtifactTranscriptTrimmed = "narratio.transcript.trimmed"
|
||||
ArtifactBoundsSession = "narratio.bounds.session"
|
||||
ArtifactTranscriptBase = artifactmodel.SourceTranscriptBase
|
||||
ArtifactTranscriptPolished = artifactmodel.SourceTranscriptPolished
|
||||
ArtifactTranscriptFinal = artifactmodel.SourceTranscriptFinal
|
||||
ArtifactTranscriptFinalTrimmed = artifactmodel.SourceTranscriptFinalTrimmed
|
||||
ArtifactBoundsSession = "narratio.bounds.session"
|
||||
|
||||
ArtifactProvenancePreviousCacheManifestInput = "manifest.inputs.previous_cache"
|
||||
ArtifactProvenancePreviousCacheFilesystem = "current_session.previous_cache"
|
||||
)
|
||||
|
||||
const (
|
||||
TranscriptPathBase = artifactmodel.TranscriptPathBase
|
||||
TranscriptPathPolished = artifactmodel.TranscriptPathPolished
|
||||
TranscriptPathFinal = artifactmodel.TranscriptPathFinal
|
||||
TranscriptPathFinalTrimmed = artifactmodel.TranscriptPathFinalTrimmed
|
||||
)
|
||||
|
||||
const (
|
||||
TranscriptOutputKindBase = artifactmodel.TranscriptOutputKindBase
|
||||
TranscriptOutputKindPolished = artifactmodel.TranscriptOutputKindPolished
|
||||
TranscriptOutputKindFinal = artifactmodel.TranscriptOutputKindFinal
|
||||
TranscriptOutputKindFinalTrimmed = artifactmodel.TranscriptOutputKindFinalTrimmed
|
||||
)
|
||||
|
||||
// ErrSessionArtifactNotFound is returned when no readable artifact exists for a known ID.
|
||||
var ErrSessionArtifactNotFound = errors.New("session artifact not found")
|
||||
var configuredArtifactSourceRE = regexp.MustCompile(`^narratio\.artifact\.[a-z][a-z0-9_]*$`)
|
||||
var configuredArtifactSourceRE = regexp.MustCompile(`^narratio\.artifact\.([a-z][a-z0-9_]*)$`)
|
||||
var previousSessionArtifactSourceRE = regexp.MustCompile(`^narratio\.previous_session\.artifact\.([a-z][a-z0-9_]*)$`)
|
||||
|
||||
type artifactContentKind string
|
||||
|
||||
@@ -40,42 +59,27 @@ type artifactSpec struct {
|
||||
ContentKind artifactContentKind
|
||||
}
|
||||
|
||||
var artifactRegistry = map[string]artifactSpec{
|
||||
ArtifactTranscriptMerged: {
|
||||
ID: ArtifactTranscriptMerged,
|
||||
CanonicalRelPath: "transcripts/merged.json",
|
||||
ProducerStage: "merge",
|
||||
OutputKind: "transcript_merged",
|
||||
ContentKind: contentTranscriptJSON,
|
||||
},
|
||||
ArtifactTranscriptPolished: {
|
||||
ID: ArtifactTranscriptPolished,
|
||||
CanonicalRelPath: "transcripts/processed.json",
|
||||
ProducerStage: "polish",
|
||||
OutputKind: "transcript_processed",
|
||||
ContentKind: contentTranscriptJSON,
|
||||
},
|
||||
ArtifactTranscriptFull: {
|
||||
ID: ArtifactTranscriptFull,
|
||||
CanonicalRelPath: "transcripts/normalized.json",
|
||||
ProducerStage: "normalize",
|
||||
OutputKind: "transcript_normalized",
|
||||
ContentKind: contentTranscriptJSON,
|
||||
},
|
||||
ArtifactTranscriptTrimmed: {
|
||||
ID: ArtifactTranscriptTrimmed,
|
||||
CanonicalRelPath: "transcripts/trimmed.json",
|
||||
ProducerStage: "trim",
|
||||
OutputKind: "transcript_trimmed",
|
||||
ContentKind: contentTranscriptJSON,
|
||||
},
|
||||
ArtifactBoundsSession: {
|
||||
var artifactRegistry = buildArtifactRegistry()
|
||||
|
||||
func buildArtifactRegistry() map[string]artifactSpec {
|
||||
registry := map[string]artifactSpec{}
|
||||
for _, transcript := range RuntimeTranscriptArtifacts() {
|
||||
registry[transcript.SourceID] = artifactSpec{
|
||||
ID: transcript.SourceID,
|
||||
CanonicalRelPath: transcript.CanonicalRelPath,
|
||||
ProducerStage: transcript.ProducerStage,
|
||||
OutputKind: transcript.OutputKind,
|
||||
ContentKind: contentTranscriptJSON,
|
||||
}
|
||||
}
|
||||
registry[ArtifactBoundsSession] = artifactSpec{
|
||||
ID: ArtifactBoundsSession,
|
||||
CanonicalRelPath: "artifacts/session_bounds.json",
|
||||
ProducerStage: "trim",
|
||||
OutputKind: "session_bounds",
|
||||
ContentKind: contentJSON,
|
||||
},
|
||||
}
|
||||
return registry
|
||||
}
|
||||
|
||||
// ResolvedSessionArtifact describes one session-level artifact lookup result.
|
||||
@@ -118,6 +122,30 @@ func IsConfiguredArtifactSource(source string) bool {
|
||||
return configuredArtifactSourceRE.MatchString(strings.TrimSpace(source))
|
||||
}
|
||||
|
||||
// ConfiguredArtifactName extracts <name> from narratio.artifact.<name>.
|
||||
func ConfiguredArtifactName(source string) (string, bool) {
|
||||
matches := configuredArtifactSourceRE.FindStringSubmatch(strings.TrimSpace(source))
|
||||
if len(matches) != 2 {
|
||||
return "", false
|
||||
}
|
||||
return matches[1], true
|
||||
}
|
||||
|
||||
// IsPreviousSessionArtifactSource returns true when source is narratio.previous_session.artifact.<name>.
|
||||
func IsPreviousSessionArtifactSource(source string) bool {
|
||||
_, ok := PreviousSessionArtifactName(source)
|
||||
return ok
|
||||
}
|
||||
|
||||
// PreviousSessionArtifactName extracts <name> from narratio.previous_session.artifact.<name>.
|
||||
func PreviousSessionArtifactName(source string) (string, bool) {
|
||||
matches := previousSessionArtifactSourceRE.FindStringSubmatch(strings.TrimSpace(source))
|
||||
if len(matches) != 2 {
|
||||
return "", false
|
||||
}
|
||||
return matches[1], true
|
||||
}
|
||||
|
||||
// ResolveSessionArtifact resolves a symbolic source to a readable local session artifact path.
|
||||
// Resolution order is manifest producer outputs first, then canonical session path fallback.
|
||||
func ResolveSessionArtifact(paths SessionPaths, m *manifest.Manifest, source string) (ResolvedSessionArtifact, error) {
|
||||
@@ -170,6 +198,9 @@ func ResolveSessionArtifact(paths SessionPaths, m *manifest.Manifest, source str
|
||||
// configured narratio.artifact.<name> sources through runtime catalog availability.
|
||||
func ResolveSessionArtifactWithCatalog(paths SessionPaths, m *manifest.Manifest, source string, catalog *ArtifactCatalog) (ResolvedSessionArtifact, error) {
|
||||
normalized := strings.TrimSpace(source)
|
||||
if IsPreviousSessionArtifactSource(normalized) {
|
||||
return ResolvePreviousSessionArtifactWithCatalog(paths, m, normalized, catalog)
|
||||
}
|
||||
if !IsConfiguredArtifactSource(normalized) {
|
||||
return ResolveSessionArtifact(paths, m, normalized)
|
||||
}
|
||||
@@ -195,6 +226,72 @@ func ResolveSessionArtifactWithCatalog(paths SessionPaths, m *manifest.Manifest,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// ResolvePreviousSessionArtifactWithCatalog resolves one canonical previous-session source id
|
||||
// to the prepared current-session previous-cache path.
|
||||
func ResolvePreviousSessionArtifactWithCatalog(
|
||||
paths SessionPaths,
|
||||
m *manifest.Manifest,
|
||||
source string,
|
||||
catalog *ArtifactCatalog,
|
||||
) (ResolvedSessionArtifact, error) {
|
||||
artifactName, ok := PreviousSessionArtifactName(source)
|
||||
if !ok {
|
||||
return ResolvedSessionArtifact{}, fmt.Errorf("unsupported previous-session artifact source %q", source)
|
||||
}
|
||||
if catalog == nil {
|
||||
return ResolvedSessionArtifact{}, fmt.Errorf("previous-session artifact source %q requires runtime artifact catalog", source)
|
||||
}
|
||||
|
||||
configuredSourceID := ConfiguredArtifactSourceID(artifactName)
|
||||
entry, ok := catalog.Lookup(configuredSourceID)
|
||||
if !ok {
|
||||
return ResolvedSessionArtifact{}, fmt.Errorf("unsupported previous-session artifact source %q", source)
|
||||
}
|
||||
candidates := previousSessionCacheCandidatePaths(paths, entry.CanonicalRelPath)
|
||||
if len(candidates) == 0 {
|
||||
return ResolvedSessionArtifact{}, &SessionArtifactNotFoundError{ArtifactID: source}
|
||||
}
|
||||
|
||||
manifestInputPaths := manifestInputPathSet(paths, m)
|
||||
fallback := ""
|
||||
for _, candidate := range candidates {
|
||||
exists, isDir, statErr := pathExists(candidate)
|
||||
if statErr != nil {
|
||||
return ResolvedSessionArtifact{}, fmt.Errorf("stat %q: %w", candidate, statErr)
|
||||
}
|
||||
if !exists || isDir {
|
||||
continue
|
||||
}
|
||||
if err := validateResolvedContent(candidate, contentText); err != nil {
|
||||
return ResolvedSessionArtifact{}, fmt.Errorf("validate %q: %w", source, err)
|
||||
}
|
||||
if _, ok := manifestInputPaths[candidate]; ok {
|
||||
return ResolvedSessionArtifact{
|
||||
ID: source,
|
||||
Path: candidate,
|
||||
ProducerStage: "prepare",
|
||||
OutputKind: "previous_session_cache",
|
||||
Provenance: ArtifactProvenancePreviousCacheManifestInput,
|
||||
}, nil
|
||||
}
|
||||
if fallback == "" {
|
||||
fallback = candidate
|
||||
}
|
||||
}
|
||||
|
||||
if fallback != "" {
|
||||
return ResolvedSessionArtifact{
|
||||
ID: source,
|
||||
Path: fallback,
|
||||
ProducerStage: "prepare",
|
||||
OutputKind: "previous_session_cache",
|
||||
Provenance: ArtifactProvenancePreviousCacheFilesystem,
|
||||
}, nil
|
||||
}
|
||||
|
||||
return ResolvedSessionArtifact{}, &SessionArtifactNotFoundError{ArtifactID: source}
|
||||
}
|
||||
|
||||
func manifestArtifactCandidates(paths SessionPaths, m *manifest.Manifest, spec artifactSpec) []ResolvedSessionArtifact {
|
||||
if m == nil || len(m.Stages) == 0 || spec.ProducerStage == "" || spec.OutputKind == "" {
|
||||
return nil
|
||||
@@ -243,6 +340,50 @@ func dedupeResolvedArtifacts(values []ResolvedSessionArtifact) []ResolvedSession
|
||||
return out
|
||||
}
|
||||
|
||||
func previousSessionCacheCandidatePaths(paths SessionPaths, canonicalRelPath string) []string {
|
||||
trimmed := strings.TrimSpace(canonicalRelPath)
|
||||
if trimmed == "" {
|
||||
return nil
|
||||
}
|
||||
normalized := filepath.ToSlash(filepath.Clean(filepath.FromSlash(trimmed)))
|
||||
if normalized == "." || normalized == "" || normalized == ".." || strings.HasPrefix(normalized, "../") || strings.HasPrefix(normalized, "/") {
|
||||
return nil
|
||||
}
|
||||
|
||||
relCandidates := []string{normalized}
|
||||
const artifactsPrefix = "artifacts/"
|
||||
if strings.HasPrefix(normalized, artifactsPrefix) && len(normalized) > len(artifactsPrefix) {
|
||||
relCandidates = append(relCandidates, strings.TrimPrefix(normalized, artifactsPrefix))
|
||||
}
|
||||
|
||||
out := make([]string, 0, len(relCandidates))
|
||||
seen := map[string]struct{}{}
|
||||
for _, rel := range relCandidates {
|
||||
abs := filepath.Clean(SessionPreviousArtifactPath(paths, rel))
|
||||
if _, ok := seen[abs]; ok {
|
||||
continue
|
||||
}
|
||||
seen[abs] = struct{}{}
|
||||
out = append(out, abs)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func manifestInputPathSet(paths SessionPaths, m *manifest.Manifest) map[string]struct{} {
|
||||
if m == nil || len(m.Inputs) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make(map[string]struct{}, len(m.Inputs))
|
||||
for _, in := range m.Inputs {
|
||||
resolved := filepath.Clean(ResolveSessionLocalPathForRead(paths, in.Path))
|
||||
if strings.TrimSpace(resolved) == "" {
|
||||
continue
|
||||
}
|
||||
out[resolved] = struct{}{}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func pathExists(path string) (exists bool, isDir bool, err error) {
|
||||
info, err := os.Stat(path)
|
||||
if err == nil {
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -22,7 +23,7 @@ func TestNormalizeSessionArtifactSource(t *testing.T) {
|
||||
{name: "legacy alias normalized unsupported", source: "normalized_transcript", wantErr: "unsupported artifact source"},
|
||||
{name: "legacy alias trimmed unsupported", source: "trimmed_transcript", wantErr: "unsupported artifact source"},
|
||||
{name: "configured source unsupported in built-in normalization", source: "narratio.artifact.session_recap", wantErr: "unsupported artifact source"},
|
||||
{name: "canonical", source: ArtifactTranscriptTrimmed, wantID: ArtifactTranscriptTrimmed},
|
||||
{name: "canonical", source: ArtifactTranscriptFinalTrimmed, wantID: ArtifactTranscriptFinalTrimmed},
|
||||
{name: "unsupported", source: "narratio.unknown", wantErr: "unsupported artifact source"},
|
||||
}
|
||||
|
||||
@@ -45,6 +46,110 @@ func TestNormalizeSessionArtifactSource(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfiguredArtifactSourceHelpers(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
source string
|
||||
wantName string
|
||||
wantMatch bool
|
||||
}{
|
||||
{
|
||||
name: "valid",
|
||||
source: "narratio.artifact.session_recap",
|
||||
wantName: "session_recap",
|
||||
wantMatch: true,
|
||||
},
|
||||
{
|
||||
name: "valid with surrounding whitespace",
|
||||
source: " narratio.artifact.player_handout ",
|
||||
wantName: "player_handout",
|
||||
wantMatch: true,
|
||||
},
|
||||
{
|
||||
name: "missing name",
|
||||
source: "narratio.artifact.",
|
||||
wantMatch: false,
|
||||
},
|
||||
{
|
||||
name: "invalid hyphen",
|
||||
source: "narratio.artifact.session-recap",
|
||||
wantMatch: false,
|
||||
},
|
||||
{
|
||||
name: "built-in",
|
||||
source: ArtifactTranscriptBase,
|
||||
wantMatch: false,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if got := IsConfiguredArtifactSource(tt.source); got != tt.wantMatch {
|
||||
t.Fatalf("IsConfiguredArtifactSource(%q) = %t, want %t", tt.source, got, tt.wantMatch)
|
||||
}
|
||||
gotName, gotOK := ConfiguredArtifactName(tt.source)
|
||||
if gotOK != tt.wantMatch {
|
||||
t.Fatalf("ConfiguredArtifactName(%q) ok = %t, want %t", tt.source, gotOK, tt.wantMatch)
|
||||
}
|
||||
if gotName != tt.wantName {
|
||||
t.Fatalf("ConfiguredArtifactName(%q) name = %q, want %q", tt.source, gotName, tt.wantName)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestPreviousSessionArtifactSourceHelpers(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
source string
|
||||
wantName string
|
||||
wantMatch bool
|
||||
}{
|
||||
{
|
||||
name: "valid",
|
||||
source: "narratio.previous_session.artifact.session_recap",
|
||||
wantName: "session_recap",
|
||||
wantMatch: true,
|
||||
},
|
||||
{
|
||||
name: "valid with surrounding whitespace",
|
||||
source: " narratio.previous_session.artifact.quest_log ",
|
||||
wantName: "quest_log",
|
||||
wantMatch: true,
|
||||
},
|
||||
{
|
||||
name: "missing name",
|
||||
source: "narratio.previous_session.artifact.",
|
||||
wantMatch: false,
|
||||
},
|
||||
{
|
||||
name: "invalid key characters",
|
||||
source: "narratio.previous_session.artifact.session-recap",
|
||||
wantMatch: false,
|
||||
},
|
||||
{
|
||||
name: "wrong prefix",
|
||||
source: "narratio.previous.artifact.session_recap",
|
||||
wantMatch: false,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
name, ok := PreviousSessionArtifactName(tt.source)
|
||||
if ok != tt.wantMatch {
|
||||
t.Fatalf("PreviousSessionArtifactName(%q) ok = %t, want %t", tt.source, ok, tt.wantMatch)
|
||||
}
|
||||
if name != tt.wantName {
|
||||
t.Fatalf("PreviousSessionArtifactName(%q) name = %q, want %q", tt.source, name, tt.wantName)
|
||||
}
|
||||
if got := IsPreviousSessionArtifactSource(tt.source); got != tt.wantMatch {
|
||||
t.Fatalf("IsPreviousSessionArtifactSource(%q) = %t, want %t", tt.source, got, tt.wantMatch)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveSessionArtifactPrefersManifestOutput(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||
@@ -55,7 +160,7 @@ func TestResolveSessionArtifactPrefersManifestOutput(t *testing.T) {
|
||||
if err := os.WriteFile(manifestPath, []byte(`{"segments":[]}`), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "normalized.json")
|
||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "final.json")
|
||||
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
@@ -65,10 +170,10 @@ func TestResolveSessionArtifactPrefersManifestOutput(t *testing.T) {
|
||||
|
||||
m := manifest.New("session", time.Now().UTC())
|
||||
m.MarkStageSucceeded("normalize", time.Now().UTC(), []manifest.ArtifactRecord{
|
||||
{Kind: "transcript_normalized", LocalPath: manifestPath, ProducerRunID: "run-123"},
|
||||
{Kind: "transcript_final", LocalPath: manifestPath, ProducerRunID: "run-123"},
|
||||
})
|
||||
|
||||
resolved, err := ResolveSessionArtifact(paths, m, ArtifactTranscriptFull)
|
||||
resolved, err := ResolveSessionArtifact(paths, m, ArtifactTranscriptFinal)
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveSessionArtifact() error = %v", err)
|
||||
}
|
||||
@@ -86,7 +191,7 @@ func TestResolveSessionArtifactPrefersManifestOutput(t *testing.T) {
|
||||
func TestResolveSessionArtifactFallsBackToCanonicalPath(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "trimmed.json")
|
||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "final.trimmed.json")
|
||||
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
@@ -94,7 +199,7 @@ func TestResolveSessionArtifactFallsBackToCanonicalPath(t *testing.T) {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
|
||||
resolved, err := ResolveSessionArtifact(paths, nil, ArtifactTranscriptTrimmed)
|
||||
resolved, err := ResolveSessionArtifact(paths, nil, ArtifactTranscriptFinalTrimmed)
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveSessionArtifact() error = %v", err)
|
||||
}
|
||||
@@ -110,7 +215,7 @@ func TestResolveSessionArtifactMissingReturnsTypedError(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||
|
||||
_, err := ResolveSessionArtifact(paths, nil, ArtifactTranscriptTrimmed)
|
||||
_, err := ResolveSessionArtifact(paths, nil, ArtifactTranscriptFinalTrimmed)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
@@ -122,7 +227,7 @@ func TestResolveSessionArtifactMissingReturnsTypedError(t *testing.T) {
|
||||
func TestResolveSessionArtifactValidatesTranscriptShape(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "processed.json")
|
||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "polished.json")
|
||||
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
@@ -142,7 +247,7 @@ func TestResolveSessionArtifactValidatesTranscriptShape(t *testing.T) {
|
||||
func TestResolveSessionArtifactWithCatalogBuiltInBehaviorUnchanged(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "trimmed.json")
|
||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "final.trimmed.json")
|
||||
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
@@ -150,7 +255,7 @@ func TestResolveSessionArtifactWithCatalogBuiltInBehaviorUnchanged(t *testing.T)
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
|
||||
resolved, err := ResolveSessionArtifactWithCatalog(paths, nil, ArtifactTranscriptTrimmed, NewArtifactCatalog())
|
||||
resolved, err := ResolveSessionArtifactWithCatalog(paths, nil, ArtifactTranscriptFinalTrimmed, NewArtifactCatalog())
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveSessionArtifactWithCatalog() error = %v", err)
|
||||
}
|
||||
@@ -266,3 +371,131 @@ func TestResolveSessionArtifactWithCatalogUnsupportedConfiguredSourceFails(t *te
|
||||
t.Fatalf("error = %q, want unsupported artifact source", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolvePreviousSessionArtifactWithCatalogPrefersManifestInputRecord(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||
|
||||
manifestBackedPath := SessionPreviousArtifactPath(paths, "artifacts/session_recap.md")
|
||||
fallbackPath := SessionPreviousArtifactPath(paths, "session_recap.md")
|
||||
if err := os.MkdirAll(filepath.Dir(manifestBackedPath), 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(fallbackPath), 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
if err := os.WriteFile(manifestBackedPath, []byte("recap from manifest input\n"), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
if err := os.WriteFile(fallbackPath, []byte("recap fallback\n"), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
|
||||
catalog := NewArtifactCatalog()
|
||||
if err := catalog.RegisterConfiguredArtifacts(
|
||||
map[string]ConfiguredArtifactDefinition{
|
||||
"session_recap": {Enabled: true, OutputPath: "artifacts/session_recap.md"},
|
||||
},
|
||||
nil,
|
||||
); err != nil {
|
||||
t.Fatalf("RegisterConfiguredArtifacts() error = %v", err)
|
||||
}
|
||||
|
||||
m := manifest.New("session", time.Now().UTC())
|
||||
m.Inputs = []manifest.InputRecord{
|
||||
{Kind: "previous_artifact", Path: manifestBackedPath},
|
||||
}
|
||||
|
||||
resolved, err := ResolvePreviousSessionArtifactWithCatalog(
|
||||
paths,
|
||||
m,
|
||||
"narratio.previous_session.artifact.session_recap",
|
||||
catalog,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("ResolvePreviousSessionArtifactWithCatalog() error = %v", err)
|
||||
}
|
||||
if resolved.Path != manifestBackedPath {
|
||||
t.Fatalf("resolved path = %q, want %q", resolved.Path, manifestBackedPath)
|
||||
}
|
||||
if resolved.Provenance != ArtifactProvenancePreviousCacheManifestInput {
|
||||
t.Fatalf("provenance = %q, want %q", resolved.Provenance, ArtifactProvenancePreviousCacheManifestInput)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolvePreviousSessionArtifactWithCatalogFallsBackToPreparedCachePath(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||
|
||||
fallbackPath := SessionPreviousArtifactPath(paths, "session_recap.md")
|
||||
if err := os.MkdirAll(filepath.Dir(fallbackPath), 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
if err := os.WriteFile(fallbackPath, []byte("recap fallback\n"), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
|
||||
catalog := NewArtifactCatalog()
|
||||
if err := catalog.RegisterConfiguredArtifacts(
|
||||
map[string]ConfiguredArtifactDefinition{
|
||||
"session_recap": {Enabled: true, OutputPath: "artifacts/session_recap.md"},
|
||||
},
|
||||
nil,
|
||||
); err != nil {
|
||||
t.Fatalf("RegisterConfiguredArtifacts() error = %v", err)
|
||||
}
|
||||
|
||||
resolved, err := ResolvePreviousSessionArtifactWithCatalog(
|
||||
paths,
|
||||
nil,
|
||||
"narratio.previous_session.artifact.session_recap",
|
||||
catalog,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("ResolvePreviousSessionArtifactWithCatalog() error = %v", err)
|
||||
}
|
||||
if resolved.Path != fallbackPath {
|
||||
t.Fatalf("resolved path = %q, want %q", resolved.Path, fallbackPath)
|
||||
}
|
||||
if resolved.Provenance != ArtifactProvenancePreviousCacheFilesystem {
|
||||
t.Fatalf("provenance = %q, want %q", resolved.Provenance, ArtifactProvenancePreviousCacheFilesystem)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPreviousSessionCacheCandidatePathsStripsArtifactsPrefix(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||
|
||||
got := previousSessionCacheCandidatePaths(paths, "artifacts/session_recap.md")
|
||||
want := []string{filepath.Join(paths.PreviousArtifactsDir, "session_recap.md")}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("previousSessionCacheCandidatePaths() = %#v, want %#v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolvePreviousSessionArtifactWithCatalogMissingReturnsTypedError(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||
catalog := NewArtifactCatalog()
|
||||
if err := catalog.RegisterConfiguredArtifacts(
|
||||
map[string]ConfiguredArtifactDefinition{
|
||||
"session_recap": {Enabled: true, OutputPath: "artifacts/session_recap.md"},
|
||||
},
|
||||
nil,
|
||||
); err != nil {
|
||||
t.Fatalf("RegisterConfiguredArtifacts() error = %v", err)
|
||||
}
|
||||
|
||||
_, err := ResolvePreviousSessionArtifactWithCatalog(
|
||||
paths,
|
||||
nil,
|
||||
"narratio.previous_session.artifact.session_recap",
|
||||
catalog,
|
||||
)
|
||||
if err == nil {
|
||||
t.Fatal("expected error, got nil")
|
||||
}
|
||||
if !errors.Is(err, ErrSessionArtifactNotFound) {
|
||||
t.Fatalf("errors.Is(err, ErrSessionArtifactNotFound)=false; err=%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -215,10 +215,10 @@ func (c *ArtifactCatalog) addEntry(entry CatalogEntry) error {
|
||||
|
||||
func runtimeBuiltInArtifactIDs() []string {
|
||||
return []string{
|
||||
ArtifactTranscriptMerged,
|
||||
ArtifactTranscriptBase,
|
||||
ArtifactTranscriptPolished,
|
||||
ArtifactTranscriptFull,
|
||||
ArtifactTranscriptTrimmed,
|
||||
ArtifactTranscriptFinal,
|
||||
ArtifactTranscriptFinalTrimmed,
|
||||
ArtifactBoundsSession,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,9 +8,9 @@ func TestArtifactCatalogRegisterBuiltInsAndLookup(t *testing.T) {
|
||||
t.Fatalf("RegisterBuiltIns() error = %v", err)
|
||||
}
|
||||
|
||||
entry, ok := catalog.Lookup(ArtifactTranscriptFull)
|
||||
entry, ok := catalog.Lookup(ArtifactTranscriptFinal)
|
||||
if !ok {
|
||||
t.Fatalf("Lookup(%q) ok = false, want true", ArtifactTranscriptFull)
|
||||
t.Fatalf("Lookup(%q) ok = false, want true", ArtifactTranscriptFinal)
|
||||
}
|
||||
if !entry.Planned {
|
||||
t.Fatalf("entry.Planned = false, want true")
|
||||
@@ -18,8 +18,8 @@ func TestArtifactCatalogRegisterBuiltInsAndLookup(t *testing.T) {
|
||||
if entry.Executable {
|
||||
t.Fatalf("entry.Executable = true, want false")
|
||||
}
|
||||
if entry.CanonicalRelPath != "transcripts/normalized.json" {
|
||||
t.Fatalf("entry.CanonicalRelPath = %q, want transcripts/normalized.json", entry.CanonicalRelPath)
|
||||
if entry.CanonicalRelPath != "transcripts/final.json" {
|
||||
t.Fatalf("entry.CanonicalRelPath = %q, want transcripts/final.json", entry.CanonicalRelPath)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -73,6 +73,8 @@ func (s *LocalStore) ensureLayout(paths SessionPaths) (SessionPaths, error) {
|
||||
paths.LogsDir,
|
||||
paths.CurrentDir,
|
||||
paths.RunsDir,
|
||||
paths.PreviousDir,
|
||||
paths.PreviousArtifactsDir,
|
||||
}
|
||||
|
||||
for _, dir := range dirs {
|
||||
|
||||
@@ -27,10 +27,15 @@ func TestEnsureLayoutCreatesExpectedDirectories(t *testing.T) {
|
||||
checkDirExists(t, paths.LogsDir)
|
||||
checkDirExists(t, paths.CurrentDir)
|
||||
checkDirExists(t, paths.RunsDir)
|
||||
checkDirExists(t, paths.PreviousDir)
|
||||
checkDirExists(t, paths.PreviousArtifactsDir)
|
||||
|
||||
if filepath.Base(paths.ManifestPath) != "manifest.json" {
|
||||
t.Fatalf("ManifestPath = %q, want basename manifest.json", paths.ManifestPath)
|
||||
}
|
||||
if filepath.Base(paths.PreviousManifestPath) != "manifest.json" {
|
||||
t.Fatalf("PreviousManifestPath = %q, want basename manifest.json", paths.PreviousManifestPath)
|
||||
}
|
||||
if filepath.Base(paths.LockPath) != ".lock" {
|
||||
t.Fatalf("LockPath = %q, want basename .lock", paths.LockPath)
|
||||
}
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
package artifacts
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
@@ -23,6 +26,9 @@ type SessionPaths struct {
|
||||
LogsDir string
|
||||
CurrentDir string
|
||||
RunsDir string
|
||||
PreviousDir string
|
||||
PreviousManifestPath string
|
||||
PreviousArtifactsDir string
|
||||
ManifestPath string
|
||||
LockPath string
|
||||
}
|
||||
@@ -42,6 +48,29 @@ func SessionRunsDirForCampaign(rootDir, campaign, sessionID string) string {
|
||||
return filepath.Join(SessionWorkDirForCampaign(rootDir, campaign, sessionID), config.PathRunsDirSegment)
|
||||
}
|
||||
|
||||
// SessionPreviousDirForCampaign returns the canonical previous-session state directory.
|
||||
func SessionPreviousDirForCampaign(rootDir, campaign, sessionID string) string {
|
||||
return filepath.Join(SessionWorkDirForCampaign(rootDir, campaign, sessionID), config.PathPreviousDirSegment)
|
||||
}
|
||||
|
||||
// SessionPreviousManifestPathForCampaign returns the canonical previous-session manifest cache path.
|
||||
func SessionPreviousManifestPathForCampaign(rootDir, campaign, sessionID string) string {
|
||||
return filepath.Join(SessionPreviousDirForCampaign(rootDir, campaign, sessionID), config.PathManifestFile)
|
||||
}
|
||||
|
||||
// SessionPreviousArtifactsDirForCampaign returns the canonical previous-session artifact cache directory.
|
||||
func SessionPreviousArtifactsDirForCampaign(rootDir, campaign, sessionID string) string {
|
||||
return filepath.Join(SessionPreviousDirForCampaign(rootDir, campaign, sessionID), config.PathArtifactsDirSegment)
|
||||
}
|
||||
|
||||
// SessionPreviousArtifactPathForCampaign returns a path under previous/artifacts for one artifact.
|
||||
func SessionPreviousArtifactPathForCampaign(rootDir, campaign, sessionID, artifactRelativePath string) string {
|
||||
return filepath.Join(
|
||||
SessionPreviousArtifactsDirForCampaign(rootDir, campaign, sessionID),
|
||||
filepath.FromSlash(previousArtifactCacheRelativePath(artifactRelativePath)),
|
||||
)
|
||||
}
|
||||
|
||||
// SessionRunRootForCampaign returns the canonical run root under runs/{run_id}.
|
||||
func SessionRunRootForCampaign(rootDir, campaign, sessionID, runID string) string {
|
||||
return filepath.Join(SessionRunsDirForCampaign(rootDir, campaign, sessionID), runID)
|
||||
@@ -62,6 +91,92 @@ func SessionSpoolAudioDir(spoolRoot, campaign, sessionID, runID string) string {
|
||||
return filepath.Join(spoolRoot, campaign, sessionID, runID, config.PathAudioDirSegment)
|
||||
}
|
||||
|
||||
// SessionSpoolDir returns the campaign/session scoped local spool root.
|
||||
func SessionSpoolDir(spoolRoot, campaign, sessionID string) string {
|
||||
return filepath.Join(spoolRoot, campaign, sessionID)
|
||||
}
|
||||
|
||||
// SessionSpoolRestoreAudioDir returns the local spool audio path for restore downloads.
|
||||
func SessionSpoolRestoreAudioDir(spoolRoot, campaign, sessionID string) string {
|
||||
return filepath.Join(SessionSpoolDir(spoolRoot, campaign, sessionID), "restore", config.PathAudioDirSegment)
|
||||
}
|
||||
|
||||
// S3AudioCachePath returns the durable local cache path for one S3 audio object.
|
||||
func S3AudioCachePath(cacheRoot, bucket, key string) (string, error) {
|
||||
root := filepath.Clean(strings.TrimSpace(cacheRoot))
|
||||
if root == "." || root == "" {
|
||||
return "", fmt.Errorf("cache root is required")
|
||||
}
|
||||
bucket = strings.Trim(strings.TrimSpace(bucket), "/")
|
||||
if bucket == "" || bucket == "." || bucket == ".." || strings.Contains(bucket, "/") || strings.Contains(bucket, `\`) {
|
||||
return "", fmt.Errorf("bucket is required and must be a single path segment")
|
||||
}
|
||||
rawKey := strings.ReplaceAll(strings.TrimSpace(key), `\`, "/")
|
||||
if strings.HasPrefix(rawKey, "/") {
|
||||
return "", fmt.Errorf("s3 key must not be absolute")
|
||||
}
|
||||
cleanKey := cleanCacheS3Key(rawKey)
|
||||
if cleanKey == "" {
|
||||
return "", fmt.Errorf("s3 key is required")
|
||||
}
|
||||
if cleanKey == ".." || strings.HasPrefix(cleanKey, "../") || strings.HasPrefix(cleanKey, "/") {
|
||||
return "", fmt.Errorf("s3 key must not escape cache root")
|
||||
}
|
||||
return filepath.Join(root, "s3", bucket, filepath.FromSlash(cleanKey)), nil
|
||||
}
|
||||
|
||||
// S3AudioCacheNamespaceDir returns the durable local cache namespace for all
|
||||
// Narratio S3 audio objects under one bucket/root prefix.
|
||||
func S3AudioCacheNamespaceDir(cacheRoot, bucket, rootPrefix string) (string, error) {
|
||||
sentinelKey := path.Join(cleanS3PathPart(rootPrefix), config.S3CampaignsSegment, ".narratio-cache-sentinel")
|
||||
sentinelPath, err := S3AudioCachePath(cacheRoot, bucket, sentinelKey)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return filepath.Dir(sentinelPath), nil
|
||||
}
|
||||
|
||||
func cleanCacheS3Key(key string) string {
|
||||
normalized := strings.ReplaceAll(strings.TrimSpace(key), `\`, "/")
|
||||
normalized = strings.Trim(normalized, "/")
|
||||
if normalized == "" {
|
||||
return ""
|
||||
}
|
||||
return path.Clean(normalized)
|
||||
}
|
||||
|
||||
// SessionPreviousDir returns the previous-session state directory for already-resolved session paths.
|
||||
func SessionPreviousDir(paths SessionPaths) string {
|
||||
return paths.PreviousDir
|
||||
}
|
||||
|
||||
// SessionPreviousManifestPath returns the previous-session manifest path for already-resolved session paths.
|
||||
func SessionPreviousManifestPath(paths SessionPaths) string {
|
||||
return paths.PreviousManifestPath
|
||||
}
|
||||
|
||||
// SessionPreviousArtifactsDir returns the previous-session artifact directory for already-resolved session paths.
|
||||
func SessionPreviousArtifactsDir(paths SessionPaths) string {
|
||||
return paths.PreviousArtifactsDir
|
||||
}
|
||||
|
||||
// SessionPreviousArtifactPath returns a path under previous/artifacts for already-resolved session paths.
|
||||
func SessionPreviousArtifactPath(paths SessionPaths, artifactRelativePath string) string {
|
||||
return filepath.Join(paths.PreviousArtifactsDir, filepath.FromSlash(previousArtifactCacheRelativePath(artifactRelativePath)))
|
||||
}
|
||||
|
||||
func previousArtifactCacheRelativePath(artifactRelativePath string) string {
|
||||
rel := filepath.ToSlash(filepath.Clean(filepath.FromSlash(strings.TrimSpace(artifactRelativePath))))
|
||||
if rel == "." {
|
||||
return ""
|
||||
}
|
||||
const artifactsPrefix = "artifacts/"
|
||||
if strings.HasPrefix(rel, artifactsPrefix) && len(rel) > len(artifactsPrefix) {
|
||||
return strings.TrimPrefix(rel, artifactsPrefix)
|
||||
}
|
||||
return rel
|
||||
}
|
||||
|
||||
func buildSessionPaths(workspaceRoot, campaign, sessionID string) SessionPaths {
|
||||
root := SessionWorkDirForCampaign(workspaceRoot, campaign, sessionID)
|
||||
return buildSessionPathsFromRoot(workspaceRoot, campaign, sessionID, root)
|
||||
@@ -84,6 +199,9 @@ func buildSessionPathsFromRoot(workspaceRoot, campaign, sessionID, root string)
|
||||
LogsDir: filepath.Join(root, config.PathLogsDirSegment),
|
||||
CurrentDir: filepath.Join(root, config.PathCurrentDirSegment),
|
||||
RunsDir: filepath.Join(root, config.PathRunsDirSegment),
|
||||
PreviousDir: filepath.Join(root, config.PathPreviousDirSegment),
|
||||
PreviousManifestPath: filepath.Join(root, config.PathPreviousDirSegment, config.PathManifestFile),
|
||||
PreviousArtifactsDir: filepath.Join(root, config.PathPreviousDirSegment, config.PathArtifactsDirSegment),
|
||||
ManifestPath: filepath.Join(root, config.PathManifestFile),
|
||||
LockPath: filepath.Join(root, config.PathLockFile),
|
||||
}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user