Compare commits
49 Commits
v0.11.0
...
ef1f650bc0
| Author | SHA1 | Date | |
|---|---|---|---|
| ef1f650bc0 | |||
| 0d02cb9fa0 | |||
| 0299b128cf | |||
| d723384888 | |||
| 54228055c8 | |||
| 23ed716450 | |||
| ab59bab044 | |||
| 71395bb076 | |||
| 79737edf79 | |||
| df2c765b7f | |||
| f050b9dd54 | |||
| 9c9cb54339 | |||
| 7657ec3ad6 | |||
| cee52aa092 | |||
| e920f3a8d5 | |||
| 591c529a09 | |||
| 7324c5a686 | |||
| d0936fb022 | |||
| 2aa074c5cf | |||
| 782d0cf3b9 | |||
| 083c01cfa0 | |||
| 2937696024 | |||
| b817a5b772 | |||
| 3022f20beb | |||
| ca1ded1821 | |||
| 3752f3ed28 | |||
| 870c2d69d5 | |||
| 135407ba7c | |||
| 228c348e42 | |||
| a813bd5a50 | |||
| d8f58dce31 | |||
| 7111edeca4 | |||
| 3aae4bbb12 | |||
| b29d8eeb50 | |||
| dffb432537 | |||
| 2dd38c7913 | |||
| bc2ade38d9 | |||
| 5be831eb13 | |||
| cae4d99a89 | |||
| e09dc0512d | |||
| ae82bc1ce0 | |||
| 01eb7aa1aa | |||
| 2ca700195c | |||
| 2b08c34539 | |||
| 79f1fc1e09 | |||
| 9c753270bd | |||
| b907cb01aa | |||
| 7824afd4a5 | |||
| 2a4e1e912c |
18
README.md
18
README.md
@@ -1,22 +1,22 @@
|
|||||||
# narratio
|
# narratio
|
||||||
|
|
||||||
Narratio is a Go orchestration application that turns D&D session audio into polished transcripts and generated session artifacts.
|
Narratio is a stage-driven Go orchestrator for turning D&D session audio into polished transcripts and generated artifacts.
|
||||||
|
|
||||||
It coordinates transcription, merge/polish/normalize/trim processing, artifact generation, archive publishing, and resumable run state in one operator workflow.
|
It runs a deterministic workflow across `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, `analyze`, and `publish`, with manifest-driven resume and restore support.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run --session-id 2026-04-04
|
narratio run 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
This command requires discoverable `pipeline.yml` and `session.yml` files (or explicit `--config` and `--session` flags).
|
This requires resolvable `pipeline.yml`, `campaign.yml`, and concrete `session.yml` (or explicit config flags).
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
- [Configuration](docs/config.md)
|
|
||||||
- [CLI Reference](docs/cli.md)
|
- [CLI Reference](docs/cli.md)
|
||||||
- [Operations and Recovery](docs/operations.md)
|
- [Configuration](docs/config.md)
|
||||||
|
- [Operations](docs/operations.md)
|
||||||
- [Troubleshooting](docs/troubleshooting.md)
|
- [Troubleshooting](docs/troubleshooting.md)
|
||||||
- [Development Guide](docs/development.md)
|
|
||||||
- [Architecture Principles](docs/architecture.md)
|
|
||||||
- [Internal Component Contracts](docs/internal/README.md)
|
- [Internal Component Contracts](docs/internal/README.md)
|
||||||
- [Config Examples](examples/)
|
- [Development Guide](docs/policy/development.md)
|
||||||
|
- [Architecture Principles](docs/policy/architecture.md)
|
||||||
|
- [Maintained Examples](examples/)
|
||||||
|
|||||||
384
docs/cli.md
384
docs/cli.md
@@ -1,61 +1,83 @@
|
|||||||
# CLI
|
# CLI Reference
|
||||||
|
|
||||||
## Shortest Useful Command
|
## Shortest Useful Command
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run --session-id 2026-04-04
|
narratio run 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
This command uses default discovery for `pipeline.yml` and `session.yml`; both files must be discoverable unless you pass explicit `--config` and `--session` paths.
|
This runs the canonical full pipeline for session `2026-04-04`.
|
||||||
|
|
||||||
## Command Overview
|
## Command Overview
|
||||||
|
|
||||||
Implemented commands:
|
Top-level commands:
|
||||||
|
|
||||||
- `run`: execute pipeline stages and persist manifest state.
|
- `run <session_id>`: run full stage order.
|
||||||
- `plan`: validate config, prepare workspace layout, and print stage run/skip decisions.
|
- `resume <session_id>`: continue from first non-succeeded stage.
|
||||||
- `resume`: continue from first non-succeeded stage unless forced.
|
- `run-stage <stage> <session_id>`: run one stage.
|
||||||
- `status`: read and print stage statuses from an existing manifest.
|
- `analyze <session_id>`: force-run analyze.
|
||||||
- `run-stage`: execute exactly one stage.
|
- `publish <session_id>`: force-run publish.
|
||||||
- `restore`: restore durable local session state from the committed remote archive state.
|
- `clean <session_id>` or `clean --all`: remove local work/spool state.
|
||||||
|
- `session <subcommand>`: session helper commands.
|
||||||
|
|
||||||
Unknown commands print usage and exit non-zero.
|
Session subcommands:
|
||||||
|
|
||||||
For config semantics, see [docs/config.md](./config.md). For operator lifecycle and recovery, see [docs/operations.md](./operations.md).
|
- `session init <session_id>`
|
||||||
|
- `session plan <session_id>`
|
||||||
|
- `session validate <session_id>`
|
||||||
|
- `session status <session_id>`
|
||||||
|
- `session restore <session_id>`
|
||||||
|
- `session artifacts <session_id>`
|
||||||
|
- `session locks <session_id>`
|
||||||
|
- `session locks add <session_id> <source>`
|
||||||
|
- `session locks remove <session_id> <source>`
|
||||||
|
|
||||||
## Complete Flag Reference
|
## Common Config Flags
|
||||||
|
|
||||||
|
Most session-aware commands accept:
|
||||||
|
|
||||||
|
- `--config <pipeline.yml>`
|
||||||
|
- `--campaign <id>`
|
||||||
|
- `--campaign-file <campaign.yml>`
|
||||||
|
- `--session <session.yml>`
|
||||||
|
- `--previous-session-id <id>`
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- `--campaign` and `--campaign-file` are mutually exclusive.
|
||||||
|
- `--session` is not used by `session init`.
|
||||||
|
- `clean --all` cannot be combined with campaign/session selectors.
|
||||||
|
|
||||||
|
## Command Reference
|
||||||
|
|
||||||
### `run`
|
### `run`
|
||||||
|
|
||||||
- `--config <path>`: optional explicit `pipeline.yml` path.
|
```bash
|
||||||
- `--session <path>`: optional explicit `session.yml` path.
|
narratio run <session_id> [--force] [--artifacts <name[,name...]>] [...common config flags]
|
||||||
- `--session-id <value>`: session template variable value.
|
```
|
||||||
- `--force`: force stage execution.
|
|
||||||
- `--artifacts <names>`: analyze artifact keys to execute (repeatable or comma-separated).
|
|
||||||
|
|
||||||
### `plan`
|
Behavior:
|
||||||
|
|
||||||
- `--config <path>`
|
- evaluates full stage order;
|
||||||
- `--session <path>`
|
- skips already-succeeded stages unless `--force` is set;
|
||||||
- `--session-id <value>`
|
- writes session and run manifests.
|
||||||
- `--force`
|
|
||||||
|
|
||||||
### `resume`
|
### `resume`
|
||||||
|
|
||||||
- `--config <path>`
|
```bash
|
||||||
- `--session <path>`
|
narratio resume <session_id> [--force] [--artifacts <name[,name...]>] [...common config flags]
|
||||||
- `--session-id <value>`
|
```
|
||||||
- `--force`
|
|
||||||
- `--artifacts <names>`: analyze artifact keys to execute (repeatable or comma-separated).
|
Behavior:
|
||||||
|
|
||||||
|
- when not forced, starts at first non-succeeded stage in manifest order;
|
||||||
|
- with `--force`, reevaluates the selected stage list as runnable.
|
||||||
|
|
||||||
### `run-stage`
|
### `run-stage`
|
||||||
|
|
||||||
- `--config <path>`
|
```bash
|
||||||
- `--session <path>`
|
narratio run-stage <stage> <session_id> [--force] [--artifacts <name[,name...]>] [...common config flags]
|
||||||
- `--session-id <value>`
|
```
|
||||||
- `--force`
|
|
||||||
- `--artifacts <names>`: analyze artifact keys to execute (repeatable or comma-separated).
|
|
||||||
- positional `<stage>`: required stage name.
|
|
||||||
|
|
||||||
Valid stage names:
|
Valid stage names:
|
||||||
|
|
||||||
@@ -66,213 +88,187 @@ Valid stage names:
|
|||||||
- `normalize`
|
- `normalize`
|
||||||
- `trim`
|
- `trim`
|
||||||
- `analyze`
|
- `analyze`
|
||||||
- `archive`
|
- `publish`
|
||||||
- `notify`
|
- `notify`
|
||||||
|
|
||||||
### `restore`
|
Rules:
|
||||||
|
|
||||||
- `--config <path>`
|
- `--artifacts` is accepted only for `analyze` and `publish` stage targets.
|
||||||
- `--session <path>`
|
|
||||||
- `--session-id <value>`
|
|
||||||
- `--dry-run`: plan restore actions without writing local files.
|
|
||||||
- `--force`: overwrite local conflicting files with remote archive files.
|
|
||||||
- `--include-audio`: include durable archived `audio/**` files in restore scope.
|
|
||||||
|
|
||||||
### `status`
|
### `analyze`
|
||||||
|
|
||||||
- `--manifest <path>`: required manifest path.
|
|
||||||
|
|
||||||
## Command Reference
|
|
||||||
|
|
||||||
### `run`
|
|
||||||
|
|
||||||
Purpose:
|
|
||||||
- Execute configured stages in canonical order.
|
|
||||||
|
|
||||||
Syntax:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] [--artifacts <name[,name...]>]
|
narratio analyze <session_id> [--artifacts <name[,name...]>] [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Success output:
|
Equivalent to:
|
||||||
- `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
|
|
||||||
|
|
||||||
Common failure cases:
|
|
||||||
- missing default config/session paths when flags omitted.
|
|
||||||
- invalid template/rendered session mismatch.
|
|
||||||
- unknown/invalid `--artifacts` value.
|
|
||||||
- `--artifacts` with unknown configured artifact key.
|
|
||||||
|
|
||||||
### `plan`
|
|
||||||
|
|
||||||
Purpose:
|
|
||||||
- Validate config, load secrets (if configured), prepare workdir, and print stage run/skip decisions.
|
|
||||||
|
|
||||||
Syntax:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio plan [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force]
|
narratio run-stage analyze <session_id> --force [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Success output includes:
|
### `publish`
|
||||||
- `narratio plan: workdir prepared at <path>`
|
|
||||||
- one line per stage (`<stage>: run|skip`)
|
|
||||||
- `totals: run=<n> skip=<n>`
|
|
||||||
|
|
||||||
Common failure cases:
|
|
||||||
- same config/session discovery and validation failures as `run`.
|
|
||||||
- secrets directory read failures when `pipeline.secrets.env_dir` is configured.
|
|
||||||
|
|
||||||
### `resume`
|
|
||||||
|
|
||||||
Purpose:
|
|
||||||
- Continue from session-manifest stage status.
|
|
||||||
|
|
||||||
Syntax:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio resume [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] [--artifacts <name[,name...]>]
|
narratio publish <session_id> [--artifacts <name[,name...]>] [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Success output:
|
Equivalent to:
|
||||||
- `narratio resume: session <session_id> has no remaining stages`
|
|
||||||
- or `narratio resume: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
|
|
||||||
|
|
||||||
Common failure cases:
|
|
||||||
- same discovery/template/validation failures as `run`.
|
|
||||||
- manifest load errors when existing manifest is unreadable.
|
|
||||||
- invalid or unknown artifact selections.
|
|
||||||
|
|
||||||
### `status`
|
|
||||||
|
|
||||||
Purpose:
|
|
||||||
- Inspect one manifest file without executing stages.
|
|
||||||
|
|
||||||
Syntax:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio status --manifest <manifest.json>
|
narratio run-stage publish <session_id> --force [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Success output includes:
|
### `clean`
|
||||||
- `session_id: <id>`
|
|
||||||
- `updated_at: <timestamp>`
|
|
||||||
- `stages:` entries (`- <stage>: <status>`)
|
|
||||||
|
|
||||||
Common failure cases:
|
|
||||||
- missing `--manifest`.
|
|
||||||
- unreadable or invalid manifest path.
|
|
||||||
|
|
||||||
### `run-stage`
|
|
||||||
|
|
||||||
Purpose:
|
|
||||||
- Execute exactly one stage.
|
|
||||||
|
|
||||||
Syntax:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run-stage [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] [--artifacts <name[,name...]>] <stage>
|
narratio clean <session_id> [--dry-run] [--clear-cache] [...common config flags]
|
||||||
|
narratio clean --all [--dry-run] [--clear-cache] [--config <pipeline.yml>]
|
||||||
```
|
```
|
||||||
|
|
||||||
Success output:
|
Behavior:
|
||||||
- `narratio run-stage: stage=<name> executed=<n> skipped=<n> force=<true|false>; manifest=<path>`
|
|
||||||
|
|
||||||
`--artifacts` behavior:
|
- session mode removes:
|
||||||
- accepted only when `<stage>` is `analyze`.
|
- `{workspace.root}/work/{campaign}/{session_id}`
|
||||||
- names are normalized (trimmed, deduplicated, sorted).
|
- `{spool.root}/{campaign}/{session_id}`
|
||||||
- unknown configured artifact keys fail.
|
- `--all` removes:
|
||||||
|
- `{workspace.root}/work/*`
|
||||||
|
- direct children under `{spool.root}`
|
||||||
|
- cache remains unless `--clear-cache` is provided.
|
||||||
|
|
||||||
Common failure cases:
|
### `session plan`
|
||||||
- missing stage positional arg.
|
|
||||||
- unknown stage name.
|
|
||||||
- using `--artifacts` with any non-`analyze` stage.
|
|
||||||
|
|
||||||
### `restore`
|
|
||||||
|
|
||||||
Purpose:
|
|
||||||
- Restore durable session state (`manifest.json`, `transcripts/**`, `artifacts/**`, and optional `audio/**`) from the committed remote archive current state.
|
|
||||||
|
|
||||||
Syntax:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio restore [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--dry-run] [--force] [--include-audio]
|
narratio session plan <session_id> [--force] [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Success output (dry-run):
|
Validates config, prepares local workdir layout, and prints run/skip decisions for each stage.
|
||||||
- `Restore plan for <campaign>/<session_id>`
|
|
||||||
- `Remote run: <run_id>`
|
|
||||||
- `Would download: <n>`
|
|
||||||
- `Would skip unchanged: <n>`
|
|
||||||
- `Conflicts: <n>`
|
|
||||||
|
|
||||||
Success output (non-dry-run):
|
### `session validate`
|
||||||
- `Restored session archive for <campaign>/<session_id>`
|
|
||||||
- `Remote run: <run_id>`
|
|
||||||
- `Downloaded: <n>`
|
|
||||||
- `Skipped unchanged: <n>`
|
|
||||||
- `Conflicts: <n>`
|
|
||||||
|
|
||||||
Common failure cases:
|
```bash
|
||||||
- storage backend is not configured.
|
narratio session validate <session_id> [...common config flags]
|
||||||
- remote `current/run_id.txt` missing/empty.
|
```
|
||||||
- remote `current/manifest.json` missing or invalid.
|
|
||||||
- remote manifest session/campaign mismatch.
|
Read-only preflight checks for config validity, required inputs, audio mode, previous-session requirements, publish outputs, and effective locks.
|
||||||
- local conflicts without `--force`.
|
|
||||||
- session lock conflict.
|
### `session status`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session status <session_id> [...common config flags]
|
||||||
|
```
|
||||||
|
|
||||||
|
Prints local manifest state and, when storage is available, remote current-state and published-output status.
|
||||||
|
|
||||||
|
### `session init`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session init <session_id> --output ./session.yml [options]
|
||||||
|
narratio session init <session_id> --remote [options]
|
||||||
|
```
|
||||||
|
|
||||||
|
Required target selection:
|
||||||
|
|
||||||
|
- exactly one of:
|
||||||
|
- `--output <path>`
|
||||||
|
- `--remote`
|
||||||
|
|
||||||
|
Options:
|
||||||
|
|
||||||
|
- `--config <pipeline.yml>`
|
||||||
|
- `--campaign <id>` or `--campaign-file <campaign.yml>`
|
||||||
|
- `--previous-session-id <id>`
|
||||||
|
- `--date <YYYY-MM-DD>`
|
||||||
|
- `--title <text>`
|
||||||
|
- `--audio-dir <path>`
|
||||||
|
- `--audio-s3-prefix <prefix>`
|
||||||
|
- `--force`
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- `--audio-dir` and `--audio-s3-prefix` are mutually exclusive.
|
||||||
|
- if campaign `session_template_file` is configured, `session init` renders it;
|
||||||
|
- generated session YAML must be concrete (no unresolved `{{ ... }}` placeholders).
|
||||||
|
|
||||||
|
### `session restore`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session restore <session_id> [--dry-run] [--force] [--include-audio] [...common config flags]
|
||||||
|
```
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
|
||||||
|
- discovers committed remote current state;
|
||||||
|
- plans local restores;
|
||||||
|
- writes `reports/restore-latest.json` on execution;
|
||||||
|
- blocks conflicting overwrites unless `--force` is set.
|
||||||
|
|
||||||
|
Default restore scope:
|
||||||
|
|
||||||
|
- `manifest.json`
|
||||||
|
- `transcripts/**`
|
||||||
|
- `artifacts/**`
|
||||||
|
- `previous/**` when required by configured previous-session inputs
|
||||||
|
|
||||||
|
`audio/**` is included only with `--include-audio`.
|
||||||
|
|
||||||
|
### `session artifacts`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session artifacts <session_id> [--remote] [...common config flags]
|
||||||
|
```
|
||||||
|
|
||||||
|
Lists effective built-in and configured artifact sources, publish rules, lock state, and optional remote published-state availability.
|
||||||
|
|
||||||
|
### `session locks`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session locks <session_id> [...common config flags]
|
||||||
|
narratio session locks add <session_id> <source> [--reason <text>] [--force] [...common config flags]
|
||||||
|
narratio session locks remove <session_id> <source> [...common config flags]
|
||||||
|
```
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
|
||||||
|
- list mode merges static `pipeline.publish.locks` with remote `{session_prefix}/locks.yml`;
|
||||||
|
- add/remove mutate only remote locks;
|
||||||
|
- static locks from pipeline config cannot be removed by CLI commands.
|
||||||
|
|
||||||
|
## `--artifacts` Selection Rules
|
||||||
|
|
||||||
|
- accepted on `run`, `resume`, `run-stage`, `analyze`, and `publish`;
|
||||||
|
- names must exist in `pipeline.scriptorium.artifacts`;
|
||||||
|
- empty entries are invalid;
|
||||||
|
- repeated names are deduplicated.
|
||||||
|
|
||||||
|
Effects:
|
||||||
|
|
||||||
|
- filters analyze execution to selected configured artifacts;
|
||||||
|
- filters publish rules that source `narratio.artifact.<name>`;
|
||||||
|
- does not filter built-in transcript/bounds publish sources.
|
||||||
|
|
||||||
## Common Workflows
|
## Common Workflows
|
||||||
|
|
||||||
Default-discovery run:
|
Run full pipeline:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run --session-id 2026-04-04
|
narratio run 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Run only selected analyze artifacts:
|
Dry-run restore plan:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run --session-id 2026-04-04 --artifacts session_recap,player_handout
|
narratio session restore 2026-04-04 --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
Resume with selected analyze artifacts:
|
Generate a concrete session file from template/default structure:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio resume --session-id 2026-04-04 --artifacts player_handout
|
narratio session init 2026-04-04 --output ./session.yml --date 2026-04-04 --title "Session 12"
|
||||||
```
|
```
|
||||||
|
|
||||||
Run only analyze stage with selected artifacts:
|
Force publish only:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run-stage --session-id 2026-04-04 --artifacts player_handout analyze
|
narratio publish 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Preview restore actions without writes:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio restore --session-id 2026-04-04 --dry-run
|
|
||||||
```
|
|
||||||
|
|
||||||
Restore and then force analyze:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio restore --session-id 2026-04-04
|
|
||||||
narratio run-stage --session-id 2026-04-04 --force analyze
|
|
||||||
```
|
|
||||||
|
|
||||||
## Diagnostic / Recovery Commands
|
|
||||||
|
|
||||||
Inspect stage status:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio status --manifest <manifest.json>
|
|
||||||
```
|
|
||||||
|
|
||||||
Get manifest path from previous output:
|
|
||||||
- `run`, `resume`, and `run-stage` print `manifest=<path>` on success.
|
|
||||||
|
|
||||||
## `--artifacts` and `--force`
|
|
||||||
|
|
||||||
- `--artifacts` filters which configured artifacts are executable when analyze runs.
|
|
||||||
- `--artifacts` does not imply `--force`.
|
|
||||||
- if analyze is already `succeeded` and `--force` is not set, runner-level skip still applies.
|
|
||||||
|
|||||||
377
docs/config.md
377
docs/config.md
@@ -1,152 +1,131 @@
|
|||||||
# Configuration
|
# Configuration Reference
|
||||||
|
|
||||||
## 1. Overview
|
## Purpose
|
||||||
|
|
||||||
Narratio loads two YAML files:
|
Narratio resolves three YAML documents:
|
||||||
|
|
||||||
- `pipeline.yml`: pipeline-level runtime configuration.
|
- `pipeline.yml`: pipeline/runtime settings
|
||||||
- `session.yml`: per-session metadata and input selection.
|
- `campaign.yml`: campaign identity and stable input defaults
|
||||||
|
- `session.yml`: session identity, metadata, and audio source selection
|
||||||
|
|
||||||
These commands load and validate both files before running:
|
## Discovery and Selection
|
||||||
|
|
||||||
- `narratio run`
|
### `pipeline.yml`
|
||||||
- `narratio plan`
|
|
||||||
- `narratio resume`
|
|
||||||
- `narratio run-stage`
|
|
||||||
- `narratio restore`
|
|
||||||
|
|
||||||
Behavior:
|
When `--config` is omitted, search order is:
|
||||||
|
|
||||||
- strict YAML decode is enabled (`KnownFields(true)`): unknown fields fail.
|
1. `/usr/local/etc/narratio/pipeline.yml`
|
||||||
- session templates render before session YAML decode.
|
2. `/etc/narratio/pipeline.yml`
|
||||||
- defaults are applied for optional pipeline fields.
|
|
||||||
- validation enforces required fields, value formats, and cross-field constraints.
|
|
||||||
|
|
||||||
## 2. Config file discovery
|
### `campaign.yml`
|
||||||
|
|
||||||
Pipeline config lookup for `run`, `plan`, `resume`, `run-stage`, and `restore`:
|
Selection rules:
|
||||||
|
|
||||||
- if `--config <path>` is provided, that path is used.
|
- if `--campaign-file` is set, use that path;
|
||||||
- if omitted, Narratio searches in order:
|
- else if `--campaign <id>` is set, use `{pipeline.campaigns.root}/{id}/campaign.yml`;
|
||||||
1. `/usr/local/etc/narratio/pipeline.yml`
|
- else use `{pipeline.campaigns.root}/{pipeline.campaigns.default_campaign_id}/campaign.yml`.
|
||||||
2. `/etc/narratio/pipeline.yml`
|
|
||||||
- first existing file wins.
|
|
||||||
|
|
||||||
## 3. Session file discovery and templating
|
### `session.yml`
|
||||||
|
|
||||||
Session config lookup for `run`, `plan`, `resume`, `run-stage`, and `restore`:
|
When `--session` is omitted, local search order is:
|
||||||
|
|
||||||
- if `--session <path>` is provided, that path is used.
|
1. `/usr/local/etc/narratio/session.yml`
|
||||||
- if omitted, Narratio searches in order:
|
2. `/etc/narratio/session.yml`
|
||||||
1. `./session.yml`
|
|
||||||
2. `/usr/local/etc/narratio/session.yml`
|
|
||||||
3. `/etc/narratio/session.yml`
|
|
||||||
- first existing file wins.
|
|
||||||
|
|
||||||
Template behavior:
|
If local session discovery fails and a `session_id` is known, Narratio attempts remote session loading from:
|
||||||
|
|
||||||
- supported placeholders:
|
- `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`
|
||||||
- `{{session_id}}`
|
|
||||||
- `{{ session_id }}`
|
|
||||||
- `--session-id <value>` supplies the placeholder value.
|
|
||||||
- unresolved placeholders fail load.
|
|
||||||
- if rendered `session_id` mismatches `--session-id`, load fails.
|
|
||||||
|
|
||||||
## 4. Minimal pipeline config
|
using configured object storage.
|
||||||
|
|
||||||
|
## Validation and Merge Rules
|
||||||
|
|
||||||
|
- YAML decode is strict (`KnownFields(true)`): unknown fields fail load.
|
||||||
|
- Session files must be concrete; unresolved `{{ ... }}` placeholders fail load.
|
||||||
|
- Pipeline defaults are applied before validation.
|
||||||
|
- Campaign and session identities must agree.
|
||||||
|
- Stable files (`speakers_file`, `autocorrect_file`, `glossary_file`) resolve from session overrides when provided, otherwise from campaign defaults.
|
||||||
|
- Exactly one audio mode must be configured in session input:
|
||||||
|
- local (`audio_dir` or `audio_files`), or
|
||||||
|
- S3 (`audio_s3.prefix`).
|
||||||
|
|
||||||
|
## Minimal Working Configuration
|
||||||
|
|
||||||
|
`pipeline.yml`
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
|
campaigns:
|
||||||
|
root: /usr/local/share/narratio/campaigns
|
||||||
|
default_campaign_id: sample-campaign
|
||||||
whisperx:
|
whisperx:
|
||||||
transcribe_url: "https://transcription.example.com/transcribe"
|
transcribe_url: https://transcription.example.com/transcribe
|
||||||
```
|
```
|
||||||
|
|
||||||
Why this is sufficient:
|
`campaign.yml`
|
||||||
|
|
||||||
- `whisperx.transcribe_url` is required.
|
|
||||||
- `workspace.root` defaults to `/var/lib/narratio`.
|
|
||||||
- optional sections (`seriatim`, `audita`, `archive`, `scriptorium`, `trim`, `normalize`, etc.) receive defaults or stay inactive.
|
|
||||||
|
|
||||||
## 5. Minimal session template
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
session_id: "{{ session_id }}"
|
campaign_id: sample-campaign
|
||||||
campaign: sample-campaign
|
inputs:
|
||||||
|
speakers_file: ./speakers.yml
|
||||||
|
autocorrect_file: ./autocorrect.yml
|
||||||
|
glossary_file: ./glossary.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
`session.yml` (local audio)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
session_id: 2026-05-03
|
||||||
inputs:
|
inputs:
|
||||||
audio_dir: ./audio
|
audio_dir: ./audio
|
||||||
speakers_file: ./examples/speakers.yml
|
|
||||||
autocorrect_file: ./examples/autocorrect.yml
|
|
||||||
glossary_file: ./examples/glossary.yml
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Usage:
|
## Secrets Handling
|
||||||
|
|
||||||
```bash
|
- Do not place raw secrets in YAML.
|
||||||
narratio run --config /path/to/pipeline.yml --session ./session.yml --session-id 2026-05-03
|
- Use env var names in config (for example `pipeline.audita.llm_api_key_env`).
|
||||||
```
|
- Optionally load env files from `pipeline.secrets.env_dir`.
|
||||||
|
- Commands that need storage/auth load filesystem secrets before constructing adapters.
|
||||||
|
|
||||||
## 6. Production-oriented config
|
## Publish Configuration Summary
|
||||||
|
|
||||||
|
Publish rules live under `pipeline.publish`.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
workspace:
|
publish:
|
||||||
root: /var/lib/narratio/workspace
|
|
||||||
cleanup_after_archive: true
|
|
||||||
|
|
||||||
storage:
|
|
||||||
backend: s3
|
|
||||||
s3:
|
|
||||||
bucket: my-dnd-archive
|
|
||||||
root_prefix: dnd
|
|
||||||
region: us-east-1
|
|
||||||
access_key_id_env: OBJECT_STORAGE_KEY_ID
|
|
||||||
secret_access_key_env: OBJECT_STORAGE_KEY
|
|
||||||
|
|
||||||
spool:
|
|
||||||
root: /var/spool/narratio
|
|
||||||
delete_audio_after_archive: true
|
|
||||||
|
|
||||||
archive:
|
|
||||||
enabled: true
|
enabled: true
|
||||||
upload_run: true
|
upload_run: true
|
||||||
promote_artifacts:
|
outputs:
|
||||||
- source: narratio.transcript.trimmed
|
- source: narratio.transcript.final_trimmed
|
||||||
dest: transcripts/trimmed.json
|
dest: transcripts/final.trimmed.json
|
||||||
required: true
|
required: true
|
||||||
- source: narratio.artifact.session_recap
|
- source: narratio.artifact.session_recap
|
||||||
dest: artifacts/session_recap.md
|
dest: artifacts/session_recap.md
|
||||||
required: true
|
required: true
|
||||||
|
locks:
|
||||||
whisperx:
|
- source: narratio.artifact.session_recap
|
||||||
transcribe_url: "https://transcription.example.com/transcribe"
|
reason: manual post-publish edits
|
||||||
|
|
||||||
scriptorium:
|
|
||||||
artifacts:
|
|
||||||
session_recap:
|
|
||||||
enabled: true
|
|
||||||
prompt_id: dnd.session_recap
|
|
||||||
output_path: artifacts/session_recap.md
|
|
||||||
inputs:
|
|
||||||
transcript:
|
|
||||||
source: narratio.transcript.trimmed
|
|
||||||
required: true
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Operational notes:
|
Rules:
|
||||||
|
|
||||||
- archive promotion is explicit and source-based via `archive.promote_artifacts`.
|
- `outputs[].source` is required.
|
||||||
- `source` is required; `dest` is optional and derived when omitted.
|
- `outputs[].dest` may be omitted when derivable from source.
|
||||||
- Narratio does not auto-promote all generated analyze artifacts.
|
- `outputs[].required` defaults to `true`.
|
||||||
- `restore` reads the same config/session inputs and restore scope is bounded by committed archive current state.
|
- static locks (`pipeline.publish.locks`) merge with remote locks (`{session_prefix}/locks.yml`), with static locks taking precedence on duplicates.
|
||||||
|
|
||||||
## 7. Full pipeline reference
|
## Full Schema
|
||||||
|
|
||||||
| Path | Type | Required | Default |
|
### Pipeline
|
||||||
|
|
||||||
|
| Field | Type | Required | Default / Rule |
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
|
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
|
||||||
| `pipeline.workspace.cleanup_after_archive` | bool | No | `false` |
|
| `pipeline.workspace.cleanup_after_publish` | bool | No | `false` |
|
||||||
| `pipeline.secrets.env_dir` | string | Conditional | none |
|
| `pipeline.campaigns.root` | string | No | `/usr/local/share/narratio/campaigns` |
|
||||||
|
| `pipeline.campaigns.default_campaign_id` | string | No | empty |
|
||||||
|
| `pipeline.secrets.env_dir` | string | No | empty |
|
||||||
| `pipeline.storage.backend` | string | No | empty |
|
| `pipeline.storage.backend` | string | No | empty |
|
||||||
| `pipeline.storage.bucket` | string | No | empty |
|
| `pipeline.storage.s3.bucket` | string | Conditional | required for S3 session-audio and for publish upload when backend is `s3` |
|
||||||
| `pipeline.storage.prefix` | string | No | empty |
|
|
||||||
| `pipeline.storage.s3.bucket` | string | Conditional | empty |
|
|
||||||
| `pipeline.storage.s3.root_prefix` | string | No | `dnd` |
|
| `pipeline.storage.s3.root_prefix` | string | No | `dnd` |
|
||||||
| `pipeline.storage.s3.region` | string | No | empty |
|
| `pipeline.storage.s3.region` | string | No | empty |
|
||||||
| `pipeline.storage.s3.endpoint` | string | No | empty |
|
| `pipeline.storage.s3.endpoint` | string | No | empty |
|
||||||
@@ -154,21 +133,26 @@ Operational notes:
|
|||||||
| `pipeline.storage.s3.access_key_id_env` | string | No | `OBJECT_STORAGE_KEY_ID` |
|
| `pipeline.storage.s3.access_key_id_env` | string | No | `OBJECT_STORAGE_KEY_ID` |
|
||||||
| `pipeline.storage.s3.secret_access_key_env` | string | No | `OBJECT_STORAGE_KEY` |
|
| `pipeline.storage.s3.secret_access_key_env` | string | No | `OBJECT_STORAGE_KEY` |
|
||||||
| `pipeline.spool.root` | string | No | `/var/spool/narratio` |
|
| `pipeline.spool.root` | string | No | `/var/spool/narratio` |
|
||||||
| `pipeline.spool.delete_audio_after_archive` | bool | No | `false` |
|
| `pipeline.spool.delete_audio_after_publish` | bool | No | `false` |
|
||||||
| `pipeline.archive.enabled` | bool | No | `true` |
|
| `pipeline.cache.root` | string | No | `/var/cache/narratio` |
|
||||||
| `pipeline.archive.upload_run` | bool | No | `true` |
|
| `pipeline.cache.s3_audio` | bool | No | `true` |
|
||||||
| `pipeline.archive.promote_artifacts[]` | list | No | trimmed transcript rule |
|
| `pipeline.publish.enabled` | bool | No | `true` |
|
||||||
| `pipeline.archive.promote_artifacts[].source` | string | Yes (per rule) | none |
|
| `pipeline.publish.upload_run` | bool | No | `true` |
|
||||||
| `pipeline.archive.promote_artifacts[].dest` | string | No | derived from source |
|
| `pipeline.publish.outputs[]` | list | No | defaults to final trimmed transcript output |
|
||||||
| `pipeline.archive.promote_artifacts[].required` | bool | No | `true` |
|
| `pipeline.publish.outputs[].source` | string | Yes (per rule) | must reference built-in or configured artifact source |
|
||||||
| `pipeline.whisperx.transcribe_url` | string | Yes | none |
|
| `pipeline.publish.outputs[].dest` | string | Conditional | derived if omitted and source supports derivation |
|
||||||
|
| `pipeline.publish.outputs[].required` | bool | No | `true` |
|
||||||
|
| `pipeline.publish.locks[]` | list | No | empty |
|
||||||
|
| `pipeline.publish.locks[].source` | string | Yes (per lock) | must reference supported publish source |
|
||||||
|
| `pipeline.publish.locks[].reason` | string | No | empty |
|
||||||
|
| `pipeline.whisperx.transcribe_url` | string | Yes | valid URL |
|
||||||
| `pipeline.whisperx.language` | string | No | `en` |
|
| `pipeline.whisperx.language` | string | No | `en` |
|
||||||
| `pipeline.whisperx.timeout` | duration string | No | `30m` |
|
| `pipeline.whisperx.timeout` | duration | No | `30m` |
|
||||||
| `pipeline.whisperx.retries` | int | No | `3` |
|
| `pipeline.whisperx.retries` | int | No | `3` |
|
||||||
| `pipeline.whisperx.retry_delay` | duration string | No | `2s` |
|
| `pipeline.whisperx.retry_delay` | duration | No | `2s` |
|
||||||
| `pipeline.whisperx.concurrency` | int | No | `2` |
|
| `pipeline.whisperx.concurrency` | int | No | `2` |
|
||||||
| `pipeline.seriatim.binary` | string | No | `seriatim` |
|
| `pipeline.seriatim.binary` | string | No | `seriatim` |
|
||||||
| `pipeline.seriatim.timeout` | duration string | No | `10m` |
|
| `pipeline.seriatim.timeout` | duration | No | `10m` |
|
||||||
| `pipeline.seriatim.output_schema` | string | No | `seriatim-intermediate` |
|
| `pipeline.seriatim.output_schema` | string | No | `seriatim-intermediate` |
|
||||||
| `pipeline.seriatim.coalesce_gap` | float | No | `3.0` |
|
| `pipeline.seriatim.coalesce_gap` | float | No | `3.0` |
|
||||||
| `pipeline.seriatim.report` | bool | No | `true` |
|
| `pipeline.seriatim.report` | bool | No | `true` |
|
||||||
@@ -177,7 +161,7 @@ Operational notes:
|
|||||||
| `pipeline.seriatim.env.backchannel_max_duration` | float | No | unset |
|
| `pipeline.seriatim.env.backchannel_max_duration` | float | No | unset |
|
||||||
| `pipeline.seriatim.env.filler_max_duration` | float | No | unset |
|
| `pipeline.seriatim.env.filler_max_duration` | float | No | unset |
|
||||||
| `pipeline.audita.binary` | string | No | `audita` |
|
| `pipeline.audita.binary` | string | No | `audita` |
|
||||||
| `pipeline.audita.timeout` | duration string | No | `3h` |
|
| `pipeline.audita.timeout` | duration | No | `3h` |
|
||||||
| `pipeline.audita.llm_api_key_env` | string | No | empty |
|
| `pipeline.audita.llm_api_key_env` | string | No | empty |
|
||||||
| `pipeline.audita.modules[]` | list[string] | No | empty |
|
| `pipeline.audita.modules[]` | list[string] | No | empty |
|
||||||
| `pipeline.audita.base_url` | string | No | empty |
|
| `pipeline.audita.base_url` | string | No | empty |
|
||||||
@@ -191,140 +175,89 @@ Operational notes:
|
|||||||
| `pipeline.audita.output_schema` | string | No | empty |
|
| `pipeline.audita.output_schema` | string | No | empty |
|
||||||
| `pipeline.audita.work_dir_retention` | string | No | empty |
|
| `pipeline.audita.work_dir_retention` | string | No | empty |
|
||||||
| `pipeline.audita.report` | bool | No | `true` |
|
| `pipeline.audita.report` | bool | No | `true` |
|
||||||
| `pipeline.normalize.output_path` | string | No | `transcripts/normalized.json` |
|
| `pipeline.normalize.output_path` | string | No | `transcripts/final.json` |
|
||||||
| `pipeline.normalize.output_schema` | string | No | `seriatim-intermediate` |
|
| `pipeline.normalize.output_schema` | string | No | `seriatim-intermediate` |
|
||||||
| `pipeline.normalize.report` | bool | No | `true` |
|
| `pipeline.normalize.report` | bool | No | `true` |
|
||||||
| `pipeline.trim.enabled` | bool | No | `false` |
|
| `pipeline.trim.enabled` | bool | No | `false` |
|
||||||
| `pipeline.trim.output_path` | string | Conditional | none |
|
| `pipeline.trim.output_path` | string | Conditional | required when trim enabled |
|
||||||
| `pipeline.trim.bounds.prompt_id` | string | Conditional | none |
|
| `pipeline.trim.bounds.prompt_id` | string | Conditional | required when trim enabled |
|
||||||
| `pipeline.trim.bounds.profile_id` | string | No | empty |
|
| `pipeline.trim.bounds.profile_id` | string | No | empty |
|
||||||
| `pipeline.trim.bounds.transcript_input_name` | string | Conditional | none |
|
| `pipeline.trim.bounds.transcript_input_name` | string | Conditional | required when trim enabled |
|
||||||
| `pipeline.trim.bounds.output_path` | string | Conditional | none |
|
| `pipeline.trim.bounds.output_path` | string | Conditional | required when trim enabled |
|
||||||
| `pipeline.trim.bounds.timeout` | duration string | No | `10m` |
|
| `pipeline.trim.bounds.timeout` | duration | No | `10m` |
|
||||||
| `pipeline.trim.bounds.render_debug` | bool | No | `false` |
|
| `pipeline.trim.bounds.render_debug` | bool | No | `false` |
|
||||||
| `pipeline.trim.bounds.render_output_path` | string | Conditional | none |
|
| `pipeline.trim.bounds.render_output_path` | string | Conditional | required when `render_debug` is true |
|
||||||
| `pipeline.trim.seriatim.report` | bool | No | `false` |
|
| `pipeline.trim.seriatim.report` | bool | No | `false` |
|
||||||
| `pipeline.scriptorium.binary` | string | No | `scriptorium` |
|
| `pipeline.scriptorium.binary` | string | No | `scriptorium` |
|
||||||
| `pipeline.scriptorium.config_path` | string | No | empty |
|
| `pipeline.scriptorium.config_path` | string | No | empty |
|
||||||
| `pipeline.scriptorium.timeout` | duration string | No | `10m` |
|
| `pipeline.scriptorium.timeout` | duration | No | `10m` |
|
||||||
| `pipeline.scriptorium.render_debug` | bool | No | `false` |
|
| `pipeline.scriptorium.render_debug` | bool | No | `false` |
|
||||||
| `pipeline.scriptorium.artifacts` | map | No | empty |
|
| `pipeline.scriptorium.artifacts` | map | No | empty |
|
||||||
| `pipeline.scriptorium.artifacts.<name>.enabled` | bool | No | `false` |
|
|
||||||
| `pipeline.scriptorium.artifacts.<name>.depends_on[]` | list[string] | No | empty |
|
|
||||||
| `pipeline.scriptorium.artifacts.<name>.render_debug` | bool | No | unset |
|
|
||||||
| `pipeline.scriptorium.artifacts.<name>.prompt_id` | string | Conditional | none |
|
|
||||||
| `pipeline.scriptorium.artifacts.<name>.profile_id` | string | No | empty |
|
|
||||||
| `pipeline.scriptorium.artifacts.<name>.output_path` | string | Conditional | none |
|
|
||||||
| `pipeline.scriptorium.artifacts.<name>.timeout` | duration string | No | empty |
|
|
||||||
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` | string | Conditional | none |
|
|
||||||
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.artifact` | string | No | empty |
|
|
||||||
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.path` | string | No | empty |
|
|
||||||
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.required` | bool | No | `false` |
|
|
||||||
| `pipeline.scriptorium.artifacts.<name>.vars.<key>` | map value | No | empty |
|
|
||||||
| `pipeline.analyzer.binary_path` | string | No | empty |
|
|
||||||
| `pipeline.analyzer.timeout` | duration string | No | empty |
|
|
||||||
| `pipeline.analyzer.artifacts.output_dir` | string | No | empty |
|
|
||||||
| `pipeline.analyzer.artifacts.types[]` | list[string] | No | empty |
|
|
||||||
| `pipeline.notification.backend` | string | No | empty |
|
| `pipeline.notification.backend` | string | No | empty |
|
||||||
| `pipeline.notification.recipient` | string | No | empty |
|
| `pipeline.notification.recipient` | string | No | empty |
|
||||||
| `pipeline.notification.timeout` | duration string | No | empty |
|
| `pipeline.notification.timeout` | duration | No | `30s` |
|
||||||
|
|
||||||
Scriptorium artifact-key and dependency rules:
|
### Scriptorium Artifact Entries
|
||||||
|
|
||||||
- artifact keys must match `^[a-z][a-z0-9_]*$`.
|
For each `pipeline.scriptorium.artifacts.<name>`:
|
||||||
- enabled artifacts require `prompt_id` and `output_path`.
|
|
||||||
- `output_path` must be relative, traversal-safe, and under `artifacts/`.
|
|
||||||
- configured artifact input sources use `narratio.artifact.<name>`.
|
|
||||||
- if input source references `narratio.artifact.<name>`, artifact `<name>` must exist and must be listed in `depends_on`.
|
|
||||||
- every `depends_on` entry must be a configured artifact key.
|
|
||||||
- self-dependency is rejected.
|
|
||||||
- enabled dependency cycles are rejected.
|
|
||||||
- any artifact referenced by `depends_on` or `narratio.artifact.<name>` source must define `output_path` (even if not enabled).
|
|
||||||
|
|
||||||
Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:
|
| Field | Type | Required | Rule |
|
||||||
|
|
||||||
- `previous_session_artifact`
|
|
||||||
- `narratio.transcript.merged`
|
|
||||||
- `narratio.transcript.polished`
|
|
||||||
- `narratio.transcript.full`
|
|
||||||
- `narratio.transcript.trimmed`
|
|
||||||
- `narratio.bounds.session`
|
|
||||||
- `narratio.artifact.<configured_artifact_key>`
|
|
||||||
|
|
||||||
`pipeline.archive.promote_artifacts[].source` values:
|
|
||||||
|
|
||||||
- `narratio.transcript.merged`
|
|
||||||
- `narratio.transcript.polished`
|
|
||||||
- `narratio.transcript.full`
|
|
||||||
- `narratio.transcript.trimmed`
|
|
||||||
- `narratio.bounds.session`
|
|
||||||
- `narratio.artifact.<configured_artifact_key>`
|
|
||||||
|
|
||||||
Archive promotion destination rules:
|
|
||||||
|
|
||||||
- `dest` must be a clean relative path (not absolute, no traversal).
|
|
||||||
- duplicate `dest` values are rejected.
|
|
||||||
- if `dest` is omitted:
|
|
||||||
- built-in sources derive their canonical destination path;
|
|
||||||
- configured sources derive from `pipeline.scriptorium.artifacts.<name>.output_path`;
|
|
||||||
- derivation failure is a config validation error.
|
|
||||||
|
|
||||||
Restore-related implications:
|
|
||||||
|
|
||||||
- restore remote identity requires archive S3 identity to resolve (`pipeline.storage.s3.bucket` and session prefix derivation inputs).
|
|
||||||
- restore scope considers only committed current state and durable paths (`manifest.json`, `transcripts/**`, `artifacts/**`, optional `audio/**`).
|
|
||||||
|
|
||||||
## 8. Full session reference
|
|
||||||
|
|
||||||
| Path | Type | Required | Default |
|
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| `session.session_id` | string | Yes | none |
|
| `enabled` | bool | No | `false` if omitted |
|
||||||
| `session.campaign` | string | Yes | none |
|
| `depends_on[]` | list[string] | No | must reference configured artifact keys; no self-reference; enabled graph must be acyclic |
|
||||||
| `session.date` | string | No | empty |
|
| `render_debug` | bool | No | per-artifact override |
|
||||||
| `session.title` | string | No | empty |
|
| `prompt_id` | string | Conditional | required when artifact is enabled |
|
||||||
| `session.inputs.audio_dir` | string | Conditional | empty |
|
| `profile_id` | string | No | empty |
|
||||||
| `session.inputs.audio_files[]` | list[string] | Conditional | empty |
|
| `output_path` | string | Conditional | required when enabled; also required when referenced by publish/output/input rules |
|
||||||
| `session.inputs.audio_s3.prefix` | string | Conditional | none |
|
| `timeout` | duration | No | artifact override |
|
||||||
| `session.inputs.speakers_file` | string | Yes | none |
|
| `inputs` | map | No | input key names must be non-empty |
|
||||||
| `session.inputs.autocorrect_file` | string | Yes | none |
|
| `vars` | map | No | values must be string or bool |
|
||||||
| `session.inputs.glossary_file` | string | Yes | none |
|
|
||||||
|
|
||||||
Audio-source rule:
|
For each artifact input `pipeline.scriptorium.artifacts.<name>.inputs.<input_name>`:
|
||||||
|
|
||||||
- configure exactly one mode:
|
| Field | Type | Required | Rule |
|
||||||
- `audio_dir`, or
|
| --- | --- | --- | --- |
|
||||||
- `audio_files` (at least one), or
|
| `source` | string | Yes | built-in runtime source, `narratio.artifact.<name>`, or `narratio.previous_session.artifact.<name>` |
|
||||||
- `audio_s3.prefix`
|
| `artifact` | string | No | optional passthrough adapter field |
|
||||||
- `audio_s3` cannot be combined with local audio fields.
|
| `path` | string | No | optional passthrough adapter field |
|
||||||
|
| `required` | bool | No | optional input requirement |
|
||||||
|
|
||||||
## 9. Secrets
|
### Campaign
|
||||||
|
|
||||||
Narratio supports filesystem-based secret injection via `pipeline.secrets.env_dir`.
|
| Field | Type | Required | Notes |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `campaign_id` | string | Yes | canonical campaign identity |
|
||||||
|
| `session_template_file` | string | No | used by `session init` when set |
|
||||||
|
| `inputs.speakers_file` | string | Yes | stable input default |
|
||||||
|
| `inputs.autocorrect_file` | string | Yes | stable input default |
|
||||||
|
| `inputs.glossary_file` | string | Yes | stable input default |
|
||||||
|
|
||||||
Behavior:
|
### Session
|
||||||
|
|
||||||
- `env_dir` may be absolute or relative.
|
| Field | Type | Required in session file | Notes |
|
||||||
- relative `env_dir` resolves from current working directory.
|
| --- | --- | --- | --- |
|
||||||
- files with valid env-var names (`[A-Za-z_][A-Za-z0-9_]*`) are loaded.
|
| `session_id` | string | Yes | must match CLI session target when provided |
|
||||||
- values are loaded from file contents with trailing newline trimming.
|
| `previous_session_id` | string | No | must not equal `session_id` |
|
||||||
- existing process env vars are preserved.
|
| `campaign` | string | No | filled from `campaign_id` during resolve if omitted |
|
||||||
- invalid names and subdirectories are skipped.
|
| `date` | string | No | metadata |
|
||||||
- missing/unreadable `env_dir` fails command execution.
|
| `title` | string | No | metadata |
|
||||||
|
| `inputs.speakers_file` | string | No | overrides campaign stable input |
|
||||||
|
| `inputs.autocorrect_file` | string | No | overrides campaign stable input |
|
||||||
|
| `inputs.glossary_file` | string | No | overrides campaign stable input |
|
||||||
|
| `inputs.audio_dir` | string | Conditional | local audio mode |
|
||||||
|
| `inputs.audio_files[]` | list[string] | Conditional | local audio mode |
|
||||||
|
| `inputs.audio_s3.prefix` | string | Conditional | S3 audio mode |
|
||||||
|
|
||||||
Guidance:
|
Audio rules:
|
||||||
|
|
||||||
- do not put secret values directly in YAML.
|
- configure local mode (`audio_dir` or `audio_files`) or S3 mode (`audio_s3.prefix`), not both.
|
||||||
- configure env var names in config and provide values via env/secrets files.
|
|
||||||
|
|
||||||
## 10. Examples
|
## Maintained Examples
|
||||||
|
|
||||||
Maintained examples:
|
|
||||||
|
|
||||||
- `examples/pipeline.minimal.yml`
|
- `examples/pipeline.minimal.yml`
|
||||||
- `examples/pipeline.production.yml`
|
- `examples/pipeline.production.yml`
|
||||||
- `examples/pipeline.full.annotated.yml`
|
- `examples/pipeline.full.annotated.yml`
|
||||||
- `examples/session.template.yml`
|
- `examples/campaigns/sample-campaign/campaign.yml`
|
||||||
- `examples/session.local-audio.yml`
|
- `examples/session.local-audio.yml`
|
||||||
- `examples/session.s3-audio.yml`
|
- `examples/session.s3-audio.yml`
|
||||||
|
- `examples/session.template.yml`
|
||||||
These examples are validated by `internal/config` tests.
|
|
||||||
|
|||||||
@@ -1,15 +1,19 @@
|
|||||||
# Integration Documentation Index
|
# Integrations Index
|
||||||
|
|
||||||
## Audience
|
## Audience
|
||||||
Developers and LLM coding agents changing Narratio's external integration contracts.
|
Developers and coding agents changing Narratio's external integration boundaries.
|
||||||
|
|
||||||
## Scope
|
## Scope
|
||||||
Implemented-only reference notes for the external systems Narratio currently integrates with.
|
`docs/integrations/` is the implementation-level reference for downstream tool adapter contracts.
|
||||||
|
|
||||||
## Integration Docs
|
These docs cover what Narratio expects from external tools and what each adapter guarantees back to stage code.
|
||||||
- `audita.md`: Audita adapter invocation and validation contract.
|
|
||||||
- `seriatim.md`: Seriatim normalize/merge/trim adapter contract.
|
|
||||||
- `scriptorium.md`: Scriptorium run/render adapter contract.
|
|
||||||
|
|
||||||
## Canonical Owner
|
## Integration Contracts
|
||||||
`docs/integrations/` is the canonical home for external integration reference notes per `docs/documentation/policy.md`.
|
- `audita.md`: transcript polishing adapter (`audita process`).
|
||||||
|
- `seriatim.md`: merge/normalize/trim adapter (`seriatim`).
|
||||||
|
- `scriptorium.md`: artifact run/render adapter (`scriptorium run|render`).
|
||||||
|
|
||||||
|
## Related Canonical Docs
|
||||||
|
- `docs/config.md`: operator-facing configuration reference.
|
||||||
|
- `docs/internal/adapters.md`: shared adapter boundary and runner wiring.
|
||||||
|
- `docs/internal/stage-*.md`: stage-specific integration usage.
|
||||||
|
|||||||
@@ -1,66 +1,60 @@
|
|||||||
# Integration: audita
|
# Integration: Audita
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define Narratio's adapter contract for transcript polishing via Audita CLI subprocess execution.
|
Define the Audita adapter contract used by the `polish` stage.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Adapter Boundary
|
||||||
Inputs (`audita.PolishRequest`):
|
Interface:
|
||||||
- merged transcript path
|
- `audita.Runner`
|
||||||
- glossary path
|
- method: `Run(ctx, PolishRequest) (PolishResult, error)`
|
||||||
- output processed transcript path
|
|
||||||
- optional report path (required when report enabled)
|
|
||||||
- work dir
|
|
||||||
- generated config path
|
|
||||||
- stdout/stderr log paths
|
|
||||||
- optional module/model/base URL and concurrency knobs
|
|
||||||
|
|
||||||
Outputs (`audita.PolishResult`):
|
Primary implementation:
|
||||||
- processed transcript path
|
- `internal/adapters/audita/SubprocessRunner`
|
||||||
- optional report path
|
|
||||||
- generated config path
|
|
||||||
- stdout/stderr log paths
|
|
||||||
- exit code, duration, invoked binary
|
|
||||||
- adapter metadata
|
|
||||||
|
|
||||||
## Boundaries
|
Execution mode:
|
||||||
Owns:
|
- subprocess invocation of `audita process`
|
||||||
- Deterministic CLI argument construction for `audita process`
|
|
||||||
- Environment bridging for API credentials
|
|
||||||
- Invocation config emission
|
|
||||||
- Output validation for processed transcript and report
|
|
||||||
|
|
||||||
Does not own:
|
## Request Contract
|
||||||
- Upstream/downstream stage orchestration
|
`PolishRequest` carries:
|
||||||
- Credential sourcing policy beyond required env-var presence check
|
- required transcript/glossary/output/work-dir paths;
|
||||||
|
- optional report path (required when report mode is enabled);
|
||||||
|
- generated config and stdout/stderr log paths;
|
||||||
|
- optional module/model/base-url/config/output-schema/concurrency settings.
|
||||||
|
|
||||||
## Config Fields Used
|
## Result Contract
|
||||||
Via `pipeline.audita.*` mapped in app/stage wiring:
|
`PolishResult` returns:
|
||||||
- `binary`, `timeout`, `llm_api_key_env`, `modules`, `base_url`, `model`
|
- processed transcript path;
|
||||||
- `transcript_description`, `config_path`, `output_schema`, `work_dir_retention`
|
- optional report path;
|
||||||
- `total_llm_concurrency`, `proposal_llm_concurrency`, `validation_model`, `validation_llm_concurrency`, `report`
|
- work dir and generated-config/log paths;
|
||||||
|
- exit code, duration, binary provenance;
|
||||||
|
- adapter metadata map.
|
||||||
|
|
||||||
## External Adapters Used
|
## Validation and Failure Semantics
|
||||||
- Shared subprocess helper (`internal/adapters/subprocess`) to run CLI and capture logs.
|
Construction fails for invalid static config values, including:
|
||||||
|
- empty binary;
|
||||||
|
- non-positive timeout;
|
||||||
|
- invalid base URL;
|
||||||
|
- invalid output schema;
|
||||||
|
- invalid work-dir retention value;
|
||||||
|
- invalid concurrency values.
|
||||||
|
|
||||||
## State and Manifest Behavior
|
Run fails for:
|
||||||
- No direct manifest writes.
|
- missing required request paths;
|
||||||
- Stage-level metadata records adapter provenance and credential-present signal.
|
- missing required credential env var when configured (`llm_api_key_env`);
|
||||||
- Generated invocation YAML is written when `GeneratedConfigPath` is provided.
|
- subprocess execution failure;
|
||||||
|
- invalid processed transcript JSON (`segments` array required);
|
||||||
|
- invalid report JSON when reporting is enabled.
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
Failure results still include output/log/config/exit metadata for diagnostics.
|
||||||
- Adapter has no skip/resume logic. Stage/runner controls this.
|
|
||||||
|
|
||||||
## Failure Behavior
|
## Deterministic Behavior
|
||||||
- Constructor validation fails on invalid binary/timeout/schema/concurrency/URL values.
|
- CLI args are built from runner config + request in a fixed order.
|
||||||
- Run fails on missing required paths, missing required credential env var, subprocess errors, invalid processed JSON shape, or invalid report JSON.
|
- Generated invocation YAML (`audita.generated.v1`) is emitted when requested.
|
||||||
- Failures preserve stdout/stderr paths in returned result metadata.
|
- Manifest writes are stage-owned; adapter itself is stateless.
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
## Config Mapping
|
||||||
- `internal/adapters/audita/subprocess_test.go`
|
Config fields consumed through runner/stage wiring are under `pipeline.audita.*`.
|
||||||
- `internal/adapters/audita/fake_test.go`
|
|
||||||
- `internal/stage/polish_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
Maintained example with Audita config:
|
||||||
- Processed output must be valid JSON with top-level `segments` array.
|
- `examples/pipeline.full.annotated.yml`
|
||||||
- When report is enabled, report output must be valid JSON.
|
- `examples/pipeline.production.yml`
|
||||||
- If `llm_api_key_env` is configured, credential must be present in environment.
|
|
||||||
|
|||||||
@@ -1,64 +1,66 @@
|
|||||||
# Integration: scriptorium
|
# Integration: Scriptorium
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define Narratio's adapter contract for Scriptorium artifact generation and render-debug subprocess invocations.
|
Define the Scriptorium adapter contract used by `analyze` and trim-bounds generation in `trim`.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Adapter Boundary
|
||||||
Inputs:
|
Interface:
|
||||||
- `RunArtifactRequest`: binary, config path, prompt/profile IDs, input map, vars map, timeout, output path, logs/config paths, optional API env and working dir
|
- `scriptorium.Runner`
|
||||||
- `RenderArtifactRequest`: same core fields for render mode
|
- methods:
|
||||||
|
- `RunArtifact(ctx, RunArtifactRequest)`
|
||||||
|
- `RenderArtifact(ctx, RenderArtifactRequest)`
|
||||||
|
|
||||||
Outputs (`ArtifactResult`):
|
Primary implementation:
|
||||||
- output path
|
- `internal/adapters/scriptorium/SubprocessRunner`
|
||||||
- stdout/stderr log paths
|
|
||||||
- generated config path
|
|
||||||
- exit code and duration
|
|
||||||
- command mode (`run` or `render`)
|
|
||||||
- prompt/profile provenance
|
|
||||||
- validation failure signal
|
|
||||||
- adapter metadata
|
|
||||||
|
|
||||||
## Boundaries
|
Execution modes:
|
||||||
Owns:
|
- `scriptorium run`
|
||||||
- Deterministic CLI arg construction for `scriptorium run` and `scriptorium render`
|
- `scriptorium render`
|
||||||
- Common request validation
|
|
||||||
- Invocation config emission
|
|
||||||
- Output existence/non-empty checks
|
|
||||||
- Validation-failure mapping for run exit code 2
|
|
||||||
|
|
||||||
Does not own:
|
## Request Contract
|
||||||
- Artifact selection policy (`analyze` stage)
|
Both request types carry:
|
||||||
- Bounds semantic validation (`trim` stage)
|
- binary/config/prompt/profile IDs;
|
||||||
|
- input map and vars map;
|
||||||
|
- output path;
|
||||||
|
- timeout;
|
||||||
|
- generated config + stdout/stderr log paths;
|
||||||
|
- optional API-key env var name;
|
||||||
|
- optional working directory.
|
||||||
|
|
||||||
## Config Fields Used
|
## Result Contract
|
||||||
Via `pipeline.scriptorium.*` and stage-level artifact config:
|
`ArtifactResult` returns:
|
||||||
- `binary`, `config_path`, `timeout`, `render_debug`
|
- output/log/generated-config paths;
|
||||||
- artifact-level `prompt_id`, `profile_id`, `timeout`, `inputs`, `vars`, `output_path`
|
- exit code and duration;
|
||||||
|
- command mode (`run` or `render`);
|
||||||
|
- prompt/profile provenance;
|
||||||
|
- `ValidationFailed` marker;
|
||||||
|
- metadata map.
|
||||||
|
|
||||||
## External Adapters Used
|
## Validation and Failure Semantics
|
||||||
- Shared subprocess helper (`internal/adapters/subprocess`).
|
Request validation fails for:
|
||||||
|
- missing binary, prompt id, or output path;
|
||||||
|
- non-positive timeout;
|
||||||
|
- empty input/var names;
|
||||||
|
- empty input path values;
|
||||||
|
- missing required credential env var when `APIKeyEnv` is set.
|
||||||
|
|
||||||
## State and Manifest Behavior
|
Run behavior:
|
||||||
- No direct manifest writes.
|
- subprocess errors propagate with context;
|
||||||
- Stage metadata records adapter outputs and command mode.
|
- `run` exit code `2` is mapped to `ValidationFailed=true`;
|
||||||
- Generated invocation YAML is written when requested.
|
- successful subprocess still fails if output file is missing or empty.
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
Render behavior:
|
||||||
- Adapter has no skip/resume logic. Stage/runner controls execution.
|
- subprocess errors propagate;
|
||||||
|
- output file must exist and be non-empty.
|
||||||
|
|
||||||
## Failure Behavior
|
## Deterministic Behavior
|
||||||
- Request validation fails for missing binary/prompt/output, invalid timeout, invalid input/var names, or missing required API env var.
|
- input and var maps are sorted into deterministic `--input` and `--var` CLI args.
|
||||||
- Subprocess errors bubble with command context.
|
- generated invocation YAML (`scriptorium.generated.v1`) is emitted when requested.
|
||||||
- `run` exit code 2 is treated as `ValidationFailed=true` and surfaced as error by calling stage.
|
- adapter is stateless and does not own artifact-selection policy.
|
||||||
- Successful subprocess still fails if output file is missing/empty.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
## Config Mapping
|
||||||
- `internal/adapters/scriptorium/subprocess_test.go`
|
Config fields consumed through runner/stage wiring are under `pipeline.scriptorium.*` plus per-artifact settings under `pipeline.scriptorium.artifacts.*`.
|
||||||
- `internal/adapters/scriptorium/fake_test.go`
|
|
||||||
- `internal/stage/analyze_test.go`
|
|
||||||
- `internal/stage/trim_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
Maintained examples with Scriptorium config:
|
||||||
- Both modes require explicit timeout > 0.
|
- `examples/pipeline.full.annotated.yml`
|
||||||
- Input/var maps are sorted into deterministic CLI argument order.
|
- `examples/pipeline.production.yml`
|
||||||
- Run-mode validation failures are represented explicitly, not silently skipped.
|
|
||||||
|
|||||||
@@ -1,60 +1,56 @@
|
|||||||
# Integration: seriatim
|
# Integration: Seriatim
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define Narratio's adapter contract for merge, normalize, and trim subprocess invocations of Seriatim.
|
Define the Seriatim adapter contract used by `merge`, `normalize`, and `trim`.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Adapter Boundary
|
||||||
Inputs:
|
Interface:
|
||||||
- `MergeRequest`: raw/normalized transcript inputs, output path, optional report, speaker/autocorrect paths, logs/config
|
- `seriatim.Runner`
|
||||||
- `NormalizeRequest`: input transcript, output path, schema, optional report, timeout/log/config
|
- methods:
|
||||||
- `TrimRequest`: input transcript, output path, keep selector, timeout/log/config
|
- `Run(ctx, MergeRequest)`
|
||||||
|
- `Normalize(ctx, NormalizeRequest)`
|
||||||
|
- `Trim(ctx, TrimRequest)`
|
||||||
|
|
||||||
Outputs:
|
Primary implementation:
|
||||||
- `MergeResult`, `NormalizeResult`, `TrimResult` with output paths, logs/config paths, exit code, duration, binary provenance, and metadata.
|
- `internal/adapters/seriatim/SubprocessRunner`
|
||||||
|
|
||||||
## Boundaries
|
Execution modes:
|
||||||
Owns:
|
- `seriatim merge`
|
||||||
- Validated deterministic CLI invocation construction
|
- `seriatim normalize`
|
||||||
- Optional env tuning propagation for merge
|
- `seriatim trim`
|
||||||
- Invocation config file emission
|
|
||||||
- JSON output validation
|
|
||||||
|
|
||||||
Does not own:
|
## Request/Result Contracts
|
||||||
- Transcript input selection/promotion logic (stage-owned)
|
- `MergeRequest`/`MergeResult`: multi-input merge to base transcript, optional report.
|
||||||
- Bounds computation (scriptorium/trim-stage-owned)
|
- `NormalizeRequest`/`NormalizeResult`: transcript normalization with explicit schema.
|
||||||
|
- `TrimRequest`/`TrimResult`: transcript trimming with required keep selector.
|
||||||
|
|
||||||
## Config Fields Used
|
Results include output/log/config paths, timing, exit code, and metadata.
|
||||||
Via `pipeline.seriatim.*` mapped in app/stage wiring:
|
|
||||||
- `binary`, `timeout`, `output_schema`, `coalesce_gap`, `report`
|
|
||||||
- `env.overlap_word_run_gap`
|
|
||||||
- `env.overlap_word_run_reorder_window`
|
|
||||||
- `env.backchannel_max_duration`
|
|
||||||
- `env.filler_max_duration`
|
|
||||||
|
|
||||||
## External Adapters Used
|
## Validation and Failure Semantics
|
||||||
- Shared subprocess helper (`internal/adapters/subprocess`).
|
Runner construction validates:
|
||||||
|
- binary presence;
|
||||||
|
- timeout > 0;
|
||||||
|
- supported output schema (`seriatim-minimal|seriatim-intermediate|seriatim-full`);
|
||||||
|
- non-negative coalesce gap.
|
||||||
|
|
||||||
## State and Manifest Behavior
|
Invocation fails on:
|
||||||
- No direct manifest writes.
|
- missing required request paths/inputs;
|
||||||
- Stage metadata consumes adapter result fields and preserves generated config/log references.
|
- invalid normalize schema override;
|
||||||
|
- subprocess failure;
|
||||||
|
- invalid JSON outputs;
|
||||||
|
- missing `segments` array for normalize/trim transcript outputs.
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
When report paths are provided/enabled, report files must parse as JSON.
|
||||||
- Adapter has no skip/resume logic. Runner controls stage execution.
|
|
||||||
|
|
||||||
## Failure Behavior
|
## Deterministic Behavior
|
||||||
- Constructor fails for invalid binary/timeout/output-schema/coalesce-gap.
|
- argument ordering is deterministic per command construction.
|
||||||
- Merge fails on missing output path/inputs/report path (if enabled), subprocess errors, invalid merged output JSON, invalid report JSON.
|
- merge env overrides are explicit (`SERIATIM_*`) and only emitted when configured.
|
||||||
- Normalize fails on missing input/output, invalid schema, subprocess errors, invalid normalized output JSON shape, invalid report JSON.
|
- generated invocation YAML (`seriatim.generated.v1`) is emitted when requested.
|
||||||
- Trim fails on missing input/output/keep selector, subprocess errors, invalid trimmed output JSON shape.
|
- adapter does not write manifests or choose stage inputs.
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
## Config Mapping
|
||||||
- `internal/adapters/seriatim/subprocess_test.go`
|
Config fields consumed through runner/stage wiring are under `pipeline.seriatim.*`.
|
||||||
- `internal/adapters/seriatim/fake_test.go`
|
|
||||||
- `internal/stage/merge_test.go`
|
|
||||||
- `internal/stage/normalize_test.go`
|
|
||||||
- `internal/stage/trim_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
Maintained examples with Seriatim config:
|
||||||
- Supported output schemas are limited to `seriatim-minimal`, `seriatim-intermediate`, `seriatim-full`.
|
- `examples/pipeline.full.annotated.yml`
|
||||||
- Normalize/trim outputs must include `segments` arrays.
|
- `examples/pipeline.production.yml`
|
||||||
- Merge/normalize/trim all route through deterministic subprocess invocation.
|
|
||||||
|
|||||||
@@ -1,29 +1,43 @@
|
|||||||
# Internal Documentation Index
|
# Internal Documentation Index
|
||||||
|
|
||||||
## Audience
|
## Audience
|
||||||
Developers and LLM coding agents changing Narratio internals.
|
Developers and coding agents changing Narratio internals.
|
||||||
|
|
||||||
## Scope
|
## Scope
|
||||||
Implementation-accurate contracts for workspace/state, manifests, stages, artifact resolution, adapter boundaries, and restore command behavior.
|
`docs/internal/` documents implemented internal contracts: stage boundaries, manifest/state behavior, artifact resolution, restore behavior, storage boundaries, and workspace invariants.
|
||||||
|
|
||||||
## Component Docs
|
User and operator behavior belongs in:
|
||||||
- `adapters.md`: external adapter map, runtime wiring, and boundary ownership.
|
- `docs/cli.md`
|
||||||
- `storage.md`: remote storage backend contracts and object-store invariants.
|
- `docs/config.md`
|
||||||
- `manifest.md`: session/run manifest schemas, lifecycle transitions, and persistence semantics.
|
- `docs/operations.md`
|
||||||
- `artifacts.md`: built-in artifact registry, runtime artifact catalog, and source-resolution behavior.
|
- `docs/troubleshooting.md`
|
||||||
- `workspace.md`: local state model, manifests, run-local layout, promotion, and cleanup invariants.
|
|
||||||
- `command-restore.md`: restore command discovery/planning/execution/reporting contract.
|
|
||||||
- `stage-prepare.md`: input materialization and provenance capture.
|
|
||||||
- `stage-transcribe.md`: WhisperX transcript generation.
|
|
||||||
- `stage-merge.md`: Seriatim normalization + merge.
|
|
||||||
- `stage-polish.md`: Audita transcript polishing.
|
|
||||||
- `stage-normalize.md`: post-polish normalization.
|
|
||||||
- `stage-trim.md`: bounds-driven transcript trimming.
|
|
||||||
- `stage-analyze.md`: dependency-ordered Scriptorium artifact generation for selected configured artifacts.
|
|
||||||
- `stage-archive.md`: archive upload and current-pointer publish contract.
|
|
||||||
|
|
||||||
## External Integration Notes
|
## Pipeline Stage Set
|
||||||
- `../integrations/README.md`: canonical location for external integration contracts (`audita.md`, `seriatim.md`, `scriptorium.md`).
|
Canonical stage order from `internal/stage.All()`:
|
||||||
|
1. `prepare`
|
||||||
|
2. `transcribe`
|
||||||
|
3. `merge`
|
||||||
|
4. `polish`
|
||||||
|
5. `normalize`
|
||||||
|
6. `trim`
|
||||||
|
7. `analyze`
|
||||||
|
8. `publish`
|
||||||
|
9. `notify` (placeholder)
|
||||||
|
|
||||||
## Canonical Owner
|
`notify` is currently a placeholder stage with optional notifier call behavior; it has no persisted pipeline outputs.
|
||||||
`docs/internal/` is the canonical home for implemented internals per `docs/documentation/policy.md`.
|
|
||||||
|
## Internal Component Docs
|
||||||
|
- `adapters.md`: external adapter boundaries and default runtime wiring.
|
||||||
|
- `artifacts.md`: canonical source IDs, runtime catalog behavior, and resolution rules.
|
||||||
|
- `manifest.md`: session and run manifest contracts.
|
||||||
|
- `storage.md`: object-store interface and S3 implementation behavior.
|
||||||
|
- `workspace.md`: local session layout, run-local layout, and cleanup guardrails.
|
||||||
|
- `command-restore.md`: restore discovery, planning, execution, and reporting.
|
||||||
|
- `stage-prepare.md`
|
||||||
|
- `stage-transcribe.md`
|
||||||
|
- `stage-merge.md`
|
||||||
|
- `stage-polish.md`
|
||||||
|
- `stage-normalize.md`
|
||||||
|
- `stage-trim.md`
|
||||||
|
- `stage-analyze.md`
|
||||||
|
- `stage-publish.md`
|
||||||
|
|||||||
@@ -1,79 +1,52 @@
|
|||||||
# Internal: Adapters
|
# Internal: Adapters
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Describe the external adapter boundaries used by Narratio stages and app orchestration, including default runtime wiring.
|
Define external integration boundaries and default adapter wiring used by app/stage orchestration.
|
||||||
|
|
||||||
## Inputs and outputs
|
## Adapter Boundaries
|
||||||
Inputs:
|
Narratio stage logic depends on adapter interfaces, not transport-specific details.
|
||||||
- Stage requests passed through adapter interfaces (for example transcription, merge/normalize/trim, polish, artifact generation, object-store operations, notifications).
|
|
||||||
- Resolved config values used to construct default adapters.
|
|
||||||
|
|
||||||
Outputs:
|
Primary adapters:
|
||||||
- Adapter-specific result structs (paths, metadata, status/attempt info, duration/exit details).
|
|
||||||
- Adapter errors returned to stage/app orchestration.
|
|
||||||
|
|
||||||
## Boundaries
|
|
||||||
Owns:
|
|
||||||
- Transport/process/SDK details at system boundaries (`HTTP`, subprocess CLI invocation, AWS SDK calls).
|
|
||||||
- Request/response contracts in `internal/adapters/*` packages.
|
|
||||||
|
|
||||||
Does not own:
|
|
||||||
- Stage sequencing, skip/force/resume decisions.
|
|
||||||
- Manifest transition logic.
|
|
||||||
- Canonical workspace path policy.
|
|
||||||
|
|
||||||
## Config fields used
|
|
||||||
Default wiring and adapter calls consume:
|
|
||||||
- `pipeline.whisperx.*`
|
|
||||||
- `pipeline.seriatim.*`
|
|
||||||
- `pipeline.audita.*`
|
|
||||||
- `pipeline.scriptorium.*`
|
|
||||||
- `pipeline.storage.*` and `pipeline.archive.*` (object-store construction/gating)
|
|
||||||
- `pipeline.notification.*` (sender boundary exists; placeholder behavior today)
|
|
||||||
|
|
||||||
## External adapters used
|
|
||||||
Runtime env boundary fields (`internal/stage.Env`):
|
|
||||||
- `whisperx.Client`
|
- `whisperx.Client`
|
||||||
- `seriatim.Runner`
|
- `seriatim.Runner`
|
||||||
- `audita.Runner`
|
- `audita.Runner`
|
||||||
- `scriptorium.Runner`
|
- `scriptorium.Runner`
|
||||||
- `storage.ObjectStore`
|
- `storage.ObjectStore`
|
||||||
- `notify.Sender`
|
- `notify.Sender`
|
||||||
- `analyzer.Runner`
|
|
||||||
|
|
||||||
Current execution usage:
|
Legacy compatibility boundary:
|
||||||
- Actively used by implemented stages: `WhisperX`, `Seriatim`, `Audita`, `Scriptorium`, `ObjectStore`, `Notifier`.
|
- `storage.Backend` remains in the storage adapter package and defaults to `NoopBackend`; current pipeline stages use `storage.ObjectStore`.
|
||||||
- Present but not used by implemented stage set: `Analyzer`, legacy `storage.Backend`.
|
|
||||||
|
|
||||||
Default construction in app runner:
|
## Ownership
|
||||||
- Auto-constructed when not injected: WhisperX HTTP client, Seriatim subprocess runner, Audita subprocess runner, Scriptorium subprocess runner, object store (only when needed), and `notify.NoopSender`.
|
Adapters own:
|
||||||
- Callers can inject test/fake implementations through `app.RunOptions.Env`.
|
- HTTP/subprocess/SDK argument and transport details.
|
||||||
|
- Backend-specific request/response mapping.
|
||||||
|
|
||||||
## State and manifest behavior
|
Adapters do not own:
|
||||||
- Adapters do not directly mutate session/run manifests.
|
- stage ordering/skip/force logic;
|
||||||
- Stages and runner own manifest writes and stage status transitions.
|
- manifest transitions;
|
||||||
- Adapter outputs are persisted indirectly through stage result mapping (outputs/logs/generated configs/metadata).
|
- canonical path policy.
|
||||||
|
|
||||||
## Skip and resume behavior
|
## Default Wiring
|
||||||
- No adapter-level skip/resume semantics.
|
`internal/app/runner.go` initializes default adapters when not injected:
|
||||||
- Skip/resume/force behavior is decided by app runner using manifest stage state.
|
- WhisperX HTTP client from pipeline config.
|
||||||
|
- Seriatim subprocess runner.
|
||||||
|
- Audita subprocess runner.
|
||||||
|
- Scriptorium subprocess runner.
|
||||||
|
- Noop notifier (`notify.NoopSender`).
|
||||||
|
- Object store only when required by selected stages/config.
|
||||||
|
|
||||||
## Failure behavior
|
Object-store construction goes through `newCommandObjectStore`, which loads configured filesystem secrets before adapter initialization.
|
||||||
- Adapter constructors validate config-derived values and fail early on invalid required inputs.
|
|
||||||
- Adapter run-time failures are returned to stage code with boundary context and are recorded as stage failures by runner logic.
|
|
||||||
- Subprocess adapters preserve stdout/stderr and generated-config paths to aid diagnosis.
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
## Failure Semantics
|
||||||
|
- Constructor errors fail stage execution setup early.
|
||||||
|
- Runtime adapter errors propagate to stage code and then manifest failure handling.
|
||||||
|
- Subprocess adapters persist stage logs/generated configs through stage-managed paths.
|
||||||
|
|
||||||
|
## Test Surfaces
|
||||||
- `internal/adapters/whisperx/http_test.go`
|
- `internal/adapters/whisperx/http_test.go`
|
||||||
- `internal/adapters/seriatim/subprocess_test.go`
|
- `internal/adapters/seriatim/subprocess_test.go`
|
||||||
- `internal/adapters/audita/subprocess_test.go`
|
- `internal/adapters/audita/subprocess_test.go`
|
||||||
- `internal/adapters/scriptorium/subprocess_test.go`
|
- `internal/adapters/scriptorium/subprocess_test.go`
|
||||||
- `internal/adapters/storage/*_test.go`
|
- `internal/adapters/storage/*_test.go`
|
||||||
- `internal/adapters/notify/fake_test.go`
|
|
||||||
- `internal/adapters/analyzer/fake_test.go`
|
|
||||||
- `internal/app/runner_test.go`
|
- `internal/app/runner_test.go`
|
||||||
|
|
||||||
## Architectural invariants
|
|
||||||
- Stage code depends on adapter interfaces, not transport-specific implementation types.
|
|
||||||
- External SDK-specific types remain inside adapter implementations.
|
|
||||||
- Default app wiring must remain deterministic and overrideable via injected env dependencies.
|
|
||||||
|
|||||||
@@ -1,87 +1,68 @@
|
|||||||
# Internal: Artifacts
|
# Internal: Artifacts
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define Narratio's artifact identity and resolution model for built-in transcript/bounds artifacts and runtime-configured analyze artifacts.
|
Define canonical artifact IDs, runtime catalog behavior, and source resolution rules for stage execution and publish output selection.
|
||||||
|
|
||||||
## Inputs and outputs
|
## Built-in Source IDs
|
||||||
Inputs:
|
- `narratio.transcript.base` -> `transcripts/base.json` (`merge`)
|
||||||
- artifact sources from config/runtime (`pipeline.scriptorium.artifacts.*.inputs.*.source`)
|
- `narratio.transcript.polished` -> `transcripts/polished.json` (`polish`)
|
||||||
- session paths and optional session manifest stage outputs
|
- `narratio.transcript.final` -> `transcripts/final.json` (`normalize`)
|
||||||
- runtime artifact catalog state for configured artifact sources
|
- `narratio.transcript.final_trimmed` -> `transcripts/final.trimmed.json` (`trim`)
|
||||||
|
- `narratio.bounds.session` -> `artifacts/session_bounds.json` (`trim`)
|
||||||
|
|
||||||
Outputs:
|
## Configured and Previous-Session Sources
|
||||||
- resolved local artifact path and provenance (`ResolvedSessionArtifact`)
|
- Configured artifact source ID: `narratio.artifact.<artifact_key>`
|
||||||
- runtime catalog entries for planned/executable/available artifacts
|
- Previous-session source ID: `narratio.previous_session.artifact.<artifact_key>`
|
||||||
- validation errors for unsupported, missing, or invalid artifact sources
|
|
||||||
|
|
||||||
## Boundaries
|
Configured and previous-session source IDs are validated by strict regex rules.
|
||||||
Owns:
|
|
||||||
- built-in artifact registry and content validation rules
|
|
||||||
- runtime artifact catalog for configured artifact source IDs
|
|
||||||
- source resolution behavior for built-in and configured artifact sources
|
|
||||||
|
|
||||||
Does not own:
|
## Runtime Catalog
|
||||||
- artifact generation (stages produce files)
|
`ArtifactCatalog` tracks:
|
||||||
- manifest transition policy
|
- `planned`: source registered for run context.
|
||||||
- archive promotion behavior
|
- `executable`: selected and enabled for analyze execution.
|
||||||
|
- `available`: local file exists and validated.
|
||||||
|
- `provenance`: availability source.
|
||||||
|
|
||||||
## Config fields used
|
Current provenance values:
|
||||||
- `pipeline.scriptorium.artifacts.<name>.enabled`
|
|
||||||
- `pipeline.scriptorium.artifacts.<name>.output_path`
|
|
||||||
- `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source`
|
|
||||||
|
|
||||||
## External adapters used
|
|
||||||
- none
|
|
||||||
|
|
||||||
## State and manifest behavior
|
|
||||||
Built-in registry entries:
|
|
||||||
|
|
||||||
| Artifact ID | Canonical file | Producer stage | Output kind |
|
|
||||||
| --- | --- | --- | --- |
|
|
||||||
| `narratio.transcript.merged` | `transcripts/merged.json` | `merge` | `transcript_merged` |
|
|
||||||
| `narratio.transcript.polished` | `transcripts/processed.json` | `polish` | `transcript_processed` |
|
|
||||||
| `narratio.transcript.full` | `transcripts/normalized.json` | `normalize` | `transcript_normalized` |
|
|
||||||
| `narratio.transcript.trimmed` | `transcripts/trimmed.json` | `trim` | `transcript_trimmed` |
|
|
||||||
| `narratio.bounds.session` | `artifacts/session_bounds.json` | `trim` | `session_bounds` |
|
|
||||||
|
|
||||||
Runtime catalog entries include built-ins and configured `narratio.artifact.<name>` sources.
|
|
||||||
|
|
||||||
Catalog states:
|
|
||||||
- `planned`: source is registered and known for this run
|
|
||||||
- `executable`: configured artifact is selected for analyze execution
|
|
||||||
- `available`: artifact has a usable file path (generated this run or reused from disk)
|
|
||||||
|
|
||||||
Resolution behavior:
|
|
||||||
- built-in sources resolve via manifest producer outputs first, then canonical fallback path
|
|
||||||
- configured `narratio.artifact.<name>` sources resolve through runtime catalog availability
|
|
||||||
- configured source lookup requires catalog context
|
|
||||||
|
|
||||||
Configured artifact provenance values:
|
|
||||||
- `generated.current_analyze_run`
|
- `generated.current_analyze_run`
|
||||||
- `filesystem.disabled_artifact_output`
|
- `filesystem.disabled_artifact_output`
|
||||||
|
- `manifest.inputs.previous_cache`
|
||||||
|
- `current_session.previous_cache`
|
||||||
|
|
||||||
Content validation:
|
## Resolution Rules
|
||||||
- transcript built-ins: JSON with top-level `segments` array
|
Built-ins:
|
||||||
- bounds built-in: valid JSON
|
1. manifest producer outputs (when present)
|
||||||
- configured artifacts: non-empty text file
|
2. canonical session path fallback
|
||||||
|
|
||||||
## Skip and resume behavior
|
Configured sources (`narratio.artifact.*`):
|
||||||
- resolver and catalog have no direct skip/resume decisions
|
- resolve only through runtime catalog availability.
|
||||||
- stage/runner skip-resume behavior consumes catalog/resolver results
|
|
||||||
|
|
||||||
## Failure behavior
|
Previous-session sources (`narratio.previous_session.artifact.*`):
|
||||||
- unsupported source -> source validation error
|
- resolve only from local `previous/` cache state.
|
||||||
- known source unavailable -> `ErrSessionArtifactNotFound`
|
- prefer manifest-backed previous input paths.
|
||||||
- configured source without catalog -> resolution error
|
- fallback to existing previous-cache filesystem paths.
|
||||||
- resolved file with invalid content -> validation error
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
Validation by content type:
|
||||||
- `internal/artifacts/artifact_resolver_test.go`
|
- transcript built-ins: JSON with top-level `segments` array.
|
||||||
- `internal/artifacts/catalog_test.go`
|
- bounds built-in: valid JSON.
|
||||||
- `internal/stage/analyze_test.go`
|
- configured/previous-session artifact files: non-empty text file.
|
||||||
- `internal/config/scriptorium_test.go`
|
|
||||||
|
|
||||||
## Architectural invariants
|
## Previous Requirement Collection
|
||||||
- built-in IDs are static and registry-backed
|
`CollectPreviousArtifactRequirements`:
|
||||||
- configured artifact IDs are runtime-derived (`narratio.artifact.<name>`) and catalog-backed
|
- scans enabled configured artifacts only;
|
||||||
- built-in/source resolution remains deterministic and validation-gated
|
- extracts only canonical previous-session sources;
|
||||||
|
- deduplicates by artifact key;
|
||||||
|
- merges required/optional (required wins);
|
||||||
|
- returns deterministic ordering and source locations.
|
||||||
|
|
||||||
|
## Key Path Helpers
|
||||||
|
`internal/artifacts/paths.go` defines canonical helpers for:
|
||||||
|
- session/work/run paths;
|
||||||
|
- previous-cache paths;
|
||||||
|
- spool/cache paths;
|
||||||
|
- S3 key layout helpers for session/run/current pointers.
|
||||||
|
|
||||||
|
## Invariants
|
||||||
|
- Source ID formats are stable contracts.
|
||||||
|
- Resolution is deterministic and manifest-aware.
|
||||||
|
- Previous-session source resolution does not call remote storage in `analyze`; remote hydration is `prepare` responsibility.
|
||||||
|
|||||||
@@ -1,86 +1,65 @@
|
|||||||
# Internal: Command Restore
|
# Internal: Command Restore
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define the implemented `narratio restore` command contract: committed remote-state discovery, deterministic planning, safe file installation, conflict policy, and restore reporting.
|
Document the implemented `narratio session restore` command contract:
|
||||||
|
- committed remote current-state discovery;
|
||||||
|
- deterministic restore plan classification;
|
||||||
|
- safe local install semantics;
|
||||||
|
- durable restore reporting.
|
||||||
|
|
||||||
## Inputs and outputs
|
## Discovery Contract
|
||||||
Inputs:
|
Restore discovers remote committed state using:
|
||||||
- CLI flags: `--config`, `--session`, `--session-id`, `--dry-run`, `--force`, `--include-audio`.
|
- `current/run_id.txt` (required, non-empty)
|
||||||
- Resolved/validated `pipeline.yml` and `session.yml`.
|
- `current/manifest.json` (required, decodable)
|
||||||
- Configured remote object store.
|
|
||||||
- Remote committed current-state markers (`current/run_id.txt`, `current/manifest.json`).
|
|
||||||
|
|
||||||
Outputs:
|
Discovered manifest identity must match requested `session_id` and `campaign`.
|
||||||
- Dry-run summary to stdout (plan + counts).
|
|
||||||
- Non-dry-run completion summary to stdout.
|
|
||||||
- Local durable session files restored under canonical session root.
|
|
||||||
- Non-dry-run restore report at `reports/restore-latest.json`.
|
|
||||||
|
|
||||||
## Boundaries
|
## Plan Contract
|
||||||
Owns:
|
Planner actions:
|
||||||
- Restore command flag parsing and command wiring.
|
- `download`
|
||||||
- Remote current-state discovery and identity validation.
|
- `skip_same`
|
||||||
- Restore plan construction and conflict classification.
|
- `conflict`
|
||||||
- Restore execution for planned downloads.
|
|
||||||
- Restore report model and persistence.
|
|
||||||
|
|
||||||
Does not own:
|
Plan behavior:
|
||||||
- Stage execution orchestration (`run`, `resume`, `run-stage`).
|
- remote list scope is the resolved session prefix;
|
||||||
- Archive publish behavior (owned by archive stage).
|
- mapping to local paths is traversal-safe;
|
||||||
- Storage transport implementation details (owned by storage adapters).
|
- actions are sorted deterministically by local relative path.
|
||||||
|
|
||||||
## Config fields used
|
Restore scope from current remote state:
|
||||||
- Config/session discovery and templating fields consumed by all commands.
|
- include `manifest.json`
|
||||||
- `pipeline.workspace.root` (local restore target root).
|
- include `transcripts/**`
|
||||||
- `pipeline.storage.*` (remote backend + archive identity derivation).
|
- include `artifacts/**`
|
||||||
- `pipeline.storage.s3.*` identity components used by archive prefix helpers.
|
- include `audio/**` only with `--include-audio`
|
||||||
- `session.session_id`
|
|
||||||
- `session.campaign`
|
|
||||||
|
|
||||||
## External adapters used
|
Explicit exclusions from current remote state mapping:
|
||||||
- `storage.ObjectStore` for `Exists`, `List`, `Download`.
|
- `current/**`
|
||||||
- `artifacts.Store` (`LocalStore`) for layout and session lock management.
|
- `runs/**`
|
||||||
- `manifest.LocalStore` for manifest decode/validation and identity checks.
|
- `logs/**`
|
||||||
|
- `reports/**`
|
||||||
|
- `config/**`
|
||||||
|
- `inputs/**`
|
||||||
|
- `previous/**`
|
||||||
|
|
||||||
## State and manifest behavior
|
Previous-cache restore files are planned separately through `previouscache.BuildPlan` when configured previous-session requirements exist.
|
||||||
- Restore is not a pipeline run and does not create a run manifest.
|
|
||||||
- Restore uses committed remote current state only:
|
|
||||||
- `current/run_id.txt` must exist and be non-empty.
|
|
||||||
- `current/manifest.json` must decode and match requested session/campaign.
|
|
||||||
- Non-dry-run writes restore files to canonical session paths.
|
|
||||||
- Manifest install behavior:
|
|
||||||
- validated before replacement.
|
|
||||||
- installed last among download actions.
|
|
||||||
- existing local manifest is preserved if restored manifest validation/install fails.
|
|
||||||
- Non-dry-run report persists summary/action status metadata in `reports/restore-latest.json`.
|
|
||||||
|
|
||||||
## Skip and resume behavior
|
## Execution Contract
|
||||||
- Restore does not participate in stage skip/resume decisions.
|
- non-manifest downloads happen before manifest install;
|
||||||
- Restore provides durable local state so subsequent stage commands can resume or rerun based on restored manifest state.
|
- `manifest.json` is installed last;
|
||||||
- Dry-run is read-only and returns plan output only.
|
- downloads use sibling temp files + atomic rename;
|
||||||
|
- manifest replacement is validated before rename;
|
||||||
|
- failed installs do not roll back previously written files.
|
||||||
|
|
||||||
## Failure behavior
|
Audio restore path:
|
||||||
- Fails when storage backend is unavailable or archive identity cannot be resolved.
|
- uses `audio.MaterializeS3Audio`;
|
||||||
- Fails when remote current pointer/manifest is missing or invalid.
|
- integrates spool and S3 audio cache paths;
|
||||||
- Fails when remote manifest identity mismatches requested campaign/session.
|
- supports cache hit reuse without object redownload.
|
||||||
- Fails on local conflicts unless `--force` is set.
|
|
||||||
- Fails fast on session lock acquisition conflict for non-dry-run execution.
|
|
||||||
- On execution failure, previously installed files remain; no rollback is performed.
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
## Reporting Contract
|
||||||
- `internal/app/restore_test.go`
|
- dry-run: summary only (no writes).
|
||||||
- `internal/app/restore_discovery_test.go`
|
- non-dry-run: writes `reports/restore-latest.json`.
|
||||||
- `internal/app/restore_plan_test.go`
|
- report captures plan counts, action status, and execution failures.
|
||||||
- `internal/app/restore_execution_test.go`
|
|
||||||
- `internal/app/restore_workflow_test.go`
|
|
||||||
- `internal/artifacts/archive_identity_test.go`
|
|
||||||
|
|
||||||
## Architectural invariants
|
## Invariants
|
||||||
- Restore relies on centralized archive identity/key helpers (`internal/artifacts`) rather than ad hoc key building.
|
- restore uses only committed remote current state as authority.
|
||||||
- `current/run_id.txt` is the remote commit marker; restore must not infer committed state from incidental files.
|
- `current/run_id.txt` is the remote commit marker.
|
||||||
- Local path mapping is traversal-safe and constrained to session root.
|
- restore is a standalone command and does not run stages.
|
||||||
- Restore scope is deterministic and path-classified:
|
|
||||||
- include `manifest.json`, `transcripts/**`, `artifacts/**`
|
|
||||||
- include `audio/**` only with `--include-audio`
|
|
||||||
- exclude `runs/**`, `logs/**`, `reports/**`, `config/**`, `inputs/**`
|
|
||||||
- Command remains standalone; no implicit `run --restore` behavior.
|
|
||||||
|
|||||||
@@ -1,81 +1,57 @@
|
|||||||
# Internal: Manifest
|
# Internal: Manifest
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Describe Narratio's durable execution state model for session-level and run-level manifests, including lifecycle transitions and persistence behavior.
|
Define durable session state (`manifest.json`) and invocation state (`runs/{run_id}/manifest.json`) contracts.
|
||||||
|
|
||||||
## Inputs and outputs
|
## Session Manifest
|
||||||
Inputs:
|
Path:
|
||||||
- Session identity and run identity from app orchestration.
|
- `{workspace.root}/work/{campaign}/{session_id}/manifest.json`
|
||||||
- Stage transition events and stage result payloads.
|
|
||||||
|
|
||||||
Outputs:
|
Primary model (`manifest.Manifest`):
|
||||||
- Session manifest at `{workspace.root}/work/{campaign}/{session_id}/manifest.json`.
|
- identity (`session_id`, `campaign`, `run_id`)
|
||||||
- Run manifest at `{workspace.root}/work/{campaign}/{session_id}/runs/{run_id}/manifest.json`.
|
- local path metadata (`local_workdir`, `local_spool_dir`)
|
||||||
|
- remote identity metadata (`s3_bucket`, `s3_session_prefix`, `s3_run_prefix`)
|
||||||
|
- `inputs` records
|
||||||
|
- durable `artifacts` records
|
||||||
|
- per-stage `stages` map
|
||||||
|
|
||||||
## Boundaries
|
Stage status enum:
|
||||||
Owns:
|
- `pending`
|
||||||
- Manifest schemas (`Manifest`, `RunManifest`, stage records, error records, input/artifact records).
|
- `running`
|
||||||
- Stage status/action transition methods.
|
- `succeeded`
|
||||||
- Persistent store contract (`manifest.Store`) and local JSON store implementation.
|
- `failed`
|
||||||
|
- `skipped`
|
||||||
|
- `stale`
|
||||||
|
- `interrupted`
|
||||||
|
|
||||||
Does not own:
|
## Run Manifest
|
||||||
- Stage implementation details.
|
Path:
|
||||||
- Path construction policy outside manifest file persistence calls.
|
- `{workspace.root}/work/{campaign}/{session_id}/runs/{run_id}/manifest.json`
|
||||||
- CLI command behavior.
|
|
||||||
|
|
||||||
## Config fields used
|
Run model (`manifest.RunManifest`):
|
||||||
Manifest package itself does not read config directly.
|
- invocation identity and `force` flag
|
||||||
|
- requested stages
|
||||||
|
- per-stage action (`run` or `skip`)
|
||||||
|
- per-stage status
|
||||||
|
- overall run status (`running`, `succeeded`, `failed`)
|
||||||
|
|
||||||
Manifest identity fields are populated by app/stage orchestration from:
|
## Persistence Semantics
|
||||||
- `session.session_id`
|
`manifest.LocalStore`:
|
||||||
- `session.campaign`
|
- validates loaded documents;
|
||||||
- `pipeline.workspace.root`
|
- normalizes missing maps/stage records;
|
||||||
- `pipeline.storage.s3.*` (when archive/S3 identity is set)
|
- writes atomically via temp file + rename;
|
||||||
|
- updates `updated_at` on save.
|
||||||
|
|
||||||
## External adapters used
|
## Execution Semantics
|
||||||
- No external service adapters.
|
Runner updates both manifests per stage transition:
|
||||||
- Uses local filesystem for persistence via `manifest.LocalStore`.
|
- mark running
|
||||||
|
- mark succeeded/failed/skipped
|
||||||
|
- persist logs/generated config refs and metadata
|
||||||
|
|
||||||
## State and manifest behavior
|
Session manifest is the authoritative stage-progress ledger across invocations.
|
||||||
Session manifest model:
|
Run manifest is invocation-scoped audit state.
|
||||||
- Tracks durable per-session stage state and provenance (`pending`, `running`, `succeeded`, `failed`, `skipped`, `stale`, `interrupted`).
|
|
||||||
- Stores resolved inputs, durable artifacts, stage logs/config refs, and stage metadata.
|
|
||||||
|
|
||||||
Run manifest model:
|
## Invariants
|
||||||
- Tracks one invocation (`run_id`) with requested stages and force mode.
|
- stage resume/skip decisions are session-manifest driven.
|
||||||
- Tracks per-stage action (`run` or `skip`) and per-stage status.
|
- force reruns stale downstream succeeded stages.
|
||||||
- Tracks overall run status (`running`, `succeeded`, `failed`).
|
- run manifest does not replace session manifest as progress authority.
|
||||||
|
|
||||||
Persistence behavior:
|
|
||||||
- Load validates required identity/timestamp fields and normalizes maps/records.
|
|
||||||
- Save updates `updated_at` and writes JSON atomically (temp file + rename).
|
|
||||||
- Session and run manifests are saved incrementally before/after stage transitions.
|
|
||||||
|
|
||||||
Relationship during execution:
|
|
||||||
- Runner updates both manifests for every stage transition.
|
|
||||||
- Session manifest is the durable pipeline-progress ledger.
|
|
||||||
- Run manifest is invocation history and audit record.
|
|
||||||
- Analyze stage outputs are persisted as `kind=scriptorium_artifact` with `source_id=narratio.artifact.<name>` for configured artifact identity.
|
|
||||||
|
|
||||||
## Skip and resume behavior
|
|
||||||
- Resume and skip decisions are based on session-manifest stage statuses.
|
|
||||||
- `--force` reruns selected stages and marks downstream succeeded stages as `stale` in session manifest.
|
|
||||||
- Run manifest records whether each stage was executed or skipped in that invocation.
|
|
||||||
|
|
||||||
## Failure behavior
|
|
||||||
- Stage failure marks both manifests failed for that stage and records error messages/timestamps.
|
|
||||||
- Save failures are returned immediately and fail the command.
|
|
||||||
- Invalid/malformed manifest files fail load with explicit validation/decode errors.
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
|
||||||
- `internal/manifest/manifest_test.go`
|
|
||||||
- `internal/manifest/run_manifest_test.go`
|
|
||||||
- `internal/manifest/store_test.go`
|
|
||||||
- `internal/app/runner_test.go`
|
|
||||||
- `internal/app/run_control_test.go`
|
|
||||||
- `internal/app/resume_run_stage_test.go`
|
|
||||||
|
|
||||||
## Architectural invariants
|
|
||||||
- Session manifest is authoritative for stage progression across invocations.
|
|
||||||
- Run manifest is invocation-scoped and never replaces session manifest as progress authority.
|
|
||||||
- Manifest writes are atomic and deterministic (JSON + newline, temp rename pattern).
|
|
||||||
|
|||||||
@@ -1,84 +1,38 @@
|
|||||||
# Stage: analyze
|
# Stage: analyze
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Execute selected configured Scriptorium artifacts in deterministic dependency order and promote successful outputs to canonical session artifact paths.
|
Execute selected configured Scriptorium artifacts in dependency order and materialize outputs.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
- configured artifacts from `pipeline.scriptorium.artifacts`
|
||||||
- configured artifact definitions from `pipeline.scriptorium.artifacts`
|
- optional selected artifact filter (`--artifacts`)
|
||||||
- selected artifact filter from runtime (`--artifacts`) when provided
|
- built-in/configured/previous-session source references in artifact inputs
|
||||||
- resolved artifact input sources declared per artifact (`inputs.*.source`)
|
|
||||||
- optional previous-session file inputs (`previous_session_artifact`)
|
|
||||||
|
|
||||||
Outputs:
|
Supported source families:
|
||||||
- one promoted output file per executed configured artifact at that artifact's configured `output_path`
|
- built-ins: `narratio.transcript.*`, `narratio.bounds.session`
|
||||||
- stage metadata containing generated artifact entries and reused disabled-artifact entries
|
- configured artifacts: `narratio.artifact.<key>`
|
||||||
|
- previous-session cache: `narratio.previous_session.artifact.<key>`
|
||||||
|
|
||||||
## Boundaries
|
## Outputs
|
||||||
Owns:
|
- one materialized output per executed configured artifact (`output_path`)
|
||||||
- runtime artifact catalog construction for analyze execution
|
- stage metadata describing selected/generated/reused artifacts
|
||||||
- selected-artifact planning and dependency ordering
|
|
||||||
- per-artifact input resolution, var resolution, timeout/render-debug resolution
|
|
||||||
- Scriptorium run/render invocation for each selected artifact
|
|
||||||
- run-local output generation and canonical promotion
|
|
||||||
|
|
||||||
Does not own:
|
## Key Behavior
|
||||||
- transcript generation/processing stages
|
- skips with metadata when Scriptorium config is missing or no executable artifacts remain.
|
||||||
- archive promotion policy
|
- builds runtime artifact catalog (built-ins + configured artifacts).
|
||||||
- per-artifact resume semantics
|
- marks non-executable configured artifacts as reusable when output files already exist.
|
||||||
|
- validates selected artifact dependency order (cycle-safe topo ordering).
|
||||||
|
- resolves required/optional inputs per artifact source definition.
|
||||||
|
- resolves previous-session sources from local `previous/` cache only.
|
||||||
|
- runs optional render-debug, then artifact execution.
|
||||||
|
- validates non-empty output files and materializes canonical outputs.
|
||||||
|
|
||||||
## Config Fields Used
|
## Failure Semantics
|
||||||
- `session.session_id`
|
- required missing configured/previous-session inputs fail.
|
||||||
- `session.campaign`
|
- missing required previous-session source includes prepare rerun guidance.
|
||||||
- `pipeline.workspace.root`
|
- dependency cycles or unavailable required dependencies fail.
|
||||||
- `pipeline.scriptorium.binary`
|
- adapter validation failures fail stage.
|
||||||
- `pipeline.scriptorium.config_path`
|
|
||||||
- `pipeline.scriptorium.timeout`
|
|
||||||
- `pipeline.scriptorium.render_debug`
|
|
||||||
- `pipeline.scriptorium.artifacts.<name>.*`
|
|
||||||
- `enabled`
|
|
||||||
- `depends_on`
|
|
||||||
- `prompt_id`
|
|
||||||
- `profile_id`
|
|
||||||
- `timeout`
|
|
||||||
- `output_path`
|
|
||||||
- `render_debug`
|
|
||||||
- `inputs`
|
|
||||||
- `vars`
|
|
||||||
|
|
||||||
## External Adapters Used
|
## Invariants
|
||||||
- Scriptorium adapter:
|
- `analyze` performs no remote storage calls for previous-session source resolution.
|
||||||
- optional `RenderArtifact` (render debug)
|
- output provenance and metadata are deterministic per execution.
|
||||||
- `RunArtifact` (artifact generation)
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- If `pipeline.scriptorium` is absent, stage returns success metadata with `skipped=true`.
|
|
||||||
- If no artifacts are configured, stage returns success metadata with `skipped=true`.
|
|
||||||
- If zero artifacts are executable after `enabled` + `--artifacts` filtering, stage returns success metadata with `skipped=true`.
|
|
||||||
- Builds runtime catalog with built-ins and configured artifacts.
|
|
||||||
- Non-executable configured artifacts are marked available only when their configured output file exists and is valid on disk.
|
|
||||||
- Executes selected configured artifacts in topological order with deterministic tie-breaking.
|
|
||||||
- For each generated artifact, records metadata fields including `name`, `source_id`, `output_kind`, `path`, `prompt_id`, `profile_id`, and `provenance`.
|
|
||||||
- Reused disabled artifacts are recorded separately in `reused_artifacts` with provenance `filesystem.disabled_artifact_output`.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies when analyze is already `succeeded` and `--force` is not set.
|
|
||||||
- Analyze remains stage-scoped for resume/skip; there is no per-artifact resume state.
|
|
||||||
- `--artifacts` filters which configured artifacts are executable when analyze runs; it does not imply `--force`.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on invalid dependency ordering, unavailable required configured inputs, invalid built-in input prerequisites, render/run adapter failures, validation-failed adapter results, or missing/empty outputs.
|
|
||||||
- Required configured dependency missing from catalog availability fails clearly before invocation.
|
|
||||||
- Optional missing inputs are omitted.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/analyze_test.go`
|
|
||||||
- `internal/artifacts/catalog_test.go`
|
|
||||||
- `internal/artifacts/artifact_resolver_test.go`
|
|
||||||
- `internal/adapters/scriptorium/subprocess_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Configured artifacts are identified by `narratio.artifact.<name>` source IDs.
|
|
||||||
- Artifact-to-artifact references rely on explicit `depends_on` declarations validated in config.
|
|
||||||
- Generated analyze outputs are treated uniformly as Scriptorium artifacts.
|
|
||||||
- Successful outputs must exist and be non-empty before promotion.
|
|
||||||
|
|||||||
@@ -1,68 +0,0 @@
|
|||||||
# Stage: archive
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Publish run records and promoted session artifacts to object storage, then atomically advance the remote current pointer.
|
|
||||||
|
|
||||||
## Inputs and Outputs
|
|
||||||
Inputs:
|
|
||||||
- session manifest and prerequisite stage records
|
|
||||||
- run root contents under `runs/{run_id}/`
|
|
||||||
- promotion rules with artifact `source` IDs and archive `dest` paths (`archive.promote_artifacts`)
|
|
||||||
|
|
||||||
Outputs:
|
|
||||||
- uploaded run files under `{session_prefix}/runs/{run_id}/...`
|
|
||||||
- uploaded promoted artifacts under `{session_prefix}/...`
|
|
||||||
- `{session_prefix}/current/manifest.json`
|
|
||||||
- `{session_prefix}/current/run_id.txt` written last
|
|
||||||
|
|
||||||
## Boundaries
|
|
||||||
Owns:
|
|
||||||
- Archive enable/disable gate behavior
|
|
||||||
- Prerequisite stage success enforcement
|
|
||||||
- Run file collection and upload (excluding `audio/`)
|
|
||||||
- Promotion rule resolution and upload
|
|
||||||
- Commit pointer publish order
|
|
||||||
|
|
||||||
Does not own:
|
|
||||||
- Stage execution before archive
|
|
||||||
- Post-archive local cleanup policy execution (handled by app cleanup logic)
|
|
||||||
|
|
||||||
## Config Fields Used
|
|
||||||
- `pipeline.archive.enabled`
|
|
||||||
- `pipeline.archive.upload_run`
|
|
||||||
- `pipeline.archive.promote_artifacts`
|
|
||||||
- `pipeline.storage.s3.bucket`
|
|
||||||
- `pipeline.storage.s3.root_prefix`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `session.campaign`
|
|
||||||
- `session.session_id`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- Object storage backend (`env.ObjectStore`) for upload/list primitives.
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Requires `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, and `analyze` status `succeeded`.
|
|
||||||
- Resolves bucket/prefix from manifest identity first, then config fallback.
|
|
||||||
- Writes metadata including:
|
|
||||||
- upload counts/paths
|
|
||||||
- `current_manifest_key`
|
|
||||||
- `current_run_id_key`
|
|
||||||
- `current_pointer_written`
|
|
||||||
- On skipped archive path, returns metadata with `skipped=true` and pointer not written.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Stage may self-skip (metadata skip) when archive disabled or run upload disabled.
|
|
||||||
- Runner-level skip also applies for previously succeeded stage unless forced.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on missing prerequisite success, missing object store when required, missing run root, missing required promotion source, upload failures, or pointer write failures.
|
|
||||||
- Pointer semantics are fail-safe: `current/run_id.txt` is not written if prior required uploads fail.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/archive_test.go`
|
|
||||||
- `internal/app/post_archive_cleanup_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Run upload excludes `audio/` subtree.
|
|
||||||
- `current/manifest.json` uploads before `current/run_id.txt`.
|
|
||||||
- `current/run_id.txt` is the remote publish commit marker.
|
|
||||||
@@ -1,63 +1,25 @@
|
|||||||
# Stage: merge
|
# Stage: merge
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Normalize per-speaker raw transcripts and merge them into one merged transcript via Seriatim.
|
Normalize raw transcript inputs and merge into base transcript via Seriatim.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
|
||||||
- `transcripts/raw/*.json`
|
- `transcripts/raw/*.json`
|
||||||
- `inputs/speakers.yml`
|
- `inputs/speakers.yml`
|
||||||
- `inputs/autocorrect.yml`
|
- `inputs/autocorrect.yml`
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- `transcripts/merged.json`
|
- `transcripts/base.json`
|
||||||
- optional `artifacts/seriatim.report.json` (when report enabled)
|
- optional `artifacts/seriatim.report.json`
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- discovers and validates raw transcript inputs.
|
||||||
- Raw transcript discovery/validation
|
- normalizes each raw transcript (`seriatim.Normalize`) into run-local scratch output.
|
||||||
- Per-input normalize calls to Seriatim
|
- merges normalized inputs (`seriatim.Run`) into base transcript.
|
||||||
- Final merge call to Seriatim
|
- validates merged transcript and optional report JSON.
|
||||||
- Run-local log/config/report path wiring
|
- materializes canonical outputs and records stage logs/generated configs.
|
||||||
- Promotion of merged/report outputs to canonical paths
|
|
||||||
|
|
||||||
Does not own:
|
## Invariants
|
||||||
- Transcript polishing or downstream artifact generation
|
- merge always consumes normalized forms of raw inputs.
|
||||||
|
- base transcript must validate before stage success.
|
||||||
## Config Fields Used
|
- report output is config-gated.
|
||||||
- `session.session_id`
|
|
||||||
- `session.campaign`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `pipeline.seriatim.binary`
|
|
||||||
- `pipeline.seriatim.timeout`
|
|
||||||
- `pipeline.seriatim.output_schema`
|
|
||||||
- `pipeline.seriatim.coalesce_gap`
|
|
||||||
- `pipeline.seriatim.report`
|
|
||||||
- `pipeline.seriatim.env.*`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- Seriatim adapter:
|
|
||||||
- `Normalize` for each raw input
|
|
||||||
- `Run` for final merge
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Reads transcript inputs from transcribe stage outputs in manifest when present; falls back to canonical raw directory.
|
|
||||||
- Writes run-local outputs/logs/config under `runs/{run_id}/merge/...` when enabled.
|
|
||||||
- Promotes canonical merged transcript and optional report.
|
|
||||||
- Records normalized-input provenance and adapter metadata in stage metadata.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies when already succeeded and not forced.
|
|
||||||
- Forced rerun of this or upstream stages can stale downstream succeeded stages via runner invalidation.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on missing/invalid raw transcripts, missing speakers/autocorrect files, normalize failure, merge failure, invalid merged output JSON, or invalid report JSON when enabled.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/merge_test.go`
|
|
||||||
- `internal/adapters/seriatim/subprocess_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Merge consumes normalized forms of each raw transcript.
|
|
||||||
- Merged transcript must validate before promotion.
|
|
||||||
- Report output is optional and gated by config.
|
|
||||||
|
|||||||
@@ -1,56 +1,22 @@
|
|||||||
# Stage: normalize
|
# Stage: normalize
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Normalize the processed transcript into a deterministic intermediate schema for trim and optionally emit a normalize report.
|
Normalize polished transcript into final transcript using Seriatim.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
- `transcripts/polished.json`
|
||||||
- `transcripts/processed.json`
|
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- `transcripts/normalized.json` (or configured normalize output path)
|
- `transcripts/final.json` (or configured normalize output path)
|
||||||
- optional `artifacts/seriatim.normalize.report.json`
|
- optional `artifacts/seriatim.normalize.report.json`
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- resolves polished transcript from manifest outputs/canonical fallback.
|
||||||
- Processed transcript discovery/validation
|
- applies `pipeline.normalize` config or default normalize config.
|
||||||
- Normalize request construction and invocation
|
- runs Seriatim normalize with configured timeout/binary.
|
||||||
- Optional normalize report wiring
|
- validates normalized transcript and optional report.
|
||||||
- Promotion of normalized transcript and optional report
|
- materializes canonical outputs and records logs/generated configs.
|
||||||
|
|
||||||
Does not own:
|
## Invariants
|
||||||
- Bounds detection or segment trimming
|
- final transcript must validate as processed transcript JSON (`segments` array).
|
||||||
|
- normalize defaults are applied when `pipeline.normalize` is unset.
|
||||||
## Config Fields Used
|
|
||||||
- `session.session_id`
|
|
||||||
- `session.campaign`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `pipeline.normalize.output_path`
|
|
||||||
- `pipeline.normalize.output_schema`
|
|
||||||
- `pipeline.normalize.report`
|
|
||||||
- `pipeline.seriatim.binary`
|
|
||||||
- `pipeline.seriatim.timeout`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- Seriatim adapter (`Normalize`).
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Reads processed transcript from polish outputs in manifest when present; falls back to canonical path.
|
|
||||||
- Uses run-local output/report/log/config paths when run layout is enabled.
|
|
||||||
- Promotes canonical normalized transcript and optional normalize report.
|
|
||||||
- Records adapter/result metadata including source path selection.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies when already succeeded and not forced.
|
|
||||||
- Forced reruns can stale downstream succeeded stages.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on missing/invalid processed transcript, adapter error, invalid normalized output, or invalid report output when report enabled.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/normalize_test.go`
|
|
||||||
- `internal/adapters/seriatim/subprocess_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Normalized output must validate as processed-transcript-compatible JSON (`segments` array required).
|
|
||||||
- Default normalize config is applied when `pipeline.normalize` is unset.
|
|
||||||
|
|||||||
@@ -1,69 +1,23 @@
|
|||||||
# Stage: polish
|
# Stage: polish
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Polish merged transcript with Audita and produce a processed transcript for downstream normalization/analyze.
|
Run Audita polishing on base transcript and produce polished transcript.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
- `transcripts/base.json`
|
||||||
- `transcripts/merged.json`
|
|
||||||
- `inputs/glossary.yml`
|
- `inputs/glossary.yml`
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- `transcripts/processed.json`
|
- `transcripts/polished.json`
|
||||||
- optional `artifacts/audita.report.json` (when report enabled)
|
- optional `artifacts/audita.report.json`
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- resolves base transcript from merge outputs/canonical fallback.
|
||||||
- Merged transcript discovery/validation
|
- invokes Audita with configured model/module/runtime options.
|
||||||
- Audita invocation request construction
|
- validates processed transcript structure (`segments` array required).
|
||||||
- Run-local logs/config/work-dir/report wiring
|
- validates optional report JSON.
|
||||||
- Promotion of processed transcript and optional report
|
- materializes canonical outputs; records logs/generated config and adapter metadata.
|
||||||
|
|
||||||
Does not own:
|
## Invariants
|
||||||
- Upstream merge normalization
|
- polished transcript schema validation is mandatory.
|
||||||
- Downstream normalize/trim/analyze logic
|
- report output is config-gated.
|
||||||
|
|
||||||
## Config Fields Used
|
|
||||||
- `session.session_id`
|
|
||||||
- `session.campaign`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `pipeline.audita.binary`
|
|
||||||
- `pipeline.audita.timeout`
|
|
||||||
- `pipeline.audita.llm_api_key_env`
|
|
||||||
- `pipeline.audita.modules`
|
|
||||||
- `pipeline.audita.base_url`
|
|
||||||
- `pipeline.audita.model`
|
|
||||||
- `pipeline.audita.transcript_description`
|
|
||||||
- `pipeline.audita.config_path`
|
|
||||||
- `pipeline.audita.output_schema`
|
|
||||||
- `pipeline.audita.work_dir_retention`
|
|
||||||
- `pipeline.audita.total_llm_concurrency`
|
|
||||||
- `pipeline.audita.proposal_llm_concurrency`
|
|
||||||
- `pipeline.audita.validation_model`
|
|
||||||
- `pipeline.audita.validation_llm_concurrency`
|
|
||||||
- `pipeline.audita.report`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- Audita adapter (`env.Audita.Run`).
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Reads merged transcript from merge manifest outputs when available; falls back to canonical merged path.
|
|
||||||
- Uses run-local output/report/log/config/scratch paths when run layout is enabled.
|
|
||||||
- Promotes canonical `transcripts/processed.json` and optional report.
|
|
||||||
- Records adapter invocation metadata, credential presence signal, and output provenance in stage metadata.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies when already succeeded and not forced.
|
|
||||||
- Forced rerun can stale downstream succeeded stages via runner invalidation.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on missing/invalid merged transcript, missing glossary, adapter error, invalid processed output shape (`segments` array required), or invalid report JSON when enabled.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/polish_test.go`
|
|
||||||
- `internal/adapters/audita/subprocess_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Processed transcript must contain a top-level `segments` array.
|
|
||||||
- Report behavior is strictly config-gated.
|
|
||||||
- Stage output canonicalization always ends at `transcripts/processed.json`.
|
|
||||||
|
|||||||
@@ -1,74 +1,42 @@
|
|||||||
# Stage: prepare
|
# Stage: prepare
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Materialize all required session inputs into canonical local workspace paths and record input provenance in the session manifest.
|
Materialize canonical current-session inputs before processing stages.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
- resolved `campaign.yml`, `session.yml`, and pipeline config
|
||||||
- `session.yml` (resolved session config)
|
- stable input files (`speakers`, `autocorrect`, `glossary`)
|
||||||
- `pipeline.resolved.yml` (materialized from resolved pipeline config)
|
|
||||||
- `speakers.yml`
|
|
||||||
- `autocorrect.yml`
|
|
||||||
- `glossary.yml`
|
|
||||||
- audio source:
|
- audio source:
|
||||||
- local (`session.inputs.audio_dir` or `session.inputs.audio_files`), or
|
- local `audio_dir`/`audio_files`, or
|
||||||
- S3 (`session.inputs.audio_s3.prefix`)
|
- S3 `audio_s3.prefix`
|
||||||
|
- enabled configured artifact input requirements for previous-session sources
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
|
- `inputs/campaign.yml`
|
||||||
- `inputs/session.yml`
|
- `inputs/session.yml`
|
||||||
- `inputs/pipeline.resolved.yml`
|
- `inputs/pipeline.resolved.yml`
|
||||||
- `inputs/speakers.yml`
|
- `inputs/speakers.yml`
|
||||||
- `inputs/autocorrect.yml`
|
- `inputs/autocorrect.yml`
|
||||||
- `inputs/glossary.yml`
|
- `inputs/glossary.yml`
|
||||||
- `audio/*.flac` in session workdir
|
- `audio/*.flac`
|
||||||
- `manifest.Inputs` records with checksums and source metadata
|
- optional `previous/manifest.json`
|
||||||
|
- optional `previous/artifacts/**`
|
||||||
|
- deterministic `manifest.inputs` entries (checksums + provenance)
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- validates required config/store state.
|
||||||
- Input path resolution and validation
|
- enforces local audio vs S3 audio mutual exclusivity.
|
||||||
- Local copy/materialization of configs and audio files
|
- materializes S3 audio through spool/cache-aware logic.
|
||||||
- S3 audio download to run-scoped spool, then copy into work audio dir
|
- scans enabled configured artifact inputs for `narratio.previous_session.artifact.*` requirements.
|
||||||
|
- when previous requirements exist:
|
||||||
|
- clears managed `previous/` state;
|
||||||
|
- builds previous-cache remote plan;
|
||||||
|
- downloads previous manifest/artifacts;
|
||||||
|
- records previous inputs in `manifest.inputs`.
|
||||||
|
|
||||||
Does not own:
|
Required previous-session inputs fail when unavailable; optional missing inputs are skipped.
|
||||||
- Transcript generation/processing
|
|
||||||
- Archive publish behavior
|
|
||||||
|
|
||||||
## Config Fields Used
|
## Invariants
|
||||||
- `session.session_id`
|
- only `prepare` hydrates canonical `previous/` cache state.
|
||||||
- `session.campaign`
|
- managed previous artifacts are stored under `previous/artifacts/**` without duplicate `artifacts/artifacts/` nesting.
|
||||||
- `session.inputs.speakers_file`
|
- `manifest.inputs` ordering is deterministic (`kind`, `path`).
|
||||||
- `session.inputs.autocorrect_file`
|
|
||||||
- `session.inputs.glossary_file`
|
|
||||||
- `session.inputs.audio_dir`
|
|
||||||
- `session.inputs.audio_files`
|
|
||||||
- `session.inputs.audio_s3.prefix`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `pipeline.spool.root`
|
|
||||||
- `pipeline.storage.s3.bucket`
|
|
||||||
- `pipeline.storage.s3.root_prefix`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- Object storage backend (`env.ObjectStore`) for S3 audio list/download when `audio_s3` is configured.
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Ensures workspace layout exists.
|
|
||||||
- Writes resolved config and input files to canonical `inputs/` paths.
|
|
||||||
- Records all prepared inputs into `manifest.Inputs` (sorted deterministically by kind/path).
|
|
||||||
- For S3 audio, records `S3Bucket`, `S3Key`, `S3Size`, `S3ETag`, and `SpoolPath` in each audio input record.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies when stage already `succeeded` and `--force` is not set.
|
|
||||||
- Stage itself is deterministic/idempotent for unchanged inputs (`copyFileIfChanged`, `writeBytesIfChanged`).
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on missing required files, invalid audio source combinations, no discoverable `.flac` files, duplicate audio basenames, missing object store for S3 mode, or S3 list/download failures.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/prepare_test.go`
|
|
||||||
- `internal/app/session_cli_test.go`
|
|
||||||
- `internal/config/load_validate_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- `audio_dir`/`audio_files` and `audio_s3` are mutually exclusive.
|
|
||||||
- Audio files must be `.flac`.
|
|
||||||
- Canonical `inputs/*` and `audio/*` paths are the durable source for downstream stages.
|
|
||||||
|
|||||||
44
docs/internal/stage-publish.md
Normal file
44
docs/internal/stage-publish.md
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
# Stage: publish
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
Upload run/session outputs to object storage and atomically advance remote current state.
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
- successful prerequisite stages: `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, `analyze`
|
||||||
|
- run root `runs/{run_id}/**`
|
||||||
|
- publish output rules (`pipeline.publish.outputs`)
|
||||||
|
- effective publish locks (static + remote merged lock set)
|
||||||
|
- local `previous/**` files when present
|
||||||
|
|
||||||
|
## Outputs
|
||||||
|
- uploaded run files under remote `runs/{run_id}/...` (excluding `audio/**`)
|
||||||
|
- uploaded selected publish outputs under session prefix
|
||||||
|
- uploaded `previous/**` files under session prefix when present
|
||||||
|
- uploaded `current/manifest.json`
|
||||||
|
- uploaded `current/run_id.txt` written last
|
||||||
|
|
||||||
|
## Key Behavior
|
||||||
|
- stage can self-skip when publish disabled or run upload disabled.
|
||||||
|
- validates prerequisite stage success and object-store availability.
|
||||||
|
- collects deterministic run file list plus run `manifest.json`.
|
||||||
|
- resolves publish output sources through runtime artifact catalog and manifest-aware resolution.
|
||||||
|
- selected artifact filter applies to configured artifact sources only.
|
||||||
|
- locked outputs are skipped intentionally (including required ones).
|
||||||
|
- optional missing outputs are skipped; required missing unlocked outputs fail.
|
||||||
|
- writes remote current manifest before current run pointer.
|
||||||
|
|
||||||
|
## Metadata Signals
|
||||||
|
Includes counts/lists for:
|
||||||
|
- run uploads
|
||||||
|
- published output uploads
|
||||||
|
- previous uploads
|
||||||
|
- skipped optional outputs
|
||||||
|
- skipped unselected outputs
|
||||||
|
- locked outputs
|
||||||
|
- current-state key paths
|
||||||
|
- `current_pointer_written`
|
||||||
|
|
||||||
|
## Invariants
|
||||||
|
- `current/run_id.txt` is the remote commit marker and is written last.
|
||||||
|
- run upload excludes `audio/**`.
|
||||||
|
- publish locks are not overridden by `--force`.
|
||||||
@@ -1,58 +1,22 @@
|
|||||||
# Stage: transcribe
|
# Stage: transcribe
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Generate per-speaker raw transcripts from prepared audio using WhisperX.
|
Generate raw per-speaker transcripts from prepared audio using WhisperX.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
- `audio/*.flac` from `prepare`
|
||||||
- `audio/*.flac` prepared by `prepare`
|
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- `transcripts/raw/<speaker>.json` for each input audio file
|
- `transcripts/raw/<speaker>.json`
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- discovers prepared audio from manifest inputs or canonical audio directory.
|
||||||
- Discovering prepared audio inputs
|
- derives speaker ID from `.flac` basename.
|
||||||
- Deriving speaker ids from audio basenames
|
- runs WhisperX with configured concurrency/retry settings.
|
||||||
- Parallel WhisperX invocation with bounded concurrency
|
- validates each output as JSON.
|
||||||
- Validating produced JSON and promoting run-local outputs
|
- writes run-local outputs then materializes canonical transcript outputs.
|
||||||
|
|
||||||
Does not own:
|
## Invariants
|
||||||
- Transcript merge/polish/normalize/trim/analyze
|
- speaker basenames must be unique.
|
||||||
|
- output path returned by adapter must match requested output path.
|
||||||
## Config Fields Used
|
- each successful output is validated before stage success.
|
||||||
- `session.session_id`
|
|
||||||
- `session.campaign`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `pipeline.whisperx.transcribe_url`
|
|
||||||
- `pipeline.whisperx.language`
|
|
||||||
- `pipeline.whisperx.timeout`
|
|
||||||
- `pipeline.whisperx.retries`
|
|
||||||
- `pipeline.whisperx.retry_delay`
|
|
||||||
- `pipeline.whisperx.concurrency`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- WhisperX adapter (`env.WhisperX.Transcribe`).
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Uses run-local output paths under `runs/{run_id}/transcribe/outputs/...` when run layout is enabled.
|
|
||||||
- Validates each generated transcript JSON before promotion.
|
|
||||||
- Promotes canonical outputs to `transcripts/raw/*.json`.
|
|
||||||
- Records per-file metadata (attempts/status/duration/output path) in stage metadata.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies for previously succeeded stage unless forced.
|
|
||||||
- On forced upstream reruns, downstream succeeded stages can be marked `stale` by runner logic.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails if no prepared audio exists, duplicate speaker basenames are detected, adapter output path mismatches expected path, any output JSON is invalid, or one worker fails.
|
|
||||||
- Cancels in-flight workers after first terminal error.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/transcribe_test.go`
|
|
||||||
- `internal/app/whisperx_wiring_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Speaker identity is derived from `.flac` basename and must be unique.
|
|
||||||
- Every successful speaker output must be valid JSON before promotion.
|
|
||||||
- Canonical raw transcript set is the only supported merge input surface.
|
|
||||||
|
|||||||
@@ -1,75 +1,28 @@
|
|||||||
# Stage: trim
|
# Stage: trim
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Optionally trim the normalized transcript to session bounds; always produce a durable trimmed transcript.
|
Produce a final-trimmed transcript; optionally generate bounds-driven trim.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
- `transcripts/final.json`
|
||||||
- `transcripts/normalized.json`
|
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- `transcripts/trimmed.json` (or configured trim output path)
|
- `transcripts/final.trimmed.json` (or configured trim output path)
|
||||||
- when trim enabled: `artifacts/session_bounds.json`
|
- when trim enabled: `artifacts/session_bounds.json`
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
When `trim.enabled=false`:
|
||||||
- Trim-enabled switch behavior
|
- copies normalized transcript to trimmed output.
|
||||||
- Bounds generation via Scriptorium artifact run
|
|
||||||
- Bounds validation against normalized transcript
|
|
||||||
- Keep-selector derivation and Seriatim trim invocation
|
|
||||||
- Copy-through behavior when disabled or bounds indicate unchanged transcript
|
|
||||||
|
|
||||||
Does not own:
|
When `trim.enabled=true`:
|
||||||
- Upstream normalization
|
- runs Scriptorium bounds artifact generation;
|
||||||
- Downstream artifact analysis
|
- optionally runs render-debug output generation;
|
||||||
|
- validates bounds payload against transcript;
|
||||||
|
- derives keep selector;
|
||||||
|
- either copies unchanged transcript or runs Seriatim trim;
|
||||||
|
- validates trimmed transcript and materializes bounds output.
|
||||||
|
|
||||||
## Config Fields Used
|
## Invariants
|
||||||
- `session.session_id`
|
- normalized transcript is required input.
|
||||||
- `session.campaign`
|
- bounds output exists only in enabled trim path.
|
||||||
- `pipeline.workspace.root`
|
- render-debug output is diagnostic and not a declared stage output.
|
||||||
- `pipeline.trim.enabled`
|
|
||||||
- `pipeline.trim.output_path`
|
|
||||||
- `pipeline.trim.bounds.prompt_id`
|
|
||||||
- `pipeline.trim.bounds.profile_id`
|
|
||||||
- `pipeline.trim.bounds.timeout`
|
|
||||||
- `pipeline.trim.bounds.output_path`
|
|
||||||
- `pipeline.trim.bounds.transcript_input_name`
|
|
||||||
- `pipeline.trim.bounds.render_debug`
|
|
||||||
- `pipeline.trim.bounds.render_output_path`
|
|
||||||
- `pipeline.seriatim.binary`
|
|
||||||
- `pipeline.seriatim.timeout`
|
|
||||||
- `pipeline.scriptorium.binary`
|
|
||||||
- `pipeline.scriptorium.config_path`
|
|
||||||
- `pipeline.scriptorium.timeout`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- Scriptorium adapter:
|
|
||||||
- optional `RenderArtifact` for bounds debug render
|
|
||||||
- `RunArtifact` for bounds output
|
|
||||||
- Seriatim adapter:
|
|
||||||
- `Trim` when bounds indicate trimming is required
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Reads normalized transcript from normalize manifest outputs when available; falls back to canonical path.
|
|
||||||
- Uses run-local outputs/logs/reports/config/scratch paths when run layout is enabled.
|
|
||||||
- Promotes canonical trimmed transcript; promotes session bounds when trim enabled.
|
|
||||||
- Records bounds diagnostics, trim action, keep selector, and adapter metadata.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies when already succeeded and not forced.
|
|
||||||
- Forced reruns can stale downstream succeeded stages.
|
|
||||||
- When `trim.enabled=false`, stage still succeeds by copying normalized to trimmed output.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on missing/invalid normalized transcript.
|
|
||||||
- With trim enabled, fails on missing adapters/config, bounds generation/validation errors, invalid bounds JSON, invalid range/segment ids, trim adapter failures, or invalid trimmed output.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/trim_test.go`
|
|
||||||
- `internal/adapters/scriptorium/subprocess_test.go`
|
|
||||||
- `internal/adapters/seriatim/subprocess_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Trim never falls back to processed transcript; normalized transcript is required input.
|
|
||||||
- `session_bounds` output exists only for enabled trim path.
|
|
||||||
- Render-debug artifacts are diagnostics and not declared stage outputs.
|
|
||||||
|
|||||||
@@ -1,71 +1,37 @@
|
|||||||
# Internal: Storage
|
# Internal: Storage
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Document Narratio's remote storage backend contracts and implementations under `internal/adapters/storage`.
|
Document remote object-store contracts and S3 implementation behavior.
|
||||||
|
|
||||||
## Inputs and outputs
|
## Primary Contract
|
||||||
Inputs:
|
`storage.ObjectStore` interface:
|
||||||
- Resolved storage config (`pipeline.storage.*`).
|
- `List(ctx, prefix)`
|
||||||
- Bucket-relative object keys and local file paths from stage/app orchestration.
|
- `Download(ctx, key, localPath)`
|
||||||
|
- `Upload(ctx, localPath, key, opts)`
|
||||||
|
- `Exists(ctx, key)`
|
||||||
|
|
||||||
Outputs:
|
Key invariant:
|
||||||
- Listed/downloaded/uploaded object metadata (`ObjectInfo`).
|
- callers pass full bucket-relative keys;
|
||||||
- Existence checks and storage-layer errors.
|
- storage implementations do not infer campaign/session/run prefixes.
|
||||||
|
|
||||||
## Boundaries
|
## Configuration
|
||||||
Owns:
|
`NewObjectStoreFromConfig` currently supports S3-backed stores from `pipeline.storage.*` config.
|
||||||
- Remote object-store interface and implementation details.
|
|
||||||
- S3 client wiring and API calls.
|
|
||||||
- Object key normalization and upload/download/list primitives.
|
|
||||||
|
|
||||||
Does not own:
|
S3 constructor behavior:
|
||||||
- Session/run prefix semantics.
|
- requires configured bucket;
|
||||||
- Archive commit order semantics.
|
- uses region/endpoint/path-style options when set;
|
||||||
- Manifest updates.
|
- resolves credentials from configured env var names (with defaults).
|
||||||
|
|
||||||
## Config fields used
|
## S3 Backend Behavior
|
||||||
- `pipeline.storage.backend`
|
- normalizes object keys.
|
||||||
- `pipeline.storage.s3.bucket`
|
- `List` paginates and returns normalized `ObjectInfo`.
|
||||||
- `pipeline.storage.s3.region`
|
- `Download` writes local files with parent directory creation.
|
||||||
- `pipeline.storage.s3.endpoint`
|
- `Upload` streams local file and returns remote metadata.
|
||||||
- `pipeline.storage.s3.force_path_style`
|
- `Exists` maps not-found responses to `false`.
|
||||||
- `pipeline.storage.s3.access_key_id_env`
|
|
||||||
- `pipeline.storage.s3.secret_access_key_env`
|
|
||||||
|
|
||||||
## External adapters used
|
## Legacy Compatibility Interface
|
||||||
Storage package contracts:
|
`storage.Backend` (with `ArchiveRequest`) remains as compatibility surface with `NoopBackend`; it is not used by current stage execution.
|
||||||
- `ObjectStore` (active remote object-store boundary): `List`, `Download`, `Upload`, `Exists`.
|
|
||||||
- `Backend` (archive request boundary): currently implemented with `NoopBackend` only.
|
|
||||||
|
|
||||||
Implementations:
|
## Invariants
|
||||||
- `S3Backend`: AWS SDK-backed `ObjectStore` implementation.
|
- storage layer is stateless regarding manifest/stage progression.
|
||||||
- `FakeBackend`: deterministic test `ObjectStore` and archive backend.
|
- publish ordering semantics are owned by stage/app code, not storage adapters.
|
||||||
- `NoopBackend`: deterministic no-op archive backend for compatibility wiring.
|
|
||||||
|
|
||||||
## State and manifest behavior
|
|
||||||
- Storage implementations are stateless with respect to manifest/session lifecycle.
|
|
||||||
- Caller supplies fully-qualified bucket-relative keys.
|
|
||||||
- Storage layer does not infer campaign/session/run/root-prefix semantics.
|
|
||||||
- Caller controls publish ordering; storage layer executes individual operations in the order invoked.
|
|
||||||
|
|
||||||
## Skip and resume behavior
|
|
||||||
- No storage-level skip/resume behavior.
|
|
||||||
- Skip/resume decisions are made by stage/app logic before storage calls occur.
|
|
||||||
|
|
||||||
## Failure behavior
|
|
||||||
- `NewObjectStoreFromConfig` fails when no remote backend is configured or required S3 config is missing.
|
|
||||||
- `S3Backend` constructor fails when required bucket is missing or AWS client setup fails.
|
|
||||||
- CRUD operations return contextual errors (including not-found behavior via `Exists`).
|
|
||||||
- Key normalization is applied before operations (`\\` to `/`, leading slash trimmed).
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
|
||||||
- `internal/adapters/storage/factory_test.go`
|
|
||||||
- `internal/adapters/storage/s3_backend_test.go`
|
|
||||||
- `internal/adapters/storage/fake_test.go`
|
|
||||||
- `internal/adapters/storage/keys_test.go`
|
|
||||||
- `internal/adapters/storage/archive.go` + consumers in stage tests (`prepare`, `archive`)
|
|
||||||
|
|
||||||
## Architectural invariants
|
|
||||||
- Callers pass full bucket-relative keys.
|
|
||||||
- Storage backends must not prepend or infer narratio prefixes.
|
|
||||||
- Remote transport details remain isolated to storage adapter implementations.
|
|
||||||
|
|||||||
@@ -1,68 +1,57 @@
|
|||||||
# Workspace internals
|
# Internal: Workspace
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define the local durable and run-local workspace model used by stages, manifests, resume, and archive.
|
Define local session layout, run-local stage layout, and cleanup guardrails.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Canonical Session Layout
|
||||||
Inputs:
|
Session root:
|
||||||
- `pipeline.workspace.root`
|
- `{workspace.root}/work/{campaign}/{session_id}`
|
||||||
- `session.campaign`
|
|
||||||
- `session.session_id`
|
|
||||||
- generated `run_id`
|
|
||||||
|
|
||||||
Outputs:
|
Core directories/files:
|
||||||
- Session manifest at `{workspace.root}/work/{campaign}/{session_id}/manifest.json`
|
- `inputs/`
|
||||||
- Run manifest at `{workspace.root}/work/{campaign}/{session_id}/runs/{run_id}/manifest.json`
|
- `audio/`
|
||||||
- Canonical durable session directories and run-local stage trees
|
- `transcripts/`
|
||||||
|
- `artifacts/`
|
||||||
|
- `reports/`
|
||||||
|
- `logs/`
|
||||||
|
- `config/`
|
||||||
|
- `current/`
|
||||||
|
- `runs/`
|
||||||
|
- `previous/`
|
||||||
|
- `manifest.json`
|
||||||
|
- `.lock`
|
||||||
|
|
||||||
## Boundaries
|
`previous/` reserved files:
|
||||||
Owns:
|
- `previous/manifest.json`
|
||||||
- Session-level path layout (`inputs/`, `audio/`, `transcripts/`, `artifacts/`, `reports/`, `logs/`, `config/`, `current/`, `runs/`)
|
- `previous/artifacts/**`
|
||||||
- Run-local stage sandbox layout under `runs/{run_id}/{stage}/`
|
|
||||||
- Session lock acquisition/release (`.lock`)
|
|
||||||
|
|
||||||
Does not own:
|
## Run-Local Stage Layout
|
||||||
- Stage business logic
|
When run context is available, stages use:
|
||||||
- Remote archive semantics (documented in `stage-archive.md`)
|
- `runs/{run_id}/{stage}/outputs/`
|
||||||
- CLI argument parsing
|
- `runs/{run_id}/{stage}/logs/`
|
||||||
|
- `runs/{run_id}/{stage}/reports/`
|
||||||
|
- `runs/{run_id}/{stage}/config/`
|
||||||
|
- `runs/{run_id}/{stage}/scratch/`
|
||||||
|
|
||||||
## Config Fields Used
|
Run-local outputs are materialized back into canonical session paths before stage success.
|
||||||
- `pipeline.workspace.root`
|
`previous/**` writes are never redirected to run-local output paths.
|
||||||
- `pipeline.workspace.cleanup_after_archive`
|
|
||||||
- `pipeline.spool.root`
|
|
||||||
- `pipeline.spool.delete_audio_after_archive`
|
|
||||||
- `session.campaign`
|
|
||||||
- `session.session_id`
|
|
||||||
|
|
||||||
## External Adapters Used
|
## Locking
|
||||||
None directly in this subsystem. Stages may use object storage adapters and then write local outputs into this layout.
|
`artifacts.LocalStore` enforces single-writer session lock via `.lock` file (`ErrLockConflict` on contention).
|
||||||
|
|
||||||
## State and Manifest Behavior
|
## Cleanup Semantics
|
||||||
- Session state is persisted in the session manifest (`manifest.Manifest`).
|
Automatic post-publish cleanup (`runPostArchiveCleanup`):
|
||||||
- Invocation history is persisted per run in run manifests under `runs/{run_id}/manifest.json`.
|
- only runs when publish actually executed and succeeded;
|
||||||
- During each run, stage outputs are often written run-local first (`runs/{run_id}/{stage}/outputs/...`) and promoted to canonical session paths after stage success.
|
- requires `uploaded=true` and `current_pointer_written=true` metadata;
|
||||||
- `manifest.Artifacts` entries record `ProducerRunID` for durable outputs.
|
- respects `pipeline.spool.delete_audio_after_publish` and `pipeline.workspace.cleanup_after_publish`;
|
||||||
- For S3 audio sessions, `prepare` records spool/work paths and S3 provenance in `manifest.Inputs`.
|
- refuses unsafe deletes (root delete, out-of-root delete, symlink paths).
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
Manual clean command:
|
||||||
- Skip/resume decisions are made in `internal/app` (`run_control.go`, `resume.go`) using stage status in the session manifest.
|
- `clean <session_id>` removes session work and spool subtree.
|
||||||
- `--force` reruns selected stages and marks downstream previously-succeeded stages as `stale`.
|
- `clean --all` removes all workspace work and spool children.
|
||||||
- Workspace layout is idempotent (`EnsureLayoutFor`) and reused across runs.
|
- durable cache is preserved unless `--clear-cache` is requested.
|
||||||
|
|
||||||
## Failure Behavior
|
## Invariants
|
||||||
- Failures preserve manifests and run-local files for inspection.
|
- campaign-aware session root is mandatory.
|
||||||
- Lock conflicts fail fast via `ErrLockConflict`.
|
- manifest-driven stage state is durable across runs.
|
||||||
- Cleanup can fail post-archive; failure is recorded in archive stage metadata and returned by the run.
|
- cleanup guardrails prevent destructive root/out-of-scope deletion.
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/artifacts/local_test.go`
|
|
||||||
- `internal/stage/run_local_test.go`
|
|
||||||
- `internal/app/run_control_test.go`
|
|
||||||
- `internal/app/resume_run_stage_test.go`
|
|
||||||
- `internal/app/post_archive_cleanup_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Session root is campaign-aware: `{workspace.root}/work/{campaign}/{session_id}`.
|
|
||||||
- Run roots are always nested: `runs/{run_id}` under the session root.
|
|
||||||
- Run-local output promotion must end in canonical session paths.
|
|
||||||
- Cleanup only targets run-scoped directories and must never delete configured root directories.
|
|
||||||
|
|||||||
@@ -1,213 +1,249 @@
|
|||||||
# Operations
|
# Operations Guide
|
||||||
|
|
||||||
This guide describes the implemented operator lifecycle for Narratio.
|
Operator workflow for running, recovering, and publishing Narratio sessions.
|
||||||
|
|
||||||
For field-level configuration, see [docs/config.md](./config.md). For full command/flag reference, see [docs/cli.md](./cli.md).
|
For command syntax, see [docs/cli.md](./cli.md). For field-level config, see [docs/config.md](./config.md).
|
||||||
|
|
||||||
## Normal workflow (S3-first path)
|
## Standard Session Workflow
|
||||||
|
|
||||||
1. Upload session `.flac` files to object storage under the configured session audio prefix.
|
1. Select pipeline/campaign/session config.
|
||||||
2. Run Narratio:
|
2. Validate session readiness:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run --session-id 2026-04-04
|
narratio session validate 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
3. Read success output:
|
3. (Optional) inspect stage decisions:
|
||||||
- `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
|
|
||||||
- use `manifest=<path>` with `status` for inspection.
|
|
||||||
|
|
||||||
Notes:
|
|
||||||
- default config/session discovery applies unless `--config` and `--session` are passed.
|
|
||||||
- S3 audio mode requires `session.inputs.audio_s3.prefix` and valid object-store access.
|
|
||||||
|
|
||||||
## Restore workflow
|
|
||||||
|
|
||||||
Use restore when local durable session state is missing or stale and archive current state is authoritative.
|
|
||||||
|
|
||||||
Dry-run (no local writes):
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio restore --session-id 2026-04-04 --dry-run
|
narratio session plan 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Execution:
|
4. Run the pipeline:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio restore --session-id 2026-04-04
|
narratio run 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Post-restore analyze rerun pattern:
|
5. Check state:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run-stage --session-id 2026-04-04 --force analyze
|
narratio session status 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Restore source-of-truth:
|
## Campaign and Session Selection
|
||||||
- remote commit marker: `current/run_id.txt`
|
|
||||||
- remote current manifest: `current/manifest.json`
|
|
||||||
|
|
||||||
Restore default scope:
|
Campaign selection priority:
|
||||||
- includes `manifest.json`, `transcripts/**`, `artifacts/**`
|
|
||||||
- includes `audio/**` only with `--include-audio`
|
|
||||||
- excludes `runs/**`, `logs/**`, `reports/**`, `config/**`, `inputs/**`, and `current/**` (except remote `current/manifest.json` as source)
|
|
||||||
|
|
||||||
## Local filesystem layout and state artifacts
|
- `--campaign-file`
|
||||||
|
- `--campaign`
|
||||||
|
- `pipeline.campaigns.default_campaign_id`
|
||||||
|
|
||||||
|
Session source priority:
|
||||||
|
|
||||||
|
- `--session`
|
||||||
|
- local default search paths
|
||||||
|
- remote session object (S3) when local session file is not found and storage is configured
|
||||||
|
|
||||||
|
## Session Initialization
|
||||||
|
|
||||||
|
Use `session init` to generate a concrete session file for local or remote use.
|
||||||
|
|
||||||
|
Local file:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session init 2026-04-04 --output ./session.yml --date 2026-04-04 --title "Session 12"
|
||||||
|
```
|
||||||
|
|
||||||
|
Remote session object:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session init 2026-04-04 --remote --force
|
||||||
|
```
|
||||||
|
|
||||||
|
If `campaign.yml` sets `session_template_file`, `session init` renders it. Template variables must resolve to concrete values.
|
||||||
|
|
||||||
|
## Stage Execution and Resume Behavior
|
||||||
|
|
||||||
|
Canonical stage order:
|
||||||
|
|
||||||
|
1. `prepare`
|
||||||
|
2. `transcribe`
|
||||||
|
3. `merge`
|
||||||
|
4. `polish`
|
||||||
|
5. `normalize`
|
||||||
|
6. `trim`
|
||||||
|
7. `analyze`
|
||||||
|
8. `publish`
|
||||||
|
9. `notify`
|
||||||
|
|
||||||
|
Execution rules:
|
||||||
|
|
||||||
|
- succeeded stages are skipped unless `--force` is set;
|
||||||
|
- `resume` starts at first non-succeeded stage;
|
||||||
|
- force rerunning a succeeded upstream stage marks succeeded downstream stages as `stale`.
|
||||||
|
|
||||||
|
Single-stage execution:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio run-stage normalize 2026-04-04 --force
|
||||||
|
```
|
||||||
|
|
||||||
|
## Artifact Selection
|
||||||
|
|
||||||
|
`--artifacts` can be used on `run`, `resume`, `run-stage`, `analyze`, and `publish`.
|
||||||
|
|
||||||
|
Selection behavior:
|
||||||
|
|
||||||
|
- validates names against `pipeline.scriptorium.artifacts`;
|
||||||
|
- filters analyze execution to selected configured artifacts;
|
||||||
|
- filters publish rules for `narratio.artifact.<name>` sources only;
|
||||||
|
- does not suppress built-in transcript or bounds publish sources.
|
||||||
|
|
||||||
|
## Publish Workflow
|
||||||
|
|
||||||
|
Run publish only:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio publish 2026-04-04
|
||||||
|
```
|
||||||
|
|
||||||
|
Equivalent:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio run-stage publish 2026-04-04 --force
|
||||||
|
```
|
||||||
|
|
||||||
|
Publish commit model:
|
||||||
|
|
||||||
|
- uploads run files under `{session_prefix}/runs/{run_id}/`;
|
||||||
|
- uploads configured published outputs;
|
||||||
|
- uploads `previous/**` cache files when present;
|
||||||
|
- writes `current/manifest.json`;
|
||||||
|
- writes `current/run_id.txt` last.
|
||||||
|
|
||||||
|
`current/run_id.txt` is the remote current-state commit marker.
|
||||||
|
|
||||||
|
## Publish Locks
|
||||||
|
|
||||||
|
Lock sources:
|
||||||
|
|
||||||
|
- static locks in `pipeline.publish.locks`
|
||||||
|
- mutable remote locks in `{session_prefix}/locks.yml`
|
||||||
|
|
||||||
|
Effective lock rules:
|
||||||
|
|
||||||
|
- static and remote locks are merged;
|
||||||
|
- static locks win on source collisions;
|
||||||
|
- locked outputs are intentional skips;
|
||||||
|
- lock add/remove commands mutate only remote lock state.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session locks 2026-04-04
|
||||||
|
narratio session locks add 2026-04-04 narratio.artifact.session_recap --reason "manual edits" --force
|
||||||
|
narratio session locks remove 2026-04-04 narratio.artifact.session_recap
|
||||||
|
```
|
||||||
|
|
||||||
|
## Restore Workflow
|
||||||
|
|
||||||
|
Use restore when local durable session state is missing or stale and remote committed current state is authoritative.
|
||||||
|
|
||||||
|
Dry run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session restore 2026-04-04 --dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
Apply:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session restore 2026-04-04
|
||||||
|
```
|
||||||
|
|
||||||
|
Default restore scope:
|
||||||
|
|
||||||
|
- `manifest.json`
|
||||||
|
- `transcripts/**`
|
||||||
|
- `artifacts/**`
|
||||||
|
- `previous/**` when needed by configured previous-session artifact inputs
|
||||||
|
|
||||||
|
Optional:
|
||||||
|
|
||||||
|
- `--include-audio` to include `audio/**`
|
||||||
|
- `--force` to overwrite local conflicts
|
||||||
|
|
||||||
|
Restore writes an execution report at `reports/restore-latest.json`.
|
||||||
|
|
||||||
|
## Local State Layout
|
||||||
|
|
||||||
Session root:
|
Session root:
|
||||||
- `{workspace.root}/work/{campaign}/{session_id}/`
|
|
||||||
|
|
||||||
Primary state:
|
- `{workspace.root}/work/{campaign}/{session_id}`
|
||||||
- `manifest.json`: session-level stage state.
|
|
||||||
- `runs/{run_id}/manifest.json`: invocation-level state.
|
|
||||||
- `.lock`: session lock while a modifying command is active.
|
|
||||||
|
|
||||||
Canonical session directories:
|
Durable session paths:
|
||||||
- `inputs/`
|
|
||||||
- `audio/`
|
|
||||||
- `transcripts/`
|
|
||||||
- `artifacts/`
|
|
||||||
- `reports/`
|
|
||||||
- `logs/`
|
|
||||||
- `config/`
|
|
||||||
- `current/`
|
|
||||||
- `runs/`
|
|
||||||
|
|
||||||
Run-local stage directories:
|
- `manifest.json`
|
||||||
- `runs/{run_id}/{stage}/` with stage-local `outputs/`, `logs/`, `reports/`, `config/`, `scratch/`.
|
- `inputs/**`
|
||||||
|
- `audio/**`
|
||||||
|
- `transcripts/**`
|
||||||
|
- `artifacts/**`
|
||||||
|
- `previous/**`
|
||||||
|
- `reports/**`
|
||||||
|
- `logs/**`
|
||||||
|
- `config/**`
|
||||||
|
- `runs/**`
|
||||||
|
|
||||||
Behavior:
|
Run-local layout:
|
||||||
- directory creation is idempotent.
|
|
||||||
- stage outputs are generally generated run-local first, then promoted to canonical paths on success.
|
|
||||||
- restore installs downloaded files to canonical session paths and does not recreate historical run sandboxes.
|
|
||||||
|
|
||||||
## Analyze artifact execution lifecycle
|
- `runs/{run_id}/{stage}/outputs`
|
||||||
|
- `runs/{run_id}/{stage}/logs`
|
||||||
|
- `runs/{run_id}/{stage}/reports`
|
||||||
|
- `runs/{run_id}/{stage}/config`
|
||||||
|
- `runs/{run_id}/{stage}/scratch`
|
||||||
|
|
||||||
Analyze executes configured artifacts from `pipeline.scriptorium.artifacts`.
|
Spool layout (runtime/transient):
|
||||||
|
|
||||||
Execution model:
|
- `{spool.root}/{campaign}/{session_id}/{run_id}/...`
|
||||||
- executable set = enabled artifacts, filtered by `--artifacts` when provided.
|
- restore audio spool under `{spool.root}/{campaign}/{session_id}/restore/audio`
|
||||||
- artifact-to-artifact dependencies are declared via `depends_on`.
|
|
||||||
- selected artifacts run in deterministic dependency order.
|
|
||||||
- after each successful artifact run, output is promoted to configured canonical `output_path`.
|
|
||||||
|
|
||||||
Configured artifact source reuse:
|
Cache layout (durable S3 audio cache):
|
||||||
- a non-executable configured artifact can satisfy inputs if its configured output file already exists and is valid.
|
|
||||||
- reused configured artifact provenance is `filesystem.disabled_artifact_output`.
|
|
||||||
|
|
||||||
`--artifacts` behavior:
|
- `{cache.root}/s3/{bucket}/...`
|
||||||
- accepted on `run`, `resume`, and `run-stage analyze`.
|
|
||||||
- filters analyze execution only; does not force stage rerun.
|
|
||||||
|
|
||||||
## Remote archive layout and publish contract
|
## Cleanup
|
||||||
|
|
||||||
When archive is enabled and run upload is enabled, archive publishes under:
|
Session-scoped cleanup:
|
||||||
|
|
||||||
- session prefix: `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/`
|
|
||||||
- run prefix: `{session_prefix}/runs/{run_id}/`
|
|
||||||
|
|
||||||
Archive uploads:
|
|
||||||
- run record files from run root (excluding `audio/`).
|
|
||||||
- promoted files from explicit `archive.promote_artifacts` rules.
|
|
||||||
|
|
||||||
Publish order:
|
|
||||||
1. upload `current/manifest.json`
|
|
||||||
2. upload `current/run_id.txt` last
|
|
||||||
|
|
||||||
`current/run_id.txt` is the remote commit marker.
|
|
||||||
|
|
||||||
Archive promotion is explicit and source-based:
|
|
||||||
- Narratio does not auto-promote all generated analyze artifacts.
|
|
||||||
- each rule resolves `source` through the artifact resolver/catalog model, then uploads to `dest`.
|
|
||||||
- missing required promotion sources fail archive stage.
|
|
||||||
- missing optional promotion sources are skipped.
|
|
||||||
- invalid resolved artifacts fail archive stage.
|
|
||||||
|
|
||||||
## Resume, retry, restore, and safe rerun behavior
|
|
||||||
|
|
||||||
Default skip:
|
|
||||||
- `run` and `run-stage` skip already-succeeded stages unless `--force` is set.
|
|
||||||
|
|
||||||
Resume:
|
|
||||||
- `resume` starts at first non-succeeded stage.
|
|
||||||
- `resume --force` runs full stage order.
|
|
||||||
|
|
||||||
Restore conflict policy:
|
|
||||||
- restore classifies local differences as conflicts.
|
|
||||||
- without `--force`, restore fails when conflicts exist.
|
|
||||||
- with `--force`, conflicting local files are overwritten by remote archive files.
|
|
||||||
|
|
||||||
Forced reruns:
|
|
||||||
- force-rerunning an upstream succeeded stage marks downstream succeeded stages as `stale`.
|
|
||||||
|
|
||||||
Safe rerun pattern:
|
|
||||||
1. rerun the changed stage with `--force`.
|
|
||||||
2. run `resume` to rebuild downstream stages.
|
|
||||||
|
|
||||||
## Cleanup behavior
|
|
||||||
|
|
||||||
Cleanup is considered only when archive stage executed and succeeded.
|
|
||||||
|
|
||||||
Cleanup toggles:
|
|
||||||
- `pipeline.spool.delete_audio_after_archive=true` deletes run-scoped spool audio.
|
|
||||||
- `pipeline.workspace.cleanup_after_archive=true` deletes run-scoped local run directory.
|
|
||||||
|
|
||||||
Cleanup eligibility gates:
|
|
||||||
- archive enabled
|
|
||||||
- archive run upload enabled
|
|
||||||
- run record upload completed
|
|
||||||
- current pointer write completed (`current/run_id.txt` written)
|
|
||||||
|
|
||||||
No cleanup for failed/incomplete/unarchived/archive-skipped runs.
|
|
||||||
|
|
||||||
## Failure and recovery playbooks
|
|
||||||
|
|
||||||
After run failure, Narratio keeps:
|
|
||||||
- session manifest
|
|
||||||
- run manifest
|
|
||||||
- run-local artifacts/logs/config/reports
|
|
||||||
|
|
||||||
Failed or incomplete runs remain local-only.
|
|
||||||
|
|
||||||
After restore failure:
|
|
||||||
- already-installed restore files remain in place.
|
|
||||||
- restore does not roll back prior successful installs.
|
|
||||||
- existing local manifest is preserved if restored manifest validation/install fails.
|
|
||||||
|
|
||||||
Recommended recovery:
|
|
||||||
|
|
||||||
1. inspect state:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio status --manifest <manifest-path>
|
narratio clean 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
2. for restore-specific checks, run:
|
Global cleanup:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio restore --session-id 2026-04-04 --dry-run
|
narratio clean --all
|
||||||
```
|
```
|
||||||
|
|
||||||
3. fix root cause (config/input/credentials/storage/service availability).
|
Dry-run and cache variants:
|
||||||
4. continue with `resume`, or targeted `run-stage --force` followed by `resume`.
|
|
||||||
|
|
||||||
## Restore report
|
```bash
|
||||||
|
narratio clean 2026-04-04 --dry-run --clear-cache
|
||||||
|
narratio clean --all --dry-run --clear-cache
|
||||||
|
```
|
||||||
|
|
||||||
Non-dry-run restore writes a durable report at:
|
Rules:
|
||||||
- `reports/restore-latest.json`
|
|
||||||
|
|
||||||
Report content includes:
|
- `clean` deletes work/spool session state;
|
||||||
- identity (`campaign`, `session_id`, `run_id`)
|
- cache is preserved unless `--clear-cache` is set;
|
||||||
- mode flags (`dry_run`, `force`, `include_audio`)
|
- automatic post-publish cleanup is gated by successful publish commit plus:
|
||||||
- plan counts and execution counts
|
- `pipeline.spool.delete_audio_after_publish=true`
|
||||||
- per-action status
|
- `pipeline.workspace.cleanup_after_publish=true`
|
||||||
|
|
||||||
Dry-run does not write restore report files.
|
## Operational Caveats
|
||||||
|
|
||||||
## Operational caveats
|
- Local and S3 audio modes are mutually exclusive.
|
||||||
|
- Publish requires prerequisite stages through analyze to be succeeded.
|
||||||
- `status` requires explicit `--manifest`; there is no session-id lookup command.
|
- Restore requires configured object storage and committed remote current state.
|
||||||
- local and S3 audio input modes are mutually exclusive.
|
- Storage-backed commands load filesystem secrets before object-store initialization.
|
||||||
- archive publish requires upstream stages through `analyze` to be `succeeded`.
|
|
||||||
- required promotion rules can fail when selected analyze artifacts did not generate a required file path.
|
|
||||||
- restore requires configured remote object storage and committed remote current state.
|
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ It coordinates specialized downstream systems rather than reimplementing their d
|
|||||||
- Audita handles transcript correction and polishing.
|
- Audita handles transcript correction and polishing.
|
||||||
- Scriptorium handles prompt execution and generated artifacts.
|
- Scriptorium handles prompt execution and generated artifacts.
|
||||||
|
|
||||||
Narratio owns orchestration, configuration loading, session/run state, local and remote path modeling, manifest persistence, stage sequencing, resume behavior, and archive semantics.
|
Narratio owns orchestration, configuration loading, session/run state, local and remote path modeling, manifest persistence, stage sequencing, resume behavior, and publish semantics.
|
||||||
|
|
||||||
Narratio should remain explicit and comprehensible. It is not intended to become a generic workflow engine.
|
Narratio should remain explicit and comprehensible. It is not intended to become a generic workflow engine.
|
||||||
|
|
||||||
@@ -80,7 +80,7 @@ It should record:
|
|||||||
- input and output refs;
|
- input and output refs;
|
||||||
- logs and generated config refs;
|
- logs and generated config refs;
|
||||||
- checksums or provenance where useful;
|
- checksums or provenance where useful;
|
||||||
- non-secret adapter and archive metadata.
|
- non-secret adapter and publish metadata.
|
||||||
|
|
||||||
Resume behavior should be manifest-driven. Filesystem state may be inspected and validated, but it should not replace manifest stage state as the source of run progress.
|
Resume behavior should be manifest-driven. Filesystem state may be inspected and validated, but it should not replace manifest stage state as the source of run progress.
|
||||||
|
|
||||||
@@ -117,19 +117,19 @@ Narratio should not become a secondary configuration system for downstream tools
|
|||||||
|
|
||||||
Local and remote paths are part of Narratio’s application contract.
|
Local and remote paths are part of Narratio’s application contract.
|
||||||
|
|
||||||
Code should use centralized path helpers for workspace, spool, session, run, artifact, log, config, and archive paths. Stages should avoid reconstructing canonical paths through scattered string concatenation.
|
Code should use centralized path helpers for workspace, spool, session, run, artifact, log, config, and publish/current paths. Stages should avoid reconstructing canonical paths through scattered string concatenation.
|
||||||
|
|
||||||
Storage backends should receive explicit bucket-relative keys. Storage implementations should not infer campaign, session, run, or root-prefix semantics.
|
Storage backends should receive explicit bucket-relative keys. Storage implementations should not infer campaign, session, run, or root-prefix semantics.
|
||||||
|
|
||||||
## Archive Invariants
|
## Publish Invariants
|
||||||
|
|
||||||
Archive behavior must preserve a clear commit boundary.
|
Publish behavior must preserve a clear commit boundary.
|
||||||
|
|
||||||
A remote run is current only after the archive stage has successfully uploaded the run record, required promoted outputs, `current/manifest.json`, and finally `current/run_id.txt`.
|
A remote run is current only after the publish stage has successfully uploaded the run record, required published outputs, `current/manifest.json`, and finally `current/run_id.txt`.
|
||||||
|
|
||||||
`current/run_id.txt` is the final remote commit marker and must be written last.
|
`current/run_id.txt` is the final remote commit marker and must be written last.
|
||||||
|
|
||||||
Failed, incomplete, skipped, or uncommitted archive attempts must not be presented as current remote state. Local cleanup is permitted only after successful archive commit and only when explicitly configured.
|
Failed, incomplete, skipped, or uncommitted publish attempts must not be presented as current remote state. Local cleanup is permitted only after successful publish commit and only when explicitly configured.
|
||||||
|
|
||||||
## Security and Privacy
|
## Security and Privacy
|
||||||
|
|
||||||
@@ -139,7 +139,7 @@ Rules:
|
|||||||
|
|
||||||
- Do not store raw secrets in pipeline or session YAML.
|
- Do not store raw secrets in pipeline or session YAML.
|
||||||
- Use environment variable names or secret-file references for secret handling.
|
- Use environment variable names or secret-file references for secret handling.
|
||||||
- Do not write raw secret values to manifests, logs, generated configs, or archive metadata.
|
- Do not write raw secret values to manifests, logs, generated configs, or publish metadata.
|
||||||
- Treat transcripts, generated artifacts, prompts, reports, and logs as potentially sensitive.
|
- Treat transcripts, generated artifacts, prompts, reports, and logs as potentially sensitive.
|
||||||
- Avoid logging transcript or prompt content unless there is a deliberate diagnostic reason.
|
- Avoid logging transcript or prompt content unless there is a deliberate diagnostic reason.
|
||||||
|
|
||||||
@@ -177,7 +177,7 @@ Tests should cover:
|
|||||||
- stage success, failure, skip, and resume behavior;
|
- stage success, failure, skip, and resume behavior;
|
||||||
- adapter command construction;
|
- adapter command construction;
|
||||||
- fake storage behavior;
|
- fake storage behavior;
|
||||||
- archive commit ordering;
|
- publish commit ordering;
|
||||||
- example config validity where practical.
|
- example config validity where practical.
|
||||||
|
|
||||||
Live S3, WhisperX, LLM, or subprocess integration tests should be explicit integration tests, not required for ordinary unit test runs.
|
Live S3, WhisperX, LLM, or subprocess integration tests should be explicit integration tests, not required for ordinary unit test runs.
|
||||||
@@ -69,11 +69,13 @@ For design principles and invariants, see [docs/architecture.md](./architecture.
|
|||||||
2. Add or update command tests (`TestExecute` and command-specific tests).
|
2. Add or update command tests (`TestExecute` and command-specific tests).
|
||||||
3. Update [docs/cli.md](./cli.md) and, if operator workflow changes, [docs/operations.md](./operations.md).
|
3. Update [docs/cli.md](./cli.md) and, if operator workflow changes, [docs/operations.md](./operations.md).
|
||||||
|
|
||||||
|
Remote-storage commands must obtain object storage through the app-level command object-store helper. Do not call `storage.NewObjectStoreFromConfig` directly from command handlers; the helper loads configured filesystem secrets before constructing the storage adapter.
|
||||||
|
|
||||||
### Add or modify stages/adapters
|
### Add or modify stages/adapters
|
||||||
|
|
||||||
1. Implement stage behavior in `internal/stage` with clear input/output boundaries.
|
1. Implement stage behavior in `internal/stage` with clear input/output boundaries.
|
||||||
2. Keep external transport/subprocess details in `internal/adapters`.
|
2. Keep external transport/subprocess details in `internal/adapters`.
|
||||||
3. Preserve manifest and promotion semantics expected by runner and archive logic.
|
3. Preserve manifest and publish-output semantics expected by runner and publish logic.
|
||||||
4. Add/update stage and adapter tests.
|
4. Add/update stage and adapter tests.
|
||||||
5. Update internal component contracts in `docs/internal/`.
|
5. Update internal component contracts in `docs/internal/`.
|
||||||
|
|
||||||
105
docs/roadmap/documentation-stage1-audit.md
Normal file
105
docs/roadmap/documentation-stage1-audit.md
Normal file
@@ -0,0 +1,105 @@
|
|||||||
|
# Documentation Pass: Stage 1 Audit
|
||||||
|
|
||||||
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
|
## Scope Reviewed
|
||||||
|
|
||||||
|
- All non-policy documentation files under `docs/`
|
||||||
|
- `README.md`
|
||||||
|
- Documentation references to maintained `examples/` files
|
||||||
|
- Documentation-related expectations in tests under `internal/**`
|
||||||
|
|
||||||
|
## File Inventory and Canonical Scope
|
||||||
|
|
||||||
|
| File | Intended audience | Canonical scope (per policy) | Primary source-of-truth anchors |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `README.md` | Users, operators | Project orientation and links | `cmd/narratio`, `internal/app/commands.go`, docs index files |
|
||||||
|
| `docs/cli.md` | Users, operators | CLI syntax, flags, command workflows | `internal/app/*.go`, `internal/app/*_test.go` |
|
||||||
|
| `docs/config.md` | Operators, advanced users | Config discovery, schema, defaults, examples | `internal/config/*.go`, `internal/config/*_test.go`, `examples/*` |
|
||||||
|
| `docs/operations.md` | Operators | Run/resume/publish/restore/cleanup workflows | `internal/app/runner.go`, `internal/app/restore*.go`, `internal/stage/archive.go`, `internal/artifacts/*.go` |
|
||||||
|
| `docs/troubleshooting.md` | Operators | Failure diagnosis and safe fixes | `internal/app`, `internal/stage`, related tests |
|
||||||
|
| `docs/internal/README.md` | Developers, LLM coding agents | Internal docs index and scope boundaries | `docs/internal/*.md`, policy docs |
|
||||||
|
| `docs/internal/adapters.md` | Developers, LLM coding agents | Adapter boundaries and ownership | `internal/adapters/*`, `internal/stage/*` |
|
||||||
|
| `docs/internal/artifacts.md` | Developers, LLM coding agents | Artifact catalog and source resolution contracts | `internal/artifacts/*`, `internal/stage/analyze.go`, `internal/stage/prepare_previous.go` |
|
||||||
|
| `docs/internal/command-restore.md` | Developers, LLM coding agents | Restore command architecture and contracts | `internal/app/restore*.go`, `internal/app/restore*_test.go` |
|
||||||
|
| `docs/internal/manifest.md` | Developers, LLM coding agents | Session/run manifest contracts and transitions | `internal/manifest/*`, `internal/app/runner.go`, `internal/stage/*` |
|
||||||
|
| `docs/internal/stage-prepare.md` | Developers, LLM coding agents | Prepare stage IO and invariants | `internal/stage/prepare.go`, `internal/stage/prepare*_test.go` |
|
||||||
|
| `docs/internal/stage-transcribe.md` | Developers, LLM coding agents | Transcribe stage IO and invariants | `internal/stage/transcribe.go`, `internal/stage/transcribe_test.go` |
|
||||||
|
| `docs/internal/stage-merge.md` | Developers, LLM coding agents | Merge stage IO and invariants | `internal/stage/merge.go`, `internal/stage/merge_test.go` |
|
||||||
|
| `docs/internal/stage-polish.md` | Developers, LLM coding agents | Polish stage IO and invariants | `internal/stage/polish.go`, `internal/stage/polish_test.go` |
|
||||||
|
| `docs/internal/stage-normalize.md` | Developers, LLM coding agents | Normalize stage IO and invariants | `internal/stage/normalize.go`, `internal/stage/normalize_test.go` |
|
||||||
|
| `docs/internal/stage-trim.md` | Developers, LLM coding agents | Trim stage IO and invariants | `internal/stage/trim.go`, `internal/stage/trim_test.go` |
|
||||||
|
| `docs/internal/stage-analyze.md` | Developers, LLM coding agents | Analyze stage artifact execution and selection | `internal/stage/analyze.go`, `internal/stage/analyze_test.go` |
|
||||||
|
| `docs/internal/stage-publish.md` | Developers, LLM coding agents | Publish-stage commit/upload invariants | `internal/stage/archive.go`, `internal/stage/archive_test.go` |
|
||||||
|
| `docs/internal/storage.md` | Developers, LLM coding agents | Storage adapter contracts and semantics | `internal/adapters/storage/*`, `internal/app/object_store.go` |
|
||||||
|
| `docs/internal/workspace.md` | Developers, LLM coding agents | Local workspace/session/run path model | `internal/artifacts/*`, `internal/app/runner.go`, `internal/stage/run_local.go` |
|
||||||
|
| `docs/integrations/README.md` | Developers, LLM coding agents | Integration docs index | `docs/integrations/*.md` |
|
||||||
|
| `docs/integrations/audita.md` | Developers, integration maintainers | Audita adapter contract | `internal/adapters/audita/*`, `internal/stage/polish.go` |
|
||||||
|
| `docs/integrations/seriatim.md` | Developers, integration maintainers | Seriatim adapter contract | `internal/adapters/seriatim/*`, `internal/stage/merge.go`, `internal/stage/normalize.go`, `internal/stage/trim.go` |
|
||||||
|
| `docs/integrations/scriptorium.md` | Developers, integration maintainers | Scriptorium adapter contract | `internal/adapters/scriptorium/*`, `internal/stage/analyze.go`, `internal/stage/trim.go` |
|
||||||
|
| `docs/roadmap/documentation.md` | Developers, maintainers | Planning and implementation sequencing for documentation pass | N/A (planning artifact) |
|
||||||
|
| `docs/roadmap/documentation-stage1-audit.md` | Developers, maintainers | Stage-1 inventory and source-of-truth audit record | N/A (planning artifact) |
|
||||||
|
|
||||||
|
## Source-of-Truth Mapping Summary
|
||||||
|
|
||||||
|
- CLI behaviors and command names are grounded in `internal/app/commands.go` and command handlers in `internal/app/*.go`.
|
||||||
|
- Stage order and canonical stage names are grounded in `internal/stage/placeholders.go` (`prepare` -> `transcribe` -> `merge` -> `polish` -> `normalize` -> `trim` -> `analyze` -> `publish` -> `notify`).
|
||||||
|
- Publish behavior and current-pointer commit semantics are grounded in `internal/stage/archive.go`.
|
||||||
|
- Config schema/defaults/validation are grounded in `internal/config/*`.
|
||||||
|
- Local/remote paths, publish keys, and workspace layout are grounded in `internal/artifacts/*`.
|
||||||
|
- Restore behavior and report contracts are grounded in `internal/app/restore*.go`.
|
||||||
|
- Maintained examples and schema compatibility are grounded in `examples/*` plus `internal/config/load_validate_test.go` (`TestExamplesLoadAndValidate`).
|
||||||
|
|
||||||
|
## Findings
|
||||||
|
|
||||||
|
### Broken or stale references
|
||||||
|
|
||||||
|
1. `README.md` linked to non-existent files:
|
||||||
|
- `docs/development.md`
|
||||||
|
- `docs/architecture.md`
|
||||||
|
2. `docs/internal/README.md` and `docs/integrations/README.md` linked to non-existent path:
|
||||||
|
- `docs/documentation/policy.md`
|
||||||
|
|
||||||
|
Stage-1 fix applied:
|
||||||
|
- Updated those links to existing policy docs under `docs/policy/`.
|
||||||
|
|
||||||
|
### Stale terminology sweep
|
||||||
|
|
||||||
|
Sweep terms used: `archive`, `promote`, `promoted`, `promote_artifacts`, `run-stage archive`.
|
||||||
|
|
||||||
|
Findings:
|
||||||
|
- User-facing docs in scope did not show obvious stale command examples requiring immediate correction.
|
||||||
|
- Internal code and tests still contain historical `archive` identifiers while user-facing command/stage naming is `publish` (for example, `internal/stage/archive.go` type names). This is acceptable for now but should be normalized deliberately, not incidentally.
|
||||||
|
|
||||||
|
Stage-1 fix applied:
|
||||||
|
- Updated clearly stale publish-related wording in test expectation messages/comments:
|
||||||
|
- `internal/app/commands_test.go`
|
||||||
|
- `internal/app/operator_helpers_test.go`
|
||||||
|
|
||||||
|
### Example path validation
|
||||||
|
|
||||||
|
- All `examples/...` paths referenced from non-policy docs resolve to existing files.
|
||||||
|
- `internal/config/load_validate_test.go` includes `TestExamplesLoadAndValidate` and points to current example files.
|
||||||
|
|
||||||
|
### Roadmap leakage into current-behavior docs
|
||||||
|
|
||||||
|
- No obvious roadmap-only behavior leakage found in non-roadmap docs during this sweep.
|
||||||
|
|
||||||
|
### Duplicate content and scope drift
|
||||||
|
|
||||||
|
- No severe duplication requiring immediate rewrite in this stage.
|
||||||
|
- Existing docs still need full content rewrite for 1.0 readiness in later stages (user/operator first, then internal/integrations), as planned.
|
||||||
|
|
||||||
|
### Canonical-home inconsistency to resolve in rewrite stages
|
||||||
|
|
||||||
|
- Policy canonical-home language names `docs/architecture.md` and `docs/development.md`, while current repository stores those policy documents under `docs/policy/`.
|
||||||
|
- Stage 1 preserves repository behavior by fixing broken links to existing files. Later rewrite stages should converge canonical-home paths and references consistently across docs.
|
||||||
|
|
||||||
|
## Stage-1 Completion Check
|
||||||
|
|
||||||
|
Completed for this stage:
|
||||||
|
- Full non-policy file inventory with audience and scope mapping.
|
||||||
|
- Source-of-truth crosswalk to code/tests.
|
||||||
|
- Stale-term, link, and example-path sweeps.
|
||||||
|
- Documentation-related stale test wording corrections.
|
||||||
|
- Minimal fixes only; broad rewrites intentionally deferred.
|
||||||
237
docs/roadmap/documentation.md
Normal file
237
docs/roadmap/documentation.md
Normal file
@@ -0,0 +1,237 @@
|
|||||||
|
# Roadmap: 1.0 Documentation Pass
|
||||||
|
|
||||||
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Prepare Narratio documentation for a 1.0 release by reviewing and rewriting
|
||||||
|
every non-policy document under `docs/` against the implemented codebase.
|
||||||
|
|
||||||
|
The finished documentation set should be accurate, concise, complete for its
|
||||||
|
audience, and compliant with:
|
||||||
|
|
||||||
|
- `docs/policy/documentation.md`
|
||||||
|
- `docs/policy/architecture.md`
|
||||||
|
- `docs/policy/development.md`
|
||||||
|
|
||||||
|
Do not modify files under `docs/policy/` during this pass.
|
||||||
|
|
||||||
|
Current behavior belongs in canonical docs. Future, planned, aspirational, or
|
||||||
|
unimplemented behavior belongs only under `docs/roadmap/`.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- `docs/*.md`
|
||||||
|
- `docs/internal/*.md`
|
||||||
|
- `docs/integrations/*.md`
|
||||||
|
- `docs/roadmap/*.md`
|
||||||
|
- documentation references to files under `examples/`
|
||||||
|
- test expectation fixes when the documentation review exposes stale or
|
||||||
|
incorrect doc/example/path expectations
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- product/runtime code changes
|
||||||
|
- feature implementation
|
||||||
|
- edits under `docs/policy/`
|
||||||
|
- adding roadmap behavior to current-behavior docs before that behavior is
|
||||||
|
implemented
|
||||||
|
|
||||||
|
## Implementation Stages
|
||||||
|
|
||||||
|
### Stage 1: Inventory and Source-of-Truth Audit
|
||||||
|
|
||||||
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
|
Create a file-by-file inventory of all non-policy docs before rewriting.
|
||||||
|
|
||||||
|
Implementation requirements:
|
||||||
|
|
||||||
|
- List every non-policy documentation file and assign its intended audience.
|
||||||
|
- Identify each document's canonical scope using `docs/policy/documentation.md`.
|
||||||
|
- Compare docs against the current code and tests, especially:
|
||||||
|
- `internal/app`
|
||||||
|
- `internal/config`
|
||||||
|
- `internal/stage`
|
||||||
|
- `internal/artifacts`
|
||||||
|
- `examples`
|
||||||
|
- relevant tests under `internal/**`
|
||||||
|
- Record stale terminology, broken links, stale example paths, duplicate
|
||||||
|
content, and roadmap-only behavior that leaked into current-behavior docs.
|
||||||
|
- Record stale test expectations related to docs, examples, paths, or command
|
||||||
|
text.
|
||||||
|
- Do not rewrite content in this stage except obvious broken links or test
|
||||||
|
corrections needed to make documentation validation meaningful.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- The rewrite has a concrete file inventory and source-of-truth map.
|
||||||
|
- The team knows which docs are canonical and which should link elsewhere.
|
||||||
|
- Known stale terms and broken references are identified before broad edits.
|
||||||
|
|
||||||
|
### Stage 2: User and Operator Docs
|
||||||
|
|
||||||
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
|
Rewrite the user-facing and operator-facing docs first.
|
||||||
|
|
||||||
|
Implementation requirements:
|
||||||
|
|
||||||
|
- Rewrite these docs as fresh, concise current-behavior references:
|
||||||
|
- `README.md`, if present
|
||||||
|
- `docs/cli.md`
|
||||||
|
- `docs/config.md`
|
||||||
|
- `docs/operations.md`
|
||||||
|
- `docs/troubleshooting.md`
|
||||||
|
- Verify every command, flag, config field, discovery rule, path, and workflow
|
||||||
|
against implemented behavior.
|
||||||
|
- Cover implemented 1.0 behavior, including:
|
||||||
|
- campaign registry selection;
|
||||||
|
- concrete session loading and template-driven `session init`;
|
||||||
|
- session-oriented helper commands;
|
||||||
|
- clean, restore, analyze, and publish workflows;
|
||||||
|
- artifact selection behavior;
|
||||||
|
- locks and published output behavior;
|
||||||
|
- workspace, spool, and cache behavior;
|
||||||
|
- secrets loading and S3-backed operation.
|
||||||
|
- Keep examples short and link to maintained files under `examples/` instead
|
||||||
|
of duplicating large config blocks.
|
||||||
|
- Fix tests only when they assert stale doc paths, example paths, command
|
||||||
|
names, or current-behavior text.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- User/operator docs are task-oriented and match actual CLI/config behavior.
|
||||||
|
- Current-behavior docs do not depend on roadmaps for normal usage.
|
||||||
|
- No current-behavior doc describes unimplemented roadmap items.
|
||||||
|
|
||||||
|
### Stage 3: Internal Developer Docs
|
||||||
|
|
||||||
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
|
Rewrite implemented internal component docs after public docs stabilize.
|
||||||
|
|
||||||
|
Implementation requirements:
|
||||||
|
|
||||||
|
- Rewrite:
|
||||||
|
- `docs/internal/README.md`
|
||||||
|
- `docs/internal/adapters.md`
|
||||||
|
- `docs/internal/artifacts.md`
|
||||||
|
- `docs/internal/command-restore.md`
|
||||||
|
- `docs/internal/manifest.md`
|
||||||
|
- `docs/internal/stage-*.md`
|
||||||
|
- `docs/internal/storage.md`
|
||||||
|
- `docs/internal/workspace.md`
|
||||||
|
- Verify stage docs against current stage names, stage ordering, manifest
|
||||||
|
records, declared inputs/outputs, adapters, path helpers, storage behavior,
|
||||||
|
publish/current-state behavior, restore behavior, cache behavior, and
|
||||||
|
workspace cleanup.
|
||||||
|
- Keep implementation details in `docs/internal/`, not in user-facing docs.
|
||||||
|
- Avoid turning internal docs into duplicate config or CLI references; link to
|
||||||
|
canonical docs when needed.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- Internal docs are accurate enough for developers and LLM coding agents to
|
||||||
|
change the system safely.
|
||||||
|
- Stage and adapter boundaries match `docs/policy/architecture.md`.
|
||||||
|
- Manifest, path, storage, and publish invariants are explicit and current.
|
||||||
|
|
||||||
|
### Stage 4: Integrations and Examples
|
||||||
|
|
||||||
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
|
Review integration docs and maintained examples after core docs are rewritten.
|
||||||
|
|
||||||
|
Implementation requirements:
|
||||||
|
|
||||||
|
- Rewrite:
|
||||||
|
- `docs/integrations/README.md`
|
||||||
|
- `docs/integrations/audita.md`
|
||||||
|
- `docs/integrations/scriptorium.md`
|
||||||
|
- `docs/integrations/seriatim.md`
|
||||||
|
- Verify integration docs against current adapter contracts and expected
|
||||||
|
downstream tool behavior.
|
||||||
|
- Confirm every referenced example file exists.
|
||||||
|
- Confirm examples match current schema and command usage.
|
||||||
|
- Run or rely on example validation tests.
|
||||||
|
- Fix tests when they reference moved, renamed, or intentionally retired
|
||||||
|
examples.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- Integration docs describe only implemented adapter expectations.
|
||||||
|
- Maintained examples are valid, secret-free, and linked from canonical docs.
|
||||||
|
- Example validation tests reflect the documented example set.
|
||||||
|
|
||||||
|
### Stage 5: Roadmap Cleanup and Final Sweep
|
||||||
|
|
||||||
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
|
Clean up roadmap state and run final documentation validation.
|
||||||
|
|
||||||
|
Implementation requirements:
|
||||||
|
|
||||||
|
- Review `docs/roadmap/**` for implemented items that should be marked
|
||||||
|
implemented, retired, or left planned.
|
||||||
|
- Keep historical and planned behavior in roadmaps only.
|
||||||
|
- Run final link/path/term sweeps.
|
||||||
|
- Run validation commands:
|
||||||
|
- `go test ./internal/config -run TestExamplesLoadAndValidate -v`
|
||||||
|
- `go test ./internal/app -run TestExecute -v`
|
||||||
|
- `go test ./...`
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- All non-policy docs are current for the 1.0 release.
|
||||||
|
- Roadmaps do not serve as required user/operator documentation.
|
||||||
|
- Tests pass after allowed documentation-related test expectation fixes.
|
||||||
|
|
||||||
|
## Required Checks
|
||||||
|
|
||||||
|
Run searches for stale terminology and references during the pass.
|
||||||
|
|
||||||
|
Stale terminology:
|
||||||
|
|
||||||
|
- `archive`
|
||||||
|
- `promote`
|
||||||
|
- `promoted`
|
||||||
|
- `promote_artifacts`
|
||||||
|
- legacy campaign path/discovery language
|
||||||
|
- old transcript names and paths
|
||||||
|
- removed CLI commands or aliases
|
||||||
|
|
||||||
|
Broken or stale references:
|
||||||
|
|
||||||
|
- missing local doc links;
|
||||||
|
- stale `examples/` paths;
|
||||||
|
- stale internal doc filenames;
|
||||||
|
- references to `docs/policy/**` as editable targets;
|
||||||
|
- command examples that no longer match the CLI.
|
||||||
|
|
||||||
|
Policy checks:
|
||||||
|
|
||||||
|
- Current-behavior docs mention only implemented behavior.
|
||||||
|
- Planned behavior appears only under `docs/roadmap/`.
|
||||||
|
- Docs do not expose raw secrets or recommend storing secrets in config.
|
||||||
|
- Docs use canonical homes:
|
||||||
|
- `docs/config.md` for config schema;
|
||||||
|
- `docs/cli.md` for command syntax;
|
||||||
|
- `docs/operations.md` for operator workflows;
|
||||||
|
- `docs/troubleshooting.md` for failure diagnosis;
|
||||||
|
- `docs/internal/` for implementation contracts;
|
||||||
|
- `docs/integrations/` for downstream tool integration notes;
|
||||||
|
- `docs/roadmap/` for future work.
|
||||||
|
|
||||||
|
## Assumptions
|
||||||
|
|
||||||
|
- `docs/policy/**` is read-only for this documentation pass.
|
||||||
|
- This pass is for 1.0 release readiness, not feature implementation.
|
||||||
|
- Product and runtime code changes are out of scope.
|
||||||
|
- Test fixes are in scope when they correct stale documentation, example, path,
|
||||||
|
command, or current-behavior expectations uncovered during the review.
|
||||||
|
- Roadmap files may remain as planning and historical records.
|
||||||
|
- Current-behavior docs must be sufficient for normal use without requiring
|
||||||
|
readers to consult roadmaps.
|
||||||
@@ -1,715 +0,0 @@
|
|||||||
# Roadmap: `narratio restore` Subcommand
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
Implemented through Step 8. This document remains as roadmap and design history for the restore feature, and as the home for future restore-related ideas (for example `run --restore`).
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Add a new `narratio restore` subcommand that hydrates a local session workspace from the current committed remote archive state.
|
|
||||||
|
|
||||||
The primary operator workflow is:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio restore --session-id 2026-04-04
|
|
||||||
narratio run-stage --force analyze
|
|
||||||
```
|
|
||||||
|
|
||||||
This should allow a new machine with no local workspace state to restore the durable session manifest, transcripts, and generated artifacts from S3, then generate new Scriptorium artifacts without re-running transcription, merge, normalize, polish, or trim.
|
|
||||||
|
|
||||||
This is intentionally a separate command. Do not fold this behavior into the `prepare` stage. The existing `prepare` stage should remain focused on materializing configured local/S3 inputs for a pipeline run.
|
|
||||||
|
|
||||||
## Goals
|
|
||||||
|
|
||||||
- Add a first-class `narratio restore` command.
|
|
||||||
- Restore the current committed remote session state into the canonical local session workspace.
|
|
||||||
- Use the existing object storage adapter boundary.
|
|
||||||
- Preserve archive commit semantics: only restore from a remote state that has a valid current commit marker.
|
|
||||||
- Restore durable session-level outputs needed for downstream stages, especially `analyze`.
|
|
||||||
- Provide safe conflict behavior by default.
|
|
||||||
- Support `--dry-run`, `--force`, and `--include-audio`.
|
|
||||||
- Keep the implementation explicit, testable, and narrow.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- Do not make `restore` a pipeline stage.
|
|
||||||
- Do not change the `prepare` stage behavior as part of this work.
|
|
||||||
- Do not add implicit restore behavior to `narratio run` in this implementation.
|
|
||||||
- Do not restore historical run-local sandboxes by default.
|
|
||||||
- Do not implement a generic remote synchronization engine.
|
|
||||||
- Do not implement bidirectional sync.
|
|
||||||
- Do not delete local files merely because they are absent remotely.
|
|
||||||
- Do not merge remote and local manifests in the first implementation.
|
|
||||||
- Do not require live S3 for the ordinary unit test suite.
|
|
||||||
|
|
||||||
## Future work explicitly out of scope
|
|
||||||
|
|
||||||
A future change may add:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio run --restore
|
|
||||||
```
|
|
||||||
|
|
||||||
That future flag should run `narratio restore` before starting the normal pipeline. Mention this as future work in roadmap/docs if useful, but do not implement it now.
|
|
||||||
|
|
||||||
## Existing architecture to preserve
|
|
||||||
|
|
||||||
### `prepare` remains input materialization
|
|
||||||
|
|
||||||
The `prepare` stage currently materializes required session inputs into canonical local workspace paths and records input provenance. It owns local copying/materialization of config and audio inputs, including S3 audio download when `session.inputs.audio_s3.prefix` is configured. It does not own transcript generation/processing or archive publish behavior.
|
|
||||||
|
|
||||||
`restore` should not be implemented by expanding `prepare`. It should be an app-level command that reuses shared helpers where appropriate.
|
|
||||||
|
|
||||||
### Workspace model
|
|
||||||
|
|
||||||
The local durable session workspace is campaign-aware:
|
|
||||||
|
|
||||||
```text
|
|
||||||
{workspace.root}/work/{campaign}/{session_id}/
|
|
||||||
```
|
|
||||||
|
|
||||||
It contains durable session paths such as:
|
|
||||||
|
|
||||||
```text
|
|
||||||
manifest.json
|
|
||||||
inputs/
|
|
||||||
audio/
|
|
||||||
transcripts/
|
|
||||||
artifacts/
|
|
||||||
reports/
|
|
||||||
logs/
|
|
||||||
config/
|
|
||||||
current/
|
|
||||||
runs/
|
|
||||||
```
|
|
||||||
|
|
||||||
Run-local sandboxes live below:
|
|
||||||
|
|
||||||
```text
|
|
||||||
runs/{run_id}/
|
|
||||||
```
|
|
||||||
|
|
||||||
Restore should target durable session-level paths, not old run-local stage sandboxes.
|
|
||||||
|
|
||||||
### Storage boundary
|
|
||||||
|
|
||||||
The storage adapter owns object-store primitives only: `List`, `Download`, `Upload`, and `Exists`.
|
|
||||||
|
|
||||||
The storage adapter must not infer root prefixes, campaign names, session IDs, run IDs, or archive layout. Restore code must construct full bucket-relative keys before calling storage.
|
|
||||||
|
|
||||||
### Archive commit boundary
|
|
||||||
|
|
||||||
A remote run is current only after the archive stage has uploaded the run record, promoted outputs, `current/manifest.json`, and finally `current/run_id.txt`.
|
|
||||||
|
|
||||||
`current/run_id.txt` is the final remote commit marker and must be written last.
|
|
||||||
|
|
||||||
Restore must not treat incomplete, skipped, failed, or uncommitted archive attempts as current remote state.
|
|
||||||
|
|
||||||
## User-facing command
|
|
||||||
|
|
||||||
Add:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio restore [flags]
|
|
||||||
```
|
|
||||||
|
|
||||||
The command should use the same configuration/session discovery conventions as `run`, `plan`, `resume`, and `run-stage` where practical:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio restore --config /path/to/pipeline.yml --session ./session.yml --session-id 2026-04-04
|
|
||||||
```
|
|
||||||
|
|
||||||
Required effective inputs:
|
|
||||||
|
|
||||||
- resolved pipeline config;
|
|
||||||
- resolved session config;
|
|
||||||
- `session.campaign`;
|
|
||||||
- `session.session_id`;
|
|
||||||
- configured remote storage backend.
|
|
||||||
|
|
||||||
Supported flags:
|
|
||||||
|
|
||||||
```text
|
|
||||||
--config <path> Existing pipeline config path behavior.
|
|
||||||
--session <path> Existing session config path behavior.
|
|
||||||
--session-id <value> Existing session template behavior.
|
|
||||||
--dry-run Plan restore actions without writing local files.
|
|
||||||
--force Overwrite conflicting local files with remote files.
|
|
||||||
--include-audio Include archived session-level audio files.
|
|
||||||
```
|
|
||||||
|
|
||||||
Do not add `--restore` to `run` in this implementation.
|
|
||||||
|
|
||||||
## Default restore scope
|
|
||||||
|
|
||||||
By default, restore:
|
|
||||||
|
|
||||||
1. Validates and reads the current remote commit marker.
|
|
||||||
2. Downloads the current remote manifest into the local session manifest path.
|
|
||||||
3. Downloads durable transcript files.
|
|
||||||
4. Downloads durable generated artifact files.
|
|
||||||
|
|
||||||
Default included remote/local durable paths:
|
|
||||||
|
|
||||||
```text
|
|
||||||
manifest.json from remote current manifest
|
|
||||||
transcripts/**
|
|
||||||
artifacts/**
|
|
||||||
```
|
|
||||||
|
|
||||||
Default excluded paths:
|
|
||||||
|
|
||||||
```text
|
|
||||||
audio/** unless --include-audio is passed
|
|
||||||
runs/** always excluded for this implementation
|
|
||||||
logs/** excluded for this implementation
|
|
||||||
reports/** excluded for this implementation unless needed for current manifest validation
|
|
||||||
config/** excluded for this implementation
|
|
||||||
inputs/** excluded for this implementation
|
|
||||||
current/** remote control metadata only; do not mirror blindly
|
|
||||||
```
|
|
||||||
|
|
||||||
If the existing archive implementation stores promoted files in a different remote layout, use the existing archive/path helpers and current archive semantics rather than inventing a parallel layout.
|
|
||||||
|
|
||||||
## Remote state discovery
|
|
||||||
|
|
||||||
Implement restore around the current committed archive state.
|
|
||||||
|
|
||||||
Expected algorithm:
|
|
||||||
|
|
||||||
1. Resolve pipeline/session config.
|
|
||||||
2. Ensure storage is configured.
|
|
||||||
3. Ensure local workspace layout exists.
|
|
||||||
4. Acquire the session lock.
|
|
||||||
5. Build the remote session archive prefix using the same helpers/policy used by archive code.
|
|
||||||
6. Check for the remote `current/run_id.txt` commit marker.
|
|
||||||
7. Read the committed run ID.
|
|
||||||
8. Download `current/manifest.json` to a temporary file.
|
|
||||||
9. Validate that the manifest is parseable and belongs to the requested campaign/session.
|
|
||||||
10. Build a restore plan from the committed remote state.
|
|
||||||
11. Execute the restore plan unless `--dry-run` is set.
|
|
||||||
12. Emit a concise summary.
|
|
||||||
|
|
||||||
Important: `current/run_id.txt` is the commit marker. Do not restore from a remote session prefix merely because files exist under `transcripts/` or `artifacts/`.
|
|
||||||
|
|
||||||
## Restore planning
|
|
||||||
|
|
||||||
Create a planning layer before writing files.
|
|
||||||
|
|
||||||
A restore plan entry should include at least:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type RestoreAction struct {
|
|
||||||
Kind RestoreActionKind
|
|
||||||
RemoteKey string
|
|
||||||
LocalPath string
|
|
||||||
Size int64
|
|
||||||
ETag string
|
|
||||||
ExistsLocal bool
|
|
||||||
SameLocal bool
|
|
||||||
Conflict bool
|
|
||||||
Reason string
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Suggested action kinds:
|
|
||||||
|
|
||||||
```text
|
|
||||||
download
|
|
||||||
skip_same
|
|
||||||
skip_missing_optional
|
|
||||||
conflict
|
|
||||||
```
|
|
||||||
|
|
||||||
The restore planner should be deterministic:
|
|
||||||
|
|
||||||
- sort remote objects by key;
|
|
||||||
- sort planned actions by local path or stable restore priority;
|
|
||||||
- write/report stable output for tests.
|
|
||||||
|
|
||||||
## Conflict and overwrite policy
|
|
||||||
|
|
||||||
Default behavior should be safe.
|
|
||||||
|
|
||||||
For each planned file:
|
|
||||||
|
|
||||||
```text
|
|
||||||
local absent:
|
|
||||||
download
|
|
||||||
|
|
||||||
local present and same as remote:
|
|
||||||
skip
|
|
||||||
|
|
||||||
local present and different:
|
|
||||||
conflict; fail restore unless --force is set
|
|
||||||
|
|
||||||
--force:
|
|
||||||
overwrite local conflicting files with remote versions
|
|
||||||
|
|
||||||
--dry-run:
|
|
||||||
do not write any files; report what would happen
|
|
||||||
```
|
|
||||||
|
|
||||||
The first implementation may use size and checksum/hash comparison where available. If remote ETag cannot be treated as a content hash, compare by downloading to a temporary file and hashing locally before deciding whether a local file is the same. Prefer correctness over assuming provider-specific ETag semantics.
|
|
||||||
|
|
||||||
Do not delete local files that are not present remotely.
|
|
||||||
|
|
||||||
## File writing and transactionality
|
|
||||||
|
|
||||||
Restore should avoid partial writes.
|
|
||||||
|
|
||||||
Implementation requirements:
|
|
||||||
|
|
||||||
- download each remote object to a temporary file under the session workspace or OS temp dir;
|
|
||||||
- validate downloaded content where possible before replacing local files;
|
|
||||||
- create parent directories as needed;
|
|
||||||
- atomically rename/copy into place only after successful download;
|
|
||||||
- do not overwrite local files unless `--force` is set;
|
|
||||||
- if a later file fails, preserve already-restored files but return a failure summary;
|
|
||||||
- never corrupt an existing local manifest on failed manifest download/parse.
|
|
||||||
|
|
||||||
Manifest restore is especially sensitive:
|
|
||||||
|
|
||||||
- download remote `current/manifest.json` to a temporary file;
|
|
||||||
- parse and validate it;
|
|
||||||
- if no local manifest exists, install it;
|
|
||||||
- if a local manifest exists and is equivalent, skip;
|
|
||||||
- if a local manifest exists and differs, fail unless `--force` is set;
|
|
||||||
- with `--force`, replace the local manifest with the remote manifest after validation;
|
|
||||||
- do not attempt a manifest merge in the initial implementation.
|
|
||||||
|
|
||||||
## Manifest semantics
|
|
||||||
|
|
||||||
`restore` is not a pipeline run and should not mark stages as running/succeeded/failed.
|
|
||||||
|
|
||||||
The restored remote manifest becomes the local session manifest. That is what allows a subsequent command such as:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio run-stage --force analyze
|
|
||||||
```
|
|
||||||
|
|
||||||
to see existing upstream stage state and canonical durable outputs.
|
|
||||||
|
|
||||||
Do not create a new run manifest for `restore`.
|
|
||||||
|
|
||||||
It is acceptable to write a restore diagnostic report outside the manifest, for example:
|
|
||||||
|
|
||||||
```text
|
|
||||||
reports/restore-latest.json
|
|
||||||
```
|
|
||||||
|
|
||||||
or a timestamped report, if that pattern fits the existing codebase. The report must not contain secrets.
|
|
||||||
|
|
||||||
## Local workspace locking
|
|
||||||
|
|
||||||
`restore` should acquire the same session lock used by ordinary pipeline operations before modifying session workspace state.
|
|
||||||
|
|
||||||
If the lock is held, fail fast with the same lock-conflict behavior used elsewhere.
|
|
||||||
|
|
||||||
`--dry-run` may still acquire the lock for consistency, but it is acceptable to avoid the lock if the codebase already has a clear read-only command pattern. Prefer safety and simplicity.
|
|
||||||
|
|
||||||
## Audio behavior
|
|
||||||
|
|
||||||
By default, do not restore audio.
|
|
||||||
|
|
||||||
If `--include-audio` is passed:
|
|
||||||
|
|
||||||
- restore archived durable session-level audio files only;
|
|
||||||
- do not use run-scoped spool paths;
|
|
||||||
- do not mutate or delete spool state;
|
|
||||||
- do not infer original `session.inputs.audio_s3.prefix` behavior;
|
|
||||||
- respect the same conflict/force/dry-run behavior used for transcripts/artifacts.
|
|
||||||
|
|
||||||
If the archive does not contain durable audio files, `--include-audio` should report that no archived audio was found rather than failing, unless the final implementation chooses to treat explicit audio restore as required. Prefer non-failure for absent archived audio unless tests or existing archive semantics suggest otherwise.
|
|
||||||
|
|
||||||
## Remote object selection
|
|
||||||
|
|
||||||
Prefer using manifest/artifact metadata when it reliably identifies durable outputs.
|
|
||||||
|
|
||||||
Also support listing committed durable archive prefixes so restore can retrieve all top-level session artifacts that may not yet be fully represented in manifest metadata.
|
|
||||||
|
|
||||||
The implementation should inspect existing archive code before choosing the final object-selection method. Do not duplicate archive path construction.
|
|
||||||
|
|
||||||
Recommended selection priority:
|
|
||||||
|
|
||||||
1. Remote current manifest path.
|
|
||||||
2. Durable promoted transcript/artifact outputs recorded in the manifest or archive metadata, if available.
|
|
||||||
3. Objects under committed durable `transcripts/` and `artifacts/` archive prefixes.
|
|
||||||
4. Objects under durable `audio/` only when `--include-audio` is passed.
|
|
||||||
|
|
||||||
Always exclude:
|
|
||||||
|
|
||||||
```text
|
|
||||||
runs/**
|
|
||||||
```
|
|
||||||
|
|
||||||
for the first implementation.
|
|
||||||
|
|
||||||
## Package and file organization
|
|
||||||
|
|
||||||
Expected areas to inspect and update:
|
|
||||||
|
|
||||||
```text
|
|
||||||
cmd/narratio/
|
|
||||||
internal/app/
|
|
||||||
internal/adapters/storage/
|
|
||||||
internal/artifacts/
|
|
||||||
internal/manifest/
|
|
||||||
docs/
|
|
||||||
examples/
|
|
||||||
```
|
|
||||||
|
|
||||||
Suggested implementation shape:
|
|
||||||
|
|
||||||
```text
|
|
||||||
internal/app/restore.go
|
|
||||||
internal/app/restore_test.go
|
|
||||||
|
|
||||||
internal/archive/restore/
|
|
||||||
planner.go
|
|
||||||
executor.go
|
|
||||||
report.go
|
|
||||||
keys.go
|
|
||||||
*_test.go
|
|
||||||
```
|
|
||||||
|
|
||||||
The exact package name may vary. Use whatever best fits the existing repository, but keep these boundaries clear:
|
|
||||||
|
|
||||||
- `internal/app` owns CLI command handling, config/session loading, lock acquisition, and wiring.
|
|
||||||
- Restore planning/execution owns remote key discovery, conflict detection, downloads, and reporting.
|
|
||||||
- `internal/adapters/storage` remains a transport boundary only.
|
|
||||||
- Workspace/path helpers remain centralized; do not scatter string concatenation.
|
|
||||||
|
|
||||||
If the repository already has an `internal/archive` or archive-stage helper package, prefer extending that rather than creating a conflicting package layout.
|
|
||||||
|
|
||||||
## CLI output
|
|
||||||
|
|
||||||
`narratio restore` should print a concise operator summary.
|
|
||||||
|
|
||||||
Example successful output:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Restored session archive for sample-campaign/2026-04-04
|
|
||||||
Remote run: 20260504T031500Z-a1b2c3
|
|
||||||
Downloaded: 4
|
|
||||||
Skipped unchanged: 2
|
|
||||||
Conflicts: 0
|
|
||||||
```
|
|
||||||
|
|
||||||
Example dry run:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Restore plan for sample-campaign/2026-04-04
|
|
||||||
Remote run: 20260504T031500Z-a1b2c3
|
|
||||||
Would download: transcripts/processed.json
|
|
||||||
Would download: transcripts/trimmed.json
|
|
||||||
Would skip unchanged: artifacts/session_recap.md
|
|
||||||
```
|
|
||||||
|
|
||||||
Example conflict:
|
|
||||||
|
|
||||||
```text
|
|
||||||
restore conflict: local artifacts/session_recap.md differs from remote archive; rerun with --force to overwrite
|
|
||||||
```
|
|
||||||
|
|
||||||
Do not print transcript or artifact content.
|
|
||||||
|
|
||||||
## Error behavior
|
|
||||||
|
|
||||||
Fail clearly when:
|
|
||||||
|
|
||||||
- storage backend is not configured;
|
|
||||||
- S3 bucket/config is missing or invalid;
|
|
||||||
- remote current commit marker is missing;
|
|
||||||
- remote current manifest is missing;
|
|
||||||
- remote manifest is invalid;
|
|
||||||
- remote manifest does not match requested campaign/session;
|
|
||||||
- local file differs from remote and `--force` is not set;
|
|
||||||
- a required remote object download fails;
|
|
||||||
- a local path would escape the session workspace;
|
|
||||||
- a remote key maps to an unsafe local path.
|
|
||||||
|
|
||||||
Skip or report non-fatal conditions when:
|
|
||||||
|
|
||||||
- optional audio restore finds no archived audio;
|
|
||||||
- an included prefix has no objects;
|
|
||||||
- a local file already matches the remote file.
|
|
||||||
|
|
||||||
## Path safety
|
|
||||||
|
|
||||||
Every restored file must map to a safe path under the session root.
|
|
||||||
|
|
||||||
Validation rules:
|
|
||||||
|
|
||||||
- local restore paths must be relative to the session root;
|
|
||||||
- reject absolute paths;
|
|
||||||
- reject `..` traversal;
|
|
||||||
- reject paths that escape through symlinks if the codebase has symlink-safe path checks;
|
|
||||||
- do not restore remote keys directly without mapping/classification;
|
|
||||||
- do not mirror arbitrary remote keys.
|
|
||||||
|
|
||||||
## Testing plan
|
|
||||||
|
|
||||||
Add focused unit tests. Do not require live S3.
|
|
||||||
|
|
||||||
### CLI tests
|
|
||||||
|
|
||||||
Add or update `internal/app` command tests for:
|
|
||||||
|
|
||||||
- `narratio restore --help`;
|
|
||||||
- restore accepts `--config`, `--session`, and `--session-id`;
|
|
||||||
- restore accepts `--dry-run`;
|
|
||||||
- restore accepts `--force`;
|
|
||||||
- restore accepts `--include-audio`;
|
|
||||||
- restore fails when storage is not configured;
|
|
||||||
- restore does not run pipeline stages.
|
|
||||||
|
|
||||||
### Restore planner tests
|
|
||||||
|
|
||||||
Test:
|
|
||||||
|
|
||||||
- missing `current/run_id.txt` fails;
|
|
||||||
- missing `current/manifest.json` fails;
|
|
||||||
- invalid manifest fails;
|
|
||||||
- wrong campaign/session manifest fails;
|
|
||||||
- default scope includes manifest/transcripts/artifacts;
|
|
||||||
- default scope excludes audio/logs/reports/config/runs;
|
|
||||||
- `--include-audio` includes durable audio;
|
|
||||||
- run-local keys are excluded;
|
|
||||||
- keys are sorted deterministically;
|
|
||||||
- unsafe remote-to-local paths are rejected.
|
|
||||||
|
|
||||||
### Conflict policy tests
|
|
||||||
|
|
||||||
Test:
|
|
||||||
|
|
||||||
- absent local file downloads;
|
|
||||||
- matching local file skips;
|
|
||||||
- differing local file conflicts by default;
|
|
||||||
- `--force` overwrites conflicts;
|
|
||||||
- `--dry-run` writes nothing;
|
|
||||||
- partial failure does not corrupt an existing local manifest.
|
|
||||||
|
|
||||||
### Storage/fake tests
|
|
||||||
|
|
||||||
Use fake storage to simulate:
|
|
||||||
|
|
||||||
- object listing;
|
|
||||||
- object download;
|
|
||||||
- missing objects;
|
|
||||||
- download failures;
|
|
||||||
- metadata/ETag behavior.
|
|
||||||
|
|
||||||
### Workspace/lock tests
|
|
||||||
|
|
||||||
Test:
|
|
||||||
|
|
||||||
- session layout is created before restore;
|
|
||||||
- session lock conflict fails;
|
|
||||||
- restored files land under the expected campaign/session workspace;
|
|
||||||
- no files are written outside the session root.
|
|
||||||
|
|
||||||
### Follow-up command workflow test
|
|
||||||
|
|
||||||
Add at least one test that simulates:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio restore --session-id 2026-04-04
|
|
||||||
narratio run-stage --force analyze
|
|
||||||
```
|
|
||||||
|
|
||||||
The test does not need to run real Scriptorium. Use existing fake/stub behavior to verify that restored transcripts and manifest state are sufficient for analyze-stage input resolution.
|
|
||||||
|
|
||||||
## Documentation updates when implemented
|
|
||||||
|
|
||||||
When the feature is implemented, update current-behavior docs:
|
|
||||||
|
|
||||||
```text
|
|
||||||
docs/cli.md
|
|
||||||
docs/operations.md
|
|
||||||
docs/internal/storage.md or docs/internal/archive/restore.md
|
|
||||||
```
|
|
||||||
|
|
||||||
If the documentation set does not yet have an internal restore document, add one consistent with the existing internal-doc style:
|
|
||||||
|
|
||||||
```text
|
|
||||||
docs/internal/command-restore.md
|
|
||||||
```
|
|
||||||
|
|
||||||
or:
|
|
||||||
|
|
||||||
```text
|
|
||||||
docs/internal/archive-restore.md
|
|
||||||
```
|
|
||||||
|
|
||||||
Do not document future `narratio run --restore` behavior outside `docs/roadmap/` until implemented.
|
|
||||||
|
|
||||||
## Implementation phases
|
|
||||||
|
|
||||||
### Phase 1: Audit existing archive and path helpers (completed)
|
|
||||||
|
|
||||||
Before coding behavior, inspect:
|
|
||||||
|
|
||||||
```text
|
|
||||||
internal/app/
|
|
||||||
internal/stage/archive*
|
|
||||||
internal/adapters/storage/
|
|
||||||
internal/artifacts/
|
|
||||||
internal/manifest/
|
|
||||||
docs/internal/stage-archive.md, if present
|
|
||||||
```
|
|
||||||
|
|
||||||
Determine:
|
|
||||||
|
|
||||||
- exact remote archive key layout;
|
|
||||||
- how root prefix/campaign/session are modeled;
|
|
||||||
- how current commit marker keys are built;
|
|
||||||
- how current manifest is uploaded;
|
|
||||||
- where promoted outputs are uploaded;
|
|
||||||
- whether helper functions already exist for remote archive keys;
|
|
||||||
- whether local workspace path helpers can safely map restore destinations.
|
|
||||||
|
|
||||||
Deliverable:
|
|
||||||
|
|
||||||
- small code comments or internal helper selection;
|
|
||||||
- no large behavior change yet unless required by tests.
|
|
||||||
|
|
||||||
### Phase 2: Add CLI surface and command wiring (completed)
|
|
||||||
|
|
||||||
Add `narratio restore` command parsing.
|
|
||||||
|
|
||||||
Wire flags:
|
|
||||||
|
|
||||||
```text
|
|
||||||
--config
|
|
||||||
--session
|
|
||||||
--session-id
|
|
||||||
--dry-run
|
|
||||||
--force
|
|
||||||
--include-audio
|
|
||||||
```
|
|
||||||
|
|
||||||
Use the existing config/session load path where practical.
|
|
||||||
|
|
||||||
Deliverable:
|
|
||||||
|
|
||||||
- command exists;
|
|
||||||
- help output is sensible;
|
|
||||||
- command validates basic inputs;
|
|
||||||
- command returns a clear “not yet implemented” or calls an empty planner if phased commits are desired;
|
|
||||||
- CLI tests pass.
|
|
||||||
|
|
||||||
### Phase 3: Implement remote current-state discovery (completed)
|
|
||||||
|
|
||||||
Add restore code that:
|
|
||||||
|
|
||||||
- creates an object store from resolved config;
|
|
||||||
- builds remote current marker key;
|
|
||||||
- reads `current/run_id.txt`;
|
|
||||||
- reads/downloads `current/manifest.json`;
|
|
||||||
- validates manifest identity;
|
|
||||||
- returns remote current-state metadata.
|
|
||||||
|
|
||||||
Deliverable:
|
|
||||||
|
|
||||||
- fake-storage tests for current-state discovery;
|
|
||||||
- no local file writes beyond temporary files.
|
|
||||||
|
|
||||||
### Phase 4: Implement restore planning (completed)
|
|
||||||
|
|
||||||
Build deterministic restore plans for default scope and `--include-audio`.
|
|
||||||
|
|
||||||
Deliverable:
|
|
||||||
|
|
||||||
- plan lists manifest, transcript, artifact files;
|
|
||||||
- plan excludes run-local data;
|
|
||||||
- plan detects local same/conflict/missing states;
|
|
||||||
- dry-run output works;
|
|
||||||
- no real file overwrite yet except temp comparisons as needed.
|
|
||||||
|
|
||||||
### Phase 5: Implement restore execution (completed)
|
|
||||||
|
|
||||||
Execute the plan safely:
|
|
||||||
|
|
||||||
- create directories;
|
|
||||||
- download to temporary files;
|
|
||||||
- validate content where practical;
|
|
||||||
- atomically install files;
|
|
||||||
- enforce default conflict failure;
|
|
||||||
- support `--force`;
|
|
||||||
- preserve existing manifest unless safe to replace.
|
|
||||||
|
|
||||||
Deliverable:
|
|
||||||
|
|
||||||
- restore works end-to-end against fake storage;
|
|
||||||
- failures are clear and do not corrupt existing local manifest.
|
|
||||||
|
|
||||||
### Phase 6: Add restore report and operator summary (completed)
|
|
||||||
|
|
||||||
Add concise stdout summary and optional JSON restore report if consistent with project diagnostics.
|
|
||||||
|
|
||||||
Deliverable:
|
|
||||||
|
|
||||||
- user-friendly output;
|
|
||||||
- durable diagnostic report if implemented;
|
|
||||||
- no content leakage.
|
|
||||||
|
|
||||||
### Phase 7: Workflow integration test (completed)
|
|
||||||
|
|
||||||
Add a test for restoring a previous session and then forcing `analyze`.
|
|
||||||
|
|
||||||
Deliverable:
|
|
||||||
|
|
||||||
- restored manifest/transcripts/artifacts are sufficient for analyze input resolution;
|
|
||||||
- no upstream stages rerun;
|
|
||||||
- no reliance on live subprocesses or S3.
|
|
||||||
|
|
||||||
### Phase 8: Documentation update (completed)
|
|
||||||
|
|
||||||
Once implemented, update current-behavior docs and internal command docs.
|
|
||||||
|
|
||||||
Also leave future `narratio run --restore` in roadmap only.
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
|
|
||||||
The feature is complete when:
|
|
||||||
|
|
||||||
- `narratio restore` exists and is documented.
|
|
||||||
- It uses the same config/session discovery semantics as other commands where practical.
|
|
||||||
- It requires configured remote storage.
|
|
||||||
- It restores only from a committed current archive state.
|
|
||||||
- It restores the current manifest, transcripts, and artifacts by default.
|
|
||||||
- It restores audio only with `--include-audio`.
|
|
||||||
- It excludes run-local sandboxes.
|
|
||||||
- It fails on local/remote conflicts by default.
|
|
||||||
- `--force` overwrites conflicts.
|
|
||||||
- `--dry-run` writes nothing.
|
|
||||||
- It uses fake storage in tests.
|
|
||||||
- It does not change `prepare` behavior.
|
|
||||||
- It does not implement `narratio run --restore`.
|
|
||||||
- It avoids AWS SDK leakage outside the storage adapter.
|
|
||||||
- It uses centralized path/key helpers rather than scattered string concatenation.
|
|
||||||
- `go test ./...` passes.
|
|
||||||
|
|
||||||
## Suggested test commands
|
|
||||||
|
|
||||||
Run focused tests first:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
go test ./internal/app -run TestExecute -v
|
|
||||||
go test ./internal/adapters/storage -v
|
|
||||||
go test ./internal/artifacts -v
|
|
||||||
go test ./internal/manifest -v
|
|
||||||
```
|
|
||||||
|
|
||||||
Then run the full suite:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
go test ./...
|
|
||||||
```
|
|
||||||
|
|
||||||
## Suggested commit message
|
|
||||||
|
|
||||||
```text
|
|
||||||
Add restore subcommand roadmap
|
|
||||||
```
|
|
||||||
@@ -1,380 +1,273 @@
|
|||||||
# Troubleshooting
|
# Troubleshooting
|
||||||
|
|
||||||
## Purpose
|
Operational diagnosis guide for common Narratio failures.
|
||||||
Canonical operator troubleshooting guide for recurring implemented Narratio failures.
|
|
||||||
|
|
||||||
## Config file discovery failure
|
## Config file not found
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- `run`, `plan`, `resume`, `run-stage`, or `restore` fails with config/session not found.
|
|
||||||
|
|
||||||
Likely Cause:
|
- command fails to resolve `pipeline.yml`, `campaign.yml`, or `session.yml`.
|
||||||
- `pipeline.yml` or `session.yml` is missing from discovery paths.
|
|
||||||
- wrong working directory when relying on `./session.yml`.
|
Likely causes:
|
||||||
|
|
||||||
|
- missing files in default search paths;
|
||||||
|
- wrong campaign selection;
|
||||||
|
- omitted explicit flags.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pwd
|
narratio session plan 2026-04-04
|
||||||
ls -l ./session.yml
|
|
||||||
ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe Fix:
|
Safe fix:
|
||||||
- pass explicit `--config` and `--session`.
|
|
||||||
- or place files in documented discovery paths.
|
|
||||||
|
|
||||||
Links:
|
- pass explicit `--config`, `--campaign` or `--campaign-file`, and `--session`.
|
||||||
- [docs/config.md](./config.md)
|
|
||||||
- [docs/cli.md](./cli.md)
|
|
||||||
|
|
||||||
## Session template rendering failure
|
## Session template placeholders rejected
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- load fails with unresolved placeholder or `session_id` mismatch.
|
|
||||||
|
|
||||||
Likely Cause:
|
- load error says session file must be concrete or contains `{{ ... }}` placeholders.
|
||||||
- templated `session.yml` used without `--session-id`.
|
|
||||||
- rendered `session_id` differs from passed `--session-id`.
|
Likely cause:
|
||||||
|
|
||||||
|
- using template content as runtime session config.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio plan --session ./session.yml --session-id 2026-04-04
|
narratio session validate 2026-04-04 --session /path/session.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe Fix:
|
Safe fix:
|
||||||
- pass `--session-id` when template placeholders are present.
|
|
||||||
- ensure rendered `session_id` matches intended run session id.
|
|
||||||
|
|
||||||
Links:
|
- generate concrete session YAML with `narratio session init`.
|
||||||
- [docs/config.md](./config.md)
|
|
||||||
|
|
||||||
## Strict YAML decode or validation failure
|
## Strict decode or schema validation failure
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- config load fails with unknown field or validation error.
|
|
||||||
|
|
||||||
Likely Cause:
|
- unknown field / invalid value error during config load.
|
||||||
- typo/stale field name.
|
|
||||||
- missing required fields or invalid constraints.
|
Likely cause:
|
||||||
|
|
||||||
|
- stale field name, typo, invalid enum, or invalid duration/path format.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04
|
narratio session plan 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe Fix:
|
Safe fix:
|
||||||
- align fields/values to canonical config reference and examples.
|
|
||||||
|
|
||||||
Links:
|
- align config with [docs/config.md](./config.md) and maintained files under `examples/`.
|
||||||
- [docs/config.md](./config.md)
|
|
||||||
- [examples/](../examples/)
|
|
||||||
|
|
||||||
## `--artifacts` selection failure
|
## Audio mode conflict
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- `run`/`resume`/`run-stage` fails with invalid or unknown artifact selection.
|
|
||||||
|
|
||||||
Likely Cause:
|
- validation fails on session audio configuration.
|
||||||
- `--artifacts` contains blank names or unknown artifact keys.
|
|
||||||
- `pipeline.scriptorium.artifacts` missing while using `--artifacts`.
|
Likely cause:
|
||||||
|
|
||||||
|
- configured both local and S3 session audio inputs.
|
||||||
|
|
||||||
|
Safe fix:
|
||||||
|
|
||||||
|
- use local mode (`audio_dir` or `audio_files`) or S3 mode (`audio_s3.prefix`), not both.
|
||||||
|
|
||||||
|
## `--artifacts` selection error
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
- unknown artifact key or invalid `--artifacts` usage.
|
||||||
|
|
||||||
|
Likely causes:
|
||||||
|
|
||||||
|
- key not defined in `pipeline.scriptorium.artifacts`;
|
||||||
|
- empty list entry (for example trailing comma);
|
||||||
|
- `run-stage` used with non-`analyze`/`publish` target.
|
||||||
|
|
||||||
|
Safe fix:
|
||||||
|
|
||||||
|
- provide only configured keys and use `--artifacts` with supported commands/stages.
|
||||||
|
|
||||||
|
## Previous-session artifact input missing
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
- prepare/analyze fails due to missing required previous-session artifact cache input.
|
||||||
|
|
||||||
|
Likely causes:
|
||||||
|
|
||||||
|
- missing `session.previous_session_id`;
|
||||||
|
- previous artifact not restored/published for source session.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout
|
narratio session validate 2026-04-04
|
||||||
|
narratio session status 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe Fix:
|
Safe fix:
|
||||||
- use configured artifact keys only.
|
|
||||||
- ensure `pipeline.scriptorium.artifacts` is defined.
|
|
||||||
|
|
||||||
Links:
|
|
||||||
- [docs/cli.md](./cli.md)
|
|
||||||
- [docs/config.md](./config.md)
|
|
||||||
|
|
||||||
## `run-stage --artifacts` on non-analyze stage
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
- `run-stage` fails with `--artifacts is only supported for stage "analyze"`.
|
|
||||||
|
|
||||||
Likely Cause:
|
|
||||||
- `--artifacts` was used with a non-`analyze` stage.
|
|
||||||
|
|
||||||
Diagnostics:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts session_recap polish
|
narratio session restore 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe Fix:
|
or rerun prepare after correcting session config:
|
||||||
- use `--artifacts` only with `run-stage ... analyze`.
|
|
||||||
|
|
||||||
Links:
|
|
||||||
- [docs/cli.md](./cli.md)
|
|
||||||
|
|
||||||
## Configured artifact dependency/input validation failure
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
- config validation fails for `depends_on`, `narratio.artifact.<name>` source, or artifact output path.
|
|
||||||
|
|
||||||
Likely Cause:
|
|
||||||
- `narratio.artifact.<name>` source missing matching `depends_on` key.
|
|
||||||
- dependency references unknown artifact key.
|
|
||||||
- dependency self-reference or enabled dependency cycle.
|
|
||||||
- artifact output path missing/invalid/outside `artifacts/` root.
|
|
||||||
|
|
||||||
Diagnostics:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04
|
narratio run-stage prepare 2026-04-04 --force
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe Fix:
|
|
||||||
- ensure artifact-to-artifact inputs have explicit `depends_on` entries using artifact keys.
|
|
||||||
- ensure referenced artifacts exist and define valid `output_path` values.
|
|
||||||
- keep output paths relative and under `artifacts/`.
|
|
||||||
|
|
||||||
Links:
|
|
||||||
- [docs/config.md](./config.md)
|
|
||||||
- [docs/internal/stage-analyze.md](./internal/stage-analyze.md)
|
|
||||||
|
|
||||||
## Required configured artifact input unavailable at analyze time
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
- analyze fails because configured input source is unavailable.
|
|
||||||
|
|
||||||
Likely Cause:
|
|
||||||
- required upstream configured artifact was not selected/executed this run.
|
|
||||||
- non-executable dependency output file is missing or invalid on disk.
|
|
||||||
|
|
||||||
Diagnostics:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio status --manifest /path/to/manifest.json
|
|
||||||
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout analyze
|
|
||||||
```
|
|
||||||
|
|
||||||
Safe Fix:
|
|
||||||
- run analyze with needed artifacts selected.
|
|
||||||
- or ensure dependency output file exists at configured path and is valid.
|
|
||||||
|
|
||||||
Links:
|
|
||||||
- [docs/operations.md](./operations.md)
|
|
||||||
- [docs/config.md](./config.md)
|
|
||||||
|
|
||||||
## Manifest/status path failure
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
- `status` fails because manifest path is missing, unreadable, or invalid.
|
|
||||||
|
|
||||||
Likely Cause:
|
|
||||||
- wrong manifest path.
|
|
||||||
- manifest removed after cleanup.
|
|
||||||
- `--manifest` omitted.
|
|
||||||
|
|
||||||
Diagnostics:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio status --manifest /path/to/manifest.json
|
|
||||||
ls -l /path/to/manifest.json
|
|
||||||
```
|
|
||||||
|
|
||||||
Safe Fix:
|
|
||||||
- use manifest path printed by `run`, `resume`, or `run-stage`.
|
|
||||||
|
|
||||||
Links:
|
|
||||||
- [docs/cli.md](./cli.md)
|
|
||||||
- [docs/operations.md](./operations.md)
|
|
||||||
|
|
||||||
## Session lock conflict (`.lock`)
|
## Session lock conflict (`.lock`)
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- `run`, `resume`, `run-stage`, or `restore` fails with lock conflict for session workdir.
|
|
||||||
|
|
||||||
Likely Cause:
|
- command fails acquiring session lock.
|
||||||
- another Narratio process is running same session.
|
|
||||||
- stale lock from interrupted prior run.
|
Likely causes:
|
||||||
|
|
||||||
|
- another process is running for the same session;
|
||||||
|
- stale lock left by interrupted process.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ls -l {workspace.root}/work/{campaign}/{session_id}/.lock
|
ls -l {workspace.root}/work/{campaign}/{session_id}/.lock
|
||||||
cat {workspace.root}/work/{campaign}/{session_id}/.lock
|
|
||||||
ps aux | grep narratio
|
ps aux | grep narratio
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe Fix:
|
Safe fix:
|
||||||
- wait for active process to finish.
|
|
||||||
- if no process is active, remove only stale session `.lock` file.
|
|
||||||
|
|
||||||
Links:
|
- wait for active process completion;
|
||||||
- [docs/operations.md](./operations.md)
|
- remove stale lock only after confirming no live process owns it.
|
||||||
- [docs/internal/workspace.md](./internal/workspace.md)
|
|
||||||
|
|
||||||
## Restore remote current pointer or manifest missing
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
- `restore` fails with remote current pointer or current manifest errors.
|
|
||||||
|
|
||||||
Likely Cause:
|
|
||||||
- `current/run_id.txt` was never published.
|
|
||||||
- `current/manifest.json` is missing for the session prefix.
|
|
||||||
- archive commit did not complete.
|
|
||||||
|
|
||||||
Diagnostics:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
|
|
||||||
```
|
|
||||||
|
|
||||||
Safe Fix:
|
|
||||||
- verify archive stage succeeded for the target session.
|
|
||||||
- rerun/archive from a healthy source workspace so current pointers are published.
|
|
||||||
|
|
||||||
Links:
|
|
||||||
- [docs/operations.md](./operations.md)
|
|
||||||
- [docs/internal/stage-archive.md](./internal/stage-archive.md)
|
|
||||||
|
|
||||||
## Restore manifest identity mismatch
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
- `restore` fails because remote manifest session or campaign does not match requested values.
|
|
||||||
|
|
||||||
Likely Cause:
|
|
||||||
- wrong `--session-id` or wrong session config selected.
|
|
||||||
- archive prefix points to a different campaign/session.
|
|
||||||
|
|
||||||
Diagnostics:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
|
|
||||||
```
|
|
||||||
|
|
||||||
Safe Fix:
|
|
||||||
- use the correct session config and `--session-id`.
|
|
||||||
- verify campaign/session identity in local config before restore.
|
|
||||||
|
|
||||||
Links:
|
|
||||||
- [docs/config.md](./config.md)
|
|
||||||
- [docs/operations.md](./operations.md)
|
|
||||||
|
|
||||||
## Restore conflict without `--force`
|
## Restore conflict without `--force`
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- `restore` fails with `restore conflict` and conflict counts.
|
|
||||||
|
|
||||||
Likely Cause:
|
- restore fails with conflict count.
|
||||||
- local durable file differs from remote file for one or more planned restore paths.
|
|
||||||
|
Likely cause:
|
||||||
|
|
||||||
|
- local durable files differ from remote restore sources.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
|
narratio session restore 2026-04-04 --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe Fix:
|
Safe fix:
|
||||||
- review planned conflicts.
|
|
||||||
- rerun with `--force` only when remote state should overwrite local state.
|
|
||||||
|
|
||||||
Links:
|
- review conflicts;
|
||||||
- [docs/cli.md](./cli.md)
|
- rerun with `--force` only when remote state should overwrite local.
|
||||||
- [docs/operations.md](./operations.md)
|
|
||||||
|
|
||||||
## Restore report expectations
|
## Restore current-state discovery failure
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- operator expects restore report file but does not find one.
|
|
||||||
|
|
||||||
Likely Cause:
|
- restore cannot find current pointer or current manifest.
|
||||||
- restore was executed in `--dry-run` mode.
|
|
||||||
- restore failed before report persistence path (for example lock acquisition failure).
|
Likely causes:
|
||||||
|
|
||||||
|
- no committed publish current state;
|
||||||
|
- storage credentials or connectivity failure.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ls -l {workspace.root}/work/{campaign}/{session_id}/reports/restore-latest.json
|
narratio session status 2026-04-04
|
||||||
|
narratio session restore 2026-04-04 --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe Fix:
|
Safe fix:
|
||||||
- run non-dry-run restore for durable report output.
|
|
||||||
- resolve lock or early preflight failures and retry.
|
|
||||||
|
|
||||||
Links:
|
- resolve storage/auth issue;
|
||||||
- [docs/operations.md](./operations.md)
|
- republish from healthy local state if current pointer is missing.
|
||||||
|
|
||||||
## Secrets env-dir or credential-env failure
|
## Publish output failure
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- startup fails loading secrets directory, or stage fails due to missing credential env vars.
|
|
||||||
|
|
||||||
Likely Cause:
|
- publish fails on missing required source, upload error, or commit write.
|
||||||
- invalid `pipeline.secrets.env_dir` path/permissions.
|
|
||||||
- required credential env var unset/empty.
|
Likely causes:
|
||||||
|
|
||||||
|
- required source file not produced;
|
||||||
|
- lock/state expectations mismatch;
|
||||||
|
- remote storage failure.
|
||||||
|
|
||||||
|
Diagnostics:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session artifacts 2026-04-04 --remote
|
||||||
|
narratio session status 2026-04-04
|
||||||
|
narratio run-stage publish 2026-04-04 --force
|
||||||
|
```
|
||||||
|
|
||||||
|
Safe fix:
|
||||||
|
|
||||||
|
- regenerate missing sources by rerunning prerequisite stages;
|
||||||
|
- correct publish source/destination rules;
|
||||||
|
- retry after storage failure is resolved.
|
||||||
|
|
||||||
|
## Secrets or storage credential failure
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
- object-store command fails at initialization/auth.
|
||||||
|
|
||||||
|
Likely causes:
|
||||||
|
|
||||||
|
- invalid `pipeline.secrets.env_dir`;
|
||||||
|
- missing credential environment variables;
|
||||||
|
- invalid S3 endpoint/bucket settings.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ls -la /path/to/secrets_dir
|
ls -la /path/to/secrets_dir
|
||||||
env | grep -E 'AUDITA|OBJECT_STORAGE|AWS|SCRIPTORIUM'
|
env | grep -E 'OBJECT_STORAGE|AWS|AUDITA|SCRIPTORIUM'
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe Fix:
|
Safe fix:
|
||||||
- fix secrets directory and credential env vars.
|
|
||||||
|
- correct secret-file path and permissions;
|
||||||
|
- provide required env vars;
|
||||||
- keep secret values out of YAML.
|
- keep secret values out of YAML.
|
||||||
|
|
||||||
Links:
|
## S3 audio prepare failure
|
||||||
- [docs/config.md](./config.md)
|
|
||||||
|
|
||||||
## S3-audio prepare failure
|
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- `prepare` fails in S3 mode (listing/downloading/no audio/backend error).
|
|
||||||
|
|
||||||
Likely Cause:
|
- prepare fails listing/downloading session S3 audio.
|
||||||
- wrong `session.inputs.audio_s3.prefix`.
|
|
||||||
- no `.flac` files at resolved prefix.
|
Likely causes:
|
||||||
- invalid/missing object-store credentials or backend config.
|
|
||||||
- mixed local+S3 audio input config.
|
- incorrect `session.inputs.audio_s3.prefix`;
|
||||||
|
- no matching `.flac` objects;
|
||||||
|
- storage connectivity or permissions failure.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 prepare
|
narratio run-stage prepare 2026-04-04 --force
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe Fix:
|
Safe fix:
|
||||||
- configure exactly one audio source mode.
|
|
||||||
- verify `.flac` files and storage access.
|
|
||||||
|
|
||||||
Links:
|
- verify prefix contents and storage access;
|
||||||
|
- keep session audio mode consistent.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [docs/cli.md](./cli.md)
|
||||||
- [docs/config.md](./config.md)
|
- [docs/config.md](./config.md)
|
||||||
- [docs/operations.md](./operations.md)
|
- [docs/operations.md](./operations.md)
|
||||||
|
- [docs/internal/stage-publish.md](./internal/stage-publish.md)
|
||||||
## Archive promotion/current-pointer failure
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
- archive fails on required promotion source missing or pointer write failure.
|
|
||||||
|
|
||||||
Likely Cause:
|
|
||||||
- required promoted file absent (including analyze outputs not generated for this run).
|
|
||||||
- storage upload failed before `current/run_id.txt` commit marker write.
|
|
||||||
|
|
||||||
Diagnostics:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio status --manifest /path/to/manifest.json
|
|
||||||
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 archive
|
|
||||||
```
|
|
||||||
|
|
||||||
Safe Fix:
|
|
||||||
- rerun or resume upstream stages to generate required files.
|
|
||||||
- adjust promotion `source`/`dest` rules to match artifacts that must exist.
|
|
||||||
- retry after storage issue is resolved.
|
|
||||||
|
|
||||||
Links:
|
|
||||||
- [docs/operations.md](./operations.md)
|
|
||||||
- [docs/config.md](./config.md)
|
|
||||||
- [docs/internal/stage-archive.md](./internal/stage-archive.md)
|
|
||||||
|
|||||||
1
examples/campaigns/sample-campaign/autocorrect.yml
Normal file
1
examples/campaigns/sample-campaign/autocorrect.yml
Normal file
@@ -0,0 +1 @@
|
|||||||
|
[]
|
||||||
6
examples/campaigns/sample-campaign/campaign.yml
Normal file
6
examples/campaigns/sample-campaign/campaign.yml
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
campaign_id: sample-campaign
|
||||||
|
session_template_file: ./session.template.yml
|
||||||
|
inputs:
|
||||||
|
speakers_file: ./speakers.yml
|
||||||
|
autocorrect_file: ./autocorrect.yml
|
||||||
|
glossary_file: ./glossary.yml
|
||||||
1
examples/campaigns/sample-campaign/glossary.yml
Normal file
1
examples/campaigns/sample-campaign/glossary.yml
Normal file
@@ -0,0 +1 @@
|
|||||||
|
[]
|
||||||
3
examples/campaigns/sample-campaign/session.template.yml
Normal file
3
examples/campaigns/sample-campaign/session.template.yml
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
session_id: "{{ session_id }}"
|
||||||
|
inputs:
|
||||||
|
audio_dir: ./audio
|
||||||
5
examples/campaigns/sample-campaign/speakers.yml
Normal file
5
examples/campaigns/sample-campaign/speakers.yml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
match:
|
||||||
|
- speaker: "Eric Rakestraw"
|
||||||
|
match:
|
||||||
|
- "Eric_Rakestraw"
|
||||||
|
- "Eric"
|
||||||
@@ -4,21 +4,18 @@
|
|||||||
workspace:
|
workspace:
|
||||||
# Optional: defaults to /var/lib/narratio.
|
# Optional: defaults to /var/lib/narratio.
|
||||||
root: /var/lib/narratio/workspace
|
root: /var/lib/narratio/workspace
|
||||||
# Optional: remove run-scoped workdir after successful archive commit.
|
# Optional: remove run-scoped workdir after successful publish commit.
|
||||||
cleanup_after_archive: false
|
cleanup_after_publish: false
|
||||||
|
|
||||||
# Optional: local secret file loader (directory of ENV_VAR_NAME files).
|
# Optional: local secret file loader (directory of ENV_VAR_NAME files).
|
||||||
# secrets:
|
# secrets:
|
||||||
# env_dir: ./secrets
|
# env_dir: ./secrets
|
||||||
|
|
||||||
storage:
|
storage:
|
||||||
# Optional storage backend selector; use "s3" for archive + S3 audio workflows.
|
# Optional storage backend selector; use "s3" for publish + S3 audio workflows.
|
||||||
backend: s3
|
backend: s3
|
||||||
# Compatibility fields retained in schema.
|
|
||||||
bucket: ""
|
|
||||||
prefix: ""
|
|
||||||
s3:
|
s3:
|
||||||
# Required when using S3 audio or S3 archive uploads.
|
# Required when using S3 audio or S3 publish uploads.
|
||||||
bucket: my-dnd-archive
|
bucket: my-dnd-archive
|
||||||
# Optional; defaults to "dnd".
|
# Optional; defaults to "dnd".
|
||||||
root_prefix: dnd
|
root_prefix: dnd
|
||||||
@@ -30,20 +27,26 @@ storage:
|
|||||||
access_key_id_env: OBJECT_STORAGE_KEY_ID
|
access_key_id_env: OBJECT_STORAGE_KEY_ID
|
||||||
secret_access_key_env: OBJECT_STORAGE_KEY
|
secret_access_key_env: OBJECT_STORAGE_KEY
|
||||||
|
|
||||||
|
campaigns:
|
||||||
|
# Optional; defaults to /usr/local/share/narratio/campaigns.
|
||||||
|
root: /usr/local/share/narratio/campaigns
|
||||||
|
# Optional command default when --campaign is omitted.
|
||||||
|
default_campaign_id: sample-campaign
|
||||||
|
|
||||||
spool:
|
spool:
|
||||||
# Optional; defaults to /var/spool/narratio.
|
# Optional; defaults to /var/spool/narratio.
|
||||||
root: /var/spool/narratio
|
root: /var/spool/narratio
|
||||||
# Optional cleanup of run-scoped spool audio after successful archive commit.
|
# Optional cleanup of run-scoped spool audio after successful publish commit.
|
||||||
delete_audio_after_archive: false
|
delete_audio_after_publish: false
|
||||||
|
|
||||||
archive:
|
publish:
|
||||||
# Optional booleans; defaults are true.
|
# Optional booleans; defaults are true.
|
||||||
enabled: true
|
enabled: true
|
||||||
upload_run: true
|
upload_run: true
|
||||||
# Optional promotion rules; sources use Narratio artifact source IDs.
|
# Optional publish output rules; sources use Narratio artifact source IDs.
|
||||||
promote_artifacts:
|
outputs:
|
||||||
- source: narratio.transcript.trimmed
|
- source: narratio.transcript.final_trimmed
|
||||||
dest: transcripts/trimmed.json
|
dest: transcripts/final.trimmed.json
|
||||||
required: true
|
required: true
|
||||||
- source: narratio.artifact.session_recap
|
- source: narratio.artifact.session_recap
|
||||||
dest: artifacts/session_recap.md
|
dest: artifacts/session_recap.md
|
||||||
@@ -96,14 +99,14 @@ audita:
|
|||||||
|
|
||||||
normalize:
|
normalize:
|
||||||
# Optional; defaults shown explicitly.
|
# Optional; defaults shown explicitly.
|
||||||
output_path: transcripts/normalized.json
|
output_path: transcripts/final.json
|
||||||
output_schema: seriatim-intermediate
|
output_schema: seriatim-intermediate
|
||||||
report: true
|
report: true
|
||||||
|
|
||||||
trim:
|
trim:
|
||||||
# Keep disabled unless bounds prompt integration is configured.
|
# Keep disabled unless bounds prompt integration is configured.
|
||||||
enabled: false
|
enabled: false
|
||||||
output_path: transcripts/trimmed.json
|
output_path: transcripts/final.trimmed.json
|
||||||
bounds:
|
bounds:
|
||||||
prompt_id: dnd.session_bounds
|
prompt_id: dnd.session_bounds
|
||||||
profile_id: local-fast
|
profile_id: local-fast
|
||||||
@@ -130,12 +133,10 @@ scriptorium:
|
|||||||
timeout: 10m
|
timeout: 10m
|
||||||
inputs:
|
inputs:
|
||||||
transcript:
|
transcript:
|
||||||
source: narratio.transcript.trimmed
|
source: narratio.transcript.final_trimmed
|
||||||
required: true
|
required: true
|
||||||
previous_recap:
|
previous_recap:
|
||||||
source: previous_session_artifact
|
source: narratio.previous_session.artifact.session_recap
|
||||||
artifact: session_recap
|
|
||||||
path: ""
|
|
||||||
required: false
|
required: false
|
||||||
vars:
|
vars:
|
||||||
session_id: true
|
session_id: true
|
||||||
@@ -160,21 +161,13 @@ scriptorium:
|
|||||||
source: narratio.artifact.session_recap
|
source: narratio.artifact.session_recap
|
||||||
required: true
|
required: true
|
||||||
transcript:
|
transcript:
|
||||||
source: narratio.transcript.trimmed
|
source: narratio.transcript.final_trimmed
|
||||||
required: true
|
required: true
|
||||||
vars:
|
vars:
|
||||||
session_id: true
|
session_id: true
|
||||||
campaign_name: true
|
campaign_name: true
|
||||||
output_kind: player_handout
|
output_kind: player_handout
|
||||||
|
|
||||||
analyzer:
|
|
||||||
# Optional adapter settings.
|
|
||||||
binary_path: ""
|
|
||||||
timeout: 2m
|
|
||||||
artifacts:
|
|
||||||
output_dir: ""
|
|
||||||
types: []
|
|
||||||
|
|
||||||
notification:
|
notification:
|
||||||
# Optional notification settings.
|
# Optional notification settings.
|
||||||
backend: ""
|
backend: ""
|
||||||
|
|||||||
@@ -1,2 +1,6 @@
|
|||||||
|
campaigns:
|
||||||
|
root: /usr/local/share/narratio/campaigns
|
||||||
|
default_campaign_id: sample-campaign
|
||||||
|
|
||||||
whisperx:
|
whisperx:
|
||||||
transcribe_url: "https://transcription.example.com/transcribe"
|
transcribe_url: "https://transcription.example.com/transcribe"
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
workspace:
|
workspace:
|
||||||
root: /var/lib/narratio/workspace
|
root: /var/lib/narratio/workspace
|
||||||
cleanup_after_archive: true
|
cleanup_after_publish: true
|
||||||
|
|
||||||
storage:
|
storage:
|
||||||
backend: s3
|
backend: s3
|
||||||
@@ -11,16 +11,20 @@ storage:
|
|||||||
access_key_id_env: OBJECT_STORAGE_KEY_ID
|
access_key_id_env: OBJECT_STORAGE_KEY_ID
|
||||||
secret_access_key_env: OBJECT_STORAGE_KEY
|
secret_access_key_env: OBJECT_STORAGE_KEY
|
||||||
|
|
||||||
|
campaigns:
|
||||||
|
root: /usr/local/share/narratio/campaigns
|
||||||
|
default_campaign_id: sample-campaign
|
||||||
|
|
||||||
spool:
|
spool:
|
||||||
root: /var/spool/narratio
|
root: /var/spool/narratio
|
||||||
delete_audio_after_archive: true
|
delete_audio_after_publish: true
|
||||||
|
|
||||||
archive:
|
publish:
|
||||||
enabled: true
|
enabled: true
|
||||||
upload_run: true
|
upload_run: true
|
||||||
promote_artifacts:
|
outputs:
|
||||||
- source: narratio.transcript.trimmed
|
- source: narratio.transcript.final_trimmed
|
||||||
dest: transcripts/trimmed.json
|
dest: transcripts/final.trimmed.json
|
||||||
required: true
|
required: true
|
||||||
- source: narratio.artifact.session_recap
|
- source: narratio.artifact.session_recap
|
||||||
dest: artifacts/session_recap.md
|
dest: artifacts/session_recap.md
|
||||||
@@ -57,7 +61,7 @@ audita:
|
|||||||
report: true
|
report: true
|
||||||
|
|
||||||
normalize:
|
normalize:
|
||||||
output_path: transcripts/normalized.json
|
output_path: transcripts/final.json
|
||||||
output_schema: seriatim-intermediate
|
output_schema: seriatim-intermediate
|
||||||
report: true
|
report: true
|
||||||
|
|
||||||
@@ -78,11 +82,10 @@ scriptorium:
|
|||||||
timeout: 10m
|
timeout: 10m
|
||||||
inputs:
|
inputs:
|
||||||
transcript:
|
transcript:
|
||||||
source: narratio.transcript.trimmed
|
source: narratio.transcript.final_trimmed
|
||||||
required: true
|
required: true
|
||||||
previous_recap:
|
previous_recap:
|
||||||
source: previous_session_artifact
|
source: narratio.previous_session.artifact.session_recap
|
||||||
artifact: session_recap
|
|
||||||
required: false
|
required: false
|
||||||
vars:
|
vars:
|
||||||
session_id: true
|
session_id: true
|
||||||
@@ -103,14 +106,11 @@ scriptorium:
|
|||||||
source: narratio.artifact.session_recap
|
source: narratio.artifact.session_recap
|
||||||
required: true
|
required: true
|
||||||
transcript:
|
transcript:
|
||||||
source: narratio.transcript.trimmed
|
source: narratio.transcript.final_trimmed
|
||||||
required: true
|
required: true
|
||||||
vars:
|
vars:
|
||||||
session_id: true
|
session_id: true
|
||||||
output_kind: player_handout
|
output_kind: player_handout
|
||||||
|
|
||||||
analyzer:
|
|
||||||
timeout: 2m
|
|
||||||
|
|
||||||
notification:
|
notification:
|
||||||
timeout: 30s
|
timeout: 30s
|
||||||
|
|||||||
@@ -1,9 +1,5 @@
|
|||||||
session_id: 2026-05-03
|
session_id: 2026-05-03
|
||||||
campaign: sample-campaign
|
|
||||||
date: 2026-05-03
|
date: 2026-05-03
|
||||||
title: Sample Session
|
title: Sample Session
|
||||||
inputs:
|
inputs:
|
||||||
audio_dir: ./audio
|
audio_dir: ./audio
|
||||||
speakers_file: ./examples/speakers.yml
|
|
||||||
autocorrect_file: ./examples/autocorrect.yml
|
|
||||||
glossary_file: ./examples/glossary.yml
|
|
||||||
|
|||||||
@@ -1,10 +1,6 @@
|
|||||||
session_id: 2026-05-03
|
session_id: 2026-05-03
|
||||||
campaign: sample-campaign
|
|
||||||
date: 2026-05-03
|
date: 2026-05-03
|
||||||
title: Sample Session
|
title: Sample Session
|
||||||
inputs:
|
inputs:
|
||||||
audio_s3:
|
audio_s3:
|
||||||
prefix: audio/
|
prefix: audio/
|
||||||
speakers_file: ./examples/speakers.yml
|
|
||||||
autocorrect_file: ./examples/autocorrect.yml
|
|
||||||
glossary_file: ./examples/glossary.yml
|
|
||||||
|
|||||||
@@ -1,7 +1,3 @@
|
|||||||
session_id: "{{ session_id }}"
|
session_id: "{{ session_id }}"
|
||||||
campaign: sample-campaign
|
|
||||||
inputs:
|
inputs:
|
||||||
audio_dir: ./audio
|
audio_dir: ./audio
|
||||||
speakers_file: ./examples/speakers.yml
|
|
||||||
autocorrect_file: ./examples/autocorrect.yml
|
|
||||||
glossary_file: ./examples/glossary.yml
|
|
||||||
|
|||||||
@@ -1,40 +0,0 @@
|
|||||||
package analyzer
|
|
||||||
|
|
||||||
import "context"
|
|
||||||
|
|
||||||
// NoopRunner is a deterministic no-op analyzer adapter.
|
|
||||||
type NoopRunner struct{}
|
|
||||||
|
|
||||||
// Run returns the requested output path with placeholder metadata.
|
|
||||||
func (n *NoopRunner) Run(ctx context.Context, req AnalyzeRequest) (AnalyzeResult, error) {
|
|
||||||
if err := ctx.Err(); err != nil {
|
|
||||||
return AnalyzeResult{}, err
|
|
||||||
}
|
|
||||||
return AnalyzeResult{ArtifactPath: req.OutputPath, Metadata: map[string]any{"placeholder": true}}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// FakeRunner captures analyze requests and returns deterministic responses.
|
|
||||||
type FakeRunner struct {
|
|
||||||
Requests []AnalyzeRequest
|
|
||||||
Err error
|
|
||||||
Result AnalyzeResult
|
|
||||||
}
|
|
||||||
|
|
||||||
// Run records request and returns configured response.
|
|
||||||
func (f *FakeRunner) Run(ctx context.Context, req AnalyzeRequest) (AnalyzeResult, error) {
|
|
||||||
if err := ctx.Err(); err != nil {
|
|
||||||
return AnalyzeResult{}, err
|
|
||||||
}
|
|
||||||
f.Requests = append(f.Requests, req)
|
|
||||||
if f.Err != nil {
|
|
||||||
return AnalyzeResult{}, f.Err
|
|
||||||
}
|
|
||||||
res := f.Result
|
|
||||||
if res.ArtifactPath == "" {
|
|
||||||
res.ArtifactPath = req.OutputPath
|
|
||||||
}
|
|
||||||
if res.Metadata == nil {
|
|
||||||
res.Metadata = map[string]any{"fake": true}
|
|
||||||
}
|
|
||||||
return res, nil
|
|
||||||
}
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
package analyzer
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"errors"
|
|
||||||
"testing"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestFakeRunnerCapturesRequestAndReturnsPath(t *testing.T) {
|
|
||||||
fake := &FakeRunner{}
|
|
||||||
req := AnalyzeRequest{ArtifactType: "session-log", OutputPath: "artifacts/session-log.md"}
|
|
||||||
|
|
||||||
res, err := fake.Run(context.Background(), req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Run() error = %v", err)
|
|
||||||
}
|
|
||||||
if len(fake.Requests) != 1 || fake.Requests[0].ArtifactType != "session-log" {
|
|
||||||
t.Fatalf("requests = %#v, want captured request", fake.Requests)
|
|
||||||
}
|
|
||||||
if res.ArtifactPath != req.OutputPath {
|
|
||||||
t.Fatalf("artifact path = %q, want %q", res.ArtifactPath, req.OutputPath)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestFakeRunnerError(t *testing.T) {
|
|
||||||
fake := &FakeRunner{Err: errors.New("boom")}
|
|
||||||
_, err := fake.Run(context.Background(), AnalyzeRequest{})
|
|
||||||
if err == nil {
|
|
||||||
t.Fatal("expected error, got nil")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
// Package analyzer declares the adapter contract for artifact analysis generation.
|
|
||||||
package analyzer
|
|
||||||
|
|
||||||
import "context"
|
|
||||||
|
|
||||||
// TODO: implement analyzer integration once the analyzer contract is finalized.
|
|
||||||
|
|
||||||
// Runner is the adapter boundary for analyzer invocations.
|
|
||||||
type Runner interface {
|
|
||||||
Run(ctx context.Context, req AnalyzeRequest) (AnalyzeResult, error)
|
|
||||||
}
|
|
||||||
|
|
||||||
// AnalyzeRequest describes one analyzer artifact generation request.
|
|
||||||
type AnalyzeRequest struct {
|
|
||||||
ArtifactType string
|
|
||||||
ProcessedTranscriptPath string
|
|
||||||
ContextReferences []string
|
|
||||||
OutputPath string
|
|
||||||
GeneratedConfigPath string
|
|
||||||
StdoutLogPath string
|
|
||||||
StderrLogPath string
|
|
||||||
}
|
|
||||||
|
|
||||||
// AnalyzeResult describes analyzer output.
|
|
||||||
type AnalyzeResult struct {
|
|
||||||
ArtifactPath string
|
|
||||||
Metadata map[string]any
|
|
||||||
}
|
|
||||||
@@ -14,7 +14,7 @@ func TestFakeRunnerCapturesRequestAndReturnsPath(t *testing.T) {
|
|||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
req := PolishRequest{
|
req := PolishRequest{
|
||||||
GeneratedConfigPath: filepath.Join(dir, "config", "audita.yml"),
|
GeneratedConfigPath: filepath.Join(dir, "config", "audita.yml"),
|
||||||
OutputProcessedPath: filepath.Join(dir, "transcripts", "processed.json"),
|
OutputProcessedPath: filepath.Join(dir, "transcripts", "polished.json"),
|
||||||
StdoutLogPath: filepath.Join(dir, "logs", "audita.stdout.log"),
|
StdoutLogPath: filepath.Join(dir, "logs", "audita.stdout.log"),
|
||||||
StderrLogPath: filepath.Join(dir, "logs", "audita.stderr.log"),
|
StderrLogPath: filepath.Join(dir, "logs", "audita.stderr.log"),
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -52,9 +52,9 @@ func TestSubprocessRunnerSuccessArgsEnvAndValidation(t *testing.T) {
|
|||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
req := PolishRequest{
|
req := PolishRequest{
|
||||||
GeneratedConfigPath: filepath.Join(dir, "audita.generated.yml"),
|
GeneratedConfigPath: filepath.Join(dir, "audita.generated.yml"),
|
||||||
MergedTranscriptPath: filepath.Join(dir, "merged.json"),
|
MergedTranscriptPath: filepath.Join(dir, "base.json"),
|
||||||
GlossaryPath: filepath.Join(dir, "glossary.yml"),
|
GlossaryPath: filepath.Join(dir, "glossary.yml"),
|
||||||
OutputProcessedPath: filepath.Join(dir, "processed.json"),
|
OutputProcessedPath: filepath.Join(dir, "polished.json"),
|
||||||
ReportPath: filepath.Join(dir, "audita.report.json"),
|
ReportPath: filepath.Join(dir, "audita.report.json"),
|
||||||
WorkDir: filepath.Join(dir, "artifacts", "audita-work"),
|
WorkDir: filepath.Join(dir, "artifacts", "audita-work"),
|
||||||
StdoutLogPath: filepath.Join(dir, "audita.stdout.log"),
|
StdoutLogPath: filepath.Join(dir, "audita.stdout.log"),
|
||||||
@@ -571,7 +571,7 @@ func mustAuditaRunner(t *testing.T, cfg SubprocessRunnerConfig) *SubprocessRunne
|
|||||||
func auditaReqForTest(t *testing.T, withReport bool) PolishRequest {
|
func auditaReqForTest(t *testing.T, withReport bool) PolishRequest {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
merged := filepath.Join(dir, "merged.json")
|
merged := filepath.Join(dir, "base.json")
|
||||||
glossary := filepath.Join(dir, "glossary.yml")
|
glossary := filepath.Join(dir, "glossary.yml")
|
||||||
writeAuditaTestFile(t, merged, `{"segments":[]}`)
|
writeAuditaTestFile(t, merged, `{"segments":[]}`)
|
||||||
writeAuditaTestFile(t, glossary, "terms: []\n")
|
writeAuditaTestFile(t, glossary, "terms: []\n")
|
||||||
@@ -579,7 +579,7 @@ func auditaReqForTest(t *testing.T, withReport bool) PolishRequest {
|
|||||||
GeneratedConfigPath: filepath.Join(dir, "audita.generated.yml"),
|
GeneratedConfigPath: filepath.Join(dir, "audita.generated.yml"),
|
||||||
MergedTranscriptPath: merged,
|
MergedTranscriptPath: merged,
|
||||||
GlossaryPath: glossary,
|
GlossaryPath: glossary,
|
||||||
OutputProcessedPath: filepath.Join(dir, "processed.json"),
|
OutputProcessedPath: filepath.Join(dir, "polished.json"),
|
||||||
WorkDir: filepath.Join(dir, "artifacts", "audita-work"),
|
WorkDir: filepath.Join(dir, "artifacts", "audita-work"),
|
||||||
StdoutLogPath: filepath.Join(dir, "audita.stdout.log"),
|
StdoutLogPath: filepath.Join(dir, "audita.stdout.log"),
|
||||||
StderrLogPath: filepath.Join(dir, "audita.stderr.log"),
|
StderrLogPath: filepath.Join(dir, "audita.stderr.log"),
|
||||||
|
|||||||
@@ -31,7 +31,7 @@ func TestSubprocessRunnerRunSuccessBuildsDeterministicArgsAndCapturesLogs(t *tes
|
|||||||
ConfigPath: "/etc/scriptorium/config.yml",
|
ConfigPath: "/etc/scriptorium/config.yml",
|
||||||
PromptID: "dnd.session_recap",
|
PromptID: "dnd.session_recap",
|
||||||
ProfileID: "local-quality",
|
ProfileID: "local-quality",
|
||||||
InputPaths: map[string]string{"transcript": filepath.Join(dir, "processed.json"), "other": filepath.Join(dir, "other.md")},
|
InputPaths: map[string]string{"transcript": filepath.Join(dir, "polished.json"), "other": filepath.Join(dir, "other.md")},
|
||||||
Vars: map[string]string{"session_id": "2026-05-03", "campaign_name": "Icewind Dale"},
|
Vars: map[string]string{"session_id": "2026-05-03", "campaign_name": "Icewind Dale"},
|
||||||
OutputPath: filepath.Join(dir, "artifacts", "session_recap.md"),
|
OutputPath: filepath.Join(dir, "artifacts", "session_recap.md"),
|
||||||
StdoutLogPath: filepath.Join(dir, "logs", "scriptorium.run.stdout.log"),
|
StdoutLogPath: filepath.Join(dir, "logs", "scriptorium.run.stdout.log"),
|
||||||
@@ -180,7 +180,7 @@ func TestSubprocessRunnerRenderSuccess(t *testing.T) {
|
|||||||
req := RenderArtifactRequest{
|
req := RenderArtifactRequest{
|
||||||
Binary: wrapper,
|
Binary: wrapper,
|
||||||
PromptID: "dnd.session_recap",
|
PromptID: "dnd.session_recap",
|
||||||
InputPaths: map[string]string{"transcript": filepath.Join(dir, "processed.json")},
|
InputPaths: map[string]string{"transcript": filepath.Join(dir, "polished.json")},
|
||||||
OutputPath: filepath.Join(dir, "artifacts", "session_recap.render.json"),
|
OutputPath: filepath.Join(dir, "artifacts", "session_recap.render.json"),
|
||||||
StdoutLogPath: filepath.Join(dir, "logs", "scriptorium.render.stdout.log"),
|
StdoutLogPath: filepath.Join(dir, "logs", "scriptorium.render.stdout.log"),
|
||||||
StderrLogPath: filepath.Join(dir, "logs", "scriptorium.render.stderr.log"),
|
StderrLogPath: filepath.Join(dir, "logs", "scriptorium.render.stderr.log"),
|
||||||
@@ -285,7 +285,7 @@ type scriptoriumHelperRecord struct {
|
|||||||
func runReqForTest(t *testing.T, binary string) RunArtifactRequest {
|
func runReqForTest(t *testing.T, binary string) RunArtifactRequest {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
transcriptPath := filepath.Join(dir, "processed.json")
|
transcriptPath := filepath.Join(dir, "polished.json")
|
||||||
writeScriptoriumFile(t, transcriptPath, `{"segments":[]}`)
|
writeScriptoriumFile(t, transcriptPath, `{"segments":[]}`)
|
||||||
return RunArtifactRequest{
|
return RunArtifactRequest{
|
||||||
Binary: binary,
|
Binary: binary,
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ func TestFakeRunnerCapturesRequestAndReturnsPath(t *testing.T) {
|
|||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
req := MergeRequest{
|
req := MergeRequest{
|
||||||
GeneratedConfigPath: filepath.Join(dir, "config", "seriatim.yml"),
|
GeneratedConfigPath: filepath.Join(dir, "config", "seriatim.yml"),
|
||||||
OutputMergedTranscriptPath: filepath.Join(dir, "transcripts", "merged.json"),
|
OutputMergedTranscriptPath: filepath.Join(dir, "transcripts", "base.json"),
|
||||||
StdoutLogPath: filepath.Join(dir, "logs", "seriatim.stdout.log"),
|
StdoutLogPath: filepath.Join(dir, "logs", "seriatim.stdout.log"),
|
||||||
StderrLogPath: filepath.Join(dir, "logs", "seriatim.stderr.log"),
|
StderrLogPath: filepath.Join(dir, "logs", "seriatim.stderr.log"),
|
||||||
}
|
}
|
||||||
@@ -57,8 +57,8 @@ func TestFakeRunnerTrimCapturesRequestAndReturnsPath(t *testing.T) {
|
|||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
req := TrimRequest{
|
req := TrimRequest{
|
||||||
GeneratedConfigPath: filepath.Join(dir, "config", "seriatim.trim.yml"),
|
GeneratedConfigPath: filepath.Join(dir, "config", "seriatim.trim.yml"),
|
||||||
InputTranscriptPath: filepath.Join(dir, "transcripts", "processed.json"),
|
InputTranscriptPath: filepath.Join(dir, "transcripts", "polished.json"),
|
||||||
OutputTrimmedPath: filepath.Join(dir, "transcripts", "trimmed.json"),
|
OutputTrimmedPath: filepath.Join(dir, "transcripts", "final.trimmed.json"),
|
||||||
KeepSelector: "1-10",
|
KeepSelector: "1-10",
|
||||||
StdoutLogPath: filepath.Join(dir, "logs", "seriatim.trim.stdout.log"),
|
StdoutLogPath: filepath.Join(dir, "logs", "seriatim.trim.stdout.log"),
|
||||||
StderrLogPath: filepath.Join(dir, "logs", "seriatim.trim.stderr.log"),
|
StderrLogPath: filepath.Join(dir, "logs", "seriatim.trim.stderr.log"),
|
||||||
@@ -105,8 +105,8 @@ func TestFakeRunnerNormalizeCapturesRequestAndReturnsPath(t *testing.T) {
|
|||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
req := NormalizeRequest{
|
req := NormalizeRequest{
|
||||||
GeneratedConfigPath: filepath.Join(dir, "config", "seriatim.normalize.yml"),
|
GeneratedConfigPath: filepath.Join(dir, "config", "seriatim.normalize.yml"),
|
||||||
InputTranscriptPath: filepath.Join(dir, "transcripts", "processed.json"),
|
InputTranscriptPath: filepath.Join(dir, "transcripts", "polished.json"),
|
||||||
OutputNormalizedPath: filepath.Join(dir, "transcripts", "normalized.json"),
|
OutputNormalizedPath: filepath.Join(dir, "transcripts", "final.json"),
|
||||||
OutputSchema: "seriatim-intermediate",
|
OutputSchema: "seriatim-intermediate",
|
||||||
ReportPath: filepath.Join(dir, "artifacts", "seriatim.normalize.report.json"),
|
ReportPath: filepath.Join(dir, "artifacts", "seriatim.normalize.report.json"),
|
||||||
StdoutLogPath: filepath.Join(dir, "logs", "seriatim.normalize.stdout.log"),
|
StdoutLogPath: filepath.Join(dir, "logs", "seriatim.normalize.stdout.log"),
|
||||||
|
|||||||
@@ -50,7 +50,7 @@ func TestSubprocessRunnerSuccessWithReportArgsAndEnv(t *testing.T) {
|
|||||||
req := MergeRequest{
|
req := MergeRequest{
|
||||||
GeneratedConfigPath: filepath.Join(dir, "seriatim.generated.yml"),
|
GeneratedConfigPath: filepath.Join(dir, "seriatim.generated.yml"),
|
||||||
InputTranscriptPaths: []string{filepath.Join(dir, "a.json"), filepath.Join(dir, "b.json")},
|
InputTranscriptPaths: []string{filepath.Join(dir, "a.json"), filepath.Join(dir, "b.json")},
|
||||||
OutputMergedTranscriptPath: filepath.Join(dir, "merged.json"),
|
OutputMergedTranscriptPath: filepath.Join(dir, "base.json"),
|
||||||
ReportPath: filepath.Join(dir, "seriatim.report.json"),
|
ReportPath: filepath.Join(dir, "seriatim.report.json"),
|
||||||
SpeakersPath: filepath.Join(dir, "speakers.yml"),
|
SpeakersPath: filepath.Join(dir, "speakers.yml"),
|
||||||
AutocorrectPath: filepath.Join(dir, "autocorrect.yml"),
|
AutocorrectPath: filepath.Join(dir, "autocorrect.yml"),
|
||||||
@@ -732,7 +732,7 @@ func mergeReqForTest(t *testing.T, withReport bool) MergeRequest {
|
|||||||
req := MergeRequest{
|
req := MergeRequest{
|
||||||
GeneratedConfigPath: filepath.Join(dir, "seriatim.generated.yml"),
|
GeneratedConfigPath: filepath.Join(dir, "seriatim.generated.yml"),
|
||||||
InputTranscriptPaths: []string{in1, in2},
|
InputTranscriptPaths: []string{in1, in2},
|
||||||
OutputMergedTranscriptPath: filepath.Join(dir, "merged.json"),
|
OutputMergedTranscriptPath: filepath.Join(dir, "base.json"),
|
||||||
StdoutLogPath: filepath.Join(dir, "seriatim.stdout.log"),
|
StdoutLogPath: filepath.Join(dir, "seriatim.stdout.log"),
|
||||||
StderrLogPath: filepath.Join(dir, "seriatim.stderr.log"),
|
StderrLogPath: filepath.Join(dir, "seriatim.stderr.log"),
|
||||||
}
|
}
|
||||||
@@ -745,11 +745,11 @@ func mergeReqForTest(t *testing.T, withReport bool) MergeRequest {
|
|||||||
func trimReqForTest(t *testing.T) TrimRequest {
|
func trimReqForTest(t *testing.T) TrimRequest {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
input := filepath.Join(dir, "processed.json")
|
input := filepath.Join(dir, "polished.json")
|
||||||
writeSeriatimFile(t, input, `{"schema":"seriatim.intermediate.v1","segments":[]}`)
|
writeSeriatimFile(t, input, `{"schema":"seriatim.intermediate.v1","segments":[]}`)
|
||||||
return TrimRequest{
|
return TrimRequest{
|
||||||
InputTranscriptPath: input,
|
InputTranscriptPath: input,
|
||||||
OutputTrimmedPath: filepath.Join(dir, "trimmed.json"),
|
OutputTrimmedPath: filepath.Join(dir, "final.trimmed.json"),
|
||||||
KeepSelector: "5-12",
|
KeepSelector: "5-12",
|
||||||
GeneratedConfigPath: filepath.Join(dir, "seriatim.trim.generated.yml"),
|
GeneratedConfigPath: filepath.Join(dir, "seriatim.trim.generated.yml"),
|
||||||
StdoutLogPath: filepath.Join(dir, "seriatim.trim.stdout.log"),
|
StdoutLogPath: filepath.Join(dir, "seriatim.trim.stdout.log"),
|
||||||
@@ -760,12 +760,12 @@ func trimReqForTest(t *testing.T) TrimRequest {
|
|||||||
func normalizeReqForTest(t *testing.T, withReport bool) NormalizeRequest {
|
func normalizeReqForTest(t *testing.T, withReport bool) NormalizeRequest {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
input := filepath.Join(dir, "processed.json")
|
input := filepath.Join(dir, "polished.json")
|
||||||
writeSeriatimFile(t, input, `{"schema":"audita.processed.v1","segments":[]}`)
|
writeSeriatimFile(t, input, `{"schema":"audita.processed.v1","segments":[]}`)
|
||||||
|
|
||||||
req := NormalizeRequest{
|
req := NormalizeRequest{
|
||||||
InputTranscriptPath: input,
|
InputTranscriptPath: input,
|
||||||
OutputNormalizedPath: filepath.Join(dir, "normalized.json"),
|
OutputNormalizedPath: filepath.Join(dir, "final.json"),
|
||||||
OutputSchema: "seriatim-intermediate",
|
OutputSchema: "seriatim-intermediate",
|
||||||
GeneratedConfigPath: filepath.Join(dir, "seriatim.normalize.generated.yml"),
|
GeneratedConfigPath: filepath.Join(dir, "seriatim.normalize.generated.yml"),
|
||||||
StdoutLogPath: filepath.Join(dir, "seriatim.normalize.stdout.log"),
|
StdoutLogPath: filepath.Join(dir, "seriatim.normalize.stdout.log"),
|
||||||
|
|||||||
@@ -29,6 +29,7 @@ type FakeBackend struct {
|
|||||||
|
|
||||||
Objects map[string]FakeObject
|
Objects map[string]FakeObject
|
||||||
Uploads []FakeUploadCall
|
Uploads []FakeUploadCall
|
||||||
|
Downloads []FakeDownloadCall
|
||||||
|
|
||||||
ListErr error
|
ListErr error
|
||||||
DownloadErr error
|
DownloadErr error
|
||||||
@@ -43,6 +44,12 @@ type FakeUploadCall struct {
|
|||||||
Options UploadOptions
|
Options UploadOptions
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// FakeDownloadCall captures one download invocation in call order.
|
||||||
|
type FakeDownloadCall struct {
|
||||||
|
Key string
|
||||||
|
LocalPath string
|
||||||
|
}
|
||||||
|
|
||||||
// Archive records request and returns configured response.
|
// Archive records request and returns configured response.
|
||||||
func (f *FakeBackend) Archive(ctx context.Context, req ArchiveRequest) (ArchiveResult, error) {
|
func (f *FakeBackend) Archive(ctx context.Context, req ArchiveRequest) (ArchiveResult, error) {
|
||||||
if err := ctx.Err(); err != nil {
|
if err := ctx.Err(); err != nil {
|
||||||
@@ -130,6 +137,10 @@ func (f *FakeBackend) Download(ctx context.Context, key, localPath string) error
|
|||||||
if !ok {
|
if !ok {
|
||||||
return fmt.Errorf("download object %q: %w", key, os.ErrNotExist)
|
return fmt.Errorf("download object %q: %w", key, os.ErrNotExist)
|
||||||
}
|
}
|
||||||
|
f.Downloads = append(f.Downloads, FakeDownloadCall{
|
||||||
|
Key: normalizeObjectKey(key),
|
||||||
|
LocalPath: localPath,
|
||||||
|
})
|
||||||
|
|
||||||
if err := os.MkdirAll(filepath.Dir(localPath), 0o755); err != nil {
|
if err := os.MkdirAll(filepath.Dir(localPath), 0o755); err != nil {
|
||||||
return fmt.Errorf("download object %q: create parent directory: %w", key, err)
|
return fmt.Errorf("download object %q: create parent directory: %w", key, err)
|
||||||
|
|||||||
@@ -56,7 +56,7 @@ func TestFakeBackendDownload(t *testing.T) {
|
|||||||
fake.SeedObject(FakeObject{Key: "audio/a.flac", Data: []byte("audio-a")})
|
fake.SeedObject(FakeObject{Key: "audio/a.flac", Data: []byte("audio-a")})
|
||||||
|
|
||||||
dst := filepath.Join(t.TempDir(), "nested", "a.flac")
|
dst := filepath.Join(t.TempDir(), "nested", "a.flac")
|
||||||
if err := fake.Download(context.Background(), "audio/a.flac", dst); err != nil {
|
if err := fake.Download(context.Background(), `audio\a.flac`, dst); err != nil {
|
||||||
t.Fatalf("Download() error = %v", err)
|
t.Fatalf("Download() error = %v", err)
|
||||||
}
|
}
|
||||||
data, err := os.ReadFile(dst)
|
data, err := os.ReadFile(dst)
|
||||||
|
|||||||
@@ -46,7 +46,7 @@ func (f *artifactSelectionFlag) Normalize() ([]string, error) {
|
|||||||
return out, nil
|
return out, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func validateSelectedAnalyzeArtifacts(cfg *config.Config, selected []string) error {
|
func validateSelectedArtifacts(cfg *config.Config, selected []string) error {
|
||||||
if len(selected) == 0 {
|
if len(selected) == 0 {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -9,36 +9,80 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/stage"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestExecuteRunStageArtifactsNonAnalyzeFails(t *testing.T) {
|
func TestExecuteRunStageArtifactsUnsupportedStageFails(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute(
|
code := Execute(
|
||||||
[]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "--artifacts", "session_recap", "polish"},
|
[]string{"run-stage", "polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||||
&stdout,
|
&stdout,
|
||||||
&stderr,
|
&stderr,
|
||||||
)
|
)
|
||||||
if code == 0 {
|
if code == 0 {
|
||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
if !strings.Contains(stderr.String(), `run-stage: --artifacts is only supported for stage "analyze"`) {
|
if !strings.Contains(stderr.String(), `run-stage: --artifacts is only supported for stages "analyze" and "publish"`) {
|
||||||
t.Fatalf("stderr = %q, want stage-gating error", stderr.String())
|
t.Fatalf("stderr = %q, want stage-gating error", stderr.String())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestExecuteRunStageArchivePropagatesSelectedArtifacts(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
|
var capturedStages []string
|
||||||
|
var capturedArtifacts []string
|
||||||
|
origExecuteStagesFn := executeStagesFn
|
||||||
|
t.Cleanup(func() {
|
||||||
|
executeStagesFn = origExecuteStagesFn
|
||||||
|
})
|
||||||
|
executeStagesFn = func(_ context.Context, _ *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||||
|
for _, s := range stages {
|
||||||
|
capturedStages = append(capturedStages, s.Name())
|
||||||
|
}
|
||||||
|
capturedArtifacts = append([]string(nil), opts.SelectedArtifacts...)
|
||||||
|
return &RunSummary{ManifestPath: filepath.Join(workspaceRoot, "manifest.json"), Executed: []string{"publish"}}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(
|
||||||
|
[]string{
|
||||||
|
"run-stage", "publish", "2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
"--artifacts", "session_recap",
|
||||||
|
},
|
||||||
|
&stdout,
|
||||||
|
&stderr,
|
||||||
|
)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if len(capturedStages) != 1 || capturedStages[0] != "publish" {
|
||||||
|
t.Fatalf("captured stages = %#v, want [publish]", capturedStages)
|
||||||
|
}
|
||||||
|
if strings.Join(capturedArtifacts, ",") != "session_recap" {
|
||||||
|
t.Fatalf("captured artifacts = %#v, want [session_recap]", capturedArtifacts)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestExecuteUnknownArtifactsFailValidation(t *testing.T) {
|
func TestExecuteUnknownArtifactsFailValidation(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute(
|
code := Execute(
|
||||||
[]string{"run", "--config", pipelinePath, "--session", sessionPath, "--artifacts", "unknown_artifact"},
|
[]string{"run", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "unknown_artifact"},
|
||||||
&stdout,
|
&stdout,
|
||||||
&stderr,
|
&stderr,
|
||||||
)
|
)
|
||||||
@@ -52,7 +96,7 @@ func TestExecuteUnknownArtifactsFailValidation(t *testing.T) {
|
|||||||
|
|
||||||
func TestRunStageArtifactsDoesNotImplyForce(t *testing.T) {
|
func TestRunStageArtifactsDoesNotImplyForce(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
@@ -65,7 +109,7 @@ func TestRunStageArtifactsDoesNotImplyForce(t *testing.T) {
|
|||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := RunStage(
|
err := RunStage(
|
||||||
context.Background(),
|
context.Background(),
|
||||||
[]string{"--config", pipelinePath, "--session", sessionPath, "--artifacts", "session_recap,session_recap", "analyze"},
|
[]string{"analyze", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap,session_recap"},
|
||||||
&out,
|
&out,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -78,12 +122,12 @@ func TestRunStageArtifactsDoesNotImplyForce(t *testing.T) {
|
|||||||
|
|
||||||
func TestResumeArtifactsWithSucceededAnalyzeSkipsUnlessForced(t *testing.T) {
|
func TestResumeArtifactsWithSucceededAnalyzeSkipsUnlessForced(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
seed := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
seed := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
||||||
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "archive", "notify"} {
|
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
||||||
seed.MarkStageSucceeded(stageName, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
seed.MarkStageSucceeded(stageName, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
||||||
}
|
}
|
||||||
if err := store.Save(context.Background(), manifestPath, seed); err != nil {
|
if err := store.Save(context.Background(), manifestPath, seed); err != nil {
|
||||||
@@ -93,7 +137,7 @@ func TestResumeArtifactsWithSucceededAnalyzeSkipsUnlessForced(t *testing.T) {
|
|||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Resume(
|
err := Resume(
|
||||||
context.Background(),
|
context.Background(),
|
||||||
[]string{"--config", pipelinePath, "--session", sessionPath, "--artifacts", "session_recap"},
|
[]string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||||
&out,
|
&out,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -104,10 +148,260 @@ func TestResumeArtifactsWithSucceededAnalyzeSkipsUnlessForced(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func writeValidConfigFilesWithScriptoriumArtifacts(t *testing.T, workspaceRoot string) (string, string) {
|
func TestExecuteAnalyzeForceRunsAnalyze(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
|
var capturedStages []string
|
||||||
|
var capturedForce bool
|
||||||
|
origExecuteStagesFn := executeStagesFn
|
||||||
|
t.Cleanup(func() {
|
||||||
|
executeStagesFn = origExecuteStagesFn
|
||||||
|
})
|
||||||
|
executeStagesFn = func(_ context.Context, _ *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||||
|
for _, s := range stages {
|
||||||
|
capturedStages = append(capturedStages, s.Name())
|
||||||
|
}
|
||||||
|
capturedForce = opts.Force
|
||||||
|
return &RunSummary{
|
||||||
|
ManifestPath: filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json"),
|
||||||
|
Executed: []string{"analyze"},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(
|
||||||
|
[]string{"analyze", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||||
|
&stdout,
|
||||||
|
&stderr,
|
||||||
|
)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if len(capturedStages) != 1 || capturedStages[0] != "analyze" {
|
||||||
|
t.Fatalf("captured stages = %#v, want [analyze]", capturedStages)
|
||||||
|
}
|
||||||
|
if !capturedForce {
|
||||||
|
t.Fatal("captured force = false, want true")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "narratio analyze: executed=1 skipped=0 force=true; manifest=") {
|
||||||
|
t.Fatalf("stdout = %q, want analyze summary", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteAnalyzePropagatesSelectedArtifacts(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
|
var capturedArtifacts []string
|
||||||
|
origExecuteStagesFn := executeStagesFn
|
||||||
|
t.Cleanup(func() {
|
||||||
|
executeStagesFn = origExecuteStagesFn
|
||||||
|
})
|
||||||
|
executeStagesFn = func(_ context.Context, _ *config.Config, _ []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||||
|
capturedArtifacts = append([]string(nil), opts.SelectedArtifacts...)
|
||||||
|
return &RunSummary{ManifestPath: filepath.Join(workspaceRoot, "manifest.json"), Executed: []string{"analyze"}}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(
|
||||||
|
[]string{
|
||||||
|
"analyze",
|
||||||
|
"2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
"--artifacts", "player_handout,session_recap",
|
||||||
|
},
|
||||||
|
&stdout,
|
||||||
|
&stderr,
|
||||||
|
)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if strings.Join(capturedArtifacts, ",") != "player_handout,session_recap" {
|
||||||
|
t.Fatalf("captured artifacts = %#v, want sorted selected artifacts", capturedArtifacts)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteAnalyzeUnknownArtifactFailsValidation(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(
|
||||||
|
[]string{"analyze", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "unknown_artifact"},
|
||||||
|
&stdout,
|
||||||
|
&stderr,
|
||||||
|
)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), `analyze: --artifacts includes unknown artifact "unknown_artifact"`) {
|
||||||
|
t.Fatalf("stderr = %q, want unknown-artifact validation error", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteAnalyzeRejectsPositionalArgsAndForceFlag(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
args []string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{name: "extra positional", args: []string{"analyze", "2026-05-03", "extra"}, want: "analyze: unexpected positional arguments"},
|
||||||
|
{name: "force flag", args: []string{"analyze", "--force"}, want: "analyze: invalid flags: flag provided but not defined: -force"},
|
||||||
|
}
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(tc.args, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), tc.want) {
|
||||||
|
t.Fatalf("stderr = %q, want %q", stderr.String(), tc.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteAnalyzeMissingConfigUsesRunStageLoadingPath(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"analyze", "2026-05-03"}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "analyze: no pipeline config path provided and no default pipeline config found; searched:") {
|
||||||
|
t.Fatalf("stderr = %q, want pipeline discovery error", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecutePublishForceRunsArchive(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
|
var capturedStages []string
|
||||||
|
var capturedForce bool
|
||||||
|
var capturedArtifacts []string
|
||||||
|
origExecuteStagesFn := executeStagesFn
|
||||||
|
t.Cleanup(func() {
|
||||||
|
executeStagesFn = origExecuteStagesFn
|
||||||
|
})
|
||||||
|
executeStagesFn = func(_ context.Context, _ *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||||
|
for _, s := range stages {
|
||||||
|
capturedStages = append(capturedStages, s.Name())
|
||||||
|
}
|
||||||
|
capturedForce = opts.Force
|
||||||
|
capturedArtifacts = append([]string(nil), opts.SelectedArtifacts...)
|
||||||
|
return &RunSummary{
|
||||||
|
ManifestPath: filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json"),
|
||||||
|
Executed: []string{"publish"},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(
|
||||||
|
[]string{"publish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||||
|
&stdout,
|
||||||
|
&stderr,
|
||||||
|
)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if len(capturedStages) != 1 || capturedStages[0] != "publish" {
|
||||||
|
t.Fatalf("captured stages = %#v, want [publish]", capturedStages)
|
||||||
|
}
|
||||||
|
if !capturedForce {
|
||||||
|
t.Fatal("captured force = false, want true")
|
||||||
|
}
|
||||||
|
if strings.Join(capturedArtifacts, ",") != "session_recap" {
|
||||||
|
t.Fatalf("captured artifacts = %#v, want [session_recap]", capturedArtifacts)
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "narratio publish: executed=1 skipped=0 force=true; manifest=") {
|
||||||
|
t.Fatalf("stdout = %q, want publish summary", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecutePublishRejectsUnsupportedArgsAndFlags(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
args []string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{name: "extra positional", args: []string{"publish", "2026-05-03", "extra"}, want: "publish: unexpected positional arguments"},
|
||||||
|
{name: "force flag", args: []string{"publish", "--force"}, want: "publish: invalid flags: flag provided but not defined: -force"},
|
||||||
|
}
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(tc.args, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), tc.want) {
|
||||||
|
t.Fatalf("stderr = %q, want %q", stderr.String(), tc.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecutePublishUnknownArtifactFailsValidation(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(
|
||||||
|
[]string{"publish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "unknown_artifact"},
|
||||||
|
&stdout,
|
||||||
|
&stderr,
|
||||||
|
)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), `publish: --artifacts includes unknown artifact "unknown_artifact"`) {
|
||||||
|
t.Fatalf("stderr = %q, want unknown-artifact validation error", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecutePublishMissingConfigUsesRunStageLoadingPath(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"publish", "2026-05-03"}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "publish: no pipeline config path provided and no default pipeline config found; searched:") {
|
||||||
|
t.Fatalf("stderr = %q, want pipeline discovery error", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteUsageIncludesAnalyzeAndPublish(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(nil, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "analyze") {
|
||||||
|
t.Fatalf("stderr = %q, want usage to include analyze", stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "publish") {
|
||||||
|
t.Fatalf("stderr = %q, want usage to include publish", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeValidConfigFilesWithScriptoriumArtifacts(t *testing.T, workspaceRoot string) (string, string, string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
f, err := os.OpenFile(pipelinePath, os.O_APPEND|os.O_WRONLY, 0)
|
f, err := os.OpenFile(pipelinePath, os.O_APPEND|os.O_WRONLY, 0)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("open pipeline config for append: %v", err)
|
t.Fatalf("open pipeline config for append: %v", err)
|
||||||
@@ -136,5 +430,5 @@ scriptorium:
|
|||||||
if _, err := f.WriteString(extra); err != nil {
|
if _, err := f.WriteString(extra); err != nil {
|
||||||
t.Fatalf("append scriptorium config: %v", err)
|
t.Fatalf("append scriptorium config: %v", err)
|
||||||
}
|
}
|
||||||
return pipelinePath, sessionPath
|
return pipelinePath, campaignPath, sessionPath
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -64,7 +64,7 @@ func TestArtifactSelectionFlagNormalize(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestValidateSelectedAnalyzeArtifacts(t *testing.T) {
|
func TestValidateSelectedArtifacts(t *testing.T) {
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
cfg *config.Config
|
cfg *config.Config
|
||||||
@@ -114,7 +114,7 @@ func TestValidateSelectedAnalyzeArtifacts(t *testing.T) {
|
|||||||
|
|
||||||
for _, tt := range tests {
|
for _, tt := range tests {
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
err := validateSelectedAnalyzeArtifacts(tt.cfg, tt.selected)
|
err := validateSelectedArtifacts(tt.cfg, tt.selected)
|
||||||
if tt.wantErr != "" {
|
if tt.wantErr != "" {
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatalf("error = nil, want %q", tt.wantErr)
|
t.Fatalf("error = nil, want %q", tt.wantErr)
|
||||||
|
|||||||
44
internal/app/campaign_config_path.go
Normal file
44
internal/app/campaign_config_path.go
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
func resolveCampaignConfigPath(pipelineCfg *config.PipelineConfig, campaignIDFlag, campaignFileFlag string) (string, error) {
|
||||||
|
campaignID := strings.TrimSpace(campaignIDFlag)
|
||||||
|
campaignFile := strings.TrimSpace(campaignFileFlag)
|
||||||
|
if campaignID != "" && campaignFile != "" {
|
||||||
|
return "", fmt.Errorf("--campaign and --campaign-file are mutually exclusive")
|
||||||
|
}
|
||||||
|
if campaignFile != "" {
|
||||||
|
return filepath.Clean(campaignFile), nil
|
||||||
|
}
|
||||||
|
if campaignID == "" && pipelineCfg != nil {
|
||||||
|
campaignID = strings.TrimSpace(pipelineCfg.Campaigns.DefaultCampaignID)
|
||||||
|
}
|
||||||
|
if campaignID == "" {
|
||||||
|
return "", fmt.Errorf("no campaign selected; pass --campaign <id> or set pipeline.campaigns.default_campaign_id")
|
||||||
|
}
|
||||||
|
if err := validateCampaignIDToken(campaignID); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if pipelineCfg == nil || strings.TrimSpace(pipelineCfg.Campaigns.Root) == "" {
|
||||||
|
return "", fmt.Errorf("pipeline.campaigns.root is required to select campaign %q", campaignID)
|
||||||
|
}
|
||||||
|
return filepath.Clean(filepath.Join(pipelineCfg.Campaigns.Root, campaignID, "campaign.yml")), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateCampaignIDToken(campaignID string) error {
|
||||||
|
if filepath.IsAbs(campaignID) ||
|
||||||
|
strings.Contains(campaignID, "/") ||
|
||||||
|
strings.Contains(campaignID, `\`) ||
|
||||||
|
campaignID == "." ||
|
||||||
|
campaignID == ".." {
|
||||||
|
return fmt.Errorf("campaign id %q must be a single path segment", campaignID)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
84
internal/app/campaign_config_path_test.go
Normal file
84
internal/app/campaign_config_path_test.go
Normal file
@@ -0,0 +1,84 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestResolveCampaignConfigPathCampaignFileWins(t *testing.T) {
|
||||||
|
explicit := filepath.Join(t.TempDir(), "custom-campaign.yml")
|
||||||
|
got, err := resolveCampaignConfigPath(&config.PipelineConfig{}, "", explicit)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("resolveCampaignConfigPath() error = %v", err)
|
||||||
|
}
|
||||||
|
if got != explicit {
|
||||||
|
t.Fatalf("path = %q, want explicit path %q", got, explicit)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveCampaignConfigPathUsesSelectedCampaignID(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
pipelineCfg := &config.PipelineConfig{}
|
||||||
|
pipelineCfg.Campaigns.Root = dir
|
||||||
|
|
||||||
|
got, err := resolveCampaignConfigPath(pipelineCfg, "icewind", "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("resolveCampaignConfigPath() error = %v", err)
|
||||||
|
}
|
||||||
|
want := filepath.Join(dir, "icewind", "campaign.yml")
|
||||||
|
if got != filepath.Clean(want) {
|
||||||
|
t.Fatalf("path = %q, want %q", got, filepath.Clean(want))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveCampaignConfigPathUsesDefaultCampaignID(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
pipelineCfg := &config.PipelineConfig{}
|
||||||
|
pipelineCfg.Campaigns.Root = dir
|
||||||
|
pipelineCfg.Campaigns.DefaultCampaignID = "dilfs"
|
||||||
|
|
||||||
|
got, err := resolveCampaignConfigPath(pipelineCfg, "", "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("resolveCampaignConfigPath() error = %v", err)
|
||||||
|
}
|
||||||
|
want := filepath.Join(dir, "dilfs", "campaign.yml")
|
||||||
|
if got != filepath.Clean(want) {
|
||||||
|
t.Fatalf("path = %q, want %q", got, filepath.Clean(want))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveCampaignConfigPathRejectsCampaignIDAndFile(t *testing.T) {
|
||||||
|
_, err := resolveCampaignConfigPath(&config.PipelineConfig{}, "dilfs", filepath.Join(t.TempDir(), "campaign.yml"))
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error, got nil")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "mutually exclusive") {
|
||||||
|
t.Fatalf("error = %q, want mutual exclusion", err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveCampaignConfigPathRequiresCampaignSelection(t *testing.T) {
|
||||||
|
_, err := resolveCampaignConfigPath(&config.PipelineConfig{}, "", "")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error, got nil")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "no campaign selected") {
|
||||||
|
t.Fatalf("error = %q, want missing selection guidance", err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveCampaignConfigPathRejectsPathLikeCampaignID(t *testing.T) {
|
||||||
|
pipelineCfg := &config.PipelineConfig{}
|
||||||
|
pipelineCfg.Campaigns.Root = t.TempDir()
|
||||||
|
|
||||||
|
_, err := resolveCampaignConfigPath(pipelineCfg, "../icewind", "")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error, got nil")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "single path segment") {
|
||||||
|
t.Fatalf("error = %q, want path segment guidance", err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
347
internal/app/clean.go
Normal file
347
internal/app/clean.go
Normal file
@@ -0,0 +1,347 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Clean removes local workspace/spool state while preserving durable cache
|
||||||
|
// state unless cache cleanup is explicitly requested.
|
||||||
|
func Clean(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
positionalSessionID, args := pullLeadingSessionID(args)
|
||||||
|
fs := flag.NewFlagSet("clean", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(io.Discard)
|
||||||
|
var flags commonConfigFlags
|
||||||
|
var all bool
|
||||||
|
var dryRun bool
|
||||||
|
var clearCache bool
|
||||||
|
addCommonConfigFlags(fs, &flags)
|
||||||
|
fs.BoolVar(&all, "all", false, "clean all local session work/spool state")
|
||||||
|
fs.BoolVar(&dryRun, "dry-run", false, "print cleanup targets without deleting")
|
||||||
|
fs.BoolVar(&clearCache, "clear-cache", false, "also clear durable S3 audio cache entries")
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return fmt.Errorf("clean: invalid flags: %w", err)
|
||||||
|
}
|
||||||
|
if positionalSessionID == "" {
|
||||||
|
if err := applyParsedSessionIDArg("clean", fs, &flags.sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if fs.NArg() != 0 {
|
||||||
|
return fmt.Errorf("clean: unexpected positional arguments")
|
||||||
|
}
|
||||||
|
if err := applyPositionalSessionID("clean", positionalSessionID, &flags.sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if all {
|
||||||
|
return cleanAllLocal(flags, dryRun, clearCache, out)
|
||||||
|
}
|
||||||
|
return cleanSession(ctx, flags, dryRun, clearCache, out)
|
||||||
|
}
|
||||||
|
|
||||||
|
func cleanSession(ctx context.Context, flags commonConfigFlags, dryRun, clearCache bool, out io.Writer) error {
|
||||||
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
|
return fmt.Errorf("clean: session_id is required unless --all is set")
|
||||||
|
}
|
||||||
|
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("clean: %w", err)
|
||||||
|
}
|
||||||
|
if cfg == nil || cfg.Pipeline == nil || cfg.Session == nil {
|
||||||
|
return fmt.Errorf("clean: resolved pipeline and session config are required")
|
||||||
|
}
|
||||||
|
campaign := strings.TrimSpace(cfg.Session.Campaign)
|
||||||
|
sessionID := strings.TrimSpace(cfg.Session.SessionID)
|
||||||
|
if campaign == "" || sessionID == "" {
|
||||||
|
return fmt.Errorf("clean: campaign and session_id are required")
|
||||||
|
}
|
||||||
|
|
||||||
|
if dryRun {
|
||||||
|
fmt.Fprintf(out, "Clean plan for %s/%s\n", campaign, sessionID)
|
||||||
|
} else {
|
||||||
|
fmt.Fprintf(out, "Cleaned %s/%s\n", campaign, sessionID)
|
||||||
|
}
|
||||||
|
|
||||||
|
workDir := artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, campaign, sessionID)
|
||||||
|
spoolDir := artifacts.SessionSpoolDir(cfg.Pipeline.Spool.Root, campaign, sessionID)
|
||||||
|
if err := reportCleanScopedDir(out, cfg.Pipeline.Workspace.Root, workDir, "clean.workspace.session", dryRun); err != nil {
|
||||||
|
return fmt.Errorf("clean: %w", err)
|
||||||
|
}
|
||||||
|
if err := reportCleanScopedDir(out, cfg.Pipeline.Spool.Root, spoolDir, "clean.spool.session", dryRun); err != nil {
|
||||||
|
return fmt.Errorf("clean: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if clearCache {
|
||||||
|
if err := cleanSessionAudioCache(ctx, cfg, dryRun, out); err != nil {
|
||||||
|
return fmt.Errorf("clean: %w", err)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
fmt.Fprintln(out, "Cache: preserved")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func cleanAllLocal(flags commonConfigFlags, dryRun, clearCache bool, out io.Writer) error {
|
||||||
|
if strings.TrimSpace(flags.campaignPath) != "" ||
|
||||||
|
strings.TrimSpace(flags.campaignFilePath) != "" ||
|
||||||
|
strings.TrimSpace(flags.sessionPath) != "" ||
|
||||||
|
strings.TrimSpace(flags.sessionID) != "" ||
|
||||||
|
strings.TrimSpace(flags.previousSessionID) != "" {
|
||||||
|
return fmt.Errorf("clean: --all cannot be combined with --campaign, --campaign-file, --session, a session_id, or --previous-session-id")
|
||||||
|
}
|
||||||
|
resolvedPipelinePath, err := resolvePipelineConfigPath(flags.pipelinePath)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("clean: %w", err)
|
||||||
|
}
|
||||||
|
pipelineCfg, err := config.LoadPipeline(resolvedPipelinePath)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("clean: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if dryRun {
|
||||||
|
fmt.Fprintln(out, "Clean plan for all local sessions")
|
||||||
|
} else {
|
||||||
|
fmt.Fprintln(out, "Cleaned all local sessions")
|
||||||
|
}
|
||||||
|
|
||||||
|
workRoot := filepath.Join(pipelineCfg.Workspace.Root, config.PathWorkDirSegment)
|
||||||
|
if err := reportCleanScopedDir(out, pipelineCfg.Workspace.Root, workRoot, "clean.workspace.all", dryRun); err != nil {
|
||||||
|
return fmt.Errorf("clean: %w", err)
|
||||||
|
}
|
||||||
|
if err := reportCleanRootChildren(out, pipelineCfg.Spool.Root, "clean.spool.all", dryRun); err != nil {
|
||||||
|
return fmt.Errorf("clean: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if clearCache {
|
||||||
|
if err := cleanAllAudioCache(pipelineCfg, dryRun, out); err != nil {
|
||||||
|
return fmt.Errorf("clean: %w", err)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
fmt.Fprintln(out, "Cache: preserved")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func reportCleanScopedDir(out io.Writer, root, target, policy string, dryRun bool) error {
|
||||||
|
dir, err := validateScopedDir(root, target, policy)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if dryRun {
|
||||||
|
if dir.Exists {
|
||||||
|
fmt.Fprintf(out, "Would delete: %s\n", dir.TargetAbs)
|
||||||
|
} else {
|
||||||
|
fmt.Fprintf(out, "Would skip missing: %s\n", dir.TargetAbs)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if !dir.Exists {
|
||||||
|
fmt.Fprintf(out, "Missing: %s\n", dir.TargetAbs)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := os.RemoveAll(dir.TargetAbs); err != nil {
|
||||||
|
return fmt.Errorf("cleanup policy %s: remove %q: %w", policy, dir.TargetAbs, err)
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "Deleted: %s\n", dir.TargetAbs)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func reportCleanRootChildren(out io.Writer, root, policy string, dryRun bool) error {
|
||||||
|
rootAbs, entries, err := cleanableRootChildren(root, policy)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if len(entries) == 0 {
|
||||||
|
if dryRun {
|
||||||
|
fmt.Fprintf(out, "Would skip empty: %s\n", rootAbs)
|
||||||
|
} else {
|
||||||
|
fmt.Fprintf(out, "Empty: %s\n", rootAbs)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
for _, entry := range entries {
|
||||||
|
if dryRun {
|
||||||
|
fmt.Fprintf(out, "Would delete: %s\n", entry)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := os.RemoveAll(entry); err != nil {
|
||||||
|
return fmt.Errorf("cleanup policy %s: remove %q: %w", policy, entry, err)
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "Deleted: %s\n", entry)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func cleanableRootChildren(root, policy string) (string, []string, error) {
|
||||||
|
cleanRoot := strings.TrimSpace(root)
|
||||||
|
if cleanRoot == "" {
|
||||||
|
return "", nil, fmt.Errorf("cleanup policy %s: root path is required", policy)
|
||||||
|
}
|
||||||
|
rootAbs, err := filepath.Abs(cleanRoot)
|
||||||
|
if err != nil {
|
||||||
|
return "", nil, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
||||||
|
}
|
||||||
|
info, err := os.Lstat(rootAbs)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return rootAbs, nil, nil
|
||||||
|
}
|
||||||
|
return "", nil, fmt.Errorf("cleanup policy %s: stat root %q: %w", policy, rootAbs, err)
|
||||||
|
}
|
||||||
|
if info.Mode()&os.ModeSymlink != 0 {
|
||||||
|
return "", nil, fmt.Errorf("cleanup policy %s: refusing to clean symlink root %q", policy, rootAbs)
|
||||||
|
}
|
||||||
|
if !info.IsDir() {
|
||||||
|
return "", nil, fmt.Errorf("cleanup policy %s: root %q is not a directory", policy, rootAbs)
|
||||||
|
}
|
||||||
|
entries, err := os.ReadDir(rootAbs)
|
||||||
|
if err != nil {
|
||||||
|
return "", nil, fmt.Errorf("cleanup policy %s: read root %q: %w", policy, rootAbs, err)
|
||||||
|
}
|
||||||
|
out := make([]string, 0, len(entries))
|
||||||
|
for _, entry := range entries {
|
||||||
|
path := filepath.Join(rootAbs, entry.Name())
|
||||||
|
info, err := os.Lstat(path)
|
||||||
|
if err != nil {
|
||||||
|
return "", nil, fmt.Errorf("cleanup policy %s: stat child %q: %w", policy, path, err)
|
||||||
|
}
|
||||||
|
if info.Mode()&os.ModeSymlink != 0 {
|
||||||
|
return "", nil, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, path)
|
||||||
|
}
|
||||||
|
out = append(out, path)
|
||||||
|
}
|
||||||
|
return rootAbs, out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func cleanSessionAudioCache(ctx context.Context, cfg *config.Config, dryRun bool, out io.Writer) error {
|
||||||
|
if cfg.Session.Inputs.AudioS3 == nil {
|
||||||
|
fmt.Fprintln(out, "Cache: skipped (session does not use audio_s3)")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Storage.S3 == nil || strings.TrimSpace(cfg.Pipeline.Storage.S3.Bucket) == "" {
|
||||||
|
return fmt.Errorf("clear cache requires pipeline.storage.s3.bucket")
|
||||||
|
}
|
||||||
|
store, err := newCommandObjectStore(ctx, cfg, nil)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("initialize object store for cache cleanup: %w", err)
|
||||||
|
}
|
||||||
|
sessionPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
audioPrefix := artifacts.S3AudioPrefix(sessionPrefix, cfg.Session.Inputs.AudioS3.Prefix)
|
||||||
|
objects, err := store.List(ctx, audioPrefix)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("list s3 audio objects under %q: %w", audioPrefix, err)
|
||||||
|
}
|
||||||
|
count := 0
|
||||||
|
for _, obj := range objects {
|
||||||
|
key := strings.TrimSpace(obj.Key)
|
||||||
|
if key == "" || strings.HasSuffix(key, "/") || !cleanIsFlac(key) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
cachePath, err := artifacts.S3AudioCachePath(cfg.Pipeline.Cache.Root, cfg.Pipeline.Storage.S3.Bucket, key)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
deleted, err := reportCleanScopedFile(out, cfg.Pipeline.Cache.Root, cachePath, "clean.cache.session", dryRun)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if deleted {
|
||||||
|
count++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if count == 0 {
|
||||||
|
fmt.Fprintf(out, "Cache: no cached S3 audio files found for %s\n", audioPrefix)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func cleanAllAudioCache(cfg *config.PipelineConfig, dryRun bool, out io.Writer) error {
|
||||||
|
if cfg.Storage.S3 == nil || strings.TrimSpace(cfg.Storage.S3.Bucket) == "" {
|
||||||
|
return fmt.Errorf("clear cache requires pipeline.storage.s3.bucket")
|
||||||
|
}
|
||||||
|
namespaceDir, err := artifacts.S3AudioCacheNamespaceDir(cfg.Cache.Root, cfg.Storage.S3.Bucket, cfg.Storage.S3.RootPrefix)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return reportCleanScopedDir(out, cfg.Cache.Root, namespaceDir, "clean.cache.all", dryRun)
|
||||||
|
}
|
||||||
|
|
||||||
|
func reportCleanScopedFile(out io.Writer, root, target, policy string, dryRun bool) (bool, error) {
|
||||||
|
file, err := validateScopedFile(root, target, policy)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if dryRun {
|
||||||
|
if file.Exists {
|
||||||
|
fmt.Fprintf(out, "Would delete cache file: %s\n", file.TargetAbs)
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "Would skip missing cache file: %s\n", file.TargetAbs)
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
if !file.Exists {
|
||||||
|
fmt.Fprintf(out, "Missing cache file: %s\n", file.TargetAbs)
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
if err := os.Remove(file.TargetAbs); err != nil {
|
||||||
|
return false, fmt.Errorf("cleanup policy %s: remove %q: %w", policy, file.TargetAbs, err)
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "Deleted cache file: %s\n", file.TargetAbs)
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateScopedFile(root, target, policy string) (scopedDir, error) {
|
||||||
|
cleanRoot := strings.TrimSpace(root)
|
||||||
|
cleanTarget := strings.TrimSpace(target)
|
||||||
|
if cleanRoot == "" {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: root path is required", policy)
|
||||||
|
}
|
||||||
|
if cleanTarget == "" {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: target path is required", policy)
|
||||||
|
}
|
||||||
|
rootAbs, err := filepath.Abs(cleanRoot)
|
||||||
|
if err != nil {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
||||||
|
}
|
||||||
|
targetAbs, err := filepath.Abs(cleanTarget)
|
||||||
|
if err != nil {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
|
||||||
|
}
|
||||||
|
rel, err := filepath.Rel(rootAbs, targetAbs)
|
||||||
|
if err != nil {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
|
||||||
|
}
|
||||||
|
if rel == "." {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
|
||||||
|
}
|
||||||
|
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
|
||||||
|
}
|
||||||
|
info, err := os.Lstat(targetAbs)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: false}, nil
|
||||||
|
}
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
|
||||||
|
}
|
||||||
|
if info.Mode()&os.ModeSymlink != 0 {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
|
||||||
|
}
|
||||||
|
if info.IsDir() {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is a directory", policy, targetAbs)
|
||||||
|
}
|
||||||
|
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func cleanIsFlac(path string) bool {
|
||||||
|
return strings.EqualFold(filepath.Ext(path), ".flac")
|
||||||
|
}
|
||||||
255
internal/app/clean_test.go
Normal file
255
internal/app/clean_test.go
Normal file
@@ -0,0 +1,255 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestExecuteCleanSessionDeletesWorkAndSpoolButPreservesCache(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
workDir := artifacts.SessionWorkDirForCampaign(workspaceRoot, "sample-campaign", "2026-05-03")
|
||||||
|
spoolDir := artifacts.SessionSpoolDir(filepath.Join(workspaceRoot, "spool"), "sample-campaign", "2026-05-03")
|
||||||
|
cachePath, err := artifacts.S3AudioCachePath(filepath.Join(workspaceRoot, "cache"), "test-bucket", "dnd/campaigns/sample-campaign/sessions/2026-05-03/audio/alice.flac")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("S3AudioCachePath() error = %v", err)
|
||||||
|
}
|
||||||
|
mustWriteTestFile(t, filepath.Join(workDir, "manifest.json"), "{}")
|
||||||
|
mustWriteTestFile(t, filepath.Join(spoolDir, "run-1", "audio", "alice.flac"), "audio")
|
||||||
|
mustWriteTestFile(t, cachePath, "cached-audio")
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
cleanAssertMissing(t, workDir)
|
||||||
|
cleanAssertMissing(t, spoolDir)
|
||||||
|
cleanAssertExists(t, cachePath)
|
||||||
|
if !strings.Contains(stdout.String(), "Cache: preserved") {
|
||||||
|
t.Fatalf("stdout = %q, want cache preserved", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteCleanSessionDryRunDeletesNothing(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
workDir := artifacts.SessionWorkDirForCampaign(workspaceRoot, "sample-campaign", "2026-05-03")
|
||||||
|
spoolDir := artifacts.SessionSpoolDir(filepath.Join(workspaceRoot, "spool"), "sample-campaign", "2026-05-03")
|
||||||
|
mustWriteTestFile(t, filepath.Join(workDir, "manifest.json"), "{}")
|
||||||
|
mustWriteTestFile(t, filepath.Join(spoolDir, "run-1", "audio", "alice.flac"), "audio")
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--dry-run"}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
cleanAssertExists(t, workDir)
|
||||||
|
cleanAssertExists(t, spoolDir)
|
||||||
|
if !strings.Contains(stdout.String(), "Would delete:") {
|
||||||
|
t.Fatalf("stdout = %q, want dry-run delete plan", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteCleanMissingSessionPathsSucceeds(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "Missing:") {
|
||||||
|
t.Fatalf("stdout = %q, want missing path output", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteCleanSessionClearCacheRemovesOnlyS3AudioCache(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
if err := os.WriteFile(sessionPath, []byte(`session_id: 2026-05-03
|
||||||
|
inputs:
|
||||||
|
audio_s3:
|
||||||
|
prefix: audio/
|
||||||
|
`), 0o644); err != nil {
|
||||||
|
t.Fatalf("write session: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
audioKey := "dnd/campaigns/sample-campaign/sessions/2026-05-03/audio/alice.flac"
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: audioKey, Data: []byte("audio")})
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
cacheRoot := filepath.Join(workspaceRoot, "cache")
|
||||||
|
cachePath, err := artifacts.S3AudioCachePath(cacheRoot, "test-bucket", audioKey)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("S3AudioCachePath() error = %v", err)
|
||||||
|
}
|
||||||
|
otherCachePath, err := artifacts.S3AudioCachePath(cacheRoot, "test-bucket", "dnd/campaigns/other/sessions/2026-05-03/audio/bob.flac")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("S3AudioCachePath() error = %v", err)
|
||||||
|
}
|
||||||
|
mustWriteTestFile(t, cachePath, "cached-audio")
|
||||||
|
mustWriteTestFile(t, otherCachePath, "other-audio")
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--clear-cache"}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
cleanAssertMissing(t, cachePath)
|
||||||
|
cleanAssertExists(t, otherCachePath)
|
||||||
|
if storeInitCalls != 1 {
|
||||||
|
t.Fatalf("object store init calls = %d, want 1", storeInitCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteCleanLocalAudioClearCacheIsNoop(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--clear-cache"}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "Cache: skipped (session does not use audio_s3)") {
|
||||||
|
t.Fatalf("stdout = %q, want local audio cache no-op", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteCleanAllDeletesWorkAndSpoolContentsButPreservesCache(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, _, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
workRoot := filepath.Join(workspaceRoot, "work")
|
||||||
|
spoolRoot := filepath.Join(workspaceRoot, "spool")
|
||||||
|
cachePath := filepath.Join(workspaceRoot, "cache", "keep.txt")
|
||||||
|
mustWriteTestFile(t, filepath.Join(workRoot, "sample-campaign", "2026-05-03", "manifest.json"), "{}")
|
||||||
|
mustWriteTestFile(t, filepath.Join(spoolRoot, "sample-campaign", "2026-05-03", "run-1", "audio", "alice.flac"), "audio")
|
||||||
|
mustWriteTestFile(t, cachePath, "cache")
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"clean", "--config", pipelinePath, "--all"}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
cleanAssertMissing(t, workRoot)
|
||||||
|
cleanAssertExists(t, spoolRoot)
|
||||||
|
cleanAssertMissing(t, filepath.Join(spoolRoot, "sample-campaign"))
|
||||||
|
cleanAssertExists(t, cachePath)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteCleanAllClearCacheRemovesS3AudioNamespaceOnly(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, _, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
cacheRoot := filepath.Join(workspaceRoot, "cache")
|
||||||
|
audioCachePath, err := artifacts.S3AudioCachePath(cacheRoot, "test-bucket", "dnd/campaigns/sample-campaign/sessions/2026-05-03/audio/alice.flac")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("S3AudioCachePath() error = %v", err)
|
||||||
|
}
|
||||||
|
otherCachePath, err := artifacts.S3AudioCachePath(cacheRoot, "test-bucket", "other-root/campaigns/sample-campaign/sessions/2026-05-03/audio/alice.flac")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("S3AudioCachePath() error = %v", err)
|
||||||
|
}
|
||||||
|
mustWriteTestFile(t, audioCachePath, "cached-audio")
|
||||||
|
mustWriteTestFile(t, otherCachePath, "other-cache")
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"clean", "--config", pipelinePath, "--all", "--clear-cache"}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
cleanAssertMissing(t, audioCachePath)
|
||||||
|
cleanAssertExists(t, otherCachePath)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteCleanAllRejectsSessionScopedFlags(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--all"}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "--all cannot be combined") {
|
||||||
|
t.Fatalf("stderr = %q, want --all conflict", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCleanRequiresSessionID(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"clean"}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "session_id is required unless --all is set") {
|
||||||
|
t.Fatalf("stderr = %q, want missing session-id", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCleanRejectsUnsafeTargets(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
outside := t.TempDir()
|
||||||
|
if err := reportCleanScopedDir(&bytes.Buffer{}, root, filepath.Join(outside, "target"), "test.outside", false); err == nil {
|
||||||
|
t.Fatal("outside target error = nil, want error")
|
||||||
|
}
|
||||||
|
if err := reportCleanScopedDir(&bytes.Buffer{}, root, root, "test.root", false); err == nil {
|
||||||
|
t.Fatal("root target error = nil, want error")
|
||||||
|
}
|
||||||
|
filePath := filepath.Join(root, "file.txt")
|
||||||
|
mustWriteTestFile(t, filePath, "file")
|
||||||
|
if err := reportCleanScopedDir(&bytes.Buffer{}, root, filePath, "test.file", false); err == nil {
|
||||||
|
t.Fatal("file target error = nil, want error")
|
||||||
|
}
|
||||||
|
symlinkPath := filepath.Join(root, "link")
|
||||||
|
if err := os.Symlink(filepath.Join(root, "missing"), symlinkPath); err != nil {
|
||||||
|
t.Fatalf("Symlink() error = %v", err)
|
||||||
|
}
|
||||||
|
if err := reportCleanScopedDir(&bytes.Buffer{}, root, symlinkPath, "test.symlink", false); err == nil {
|
||||||
|
t.Fatal("symlink target error = nil, want error")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestClearIsNotCommandAlias(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"clear"}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), `unknown command: "clear"`) {
|
||||||
|
t.Fatalf("stderr = %q, want unknown clear command", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func cleanAssertExists(t *testing.T, path string) {
|
||||||
|
t.Helper()
|
||||||
|
if _, err := os.Stat(path); err != nil {
|
||||||
|
t.Fatalf("expected %q to exist: %v", path, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func cleanAssertMissing(t *testing.T, path string) {
|
||||||
|
t.Helper()
|
||||||
|
if _, err := os.Stat(path); !os.IsNotExist(err) {
|
||||||
|
t.Fatalf("expected %q to be missing, stat err=%v", path, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -7,7 +7,7 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
)
|
)
|
||||||
|
|
||||||
var supportedCommands = []string{"run", "plan", "status", "resume", "run-stage", "restore"}
|
var supportedCommands = []string{"run", "run-stage", "resume", "analyze", "publish", "clean", "session"}
|
||||||
|
|
||||||
// Execute dispatches CLI commands and returns a process exit code.
|
// Execute dispatches CLI commands and returns a process exit code.
|
||||||
func Execute(args []string, stdout, stderr io.Writer) int {
|
func Execute(args []string, stdout, stderr io.Writer) int {
|
||||||
@@ -24,16 +24,18 @@ func Execute(args []string, stdout, stderr io.Writer) int {
|
|||||||
switch cmd {
|
switch cmd {
|
||||||
case "run":
|
case "run":
|
||||||
err = Run(ctx, cmdArgs, stdout)
|
err = Run(ctx, cmdArgs, stdout)
|
||||||
case "plan":
|
|
||||||
err = Plan(ctx, cmdArgs, stdout)
|
|
||||||
case "status":
|
|
||||||
err = Status(ctx, cmdArgs, stdout)
|
|
||||||
case "resume":
|
case "resume":
|
||||||
err = Resume(ctx, cmdArgs, stdout)
|
err = Resume(ctx, cmdArgs, stdout)
|
||||||
case "run-stage":
|
case "run-stage":
|
||||||
err = RunStage(ctx, cmdArgs, stdout)
|
err = RunStage(ctx, cmdArgs, stdout)
|
||||||
case "restore":
|
case "analyze":
|
||||||
err = Restore(ctx, cmdArgs, stdout)
|
err = Analyze(ctx, cmdArgs, stdout)
|
||||||
|
case "publish":
|
||||||
|
err = Publish(ctx, cmdArgs, stdout)
|
||||||
|
case "session":
|
||||||
|
err = Session(ctx, cmdArgs, stdout)
|
||||||
|
case "clean":
|
||||||
|
err = Clean(ctx, cmdArgs, stdout)
|
||||||
default:
|
default:
|
||||||
fmt.Fprintf(stderr, "unknown command: %q\n\n", cmd)
|
fmt.Fprintf(stderr, "unknown command: %q\n\n", cmd)
|
||||||
printUsage(stderr)
|
printUsage(stderr)
|
||||||
|
|||||||
@@ -24,19 +24,18 @@ func TestExecuteValidCommands(t *testing.T) {
|
|||||||
}))
|
}))
|
||||||
defer srv.Close()
|
defer srv.Close()
|
||||||
|
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||||
manifestPath := writeManifestPathForExecute(t)
|
|
||||||
|
|
||||||
cases := []struct {
|
cases := []struct {
|
||||||
name string
|
name string
|
||||||
args []string
|
args []string
|
||||||
wantOut string
|
wantOut string
|
||||||
}{
|
}{
|
||||||
{name: "run", args: []string{"run", "--config", pipelinePath, "--session", sessionPath}, wantOut: "narratio run: session 2026-05-03; executed=9 skipped=0; manifest="},
|
{name: "run", args: []string{"run", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "narratio run: session 2026-05-03; executed=9 skipped=0; manifest="},
|
||||||
{name: "plan", args: []string{"plan", "--config", pipelinePath, "--session", sessionPath}, wantOut: "prepare: skip\ntranscribe: skip\nmerge: skip\npolish: skip\nnormalize: skip\ntrim: skip\nanalyze: skip\narchive: skip\nnotify: skip"},
|
{name: "session plan", args: []string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "prepare: skip\ntranscribe: skip\nmerge: skip\npolish: skip\nnormalize: skip\ntrim: skip\nanalyze: skip\npublish: skip\nnotify: skip"},
|
||||||
{name: "status", args: []string{"status", "--manifest", manifestPath}, wantOut: "session_id: 2026-05-03"},
|
{name: "session status", args: []string{"session", "status", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "Session: 2026-05-03"},
|
||||||
{name: "resume", args: []string{"resume", "--config", pipelinePath, "--session", sessionPath}, wantOut: "narratio resume: session 2026-05-03 has no remaining stages"},
|
{name: "resume", args: []string{"resume", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "narratio resume: session 2026-05-03 has no remaining stages"},
|
||||||
{name: "run-stage", args: []string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "polish"}, wantOut: "narratio run-stage: stage=polish executed=0 skipped=1 force=false; manifest="},
|
{name: "run-stage", args: []string{"run-stage", "polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "narratio run-stage: stage=polish executed=0 skipped=1 force=false; manifest="},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tc := range cases {
|
for _, tc := range cases {
|
||||||
@@ -64,13 +63,13 @@ func TestExecuteMissingRequiredFlags(t *testing.T) {
|
|||||||
args []string
|
args []string
|
||||||
want string
|
want string
|
||||||
}{
|
}{
|
||||||
{name: "run missing flags", args: []string{"run"}, want: "run: no pipeline config path provided and no default pipeline config found; searched:"},
|
{name: "run missing session", args: []string{"run"}, want: "run: session_id is required"},
|
||||||
{name: "plan missing flags", args: []string{"plan"}, want: "plan: no pipeline config path provided and no default pipeline config found; searched:"},
|
{name: "plan old top-level removed", args: []string{"plan"}, want: `unknown command: "plan"`},
|
||||||
{name: "status missing flags", args: []string{"status"}, want: "status: --manifest is required"},
|
{name: "status old top-level removed", args: []string{"status"}, want: `unknown command: "status"`},
|
||||||
{name: "resume missing flags", args: []string{"resume"}, want: "resume: no pipeline config path provided and no default pipeline config found; searched:"},
|
{name: "resume missing session", args: []string{"resume"}, want: "resume: session_id is required"},
|
||||||
{name: "run-stage missing name", args: []string{"run-stage", "--config", "a", "--session", "b"}, want: "run-stage: expected exactly one stage name"},
|
{name: "run-stage missing name", args: []string{"run-stage", "--config", "a", "--session", "b"}, want: "run-stage: expected stage name and session_id"},
|
||||||
{name: "run-stage missing config flags", args: []string{"run-stage", "polish"}, want: "run-stage: no pipeline config path provided and no default pipeline config found; searched:"},
|
{name: "run-stage missing session", args: []string{"run-stage", "polish"}, want: "run-stage: expected stage name and session_id"},
|
||||||
{name: "run missing config uses defaults", args: []string{"run", "--session", "session.yml"}, want: "run: no pipeline config path provided and no default pipeline config found; searched:"},
|
{name: "run missing config uses defaults", args: []string{"run", "2026-05-03", "--session", "session.yml"}, want: "run: no pipeline config path provided and no default pipeline config found; searched:"},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tc := range cases {
|
for _, tc := range cases {
|
||||||
@@ -94,12 +93,12 @@ func TestExecuteMissingRequiredFlags(t *testing.T) {
|
|||||||
|
|
||||||
func TestExecuteRunStageUnknownFails(t *testing.T) {
|
func TestExecuteRunStageUnknownFails(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, "https://example.com/transcribe")
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, "https://example.com/transcribe")
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
|
|
||||||
code := Execute([]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "unknown"}, &stdout, &stderr)
|
code := Execute([]string{"run-stage", "unknown", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
if code == 0 {
|
if code == 0 {
|
||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
@@ -108,16 +107,32 @@ func TestExecuteRunStageUnknownFails(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteRunStageNormalizeIsAccepted(t *testing.T) {
|
func TestExecuteRunStageArchiveAliasFails(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, "https://example.com/transcribe")
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, "https://example.com/transcribe")
|
||||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "processed.json"), `{"segments":[{"id":1}]}`)
|
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
|
|
||||||
code := Execute([]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "normalize"}, &stdout, &stderr)
|
code := Execute([]string{"run-stage", "archive", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), `unknown stage "archive"`) {
|
||||||
|
t.Fatalf("stderr = %q, want unknown stage alias error", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRunStageNormalizeIsAccepted(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, "https://example.com/transcribe")
|
||||||
|
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||||
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "polished.json"), `{"segments":[{"id":1}]}`)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
|
||||||
|
code := Execute([]string{"run-stage", "normalize", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
@@ -136,19 +151,19 @@ func TestExecuteRunStageTranscribeUsesConfiguredWhisperXServer(t *testing.T) {
|
|||||||
}))
|
}))
|
||||||
defer srv.Close()
|
defer srv.Close()
|
||||||
|
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
|
|
||||||
code := Execute([]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "prepare"}, &stdout, &stderr)
|
code := Execute([]string{"run-stage", "prepare", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("prepare exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("prepare exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
stdout.Reset()
|
stdout.Reset()
|
||||||
stderr.Reset()
|
stderr.Reset()
|
||||||
|
|
||||||
code = Execute([]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "--force", "transcribe"}, &stdout, &stderr)
|
code = Execute([]string{"run-stage", "transcribe", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("transcribe exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("transcribe exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
@@ -188,6 +203,7 @@ func TestExecuteRunStagePolishLoadsCredentialFromSecretsDir(t *testing.T) {
|
|||||||
t.Setenv("GO_WANT_APP_AUDITA_HELPER", "1")
|
t.Setenv("GO_WANT_APP_AUDITA_HELPER", "1")
|
||||||
|
|
||||||
pipelinePath := filepath.Join(configDir, "pipeline.yml")
|
pipelinePath := filepath.Join(configDir, "pipeline.yml")
|
||||||
|
campaignPath := writeAppTestCampaignConfig(t, configDir)
|
||||||
sessionPath := filepath.Join(configDir, "session.yml")
|
sessionPath := filepath.Join(configDir, "session.yml")
|
||||||
pipelineYAML := `workspace:
|
pipelineYAML := `workspace:
|
||||||
root: ` + workspaceRoot + `
|
root: ` + workspaceRoot + `
|
||||||
@@ -202,8 +218,6 @@ seriatim:
|
|||||||
audita:
|
audita:
|
||||||
binary: ` + auditaBinary + `
|
binary: ` + auditaBinary + `
|
||||||
llm_api_key_env: OPENROUTER_API_KEY
|
llm_api_key_env: OPENROUTER_API_KEY
|
||||||
analyzer:
|
|
||||||
timeout: 20m
|
|
||||||
notification:
|
notification:
|
||||||
timeout: 10s
|
timeout: 10s
|
||||||
`
|
`
|
||||||
@@ -234,12 +248,12 @@ inputs:
|
|||||||
})
|
})
|
||||||
|
|
||||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", sessionID)
|
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", sessionID)
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "merged.json"), `{"schema":"seriatim-intermediate","segments":[]}`)
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "base.json"), `{"schema":"seriatim-intermediate","segments":[]}`)
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "[]\n")
|
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "[]\n")
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"run-stage", "--config", pipelinePath, "--session", sessionPath, "--force", "polish"}, &stdout, &stderr)
|
code := Execute([]string{"run-stage", "polish", sessionID, "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
@@ -252,6 +266,7 @@ func TestExecuteRunFailsWhenConfiguredSecretsDirMissing(t *testing.T) {
|
|||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
configDir := t.TempDir()
|
configDir := t.TempDir()
|
||||||
pipelinePath := filepath.Join(configDir, "pipeline.yml")
|
pipelinePath := filepath.Join(configDir, "pipeline.yml")
|
||||||
|
campaignPath := writeAppTestCampaignConfig(t, configDir)
|
||||||
sessionPath := filepath.Join(configDir, "session.yml")
|
sessionPath := filepath.Join(configDir, "session.yml")
|
||||||
|
|
||||||
pipelineYAML := `workspace:
|
pipelineYAML := `workspace:
|
||||||
@@ -266,8 +281,6 @@ seriatim:
|
|||||||
binary: seriatim
|
binary: seriatim
|
||||||
audita:
|
audita:
|
||||||
binary: audita
|
binary: audita
|
||||||
analyzer:
|
|
||||||
timeout: 20m
|
|
||||||
notification:
|
notification:
|
||||||
timeout: 10s
|
timeout: 10s
|
||||||
`
|
`
|
||||||
@@ -288,7 +301,7 @@ inputs:
|
|||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"run", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
code := Execute([]string{"run", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
if code == 0 {
|
if code == 0 {
|
||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
@@ -305,16 +318,17 @@ func TestExecuteUsesDefaultPipelineConfigPathWhenConfigFlagOmitted(t *testing.T)
|
|||||||
}))
|
}))
|
||||||
defer srv.Close()
|
defer srv.Close()
|
||||||
|
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||||
originalDefaults := append([]string(nil), config.DefaultPipelineConfigSearchPaths...)
|
originalDefaults := append([]string(nil), config.DefaultPipelineConfigSearchPaths...)
|
||||||
config.DefaultPipelineConfigSearchPaths = []string{pipelinePath}
|
config.DefaultPipelineConfigSearchPaths = []string{pipelinePath}
|
||||||
defer func() {
|
defer func() {
|
||||||
config.DefaultPipelineConfigSearchPaths = originalDefaults
|
config.DefaultPipelineConfigSearchPaths = originalDefaults
|
||||||
}()
|
}()
|
||||||
|
_ = campaignPath
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"run", "--session", sessionPath}, &stdout, &stderr)
|
code := Execute([]string{"run", "2026-05-03", "--session", sessionPath}, &stdout, &stderr)
|
||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
@@ -323,6 +337,86 @@ func TestExecuteUsesDefaultPipelineConfigPathWhenConfigFlagOmitted(t *testing.T)
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestExecuteMissingCampaignConfigReportsRegistryPath(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
if err := os.Remove(campaignPath); err != nil {
|
||||||
|
t.Fatalf("remove campaign config: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"run", "2026-05-03", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if stdout.Len() != 0 {
|
||||||
|
t.Fatalf("stdout = %q, want empty", stdout.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "load campaign config") {
|
||||||
|
t.Fatalf("stderr = %q, want campaign discovery failure", stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), filepath.ToSlash(filepath.Join("campaigns", "sample-campaign", "campaign.yml"))) {
|
||||||
|
t.Fatalf("stderr = %q, want campaign registry path", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteUsesPipelineDefaultCampaignID(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, _, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "status", "2026-05-03", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "Campaign: sample-campaign") {
|
||||||
|
t.Fatalf("stdout = %q, want default campaign", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteCampaignIDSelectsRegistryCampaign(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
campaignRoot := filepath.Dir(filepath.Dir(campaignPath))
|
||||||
|
otherDir := filepath.Join(campaignRoot, "icewind")
|
||||||
|
mustWriteTestFile(t, filepath.Join(otherDir, "campaign.yml"), `campaign_id: icewind
|
||||||
|
inputs:
|
||||||
|
speakers_file: ./speakers.yml
|
||||||
|
autocorrect_file: ./autocorrect.yml
|
||||||
|
glossary_file: ./glossary.yml
|
||||||
|
`)
|
||||||
|
mustWriteTestFile(t, filepath.Join(otherDir, "speakers.yml"), "match:\n - speaker: Alice\n match: [\"alice\"]\n")
|
||||||
|
mustWriteTestFile(t, filepath.Join(otherDir, "autocorrect.yml"), "[]\n")
|
||||||
|
mustWriteTestFile(t, filepath.Join(otherDir, "glossary.yml"), "[]\n")
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "status", "2026-05-03", "--config", pipelinePath, "--campaign", "icewind", "--session", sessionPath}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "Campaign: icewind") {
|
||||||
|
t.Fatalf("stdout = %q, want selected campaign", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRejectsCampaignIDAndCampaignFile(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "status", "2026-05-03", "--config", pipelinePath, "--campaign", "sample-campaign", "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "mutually exclusive") {
|
||||||
|
t.Fatalf("stderr = %q, want mutually exclusive error", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestExecuteInvalidCommand(t *testing.T) {
|
func TestExecuteInvalidCommand(t *testing.T) {
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
@@ -359,11 +453,14 @@ func TestExecuteMissingCommand(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func writeValidConfigFiles(t *testing.T, workspaceRoot string, transcribeURL ...string) (string, string) {
|
func writeValidConfigFiles(t *testing.T, workspaceRoot string, transcribeURL ...string) (string, string, string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
pipelinePath := filepath.Join(dir, "pipeline.yml")
|
pipelinePath := filepath.Join(dir, "pipeline.yml")
|
||||||
|
campaignRoot := filepath.Join(dir, "campaigns")
|
||||||
|
campaignDir := filepath.Join(campaignRoot, "sample-campaign")
|
||||||
|
campaignPath := filepath.Join(campaignDir, "campaign.yml")
|
||||||
sessionPath := filepath.Join(dir, "session.yml")
|
sessionPath := filepath.Join(dir, "session.yml")
|
||||||
url := "https://example.com/transcribe"
|
url := "https://example.com/transcribe"
|
||||||
if len(transcribeURL) > 0 && strings.TrimSpace(transcribeURL[0]) != "" {
|
if len(transcribeURL) > 0 && strings.TrimSpace(transcribeURL[0]) != "" {
|
||||||
@@ -377,11 +474,18 @@ func writeValidConfigFiles(t *testing.T, workspaceRoot string, transcribeURL ...
|
|||||||
|
|
||||||
pipelineYAML := `workspace:
|
pipelineYAML := `workspace:
|
||||||
root: ` + workspaceRoot + `
|
root: ` + workspaceRoot + `
|
||||||
|
campaigns:
|
||||||
|
root: ` + campaignRoot + `
|
||||||
|
default_campaign_id: sample-campaign
|
||||||
|
cache:
|
||||||
|
root: ` + filepath.Join(workspaceRoot, "cache") + `
|
||||||
|
spool:
|
||||||
|
root: ` + filepath.Join(workspaceRoot, "spool") + `
|
||||||
storage:
|
storage:
|
||||||
backend: s3
|
backend: s3
|
||||||
s3:
|
s3:
|
||||||
bucket: test-bucket
|
bucket: test-bucket
|
||||||
archive:
|
publish:
|
||||||
enabled: true
|
enabled: true
|
||||||
upload_run: false
|
upload_run: false
|
||||||
whisperx:
|
whisperx:
|
||||||
@@ -398,18 +502,16 @@ seriatim:
|
|||||||
report: true
|
report: true
|
||||||
audita:
|
audita:
|
||||||
binary: ` + auditaBinary + `
|
binary: ` + auditaBinary + `
|
||||||
analyzer:
|
|
||||||
timeout: 20m
|
|
||||||
artifacts:
|
|
||||||
output_dir: artifacts
|
|
||||||
notification:
|
notification:
|
||||||
timeout: 10s
|
timeout: 10s
|
||||||
`
|
`
|
||||||
|
|
||||||
sessionYAML := `session_id: 2026-05-03
|
sessionYAML := `session_id: 2026-05-03
|
||||||
campaign: sample-campaign
|
|
||||||
inputs:
|
inputs:
|
||||||
audio_dir: ./audio
|
audio_dir: ./audio
|
||||||
|
`
|
||||||
|
campaignYAML := `campaign_id: sample-campaign
|
||||||
|
inputs:
|
||||||
speakers_file: ./speakers.yml
|
speakers_file: ./speakers.yml
|
||||||
autocorrect_file: ./autocorrect.yml
|
autocorrect_file: ./autocorrect.yml
|
||||||
glossary_file: ./glossary.yml
|
glossary_file: ./glossary.yml
|
||||||
@@ -418,16 +520,37 @@ inputs:
|
|||||||
if err := os.WriteFile(pipelinePath, []byte(pipelineYAML), 0o644); err != nil {
|
if err := os.WriteFile(pipelinePath, []byte(pipelineYAML), 0o644); err != nil {
|
||||||
t.Fatalf("write pipeline config: %v", err)
|
t.Fatalf("write pipeline config: %v", err)
|
||||||
}
|
}
|
||||||
|
if err := os.MkdirAll(campaignDir, 0o755); err != nil {
|
||||||
|
t.Fatalf("create campaign dir: %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(campaignPath, []byte(campaignYAML), 0o644); err != nil {
|
||||||
|
t.Fatalf("write campaign config: %v", err)
|
||||||
|
}
|
||||||
if err := os.WriteFile(sessionPath, []byte(sessionYAML), 0o644); err != nil {
|
if err := os.WriteFile(sessionPath, []byte(sessionYAML), 0o644); err != nil {
|
||||||
t.Fatalf("write session config: %v", err)
|
t.Fatalf("write session config: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
mustWriteTestFile(t, filepath.Join(dir, "speakers.yml"), "match:\n - speaker: Alice\n match: [\"alice\"]\n")
|
mustWriteTestFile(t, filepath.Join(campaignDir, "speakers.yml"), "match:\n - speaker: Alice\n match: [\"alice\"]\n")
|
||||||
mustWriteTestFile(t, filepath.Join(dir, "autocorrect.yml"), "[]\n")
|
mustWriteTestFile(t, filepath.Join(campaignDir, "autocorrect.yml"), "[]\n")
|
||||||
mustWriteTestFile(t, filepath.Join(dir, "glossary.yml"), "[]\n")
|
mustWriteTestFile(t, filepath.Join(campaignDir, "glossary.yml"), "[]\n")
|
||||||
mustWriteTestFile(t, filepath.Join(dir, "audio", "alice.flac"), "audio-bytes")
|
mustWriteTestFile(t, filepath.Join(dir, "audio", "alice.flac"), "audio-bytes")
|
||||||
|
|
||||||
return pipelinePath, sessionPath
|
return pipelinePath, campaignPath, sessionPath
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeAppTestCampaignConfig(t *testing.T, dir string) string {
|
||||||
|
t.Helper()
|
||||||
|
campaignPath := filepath.Join(dir, "campaign.yml")
|
||||||
|
campaignYAML := `campaign_id: sample-campaign
|
||||||
|
inputs:
|
||||||
|
speakers_file: ./speakers.yml
|
||||||
|
autocorrect_file: ./autocorrect.yml
|
||||||
|
glossary_file: ./glossary.yml
|
||||||
|
`
|
||||||
|
if err := os.WriteFile(campaignPath, []byte(campaignYAML), 0o644); err != nil {
|
||||||
|
t.Fatalf("write campaign.yml: %v", err)
|
||||||
|
}
|
||||||
|
return campaignPath
|
||||||
}
|
}
|
||||||
|
|
||||||
func writeManifestPathForExecute(t *testing.T) string {
|
func writeManifestPathForExecute(t *testing.T) string {
|
||||||
|
|||||||
157
internal/app/config_loader.go
Normal file
157
internal/app/config_loader.go
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
type pipelineCampaignConfig struct {
|
||||||
|
PipelinePath string
|
||||||
|
CampaignPath string
|
||||||
|
Pipeline *config.PipelineConfig
|
||||||
|
Campaign *config.CampaignConfig
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadCommandConfig(ctx context.Context, pipelineFlag, campaignFlag, campaignFileFlag, sessionFlag string, sessionOpts config.SessionLoadOptions) (*config.Config, error) {
|
||||||
|
base, err := loadPipelineCampaignConfig(pipelineFlag, campaignFlag, campaignFileFlag)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
if explicitSession := strings.TrimSpace(sessionFlag); explicitSession != "" {
|
||||||
|
return config.LoadWithSessionOptions(base.PipelinePath, base.CampaignPath, explicitSession, sessionOpts)
|
||||||
|
}
|
||||||
|
|
||||||
|
discoveredSession, err := discoverSessionConfigPathWithCandidates(config.DefaultSessionConfigSearchPaths)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if discoveredSession.Path != "" {
|
||||||
|
return config.LoadWithSessionOptions(base.PipelinePath, base.CampaignPath, discoveredSession.Path, sessionOpts)
|
||||||
|
}
|
||||||
|
|
||||||
|
sessionID := strings.TrimSpace(sessionOpts.SessionID)
|
||||||
|
if sessionID == "" {
|
||||||
|
return nil, missingSessionConfigError(discoveredSession.Searched, "remote session loading requires a session_id")
|
||||||
|
}
|
||||||
|
|
||||||
|
sessionPrefix := artifacts.S3SessionPrefix(base.Pipeline.Storage.S3.RootPrefix, config.CampaignID(base.Campaign), sessionID)
|
||||||
|
remoteKey := artifacts.S3SessionConfigKey(sessionPrefix)
|
||||||
|
partialCfg := &config.Config{
|
||||||
|
Pipeline: base.Pipeline,
|
||||||
|
Campaign: base.Campaign,
|
||||||
|
PipelinePath: base.PipelinePath,
|
||||||
|
CampaignPath: base.CampaignPath,
|
||||||
|
}
|
||||||
|
store, err := newCommandObjectStore(ctx, partialCfg, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, missingSessionConfigError(discoveredSession.Searched, fmt.Sprintf("remote session %q unavailable: %v", remoteKey, err))
|
||||||
|
}
|
||||||
|
|
||||||
|
sessionInfo, err := findRemoteSessionConfig(ctx, store, sessionPrefix, remoteKey)
|
||||||
|
if err != nil {
|
||||||
|
return nil, missingSessionConfigError(discoveredSession.Searched, err.Error())
|
||||||
|
}
|
||||||
|
sessionTempPath, err := downloadRemoteSessionConfig(ctx, store, remoteKey)
|
||||||
|
if err != nil {
|
||||||
|
return nil, missingSessionConfigError(discoveredSession.Searched, fmt.Sprintf("remote session %q download failed: %v", remoteKey, err))
|
||||||
|
}
|
||||||
|
sessionBytes, err := os.ReadFile(sessionTempPath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read downloaded remote session %q: %w", sessionTempPath, err)
|
||||||
|
}
|
||||||
|
sessionCfg, err := config.LoadSessionBytesWithOptions("s3://"+s3BucketName(base.Pipeline)+"/"+remoteKey, sessionBytes, sessionOpts)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
return config.Resolve(
|
||||||
|
base.PipelinePath,
|
||||||
|
base.Pipeline,
|
||||||
|
base.CampaignPath,
|
||||||
|
base.Campaign,
|
||||||
|
sessionTempPath,
|
||||||
|
sessionCfg,
|
||||||
|
config.SessionSource{
|
||||||
|
Source: "session_config.s3",
|
||||||
|
LocalPath: sessionTempPath,
|
||||||
|
S3Bucket: s3BucketName(base.Pipeline),
|
||||||
|
S3Key: remoteKey,
|
||||||
|
S3Size: sessionInfo.Size,
|
||||||
|
S3ETag: sessionInfo.ETag,
|
||||||
|
SpoolPath: sessionTempPath,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadPipelineCampaignConfig(pipelineFlag, campaignFlag, campaignFileFlag string) (*pipelineCampaignConfig, error) {
|
||||||
|
resolvedPipelinePath, err := resolvePipelineConfigPath(pipelineFlag)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
pipelineCfg, err := config.LoadPipeline(resolvedPipelinePath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
resolvedCampaignPath, err := resolveCampaignConfigPath(pipelineCfg, campaignFlag, campaignFileFlag)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
campaignCfg, err := config.LoadCampaign(resolvedCampaignPath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if selectedID := strings.TrimSpace(campaignFlag); selectedID != "" && strings.TrimSpace(campaignFileFlag) == "" {
|
||||||
|
if got := config.CampaignID(campaignCfg); got != selectedID {
|
||||||
|
return nil, fmt.Errorf("campaign config %q invalid: campaign_id %q does not match selected campaign %q", resolvedCampaignPath, got, selectedID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &pipelineCampaignConfig{
|
||||||
|
PipelinePath: resolvedPipelinePath,
|
||||||
|
CampaignPath: resolvedCampaignPath,
|
||||||
|
Pipeline: pipelineCfg,
|
||||||
|
Campaign: campaignCfg,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func findRemoteSessionConfig(ctx context.Context, store storage.ObjectStore, sessionPrefix, remoteKey string) (storage.ObjectInfo, error) {
|
||||||
|
objects, err := store.List(ctx, sessionPrefix)
|
||||||
|
if err != nil {
|
||||||
|
return storage.ObjectInfo{}, fmt.Errorf("remote session %q list failed: %w", remoteKey, err)
|
||||||
|
}
|
||||||
|
for _, obj := range objects {
|
||||||
|
if obj.Key == remoteKey {
|
||||||
|
return obj, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return storage.ObjectInfo{}, fmt.Errorf("remote session %q not found", remoteKey)
|
||||||
|
}
|
||||||
|
|
||||||
|
func downloadRemoteSessionConfig(ctx context.Context, store storage.ObjectStore, remoteKey string) (string, error) {
|
||||||
|
f, err := os.CreateTemp("", "narratio-session-*.yml")
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("create temp file: %w", err)
|
||||||
|
}
|
||||||
|
path := f.Name()
|
||||||
|
if err := f.Close(); err != nil {
|
||||||
|
return "", fmt.Errorf("close temp file %q: %w", path, err)
|
||||||
|
}
|
||||||
|
if err := store.Download(ctx, remoteKey, path); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return filepath.Clean(path), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func s3BucketName(cfg *config.PipelineConfig) string {
|
||||||
|
if cfg == nil || cfg.Storage.S3 == nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return strings.TrimSpace(cfg.Storage.S3.Bucket)
|
||||||
|
}
|
||||||
21
internal/app/object_store.go
Normal file
21
internal/app/object_store.go
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
func newCommandObjectStore(ctx context.Context, cfg *config.Config, logger *slog.Logger) (storage.ObjectStore, error) {
|
||||||
|
if _, err := loadSecretsFromConfig(cfg, logger); err != nil {
|
||||||
|
return nil, fmt.Errorf("load secrets from files: %w", err)
|
||||||
|
}
|
||||||
|
store, err := newObjectStoreFromConfigFn(ctx, cfg)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("initialize object store backend: %w", err)
|
||||||
|
}
|
||||||
|
return store, nil
|
||||||
|
}
|
||||||
165
internal/app/object_store_test.go
Normal file
165
internal/app/object_store_test.go
Normal file
@@ -0,0 +1,165 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestNewCommandObjectStoreLoadsSecretsBeforeFactory(t *testing.T) {
|
||||||
|
accessKeyEnv := "NARRATIO_TEST_COMMAND_STORE_KEY_ID"
|
||||||
|
secretKeyEnv := "NARRATIO_TEST_COMMAND_STORE_SECRET"
|
||||||
|
restoreEnvAfterTest(t, accessKeyEnv, secretKeyEnv)
|
||||||
|
|
||||||
|
secretsDir := t.TempDir()
|
||||||
|
mustWriteSecretFile(t, filepath.Join(secretsDir, accessKeyEnv), "loaded-key-id\n")
|
||||||
|
mustWriteSecretFile(t, filepath.Join(secretsDir, secretKeyEnv), "loaded-secret\n")
|
||||||
|
|
||||||
|
cfg := commandObjectStoreTestConfig(secretsDir)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
called := false
|
||||||
|
origStoreFn := newObjectStoreFromConfigFn
|
||||||
|
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||||
|
called = true
|
||||||
|
if got := os.Getenv(accessKeyEnv); got != "loaded-key-id" {
|
||||||
|
return nil, errors.New("access key was not loaded before object store init")
|
||||||
|
}
|
||||||
|
if got := os.Getenv(secretKeyEnv); got != "loaded-secret" {
|
||||||
|
return nil, errors.New("secret key was not loaded before object store init")
|
||||||
|
}
|
||||||
|
return fake, nil
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
newObjectStoreFromConfigFn = origStoreFn
|
||||||
|
})
|
||||||
|
|
||||||
|
store, err := newCommandObjectStore(context.Background(), cfg, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("newCommandObjectStore() error = %v", err)
|
||||||
|
}
|
||||||
|
if store != fake {
|
||||||
|
t.Fatalf("store = %#v, want fake backend", store)
|
||||||
|
}
|
||||||
|
if !called {
|
||||||
|
t.Fatal("object store factory was not called")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewCommandObjectStorePreservesExistingEnv(t *testing.T) {
|
||||||
|
accessKeyEnv := "NARRATIO_TEST_COMMAND_STORE_EXISTING_KEY_ID"
|
||||||
|
secretKeyEnv := "NARRATIO_TEST_COMMAND_STORE_EXISTING_SECRET"
|
||||||
|
t.Setenv(accessKeyEnv, "existing-key-id")
|
||||||
|
t.Setenv(secretKeyEnv, "existing-secret")
|
||||||
|
|
||||||
|
secretsDir := t.TempDir()
|
||||||
|
mustWriteSecretFile(t, filepath.Join(secretsDir, accessKeyEnv), "file-key-id\n")
|
||||||
|
mustWriteSecretFile(t, filepath.Join(secretsDir, secretKeyEnv), "file-secret\n")
|
||||||
|
|
||||||
|
cfg := commandObjectStoreTestConfig(secretsDir)
|
||||||
|
origStoreFn := newObjectStoreFromConfigFn
|
||||||
|
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||||
|
if got := os.Getenv(accessKeyEnv); got != "existing-key-id" {
|
||||||
|
return nil, errors.New("existing access key was overwritten")
|
||||||
|
}
|
||||||
|
if got := os.Getenv(secretKeyEnv); got != "existing-secret" {
|
||||||
|
return nil, errors.New("existing secret key was overwritten")
|
||||||
|
}
|
||||||
|
return &storage.FakeBackend{}, nil
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
newObjectStoreFromConfigFn = origStoreFn
|
||||||
|
})
|
||||||
|
|
||||||
|
if _, err := newCommandObjectStore(context.Background(), cfg, nil); err != nil {
|
||||||
|
t.Fatalf("newCommandObjectStore() error = %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewCommandObjectStoreSecretErrorStopsFactory(t *testing.T) {
|
||||||
|
cfg := commandObjectStoreTestConfig(filepath.Join(t.TempDir(), "missing"))
|
||||||
|
called := false
|
||||||
|
origStoreFn := newObjectStoreFromConfigFn
|
||||||
|
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||||
|
called = true
|
||||||
|
return &storage.FakeBackend{}, nil
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
newObjectStoreFromConfigFn = origStoreFn
|
||||||
|
})
|
||||||
|
|
||||||
|
_, err := newCommandObjectStore(context.Background(), cfg, nil)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error, got nil")
|
||||||
|
}
|
||||||
|
if called {
|
||||||
|
t.Fatal("object store factory was called after secret load failure")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "load secrets from files") {
|
||||||
|
t.Fatalf("error = %q, want secret loading context", err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewCommandObjectStoreFactoryErrorIsContextual(t *testing.T) {
|
||||||
|
cfg := commandObjectStoreTestConfig("")
|
||||||
|
origStoreFn := newObjectStoreFromConfigFn
|
||||||
|
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||||
|
return nil, errors.New("factory boom")
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
newObjectStoreFromConfigFn = origStoreFn
|
||||||
|
})
|
||||||
|
|
||||||
|
_, err := newCommandObjectStore(context.Background(), cfg, nil)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error, got nil")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "initialize object store backend") || !strings.Contains(err.Error(), "factory boom") {
|
||||||
|
t.Fatalf("error = %q, want factory context", err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func commandObjectStoreTestConfig(secretsDir string) *config.Config {
|
||||||
|
cfg := &config.Config{
|
||||||
|
Pipeline: &config.PipelineConfig{
|
||||||
|
Storage: config.StorageConfig{
|
||||||
|
Backend: "s3",
|
||||||
|
S3: &config.StorageS3Config{
|
||||||
|
Bucket: "test-bucket",
|
||||||
|
AccessKeyIDEnv: "NARRATIO_TEST_COMMAND_STORE_KEY_ID",
|
||||||
|
SecretKeyEnv: "NARRATIO_TEST_COMMAND_STORE_SECRET",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(secretsDir) != "" {
|
||||||
|
cfg.Pipeline.Secrets = &config.SecretsConfig{EnvDir: secretsDir}
|
||||||
|
}
|
||||||
|
return cfg
|
||||||
|
}
|
||||||
|
|
||||||
|
func restoreEnvAfterTest(t *testing.T, names ...string) {
|
||||||
|
t.Helper()
|
||||||
|
originals := make(map[string]string, len(names))
|
||||||
|
present := make(map[string]bool, len(names))
|
||||||
|
for _, name := range names {
|
||||||
|
value, ok := os.LookupEnv(name)
|
||||||
|
originals[name] = value
|
||||||
|
present[name] = ok
|
||||||
|
_ = os.Unsetenv(name)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
for _, name := range names {
|
||||||
|
if present[name] {
|
||||||
|
_ = os.Setenv(name, originals[name])
|
||||||
|
} else {
|
||||||
|
_ = os.Unsetenv(name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
1145
internal/app/operator_helpers.go
Normal file
1145
internal/app/operator_helpers.go
Normal file
File diff suppressed because it is too large
Load Diff
962
internal/app/operator_helpers_test.go
Normal file
962
internal/app/operator_helpers_test.go
Normal file
@@ -0,0 +1,962 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestExecuteSessionInitRemoteWritesCanonicalSessionConfig(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "init", "2026-06-07",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--title", "The Black Cabin",
|
||||||
|
"--remote",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
key := artifacts.S3SessionConfigKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-06-07"))
|
||||||
|
obj, ok := fake.Objects[key]
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("remote session key %q not uploaded; objects=%v", key, fake.Objects)
|
||||||
|
}
|
||||||
|
if !strings.Contains(string(obj.Data), `session_id: "2026-06-07"`) || !strings.Contains(string(obj.Data), "prefix: audio/") {
|
||||||
|
t.Fatalf("remote session data = %q", string(obj.Data))
|
||||||
|
}
|
||||||
|
if storeInitCalls != 1 {
|
||||||
|
t.Fatalf("object store init calls = %d, want 1", storeInitCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitRemoteUsesDefaultConfigDiscovery(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
withDefaultPipelineCampaignConfigs(t, pipelinePath, campaignPath)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "init", "2026-06-07",
|
||||||
|
"--remote",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
key := artifacts.S3SessionConfigKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-06-07"))
|
||||||
|
if _, ok := fake.Objects[key]; !ok {
|
||||||
|
t.Fatalf("remote session key %q not uploaded; objects=%v", key, fake.Objects)
|
||||||
|
}
|
||||||
|
if storeInitCalls != 1 {
|
||||||
|
t.Fatalf("object store init calls = %d, want 1", storeInitCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitLocalUsesDefaultConfigDiscovery(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
withDefaultPipelineCampaignConfigs(t, pipelinePath, campaignPath)
|
||||||
|
outputPath := filepath.Join(t.TempDir(), "session.yml")
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "init", "2026-06-07",
|
||||||
|
"--output", outputPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(outputPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read generated session: %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(string(data), `session_id: "2026-06-07"`) || !strings.Contains(string(data), "prefix: audio/") {
|
||||||
|
t.Fatalf("generated session = %q", string(data))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitExplicitConfigWinsOverDefaults(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
defaultPipeline, defaultCampaign, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
withDefaultPipelineCampaignConfigs(t, defaultPipeline, defaultCampaign)
|
||||||
|
|
||||||
|
explicitDir := t.TempDir()
|
||||||
|
explicitCampaign := filepath.Join(explicitDir, "campaign.yml")
|
||||||
|
if err := os.WriteFile(explicitCampaign, []byte(`campaign_id: explicit-campaign
|
||||||
|
inputs:
|
||||||
|
speakers_file: ./speakers.yml
|
||||||
|
autocorrect_file: ./autocorrect.yml
|
||||||
|
glossary_file: ./glossary.yml
|
||||||
|
`), 0o644); err != nil {
|
||||||
|
t.Fatalf("write explicit campaign: %v", err)
|
||||||
|
}
|
||||||
|
mustWriteTestFile(t, filepath.Join(explicitDir, "speakers.yml"), "match:\n - speaker: Alice\n match: [\"alice\"]\n")
|
||||||
|
mustWriteTestFile(t, filepath.Join(explicitDir, "autocorrect.yml"), "[]\n")
|
||||||
|
mustWriteTestFile(t, filepath.Join(explicitDir, "glossary.yml"), "[]\n")
|
||||||
|
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "init", "2026-06-07",
|
||||||
|
"--config", defaultPipeline,
|
||||||
|
"--campaign-file", explicitCampaign,
|
||||||
|
"--remote",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
explicitKey := artifacts.S3SessionConfigKey(artifacts.S3SessionPrefix("dnd", "explicit-campaign", "2026-06-07"))
|
||||||
|
if _, ok := fake.Objects[explicitKey]; !ok {
|
||||||
|
t.Fatalf("explicit campaign remote key %q not uploaded; objects=%v", explicitKey, fake.Objects)
|
||||||
|
}
|
||||||
|
defaultKey := artifacts.S3SessionConfigKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-06-07"))
|
||||||
|
if _, ok := fake.Objects[defaultKey]; ok {
|
||||||
|
t.Fatalf("default campaign key %q uploaded despite explicit campaign override", defaultKey)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitRequiresSessionID(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
withDefaultPipelineCampaignConfigs(t, pipelinePath, campaignPath)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "init", "--remote"}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "session init: session_id is required") {
|
||||||
|
t.Fatalf("stderr = %q, want session-id required error", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitMissingDefaultConfigReportsSearchedPaths(t *testing.T) {
|
||||||
|
origPipelineDefaults := append([]string(nil), config.DefaultPipelineConfigSearchPaths...)
|
||||||
|
config.DefaultPipelineConfigSearchPaths = []string{filepath.Join(t.TempDir(), "missing-pipeline.yml")}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
config.DefaultPipelineConfigSearchPaths = origPipelineDefaults
|
||||||
|
})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "init", "2026-06-07", "--remote"}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "session init: no pipeline config path provided and no default pipeline config found; searched:") {
|
||||||
|
t.Fatalf("stderr = %q, want default pipeline searched-path error", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitRemoteLoadsSecretsBeforeObjectStoreInit(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
withDefaultPipelineCampaignConfigs(t, pipelinePath, campaignPath)
|
||||||
|
accessKeyEnv := "NARRATIO_TEST_SESSION_INIT_OBJECT_KEY_ID"
|
||||||
|
secretKeyEnv := "NARRATIO_TEST_SESSION_INIT_OBJECT_SECRET"
|
||||||
|
restoreEnvAfterTest(t, accessKeyEnv, secretKeyEnv)
|
||||||
|
secretsDir := t.TempDir()
|
||||||
|
mustWriteTestFile(t, filepath.Join(secretsDir, accessKeyEnv), "test-key-id\n")
|
||||||
|
mustWriteTestFile(t, filepath.Join(secretsDir, secretKeyEnv), "test-secret\n")
|
||||||
|
addSecretsToPipelineConfig(t, pipelinePath, secretsDir, accessKeyEnv, secretKeyEnv)
|
||||||
|
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
origStoreFn := newObjectStoreFromConfigFn
|
||||||
|
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||||
|
if os.Getenv(accessKeyEnv) != "test-key-id" || os.Getenv(secretKeyEnv) != "test-secret" {
|
||||||
|
return nil, fmt.Errorf("secrets were not loaded before object store init")
|
||||||
|
}
|
||||||
|
return fake, nil
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
newObjectStoreFromConfigFn = origStoreFn
|
||||||
|
})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "init", "2026-06-07", "--remote"}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitLocalRendersCampaignTemplate(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
writeSessionInitTemplate(t, campaignPath, `session_id: "{{ session_id }}"
|
||||||
|
previous_session_id: "{{ previous_session_id }}"
|
||||||
|
date: "{{ date }}"
|
||||||
|
title: "{{ title }}"
|
||||||
|
inputs:
|
||||||
|
audio_s3:
|
||||||
|
prefix: "{{ audio_s3_prefix }}"
|
||||||
|
`)
|
||||||
|
outputPath := filepath.Join(t.TempDir(), "session.yml")
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "init", "2026-06-07",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--previous-session-id", "2026-05-31",
|
||||||
|
"--date", "2026-06-07",
|
||||||
|
"--title", "The Black Cabin",
|
||||||
|
"--audio-s3-prefix", "audio/",
|
||||||
|
"--output", outputPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(outputPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read generated session: %v", err)
|
||||||
|
}
|
||||||
|
got := string(data)
|
||||||
|
for _, want := range []string{
|
||||||
|
`session_id: "2026-06-07"`,
|
||||||
|
`previous_session_id: "2026-05-31"`,
|
||||||
|
`date: "2026-06-07"`,
|
||||||
|
`title: "The Black Cabin"`,
|
||||||
|
`prefix: "audio/"`,
|
||||||
|
} {
|
||||||
|
if !strings.Contains(got, want) {
|
||||||
|
t.Fatalf("generated session = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.Contains(got, "{{") {
|
||||||
|
t.Fatalf("generated session still contains template placeholder: %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitRemoteRendersCampaignTemplate(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
writeSessionInitTemplate(t, campaignPath, `session_id: "{{ session_id }}"
|
||||||
|
inputs:
|
||||||
|
audio_s3:
|
||||||
|
prefix: audio/
|
||||||
|
`)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "init", "2026-06-07",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--remote",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
key := artifacts.S3SessionConfigKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-06-07"))
|
||||||
|
obj, ok := fake.Objects[key]
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("remote session key %q not uploaded; objects=%v", key, fake.Objects)
|
||||||
|
}
|
||||||
|
if strings.Contains(string(obj.Data), "{{") || !strings.Contains(string(obj.Data), `session_id: "2026-06-07"`) {
|
||||||
|
t.Fatalf("remote session data = %q, want rendered concrete session", string(obj.Data))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitTemplatePathIsCampaignRelative(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
templateDir := filepath.Join(filepath.Dir(campaignPath), "templates")
|
||||||
|
if err := os.MkdirAll(templateDir, 0o755); err != nil {
|
||||||
|
t.Fatalf("mkdir template dir: %v", err)
|
||||||
|
}
|
||||||
|
templatePath := filepath.Join(templateDir, "session.template.yml")
|
||||||
|
if err := os.WriteFile(templatePath, []byte(`session_id: "{{ session_id }}"
|
||||||
|
inputs:
|
||||||
|
audio_dir: ./audio
|
||||||
|
`), 0o644); err != nil {
|
||||||
|
t.Fatalf("write session template: %v", err)
|
||||||
|
}
|
||||||
|
addSessionTemplateToCampaign(t, campaignPath, "./templates/session.template.yml")
|
||||||
|
outputPath := filepath.Join(t.TempDir(), "session.yml")
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "init", "2026-06-07",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--output", outputPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(outputPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read generated session: %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(string(data), `session_id: "2026-06-07"`) {
|
||||||
|
t.Fatalf("generated session = %q, want campaign-relative template output", string(data))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitTemplateMissingVariableFails(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
writeSessionInitTemplate(t, campaignPath, `session_id: "{{ session_id }}"
|
||||||
|
date: "{{ date }}"
|
||||||
|
inputs:
|
||||||
|
audio_s3:
|
||||||
|
prefix: audio/
|
||||||
|
`)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "init", "2026-06-07",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--remote",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "missing required template variable value(s): date") {
|
||||||
|
t.Fatalf("stderr = %q, want missing date variable", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitTemplateUnusedFlagFails(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
writeSessionInitTemplate(t, campaignPath, `session_id: "{{ session_id }}"
|
||||||
|
inputs:
|
||||||
|
audio_s3:
|
||||||
|
prefix: audio/
|
||||||
|
`)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "init", "2026-06-07",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--title", "Unused Title",
|
||||||
|
"--remote",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "unused template variable value(s): title") {
|
||||||
|
t.Fatalf("stderr = %q, want unused title variable", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitTemplateStrictDecodeFailure(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
writeSessionInitTemplate(t, campaignPath, `session_id: "{{ session_id }}"
|
||||||
|
unknown: true
|
||||||
|
inputs:
|
||||||
|
audio_s3:
|
||||||
|
prefix: audio/
|
||||||
|
`)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "init", "2026-06-07",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--remote",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "strict decode failed") {
|
||||||
|
t.Fatalf("stderr = %q, want strict decode error", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionValidateLoadsSecretsBeforeObjectStoreInit(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
accessKeyEnv := "NARRATIO_TEST_VALIDATE_OBJECT_KEY_ID"
|
||||||
|
secretKeyEnv := "NARRATIO_TEST_VALIDATE_OBJECT_SECRET"
|
||||||
|
restoreEnvAfterTest(t, accessKeyEnv, secretKeyEnv)
|
||||||
|
secretsDir := t.TempDir()
|
||||||
|
mustWriteTestFile(t, filepath.Join(secretsDir, accessKeyEnv), "test-key-id\n")
|
||||||
|
mustWriteTestFile(t, filepath.Join(secretsDir, secretKeyEnv), "test-secret\n")
|
||||||
|
addSecretsToPipelineConfig(t, pipelinePath, secretsDir, accessKeyEnv, secretKeyEnv)
|
||||||
|
if err := os.WriteFile(sessionPath, []byte(`session_id: 2026-05-03
|
||||||
|
inputs:
|
||||||
|
audio_s3:
|
||||||
|
prefix: audio/
|
||||||
|
`), 0o644); err != nil {
|
||||||
|
t.Fatalf("write session: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
audioKey := artifacts.S3PublishedOutputKey(artifacts.S3AudioPrefix(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"), "audio/"), "alice.flac")
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: audioKey, Data: []byte("audio")})
|
||||||
|
origStoreFn := newObjectStoreFromConfigFn
|
||||||
|
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||||
|
if os.Getenv(accessKeyEnv) != "test-key-id" || os.Getenv(secretKeyEnv) != "test-secret" {
|
||||||
|
return nil, fmt.Errorf("secrets were not loaded before object store init")
|
||||||
|
}
|
||||||
|
return fake, nil
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
newObjectStoreFromConfigFn = origStoreFn
|
||||||
|
})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "validate", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stdout=%q stderr=%q", code, stdout.String(), stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "OK audio") {
|
||||||
|
t.Fatalf("stdout = %q, want OK audio", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteLocksAddListAndRemoveUseRemoteLockStore(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
"--reason", "manual edit",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("locks add exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
key := artifacts.S3SessionLocksKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"))
|
||||||
|
obj, ok := fake.Objects[key]
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("remote locks key %q not uploaded", key)
|
||||||
|
}
|
||||||
|
if !strings.Contains(string(obj.Data), "source: narratio.transcript.final_trimmed") || !strings.Contains(string(obj.Data), "reason: manual edit") {
|
||||||
|
t.Fatalf("lock store data = %q", string(obj.Data))
|
||||||
|
}
|
||||||
|
|
||||||
|
stdout.Reset()
|
||||||
|
stderr.Reset()
|
||||||
|
code = Execute([]string{
|
||||||
|
"session", "locks", "2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("locks list exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "- narratio.transcript.final_trimmed origin=remote") {
|
||||||
|
t.Fatalf("stdout = %q, want remote lock", stdout.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
stdout.Reset()
|
||||||
|
stderr.Reset()
|
||||||
|
code = Execute([]string{
|
||||||
|
"session", "locks", "remove", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("locks remove exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
store, err := config.LoadPublishLockStoreBytes("locks.yml", fake.Objects[key].Data, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("LoadPublishLockStoreBytes() error = %v", err)
|
||||||
|
}
|
||||||
|
if len(store.Locks) != 0 {
|
||||||
|
t.Fatalf("locks after remove = %#v, want empty", store.Locks)
|
||||||
|
}
|
||||||
|
if storeInitCalls != 3 {
|
||||||
|
t.Fatalf("object store init calls = %d, want 3", storeInitCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteLocksAddDuplicateRequiresForce(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
"--reason", "first",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("initial locks add exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
stdout.Reset()
|
||||||
|
stderr.Reset()
|
||||||
|
code = Execute([]string{
|
||||||
|
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
"--reason", "second",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("duplicate locks add exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "pass --force to update") {
|
||||||
|
t.Fatalf("stderr = %q, want force guidance", stderr.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
stdout.Reset()
|
||||||
|
stderr.Reset()
|
||||||
|
code = Execute([]string{
|
||||||
|
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
"--reason", "second",
|
||||||
|
"--force",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("forced locks add exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
key := artifacts.S3SessionLocksKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"))
|
||||||
|
if !strings.Contains(string(fake.Objects[key].Data), "reason: second") {
|
||||||
|
t.Fatalf("lock store data = %q, want updated reason", string(fake.Objects[key].Data))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteLocksRequireSessionID(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
args []string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{"list", []string{"session", "locks"}, "locks: session_id is required"},
|
||||||
|
{"add", []string{"session", "locks", "add", "narratio.transcript.final_trimmed"}, "locks add: expected session_id and source id"},
|
||||||
|
{"remove", []string{"session", "locks", "remove", "narratio.transcript.final_trimmed"}, "locks remove: expected session_id and source id"},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(tt.args, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), tt.want) {
|
||||||
|
t.Fatalf("stderr = %q, want %q", stderr.String(), tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteLocksCannotModifyStaticLocks(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
addStaticArchiveLockToPipelineConfig(t, pipelinePath, "narratio.transcript.final_trimmed")
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("locks add static lock exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "locked by pipeline config") {
|
||||||
|
t.Fatalf("stderr = %q, want static lock error", stderr.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
stdout.Reset()
|
||||||
|
stderr.Reset()
|
||||||
|
code = Execute([]string{
|
||||||
|
"session", "locks", "remove", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("locks remove static lock exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "locked by pipeline config") {
|
||||||
|
t.Fatalf("stderr = %q, want static lock error", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteTopLevelLockAndUnlockAreRemoved(t *testing.T) {
|
||||||
|
tests := []string{"lock", "unlock"}
|
||||||
|
for _, cmd := range tests {
|
||||||
|
t.Run(cmd, func(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{cmd, "narratio.transcript.final_trimmed"}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), `unknown command: "`+cmd+`"`) {
|
||||||
|
t.Fatalf("stderr = %q, want unknown command", stderr.String())
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func withDefaultPipelineCampaignConfigs(t *testing.T, pipelinePath, campaignPath string) {
|
||||||
|
t.Helper()
|
||||||
|
origPipelineDefaults := append([]string(nil), config.DefaultPipelineConfigSearchPaths...)
|
||||||
|
config.DefaultPipelineConfigSearchPaths = []string{pipelinePath}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
config.DefaultPipelineConfigSearchPaths = origPipelineDefaults
|
||||||
|
})
|
||||||
|
_ = campaignPath
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeSessionInitTemplate(t *testing.T, campaignPath, templateYAML string) {
|
||||||
|
t.Helper()
|
||||||
|
templatePath := filepath.Join(filepath.Dir(campaignPath), "session.template.yml")
|
||||||
|
if err := os.WriteFile(templatePath, []byte(templateYAML), 0o644); err != nil {
|
||||||
|
t.Fatalf("write session template: %v", err)
|
||||||
|
}
|
||||||
|
addSessionTemplateToCampaign(t, campaignPath, "./session.template.yml")
|
||||||
|
}
|
||||||
|
|
||||||
|
func addSessionTemplateToCampaign(t *testing.T, campaignPath, templateFile string) {
|
||||||
|
t.Helper()
|
||||||
|
data, err := os.ReadFile(campaignPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read campaign config: %v", err)
|
||||||
|
}
|
||||||
|
if strings.Contains(string(data), "session_template_file:") {
|
||||||
|
t.Fatalf("campaign config already has session_template_file: %q", string(data))
|
||||||
|
}
|
||||||
|
updated := "session_template_file: " + templateFile + "\n" + string(data)
|
||||||
|
if err := os.WriteFile(campaignPath, []byte(updated), 0o644); err != nil {
|
||||||
|
t.Fatalf("write campaign config: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteArtifactsListRemoteReportsPromotedAvailability(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
addArchivePromotionsToPipeline(t, pipelinePath, `
|
||||||
|
outputs:
|
||||||
|
- source: narratio.transcript.final_trimmed
|
||||||
|
dest: transcripts/final.trimmed.json
|
||||||
|
required: true
|
||||||
|
`)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
trimmedKey := artifacts.S3PublishedOutputKey(
|
||||||
|
artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"),
|
||||||
|
"transcripts/final.trimmed.json",
|
||||||
|
)
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: trimmedKey, Data: []byte(`{"segments":[]}`)})
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "artifacts", "2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
"--remote",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "narratio.transcript.final_trimmed remote=published") {
|
||||||
|
t.Fatalf("stdout = %q, want promoted remote availability", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteArtifactsListRemoteUsesPromotionDestinations(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
addArchivePromotionsToPipeline(t, pipelinePath, `
|
||||||
|
outputs:
|
||||||
|
- source: narratio.transcript.final
|
||||||
|
dest: transcripts/full.json
|
||||||
|
required: true
|
||||||
|
- source: narratio.bounds.session
|
||||||
|
dest: transcripts/bounds.json
|
||||||
|
required: true
|
||||||
|
`)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: artifacts.S3PublishedOutputKey(sessionPrefix, "transcripts/full.json"), Data: []byte(`{"segments":[]}`)})
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: artifacts.S3PublishedOutputKey(sessionPrefix, "transcripts/bounds.json"), Data: []byte(`{}`)})
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "artifacts", "2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
"--remote",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
out := stdout.String()
|
||||||
|
for _, unwanted := range []string{
|
||||||
|
"narratio.transcript.final remote=missing",
|
||||||
|
"narratio.bounds.session remote=missing",
|
||||||
|
} {
|
||||||
|
if strings.Contains(out, unwanted) {
|
||||||
|
t.Fatalf("stdout = %q, did not want catalog remote marker %q", out, unwanted)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, want := range []string{
|
||||||
|
"narratio.transcript.final dest=transcripts/full.json remote=published",
|
||||||
|
"narratio.bounds.session dest=transcripts/bounds.json remote=published",
|
||||||
|
} {
|
||||||
|
if !strings.Contains(out, want) {
|
||||||
|
t.Fatalf("stdout = %q, want %q", out, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteStatusReportsRemoteArtifactCatalog(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
addArchivePromotionsToPipeline(t, pipelinePath, `
|
||||||
|
outputs:
|
||||||
|
- source: narratio.transcript.final_trimmed
|
||||||
|
dest: transcripts/final.trimmed.json
|
||||||
|
required: true
|
||||||
|
- source: narratio.transcript.final
|
||||||
|
dest: transcripts/full.json
|
||||||
|
required: true
|
||||||
|
`)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
|
||||||
|
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
||||||
|
trimmedKey := artifacts.S3PublishedOutputKey(sessionPrefix, "transcripts/final.trimmed.json")
|
||||||
|
fullKey := artifacts.S3PublishedOutputKey(sessionPrefix, "transcripts/full.json")
|
||||||
|
lockKey := artifacts.S3SessionLocksKey(sessionPrefix)
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, "2026-05-03", "sample-campaign")})
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: trimmedKey, Data: []byte(`{"segments":[]}`)})
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: fullKey, Data: []byte(`{"segments":[]}`)})
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: lockKey, Data: []byte("locks:\n - source: narratio.transcript.final_trimmed\n reason: remote review\n")})
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "status", "2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
out := stdout.String()
|
||||||
|
for _, want := range []string{
|
||||||
|
"Remote outputs:",
|
||||||
|
"Built-in:",
|
||||||
|
"Configured:",
|
||||||
|
"Previous-session:",
|
||||||
|
"Published:",
|
||||||
|
"narratio.transcript.final_trimmed locked",
|
||||||
|
"narratio.transcript.final_trimmed locked remote=published",
|
||||||
|
"narratio.transcript.final dest=transcripts/full.json remote=published",
|
||||||
|
} {
|
||||||
|
if !strings.Contains(out, want) {
|
||||||
|
t.Fatalf("stdout = %q, want %q", out, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.Contains(out, "narratio.transcript.base remote=missing") {
|
||||||
|
t.Fatalf("stdout = %q, did not want catalog remote marker", out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteStatusReportsRemoteArtifactCatalogErrorsWithoutFailing(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
addArchivePromotionsToPipeline(t, pipelinePath, `
|
||||||
|
outputs:
|
||||||
|
- source: narratio.transcript.final_trimmed
|
||||||
|
dest: transcripts/final.trimmed.json
|
||||||
|
required: true
|
||||||
|
`)
|
||||||
|
fake := &storage.FakeBackend{ExistsErr: fmt.Errorf("exists failed")}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "status", "2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
out := stdout.String()
|
||||||
|
if !strings.Contains(out, "Remote publish: missing or unavailable:") {
|
||||||
|
t.Fatalf("stdout = %q, want remote publish unavailable state", out)
|
||||||
|
}
|
||||||
|
if !strings.Contains(out, "Remote outputs:") || !strings.Contains(out, "narratio.transcript.final_trimmed remote=error") {
|
||||||
|
t.Fatalf("stdout = %q, want remote output error state", out)
|
||||||
|
}
|
||||||
|
if !strings.Contains(out, "Publish locks: error:") {
|
||||||
|
t.Fatalf("stdout = %q, want publish locks error", out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteArchiveLoadsRemoteLocks(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidArchiveConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
lockKey := artifacts.S3SessionLocksKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"))
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: lockKey, Data: []byte("locks:\n - source: narratio.transcript.final_trimmed\n reason: remote review\n")})
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||||
|
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze"} {
|
||||||
|
// The publish stage only checks the manifest statuses and source files.
|
||||||
|
_ = stageName
|
||||||
|
}
|
||||||
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.trimmed.json"), `{"segments":[]}`)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"run-stage", "publish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
promotedKey := artifacts.S3PublishedOutputKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"), "transcripts/final.trimmed.json")
|
||||||
|
if _, ok := fake.Objects[promotedKey]; ok {
|
||||||
|
t.Fatalf("locked promoted key %q was uploaded", promotedKey)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func addArchivePromotionsToPipeline(t *testing.T, pipelinePath, archiveYAML string) {
|
||||||
|
t.Helper()
|
||||||
|
data, err := os.ReadFile(pipelinePath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read pipeline: %v", err)
|
||||||
|
}
|
||||||
|
updated := strings.Replace(string(data), " upload_run: false\n", " upload_run: false\n"+archiveYAML, 1)
|
||||||
|
if updated == string(data) {
|
||||||
|
t.Fatalf("pipeline %q did not contain archive upload_run marker", pipelinePath)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
|
||||||
|
t.Fatalf("write pipeline: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeValidArchiveConfigFiles(t *testing.T, workspaceRoot string) (string, string, string) {
|
||||||
|
t.Helper()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
data, err := os.ReadFile(pipelinePath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read pipeline: %v", err)
|
||||||
|
}
|
||||||
|
updated := strings.Replace(string(data), "upload_run: false", "upload_run: true", 1)
|
||||||
|
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
|
||||||
|
t.Fatalf("write pipeline: %v", err)
|
||||||
|
}
|
||||||
|
ctx := context.Background()
|
||||||
|
cfg, err := config.LoadWithSessionOptions(pipelinePath, campaignPath, sessionPath, config.SessionLoadOptions{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("LoadWithSessionOptions() error = %v", err)
|
||||||
|
}
|
||||||
|
store := &manifest.LocalStore{}
|
||||||
|
m := manifest.New("2026-05-03", nowUTC())
|
||||||
|
m.Campaign = "sample-campaign"
|
||||||
|
m.RunID = "20260521T160000Z-test"
|
||||||
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze"} {
|
||||||
|
m.MarkStageSucceeded(name, nowUTC(), nil)
|
||||||
|
}
|
||||||
|
path := artifacts.SessionManifestPathForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
if err := store.Save(ctx, path, m); err != nil {
|
||||||
|
t.Fatalf("save manifest: %v", err)
|
||||||
|
}
|
||||||
|
runManifestPath := artifacts.SessionRunManifestPathForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID, m.RunID)
|
||||||
|
if err := os.MkdirAll(filepath.Dir(runManifestPath), 0o755); err != nil {
|
||||||
|
t.Fatalf("mkdir run manifest: %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(runManifestPath, []byte("{}\n"), 0o644); err != nil {
|
||||||
|
t.Fatalf("write run manifest: %v", err)
|
||||||
|
}
|
||||||
|
return pipelinePath, campaignPath, sessionPath
|
||||||
|
}
|
||||||
|
|
||||||
|
func addStaticArchiveLockToPipelineConfig(t *testing.T, pipelinePath, source string) {
|
||||||
|
t.Helper()
|
||||||
|
data, err := os.ReadFile(pipelinePath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read pipeline: %v", err)
|
||||||
|
}
|
||||||
|
updated := strings.Replace(
|
||||||
|
string(data),
|
||||||
|
"publish:\n enabled: true\n upload_run: false\n",
|
||||||
|
"publish:\n enabled: true\n upload_run: false\n locks:\n - source: "+source+"\n reason: static review\n",
|
||||||
|
1,
|
||||||
|
)
|
||||||
|
if updated == string(data) {
|
||||||
|
t.Fatalf("archive section not found in pipeline config")
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
|
||||||
|
t.Fatalf("write pipeline: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"os"
|
"os"
|
||||||
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
@@ -16,35 +17,45 @@ import (
|
|||||||
|
|
||||||
// Plan validates configuration, prepares the local workdir, and prints stage order.
|
// Plan validates configuration, prepares the local workdir, and prints stage order.
|
||||||
func Plan(ctx context.Context, args []string, out io.Writer) error {
|
func Plan(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
positionalSessionID, args := pullLeadingSessionID(args)
|
||||||
fs := flag.NewFlagSet("plan", flag.ContinueOnError)
|
fs := flag.NewFlagSet("plan", flag.ContinueOnError)
|
||||||
fs.SetOutput(io.Discard)
|
fs.SetOutput(io.Discard)
|
||||||
|
|
||||||
var pipelinePath string
|
var pipelinePath string
|
||||||
|
var campaignPath string
|
||||||
|
var campaignFilePath string
|
||||||
var sessionPath string
|
var sessionPath string
|
||||||
var sessionID string
|
var sessionID string
|
||||||
|
var previousSessionID string
|
||||||
var force bool
|
var force bool
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||||
|
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||||
|
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||||
fs.StringVar(&sessionID, "session-id", "", "session identifier for session.yml templates")
|
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||||
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
if err := fs.Parse(args); err != nil {
|
||||||
return fmt.Errorf("plan: invalid flags: %w", err)
|
return fmt.Errorf("plan: invalid flags: %w", err)
|
||||||
}
|
}
|
||||||
|
if positionalSessionID == "" {
|
||||||
|
if err := applyParsedSessionIDArg("plan", fs, &sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
if fs.NArg() != 0 {
|
if fs.NArg() != 0 {
|
||||||
return fmt.Errorf("plan: unexpected positional arguments")
|
return fmt.Errorf("plan: unexpected positional arguments")
|
||||||
}
|
}
|
||||||
resolvedPipelinePath, err := resolvePipelineConfigPath(pipelinePath)
|
if err := applyPositionalSessionID("plan", positionalSessionID, &sessionID); err != nil {
|
||||||
if err != nil {
|
return err
|
||||||
return fmt.Errorf("plan: %w", err)
|
|
||||||
}
|
}
|
||||||
resolvedSessionPath, err := resolveSessionConfigPath(sessionPath)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("plan: %w", err)
|
|
||||||
}
|
}
|
||||||
|
if strings.TrimSpace(sessionID) == "" {
|
||||||
cfg, err := config.LoadWithSessionOptions(resolvedPipelinePath, resolvedSessionPath, config.SessionLoadOptions{
|
return fmt.Errorf("plan: session_id is required")
|
||||||
|
}
|
||||||
|
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
||||||
SessionID: sessionID,
|
SessionID: sessionID,
|
||||||
|
PreviousSessionID: previousSessionID,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("plan: %w", err)
|
return fmt.Errorf("plan: %w", err)
|
||||||
@@ -72,7 +83,7 @@ func Plan(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
|
|
||||||
runCount := 0
|
runCount := 0
|
||||||
skipCount := 0
|
skipCount := 0
|
||||||
if _, err := fmt.Fprintf(out, "narratio plan: workdir prepared at %s\n", paths.Root); err != nil {
|
if _, err := fmt.Fprintf(out, "narratio session plan: workdir prepared at %s\n", paths.Root); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
for _, d := range decisions {
|
for _, d := range decisions {
|
||||||
|
|||||||
@@ -15,19 +15,19 @@ import (
|
|||||||
|
|
||||||
func TestPlanCreatesAndReusesWorkdir(t *testing.T) {
|
func TestPlanCreatesAndReusesWorkdir(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
args := []string{"--config", pipelinePath, "--session", sessionPath}
|
args := []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}
|
||||||
|
|
||||||
if err := Plan(context.Background(), args, &out); err != nil {
|
if err := Plan(context.Background(), args, &out); err != nil {
|
||||||
t.Fatalf("first Plan() error = %v", err)
|
t.Fatalf("first Plan() error = %v", err)
|
||||||
}
|
}
|
||||||
got := out.String()
|
got := out.String()
|
||||||
if !strings.Contains(got, "narratio plan: workdir prepared at") {
|
if !strings.Contains(got, "narratio session plan: workdir prepared at") {
|
||||||
t.Fatalf("first output = %q, want workdir prepared", got)
|
t.Fatalf("first output = %q, want workdir prepared", got)
|
||||||
}
|
}
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "archive", "notify"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
||||||
if !strings.Contains(got, name+": run") {
|
if !strings.Contains(got, name+": run") {
|
||||||
t.Fatalf("first output = %q, missing stage %q", got, name)
|
t.Fatalf("first output = %q, missing stage %q", got, name)
|
||||||
}
|
}
|
||||||
@@ -55,14 +55,14 @@ func TestPlanCreatesAndReusesWorkdir(t *testing.T) {
|
|||||||
if err := Plan(context.Background(), args, &out); err != nil {
|
if err := Plan(context.Background(), args, &out); err != nil {
|
||||||
t.Fatalf("second Plan() error = %v", err)
|
t.Fatalf("second Plan() error = %v", err)
|
||||||
}
|
}
|
||||||
if !strings.Contains(out.String(), "narratio plan: workdir prepared at") {
|
if !strings.Contains(out.String(), "narratio session plan: workdir prepared at") {
|
||||||
t.Fatalf("second output = %q, want workdir prepared", out.String())
|
t.Fatalf("second output = %q, want workdir prepared", out.String())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPlanShowsRunAndSkipFromManifest(t *testing.T) {
|
func TestPlanShowsRunAndSkipFromManifest(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
@@ -74,7 +74,7 @@ func TestPlanShowsRunAndSkipFromManifest(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
if err := Plan(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath}, &out); err != nil {
|
if err := Plan(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out); err != nil {
|
||||||
t.Fatalf("Plan() error = %v", err)
|
t.Fatalf("Plan() error = %v", err)
|
||||||
}
|
}
|
||||||
got := out.String()
|
got := out.String()
|
||||||
@@ -93,6 +93,7 @@ func TestPlanFailsWhenConfiguredSecretsDirMissing(t *testing.T) {
|
|||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
configDir := t.TempDir()
|
configDir := t.TempDir()
|
||||||
pipelinePath := filepath.Join(configDir, "pipeline.yml")
|
pipelinePath := filepath.Join(configDir, "pipeline.yml")
|
||||||
|
campaignPath := writeAppTestCampaignConfig(t, configDir)
|
||||||
sessionPath := filepath.Join(configDir, "session.yml")
|
sessionPath := filepath.Join(configDir, "session.yml")
|
||||||
|
|
||||||
pipelineYAML := `workspace:
|
pipelineYAML := `workspace:
|
||||||
@@ -107,8 +108,6 @@ seriatim:
|
|||||||
binary: seriatim
|
binary: seriatim
|
||||||
audita:
|
audita:
|
||||||
binary: audita
|
binary: audita
|
||||||
analyzer:
|
|
||||||
timeout: 20m
|
|
||||||
notification:
|
notification:
|
||||||
timeout: 10s
|
timeout: 10s
|
||||||
`
|
`
|
||||||
@@ -128,7 +127,7 @@ inputs:
|
|||||||
}
|
}
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Plan(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath}, &out)
|
err := Plan(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected error, got nil")
|
t.Fatal("expected error, got nil")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ import "testing"
|
|||||||
|
|
||||||
func TestBuildFullPlanOrder(t *testing.T) {
|
func TestBuildFullPlanOrder(t *testing.T) {
|
||||||
got := BuildFullPlan()
|
got := BuildFullPlan()
|
||||||
want := []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "archive", "notify"}
|
want := []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"}
|
||||||
if len(got) != len(want) {
|
if len(got) != len(want) {
|
||||||
t.Fatalf("len(plan) = %d, want %d", len(got), len(want))
|
t.Fatalf("len(plan) = %d, want %d", len(got), len(want))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -17,8 +17,8 @@ func runPostArchiveCleanup(ctx context.Context, env *Env, manifestPath string, m
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
spoolRequested := env.Config.Pipeline.Spool.DeleteAudioAfterArchive
|
spoolRequested := env.Config.Pipeline.Spool.DeleteAudioAfterPublish
|
||||||
workRequested := env.Config.Pipeline.Workspace.CleanupAfterArchive
|
workRequested := env.Config.Pipeline.Workspace.CleanupAfterPublish
|
||||||
if !spoolRequested && !workRequested {
|
if !spoolRequested && !workRequested {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -63,9 +63,9 @@ func runPostArchiveCleanup(ctx context.Context, env *Env, manifestPath string, m
|
|||||||
}
|
}
|
||||||
|
|
||||||
if spoolRequested {
|
if spoolRequested {
|
||||||
if err := removeRunScopedDir(strings.TrimSpace(env.Config.Pipeline.Spool.Root), spoolDir, "pipeline.spool.delete_audio_after_archive"); err != nil {
|
if err := removeRunScopedDir(strings.TrimSpace(env.Config.Pipeline.Spool.Root), spoolDir, "pipeline.spool.delete_audio_after_publish"); err != nil {
|
||||||
sr.Metadata["cleanup_failed"] = true
|
sr.Metadata["cleanup_failed"] = true
|
||||||
sr.Metadata["cleanup_failed_policy"] = "pipeline.spool.delete_audio_after_archive"
|
sr.Metadata["cleanup_failed_policy"] = "pipeline.spool.delete_audio_after_publish"
|
||||||
sr.Metadata["cleanup_failed_path"] = spoolDir
|
sr.Metadata["cleanup_failed_path"] = spoolDir
|
||||||
_ = env.ManifestStore.Save(ctx, manifestPath, m)
|
_ = env.ManifestStore.Save(ctx, manifestPath, m)
|
||||||
return err
|
return err
|
||||||
@@ -82,9 +82,9 @@ func runPostArchiveCleanup(ctx context.Context, env *Env, manifestPath string, m
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
if err := removeRunScopedDir(strings.TrimSpace(env.Config.Pipeline.Workspace.Root), workDir, "pipeline.workspace.cleanup_after_archive"); err != nil {
|
if err := removeRunScopedDir(strings.TrimSpace(env.Config.Pipeline.Workspace.Root), workDir, "pipeline.workspace.cleanup_after_publish"); err != nil {
|
||||||
sr.Metadata["cleanup_failed"] = true
|
sr.Metadata["cleanup_failed"] = true
|
||||||
sr.Metadata["cleanup_failed_policy"] = "pipeline.workspace.cleanup_after_archive"
|
sr.Metadata["cleanup_failed_policy"] = "pipeline.workspace.cleanup_after_publish"
|
||||||
sr.Metadata["cleanup_failed_path"] = workDir
|
sr.Metadata["cleanup_failed_path"] = workDir
|
||||||
_ = env.ManifestStore.Save(ctx, manifestPath, m)
|
_ = env.ManifestStore.Save(ctx, manifestPath, m)
|
||||||
return err
|
return err
|
||||||
@@ -100,17 +100,17 @@ func archiveStageRecordForCleanup(m *manifest.Manifest, executed []string) *mani
|
|||||||
if m == nil {
|
if m == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
archiveRan := false
|
publishRan := false
|
||||||
for _, name := range executed {
|
for _, name := range executed {
|
||||||
if name == "archive" {
|
if name == "publish" {
|
||||||
archiveRan = true
|
publishRan = true
|
||||||
break
|
break
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if !archiveRan {
|
if !publishRan {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
sr := m.Stages["archive"]
|
sr := m.Stages["publish"]
|
||||||
if sr == nil || sr.Status != manifest.StatusSucceeded {
|
if sr == nil || sr.Status != manifest.StatusSucceeded {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -118,88 +118,105 @@ func archiveStageRecordForCleanup(m *manifest.Manifest, executed []string) *mani
|
|||||||
}
|
}
|
||||||
|
|
||||||
func archiveCleanupEligible(cfg *config.Config, sr *manifest.StageRecord) (bool, string) {
|
func archiveCleanupEligible(cfg *config.Config, sr *manifest.StageRecord) (bool, string) {
|
||||||
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Archive == nil {
|
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Publish == nil {
|
||||||
return false, "archive configuration is missing"
|
return false, "publish configuration is missing"
|
||||||
}
|
}
|
||||||
enabled := true
|
enabled := true
|
||||||
if cfg.Pipeline.Archive.Enabled != nil {
|
if cfg.Pipeline.Publish.Enabled != nil {
|
||||||
enabled = *cfg.Pipeline.Archive.Enabled
|
enabled = *cfg.Pipeline.Publish.Enabled
|
||||||
}
|
}
|
||||||
if !enabled {
|
if !enabled {
|
||||||
return false, "archive.enabled is false"
|
return false, "publish.enabled is false"
|
||||||
}
|
}
|
||||||
uploadRun := true
|
uploadRun := true
|
||||||
if cfg.Pipeline.Archive.UploadRun != nil {
|
if cfg.Pipeline.Publish.UploadRun != nil {
|
||||||
uploadRun = *cfg.Pipeline.Archive.UploadRun
|
uploadRun = *cfg.Pipeline.Publish.UploadRun
|
||||||
}
|
}
|
||||||
if !uploadRun {
|
if !uploadRun {
|
||||||
return false, "archive.upload_run is false"
|
return false, "publish.upload_run is false"
|
||||||
}
|
}
|
||||||
if sr == nil || sr.Metadata == nil {
|
if sr == nil || sr.Metadata == nil {
|
||||||
return false, "archive metadata is missing"
|
return false, "publish metadata is missing"
|
||||||
}
|
}
|
||||||
if skipped, _ := sr.Metadata["skipped"].(bool); skipped {
|
if skipped, _ := sr.Metadata["skipped"].(bool); skipped {
|
||||||
return false, "archive stage was skipped"
|
return false, "publish stage was skipped"
|
||||||
}
|
}
|
||||||
if uploaded, _ := sr.Metadata["uploaded"].(bool); !uploaded {
|
if uploaded, _ := sr.Metadata["uploaded"].(bool); !uploaded {
|
||||||
return false, "archive did not upload run record"
|
return false, "publish did not upload run record"
|
||||||
}
|
}
|
||||||
if pointer, _ := sr.Metadata["current_pointer_written"].(bool); !pointer {
|
if pointer, _ := sr.Metadata["current_pointer_written"].(bool); !pointer {
|
||||||
return false, "archive did not write current pointer"
|
return false, "publish did not write current pointer"
|
||||||
}
|
}
|
||||||
if strings.TrimSpace(asString(sr.Metadata["current_run_id_key"])) == "" {
|
if strings.TrimSpace(asString(sr.Metadata["current_run_id_key"])) == "" {
|
||||||
return false, "archive current run pointer key is missing"
|
return false, "publish current run pointer key is missing"
|
||||||
}
|
}
|
||||||
return true, ""
|
return true, ""
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type scopedDir struct {
|
||||||
|
RootAbs string
|
||||||
|
TargetAbs string
|
||||||
|
Exists bool
|
||||||
|
}
|
||||||
|
|
||||||
func removeRunScopedDir(root, target, policy string) error {
|
func removeRunScopedDir(root, target, policy string) error {
|
||||||
|
dir, err := validateScopedDir(root, target, policy)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !dir.Exists {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := os.RemoveAll(dir.TargetAbs); err != nil {
|
||||||
|
return fmt.Errorf("cleanup policy %s: remove %q: %w", policy, dir.TargetAbs, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateScopedDir(root, target, policy string) (scopedDir, error) {
|
||||||
cleanRoot := strings.TrimSpace(root)
|
cleanRoot := strings.TrimSpace(root)
|
||||||
cleanTarget := strings.TrimSpace(target)
|
cleanTarget := strings.TrimSpace(target)
|
||||||
if cleanRoot == "" {
|
if cleanRoot == "" {
|
||||||
return fmt.Errorf("cleanup policy %s: root path is required", policy)
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: root path is required", policy)
|
||||||
}
|
}
|
||||||
if cleanTarget == "" {
|
if cleanTarget == "" {
|
||||||
return fmt.Errorf("cleanup policy %s: target path is required", policy)
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: target path is required", policy)
|
||||||
}
|
}
|
||||||
|
|
||||||
rootAbs, err := filepath.Abs(cleanRoot)
|
rootAbs, err := filepath.Abs(cleanRoot)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
||||||
}
|
}
|
||||||
targetAbs, err := filepath.Abs(cleanTarget)
|
targetAbs, err := filepath.Abs(cleanTarget)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
rel, err := filepath.Rel(rootAbs, targetAbs)
|
rel, err := filepath.Rel(rootAbs, targetAbs)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
|
||||||
}
|
}
|
||||||
if rel == "." {
|
if rel == "." {
|
||||||
return fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
|
||||||
}
|
}
|
||||||
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||||
return fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
|
||||||
}
|
}
|
||||||
|
|
||||||
info, err := os.Lstat(targetAbs)
|
info, err := os.Lstat(targetAbs)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if os.IsNotExist(err) {
|
if os.IsNotExist(err) {
|
||||||
return nil
|
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: false}, nil
|
||||||
}
|
}
|
||||||
return fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
|
||||||
}
|
}
|
||||||
if info.Mode()&os.ModeSymlink != 0 {
|
if info.Mode()&os.ModeSymlink != 0 {
|
||||||
return fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
|
||||||
}
|
}
|
||||||
if !info.IsDir() {
|
if !info.IsDir() {
|
||||||
return fmt.Errorf("cleanup policy %s: target %q is not a directory", policy, targetAbs)
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is not a directory", policy, targetAbs)
|
||||||
}
|
}
|
||||||
if err := os.RemoveAll(targetAbs); err != nil {
|
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: true}, nil
|
||||||
return fmt.Errorf("cleanup policy %s: remove %q: %w", policy, targetAbs, err)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func asString(v any) string {
|
func asString(v any) string {
|
||||||
|
|||||||
@@ -20,11 +20,11 @@ type archiveSuccessStage struct {
|
|||||||
metadata map[string]any
|
metadata map[string]any
|
||||||
}
|
}
|
||||||
|
|
||||||
func (archiveSuccessStage) Name() string { return "archive" }
|
func (archiveSuccessStage) Name() string { return "publish" }
|
||||||
func (archiveSuccessStage) Declares() stage.IODecl { return stage.IODecl{} }
|
func (archiveSuccessStage) Declares() stage.IODecl { return stage.IODecl{} }
|
||||||
func (s archiveSuccessStage) Run(_ context.Context, _ *stage.Env, _ *manifest.Manifest) (*stage.StageResult, error) {
|
func (s archiveSuccessStage) Run(_ context.Context, _ *stage.Env, _ *manifest.Manifest) (*stage.StageResult, error) {
|
||||||
md := map[string]any{
|
md := map[string]any{
|
||||||
"stage": "archive",
|
"stage": "publish",
|
||||||
"uploaded": true,
|
"uploaded": true,
|
||||||
"current_pointer_written": true,
|
"current_pointer_written": true,
|
||||||
"current_run_id_key": "dnd/campaigns/sample-campaign/sessions/2026-05-03/current/run_id.txt",
|
"current_run_id_key": "dnd/campaigns/sample-campaign/sessions/2026-05-03/current/run_id.txt",
|
||||||
@@ -45,8 +45,8 @@ func (notifyFailStage) Run(_ context.Context, _ *stage.Env, _ *manifest.Manifest
|
|||||||
|
|
||||||
func TestPostArchiveCleanupDisabledKeepsLocalDirs(t *testing.T) {
|
func TestPostArchiveCleanupDisabledKeepsLocalDirs(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = false
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = false
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = false
|
cfg.Pipeline.Workspace.CleanupAfterPublish = false
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
@@ -59,8 +59,8 @@ func TestPostArchiveCleanupDisabledKeepsLocalDirs(t *testing.T) {
|
|||||||
|
|
||||||
func TestPostArchiveCleanupSpoolOnly(t *testing.T) {
|
func TestPostArchiveCleanupSpoolOnly(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = false
|
cfg.Pipeline.Workspace.CleanupAfterPublish = false
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
@@ -73,8 +73,8 @@ func TestPostArchiveCleanupSpoolOnly(t *testing.T) {
|
|||||||
|
|
||||||
func TestPostArchiveCleanupWorkdirOnly(t *testing.T) {
|
func TestPostArchiveCleanupWorkdirOnly(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = false
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = false
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
@@ -82,14 +82,15 @@ func TestPostArchiveCleanupWorkdirOnly(t *testing.T) {
|
|||||||
|
|
||||||
assertExists(t, cfg.Pipeline.Workspace.Root)
|
assertExists(t, cfg.Pipeline.Workspace.Root)
|
||||||
assertExists(t, seed.otherRunDir)
|
assertExists(t, seed.otherRunDir)
|
||||||
|
assertExists(t, seed.previousCachePath)
|
||||||
assertMissing(t, seed.runWorkDir)
|
assertMissing(t, seed.runWorkDir)
|
||||||
assertExists(t, seed.spoolAudioDir)
|
assertExists(t, seed.spoolAudioDir)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupBothPolicies(t *testing.T) {
|
func TestPostArchiveCleanupBothPolicies(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
@@ -98,16 +99,17 @@ func TestPostArchiveCleanupBothPolicies(t *testing.T) {
|
|||||||
assertMissing(t, seed.spoolAudioDir)
|
assertMissing(t, seed.spoolAudioDir)
|
||||||
assertMissing(t, seed.runWorkDir)
|
assertMissing(t, seed.runWorkDir)
|
||||||
assertExists(t, seed.otherRunDir)
|
assertExists(t, seed.otherRunDir)
|
||||||
|
assertExists(t, seed.previousCachePath)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenArchiveFails(t *testing.T) {
|
func TestPostArchiveCleanupNotRunWhenArchiveFails(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
_, err := executeStages(context.Background(), cfg, []stage.Stage{failingStage{name: "archive", err: errors.New("archive failed")}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
_, err := executeStages(context.Background(), cfg, []stage.Stage{failingStage{name: "publish", err: errors.New("archive failed")}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
||||||
if err == nil || !strings.Contains(err.Error(), "stage \"archive\" failed") {
|
if err == nil || !strings.Contains(err.Error(), "stage \"publish\" failed") {
|
||||||
t.Fatalf("executeStages() error = %v, want archive failure", err)
|
t.Fatalf("executeStages() error = %v, want publish failure", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
assertExists(t, seed.spoolAudioDir)
|
assertExists(t, seed.spoolAudioDir)
|
||||||
@@ -116,8 +118,8 @@ func TestPostArchiveCleanupNotRunWhenArchiveFails(t *testing.T) {
|
|||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenArchiveSkipped(t *testing.T) {
|
func TestPostArchiveCleanupNotRunWhenArchiveSkipped(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{metadata: map[string]any{"skipped": true}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{metadata: map[string]any{"skipped": true}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
@@ -129,8 +131,8 @@ func TestPostArchiveCleanupNotRunWhenArchiveSkipped(t *testing.T) {
|
|||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenCurrentPointerMissing(t *testing.T) {
|
func TestPostArchiveCleanupNotRunWhenCurrentPointerMissing(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{metadata: map[string]any{"current_pointer_written": false}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{metadata: map[string]any{"current_pointer_written": false}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
@@ -142,9 +144,9 @@ func TestPostArchiveCleanupNotRunWhenCurrentPointerMissing(t *testing.T) {
|
|||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenArchiveUploadDisabled(t *testing.T) {
|
func TestPostArchiveCleanupNotRunWhenArchiveUploadDisabled(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
cfg.Pipeline.Archive.UploadRun = boolPtr(false)
|
cfg.Pipeline.Publish.UploadRun = boolPtr(false)
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
@@ -156,8 +158,8 @@ func TestPostArchiveCleanupNotRunWhenArchiveUploadDisabled(t *testing.T) {
|
|||||||
|
|
||||||
func TestPostArchiveCleanupWaitsUntilAllStagesSucceed(t *testing.T) {
|
func TestPostArchiveCleanupWaitsUntilAllStagesSucceed(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
_, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}, notifyFailStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
_, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}, notifyFailStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
||||||
if err == nil || !strings.Contains(err.Error(), "stage \"notify\" failed") {
|
if err == nil || !strings.Contains(err.Error(), "stage \"notify\" failed") {
|
||||||
@@ -170,8 +172,8 @@ func TestPostArchiveCleanupWaitsUntilAllStagesSucceed(t *testing.T) {
|
|||||||
|
|
||||||
func TestPostArchiveCleanupFailsOnUnsafePath(t *testing.T) {
|
func TestPostArchiveCleanupFailsOnUnsafePath(t *testing.T) {
|
||||||
cfg, _ := cleanupFixtureConfig(t)
|
cfg, _ := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = false
|
cfg.Pipeline.Workspace.CleanupAfterPublish = false
|
||||||
|
|
||||||
manifestPath := manifestPathFor(cfg)
|
manifestPath := manifestPathFor(cfg)
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
@@ -192,19 +194,19 @@ func TestPostArchiveCleanupFailsOnUnsafePath(t *testing.T) {
|
|||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenPromotionIsMissing(t *testing.T) {
|
func TestPostArchiveCleanupNotRunWhenPromotionIsMissing(t *testing.T) {
|
||||||
cfg, seed, runID := archiveStageCleanupFixture(t)
|
cfg, seed, runID := archiveStageCleanupFixture(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
cfg.Pipeline.Archive.PromoteArtifacts = []config.ArchivePromotionRule{
|
cfg.Pipeline.Publish.Outputs = []config.PublishOutputRule{
|
||||||
{Source: "narratio.transcript.merged", Dest: "transcripts/merged.json", Required: boolPtr(true)},
|
{Source: "narratio.transcript.base", Dest: "transcripts/base.json", Required: boolPtr(true)},
|
||||||
}
|
}
|
||||||
|
|
||||||
archiveStageImpl, err := stage.Select("archive")
|
archiveStageImpl, err := stage.Select("publish")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Select(archive) error = %v", err)
|
t.Fatalf("Select(publish) error = %v", err)
|
||||||
}
|
}
|
||||||
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
||||||
if err == nil || !strings.Contains(err.Error(), "required promotion source unavailable") {
|
if err == nil || !strings.Contains(err.Error(), "required output source unavailable") {
|
||||||
t.Fatalf("executeStages() error = %v, want promotion-missing failure", err)
|
t.Fatalf("executeStages() error = %v, want required output source unavailable failure", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
assertExists(t, seed.spoolAudioDir)
|
assertExists(t, seed.spoolAudioDir)
|
||||||
@@ -215,13 +217,13 @@ func TestPostArchiveCleanupNotRunWhenPromotionIsMissing(t *testing.T) {
|
|||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenCurrentManifestUploadFails(t *testing.T) {
|
func TestPostArchiveCleanupNotRunWhenCurrentManifestUploadFails(t *testing.T) {
|
||||||
cfg, seed, _ := archiveStageCleanupFixture(t)
|
cfg, seed, _ := archiveStageCleanupFixture(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
failKey := seed.sessionPrefix + "current/manifest.json"
|
failKey := seed.sessionPrefix + "current/manifest.json"
|
||||||
|
|
||||||
archiveStageImpl, err := stage.Select("archive")
|
archiveStageImpl, err := stage.Select("publish")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Select(archive) error = %v", err)
|
t.Fatalf("Select(publish) error = %v", err)
|
||||||
}
|
}
|
||||||
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{
|
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{
|
||||||
Env: &Env{ObjectStore: &failKeyStore{delegate: &storage.FakeBackend{}, failKey: failKey}},
|
Env: &Env{ObjectStore: &failKeyStore{delegate: &storage.FakeBackend{}, failKey: failKey}},
|
||||||
@@ -236,13 +238,13 @@ func TestPostArchiveCleanupNotRunWhenCurrentManifestUploadFails(t *testing.T) {
|
|||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenCurrentPointerUploadFails(t *testing.T) {
|
func TestPostArchiveCleanupNotRunWhenCurrentPointerUploadFails(t *testing.T) {
|
||||||
cfg, seed, _ := archiveStageCleanupFixture(t)
|
cfg, seed, _ := archiveStageCleanupFixture(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterArchive = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterArchive = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
failKey := seed.sessionPrefix + "current/run_id.txt"
|
failKey := seed.sessionPrefix + "current/run_id.txt"
|
||||||
|
|
||||||
archiveStageImpl, err := stage.Select("archive")
|
archiveStageImpl, err := stage.Select("publish")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Select(archive) error = %v", err)
|
t.Fatalf("Select(publish) error = %v", err)
|
||||||
}
|
}
|
||||||
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{
|
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{
|
||||||
Env: &Env{ObjectStore: &failKeyStore{delegate: &storage.FakeBackend{}, failKey: failKey}},
|
Env: &Env{ObjectStore: &failKeyStore{delegate: &storage.FakeBackend{}, failKey: failKey}},
|
||||||
@@ -260,6 +262,7 @@ type cleanupSeed struct {
|
|||||||
otherRunDir string
|
otherRunDir string
|
||||||
spoolAudioDir string
|
spoolAudioDir string
|
||||||
localSourceAudio string
|
localSourceAudio string
|
||||||
|
previousCachePath string
|
||||||
sessionPrefix string
|
sessionPrefix string
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -267,18 +270,25 @@ func cleanupFixtureConfig(t *testing.T) (*config.Config, cleanupSeed) {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
cfg := testConfig(t)
|
cfg := testConfig(t)
|
||||||
cfg.Pipeline.Archive = &config.ArchiveConfig{Enabled: boolPtr(true), UploadRun: boolPtr(true)}
|
cfg.Pipeline.Publish = &config.PublishConfig{Enabled: boolPtr(true), UploadRun: boolPtr(true)}
|
||||||
cfg.Pipeline.Spool.Root = filepath.Join(t.TempDir(), "spool")
|
cfg.Pipeline.Spool.Root = filepath.Join(t.TempDir(), "spool")
|
||||||
|
|
||||||
runID := "20260516T010203Z-1a2b3c4d"
|
runID := "20260516T010203Z-1a2b3c4d"
|
||||||
runWorkDir := artifacts.SessionRunRootForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID, runID)
|
runWorkDir := artifacts.SessionRunRootForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID, runID)
|
||||||
otherRunDir := artifacts.SessionRunRootForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID, "20260516T010204Z-5e6f7a8b")
|
otherRunDir := artifacts.SessionRunRootForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID, "20260516T010204Z-5e6f7a8b")
|
||||||
spoolAudioDir := artifacts.SessionSpoolAudioDir(cfg.Pipeline.Spool.Root, cfg.Session.Campaign, cfg.Session.SessionID, runID)
|
spoolAudioDir := artifacts.SessionSpoolAudioDir(cfg.Pipeline.Spool.Root, cfg.Session.Campaign, cfg.Session.SessionID, runID)
|
||||||
|
previousCachePath := artifacts.SessionPreviousArtifactPathForCampaign(
|
||||||
|
cfg.Pipeline.Workspace.Root,
|
||||||
|
cfg.Session.Campaign,
|
||||||
|
cfg.Session.SessionID,
|
||||||
|
"session_recap.md",
|
||||||
|
)
|
||||||
|
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "manifest.json"), "{}\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "manifest.json"), "{}\n")
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "logs", "stage.log"), "log\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "logs", "stage.log"), "log\n")
|
||||||
mustWriteFile(t, filepath.Join(otherRunDir, "logs", "stage.log"), "other\n")
|
mustWriteFile(t, filepath.Join(otherRunDir, "logs", "stage.log"), "other\n")
|
||||||
mustWriteFile(t, filepath.Join(spoolAudioDir, "speaker.flac"), "flac\n")
|
mustWriteFile(t, filepath.Join(spoolAudioDir, "speaker.flac"), "flac\n")
|
||||||
|
mustWriteFile(t, previousCachePath, "# previous recap\n")
|
||||||
|
|
||||||
localSourceAudio := filepath.Join(filepath.Dir(cfg.SessionPath), "audio", "alice.flac")
|
localSourceAudio := filepath.Join(filepath.Dir(cfg.SessionPath), "audio", "alice.flac")
|
||||||
mustWriteFile(t, localSourceAudio, "source\n")
|
mustWriteFile(t, localSourceAudio, "source\n")
|
||||||
@@ -305,6 +315,7 @@ func cleanupFixtureConfig(t *testing.T) (*config.Config, cleanupSeed) {
|
|||||||
otherRunDir: otherRunDir,
|
otherRunDir: otherRunDir,
|
||||||
spoolAudioDir: spoolAudioDir,
|
spoolAudioDir: spoolAudioDir,
|
||||||
localSourceAudio: localSourceAudio,
|
localSourceAudio: localSourceAudio,
|
||||||
|
previousCachePath: previousCachePath,
|
||||||
sessionPrefix: seed.S3SessionPrefix,
|
sessionPrefix: seed.S3SessionPrefix,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -318,11 +329,11 @@ func archiveStageCleanupFixture(t *testing.T) (*config.Config, cleanupSeed, stri
|
|||||||
Bucket: "my-dnd-archive",
|
Bucket: "my-dnd-archive",
|
||||||
RootPrefix: "dnd",
|
RootPrefix: "dnd",
|
||||||
}
|
}
|
||||||
cfg.Pipeline.Archive = &config.ArchiveConfig{
|
cfg.Pipeline.Publish = &config.PublishConfig{
|
||||||
Enabled: boolPtr(true),
|
Enabled: boolPtr(true),
|
||||||
UploadRun: boolPtr(true),
|
UploadRun: boolPtr(true),
|
||||||
PromoteArtifacts: []config.ArchivePromotionRule{
|
Outputs: []config.PublishOutputRule{
|
||||||
{Source: "narratio.transcript.trimmed", Dest: "transcripts/trimmed.json", Required: boolPtr(true)},
|
{Source: "narratio.transcript.final_trimmed", Dest: "transcripts/final.trimmed.json", Required: boolPtr(true)},
|
||||||
{Source: "narratio.artifact.session_recap", Dest: "artifacts/session_recap.md", Required: boolPtr(true)},
|
{Source: "narratio.artifact.session_recap", Dest: "artifacts/session_recap.md", Required: boolPtr(true)},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -360,14 +371,14 @@ func writeArchiveFixtureRunFiles(t *testing.T, runWorkDir, sessionRoot string) {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "prepare", "inputs", "session.yml"), "session_id: 2026-05-03\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "prepare", "inputs", "session.yml"), "session_id: 2026-05-03\n")
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "transcribe", "outputs", "transcripts", "raw", "speaker.json"), "{}\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "transcribe", "outputs", "transcripts", "raw", "speaker.json"), "{}\n")
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "trim", "outputs", "transcripts", "trimmed.json"), "{\"segments\":[]}\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "trim", "outputs", "transcripts", "final.trimmed.json"), "{\"segments\":[]}\n")
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "analyze", "outputs", "artifacts", "session_recap.md"), "# recap\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "analyze", "outputs", "artifacts", "session_recap.md"), "# recap\n")
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "polish", "reports", "audita.report.json"), "{}\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "polish", "reports", "audita.report.json"), "{}\n")
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "merge", "config", "seriatim.generated.yml"), "key: value\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "merge", "config", "seriatim.generated.yml"), "key: value\n")
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "logs", "audita.stderr.log"), "stderr\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "logs", "audita.stderr.log"), "stderr\n")
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "manifest.json"), "{}\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "manifest.json"), "{}\n")
|
||||||
|
|
||||||
mustWriteFile(t, filepath.Join(sessionRoot, "transcripts", "trimmed.json"), "{\"segments\":[]}\n")
|
mustWriteFile(t, filepath.Join(sessionRoot, "transcripts", "final.trimmed.json"), "{\"segments\":[]}\n")
|
||||||
mustWriteFile(t, filepath.Join(sessionRoot, "artifacts", "session_recap.md"), "# recap\n")
|
mustWriteFile(t, filepath.Join(sessionRoot, "artifacts", "session_recap.md"), "# recap\n")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
157
internal/app/remote_locks.go
Normal file
157
internal/app/remote_locks.go
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
type effectiveLocks struct {
|
||||||
|
Static []config.PublishLockRule
|
||||||
|
Remote []config.PublishLockRule
|
||||||
|
All []config.PublishLockRule
|
||||||
|
Key string
|
||||||
|
}
|
||||||
|
|
||||||
|
func remoteLocksKey(cfg *config.Config) (string, error) {
|
||||||
|
if cfg == nil || cfg.Pipeline == nil || cfg.Session == nil {
|
||||||
|
return "", fmt.Errorf("resolved config is required")
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Storage.S3 == nil {
|
||||||
|
return "", fmt.Errorf("pipeline.storage.s3 configuration is required")
|
||||||
|
}
|
||||||
|
sessionPrefix := artifacts.S3SessionPrefix(
|
||||||
|
cfg.Pipeline.Storage.S3.RootPrefix,
|
||||||
|
cfg.Session.Campaign,
|
||||||
|
cfg.Session.SessionID,
|
||||||
|
)
|
||||||
|
return artifacts.S3SessionLocksKey(sessionPrefix), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadRemoteLockStore(ctx context.Context, cfg *config.Config, store storage.ObjectStore) (*config.PublishLockStore, string, error) {
|
||||||
|
key, err := remoteLocksKey(cfg)
|
||||||
|
if err != nil {
|
||||||
|
return nil, "", err
|
||||||
|
}
|
||||||
|
exists, err := store.Exists(ctx, key)
|
||||||
|
if err != nil {
|
||||||
|
return nil, key, fmt.Errorf("check remote locks %q: %w", key, err)
|
||||||
|
}
|
||||||
|
if !exists {
|
||||||
|
return &config.PublishLockStore{}, key, nil
|
||||||
|
}
|
||||||
|
tmp, err := downloadObjectToTemp(ctx, store, key, "narratio-locks-*.yml")
|
||||||
|
if err != nil {
|
||||||
|
return nil, key, fmt.Errorf("download remote locks %q: %w", key, err)
|
||||||
|
}
|
||||||
|
defer func() { _ = os.Remove(tmp) }()
|
||||||
|
data, err := os.ReadFile(tmp)
|
||||||
|
if err != nil {
|
||||||
|
return nil, key, fmt.Errorf("read remote locks %q: %w", key, err)
|
||||||
|
}
|
||||||
|
lockStore, err := config.LoadPublishLockStoreBytes("s3://"+s3BucketName(cfg.Pipeline)+"/"+key, data, cfg.Pipeline.Scriptorium)
|
||||||
|
if err != nil {
|
||||||
|
return nil, key, err
|
||||||
|
}
|
||||||
|
return lockStore, key, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadEffectiveLocks(ctx context.Context, cfg *config.Config, store storage.ObjectStore) (*effectiveLocks, error) {
|
||||||
|
staticLocks := staticArchiveLocks(cfg)
|
||||||
|
if store == nil {
|
||||||
|
return &effectiveLocks{
|
||||||
|
Static: staticLocks,
|
||||||
|
All: append([]config.PublishLockRule(nil), staticLocks...),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
lockStore, key, err := loadRemoteLockStore(ctx, cfg, store)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
remoteLocks := append([]config.PublishLockRule(nil), lockStore.Locks...)
|
||||||
|
return &effectiveLocks{
|
||||||
|
Static: staticLocks,
|
||||||
|
Remote: remoteLocks,
|
||||||
|
All: config.MergePublishLockRules(staticLocks, remoteLocks),
|
||||||
|
Key: key,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func staticArchiveLocks(cfg *config.Config) []config.PublishLockRule {
|
||||||
|
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Publish == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return append([]config.PublishLockRule(nil), cfg.Pipeline.Publish.Locks...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func applyEffectiveLocks(cfg *config.Config, locks []config.PublishLockRule) {
|
||||||
|
if cfg == nil || cfg.Pipeline == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Publish == nil {
|
||||||
|
cfg.Pipeline.Publish = &config.PublishConfig{}
|
||||||
|
}
|
||||||
|
cfg.Pipeline.Publish.Locks = append([]config.PublishLockRule(nil), locks...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func uploadRemoteLockStore(ctx context.Context, store storage.ObjectStore, key string, lockStore *config.PublishLockStore) error {
|
||||||
|
data, err := config.MarshalPublishLockStore(lockStore)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
tmp, err := os.CreateTemp("", "narratio-locks-upload-*.yml")
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("create lock store temp file: %w", err)
|
||||||
|
}
|
||||||
|
tmpPath := tmp.Name()
|
||||||
|
defer func() { _ = os.Remove(tmpPath) }()
|
||||||
|
if _, err := tmp.Write(data); err != nil {
|
||||||
|
_ = tmp.Close()
|
||||||
|
return fmt.Errorf("write lock store temp file: %w", err)
|
||||||
|
}
|
||||||
|
if err := tmp.Close(); err != nil {
|
||||||
|
return fmt.Errorf("close lock store temp file: %w", err)
|
||||||
|
}
|
||||||
|
if _, err := store.Upload(ctx, tmpPath, key, storage.UploadOptions{ContentType: "application/x-yaml; charset=utf-8"}); err != nil {
|
||||||
|
return fmt.Errorf("upload remote locks %q: %w", key, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func lockSourceSet(locks []config.PublishLockRule) map[string]config.PublishLockRule {
|
||||||
|
out := make(map[string]config.PublishLockRule, len(locks))
|
||||||
|
for _, lock := range locks {
|
||||||
|
source := strings.TrimSpace(lock.Source)
|
||||||
|
if source == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
lock.Source = source
|
||||||
|
lock.Reason = strings.TrimSpace(lock.Reason)
|
||||||
|
out[source] = lock
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeLocalFile(path string, data []byte, force bool) error {
|
||||||
|
cleaned := filepath.Clean(strings.TrimSpace(path))
|
||||||
|
if cleaned == "" || cleaned == "." {
|
||||||
|
return fmt.Errorf("output path is required")
|
||||||
|
}
|
||||||
|
if !force {
|
||||||
|
if _, err := os.Stat(cleaned); err == nil {
|
||||||
|
return fmt.Errorf("output file %q already exists; pass --force to overwrite", cleaned)
|
||||||
|
} else if err != nil && !os.IsNotExist(err) {
|
||||||
|
return fmt.Errorf("check output file %q: %w", cleaned, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(filepath.Dir(cleaned), 0o755); err != nil {
|
||||||
|
return fmt.Errorf("create output directory: %w", err)
|
||||||
|
}
|
||||||
|
return os.WriteFile(cleaned, data, 0o644)
|
||||||
|
}
|
||||||
294
internal/app/remote_session_test.go
Normal file
294
internal/app/remote_session_test.go
Normal file
@@ -0,0 +1,294 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestExecuteRemoteSessionFallbackLoadsFromObjectStore(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
remoteKey := seedRemoteSessionConfig(t, fake, "2026-05-03", `session_id: 2026-05-03
|
||||||
|
inputs:
|
||||||
|
audio_s3:
|
||||||
|
prefix: audio/
|
||||||
|
`)
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if storeInitCalls != 1 {
|
||||||
|
t.Fatalf("object store init calls = %d, want 1", storeInitCalls)
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "narratio session plan: workdir prepared") {
|
||||||
|
t.Fatalf("stdout = %q, want plan output", stdout.String())
|
||||||
|
}
|
||||||
|
if _, ok := fake.Objects[remoteKey]; !ok {
|
||||||
|
t.Fatalf("remote session key %q was not seeded", remoteKey)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRemoteSessionFallbackLoadsSecretsBeforeObjectStoreInit(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
accessKeyEnv := "NARRATIO_TEST_REMOTE_SESSION_KEY_ID"
|
||||||
|
secretKeyEnv := "NARRATIO_TEST_REMOTE_SESSION_SECRET"
|
||||||
|
restoreEnvAfterTest(t, accessKeyEnv, secretKeyEnv)
|
||||||
|
secretsDir := t.TempDir()
|
||||||
|
mustWriteTestFile(t, filepath.Join(secretsDir, accessKeyEnv), "remote-session-key-id\n")
|
||||||
|
mustWriteTestFile(t, filepath.Join(secretsDir, secretKeyEnv), "remote-session-secret\n")
|
||||||
|
addSecretsToPipelineConfig(t, pipelinePath, secretsDir, accessKeyEnv, secretKeyEnv)
|
||||||
|
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
seedRemoteSessionConfig(t, fake, "2026-05-03", `session_id: 2026-05-03
|
||||||
|
inputs:
|
||||||
|
audio_s3:
|
||||||
|
prefix: audio/
|
||||||
|
`)
|
||||||
|
origStoreFn := newObjectStoreFromConfigFn
|
||||||
|
origSessionDefaults := append([]string(nil), config.DefaultSessionConfigSearchPaths...)
|
||||||
|
config.DefaultSessionConfigSearchPaths = []string{filepath.Join(t.TempDir(), "session.yml")}
|
||||||
|
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||||
|
if os.Getenv(accessKeyEnv) != "remote-session-key-id" || os.Getenv(secretKeyEnv) != "remote-session-secret" {
|
||||||
|
return nil, fmt.Errorf("secrets were not loaded before remote session object store init")
|
||||||
|
}
|
||||||
|
return fake, nil
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
newObjectStoreFromConfigFn = origStoreFn
|
||||||
|
config.DefaultSessionConfigSearchPaths = origSessionDefaults
|
||||||
|
})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stdout=%q stderr=%q", code, stdout.String(), stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteExplicitLocalSessionPrecedenceSkipsRemote(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if storeInitCalls != 0 {
|
||||||
|
t.Fatalf("object store init calls = %d, want 0", storeInitCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteLocalSessionDiscoveryPrecedenceSkipsRemote(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if storeInitCalls != 0 {
|
||||||
|
t.Fatalf("object store init calls = %d, want 0", storeInitCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRemoteSessionMissingObjectFailsClearly(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
missingSessionPath := filepath.Join(t.TempDir(), "session.yml")
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{missingSessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "remote session") || !strings.Contains(stderr.String(), "session.yml") || !strings.Contains(stderr.String(), "not found") {
|
||||||
|
t.Fatalf("stderr = %q, want remote session not found context", stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), missingSessionPath) {
|
||||||
|
t.Fatalf("stderr = %q, want local searched path", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRemoteSessionRequiresSessionID(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, &storage.FakeBackend{}, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "plan", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "plan: session_id is required") {
|
||||||
|
t.Fatalf("stderr = %q, want session_id guidance", stderr.String())
|
||||||
|
}
|
||||||
|
if storeInitCalls != 0 {
|
||||||
|
t.Fatalf("object store init calls = %d, want 0", storeInitCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRemoteSessionStorageInitErrorFailsClearly(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
origStoreFn := newObjectStoreFromConfigFn
|
||||||
|
origSessionDefaults := append([]string(nil), config.DefaultSessionConfigSearchPaths...)
|
||||||
|
config.DefaultSessionConfigSearchPaths = []string{filepath.Join(t.TempDir(), "session.yml")}
|
||||||
|
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||||
|
return nil, errors.New("storage unavailable")
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
newObjectStoreFromConfigFn = origStoreFn
|
||||||
|
config.DefaultSessionConfigSearchPaths = origSessionDefaults
|
||||||
|
})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "storage unavailable") || !strings.Contains(stderr.String(), "remote session") {
|
||||||
|
t.Fatalf("stderr = %q, want remote storage context", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRemoteSessionMalformedYAMLFailsStrictDecode(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
seedRemoteSessionConfig(t, fake, "2026-05-03", "session_id: 2026-05-03\nunknown: true\n")
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "strict decode failed") {
|
||||||
|
t.Fatalf("stderr = %q, want strict decode context", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRemoteSessionTemplateFailsConcreteSessionCheck(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
seedRemoteSessionConfig(t, fake, "2026-05-03", `session_id: "{{ session_id }}"
|
||||||
|
inputs:
|
||||||
|
audio_s3:
|
||||||
|
prefix: audio/
|
||||||
|
`)
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "session.yml must be concrete") || !strings.Contains(stderr.String(), "run narratio session init") {
|
||||||
|
t.Fatalf("stderr = %q, want concrete session guidance", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRemoteSessionMismatchFails(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
seedRemoteSessionConfig(t, fake, "2026-05-03", "session_id: 2026-05-04\ninputs:\n audio_s3:\n prefix: audio/\n")
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "session_id mismatch") {
|
||||||
|
t.Fatalf("stderr = %q, want session_id mismatch", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func restoreAppConfigTestGlobals(t *testing.T, fake *storage.FakeBackend, storeInitCalls *int, sessionDefaults []string) {
|
||||||
|
t.Helper()
|
||||||
|
origStoreFn := newObjectStoreFromConfigFn
|
||||||
|
origSessionDefaults := append([]string(nil), config.DefaultSessionConfigSearchPaths...)
|
||||||
|
config.DefaultSessionConfigSearchPaths = append([]string(nil), sessionDefaults...)
|
||||||
|
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||||
|
if storeInitCalls != nil {
|
||||||
|
(*storeInitCalls)++
|
||||||
|
}
|
||||||
|
return fake, nil
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
newObjectStoreFromConfigFn = origStoreFn
|
||||||
|
config.DefaultSessionConfigSearchPaths = origSessionDefaults
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func seedRemoteSessionConfig(t *testing.T, fake *storage.FakeBackend, sessionID, content string) string {
|
||||||
|
t.Helper()
|
||||||
|
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", sessionID)
|
||||||
|
remoteKey := artifacts.S3SessionConfigKey(sessionPrefix)
|
||||||
|
fake.SeedObject(storage.FakeObject{
|
||||||
|
Key: remoteKey,
|
||||||
|
Data: []byte(content),
|
||||||
|
ETag: "remote-session-etag",
|
||||||
|
})
|
||||||
|
return remoteKey
|
||||||
|
}
|
||||||
|
|
||||||
|
func addSecretsToPipelineConfig(t *testing.T, pipelinePath, secretsDir, accessKeyEnv, secretKeyEnv string) {
|
||||||
|
t.Helper()
|
||||||
|
pipelineData, err := os.ReadFile(pipelinePath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read pipeline: %v", err)
|
||||||
|
}
|
||||||
|
pipelineYAML := strings.Replace(
|
||||||
|
string(pipelineData),
|
||||||
|
"storage:\n backend: s3\n s3:\n bucket: test-bucket\n",
|
||||||
|
"storage:\n backend: s3\n s3:\n bucket: test-bucket\n access_key_id_env: "+accessKeyEnv+"\n secret_access_key_env: "+secretKeyEnv+"\nsecrets:\n env_dir: "+secretsDir+"\n",
|
||||||
|
1,
|
||||||
|
)
|
||||||
|
if err := os.WriteFile(pipelinePath, []byte(pipelineYAML), 0o644); err != nil {
|
||||||
|
t.Fatalf("write pipeline: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -8,6 +8,7 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"os"
|
"os"
|
||||||
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
@@ -22,23 +23,29 @@ var executeRestorePlanFn = executeRestorePlan
|
|||||||
|
|
||||||
// Restore validates restore CLI/config inputs and storage preflight for future restore phases.
|
// Restore validates restore CLI/config inputs and storage preflight for future restore phases.
|
||||||
func Restore(ctx context.Context, args []string, out io.Writer) error {
|
func Restore(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
positionalSessionID, args := pullLeadingSessionID(args)
|
||||||
fs := flag.NewFlagSet("restore", flag.ContinueOnError)
|
fs := flag.NewFlagSet("restore", flag.ContinueOnError)
|
||||||
fs.SetOutput(out)
|
fs.SetOutput(out)
|
||||||
|
|
||||||
var pipelinePath string
|
var pipelinePath string
|
||||||
|
var campaignPath string
|
||||||
|
var campaignFilePath string
|
||||||
var sessionPath string
|
var sessionPath string
|
||||||
var sessionID string
|
var sessionID string
|
||||||
|
var previousSessionID string
|
||||||
var dryRun bool
|
var dryRun bool
|
||||||
var force bool
|
var force bool
|
||||||
var includeAudio bool
|
var includeAudio bool
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||||
|
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||||
|
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||||
fs.StringVar(&sessionID, "session-id", "", "session identifier for session.yml templates")
|
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||||
fs.BoolVar(&dryRun, "dry-run", false, "plan restore actions without writing local files")
|
fs.BoolVar(&dryRun, "dry-run", false, "plan restore actions without writing local files")
|
||||||
fs.BoolVar(&force, "force", false, "overwrite local conflicts with remote state")
|
fs.BoolVar(&force, "force", false, "overwrite local conflicts with remote state")
|
||||||
fs.BoolVar(&includeAudio, "include-audio", false, "include archived session-level audio objects")
|
fs.BoolVar(&includeAudio, "include-audio", false, "include archived session-level audio objects")
|
||||||
fs.Usage = func() {
|
fs.Usage = func() {
|
||||||
_, _ = fmt.Fprintln(out, "Usage: narratio restore [--config <path>] [--session <path>] [--session-id <value>] [--dry-run] [--force] [--include-audio]")
|
_, _ = fmt.Fprintln(out, "Usage: narratio session restore <session_id> [--config <path>] [--campaign <id>] [--campaign-file <path>] [--session <path>] [--previous-session-id <value>] [--dry-run] [--force] [--include-audio]")
|
||||||
_, _ = fmt.Fprintln(out)
|
_, _ = fmt.Fprintln(out)
|
||||||
_, _ = fmt.Fprintln(out, "Flags:")
|
_, _ = fmt.Fprintln(out, "Flags:")
|
||||||
fs.PrintDefaults()
|
fs.PrintDefaults()
|
||||||
@@ -50,20 +57,24 @@ func Restore(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
}
|
}
|
||||||
return fmt.Errorf("restore: invalid flags: %w", err)
|
return fmt.Errorf("restore: invalid flags: %w", err)
|
||||||
}
|
}
|
||||||
|
if positionalSessionID == "" {
|
||||||
|
if err := applyParsedSessionIDArg("restore", fs, &sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
if fs.NArg() != 0 {
|
if fs.NArg() != 0 {
|
||||||
return fmt.Errorf("restore: unexpected positional arguments")
|
return fmt.Errorf("restore: unexpected positional arguments")
|
||||||
}
|
}
|
||||||
resolvedPipelinePath, err := resolvePipelineConfigPath(pipelinePath)
|
if err := applyPositionalSessionID("restore", positionalSessionID, &sessionID); err != nil {
|
||||||
if err != nil {
|
return err
|
||||||
return fmt.Errorf("restore: %w", err)
|
|
||||||
}
|
}
|
||||||
resolvedSessionPath, err := resolveSessionConfigPath(sessionPath)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("restore: %w", err)
|
|
||||||
}
|
}
|
||||||
|
if strings.TrimSpace(sessionID) == "" {
|
||||||
cfg, err := config.LoadWithSessionOptions(resolvedPipelinePath, resolvedSessionPath, config.SessionLoadOptions{
|
return fmt.Errorf("restore: session_id is required")
|
||||||
|
}
|
||||||
|
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
||||||
SessionID: sessionID,
|
SessionID: sessionID,
|
||||||
|
PreviousSessionID: previousSessionID,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("restore: %w", err)
|
return fmt.Errorf("restore: %w", err)
|
||||||
@@ -71,11 +82,7 @@ func Restore(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
if err := config.Validate(cfg); err != nil {
|
if err := config.Validate(cfg); err != nil {
|
||||||
return fmt.Errorf("restore: %w", err)
|
return fmt.Errorf("restore: %w", err)
|
||||||
}
|
}
|
||||||
if _, err := loadSecretsFromConfig(cfg, logging.NewLogger(os.Stderr, slog.LevelInfo)); err != nil {
|
objectStore, err := newCommandObjectStore(ctx, cfg, logging.NewLogger(os.Stderr, slog.LevelInfo))
|
||||||
return fmt.Errorf("restore: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
objectStore, err := newObjectStoreFromConfigFn(ctx, cfg)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("restore: %w", err)
|
return fmt.Errorf("restore: %w", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ import (
|
|||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/audio"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
)
|
)
|
||||||
@@ -92,6 +93,10 @@ func executeRestoreDownloadAction(
|
|||||||
return fmt.Errorf("restore plan local path mismatch for %q", action.LocalRelativePath)
|
return fmt.Errorf("restore plan local path mismatch for %q", action.LocalRelativePath)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if restoreActionIsAudio(action) {
|
||||||
|
return executeRestoreAudioAction(ctx, cfg, safeLocalPath, action, store)
|
||||||
|
}
|
||||||
|
|
||||||
tmpPath, err := downloadObjectToSiblingTemp(ctx, store, action.RemoteKey, safeLocalPath)
|
tmpPath, err := downloadObjectToSiblingTemp(ctx, store, action.RemoteKey, safeLocalPath)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("download to temp file: %w", err)
|
return fmt.Errorf("download to temp file: %w", err)
|
||||||
@@ -120,6 +125,38 @@ func executeRestoreDownloadAction(
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func executeRestoreAudioAction(
|
||||||
|
ctx context.Context,
|
||||||
|
cfg *config.Config,
|
||||||
|
safeLocalPath string,
|
||||||
|
action RestoreAction,
|
||||||
|
store storage.ObjectStore,
|
||||||
|
) error {
|
||||||
|
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Storage.S3 == nil || cfg.Session == nil {
|
||||||
|
return fmt.Errorf("resolved s3 config and session are required")
|
||||||
|
}
|
||||||
|
spoolDir := artifacts.SessionSpoolRestoreAudioDir(cfg.Pipeline.Spool.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
spoolPath := filepath.Join(spoolDir, filepath.Base(safeLocalPath))
|
||||||
|
cacheEnabled := cfg.Pipeline.Cache.S3Audio == nil || *cfg.Pipeline.Cache.S3Audio
|
||||||
|
_, err := audio.MaterializeS3Audio(ctx, audio.S3MaterializeRequest{
|
||||||
|
Store: store,
|
||||||
|
Object: storage.ObjectInfo{
|
||||||
|
Key: action.RemoteKey,
|
||||||
|
Size: action.Size,
|
||||||
|
ETag: action.ETag,
|
||||||
|
},
|
||||||
|
Bucket: strings.TrimSpace(cfg.Pipeline.Storage.S3.Bucket),
|
||||||
|
CacheRoot: strings.TrimSpace(cfg.Pipeline.Cache.Root),
|
||||||
|
CacheEnabled: cacheEnabled,
|
||||||
|
SpoolPath: spoolPath,
|
||||||
|
DestPath: safeLocalPath,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("materialize audio: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
func downloadObjectToSiblingTemp(ctx context.Context, store storage.ObjectStore, remoteKey, destPath string) (string, error) {
|
func downloadObjectToSiblingTemp(ctx context.Context, store storage.ObjectStore, remoteKey, destPath string) (string, error) {
|
||||||
if strings.TrimSpace(destPath) == "" {
|
if strings.TrimSpace(destPath) == "" {
|
||||||
return "", fmt.Errorf("destination path is required")
|
return "", fmt.Errorf("destination path is required")
|
||||||
|
|||||||
@@ -18,10 +18,10 @@ import (
|
|||||||
|
|
||||||
func TestExecuteRestoreNonDryRunRestoresDurableFiles(t *testing.T) {
|
func TestExecuteRestoreNonDryRunRestoresDurableFiles(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
fake := &storage.FakeBackend{}
|
fake := &storage.FakeBackend{}
|
||||||
cfg, sessionPrefix, manifestKey, runIDKey := seedRestoreCommittedState(t, fake, pipelinePath, sessionPath)
|
cfg, sessionPrefix, manifestKey, runIDKey := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||||
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte(`{"segments":[1,2,3]}`))
|
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte(`{"segments":[1,2,3]}`))
|
||||||
seedRestoreObject(fake, sessionPrefix+"artifacts/session_recap.md", []byte("# recap\n"))
|
seedRestoreObject(fake, sessionPrefix+"artifacts/session_recap.md", []byte("# recap\n"))
|
||||||
seedRestoreObject(fake, sessionPrefix+"audio/alice.flac", []byte("remote-audio"))
|
seedRestoreObject(fake, sessionPrefix+"audio/alice.flac", []byte("remote-audio"))
|
||||||
@@ -32,7 +32,7 @@ func TestExecuteRestoreNonDryRunRestoresDurableFiles(t *testing.T) {
|
|||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"restore", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
@@ -64,17 +64,17 @@ func TestExecuteRestoreNonDryRunRestoresDurableFiles(t *testing.T) {
|
|||||||
|
|
||||||
func TestExecuteRestoreIncludeAudioRestoresAudio(t *testing.T) {
|
func TestExecuteRestoreIncludeAudioRestoresAudio(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
fake := &storage.FakeBackend{}
|
fake := &storage.FakeBackend{}
|
||||||
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, sessionPath)
|
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||||
seedRestoreObject(fake, sessionPrefix+"audio/alice.flac", []byte("remote-audio"))
|
seedRestoreObject(fake, sessionPrefix+"audio/alice.flac", []byte("remote-audio"))
|
||||||
|
|
||||||
restoreWithStoreAndRealPhases(t, fake)
|
restoreWithStoreAndRealPhases(t, fake)
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"restore", "--config", pipelinePath, "--session", sessionPath, "--include-audio"}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--include-audio"}, &stdout, &stderr)
|
||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
@@ -87,12 +87,110 @@ func TestExecuteRestoreIncludeAudioRestoresAudio(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteRestoreConflictWithoutForceDoesNotOverwrite(t *testing.T) {
|
func TestExecuteRestoreIncludeAudioUsesCacheAfterWorkspaceDeletion(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
fake := &storage.FakeBackend{}
|
fake := &storage.FakeBackend{}
|
||||||
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, sessionPath)
|
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||||
|
audioKey := sessionPrefix + "audio/alice.flac"
|
||||||
|
seedRestoreObject(fake, audioKey, []byte("remote-audio"))
|
||||||
|
|
||||||
|
restoreWithStoreAndRealPhases(t, fake)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--include-audio"}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("first restore exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if got := fakeDownloadCount(fake, audioKey); got != 1 {
|
||||||
|
t.Fatalf("audio downloads after first restore = %d, want 1", got)
|
||||||
|
}
|
||||||
|
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
mustReadEquals(t, filepath.Join(sessionRoot, "audio", "alice.flac"), "remote-audio")
|
||||||
|
|
||||||
|
if err := os.RemoveAll(sessionRoot); err != nil {
|
||||||
|
t.Fatalf("remove session root: %v", err)
|
||||||
|
}
|
||||||
|
stdout.Reset()
|
||||||
|
stderr.Reset()
|
||||||
|
code = Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--include-audio"}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("second restore exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if got := fakeDownloadCount(fake, audioKey); got != 1 {
|
||||||
|
t.Fatalf("audio downloads after cached restore = %d, want still 1", got)
|
||||||
|
}
|
||||||
|
mustReadEquals(t, filepath.Join(sessionRoot, "audio", "alice.flac"), "remote-audio")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRestoreRestoresPreviousCacheWhenPresent(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
appendRestoreWorkflowPreviousInputConfig(t, pipelinePath, sessionPath)
|
||||||
|
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
cfg, _, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||||
|
seedRestorePreviousCurrent(t, fake, cfg, "# previous recap\n")
|
||||||
|
|
||||||
|
restoreWithStoreAndRealPhases(t, fake)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
previousManifestBytes, err := os.ReadFile(filepath.Join(sessionRoot, "previous", "manifest.json"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read restored previous manifest: %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(string(previousManifestBytes), `"session_id":"2026-04-26"`) {
|
||||||
|
t.Fatalf("restored previous manifest = %q, want previous session id", string(previousManifestBytes))
|
||||||
|
}
|
||||||
|
mustReadEquals(t, filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md"), "# previous recap\n")
|
||||||
|
report := mustReadRestoreReport(t, filepath.Join(sessionRoot, "reports", "restore-latest.json"))
|
||||||
|
if report.Execution.Downloaded != 3 {
|
||||||
|
t.Fatalf("report execution.downloaded = %d, want 3", report.Execution.Downloaded)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRestoreDryRunReportsPreviousCacheWithoutWriting(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
appendRestoreWorkflowPreviousInputConfig(t, pipelinePath, sessionPath)
|
||||||
|
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
cfg, _, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||||
|
seedRestorePreviousCurrent(t, fake, cfg, "# previous recap\n")
|
||||||
|
|
||||||
|
restoreWithStoreAndRealPhases(t, fake)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--dry-run"}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "previous/artifacts/session_recap.md") {
|
||||||
|
t.Fatalf("stdout = %q, want planned previous-cache artifact", stdout.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
if _, err := os.Stat(filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md")); !os.IsNotExist(err) {
|
||||||
|
t.Fatalf("previous artifact should not be written during dry-run; stat err=%v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRestoreConflictWithoutForceDoesNotOverwrite(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||||
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
||||||
|
|
||||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
@@ -102,7 +200,7 @@ func TestExecuteRestoreConflictWithoutForceDoesNotOverwrite(t *testing.T) {
|
|||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"restore", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
if code == 0 {
|
if code == 0 {
|
||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
@@ -121,10 +219,10 @@ func TestExecuteRestoreConflictWithoutForceDoesNotOverwrite(t *testing.T) {
|
|||||||
|
|
||||||
func TestExecuteRestoreForceOverwritesDifferingFile(t *testing.T) {
|
func TestExecuteRestoreForceOverwritesDifferingFile(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
fake := &storage.FakeBackend{}
|
fake := &storage.FakeBackend{}
|
||||||
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, sessionPath)
|
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||||
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
||||||
|
|
||||||
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
@@ -134,7 +232,7 @@ func TestExecuteRestoreForceOverwritesDifferingFile(t *testing.T) {
|
|||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"restore", "--config", pipelinePath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
@@ -145,12 +243,35 @@ func TestExecuteRestoreForceOverwritesDifferingFile(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteRestoreLockConflictFailsAndWritesNothing(t *testing.T) {
|
func TestExecuteRestoreForceOverwritesDifferingPreviousCacheFile(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
appendRestoreWorkflowPreviousInputConfig(t, pipelinePath, sessionPath)
|
||||||
|
|
||||||
fake := &storage.FakeBackend{}
|
fake := &storage.FakeBackend{}
|
||||||
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, sessionPath)
|
cfg, _, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||||
|
seedRestorePreviousCurrent(t, fake, cfg, "# remote previous recap\n")
|
||||||
|
|
||||||
|
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
mustWriteTestFile(t, filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md"), "# local previous recap\n")
|
||||||
|
|
||||||
|
restoreWithStoreAndRealPhases(t, fake)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
mustReadEquals(t, filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md"), "# remote previous recap\n")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRestoreLockConflictFailsAndWritesNothing(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
cfg, sessionPrefix, _, _ := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||||
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
||||||
|
|
||||||
store := artifacts.NewLocalStore(workspaceRoot)
|
store := artifacts.NewLocalStore(workspaceRoot)
|
||||||
@@ -164,7 +285,7 @@ func TestExecuteRestoreLockConflictFailsAndWritesNothing(t *testing.T) {
|
|||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"restore", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
if code == 0 {
|
if code == 0 {
|
||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
@@ -180,10 +301,10 @@ func TestExecuteRestoreLockConflictFailsAndWritesNothing(t *testing.T) {
|
|||||||
|
|
||||||
func TestExecuteRestoreInvalidManifestDoesNotCorruptExistingManifest(t *testing.T) {
|
func TestExecuteRestoreInvalidManifestDoesNotCorruptExistingManifest(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
base := &storage.FakeBackend{}
|
base := &storage.FakeBackend{}
|
||||||
cfg, sessionPrefix, manifestKey, _ := seedRestoreCommittedState(t, base, pipelinePath, sessionPath)
|
cfg, sessionPrefix, manifestKey, _ := seedRestoreCommittedState(t, base, pipelinePath, campaignPath, sessionPath)
|
||||||
seedRestoreObject(base, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
seedRestoreObject(base, sessionPrefix+"transcripts/full.json", []byte("remote-transcript"))
|
||||||
|
|
||||||
toggled := &stagedManifestDownloadStore{
|
toggled := &stagedManifestDownloadStore{
|
||||||
@@ -211,7 +332,7 @@ func TestExecuteRestoreInvalidManifestDoesNotCorruptExistingManifest(t *testing.
|
|||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"restore", "--config", pipelinePath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||||
if code == 0 {
|
if code == 0 {
|
||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
@@ -295,9 +416,9 @@ func restoreWithStoreAndRealPhases(t *testing.T, objectStore storage.ObjectStore
|
|||||||
executeRestorePlanFn = executeRestorePlan
|
executeRestorePlanFn = executeRestorePlan
|
||||||
}
|
}
|
||||||
|
|
||||||
func seedRestoreCommittedState(t *testing.T, fake *storage.FakeBackend, pipelinePath, sessionPath string) (*config.Config, string, string, string) {
|
func seedRestoreCommittedState(t *testing.T, fake *storage.FakeBackend, pipelinePath, campaignPath, sessionPath string) (*config.Config, string, string, string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
cfg, err := config.LoadWithSessionOptions(pipelinePath, sessionPath, config.SessionLoadOptions{})
|
cfg, err := config.LoadWithSessionOptions(pipelinePath, campaignPath, sessionPath, config.SessionLoadOptions{})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("LoadWithSessionOptions() error = %v", err)
|
t.Fatalf("LoadWithSessionOptions() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -314,6 +435,50 @@ func seedRestoreCommittedState(t *testing.T, fake *storage.FakeBackend, pipeline
|
|||||||
return cfg, sessionPrefix, manifestKey, runIDKey
|
return cfg, sessionPrefix, manifestKey, runIDKey
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func appendRestoreWorkflowPreviousInputConfig(t *testing.T, pipelinePath, sessionPath string) {
|
||||||
|
t.Helper()
|
||||||
|
appendRestoreWorkflowScriptoriumConfig(t, pipelinePath, `
|
||||||
|
scriptorium:
|
||||||
|
binary: scriptorium
|
||||||
|
artifacts:
|
||||||
|
session_recap:
|
||||||
|
enabled: true
|
||||||
|
prompt_id: dnd.session_recap
|
||||||
|
output_path: artifacts/session_recap.md
|
||||||
|
inputs:
|
||||||
|
previous_recap:
|
||||||
|
source: narratio.previous_session.artifact.session_recap
|
||||||
|
required: true
|
||||||
|
`)
|
||||||
|
appendRestoreWorkflowScriptoriumConfig(t, sessionPath, `
|
||||||
|
previous_session_id: 2026-04-26
|
||||||
|
`)
|
||||||
|
}
|
||||||
|
|
||||||
|
func seedRestorePreviousCurrent(t *testing.T, fake *storage.FakeBackend, cfg *config.Config, artifactBody string) {
|
||||||
|
t.Helper()
|
||||||
|
seedRestorePreviousCurrentManifestOnly(t, fake, cfg)
|
||||||
|
previousPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.PreviousSessionID)
|
||||||
|
seedRestoreObject(fake, previousPrefix+"artifacts/session_recap.md", []byte(artifactBody))
|
||||||
|
}
|
||||||
|
|
||||||
|
func seedRestorePreviousCurrentManifestOnly(t *testing.T, fake *storage.FakeBackend, cfg *config.Config) {
|
||||||
|
t.Helper()
|
||||||
|
previousPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.PreviousSessionID)
|
||||||
|
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(previousPrefix)
|
||||||
|
previousRunID := "20260426T010203Z-a1b2c3d4"
|
||||||
|
seedRestoreObject(fake, runIDKey, []byte(previousRunID+"\n"))
|
||||||
|
|
||||||
|
m := manifest.New(cfg.Session.PreviousSessionID, nowUTC())
|
||||||
|
m.Campaign = cfg.Session.Campaign
|
||||||
|
m.RunID = previousRunID
|
||||||
|
data, err := json.Marshal(m)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("marshal previous restore manifest: %v", err)
|
||||||
|
}
|
||||||
|
seedRestoreObject(fake, manifestKey, append(data, '\n'))
|
||||||
|
}
|
||||||
|
|
||||||
func mustReadEquals(t *testing.T, path, want string) {
|
func mustReadEquals(t *testing.T, path, want string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
data, err := os.ReadFile(path)
|
data, err := os.ReadFile(path)
|
||||||
@@ -325,6 +490,16 @@ func mustReadEquals(t *testing.T, path, want string) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func fakeDownloadCount(fake *storage.FakeBackend, key string) int {
|
||||||
|
count := 0
|
||||||
|
for _, call := range fake.Downloads {
|
||||||
|
if call.Key == key {
|
||||||
|
count++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return count
|
||||||
|
}
|
||||||
|
|
||||||
type stagedManifestDownloadStore struct {
|
type stagedManifestDownloadStore struct {
|
||||||
delegate *storage.FakeBackend
|
delegate *storage.FakeBackend
|
||||||
manifestKey string
|
manifestKey string
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/previouscache"
|
||||||
)
|
)
|
||||||
|
|
||||||
// RestoreActionKind identifies one restore planner action.
|
// RestoreActionKind identifies one restore planner action.
|
||||||
@@ -114,6 +115,12 @@ func buildRestorePlan(ctx context.Context, cfg *config.Config, current *RemoteCu
|
|||||||
actions = append(actions, action)
|
actions = append(actions, action)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
previousActions, err := buildPreviousCacheRestoreActions(ctx, cfg, sessionPaths, store, opts.Force)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
actions = append(actions, previousActions...)
|
||||||
|
|
||||||
sort.Slice(actions, func(i, j int) bool {
|
sort.Slice(actions, func(i, j int) bool {
|
||||||
if actions[i].LocalRelativePath == actions[j].LocalRelativePath {
|
if actions[i].LocalRelativePath == actions[j].LocalRelativePath {
|
||||||
return actions[i].RemoteKey < actions[j].RemoteKey
|
return actions[i].RemoteKey < actions[j].RemoteKey
|
||||||
@@ -194,6 +201,9 @@ func restoreLocalRelativePathForKey(sessionPrefix, currentManifestKey, key strin
|
|||||||
if cleanRel == config.PathArtifactsDirSegment || strings.HasPrefix(cleanRel, config.PathArtifactsDirSegment+"/") {
|
if cleanRel == config.PathArtifactsDirSegment || strings.HasPrefix(cleanRel, config.PathArtifactsDirSegment+"/") {
|
||||||
return cleanRel, true, nil
|
return cleanRel, true, nil
|
||||||
}
|
}
|
||||||
|
if cleanRel == config.PathPreviousDirSegment || strings.HasPrefix(cleanRel, config.PathPreviousDirSegment+"/") {
|
||||||
|
return "", false, nil
|
||||||
|
}
|
||||||
if includeAudio && (cleanRel == config.PathAudioDirSegment || strings.HasPrefix(cleanRel, config.PathAudioDirSegment+"/")) {
|
if includeAudio && (cleanRel == config.PathAudioDirSegment || strings.HasPrefix(cleanRel, config.PathAudioDirSegment+"/")) {
|
||||||
return cleanRel, true, nil
|
return cleanRel, true, nil
|
||||||
}
|
}
|
||||||
@@ -220,6 +230,35 @@ func joinWithinSessionRoot(sessionRoot, relative string) (string, error) {
|
|||||||
return abs, nil
|
return abs, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func buildPreviousCacheRestoreActions(
|
||||||
|
ctx context.Context,
|
||||||
|
cfg *config.Config,
|
||||||
|
sessionPaths artifacts.SessionPaths,
|
||||||
|
store storage.ObjectStore,
|
||||||
|
force bool,
|
||||||
|
) ([]RestoreAction, error) {
|
||||||
|
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Scriptorium == nil {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
requirements := artifacts.CollectPreviousArtifactRequirements(cfg.Pipeline.Scriptorium.Artifacts)
|
||||||
|
if len(requirements) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
plan, err := previouscache.BuildPlan(ctx, cfg, sessionPaths, requirements, store)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("plan previous-session cache restore: %w", err)
|
||||||
|
}
|
||||||
|
actions := make([]RestoreAction, 0, len(plan.Records))
|
||||||
|
for _, record := range plan.Records {
|
||||||
|
action, err := classifyRestoreAction(ctx, store, storage.ObjectInfo{Key: record.RemoteKey}, record.LocalRelativePath, record.LocalPath, force)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("classify previous-session cache object %q: %w", record.RemoteKey, err)
|
||||||
|
}
|
||||||
|
actions = append(actions, action)
|
||||||
|
}
|
||||||
|
return actions, nil
|
||||||
|
}
|
||||||
|
|
||||||
func classifyRestoreAction(
|
func classifyRestoreAction(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
store storage.ObjectStore,
|
store storage.ObjectStore,
|
||||||
@@ -254,6 +293,35 @@ func classifyRestoreAction(
|
|||||||
return action, nil
|
return action, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if restoreRelativePathIsAudio(localRelPath) {
|
||||||
|
if object.Size > 0 {
|
||||||
|
if info.Size() == object.Size {
|
||||||
|
action.Kind = RestoreActionSkipSame
|
||||||
|
action.SameLocal = true
|
||||||
|
action.Reason = "local audio size matches remote content"
|
||||||
|
return action, nil
|
||||||
|
}
|
||||||
|
if force {
|
||||||
|
action.Kind = RestoreActionDownload
|
||||||
|
action.Reason = "local audio differs (size mismatch); overwrite with --force"
|
||||||
|
return action, nil
|
||||||
|
}
|
||||||
|
action.Kind = RestoreActionConflict
|
||||||
|
action.Conflict = true
|
||||||
|
action.Reason = "local audio differs (size mismatch)"
|
||||||
|
return action, nil
|
||||||
|
}
|
||||||
|
if force {
|
||||||
|
action.Kind = RestoreActionDownload
|
||||||
|
action.Reason = "local audio exists; remote size unavailable; overwrite with --force"
|
||||||
|
return action, nil
|
||||||
|
}
|
||||||
|
action.Kind = RestoreActionConflict
|
||||||
|
action.Conflict = true
|
||||||
|
action.Reason = "local audio exists; remote size unavailable"
|
||||||
|
return action, nil
|
||||||
|
}
|
||||||
|
|
||||||
if object.Size > 0 && info.Size() != object.Size {
|
if object.Size > 0 && info.Size() != object.Size {
|
||||||
if force {
|
if force {
|
||||||
action.Kind = RestoreActionDownload
|
action.Kind = RestoreActionDownload
|
||||||
@@ -300,6 +368,15 @@ func classifyRestoreAction(
|
|||||||
return action, nil
|
return action, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func restoreActionIsAudio(action RestoreAction) bool {
|
||||||
|
return restoreRelativePathIsAudio(action.LocalRelativePath)
|
||||||
|
}
|
||||||
|
|
||||||
|
func restoreRelativePathIsAudio(rel string) bool {
|
||||||
|
cleanRel := path.Clean(strings.TrimSpace(rel))
|
||||||
|
return cleanRel == config.PathAudioDirSegment || strings.HasPrefix(cleanRel, config.PathAudioDirSegment+"/")
|
||||||
|
}
|
||||||
|
|
||||||
func writeRestorePlan(out io.Writer, current *RemoteCurrentState, plan *RestorePlan, opts RestorePlanOptions) error {
|
func writeRestorePlan(out io.Writer, current *RemoteCurrentState, plan *RestorePlan, opts RestorePlanOptions) error {
|
||||||
if out == nil {
|
if out == nil {
|
||||||
return fmt.Errorf("output writer is required")
|
return fmt.Errorf("output writer is required")
|
||||||
|
|||||||
@@ -58,6 +58,139 @@ func TestRestorePlanIncludeAudio(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRestorePlanExistingAudioUsesSizeWithoutRemoteChecksumDownload(t *testing.T) {
|
||||||
|
cfg := restorePlanConfig(t)
|
||||||
|
current := restorePlanCurrentState(t, cfg)
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
|
||||||
|
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||||
|
seedRestoreObject(store, current.SessionPrefix+"audio/alice.flac", []byte("audio"))
|
||||||
|
sessionRoot := artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
mustWriteTestFile(t, filepath.Join(sessionRoot, "audio", "alice.flac"), "local")
|
||||||
|
|
||||||
|
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{IncludeAudio: true})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||||
|
}
|
||||||
|
if len(store.Downloads) != 0 {
|
||||||
|
t.Fatalf("downloads = %d, want no remote checksum download for audio", len(store.Downloads))
|
||||||
|
}
|
||||||
|
actionByRel := map[string]RestoreAction{}
|
||||||
|
for _, action := range plan.Actions {
|
||||||
|
actionByRel[action.LocalRelativePath] = action
|
||||||
|
}
|
||||||
|
audioAction := actionByRel["audio/alice.flac"]
|
||||||
|
if audioAction.Kind != RestoreActionSkipSame {
|
||||||
|
t.Fatalf("audio action kind = %q, want %q", audioAction.Kind, RestoreActionSkipSame)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRestorePlanIncludesPreviousCacheByDefault(t *testing.T) {
|
||||||
|
cfg := restorePlanConfig(t)
|
||||||
|
configureRestorePlanPreviousRequirement(cfg, true)
|
||||||
|
current := restorePlanCurrentState(t, cfg)
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
|
||||||
|
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||||
|
seedRestorePreviousCurrent(t, store, cfg, "# previous recap\n")
|
||||||
|
|
||||||
|
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
got := actionRelPaths(plan.Actions)
|
||||||
|
want := []string{"manifest.json", "previous/artifacts/session_recap.md", "previous/manifest.json"}
|
||||||
|
if !reflect.DeepEqual(got, want) {
|
||||||
|
t.Fatalf("action local paths = %#v, want %#v", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRestorePlanIgnoresCurrentSessionArchivedPreviousCache(t *testing.T) {
|
||||||
|
cfg := restorePlanConfig(t)
|
||||||
|
current := restorePlanCurrentState(t, cfg)
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
|
||||||
|
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||||
|
seedRestoreObject(store, current.SessionPrefix+"previous/manifest.json", []byte(`{"session_id":"2026-04-26"}`))
|
||||||
|
seedRestoreObject(store, current.SessionPrefix+"previous/artifacts/session_recap.md", []byte("# previous recap\n"))
|
||||||
|
|
||||||
|
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
got := actionRelPaths(plan.Actions)
|
||||||
|
want := []string{"manifest.json"}
|
||||||
|
if !reflect.DeepEqual(got, want) {
|
||||||
|
t.Fatalf("action local paths = %#v, want %#v", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRestorePlanMissingOptionalPreviousCacheSkipsArtifact(t *testing.T) {
|
||||||
|
cfg := restorePlanConfig(t)
|
||||||
|
configureRestorePlanPreviousRequirement(cfg, false)
|
||||||
|
current := restorePlanCurrentState(t, cfg)
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
|
||||||
|
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||||
|
seedRestorePreviousCurrentManifestOnly(t, store, cfg)
|
||||||
|
|
||||||
|
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
got := actionRelPaths(plan.Actions)
|
||||||
|
want := []string{"manifest.json", "previous/manifest.json"}
|
||||||
|
if !reflect.DeepEqual(got, want) {
|
||||||
|
t.Fatalf("action local paths = %#v, want %#v", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRestorePlanMissingRequiredPreviousCacheFails(t *testing.T) {
|
||||||
|
cfg := restorePlanConfig(t)
|
||||||
|
configureRestorePlanPreviousRequirement(cfg, true)
|
||||||
|
current := restorePlanCurrentState(t, cfg)
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
|
||||||
|
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||||
|
seedRestorePreviousCurrentManifestOnly(t, store, cfg)
|
||||||
|
|
||||||
|
_, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "required previous-session artifact") {
|
||||||
|
t.Fatalf("buildRestorePlan() error = %v, want required previous artifact failure", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRestorePlanPreviousCacheConflictRequiresForce(t *testing.T) {
|
||||||
|
cfg := restorePlanConfig(t)
|
||||||
|
configureRestorePlanPreviousRequirement(cfg, true)
|
||||||
|
current := restorePlanCurrentState(t, cfg)
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
|
||||||
|
seedRestoreObject(store, current.CurrentManifestKey, []byte(`{"session_id":"2026-05-03"}`))
|
||||||
|
seedRestorePreviousCurrent(t, store, cfg, "# remote previous recap\n")
|
||||||
|
sessionRoot := artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
mustWriteTestFile(t, filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md"), "# local previous recap\n")
|
||||||
|
|
||||||
|
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("buildRestorePlan() error = %v", err)
|
||||||
|
}
|
||||||
|
if plan.ConflictCount != 1 {
|
||||||
|
t.Fatalf("ConflictCount = %d, want 1", plan.ConflictCount)
|
||||||
|
}
|
||||||
|
|
||||||
|
plan, err = buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{Force: true})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("buildRestorePlan(force) error = %v", err)
|
||||||
|
}
|
||||||
|
if plan.ConflictCount != 0 {
|
||||||
|
t.Fatalf("force ConflictCount = %d, want 0", plan.ConflictCount)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestRestorePlanClassifiesSameAndConflict(t *testing.T) {
|
func TestRestorePlanClassifiesSameAndConflict(t *testing.T) {
|
||||||
cfg := restorePlanConfig(t)
|
cfg := restorePlanConfig(t)
|
||||||
current := restorePlanCurrentState(t, cfg)
|
current := restorePlanCurrentState(t, cfg)
|
||||||
@@ -159,6 +292,10 @@ func restorePlanConfig(t *testing.T) *config.Config {
|
|||||||
return &config.Config{
|
return &config.Config{
|
||||||
Pipeline: &config.PipelineConfig{
|
Pipeline: &config.PipelineConfig{
|
||||||
Workspace: config.WorkspaceConfig{Root: workspaceRoot},
|
Workspace: config.WorkspaceConfig{Root: workspaceRoot},
|
||||||
|
Storage: config.StorageConfig{S3: &config.StorageS3Config{
|
||||||
|
Bucket: "test-bucket",
|
||||||
|
RootPrefix: "dnd",
|
||||||
|
}},
|
||||||
},
|
},
|
||||||
Session: &config.SessionConfig{
|
Session: &config.SessionConfig{
|
||||||
SessionID: "2026-05-03",
|
SessionID: "2026-05-03",
|
||||||
@@ -167,6 +304,24 @@ func restorePlanConfig(t *testing.T) *config.Config {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func configureRestorePlanPreviousRequirement(cfg *config.Config, required bool) {
|
||||||
|
cfg.Session.PreviousSessionID = "2026-04-26"
|
||||||
|
cfg.Pipeline.Scriptorium = &config.ScriptoriumConfig{
|
||||||
|
Artifacts: map[string]config.ScriptoriumArtifactConfig{
|
||||||
|
"session_recap": {
|
||||||
|
Enabled: true,
|
||||||
|
OutputPath: "artifacts/session_recap.md",
|
||||||
|
Inputs: map[string]config.ScriptoriumInputConfig{
|
||||||
|
"previous_recap": {
|
||||||
|
Source: "narratio.previous_session.artifact.session_recap",
|
||||||
|
Required: required,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func restorePlanCurrentState(t *testing.T, cfg *config.Config) *RemoteCurrentState {
|
func restorePlanCurrentState(t *testing.T, cfg *config.Config) *RemoteCurrentState {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
sessionPrefix := artifacts.S3SessionPrefix("dnd", cfg.Session.Campaign, cfg.Session.SessionID)
|
sessionPrefix := artifacts.S3SessionPrefix("dnd", cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ func TestExecuteRestoreHelp(t *testing.T) {
|
|||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
|
|
||||||
code := Execute([]string{"restore", "--help"}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "--help"}, &stdout, &stderr)
|
||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("exit code = %d, want 0", code)
|
t.Fatalf("exit code = %d, want 0", code)
|
||||||
}
|
}
|
||||||
@@ -26,12 +26,15 @@ func TestExecuteRestoreHelp(t *testing.T) {
|
|||||||
t.Fatalf("stderr = %q, want empty", stderr.String())
|
t.Fatalf("stderr = %q, want empty", stderr.String())
|
||||||
}
|
}
|
||||||
out := stdout.String()
|
out := stdout.String()
|
||||||
if !strings.Contains(out, "Usage: narratio restore") {
|
if !strings.Contains(out, "Usage: narratio session restore <session_id>") {
|
||||||
t.Fatalf("stdout = %q, want restore usage", out)
|
t.Fatalf("stdout = %q, want restore usage", out)
|
||||||
}
|
}
|
||||||
if !strings.Contains(out, "--include-audio") {
|
if !strings.Contains(out, "--include-audio") {
|
||||||
t.Fatalf("stdout = %q, want --include-audio flag", out)
|
t.Fatalf("stdout = %q, want --include-audio flag", out)
|
||||||
}
|
}
|
||||||
|
if !strings.Contains(out, "--campaign") {
|
||||||
|
t.Fatalf("stdout = %q, want --campaign flag", out)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteRestoreRecognizedAndReturnsNYI(t *testing.T) {
|
func TestExecuteRestoreRecognizedAndReturnsNYI(t *testing.T) {
|
||||||
@@ -70,16 +73,16 @@ func TestExecuteRestoreRecognizedAndReturnsNYI(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute(
|
code := Execute(
|
||||||
[]string{
|
[]string{
|
||||||
"restore",
|
"session", "restore", "2026-05-03",
|
||||||
"--config", pipelinePath,
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
"--session", sessionPath,
|
"--session", sessionPath,
|
||||||
"--session-id", "2026-05-03",
|
|
||||||
"--dry-run",
|
"--dry-run",
|
||||||
"--force",
|
"--force",
|
||||||
"--include-audio",
|
"--include-audio",
|
||||||
@@ -116,11 +119,11 @@ func TestExecuteRestoreRecognizedAndReturnsNYI(t *testing.T) {
|
|||||||
|
|
||||||
func TestExecuteRestoreRejectsUnexpectedPositionalArguments(t *testing.T) {
|
func TestExecuteRestoreRejectsUnexpectedPositionalArguments(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"restore", "--config", pipelinePath, "--session", sessionPath, "extra"}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "extra"}, &stdout, &stderr)
|
||||||
if code == 0 {
|
if code == 0 {
|
||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
@@ -138,11 +141,11 @@ func TestExecuteRestoreFailsWhenStorageBackendNotConfigured(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeRestoreConfigWithoutStorage(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeRestoreConfigWithoutStorage(t, workspaceRoot)
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"restore", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
if code == 0 {
|
if code == 0 {
|
||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
@@ -166,11 +169,11 @@ func TestExecuteRestoreDiscoveryErrorSurfaced(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"restore", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
if code == 0 {
|
if code == 0 {
|
||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
@@ -208,7 +211,7 @@ func TestExecuteRestoreLoadsSecretsBeforeObjectStoreInit(t *testing.T) {
|
|||||||
restoreEnv(secretKeyEnv)
|
restoreEnv(secretKeyEnv)
|
||||||
|
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
secretsDir := filepath.Join(t.TempDir(), "secrets")
|
secretsDir := filepath.Join(t.TempDir(), "secrets")
|
||||||
mustWriteTestFile(t, filepath.Join(secretsDir, accessKeyEnv), "test-access-key-id\n")
|
mustWriteTestFile(t, filepath.Join(secretsDir, accessKeyEnv), "test-access-key-id\n")
|
||||||
mustWriteTestFile(t, filepath.Join(secretsDir, secretKeyEnv), "test-secret-key\n")
|
mustWriteTestFile(t, filepath.Join(secretsDir, secretKeyEnv), "test-secret-key\n")
|
||||||
@@ -259,10 +262,10 @@ func TestExecuteRestoreLoadsSecretsBeforeObjectStoreInit(t *testing.T) {
|
|||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute(
|
code := Execute(
|
||||||
[]string{
|
[]string{
|
||||||
"restore",
|
"session", "restore", "2026-05-03",
|
||||||
"--config", pipelinePath,
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
"--session", sessionPath,
|
"--session", sessionPath,
|
||||||
"--session-id", "2026-05-03",
|
|
||||||
"--dry-run",
|
"--dry-run",
|
||||||
},
|
},
|
||||||
&stdout,
|
&stdout,
|
||||||
@@ -307,10 +310,10 @@ func TestExecuteRestoreNonDryRunConflictFailsBeforeNYI(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"restore", "--config", pipelinePath, "--session", sessionPath}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
if code == 0 {
|
if code == 0 {
|
||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
@@ -359,10 +362,10 @@ func TestExecuteRestoreNonDryRunForceExecutesPlan(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"restore", "--config", pipelinePath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
code := Execute([]string{"session", "restore", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &stdout, &stderr)
|
||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
@@ -374,11 +377,12 @@ func TestExecuteRestoreNonDryRunForceExecutesPlan(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func writeRestoreConfigWithoutStorage(t *testing.T, workspaceRoot string) (string, string) {
|
func writeRestoreConfigWithoutStorage(t *testing.T, workspaceRoot string) (string, string, string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
pipelinePath := filepath.Join(dir, "pipeline.yml")
|
pipelinePath := filepath.Join(dir, "pipeline.yml")
|
||||||
|
campaignPath := writeAppTestCampaignConfig(t, dir)
|
||||||
sessionPath := filepath.Join(dir, "session.yml")
|
sessionPath := filepath.Join(dir, "session.yml")
|
||||||
|
|
||||||
pipelineYAML := `workspace:
|
pipelineYAML := `workspace:
|
||||||
@@ -405,5 +409,5 @@ inputs:
|
|||||||
mustWriteTestFile(t, filepath.Join(dir, "glossary.yml"), "[]\n")
|
mustWriteTestFile(t, filepath.Join(dir, "glossary.yml"), "[]\n")
|
||||||
mustWriteTestFile(t, filepath.Join(dir, "audio", "alice.flac"), "audio-bytes")
|
mustWriteTestFile(t, filepath.Join(dir, "audio", "alice.flac"), "audio-bytes")
|
||||||
|
|
||||||
return pipelinePath, sessionPath
|
return pipelinePath, campaignPath, sessionPath
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -19,10 +19,10 @@ import (
|
|||||||
|
|
||||||
func TestRestoreThenRunStageForceAnalyzeUsesRestoredDurableState(t *testing.T) {
|
func TestRestoreThenRunStageForceAnalyzeUsesRestoredDurableState(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
fake := &storage.FakeBackend{}
|
fake := &storage.FakeBackend{}
|
||||||
cfg, sessionPrefix, manifestKey, runIDKey := seedRestoreCommittedState(t, fake, pipelinePath, sessionPath)
|
cfg, sessionPrefix, manifestKey, runIDKey := seedRestoreCommittedState(t, fake, pipelinePath, campaignPath, sessionPath)
|
||||||
seedRestoreObject(fake, runIDKey, []byte("20260519T010203Z-a1b2c3d4\n"))
|
seedRestoreObject(fake, runIDKey, []byte("20260519T010203Z-a1b2c3d4\n"))
|
||||||
seedRestoreObject(fake, manifestKey, restoreWorkflowManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign))
|
seedRestoreObject(fake, manifestKey, restoreWorkflowManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign))
|
||||||
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte(`{"segments":[1,2,3]}`+"\n"))
|
seedRestoreObject(fake, sessionPrefix+"transcripts/full.json", []byte(`{"segments":[1,2,3]}`+"\n"))
|
||||||
@@ -46,10 +46,10 @@ func TestRestoreThenRunStageForceAnalyzeUsesRestoredDurableState(t *testing.T) {
|
|||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
restoreCode := Execute(
|
restoreCode := Execute(
|
||||||
[]string{
|
[]string{
|
||||||
"restore",
|
"session", "restore", cfg.Session.SessionID,
|
||||||
"--config", pipelinePath,
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
"--session", sessionPath,
|
"--session", sessionPath,
|
||||||
"--session-id", cfg.Session.SessionID,
|
|
||||||
},
|
},
|
||||||
&stdout,
|
&stdout,
|
||||||
&stderr,
|
&stderr,
|
||||||
@@ -84,13 +84,12 @@ func TestRestoreThenRunStageForceAnalyzeUsesRestoredDurableState(t *testing.T) {
|
|||||||
stderr.Reset()
|
stderr.Reset()
|
||||||
runStageCode := Execute(
|
runStageCode := Execute(
|
||||||
[]string{
|
[]string{
|
||||||
"run-stage",
|
"run-stage", "analyze", cfg.Session.SessionID,
|
||||||
"--config", pipelinePath,
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
"--session", sessionPath,
|
"--session", sessionPath,
|
||||||
"--session-id", cfg.Session.SessionID,
|
|
||||||
"--force",
|
"--force",
|
||||||
"--artifacts", "player_handout",
|
"--artifacts", "player_handout",
|
||||||
"analyze",
|
|
||||||
},
|
},
|
||||||
&stdout,
|
&stdout,
|
||||||
&stderr,
|
&stderr,
|
||||||
@@ -155,6 +154,123 @@ func TestRestoreThenRunStageForceAnalyzeUsesRestoredDurableState(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRestoreThenAnalyzeUsesRestoredPreviousCacheWithoutObjectStore(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
appendRestoreWorkflowScriptoriumConfig(t, pipelinePath, `
|
||||||
|
scriptorium:
|
||||||
|
binary: scriptorium
|
||||||
|
artifacts:
|
||||||
|
session_recap:
|
||||||
|
enabled: true
|
||||||
|
prompt_id: dnd.session_recap
|
||||||
|
output_path: artifacts/session_recap.md
|
||||||
|
inputs:
|
||||||
|
transcript:
|
||||||
|
source: narratio.transcript.final_trimmed
|
||||||
|
required: true
|
||||||
|
previous_recap:
|
||||||
|
source: narratio.previous_session.artifact.session_recap
|
||||||
|
required: true
|
||||||
|
`)
|
||||||
|
appendRestoreWorkflowScriptoriumConfig(t, sessionPath, `
|
||||||
|
previous_session_id: 2026-04-26
|
||||||
|
`)
|
||||||
|
|
||||||
|
fakeStore := &storage.FakeBackend{}
|
||||||
|
cfg, sessionPrefix, manifestKey, runIDKey := seedRestoreCommittedState(t, fakeStore, pipelinePath, campaignPath, sessionPath)
|
||||||
|
seedRestoreObject(fakeStore, runIDKey, []byte("20260519T010203Z-a1b2c3d4\n"))
|
||||||
|
seedRestoreObject(fakeStore, manifestKey, restoreWorkflowManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign))
|
||||||
|
seedRestoreObject(fakeStore, sessionPrefix+"transcripts/final.trimmed.json", []byte(`{"segments":[]}`+"\n"))
|
||||||
|
seedRestorePreviousCurrent(t, fakeStore, cfg, "# previous recap\n")
|
||||||
|
|
||||||
|
restoreWithStoreAndRealPhases(t, fakeStore)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
restoreCode := Execute(
|
||||||
|
[]string{
|
||||||
|
"session", "restore", cfg.Session.SessionID,
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
},
|
||||||
|
&stdout,
|
||||||
|
&stderr,
|
||||||
|
)
|
||||||
|
if restoreCode != 0 {
|
||||||
|
t.Fatalf("restore exit code = %d, want 0; stderr=%q", restoreCode, stderr.String())
|
||||||
|
}
|
||||||
|
if stderr.Len() != 0 {
|
||||||
|
t.Fatalf("restore stderr = %q, want empty", stderr.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
sessionRoot := artifacts.SessionWorkDirForCampaign(workspaceRoot, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
mustReadEquals(t, filepath.Join(sessionRoot, "transcripts", "final.trimmed.json"), `{"segments":[]}`+"\n")
|
||||||
|
previousManifestBytes, err := os.ReadFile(filepath.Join(sessionRoot, "previous", "manifest.json"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read restored previous manifest: %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(string(previousManifestBytes), `"session_id":"2026-04-26"`) {
|
||||||
|
t.Fatalf("restored previous manifest = %q, want previous session id", string(previousManifestBytes))
|
||||||
|
}
|
||||||
|
mustReadEquals(t, filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md"), "# previous recap\n")
|
||||||
|
|
||||||
|
scriptoriumFake := &scriptorium.FakeRunner{}
|
||||||
|
origExecuteStagesFn := executeStagesFn
|
||||||
|
origObjectStoreFn := newObjectStoreFromConfigFn
|
||||||
|
objectStoreConstructed := false
|
||||||
|
t.Cleanup(func() {
|
||||||
|
executeStagesFn = origExecuteStagesFn
|
||||||
|
newObjectStoreFromConfigFn = origObjectStoreFn
|
||||||
|
})
|
||||||
|
executeStagesFn = func(ctx context.Context, cfg *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||||
|
if opts.Env == nil {
|
||||||
|
opts.Env = &Env{}
|
||||||
|
}
|
||||||
|
opts.Env.Scriptorium = scriptoriumFake
|
||||||
|
return executeStages(ctx, cfg, stages, opts)
|
||||||
|
}
|
||||||
|
newObjectStoreFromConfigFn = func(context.Context, *config.Config) (storage.ObjectStore, error) {
|
||||||
|
objectStoreConstructed = true
|
||||||
|
return nil, context.Canceled
|
||||||
|
}
|
||||||
|
|
||||||
|
stdout.Reset()
|
||||||
|
stderr.Reset()
|
||||||
|
runStageCode := Execute(
|
||||||
|
[]string{
|
||||||
|
"run-stage", "analyze", cfg.Session.SessionID,
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
"--force",
|
||||||
|
"--artifacts", "session_recap",
|
||||||
|
},
|
||||||
|
&stdout,
|
||||||
|
&stderr,
|
||||||
|
)
|
||||||
|
if runStageCode != 0 {
|
||||||
|
t.Fatalf("run-stage exit code = %d, want 0; stderr=%q", runStageCode, stderr.String())
|
||||||
|
}
|
||||||
|
if stderr.Len() != 0 {
|
||||||
|
t.Fatalf("run-stage stderr = %q, want empty", stderr.String())
|
||||||
|
}
|
||||||
|
if objectStoreConstructed {
|
||||||
|
t.Fatal("analyze run-stage should not construct object store for previous-session input resolution")
|
||||||
|
}
|
||||||
|
if len(scriptoriumFake.RunRequests) != 1 {
|
||||||
|
t.Fatalf("scriptorium run requests = %d, want 1", len(scriptoriumFake.RunRequests))
|
||||||
|
}
|
||||||
|
req := scriptoriumFake.RunRequests[0]
|
||||||
|
if got := req.InputPaths["transcript"]; got != filepath.Join(sessionRoot, "transcripts", "final.trimmed.json") {
|
||||||
|
t.Fatalf("transcript input = %q, want trimmed transcript path", got)
|
||||||
|
}
|
||||||
|
if got := req.InputPaths["previous_recap"]; got != filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md") {
|
||||||
|
t.Fatalf("previous_recap input = %q, want restored previous cache path", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func restoreWorkflowManifestJSON(t *testing.T, sessionID, campaign string) []byte {
|
func restoreWorkflowManifestJSON(t *testing.T, sessionID, campaign string) []byte {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
@@ -176,3 +292,15 @@ func restoreWorkflowManifestJSON(t *testing.T, sessionID, campaign string) []byt
|
|||||||
}
|
}
|
||||||
return data
|
return data
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func appendRestoreWorkflowScriptoriumConfig(t *testing.T, pipelinePath, extra string) {
|
||||||
|
t.Helper()
|
||||||
|
f, err := os.OpenFile(pipelinePath, os.O_APPEND|os.O_WRONLY, 0)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("open pipeline config for append: %v", err)
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
if _, err := f.WriteString(extra); err != nil {
|
||||||
|
t.Fatalf("append pipeline config: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"flag"
|
"flag"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
@@ -13,37 +14,47 @@ import (
|
|||||||
|
|
||||||
// Resume continues execution from the first non-succeeded stage in the manifest.
|
// Resume continues execution from the first non-succeeded stage in the manifest.
|
||||||
func Resume(ctx context.Context, args []string, out io.Writer) error {
|
func Resume(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
positionalSessionID, args := pullLeadingSessionID(args)
|
||||||
fs := flag.NewFlagSet("resume", flag.ContinueOnError)
|
fs := flag.NewFlagSet("resume", flag.ContinueOnError)
|
||||||
fs.SetOutput(io.Discard)
|
fs.SetOutput(io.Discard)
|
||||||
|
|
||||||
var pipelinePath string
|
var pipelinePath string
|
||||||
|
var campaignPath string
|
||||||
|
var campaignFilePath string
|
||||||
var sessionPath string
|
var sessionPath string
|
||||||
var sessionID string
|
var sessionID string
|
||||||
|
var previousSessionID string
|
||||||
var force bool
|
var force bool
|
||||||
var selectedArtifacts artifactSelectionFlag
|
var selectedArtifacts artifactSelectionFlag
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||||
|
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||||
|
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||||
fs.StringVar(&sessionID, "session-id", "", "session identifier for session.yml templates")
|
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||||
fs.BoolVar(&force, "force", false, "force stage execution")
|
fs.BoolVar(&force, "force", false, "force stage execution")
|
||||||
fs.Var(&selectedArtifacts, "artifacts", "artifact names to execute during analyze (comma-separated or repeatable)")
|
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute and publish (comma-separated or repeatable)")
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
if err := fs.Parse(args); err != nil {
|
||||||
return fmt.Errorf("resume: invalid flags: %w", err)
|
return fmt.Errorf("resume: invalid flags: %w", err)
|
||||||
}
|
}
|
||||||
|
if positionalSessionID == "" {
|
||||||
|
if err := applyParsedSessionIDArg("resume", fs, &sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
if fs.NArg() != 0 {
|
if fs.NArg() != 0 {
|
||||||
return fmt.Errorf("resume: unexpected positional arguments")
|
return fmt.Errorf("resume: unexpected positional arguments")
|
||||||
}
|
}
|
||||||
resolvedPipelinePath, err := resolvePipelineConfigPath(pipelinePath)
|
if err := applyPositionalSessionID("resume", positionalSessionID, &sessionID); err != nil {
|
||||||
if err != nil {
|
return err
|
||||||
return fmt.Errorf("resume: %w", err)
|
|
||||||
}
|
}
|
||||||
resolvedSessionPath, err := resolveSessionConfigPath(sessionPath)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("resume: %w", err)
|
|
||||||
}
|
}
|
||||||
|
if strings.TrimSpace(sessionID) == "" {
|
||||||
cfg, err := config.LoadWithSessionOptions(resolvedPipelinePath, resolvedSessionPath, config.SessionLoadOptions{
|
return fmt.Errorf("resume: session_id is required")
|
||||||
|
}
|
||||||
|
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
||||||
SessionID: sessionID,
|
SessionID: sessionID,
|
||||||
|
PreviousSessionID: previousSessionID,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("resume: %w", err)
|
return fmt.Errorf("resume: %w", err)
|
||||||
@@ -55,7 +66,7 @@ func Resume(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("resume: invalid --artifacts: %w", err)
|
return fmt.Errorf("resume: invalid --artifacts: %w", err)
|
||||||
}
|
}
|
||||||
if err := validateSelectedAnalyzeArtifacts(cfg, normalizedArtifacts); err != nil {
|
if err := validateSelectedArtifacts(cfg, normalizedArtifacts); err != nil {
|
||||||
return fmt.Errorf("resume: %w", err)
|
return fmt.Errorf("resume: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import (
|
|||||||
|
|
||||||
func TestResumeStartsAfterCompletedStages(t *testing.T) {
|
func TestResumeStartsAfterCompletedStages(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
@@ -32,7 +32,7 @@ func TestResumeStartsAfterCompletedStages(t *testing.T) {
|
|||||||
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Resume(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath}, &out)
|
err := Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Resume() error = %v", err)
|
t.Fatalf("Resume() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -51,12 +51,12 @@ func TestResumeStartsAfterCompletedStages(t *testing.T) {
|
|||||||
|
|
||||||
func TestResumeNoRemainingStages(t *testing.T) {
|
func TestResumeNoRemainingStages(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
m := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
m := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "archive", "notify"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
||||||
m.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
m.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
||||||
}
|
}
|
||||||
if err := store.Save(context.Background(), manifestPath, m); err != nil {
|
if err := store.Save(context.Background(), manifestPath, m); err != nil {
|
||||||
@@ -64,7 +64,7 @@ func TestResumeNoRemainingStages(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Resume(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath}, &out)
|
err := Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Resume() error = %v", err)
|
t.Fatalf("Resume() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -80,12 +80,12 @@ func TestResumeForceRerunsSucceeded(t *testing.T) {
|
|||||||
_, _ = w.Write([]byte(`{"source":"resume-force-test","segments":[{"speaker":"alice"}]}`))
|
_, _ = w.Write([]byte(`{"source":"resume-force-test","segments":[{"speaker":"alice"}]}`))
|
||||||
}))
|
}))
|
||||||
defer srv.Close()
|
defer srv.Close()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
m := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
m := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "archive", "notify"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
||||||
m.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
m.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
||||||
}
|
}
|
||||||
if err := store.Save(context.Background(), manifestPath, m); err != nil {
|
if err := store.Save(context.Background(), manifestPath, m); err != nil {
|
||||||
@@ -93,7 +93,7 @@ func TestResumeForceRerunsSucceeded(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Resume(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "--force"}, &out)
|
err := Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Resume() error = %v", err)
|
t.Fatalf("Resume() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -104,14 +104,14 @@ func TestResumeForceRerunsSucceeded(t *testing.T) {
|
|||||||
|
|
||||||
func TestRunStageExecutesOnlySelectedStage(t *testing.T) {
|
func TestRunStageExecutesOnlySelectedStage(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "merged.json"), `{"segments":[]}`)
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "base.json"), `{"segments":[]}`)
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "polish"}, &out)
|
err := RunStage(context.Background(), []string{"polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("RunStage() error = %v", err)
|
t.Fatalf("RunStage() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -134,10 +134,10 @@ func TestRunStageExecutesOnlySelectedStage(t *testing.T) {
|
|||||||
|
|
||||||
func TestRunStageSkipAndForce(t *testing.T) {
|
func TestRunStageSkipAndForce(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "merged.json"), `{"segments":[]}`)
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "base.json"), `{"segments":[]}`)
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
@@ -148,7 +148,7 @@ func TestRunStageSkipAndForce(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "polish"}, &out)
|
err := RunStage(context.Background(), []string{"polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("RunStage() error = %v", err)
|
t.Fatalf("RunStage() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -157,7 +157,7 @@ func TestRunStageSkipAndForce(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
out.Reset()
|
out.Reset()
|
||||||
err = RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "--force", "polish"}, &out)
|
err = RunStage(context.Background(), []string{"polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("RunStage(force) error = %v", err)
|
t.Fatalf("RunStage(force) error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -168,15 +168,15 @@ func TestRunStageSkipAndForce(t *testing.T) {
|
|||||||
|
|
||||||
func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing.T) {
|
func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "merged.json"), `{"segments":[]}`)
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "base.json"), `{"segments":[]}`)
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
seed := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
seed := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "archive", "notify"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
||||||
seed.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
seed.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
||||||
}
|
}
|
||||||
if err := store.Save(context.Background(), manifestPath, seed); err != nil {
|
if err := store.Save(context.Background(), manifestPath, seed); err != nil {
|
||||||
@@ -184,7 +184,7 @@ func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing
|
|||||||
}
|
}
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "--force", "polish"}, &out)
|
err := RunStage(context.Background(), []string{"polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("RunStage(force) error = %v", err)
|
t.Fatalf("RunStage(force) error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -196,14 +196,14 @@ func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("load manifest after force: %v", err)
|
t.Fatalf("load manifest after force: %v", err)
|
||||||
}
|
}
|
||||||
for _, name := range []string{"normalize", "trim", "analyze", "archive", "notify"} {
|
for _, name := range []string{"normalize", "trim", "analyze", "publish", "notify"} {
|
||||||
if afterForce.Stages[name] == nil || afterForce.Stages[name].Status != manifest.StatusStale {
|
if afterForce.Stages[name] == nil || afterForce.Stages[name].Status != manifest.StatusStale {
|
||||||
t.Fatalf("stage %q = %#v, want stale", name, afterForce.Stages[name])
|
t.Fatalf("stage %q = %#v, want stale", name, afterForce.Stages[name])
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
out.Reset()
|
out.Reset()
|
||||||
err = Resume(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath}, &out)
|
err = Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Resume() error = %v", err)
|
t.Fatalf("Resume() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -214,13 +214,13 @@ func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing
|
|||||||
|
|
||||||
func TestRunStageTrimExecutes(t *testing.T) {
|
func TestRunStageTrimExecutes(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "normalized.json"), `{"segments":[{"id":1},{"id":2}]}`)
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.json"), `{"segments":[{"id":1},{"id":2}]}`)
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "trim"}, &out)
|
err := RunStage(context.Background(), []string{"trim", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("RunStage(trim) error = %v", err)
|
t.Fatalf("RunStage(trim) error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -243,13 +243,13 @@ func TestRunStageTrimExecutes(t *testing.T) {
|
|||||||
|
|
||||||
func TestRunStageNormalizeExecutes(t *testing.T) {
|
func TestRunStageNormalizeExecutes(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "processed.json"), `{"segments":[{"id":1},{"id":2}]}`)
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "polished.json"), `{"segments":[{"id":1},{"id":2}]}`)
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "normalize"}, &out)
|
err := RunStage(context.Background(), []string{"normalize", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("RunStage(normalize) error = %v", err)
|
t.Fatalf("RunStage(normalize) error = %v", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,43 +5,54 @@ import (
|
|||||||
"flag"
|
"flag"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Run executes the pipeline plan and persists manifest state.
|
// Run executes the pipeline plan and persists manifest state.
|
||||||
func Run(ctx context.Context, args []string, out io.Writer) error {
|
func Run(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
positionalSessionID, args := pullLeadingSessionID(args)
|
||||||
fs := flag.NewFlagSet("run", flag.ContinueOnError)
|
fs := flag.NewFlagSet("run", flag.ContinueOnError)
|
||||||
fs.SetOutput(io.Discard)
|
fs.SetOutput(io.Discard)
|
||||||
|
|
||||||
var pipelinePath string
|
var pipelinePath string
|
||||||
|
var campaignPath string
|
||||||
|
var campaignFilePath string
|
||||||
var sessionPath string
|
var sessionPath string
|
||||||
var sessionID string
|
var sessionID string
|
||||||
|
var previousSessionID string
|
||||||
var force bool
|
var force bool
|
||||||
var selectedArtifacts artifactSelectionFlag
|
var selectedArtifacts artifactSelectionFlag
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||||
|
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||||
|
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||||
fs.StringVar(&sessionID, "session-id", "", "session identifier for session.yml templates")
|
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||||
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
||||||
fs.Var(&selectedArtifacts, "artifacts", "artifact names to execute during analyze (comma-separated or repeatable)")
|
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute and publish (comma-separated or repeatable)")
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
if err := fs.Parse(args); err != nil {
|
||||||
return fmt.Errorf("run: invalid flags: %w", err)
|
return fmt.Errorf("run: invalid flags: %w", err)
|
||||||
}
|
}
|
||||||
|
if positionalSessionID == "" {
|
||||||
|
if err := applyParsedSessionIDArg("run", fs, &sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
if fs.NArg() != 0 {
|
if fs.NArg() != 0 {
|
||||||
return fmt.Errorf("run: unexpected positional arguments")
|
return fmt.Errorf("run: unexpected positional arguments")
|
||||||
}
|
}
|
||||||
resolvedPipelinePath, err := resolvePipelineConfigPath(pipelinePath)
|
if err := applyPositionalSessionID("run", positionalSessionID, &sessionID); err != nil {
|
||||||
if err != nil {
|
return err
|
||||||
return fmt.Errorf("run: %w", err)
|
|
||||||
}
|
}
|
||||||
resolvedSessionPath, err := resolveSessionConfigPath(sessionPath)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("run: %w", err)
|
|
||||||
}
|
}
|
||||||
|
if strings.TrimSpace(sessionID) == "" {
|
||||||
cfg, err := config.LoadWithSessionOptions(resolvedPipelinePath, resolvedSessionPath, config.SessionLoadOptions{
|
return fmt.Errorf("run: session_id is required")
|
||||||
|
}
|
||||||
|
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
||||||
SessionID: sessionID,
|
SessionID: sessionID,
|
||||||
|
PreviousSessionID: previousSessionID,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("run: %w", err)
|
return fmt.Errorf("run: %w", err)
|
||||||
@@ -53,7 +64,7 @@ func Run(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("run: invalid --artifacts: %w", err)
|
return fmt.Errorf("run: invalid --artifacts: %w", err)
|
||||||
}
|
}
|
||||||
if err := validateSelectedAnalyzeArtifacts(cfg, normalizedArtifacts); err != nil {
|
if err := validateSelectedArtifacts(cfg, normalizedArtifacts); err != nil {
|
||||||
return fmt.Errorf("run: %w", err)
|
return fmt.Errorf("run: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -44,7 +44,7 @@ func TestDecideStageActions(t *testing.T) {
|
|||||||
|
|
||||||
func TestDownstreamStageNames(t *testing.T) {
|
func TestDownstreamStageNames(t *testing.T) {
|
||||||
got := downstreamStageNames("polish")
|
got := downstreamStageNames("polish")
|
||||||
want := []string{"normalize", "trim", "analyze", "archive", "notify"}
|
want := []string{"normalize", "trim", "analyze", "publish", "notify"}
|
||||||
if !reflect.DeepEqual(got, want) {
|
if !reflect.DeepEqual(got, want) {
|
||||||
t.Fatalf("downstreamStageNames(polish) = %#v, want %#v", got, want)
|
t.Fatalf("downstreamStageNames(polish) = %#v, want %#v", got, want)
|
||||||
}
|
}
|
||||||
@@ -65,11 +65,11 @@ func TestInvalidateDownstreamSucceededStages(t *testing.T) {
|
|||||||
m.MarkStageSucceeded("normalize", now, nil)
|
m.MarkStageSucceeded("normalize", now, nil)
|
||||||
m.MarkStageSucceeded("trim", now, nil)
|
m.MarkStageSucceeded("trim", now, nil)
|
||||||
m.MarkStageFailed("analyze", now, "analysis failed")
|
m.MarkStageFailed("analyze", now, "analysis failed")
|
||||||
m.MarkStageSucceeded("archive", now, nil)
|
m.MarkStageSucceeded("publish", now, nil)
|
||||||
m.MarkStageSucceeded("notify", now, nil)
|
m.MarkStageSucceeded("notify", now, nil)
|
||||||
|
|
||||||
got := invalidateDownstreamSucceededStages(m, "polish", now.Add(1*time.Second))
|
got := invalidateDownstreamSucceededStages(m, "polish", now.Add(1*time.Second))
|
||||||
want := []string{"normalize", "trim", "archive", "notify"}
|
want := []string{"normalize", "trim", "publish", "notify"}
|
||||||
if !reflect.DeepEqual(got, want) {
|
if !reflect.DeepEqual(got, want) {
|
||||||
t.Fatalf("invalidateDownstreamSucceededStages() = %#v, want %#v", got, want)
|
t.Fatalf("invalidateDownstreamSucceededStages() = %#v, want %#v", got, want)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,79 +5,88 @@ import (
|
|||||||
"flag"
|
"flag"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
)
|
)
|
||||||
|
|
||||||
// RunStage executes exactly one selected stage.
|
// RunStage executes exactly one selected stage.
|
||||||
func RunStage(ctx context.Context, args []string, out io.Writer) error {
|
func RunStage(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
var stageName string
|
||||||
|
var positionalSessionID string
|
||||||
|
if len(args) >= 2 && !isCLIFlagToken(args[0]) && !isCLIFlagToken(args[1]) {
|
||||||
|
stageName = strings.TrimSpace(args[0])
|
||||||
|
positionalSessionID = strings.TrimSpace(args[1])
|
||||||
|
args = append([]string(nil), args[2:]...)
|
||||||
|
}
|
||||||
|
|
||||||
fs := flag.NewFlagSet("run-stage", flag.ContinueOnError)
|
fs := flag.NewFlagSet("run-stage", flag.ContinueOnError)
|
||||||
fs.SetOutput(io.Discard)
|
fs.SetOutput(io.Discard)
|
||||||
|
|
||||||
var pipelinePath string
|
var pipelinePath string
|
||||||
|
var campaignPath string
|
||||||
|
var campaignFilePath string
|
||||||
var sessionPath string
|
var sessionPath string
|
||||||
var sessionID string
|
var sessionID string
|
||||||
|
var previousSessionID string
|
||||||
var force bool
|
var force bool
|
||||||
var selectedArtifacts artifactSelectionFlag
|
var selectedArtifacts artifactSelectionFlag
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||||
|
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||||
|
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||||
fs.StringVar(&sessionID, "session-id", "", "session identifier for session.yml templates")
|
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||||
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
||||||
fs.Var(&selectedArtifacts, "artifacts", "artifact names to execute during analyze (comma-separated or repeatable)")
|
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute or publish (comma-separated or repeatable)")
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
if err := fs.Parse(args); err != nil {
|
||||||
return fmt.Errorf("run-stage: invalid flags: %w", err)
|
return fmt.Errorf("run-stage: invalid flags: %w", err)
|
||||||
}
|
}
|
||||||
if fs.NArg() != 1 {
|
if stageName == "" {
|
||||||
return fmt.Errorf("run-stage: expected exactly one stage name")
|
switch fs.NArg() {
|
||||||
|
case 2:
|
||||||
|
stageName = strings.TrimSpace(fs.Arg(0))
|
||||||
|
positionalSessionID = strings.TrimSpace(fs.Arg(1))
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("run-stage: expected stage name and session_id")
|
||||||
|
}
|
||||||
|
} else if fs.NArg() != 0 {
|
||||||
|
return fmt.Errorf("run-stage: unexpected positional arguments")
|
||||||
|
}
|
||||||
|
if err := applyPositionalSessionID("run-stage", positionalSessionID, &sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(sessionID) == "" {
|
||||||
|
return fmt.Errorf("run-stage: session_id is required")
|
||||||
}
|
}
|
||||||
stageName := fs.Arg(0)
|
|
||||||
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("run-stage: invalid --artifacts: %w", err)
|
return fmt.Errorf("run-stage: invalid --artifacts: %w", err)
|
||||||
}
|
}
|
||||||
if len(normalizedArtifacts) > 0 && stageName != "analyze" {
|
if len(normalizedArtifacts) > 0 && stageName != "analyze" && stageName != "publish" {
|
||||||
return fmt.Errorf("run-stage: --artifacts is only supported for stage \"analyze\"")
|
return fmt.Errorf("run-stage: --artifacts is only supported for stages \"analyze\" and \"publish\"")
|
||||||
}
|
|
||||||
stages, err := BuildSingleStagePlan(stageName)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("run-stage: %w", err)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
resolvedPipelinePath, err := resolvePipelineConfigPath(pipelinePath)
|
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
||||||
if err != nil {
|
CommandName: "run-stage",
|
||||||
return fmt.Errorf("run-stage: %w", err)
|
StageName: stageName,
|
||||||
}
|
PipelinePath: pipelinePath,
|
||||||
resolvedSessionPath, err := resolveSessionConfigPath(sessionPath)
|
CampaignPath: campaignPath,
|
||||||
if err != nil {
|
CampaignFilePath: campaignFilePath,
|
||||||
return fmt.Errorf("run-stage: %w", err)
|
SessionPath: sessionPath,
|
||||||
}
|
|
||||||
|
|
||||||
cfg, err := config.LoadWithSessionOptions(resolvedPipelinePath, resolvedSessionPath, config.SessionLoadOptions{
|
|
||||||
SessionID: sessionID,
|
SessionID: sessionID,
|
||||||
})
|
PreviousSessionID: previousSessionID,
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("run-stage: %w", err)
|
|
||||||
}
|
|
||||||
if err := config.Validate(cfg); err != nil {
|
|
||||||
return fmt.Errorf("run-stage: %w", err)
|
|
||||||
}
|
|
||||||
if err := validateSelectedAnalyzeArtifacts(cfg, normalizedArtifacts); err != nil {
|
|
||||||
return fmt.Errorf("run-stage: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
summary, err := executeStagesFn(ctx, cfg, stages, RunOptions{
|
|
||||||
Force: force,
|
Force: force,
|
||||||
SelectedArtifacts: normalizedArtifacts,
|
SelectedArtifacts: normalizedArtifacts,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("run-stage: %w", err)
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
_, err = fmt.Fprintf(
|
_, err = fmt.Fprintf(
|
||||||
out,
|
out,
|
||||||
"narratio run-stage: stage=%s executed=%d skipped=%d force=%t; manifest=%s\n",
|
"narratio run-stage: stage=%s executed=%d skipped=%d force=%t; manifest=%s\n",
|
||||||
stages[0].Name(),
|
stageName,
|
||||||
len(summary.Executed),
|
len(summary.Executed),
|
||||||
len(summary.Skipped),
|
len(summary.Skipped),
|
||||||
force,
|
force,
|
||||||
@@ -85,3 +94,184 @@ func RunStage(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
)
|
)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Analyze force-runs the analyze stage.
|
||||||
|
func Analyze(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
positionalSessionID, args := pullLeadingSessionID(args)
|
||||||
|
fs := flag.NewFlagSet("analyze", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(io.Discard)
|
||||||
|
|
||||||
|
var pipelinePath string
|
||||||
|
var campaignPath string
|
||||||
|
var campaignFilePath string
|
||||||
|
var sessionPath string
|
||||||
|
var sessionID string
|
||||||
|
var previousSessionID string
|
||||||
|
var selectedArtifacts artifactSelectionFlag
|
||||||
|
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||||
|
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||||
|
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||||
|
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||||
|
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||||
|
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute during analyze (comma-separated or repeatable)")
|
||||||
|
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return fmt.Errorf("analyze: invalid flags: %w", err)
|
||||||
|
}
|
||||||
|
if positionalSessionID == "" {
|
||||||
|
if err := applyParsedSessionIDArg("analyze", fs, &sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if fs.NArg() != 0 {
|
||||||
|
return fmt.Errorf("analyze: unexpected positional arguments")
|
||||||
|
}
|
||||||
|
if err := applyPositionalSessionID("analyze", positionalSessionID, &sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(sessionID) == "" {
|
||||||
|
return fmt.Errorf("analyze: session_id is required")
|
||||||
|
}
|
||||||
|
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("analyze: invalid --artifacts: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
||||||
|
CommandName: "analyze",
|
||||||
|
StageName: "analyze",
|
||||||
|
PipelinePath: pipelinePath,
|
||||||
|
CampaignPath: campaignPath,
|
||||||
|
CampaignFilePath: campaignFilePath,
|
||||||
|
SessionPath: sessionPath,
|
||||||
|
SessionID: sessionID,
|
||||||
|
PreviousSessionID: previousSessionID,
|
||||||
|
Force: true,
|
||||||
|
SelectedArtifacts: normalizedArtifacts,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err = fmt.Fprintf(
|
||||||
|
out,
|
||||||
|
"narratio analyze: executed=%d skipped=%d force=true; manifest=%s\n",
|
||||||
|
len(summary.Executed),
|
||||||
|
len(summary.Skipped),
|
||||||
|
summary.ManifestPath,
|
||||||
|
)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Publish force-runs the publish stage.
|
||||||
|
func Publish(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
positionalSessionID, args := pullLeadingSessionID(args)
|
||||||
|
fs := flag.NewFlagSet("publish", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(io.Discard)
|
||||||
|
|
||||||
|
var pipelinePath string
|
||||||
|
var campaignPath string
|
||||||
|
var campaignFilePath string
|
||||||
|
var sessionPath string
|
||||||
|
var sessionID string
|
||||||
|
var previousSessionID string
|
||||||
|
var selectedArtifacts artifactSelectionFlag
|
||||||
|
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||||
|
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||||
|
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||||
|
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
||||||
|
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
||||||
|
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to publish (comma-separated or repeatable)")
|
||||||
|
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return fmt.Errorf("publish: invalid flags: %w", err)
|
||||||
|
}
|
||||||
|
if positionalSessionID == "" {
|
||||||
|
if err := applyParsedSessionIDArg("publish", fs, &sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if fs.NArg() != 0 {
|
||||||
|
return fmt.Errorf("publish: unexpected positional arguments")
|
||||||
|
}
|
||||||
|
if err := applyPositionalSessionID("publish", positionalSessionID, &sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(sessionID) == "" {
|
||||||
|
return fmt.Errorf("publish: session_id is required")
|
||||||
|
}
|
||||||
|
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("publish: invalid --artifacts: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
||||||
|
CommandName: "publish",
|
||||||
|
StageName: "publish",
|
||||||
|
PipelinePath: pipelinePath,
|
||||||
|
CampaignPath: campaignPath,
|
||||||
|
CampaignFilePath: campaignFilePath,
|
||||||
|
SessionPath: sessionPath,
|
||||||
|
SessionID: sessionID,
|
||||||
|
PreviousSessionID: previousSessionID,
|
||||||
|
Force: true,
|
||||||
|
SelectedArtifacts: normalizedArtifacts,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err = fmt.Fprintf(
|
||||||
|
out,
|
||||||
|
"narratio publish: executed=%d skipped=%d force=true; manifest=%s\n",
|
||||||
|
len(summary.Executed),
|
||||||
|
len(summary.Skipped),
|
||||||
|
summary.ManifestPath,
|
||||||
|
)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
type singleStageCommand struct {
|
||||||
|
CommandName string
|
||||||
|
StageName string
|
||||||
|
PipelinePath string
|
||||||
|
CampaignPath string
|
||||||
|
CampaignFilePath string
|
||||||
|
SessionPath string
|
||||||
|
SessionID string
|
||||||
|
PreviousSessionID string
|
||||||
|
Force bool
|
||||||
|
SelectedArtifacts []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func runSingleStageCommand(ctx context.Context, req singleStageCommand) (*RunSummary, error) {
|
||||||
|
stages, err := BuildSingleStagePlan(req.StageName)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", req.CommandName, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg, err := loadCommandConfig(ctx, req.PipelinePath, req.CampaignPath, req.CampaignFilePath, req.SessionPath, config.SessionLoadOptions{
|
||||||
|
SessionID: req.SessionID,
|
||||||
|
PreviousSessionID: req.PreviousSessionID,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", req.CommandName, err)
|
||||||
|
}
|
||||||
|
if err := config.Validate(cfg); err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", req.CommandName, err)
|
||||||
|
}
|
||||||
|
if err := validateSelectedArtifacts(cfg, req.SelectedArtifacts); err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", req.CommandName, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
summary, err := executeStagesFn(ctx, cfg, stages, RunOptions{
|
||||||
|
Force: req.Force,
|
||||||
|
SelectedArtifacts: req.SelectedArtifacts,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", req.CommandName, err)
|
||||||
|
}
|
||||||
|
return summary, nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -46,7 +46,7 @@ func executeStages(ctx context.Context, cfg *config.Config, stages []stage.Stage
|
|||||||
if env.Config == nil {
|
if env.Config == nil {
|
||||||
env.Config = cfg
|
env.Config = cfg
|
||||||
}
|
}
|
||||||
env.SelectedAnalyzeArtifacts = append([]string(nil), opts.SelectedArtifacts...)
|
env.SelectedArtifactKeys = append([]string(nil), opts.SelectedArtifacts...)
|
||||||
if env.ArtifactStore == nil {
|
if env.ArtifactStore == nil {
|
||||||
env.ArtifactStore = artifacts.NewLocalStore(cfg.Pipeline.Workspace.Root)
|
env.ArtifactStore = artifacts.NewLocalStore(cfg.Pipeline.Workspace.Root)
|
||||||
}
|
}
|
||||||
@@ -87,12 +87,19 @@ func executeStages(ctx context.Context, cfg *config.Config, stages []stage.Stage
|
|||||||
env.Storage = &storage.NoopBackend{}
|
env.Storage = &storage.NoopBackend{}
|
||||||
}
|
}
|
||||||
if env.ObjectStore == nil && needsObjectStoreForRun(env.Config, stages) {
|
if env.ObjectStore == nil && needsObjectStoreForRun(env.Config, stages) {
|
||||||
objectStore, err := storage.NewObjectStoreFromConfig(ctx, env.Config)
|
objectStore, err := newCommandObjectStore(ctx, env.Config, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("initialize object store backend: %w", err)
|
return nil, err
|
||||||
}
|
}
|
||||||
env.ObjectStore = objectStore
|
env.ObjectStore = objectStore
|
||||||
}
|
}
|
||||||
|
if needsRemoteLocksForRun(env.Config, stages) {
|
||||||
|
locks, err := loadEffectiveLocks(ctx, env.Config, env.ObjectStore)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("load remote archive locks: %w", err)
|
||||||
|
}
|
||||||
|
applyEffectiveLocks(env.Config, locks.All)
|
||||||
|
}
|
||||||
if env.Notifier == nil {
|
if env.Notifier == nil {
|
||||||
env.Notifier = ¬ify.NoopSender{}
|
env.Notifier = ¬ify.NoopSender{}
|
||||||
}
|
}
|
||||||
@@ -555,17 +562,56 @@ func needsObjectStoreForRun(cfg *config.Config, stages []stage.Stage) bool {
|
|||||||
if cfg.Session.Inputs.AudioS3 != nil && stageRequested("prepare") {
|
if cfg.Session.Inputs.AudioS3 != nil && stageRequested("prepare") {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
if !stageRequested("archive") {
|
if stageRequested("prepare") {
|
||||||
|
requirements := artifacts.CollectPreviousArtifactRequirements(configuredScriptoriumArtifacts(cfg))
|
||||||
|
if len(requirements) > 0 && strings.TrimSpace(cfg.Session.PreviousSessionID) != "" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !stageRequested("publish") {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
if cfg.Pipeline.Archive == nil {
|
if cfg.Pipeline.Publish == nil {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
if cfg.Pipeline.Archive.Enabled != nil && !*cfg.Pipeline.Archive.Enabled {
|
if cfg.Pipeline.Publish.Enabled != nil && !*cfg.Pipeline.Publish.Enabled {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
if cfg.Pipeline.Archive.UploadRun != nil && !*cfg.Pipeline.Archive.UploadRun {
|
if cfg.Pipeline.Publish.UploadRun != nil && !*cfg.Pipeline.Publish.UploadRun {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func needsRemoteLocksForRun(cfg *config.Config, stages []stage.Stage) bool {
|
||||||
|
if cfg == nil || cfg.Pipeline == nil || cfg.Session == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
publishRequested := false
|
||||||
|
for _, s := range stages {
|
||||||
|
if s != nil && s.Name() == "publish" {
|
||||||
|
publishRequested = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !publishRequested {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Publish == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Publish.Enabled != nil && !*cfg.Pipeline.Publish.Enabled {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Publish.UploadRun != nil && !*cfg.Pipeline.Publish.UploadRun {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return cfg.Pipeline.Storage.S3 != nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func configuredScriptoriumArtifacts(cfg *config.Config) map[string]config.ScriptoriumArtifactConfig {
|
||||||
|
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Scriptorium == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return cfg.Pipeline.Scriptorium.Artifacts
|
||||||
|
}
|
||||||
|
|||||||
@@ -54,7 +54,7 @@ func (s captureSelectedArtifactsStage) Name() string { return s.name }
|
|||||||
func (s captureSelectedArtifactsStage) Declares() stage.IODecl { return stage.IODecl{} }
|
func (s captureSelectedArtifactsStage) Declares() stage.IODecl { return stage.IODecl{} }
|
||||||
func (s captureSelectedArtifactsStage) Run(_ context.Context, env *stage.Env, _ *manifest.Manifest) (*stage.StageResult, error) {
|
func (s captureSelectedArtifactsStage) Run(_ context.Context, env *stage.Env, _ *manifest.Manifest) (*stage.StageResult, error) {
|
||||||
if s.captured != nil {
|
if s.captured != nil {
|
||||||
*s.captured = append((*s.captured)[:0], env.SelectedAnalyzeArtifacts...)
|
*s.captured = append((*s.captured)[:0], env.SelectedArtifactKeys...)
|
||||||
}
|
}
|
||||||
return &stage.StageResult{Metadata: map[string]any{"captured": true}}, nil
|
return &stage.StageResult{Metadata: map[string]any{"captured": true}}, nil
|
||||||
}
|
}
|
||||||
@@ -78,12 +78,12 @@ type selectedAnalyzeArtifactStage struct {
|
|||||||
func (s selectedAnalyzeArtifactStage) Name() string { return "analyze" }
|
func (s selectedAnalyzeArtifactStage) Name() string { return "analyze" }
|
||||||
func (s selectedAnalyzeArtifactStage) Declares() stage.IODecl { return stage.IODecl{} }
|
func (s selectedAnalyzeArtifactStage) Declares() stage.IODecl { return stage.IODecl{} }
|
||||||
func (s selectedAnalyzeArtifactStage) Run(_ context.Context, env *stage.Env, m *manifest.Manifest) (*stage.StageResult, error) {
|
func (s selectedAnalyzeArtifactStage) Run(_ context.Context, env *stage.Env, m *manifest.Manifest) (*stage.StageResult, error) {
|
||||||
if len(env.SelectedAnalyzeArtifacts) != len(s.expected) {
|
if len(env.SelectedArtifactKeys) != len(s.expected) {
|
||||||
return nil, fmt.Errorf("selected artifacts len = %d, want %d", len(env.SelectedAnalyzeArtifacts), len(s.expected))
|
return nil, fmt.Errorf("selected artifacts len = %d, want %d", len(env.SelectedArtifactKeys), len(s.expected))
|
||||||
}
|
}
|
||||||
for i := range s.expected {
|
for i := range s.expected {
|
||||||
if env.SelectedAnalyzeArtifacts[i] != s.expected[i] {
|
if env.SelectedArtifactKeys[i] != s.expected[i] {
|
||||||
return nil, fmt.Errorf("selected artifacts[%d] = %q, want %q", i, env.SelectedAnalyzeArtifacts[i], s.expected[i])
|
return nil, fmt.Errorf("selected artifacts[%d] = %q, want %q", i, env.SelectedArtifactKeys[i], s.expected[i])
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -199,16 +199,65 @@ func TestExecuteStagesAnalyzeOutputsPersistAsScriptoriumArtifacts(t *testing.T)
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteStagesArchiveFailsWhenRequiredRecapPromotionMissingForSelectedArtifacts(t *testing.T) {
|
func TestNeedsObjectStoreForRunPrepareWithPreviousRequirements(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
previousSessionID string
|
||||||
|
want bool
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "previous session configured",
|
||||||
|
previousSessionID: "2026-05-10",
|
||||||
|
want: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "previous session missing",
|
||||||
|
previousSessionID: "",
|
||||||
|
want: false,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
cfg := &config.Config{
|
||||||
|
Pipeline: &config.PipelineConfig{
|
||||||
|
Scriptorium: &config.ScriptoriumConfig{
|
||||||
|
Artifacts: map[string]config.ScriptoriumArtifactConfig{
|
||||||
|
"session_recap": {
|
||||||
|
Enabled: true,
|
||||||
|
Inputs: map[string]config.ScriptoriumInputConfig{
|
||||||
|
"previous_recap": {
|
||||||
|
Source: "narratio.previous_session.artifact.session_recap",
|
||||||
|
Required: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
Session: &config.SessionConfig{
|
||||||
|
PreviousSessionID: tt.previousSessionID,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
got := needsObjectStoreForRun(cfg, []stage.Stage{countingStage{name: "prepare", runs: new(int)}})
|
||||||
|
if got != tt.want {
|
||||||
|
t.Fatalf("needsObjectStoreForRun() = %v, want %v", got, tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testing.T) {
|
||||||
cfg := testConfig(t)
|
cfg := testConfig(t)
|
||||||
cfg.Pipeline.Storage.S3 = &config.StorageS3Config{
|
cfg.Pipeline.Storage.S3 = &config.StorageS3Config{
|
||||||
Bucket: "my-dnd-archive",
|
Bucket: "my-dnd-archive",
|
||||||
RootPrefix: "dnd",
|
RootPrefix: "dnd",
|
||||||
}
|
}
|
||||||
cfg.Pipeline.Archive = &config.ArchiveConfig{
|
cfg.Pipeline.Publish = &config.PublishConfig{
|
||||||
Enabled: boolPtr(true),
|
Enabled: boolPtr(true),
|
||||||
UploadRun: boolPtr(true),
|
UploadRun: boolPtr(true),
|
||||||
PromoteArtifacts: []config.ArchivePromotionRule{
|
Outputs: []config.PublishOutputRule{
|
||||||
{Source: "narratio.artifact.session_recap", Dest: "artifacts/session_recap.md", Required: boolPtr(true)},
|
{Source: "narratio.artifact.session_recap", Dest: "artifacts/session_recap.md", Required: boolPtr(true)},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -234,12 +283,12 @@ func TestExecuteStagesArchiveFailsWhenRequiredRecapPromotionMissingForSelectedAr
|
|||||||
t.Fatalf("Save manifest error = %v", err)
|
t.Fatalf("Save manifest error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
archiveStageImpl, err := stage.Select("archive")
|
archiveStageImpl, err := stage.Select("publish")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Select(archive) error = %v", err)
|
t.Fatalf("Select(publish) error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
_, err = executeStages(
|
summary, err := executeStages(
|
||||||
context.Background(),
|
context.Background(),
|
||||||
cfg,
|
cfg,
|
||||||
[]stage.Stage{
|
[]stage.Stage{
|
||||||
@@ -251,11 +300,28 @@ func TestExecuteStagesArchiveFailsWhenRequiredRecapPromotionMissingForSelectedAr
|
|||||||
Env: &Env{ObjectStore: &storage.FakeBackend{}},
|
Env: &Env{ObjectStore: &storage.FakeBackend{}},
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
if err == nil {
|
if err != nil {
|
||||||
t.Fatal("expected archive promotion failure, got nil")
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
}
|
}
|
||||||
if !strings.Contains(err.Error(), "required promotion source unavailable") {
|
if len(summary.Executed) != 2 || summary.Executed[0] != "analyze" || summary.Executed[1] != "publish" {
|
||||||
t.Fatalf("error = %q, want required promotion source unavailable", err.Error())
|
t.Fatalf("executed = %#v, want analyze and archive", summary.Executed)
|
||||||
|
}
|
||||||
|
|
||||||
|
loadedManifest, err := store.Load(context.Background(), summary.ManifestPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Load manifest error = %v", err)
|
||||||
|
}
|
||||||
|
meta := loadedManifest.Stages["publish"].Metadata
|
||||||
|
skipped, ok := meta["skipped_unselected_outputs"].([]any)
|
||||||
|
if !ok || len(skipped) != 1 {
|
||||||
|
t.Fatalf("skipped_unselected_outputs = %#v, want one item", meta["skipped_unselected_outputs"])
|
||||||
|
}
|
||||||
|
item, ok := skipped[0].(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("skipped item = %#v, want object", skipped[0])
|
||||||
|
}
|
||||||
|
if item["source"] != "narratio.artifact.session_recap" || item["dest"] != "artifacts/session_recap.md" || item["required"] != true {
|
||||||
|
t.Fatalf("skipped item = %#v, want required session_recap promotion", item)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -276,7 +342,7 @@ func TestExecuteStagesPlaceholderSuccessUpdatesManifest(t *testing.T) {
|
|||||||
t.Fatalf("Load manifest error = %v", err)
|
t.Fatalf("Load manifest error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "archive", "notify"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
||||||
sr := m.Stages[name]
|
sr := m.Stages[name]
|
||||||
if sr == nil {
|
if sr == nil {
|
||||||
t.Fatalf("missing stage record %q", name)
|
t.Fatalf("missing stage record %q", name)
|
||||||
@@ -359,9 +425,9 @@ func TestExecuteStagesPlaceholderSuccessUpdatesManifest(t *testing.T) {
|
|||||||
}
|
}
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if name == "archive" {
|
if name == "publish" {
|
||||||
if sr.Metadata == nil || sr.Metadata["stage"] != "archive" {
|
if sr.Metadata == nil || sr.Metadata["stage"] != "publish" {
|
||||||
t.Fatalf("archive metadata missing stage=archive: %#v", sr.Metadata)
|
t.Fatalf("archive metadata missing stage=publish: %#v", sr.Metadata)
|
||||||
}
|
}
|
||||||
if sr.Metadata["skipped"] != true {
|
if sr.Metadata["skipped"] != true {
|
||||||
t.Fatalf("archive metadata missing skipped=true for test config without archive section: %#v", sr.Metadata)
|
t.Fatalf("archive metadata missing skipped=true for test config without archive section: %#v", sr.Metadata)
|
||||||
@@ -456,7 +522,7 @@ func TestExecuteStagesForceSuccessInvalidatesDownstreamSucceededStages(t *testin
|
|||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
|
|
||||||
existing := manifest.New(cfg.Session.SessionID, time.Date(2026, 5, 3, 1, 0, 0, 0, time.UTC))
|
existing := manifest.New(cfg.Session.SessionID, time.Date(2026, 5, 3, 1, 0, 0, 0, time.UTC))
|
||||||
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "archive", "notify"} {
|
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "publish", "notify"} {
|
||||||
existing.MarkStageSucceeded(stageName, time.Date(2026, 5, 3, 1, 1, 0, 0, time.UTC), nil)
|
existing.MarkStageSucceeded(stageName, time.Date(2026, 5, 3, 1, 1, 0, 0, time.UTC), nil)
|
||||||
}
|
}
|
||||||
existing.MarkStageFailed("analyze", time.Date(2026, 5, 3, 1, 1, 0, 0, time.UTC), "previous analyze failure")
|
existing.MarkStageFailed("analyze", time.Date(2026, 5, 3, 1, 1, 0, 0, time.UTC), "previous analyze failure")
|
||||||
@@ -487,7 +553,7 @@ func TestExecuteStagesForceSuccessInvalidatesDownstreamSucceededStages(t *testin
|
|||||||
if loaded.Stages["polish"] == nil || loaded.Stages["polish"].Status != manifest.StatusSucceeded {
|
if loaded.Stages["polish"] == nil || loaded.Stages["polish"].Status != manifest.StatusSucceeded {
|
||||||
t.Fatalf("polish status = %#v, want succeeded", loaded.Stages["polish"])
|
t.Fatalf("polish status = %#v, want succeeded", loaded.Stages["polish"])
|
||||||
}
|
}
|
||||||
for _, stageName := range []string{"normalize", "trim", "archive", "notify"} {
|
for _, stageName := range []string{"normalize", "trim", "publish", "notify"} {
|
||||||
if loaded.Stages[stageName] == nil || loaded.Stages[stageName].Status != manifest.StatusStale {
|
if loaded.Stages[stageName] == nil || loaded.Stages[stageName].Status != manifest.StatusStale {
|
||||||
t.Fatalf("%s status = %#v, want stale", stageName, loaded.Stages[stageName])
|
t.Fatalf("%s status = %#v, want stale", stageName, loaded.Stages[stageName])
|
||||||
}
|
}
|
||||||
@@ -695,7 +761,7 @@ func TestExecuteStagesRunLocalArtifactsAndCanonicalPromotion(t *testing.T) {
|
|||||||
filepath.Join(runRoot, "merge", "logs", "seriatim.stdout.log"),
|
filepath.Join(runRoot, "merge", "logs", "seriatim.stdout.log"),
|
||||||
filepath.Join(runRoot, "polish", "config", "audita.generated.yml"),
|
filepath.Join(runRoot, "polish", "config", "audita.generated.yml"),
|
||||||
filepath.Join(runRoot, "normalize", "logs", "seriatim.normalize.stdout.log"),
|
filepath.Join(runRoot, "normalize", "logs", "seriatim.normalize.stdout.log"),
|
||||||
filepath.Join(runRoot, "trim", "outputs", "transcripts", "trimmed.json"),
|
filepath.Join(runRoot, "trim", "outputs", "transcripts", "final.trimmed.json"),
|
||||||
}
|
}
|
||||||
for _, p := range runLocalChecks {
|
for _, p := range runLocalChecks {
|
||||||
if _, statErr := os.Stat(p); statErr != nil {
|
if _, statErr := os.Stat(p); statErr != nil {
|
||||||
@@ -705,10 +771,10 @@ func TestExecuteStagesRunLocalArtifactsAndCanonicalPromotion(t *testing.T) {
|
|||||||
|
|
||||||
canonicalChecks := []string{
|
canonicalChecks := []string{
|
||||||
filepath.Join(paths.TranscriptsRawDir, "alice.json"),
|
filepath.Join(paths.TranscriptsRawDir, "alice.json"),
|
||||||
filepath.Join(paths.TranscriptsDir, "merged.json"),
|
filepath.Join(paths.TranscriptsDir, "base.json"),
|
||||||
filepath.Join(paths.TranscriptsDir, "processed.json"),
|
filepath.Join(paths.TranscriptsDir, "polished.json"),
|
||||||
filepath.Join(paths.TranscriptsDir, "normalized.json"),
|
filepath.Join(paths.TranscriptsDir, "final.json"),
|
||||||
filepath.Join(paths.TranscriptsDir, "trimmed.json"),
|
filepath.Join(paths.TranscriptsDir, "final.trimmed.json"),
|
||||||
}
|
}
|
||||||
for _, p := range canonicalChecks {
|
for _, p := range canonicalChecks {
|
||||||
if _, statErr := os.Stat(p); statErr != nil {
|
if _, statErr := os.Stat(p); statErr != nil {
|
||||||
@@ -798,7 +864,7 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
|
|||||||
{name: "merge", env: &Env{Seriatim: &seriatim.FakeRunner{Err: errors.New("merge fail")}}},
|
{name: "merge", env: &Env{Seriatim: &seriatim.FakeRunner{Err: errors.New("merge fail")}}},
|
||||||
{name: "polish", env: &Env{Audita: &audita.FakeRunner{Err: errors.New("polish fail")}}},
|
{name: "polish", env: &Env{Audita: &audita.FakeRunner{Err: errors.New("polish fail")}}},
|
||||||
{name: "analyze", env: &Env{Scriptorium: &scriptorium.FakeRunner{RunErr: errors.New("analyze fail")}}},
|
{name: "analyze", env: &Env{Scriptorium: &scriptorium.FakeRunner{RunErr: errors.New("analyze fail")}}},
|
||||||
{name: "archive", env: &Env{ObjectStore: &storage.FakeBackend{UploadErr: errors.New("archive fail")}}},
|
{name: "publish", env: &Env{ObjectStore: &storage.FakeBackend{UploadErr: errors.New("archive fail")}}},
|
||||||
{name: "notify", env: &Env{Notifier: ¬ify.FakeSender{Err: errors.New("notify fail")}}},
|
{name: "notify", env: &Env{Notifier: ¬ify.FakeSender{Err: errors.New("notify fail")}}},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -853,7 +919,7 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
|
|||||||
if ensureErr != nil {
|
if ensureErr != nil {
|
||||||
t.Fatalf("EnsureLayout() error = %v", ensureErr)
|
t.Fatalf("EnsureLayout() error = %v", ensureErr)
|
||||||
}
|
}
|
||||||
if err := os.WriteFile(filepath.Join(paths.TranscriptsDir, "merged.json"), []byte(`{"segments":[]}`), 0o644); err != nil {
|
if err := os.WriteFile(filepath.Join(paths.TranscriptsDir, "base.json"), []byte(`{"segments":[]}`), 0o644); err != nil {
|
||||||
t.Fatalf("write merged transcript: %v", err)
|
t.Fatalf("write merged transcript: %v", err)
|
||||||
}
|
}
|
||||||
if err := os.WriteFile(filepath.Join(paths.InputsDir, "glossary.yml"), []byte("terms: []\n"), 0o644); err != nil {
|
if err := os.WriteFile(filepath.Join(paths.InputsDir, "glossary.yml"), []byte("terms: []\n"), 0o644); err != nil {
|
||||||
@@ -865,7 +931,7 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
|
|||||||
if ensureErr != nil {
|
if ensureErr != nil {
|
||||||
t.Fatalf("EnsureLayout() error = %v", ensureErr)
|
t.Fatalf("EnsureLayout() error = %v", ensureErr)
|
||||||
}
|
}
|
||||||
if err := os.WriteFile(filepath.Join(paths.TranscriptsDir, "processed.json"), []byte(`{"segments":[]}`), 0o644); err != nil {
|
if err := os.WriteFile(filepath.Join(paths.TranscriptsDir, "polished.json"), []byte(`{"segments":[]}`), 0o644); err != nil {
|
||||||
t.Fatalf("write processed transcript: %v", err)
|
t.Fatalf("write processed transcript: %v", err)
|
||||||
}
|
}
|
||||||
cfg.Pipeline.Scriptorium = &config.ScriptoriumConfig{
|
cfg.Pipeline.Scriptorium = &config.ScriptoriumConfig{
|
||||||
@@ -883,8 +949,8 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
|
|||||||
},
|
},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if tc.name == "archive" {
|
if tc.name == "publish" {
|
||||||
cfg.Pipeline.Archive = &config.ArchiveConfig{
|
cfg.Pipeline.Publish = &config.PublishConfig{
|
||||||
Enabled: boolPtr(true),
|
Enabled: boolPtr(true),
|
||||||
UploadRun: boolPtr(true),
|
UploadRun: boolPtr(true),
|
||||||
}
|
}
|
||||||
@@ -945,10 +1011,12 @@ func testConfig(t *testing.T) *config.Config {
|
|||||||
workspace := t.TempDir()
|
workspace := t.TempDir()
|
||||||
cfgDir := t.TempDir()
|
cfgDir := t.TempDir()
|
||||||
sessionPath := filepath.Join(cfgDir, "session.yml")
|
sessionPath := filepath.Join(cfgDir, "session.yml")
|
||||||
|
campaignPath := filepath.Join(cfgDir, "campaign.yml")
|
||||||
pipelinePath := filepath.Join(cfgDir, "pipeline.yml")
|
pipelinePath := filepath.Join(cfgDir, "pipeline.yml")
|
||||||
|
|
||||||
mustWriteFile(t, pipelinePath, "workspace:\n root: "+workspace+"\n")
|
mustWriteFile(t, pipelinePath, "workspace:\n root: "+workspace+"\n")
|
||||||
mustWriteFile(t, sessionPath, "session_id: 2026-05-03\ncampaign: sample-campaign\n")
|
mustWriteFile(t, campaignPath, "campaign_id: sample-campaign\ninputs:\n speakers_file: ./speakers.yml\n autocorrect_file: ./autocorrect.yml\n glossary_file: ./glossary.yml\n")
|
||||||
|
mustWriteFile(t, sessionPath, "session_id: 2026-05-03\ncampaign: sample-campaign\ninputs:\n audio_dir: ./audio\n")
|
||||||
mustWriteFile(t, filepath.Join(cfgDir, "speakers.yml"), "alice: alice.flac\n")
|
mustWriteFile(t, filepath.Join(cfgDir, "speakers.yml"), "alice: alice.flac\n")
|
||||||
mustWriteFile(t, filepath.Join(cfgDir, "autocorrect.yml"), "[]\n")
|
mustWriteFile(t, filepath.Join(cfgDir, "autocorrect.yml"), "[]\n")
|
||||||
mustWriteFile(t, filepath.Join(cfgDir, "glossary.yml"), "[]\n")
|
mustWriteFile(t, filepath.Join(cfgDir, "glossary.yml"), "[]\n")
|
||||||
@@ -956,8 +1024,27 @@ func testConfig(t *testing.T) *config.Config {
|
|||||||
|
|
||||||
return &config.Config{
|
return &config.Config{
|
||||||
Pipeline: &config.PipelineConfig{Workspace: config.WorkspaceConfig{Root: workspace}},
|
Pipeline: &config.PipelineConfig{Workspace: config.WorkspaceConfig{Root: workspace}},
|
||||||
|
Campaign: &config.CampaignConfig{CampaignID: "sample-campaign"},
|
||||||
PipelinePath: pipelinePath,
|
PipelinePath: pipelinePath,
|
||||||
|
CampaignPath: campaignPath,
|
||||||
SessionPath: sessionPath,
|
SessionPath: sessionPath,
|
||||||
|
StableInputs: config.ResolvedStableInputs{
|
||||||
|
SpeakersFile: config.ResolvedInputFile{
|
||||||
|
Path: "./speakers.yml",
|
||||||
|
ConfigPath: campaignPath,
|
||||||
|
Source: "campaign_config",
|
||||||
|
},
|
||||||
|
AutocorrectFile: config.ResolvedInputFile{
|
||||||
|
Path: "./autocorrect.yml",
|
||||||
|
ConfigPath: campaignPath,
|
||||||
|
Source: "campaign_config",
|
||||||
|
},
|
||||||
|
GlossaryFile: config.ResolvedInputFile{
|
||||||
|
Path: "./glossary.yml",
|
||||||
|
ConfigPath: campaignPath,
|
||||||
|
Source: "campaign_config",
|
||||||
|
},
|
||||||
|
},
|
||||||
Session: &config.SessionConfig{
|
Session: &config.SessionConfig{
|
||||||
SessionID: "2026-05-03",
|
SessionID: "2026-05-03",
|
||||||
Campaign: "sample-campaign",
|
Campaign: "sample-campaign",
|
||||||
@@ -974,15 +1061,20 @@ func testConfig(t *testing.T) *config.Config {
|
|||||||
func TestBuildDefaultRunnersWithOmittedToolSections(t *testing.T) {
|
func TestBuildDefaultRunnersWithOmittedToolSections(t *testing.T) {
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
pipelinePath := filepath.Join(dir, "pipeline.yml")
|
pipelinePath := filepath.Join(dir, "pipeline.yml")
|
||||||
|
campaignPath := filepath.Join(dir, "campaign.yml")
|
||||||
sessionPath := filepath.Join(dir, "session.yml")
|
sessionPath := filepath.Join(dir, "session.yml")
|
||||||
pipelineYAML := `workspace:
|
pipelineYAML := `workspace:
|
||||||
root: ` + t.TempDir() + `
|
root: ` + t.TempDir() + `
|
||||||
whisperx:
|
whisperx:
|
||||||
transcribe_url: https://example.com/transcribe
|
transcribe_url: https://example.com/transcribe
|
||||||
analyzer:
|
|
||||||
timeout: 20m
|
|
||||||
notification:
|
notification:
|
||||||
timeout: 10s
|
timeout: 10s
|
||||||
|
`
|
||||||
|
campaignYAML := `campaign_id: sample-campaign
|
||||||
|
inputs:
|
||||||
|
speakers_file: ./speakers.yml
|
||||||
|
autocorrect_file: ./autocorrect.yml
|
||||||
|
glossary_file: ./glossary.yml
|
||||||
`
|
`
|
||||||
sessionYAML := `session_id: 2026-05-03
|
sessionYAML := `session_id: 2026-05-03
|
||||||
campaign: sample-campaign
|
campaign: sample-campaign
|
||||||
@@ -993,6 +1085,7 @@ inputs:
|
|||||||
glossary_file: ./glossary.yml
|
glossary_file: ./glossary.yml
|
||||||
`
|
`
|
||||||
mustWriteFile(t, pipelinePath, pipelineYAML)
|
mustWriteFile(t, pipelinePath, pipelineYAML)
|
||||||
|
mustWriteFile(t, campaignPath, campaignYAML)
|
||||||
mustWriteFile(t, sessionPath, sessionYAML)
|
mustWriteFile(t, sessionPath, sessionYAML)
|
||||||
|
|
||||||
cfg, err := config.Load(pipelinePath, sessionPath)
|
cfg, err := config.Load(pipelinePath, sessionPath)
|
||||||
|
|||||||
43
internal/app/session_args.go
Normal file
43
internal/app/session_args.go
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
func isCLIFlagToken(arg string) bool {
|
||||||
|
return strings.HasPrefix(arg, "-") && arg != "-"
|
||||||
|
}
|
||||||
|
|
||||||
|
func pullLeadingSessionID(args []string) (string, []string) {
|
||||||
|
if len(args) == 0 || isCLIFlagToken(args[0]) {
|
||||||
|
return "", args
|
||||||
|
}
|
||||||
|
rest := append([]string(nil), args[1:]...)
|
||||||
|
return strings.TrimSpace(args[0]), rest
|
||||||
|
}
|
||||||
|
|
||||||
|
func applyPositionalSessionID(command, positional string, sessionID *string) error {
|
||||||
|
positional = strings.TrimSpace(positional)
|
||||||
|
if positional == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
existing := strings.TrimSpace(*sessionID)
|
||||||
|
if existing != "" && existing != positional {
|
||||||
|
return fmt.Errorf("%s: positional session id %q does not match expected session id %q", command, positional, existing)
|
||||||
|
}
|
||||||
|
*sessionID = positional
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func applyParsedSessionIDArg(command string, fs *flag.FlagSet, sessionID *string) error {
|
||||||
|
switch fs.NArg() {
|
||||||
|
case 0:
|
||||||
|
return nil
|
||||||
|
case 1:
|
||||||
|
return applyPositionalSessionID(command, fs.Arg(0), sessionID)
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("%s: unexpected positional arguments", command)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -4,16 +4,18 @@ import (
|
|||||||
"bytes"
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestPlanUsesDiscoveredSessionTemplateWithSessionID(t *testing.T) {
|
func TestPlanRejectsDiscoveredSessionTemplate(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
sessionTemplate := `session_id: "{{ session_id }}"
|
sessionTemplate := `session_id: "{{ session_id }}"
|
||||||
|
previous_session_id: "{{ previous_session_id }}"
|
||||||
campaign: sample-campaign
|
campaign: sample-campaign
|
||||||
inputs:
|
inputs:
|
||||||
audio_dir: ./audio
|
audio_dir: ./audio
|
||||||
@@ -25,31 +27,34 @@ inputs:
|
|||||||
t.Fatalf("write session template: %v", err)
|
t.Fatalf("write session template: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
cwd := filepath.Dir(sessionPath)
|
origSessionDefaults := append([]string(nil), config.DefaultSessionConfigSearchPaths...)
|
||||||
originalWD, err := os.Getwd()
|
config.DefaultSessionConfigSearchPaths = []string{sessionPath}
|
||||||
if err != nil {
|
t.Cleanup(func() { config.DefaultSessionConfigSearchPaths = origSessionDefaults })
|
||||||
t.Fatalf("Getwd(): %v", err)
|
|
||||||
}
|
|
||||||
if err := os.Chdir(cwd); err != nil {
|
|
||||||
t.Fatalf("Chdir(%q): %v", cwd, err)
|
|
||||||
}
|
|
||||||
t.Cleanup(func() { _ = os.Chdir(originalWD) })
|
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
if err := Plan(context.Background(), []string{"--config", pipelinePath, "--session-id", "2026-04-04"}, &out); err != nil {
|
err := Plan(context.Background(), []string{
|
||||||
t.Fatalf("Plan() error = %v", err)
|
"2026-04-04",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--previous-session-id", "2026-03-28",
|
||||||
|
}, &out)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error, got nil")
|
||||||
}
|
}
|
||||||
if !strings.Contains(out.String(), "narratio plan: workdir prepared") {
|
if !strings.Contains(err.Error(), "session.yml must be concrete") {
|
||||||
t.Fatalf("output = %q, want plan output", out.String())
|
t.Fatalf("error = %q, want concrete session guidance", err.Error())
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "run narratio session init") {
|
||||||
|
t.Fatalf("error = %q, want session init guidance", err.Error())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPlanFailsWhenSessionIDMismatchesConcreteSession(t *testing.T) {
|
func TestPlanFailsWhenSessionIDMismatchesConcreteSession(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Plan(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "--session-id", "2026-04-04"}, &out)
|
err := Plan(context.Background(), []string{"2026-04-04", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected error, got nil")
|
t.Fatal("expected error, got nil")
|
||||||
}
|
}
|
||||||
@@ -58,12 +63,45 @@ func TestPlanFailsWhenSessionIDMismatchesConcreteSession(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRunStageAcceptsSessionIDFlagAndParsesStageName(t *testing.T) {
|
func TestPlanFailsWhenPreviousSessionIDMismatchesConcreteSession(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
sessionYAML := `session_id: 2026-05-03
|
||||||
|
previous_session_id: 2026-04-26
|
||||||
|
campaign: sample-campaign
|
||||||
|
inputs:
|
||||||
|
audio_dir: ./audio
|
||||||
|
speakers_file: ./speakers.yml
|
||||||
|
autocorrect_file: ./autocorrect.yml
|
||||||
|
glossary_file: ./glossary.yml
|
||||||
|
`
|
||||||
|
if err := os.WriteFile(sessionPath, []byte(sessionYAML), 0o644); err != nil {
|
||||||
|
t.Fatalf("write session.yml: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := RunStage(context.Background(), []string{"--config", pipelinePath, "--session", sessionPath, "--session-id", "2026-05-03", "prepare"}, &out)
|
err := Plan(context.Background(), []string{
|
||||||
|
"2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
"--previous-session-id", "2026-04-25",
|
||||||
|
}, &out)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error, got nil")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "previous_session_id mismatch") {
|
||||||
|
t.Fatalf("error = %q, want mismatch context", err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunStageAcceptsPositionalSessionIDAndParsesStageName(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
var out bytes.Buffer
|
||||||
|
err := RunStage(context.Background(), []string{"prepare", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("RunStage() error = %v", err)
|
t.Fatalf("RunStage() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -73,7 +111,7 @@ func TestRunStageAcceptsSessionIDFlagAndParsesStageName(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestResolveSessionConfigPathErrorIncludesSearchedPaths(t *testing.T) {
|
func TestResolveSessionConfigPathErrorIncludesSearchedPaths(t *testing.T) {
|
||||||
_, err := resolveSessionConfigPathWithCandidates("", []string{"./session.yml", "/usr/local/etc/narratio/session.yml", "/etc/narratio/session.yml"})
|
_, err := resolveSessionConfigPathWithCandidates("", []string{"/usr/local/etc/narratio/session.yml", "/etc/narratio/session.yml"})
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected error, got nil")
|
t.Fatal("expected error, got nil")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -19,6 +19,22 @@ func resolveSessionConfigPathWithCandidates(flagValue string, candidates []strin
|
|||||||
return explicit, nil
|
return explicit, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
resolved, err := discoverSessionConfigPathWithCandidates(candidates)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if resolved.Path != "" {
|
||||||
|
return resolved.Path, nil
|
||||||
|
}
|
||||||
|
return "", missingSessionConfigError(resolved.Searched, "")
|
||||||
|
}
|
||||||
|
|
||||||
|
type sessionConfigDiscovery struct {
|
||||||
|
Path string
|
||||||
|
Searched []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func discoverSessionConfigPathWithCandidates(candidates []string) (sessionConfigDiscovery, error) {
|
||||||
ordered := make([]string, 0, len(candidates))
|
ordered := make([]string, 0, len(candidates))
|
||||||
for _, raw := range candidates {
|
for _, raw := range candidates {
|
||||||
path := strings.TrimSpace(raw)
|
path := strings.TrimSpace(raw)
|
||||||
@@ -31,19 +47,32 @@ func resolveSessionConfigPathWithCandidates(flagValue string, candidates []strin
|
|||||||
if info.IsDir() {
|
if info.IsDir() {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
return filepath.Clean(path), nil
|
return sessionConfigDiscovery{Path: filepath.Clean(path), Searched: ordered}, nil
|
||||||
}
|
}
|
||||||
if errors.Is(err, os.ErrNotExist) {
|
if errors.Is(err, os.ErrNotExist) {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
return "", fmt.Errorf("check default session config %q: %w", path, err)
|
return sessionConfigDiscovery{}, fmt.Errorf("check default session config %q: %w", path, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
return sessionConfigDiscovery{Searched: ordered}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func missingSessionConfigError(searched []string, remoteDetail string) error {
|
||||||
|
ordered := append([]string(nil), searched...)
|
||||||
if len(ordered) == 0 {
|
if len(ordered) == 0 {
|
||||||
return "", fmt.Errorf("no session config path provided and no default locations configured")
|
if strings.TrimSpace(remoteDetail) != "" {
|
||||||
|
return fmt.Errorf("no session config path provided and no default locations configured; %s", remoteDetail)
|
||||||
}
|
}
|
||||||
return "", fmt.Errorf(
|
return fmt.Errorf("no session config path provided and no default locations configured")
|
||||||
"no session config path provided and no default session config found; searched: %s; pass --session to use an explicit path",
|
}
|
||||||
|
msg := fmt.Sprintf(
|
||||||
|
"no session config path provided and no default session config found; searched: %s",
|
||||||
strings.Join(ordered, ", "),
|
strings.Join(ordered, ", "),
|
||||||
)
|
)
|
||||||
|
if strings.TrimSpace(remoteDetail) != "" {
|
||||||
|
msg += "; " + strings.TrimSpace(remoteDetail)
|
||||||
|
}
|
||||||
|
msg += "; pass --session to use an explicit path"
|
||||||
|
return fmt.Errorf("%s", msg)
|
||||||
}
|
}
|
||||||
|
|||||||
353
internal/app/session_oriented_cli_test.go
Normal file
353
internal/app/session_oriented_cli_test.go
Normal file
@@ -0,0 +1,353 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/stage"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestExecuteRunAcceptsPositionalSessionID(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
var capturedSessionID string
|
||||||
|
origExecuteStagesFn := executeStagesFn
|
||||||
|
t.Cleanup(func() { executeStagesFn = origExecuteStagesFn })
|
||||||
|
executeStagesFn = func(_ context.Context, cfg *config.Config, _ []stage.Stage, _ RunOptions) (*RunSummary, error) {
|
||||||
|
capturedSessionID = cfg.Session.SessionID
|
||||||
|
return &RunSummary{
|
||||||
|
SessionID: cfg.Session.SessionID,
|
||||||
|
ManifestPath: filepath.Join(workspaceRoot, "manifest.json"),
|
||||||
|
Executed: []string{"prepare"},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"run",
|
||||||
|
"2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "narratio run: session 2026-05-03") {
|
||||||
|
t.Fatalf("stdout = %q, want run summary", stdout.String())
|
||||||
|
}
|
||||||
|
if capturedSessionID != "2026-05-03" {
|
||||||
|
t.Fatalf("captured session = %q, want positional session id", capturedSessionID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecutePositionalSessionIDMismatchFails(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"run",
|
||||||
|
"2026-05-04",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "session_id mismatch") {
|
||||||
|
t.Fatalf("stderr = %q, want session mismatch", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionIDFlagFails(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"session", "status", "2026-05-03", "--session-id", "2026-05-04"}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "flag provided but not defined: -session-id") {
|
||||||
|
t.Fatalf("stderr = %q, want invalid --session-id flag", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteRemoteSessionFallbackUsesPositionalSessionID(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{filepath.Join(t.TempDir(), "session.yml")})
|
||||||
|
remoteKey := seedRemoteSessionConfig(t, fake, "2026-06-07", `session_id: 2026-06-07
|
||||||
|
inputs:
|
||||||
|
audio_s3:
|
||||||
|
prefix: audio/
|
||||||
|
`)
|
||||||
|
origExecuteStagesFn := executeStagesFn
|
||||||
|
t.Cleanup(func() { executeStagesFn = origExecuteStagesFn })
|
||||||
|
executeStagesFn = func(_ context.Context, cfg *config.Config, _ []stage.Stage, _ RunOptions) (*RunSummary, error) {
|
||||||
|
return &RunSummary{
|
||||||
|
SessionID: cfg.Session.SessionID,
|
||||||
|
ManifestPath: filepath.Join(workspaceRoot, "manifest.json"),
|
||||||
|
Executed: []string{"prepare"},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"run",
|
||||||
|
"2026-06-07",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
downloaded := false
|
||||||
|
for _, call := range fake.Downloads {
|
||||||
|
if call.Key == remoteKey {
|
||||||
|
downloaded = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !downloaded {
|
||||||
|
t.Fatalf("remote session %q was not downloaded; downloads=%v", remoteKey, fake.Downloads)
|
||||||
|
}
|
||||||
|
if storeInitCalls == 0 {
|
||||||
|
t.Fatal("object store was not initialized")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteWorkflowCommandsAcceptPositionalSessionID(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
args []string
|
||||||
|
wantStage string
|
||||||
|
wantForce bool
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "resume",
|
||||||
|
args: []string{"resume", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||||
|
wantStage: "prepare",
|
||||||
|
wantForce: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "analyze",
|
||||||
|
args: []string{"analyze", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||||
|
wantStage: "analyze",
|
||||||
|
wantForce: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "publish",
|
||||||
|
args: []string{"publish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||||
|
wantStage: "publish",
|
||||||
|
wantForce: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "run-stage",
|
||||||
|
args: []string{"run-stage", "publish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||||
|
wantStage: "publish",
|
||||||
|
wantForce: false,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
var capturedStages []string
|
||||||
|
var capturedForce bool
|
||||||
|
var capturedArtifacts []string
|
||||||
|
origExecuteStagesFn := executeStagesFn
|
||||||
|
t.Cleanup(func() { executeStagesFn = origExecuteStagesFn })
|
||||||
|
executeStagesFn = func(_ context.Context, _ *config.Config, stages []stage.Stage, opts RunOptions) (*RunSummary, error) {
|
||||||
|
for _, s := range stages {
|
||||||
|
capturedStages = append(capturedStages, s.Name())
|
||||||
|
}
|
||||||
|
capturedForce = opts.Force
|
||||||
|
capturedArtifacts = append([]string(nil), opts.SelectedArtifacts...)
|
||||||
|
return &RunSummary{
|
||||||
|
SessionID: "2026-05-03",
|
||||||
|
ManifestPath: filepath.Join(workspaceRoot, "manifest.json"),
|
||||||
|
Executed: []string{tt.wantStage},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(tt.args, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if len(capturedStages) == 0 || capturedStages[0] != tt.wantStage {
|
||||||
|
t.Fatalf("captured stages = %#v, want first %q", capturedStages, tt.wantStage)
|
||||||
|
}
|
||||||
|
if capturedForce != tt.wantForce {
|
||||||
|
t.Fatalf("captured force = %t, want %t", capturedForce, tt.wantForce)
|
||||||
|
}
|
||||||
|
if tt.name == "analyze" || tt.name == "publish" || tt.name == "run-stage" {
|
||||||
|
if strings.Join(capturedArtifacts, ",") != "session_recap" {
|
||||||
|
t.Fatalf("captured artifacts = %#v, want [session_recap]", capturedArtifacts)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionSubcommandsAcceptPositionalSessionID(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
|
||||||
|
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||||
|
fake.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, "2026-05-03", "sample-campaign")})
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
args []string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "validate",
|
||||||
|
args: []string{"session", "validate", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||||
|
want: "OK config",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "status",
|
||||||
|
args: []string{"session", "status", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||||
|
want: "Session: 2026-05-03",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "plan",
|
||||||
|
args: []string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||||
|
want: "narratio session plan: workdir prepared",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "artifacts",
|
||||||
|
args: []string{"session", "artifacts", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||||
|
want: "Built-in:",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "locks",
|
||||||
|
args: []string{"session", "locks", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||||
|
want: "Publish locks:",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(tt.args, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stdout=%q stderr=%q", code, stdout.String(), stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), tt.want) {
|
||||||
|
t.Fatalf("stdout = %q, want %q", stdout.String(), tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionInitAcceptsPositionalSessionID(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
outputPath := filepath.Join(t.TempDir(), "session.yml")
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "init", "2026-06-07",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--output", outputPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(outputPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read generated session: %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(string(data), `session_id: "2026-06-07"`) {
|
||||||
|
t.Fatalf("generated session = %q, want positional session id", string(data))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionLocksMutationAcceptsPositionalSessionID(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
"--reason", "review",
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("locks add exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
key := artifacts.S3SessionLocksKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"))
|
||||||
|
if !strings.Contains(string(fake.Objects[key].Data), "reason: review") {
|
||||||
|
t.Fatalf("lock store data = %q, want reason", string(fake.Objects[key].Data))
|
||||||
|
}
|
||||||
|
|
||||||
|
stdout.Reset()
|
||||||
|
stderr.Reset()
|
||||||
|
code = Execute([]string{
|
||||||
|
"session", "locks", "remove", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("locks remove exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
store, err := config.LoadPublishLockStoreBytes("locks.yml", fake.Objects[key].Data, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("LoadPublishLockStoreBytes() error = %v", err)
|
||||||
|
}
|
||||||
|
if len(store.Locks) != 0 {
|
||||||
|
t.Fatalf("locks after remove = %#v, want empty", store.Locks)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteCleanAcceptsPositionalSessionID(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
workDir := artifacts.SessionWorkDirForCampaign(workspaceRoot, "sample-campaign", "2026-05-03")
|
||||||
|
spoolDir := artifacts.SessionSpoolDir(filepath.Join(workspaceRoot, "spool"), "sample-campaign", "2026-05-03")
|
||||||
|
mustWriteTestFile(t, filepath.Join(workDir, "manifest.json"), "{}")
|
||||||
|
mustWriteTestFile(t, filepath.Join(spoolDir, "run-1", "audio", "alice.flac"), "audio")
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{"clean", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
cleanAssertMissing(t, workDir)
|
||||||
|
cleanAssertMissing(t, spoolDir)
|
||||||
|
}
|
||||||
@@ -1,67 +0,0 @@
|
|||||||
package app
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"flag"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"sort"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
|
||||||
)
|
|
||||||
|
|
||||||
// Status reads and prints stage statuses from an existing manifest.
|
|
||||||
func Status(ctx context.Context, args []string, out io.Writer) error {
|
|
||||||
fs := flag.NewFlagSet("status", flag.ContinueOnError)
|
|
||||||
fs.SetOutput(io.Discard)
|
|
||||||
|
|
||||||
var manifestPath string
|
|
||||||
fs.StringVar(&manifestPath, "manifest", "", "path to manifest.json")
|
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
|
||||||
return fmt.Errorf("status: invalid flags: %w", err)
|
|
||||||
}
|
|
||||||
if fs.NArg() != 0 {
|
|
||||||
return fmt.Errorf("status: unexpected positional arguments")
|
|
||||||
}
|
|
||||||
if manifestPath == "" {
|
|
||||||
return fmt.Errorf("status: --manifest is required")
|
|
||||||
}
|
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
|
||||||
m, err := store.Load(ctx, manifestPath)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("status: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if _, err := fmt.Fprintf(out, "session_id: %s\n", m.SessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if _, err := fmt.Fprintf(out, "updated_at: %s\n", m.UpdatedAt.UTC().Format("2006-01-02T15:04:05Z07:00")); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(m.Stages) == 0 {
|
|
||||||
_, err := fmt.Fprintln(out, "stages: no stages recorded")
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
if _, err := fmt.Fprintln(out, "stages:"); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
names := make([]string, 0, len(m.Stages))
|
|
||||||
for name := range m.Stages {
|
|
||||||
names = append(names, name)
|
|
||||||
}
|
|
||||||
sort.Strings(names)
|
|
||||||
|
|
||||||
for _, name := range names {
|
|
||||||
status := m.Stages[name].Status
|
|
||||||
if _, err := fmt.Fprintf(out, "- %s: %s\n", name, status); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
@@ -3,71 +3,28 @@ package app
|
|||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestStatusCommandReadsManifest(t *testing.T) {
|
func TestStatusCommandRequiresSessionID(t *testing.T) {
|
||||||
manifestPath := writeManifestForStatus(t)
|
|
||||||
|
|
||||||
var out bytes.Buffer
|
|
||||||
err := Status(context.Background(), []string{"--manifest", manifestPath}, &out)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Status() error = %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
s := out.String()
|
|
||||||
if !strings.Contains(s, "session_id: 2026-05-03") {
|
|
||||||
t.Fatalf("output = %q, want session_id", s)
|
|
||||||
}
|
|
||||||
if !strings.Contains(s, "- merge: succeeded") {
|
|
||||||
t.Fatalf("output = %q, want stage status", s)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestStatusCommandMissingManifestFlag(t *testing.T) {
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Status(context.Background(), nil, &out)
|
err := Status(context.Background(), nil, &out)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected error, got nil")
|
t.Fatal("expected error, got nil")
|
||||||
}
|
}
|
||||||
if !strings.Contains(err.Error(), "--manifest is required") {
|
if !strings.Contains(err.Error(), "status: session_id is required") {
|
||||||
t.Fatalf("error = %q, want missing manifest flag", err.Error())
|
t.Fatalf("error = %q, want missing session_id error", err.Error())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestStatusCommandBadManifest(t *testing.T) {
|
func TestStatusCommandRejectsManifestFlag(t *testing.T) {
|
||||||
dir := t.TempDir()
|
|
||||||
path := filepath.Join(dir, "manifest.json")
|
|
||||||
if err := os.WriteFile(path, []byte("{not-json"), 0o644); err != nil {
|
|
||||||
t.Fatalf("WriteFile() error = %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Status(context.Background(), []string{"--manifest", path}, &out)
|
err := Status(context.Background(), []string{"2026-05-03", "--manifest", "manifest.json"}, &out)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected error, got nil")
|
t.Fatal("expected error, got nil")
|
||||||
}
|
}
|
||||||
if !strings.Contains(err.Error(), "decode manifest") {
|
if !strings.Contains(err.Error(), "status: invalid flags: flag provided but not defined: -manifest") {
|
||||||
t.Fatalf("error = %q, want decode error", err.Error())
|
t.Fatalf("error = %q, want invalid manifest flag", err.Error())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func writeManifestForStatus(t *testing.T) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
|
||||||
m := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
|
||||||
m.MarkStageSucceeded("merge", time.Date(2026, 5, 3, 10, 5, 0, 0, time.UTC), nil)
|
|
||||||
|
|
||||||
path := filepath.Join(t.TempDir(), "manifest.json")
|
|
||||||
if err := store.Save(context.Background(), path, m); err != nil {
|
|
||||||
t.Fatalf("Save() error = %v", err)
|
|
||||||
}
|
|
||||||
return path
|
|
||||||
}
|
|
||||||
|
|||||||
84
internal/artifactmodel/transcripts.go
Normal file
84
internal/artifactmodel/transcripts.go
Normal file
@@ -0,0 +1,84 @@
|
|||||||
|
package artifactmodel
|
||||||
|
|
||||||
|
import "strings"
|
||||||
|
|
||||||
|
const (
|
||||||
|
SourceTranscriptBase = "narratio.transcript.base"
|
||||||
|
SourceTranscriptPolished = "narratio.transcript.polished"
|
||||||
|
SourceTranscriptFinal = "narratio.transcript.final"
|
||||||
|
SourceTranscriptFinalTrimmed = "narratio.transcript.final_trimmed"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
TranscriptPathBase = "transcripts/base.json"
|
||||||
|
TranscriptPathPolished = "transcripts/polished.json"
|
||||||
|
TranscriptPathFinal = "transcripts/final.json"
|
||||||
|
TranscriptPathFinalTrimmed = "transcripts/final.trimmed.json"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
TranscriptOutputKindBase = "transcript_base"
|
||||||
|
TranscriptOutputKindPolished = "transcript_polished"
|
||||||
|
TranscriptOutputKindFinal = "transcript_final"
|
||||||
|
TranscriptOutputKindFinalTrimmed = "transcript_final_trimmed"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TranscriptArtifactSpec describes one built-in transcript artifact mapping.
|
||||||
|
type TranscriptArtifactSpec struct {
|
||||||
|
SourceID string
|
||||||
|
CanonicalRelPath string
|
||||||
|
ProducerStage string
|
||||||
|
OutputKind string
|
||||||
|
}
|
||||||
|
|
||||||
|
var runtimeTranscriptArtifacts = []TranscriptArtifactSpec{
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptBase,
|
||||||
|
CanonicalRelPath: TranscriptPathBase,
|
||||||
|
ProducerStage: "merge",
|
||||||
|
OutputKind: TranscriptOutputKindBase,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptPolished,
|
||||||
|
CanonicalRelPath: TranscriptPathPolished,
|
||||||
|
ProducerStage: "polish",
|
||||||
|
OutputKind: TranscriptOutputKindPolished,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptFinal,
|
||||||
|
CanonicalRelPath: TranscriptPathFinal,
|
||||||
|
ProducerStage: "normalize",
|
||||||
|
OutputKind: TranscriptOutputKindFinal,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptFinalTrimmed,
|
||||||
|
CanonicalRelPath: TranscriptPathFinalTrimmed,
|
||||||
|
ProducerStage: "trim",
|
||||||
|
OutputKind: TranscriptOutputKindFinalTrimmed,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
// RuntimeTranscriptArtifacts returns transcript mappings in pipeline order.
|
||||||
|
func RuntimeTranscriptArtifacts() []TranscriptArtifactSpec {
|
||||||
|
return cloneTranscriptSpecs(runtimeTranscriptArtifacts)
|
||||||
|
}
|
||||||
|
|
||||||
|
// LookupRuntimeTranscriptArtifact returns runtime transcript metadata by source ID.
|
||||||
|
func LookupRuntimeTranscriptArtifact(sourceID string) (TranscriptArtifactSpec, bool) {
|
||||||
|
trimmed := strings.TrimSpace(sourceID)
|
||||||
|
for _, spec := range runtimeTranscriptArtifacts {
|
||||||
|
if spec.SourceID == trimmed {
|
||||||
|
return spec, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return TranscriptArtifactSpec{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
func cloneTranscriptSpecs(specs []TranscriptArtifactSpec) []TranscriptArtifactSpec {
|
||||||
|
if len(specs) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
out := make([]TranscriptArtifactSpec, len(specs))
|
||||||
|
copy(out, specs)
|
||||||
|
return out
|
||||||
|
}
|
||||||
@@ -9,20 +9,39 @@ import (
|
|||||||
"regexp"
|
"regexp"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifactmodel"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
ArtifactTranscriptMerged = "narratio.transcript.merged"
|
ArtifactTranscriptBase = artifactmodel.SourceTranscriptBase
|
||||||
ArtifactTranscriptPolished = "narratio.transcript.polished"
|
ArtifactTranscriptPolished = artifactmodel.SourceTranscriptPolished
|
||||||
ArtifactTranscriptFull = "narratio.transcript.full"
|
ArtifactTranscriptFinal = artifactmodel.SourceTranscriptFinal
|
||||||
ArtifactTranscriptTrimmed = "narratio.transcript.trimmed"
|
ArtifactTranscriptFinalTrimmed = artifactmodel.SourceTranscriptFinalTrimmed
|
||||||
ArtifactBoundsSession = "narratio.bounds.session"
|
ArtifactBoundsSession = "narratio.bounds.session"
|
||||||
|
|
||||||
|
ArtifactProvenancePreviousCacheManifestInput = "manifest.inputs.previous_cache"
|
||||||
|
ArtifactProvenancePreviousCacheFilesystem = "current_session.previous_cache"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
TranscriptPathBase = artifactmodel.TranscriptPathBase
|
||||||
|
TranscriptPathPolished = artifactmodel.TranscriptPathPolished
|
||||||
|
TranscriptPathFinal = artifactmodel.TranscriptPathFinal
|
||||||
|
TranscriptPathFinalTrimmed = artifactmodel.TranscriptPathFinalTrimmed
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
TranscriptOutputKindBase = artifactmodel.TranscriptOutputKindBase
|
||||||
|
TranscriptOutputKindPolished = artifactmodel.TranscriptOutputKindPolished
|
||||||
|
TranscriptOutputKindFinal = artifactmodel.TranscriptOutputKindFinal
|
||||||
|
TranscriptOutputKindFinalTrimmed = artifactmodel.TranscriptOutputKindFinalTrimmed
|
||||||
)
|
)
|
||||||
|
|
||||||
// ErrSessionArtifactNotFound is returned when no readable artifact exists for a known ID.
|
// ErrSessionArtifactNotFound is returned when no readable artifact exists for a known ID.
|
||||||
var ErrSessionArtifactNotFound = errors.New("session artifact not found")
|
var ErrSessionArtifactNotFound = errors.New("session artifact not found")
|
||||||
var configuredArtifactSourceRE = regexp.MustCompile(`^narratio\.artifact\.[a-z][a-z0-9_]*$`)
|
var configuredArtifactSourceRE = regexp.MustCompile(`^narratio\.artifact\.([a-z][a-z0-9_]*)$`)
|
||||||
|
var previousSessionArtifactSourceRE = regexp.MustCompile(`^narratio\.previous_session\.artifact\.([a-z][a-z0-9_]*)$`)
|
||||||
|
|
||||||
type artifactContentKind string
|
type artifactContentKind string
|
||||||
|
|
||||||
@@ -40,42 +59,27 @@ type artifactSpec struct {
|
|||||||
ContentKind artifactContentKind
|
ContentKind artifactContentKind
|
||||||
}
|
}
|
||||||
|
|
||||||
var artifactRegistry = map[string]artifactSpec{
|
var artifactRegistry = buildArtifactRegistry()
|
||||||
ArtifactTranscriptMerged: {
|
|
||||||
ID: ArtifactTranscriptMerged,
|
func buildArtifactRegistry() map[string]artifactSpec {
|
||||||
CanonicalRelPath: "transcripts/merged.json",
|
registry := map[string]artifactSpec{}
|
||||||
ProducerStage: "merge",
|
for _, transcript := range RuntimeTranscriptArtifacts() {
|
||||||
OutputKind: "transcript_merged",
|
registry[transcript.SourceID] = artifactSpec{
|
||||||
|
ID: transcript.SourceID,
|
||||||
|
CanonicalRelPath: transcript.CanonicalRelPath,
|
||||||
|
ProducerStage: transcript.ProducerStage,
|
||||||
|
OutputKind: transcript.OutputKind,
|
||||||
ContentKind: contentTranscriptJSON,
|
ContentKind: contentTranscriptJSON,
|
||||||
},
|
}
|
||||||
ArtifactTranscriptPolished: {
|
}
|
||||||
ID: ArtifactTranscriptPolished,
|
registry[ArtifactBoundsSession] = artifactSpec{
|
||||||
CanonicalRelPath: "transcripts/processed.json",
|
|
||||||
ProducerStage: "polish",
|
|
||||||
OutputKind: "transcript_processed",
|
|
||||||
ContentKind: contentTranscriptJSON,
|
|
||||||
},
|
|
||||||
ArtifactTranscriptFull: {
|
|
||||||
ID: ArtifactTranscriptFull,
|
|
||||||
CanonicalRelPath: "transcripts/normalized.json",
|
|
||||||
ProducerStage: "normalize",
|
|
||||||
OutputKind: "transcript_normalized",
|
|
||||||
ContentKind: contentTranscriptJSON,
|
|
||||||
},
|
|
||||||
ArtifactTranscriptTrimmed: {
|
|
||||||
ID: ArtifactTranscriptTrimmed,
|
|
||||||
CanonicalRelPath: "transcripts/trimmed.json",
|
|
||||||
ProducerStage: "trim",
|
|
||||||
OutputKind: "transcript_trimmed",
|
|
||||||
ContentKind: contentTranscriptJSON,
|
|
||||||
},
|
|
||||||
ArtifactBoundsSession: {
|
|
||||||
ID: ArtifactBoundsSession,
|
ID: ArtifactBoundsSession,
|
||||||
CanonicalRelPath: "artifacts/session_bounds.json",
|
CanonicalRelPath: "artifacts/session_bounds.json",
|
||||||
ProducerStage: "trim",
|
ProducerStage: "trim",
|
||||||
OutputKind: "session_bounds",
|
OutputKind: "session_bounds",
|
||||||
ContentKind: contentJSON,
|
ContentKind: contentJSON,
|
||||||
},
|
}
|
||||||
|
return registry
|
||||||
}
|
}
|
||||||
|
|
||||||
// ResolvedSessionArtifact describes one session-level artifact lookup result.
|
// ResolvedSessionArtifact describes one session-level artifact lookup result.
|
||||||
@@ -118,6 +122,30 @@ func IsConfiguredArtifactSource(source string) bool {
|
|||||||
return configuredArtifactSourceRE.MatchString(strings.TrimSpace(source))
|
return configuredArtifactSourceRE.MatchString(strings.TrimSpace(source))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ConfiguredArtifactName extracts <name> from narratio.artifact.<name>.
|
||||||
|
func ConfiguredArtifactName(source string) (string, bool) {
|
||||||
|
matches := configuredArtifactSourceRE.FindStringSubmatch(strings.TrimSpace(source))
|
||||||
|
if len(matches) != 2 {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return matches[1], true
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsPreviousSessionArtifactSource returns true when source is narratio.previous_session.artifact.<name>.
|
||||||
|
func IsPreviousSessionArtifactSource(source string) bool {
|
||||||
|
_, ok := PreviousSessionArtifactName(source)
|
||||||
|
return ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// PreviousSessionArtifactName extracts <name> from narratio.previous_session.artifact.<name>.
|
||||||
|
func PreviousSessionArtifactName(source string) (string, bool) {
|
||||||
|
matches := previousSessionArtifactSourceRE.FindStringSubmatch(strings.TrimSpace(source))
|
||||||
|
if len(matches) != 2 {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return matches[1], true
|
||||||
|
}
|
||||||
|
|
||||||
// ResolveSessionArtifact resolves a symbolic source to a readable local session artifact path.
|
// ResolveSessionArtifact resolves a symbolic source to a readable local session artifact path.
|
||||||
// Resolution order is manifest producer outputs first, then canonical session path fallback.
|
// Resolution order is manifest producer outputs first, then canonical session path fallback.
|
||||||
func ResolveSessionArtifact(paths SessionPaths, m *manifest.Manifest, source string) (ResolvedSessionArtifact, error) {
|
func ResolveSessionArtifact(paths SessionPaths, m *manifest.Manifest, source string) (ResolvedSessionArtifact, error) {
|
||||||
@@ -170,6 +198,9 @@ func ResolveSessionArtifact(paths SessionPaths, m *manifest.Manifest, source str
|
|||||||
// configured narratio.artifact.<name> sources through runtime catalog availability.
|
// configured narratio.artifact.<name> sources through runtime catalog availability.
|
||||||
func ResolveSessionArtifactWithCatalog(paths SessionPaths, m *manifest.Manifest, source string, catalog *ArtifactCatalog) (ResolvedSessionArtifact, error) {
|
func ResolveSessionArtifactWithCatalog(paths SessionPaths, m *manifest.Manifest, source string, catalog *ArtifactCatalog) (ResolvedSessionArtifact, error) {
|
||||||
normalized := strings.TrimSpace(source)
|
normalized := strings.TrimSpace(source)
|
||||||
|
if IsPreviousSessionArtifactSource(normalized) {
|
||||||
|
return ResolvePreviousSessionArtifactWithCatalog(paths, m, normalized, catalog)
|
||||||
|
}
|
||||||
if !IsConfiguredArtifactSource(normalized) {
|
if !IsConfiguredArtifactSource(normalized) {
|
||||||
return ResolveSessionArtifact(paths, m, normalized)
|
return ResolveSessionArtifact(paths, m, normalized)
|
||||||
}
|
}
|
||||||
@@ -195,6 +226,72 @@ func ResolveSessionArtifactWithCatalog(paths SessionPaths, m *manifest.Manifest,
|
|||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ResolvePreviousSessionArtifactWithCatalog resolves one canonical previous-session source id
|
||||||
|
// to the prepared current-session previous-cache path.
|
||||||
|
func ResolvePreviousSessionArtifactWithCatalog(
|
||||||
|
paths SessionPaths,
|
||||||
|
m *manifest.Manifest,
|
||||||
|
source string,
|
||||||
|
catalog *ArtifactCatalog,
|
||||||
|
) (ResolvedSessionArtifact, error) {
|
||||||
|
artifactName, ok := PreviousSessionArtifactName(source)
|
||||||
|
if !ok {
|
||||||
|
return ResolvedSessionArtifact{}, fmt.Errorf("unsupported previous-session artifact source %q", source)
|
||||||
|
}
|
||||||
|
if catalog == nil {
|
||||||
|
return ResolvedSessionArtifact{}, fmt.Errorf("previous-session artifact source %q requires runtime artifact catalog", source)
|
||||||
|
}
|
||||||
|
|
||||||
|
configuredSourceID := ConfiguredArtifactSourceID(artifactName)
|
||||||
|
entry, ok := catalog.Lookup(configuredSourceID)
|
||||||
|
if !ok {
|
||||||
|
return ResolvedSessionArtifact{}, fmt.Errorf("unsupported previous-session artifact source %q", source)
|
||||||
|
}
|
||||||
|
candidates := previousSessionCacheCandidatePaths(paths, entry.CanonicalRelPath)
|
||||||
|
if len(candidates) == 0 {
|
||||||
|
return ResolvedSessionArtifact{}, &SessionArtifactNotFoundError{ArtifactID: source}
|
||||||
|
}
|
||||||
|
|
||||||
|
manifestInputPaths := manifestInputPathSet(paths, m)
|
||||||
|
fallback := ""
|
||||||
|
for _, candidate := range candidates {
|
||||||
|
exists, isDir, statErr := pathExists(candidate)
|
||||||
|
if statErr != nil {
|
||||||
|
return ResolvedSessionArtifact{}, fmt.Errorf("stat %q: %w", candidate, statErr)
|
||||||
|
}
|
||||||
|
if !exists || isDir {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := validateResolvedContent(candidate, contentText); err != nil {
|
||||||
|
return ResolvedSessionArtifact{}, fmt.Errorf("validate %q: %w", source, err)
|
||||||
|
}
|
||||||
|
if _, ok := manifestInputPaths[candidate]; ok {
|
||||||
|
return ResolvedSessionArtifact{
|
||||||
|
ID: source,
|
||||||
|
Path: candidate,
|
||||||
|
ProducerStage: "prepare",
|
||||||
|
OutputKind: "previous_session_cache",
|
||||||
|
Provenance: ArtifactProvenancePreviousCacheManifestInput,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
if fallback == "" {
|
||||||
|
fallback = candidate
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if fallback != "" {
|
||||||
|
return ResolvedSessionArtifact{
|
||||||
|
ID: source,
|
||||||
|
Path: fallback,
|
||||||
|
ProducerStage: "prepare",
|
||||||
|
OutputKind: "previous_session_cache",
|
||||||
|
Provenance: ArtifactProvenancePreviousCacheFilesystem,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return ResolvedSessionArtifact{}, &SessionArtifactNotFoundError{ArtifactID: source}
|
||||||
|
}
|
||||||
|
|
||||||
func manifestArtifactCandidates(paths SessionPaths, m *manifest.Manifest, spec artifactSpec) []ResolvedSessionArtifact {
|
func manifestArtifactCandidates(paths SessionPaths, m *manifest.Manifest, spec artifactSpec) []ResolvedSessionArtifact {
|
||||||
if m == nil || len(m.Stages) == 0 || spec.ProducerStage == "" || spec.OutputKind == "" {
|
if m == nil || len(m.Stages) == 0 || spec.ProducerStage == "" || spec.OutputKind == "" {
|
||||||
return nil
|
return nil
|
||||||
@@ -243,6 +340,50 @@ func dedupeResolvedArtifacts(values []ResolvedSessionArtifact) []ResolvedSession
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func previousSessionCacheCandidatePaths(paths SessionPaths, canonicalRelPath string) []string {
|
||||||
|
trimmed := strings.TrimSpace(canonicalRelPath)
|
||||||
|
if trimmed == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
normalized := filepath.ToSlash(filepath.Clean(filepath.FromSlash(trimmed)))
|
||||||
|
if normalized == "." || normalized == "" || normalized == ".." || strings.HasPrefix(normalized, "../") || strings.HasPrefix(normalized, "/") {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
relCandidates := []string{normalized}
|
||||||
|
const artifactsPrefix = "artifacts/"
|
||||||
|
if strings.HasPrefix(normalized, artifactsPrefix) && len(normalized) > len(artifactsPrefix) {
|
||||||
|
relCandidates = append(relCandidates, strings.TrimPrefix(normalized, artifactsPrefix))
|
||||||
|
}
|
||||||
|
|
||||||
|
out := make([]string, 0, len(relCandidates))
|
||||||
|
seen := map[string]struct{}{}
|
||||||
|
for _, rel := range relCandidates {
|
||||||
|
abs := filepath.Clean(SessionPreviousArtifactPath(paths, rel))
|
||||||
|
if _, ok := seen[abs]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seen[abs] = struct{}{}
|
||||||
|
out = append(out, abs)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func manifestInputPathSet(paths SessionPaths, m *manifest.Manifest) map[string]struct{} {
|
||||||
|
if m == nil || len(m.Inputs) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
out := make(map[string]struct{}, len(m.Inputs))
|
||||||
|
for _, in := range m.Inputs {
|
||||||
|
resolved := filepath.Clean(ResolveSessionLocalPathForRead(paths, in.Path))
|
||||||
|
if strings.TrimSpace(resolved) == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out[resolved] = struct{}{}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
func pathExists(path string) (exists bool, isDir bool, err error) {
|
func pathExists(path string) (exists bool, isDir bool, err error) {
|
||||||
info, err := os.Stat(path)
|
info, err := os.Stat(path)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"reflect"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -22,7 +23,7 @@ func TestNormalizeSessionArtifactSource(t *testing.T) {
|
|||||||
{name: "legacy alias normalized unsupported", source: "normalized_transcript", wantErr: "unsupported artifact source"},
|
{name: "legacy alias normalized unsupported", source: "normalized_transcript", wantErr: "unsupported artifact source"},
|
||||||
{name: "legacy alias trimmed unsupported", source: "trimmed_transcript", wantErr: "unsupported artifact source"},
|
{name: "legacy alias trimmed unsupported", source: "trimmed_transcript", wantErr: "unsupported artifact source"},
|
||||||
{name: "configured source unsupported in built-in normalization", source: "narratio.artifact.session_recap", wantErr: "unsupported artifact source"},
|
{name: "configured source unsupported in built-in normalization", source: "narratio.artifact.session_recap", wantErr: "unsupported artifact source"},
|
||||||
{name: "canonical", source: ArtifactTranscriptTrimmed, wantID: ArtifactTranscriptTrimmed},
|
{name: "canonical", source: ArtifactTranscriptFinalTrimmed, wantID: ArtifactTranscriptFinalTrimmed},
|
||||||
{name: "unsupported", source: "narratio.unknown", wantErr: "unsupported artifact source"},
|
{name: "unsupported", source: "narratio.unknown", wantErr: "unsupported artifact source"},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -45,6 +46,110 @@ func TestNormalizeSessionArtifactSource(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestConfiguredArtifactSourceHelpers(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
source string
|
||||||
|
wantName string
|
||||||
|
wantMatch bool
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "valid",
|
||||||
|
source: "narratio.artifact.session_recap",
|
||||||
|
wantName: "session_recap",
|
||||||
|
wantMatch: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "valid with surrounding whitespace",
|
||||||
|
source: " narratio.artifact.player_handout ",
|
||||||
|
wantName: "player_handout",
|
||||||
|
wantMatch: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "missing name",
|
||||||
|
source: "narratio.artifact.",
|
||||||
|
wantMatch: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "invalid hyphen",
|
||||||
|
source: "narratio.artifact.session-recap",
|
||||||
|
wantMatch: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "built-in",
|
||||||
|
source: ArtifactTranscriptBase,
|
||||||
|
wantMatch: false,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
if got := IsConfiguredArtifactSource(tt.source); got != tt.wantMatch {
|
||||||
|
t.Fatalf("IsConfiguredArtifactSource(%q) = %t, want %t", tt.source, got, tt.wantMatch)
|
||||||
|
}
|
||||||
|
gotName, gotOK := ConfiguredArtifactName(tt.source)
|
||||||
|
if gotOK != tt.wantMatch {
|
||||||
|
t.Fatalf("ConfiguredArtifactName(%q) ok = %t, want %t", tt.source, gotOK, tt.wantMatch)
|
||||||
|
}
|
||||||
|
if gotName != tt.wantName {
|
||||||
|
t.Fatalf("ConfiguredArtifactName(%q) name = %q, want %q", tt.source, gotName, tt.wantName)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPreviousSessionArtifactSourceHelpers(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
source string
|
||||||
|
wantName string
|
||||||
|
wantMatch bool
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "valid",
|
||||||
|
source: "narratio.previous_session.artifact.session_recap",
|
||||||
|
wantName: "session_recap",
|
||||||
|
wantMatch: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "valid with surrounding whitespace",
|
||||||
|
source: " narratio.previous_session.artifact.quest_log ",
|
||||||
|
wantName: "quest_log",
|
||||||
|
wantMatch: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "missing name",
|
||||||
|
source: "narratio.previous_session.artifact.",
|
||||||
|
wantMatch: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "invalid key characters",
|
||||||
|
source: "narratio.previous_session.artifact.session-recap",
|
||||||
|
wantMatch: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "wrong prefix",
|
||||||
|
source: "narratio.previous.artifact.session_recap",
|
||||||
|
wantMatch: false,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
name, ok := PreviousSessionArtifactName(tt.source)
|
||||||
|
if ok != tt.wantMatch {
|
||||||
|
t.Fatalf("PreviousSessionArtifactName(%q) ok = %t, want %t", tt.source, ok, tt.wantMatch)
|
||||||
|
}
|
||||||
|
if name != tt.wantName {
|
||||||
|
t.Fatalf("PreviousSessionArtifactName(%q) name = %q, want %q", tt.source, name, tt.wantName)
|
||||||
|
}
|
||||||
|
if got := IsPreviousSessionArtifactSource(tt.source); got != tt.wantMatch {
|
||||||
|
t.Fatalf("IsPreviousSessionArtifactSource(%q) = %t, want %t", tt.source, got, tt.wantMatch)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestResolveSessionArtifactPrefersManifestOutput(t *testing.T) {
|
func TestResolveSessionArtifactPrefersManifestOutput(t *testing.T) {
|
||||||
workspace := t.TempDir()
|
workspace := t.TempDir()
|
||||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
@@ -55,7 +160,7 @@ func TestResolveSessionArtifactPrefersManifestOutput(t *testing.T) {
|
|||||||
if err := os.WriteFile(manifestPath, []byte(`{"segments":[]}`), 0o644); err != nil {
|
if err := os.WriteFile(manifestPath, []byte(`{"segments":[]}`), 0o644); err != nil {
|
||||||
t.Fatalf("WriteFile() error = %v", err)
|
t.Fatalf("WriteFile() error = %v", err)
|
||||||
}
|
}
|
||||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "normalized.json")
|
canonicalPath := filepath.Join(paths.TranscriptsDir, "final.json")
|
||||||
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
||||||
t.Fatalf("MkdirAll() error = %v", err)
|
t.Fatalf("MkdirAll() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -65,10 +170,10 @@ func TestResolveSessionArtifactPrefersManifestOutput(t *testing.T) {
|
|||||||
|
|
||||||
m := manifest.New("session", time.Now().UTC())
|
m := manifest.New("session", time.Now().UTC())
|
||||||
m.MarkStageSucceeded("normalize", time.Now().UTC(), []manifest.ArtifactRecord{
|
m.MarkStageSucceeded("normalize", time.Now().UTC(), []manifest.ArtifactRecord{
|
||||||
{Kind: "transcript_normalized", LocalPath: manifestPath, ProducerRunID: "run-123"},
|
{Kind: "transcript_final", LocalPath: manifestPath, ProducerRunID: "run-123"},
|
||||||
})
|
})
|
||||||
|
|
||||||
resolved, err := ResolveSessionArtifact(paths, m, ArtifactTranscriptFull)
|
resolved, err := ResolveSessionArtifact(paths, m, ArtifactTranscriptFinal)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ResolveSessionArtifact() error = %v", err)
|
t.Fatalf("ResolveSessionArtifact() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -86,7 +191,7 @@ func TestResolveSessionArtifactPrefersManifestOutput(t *testing.T) {
|
|||||||
func TestResolveSessionArtifactFallsBackToCanonicalPath(t *testing.T) {
|
func TestResolveSessionArtifactFallsBackToCanonicalPath(t *testing.T) {
|
||||||
workspace := t.TempDir()
|
workspace := t.TempDir()
|
||||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "trimmed.json")
|
canonicalPath := filepath.Join(paths.TranscriptsDir, "final.trimmed.json")
|
||||||
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
||||||
t.Fatalf("MkdirAll() error = %v", err)
|
t.Fatalf("MkdirAll() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -94,7 +199,7 @@ func TestResolveSessionArtifactFallsBackToCanonicalPath(t *testing.T) {
|
|||||||
t.Fatalf("WriteFile() error = %v", err)
|
t.Fatalf("WriteFile() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
resolved, err := ResolveSessionArtifact(paths, nil, ArtifactTranscriptTrimmed)
|
resolved, err := ResolveSessionArtifact(paths, nil, ArtifactTranscriptFinalTrimmed)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ResolveSessionArtifact() error = %v", err)
|
t.Fatalf("ResolveSessionArtifact() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -110,7 +215,7 @@ func TestResolveSessionArtifactMissingReturnsTypedError(t *testing.T) {
|
|||||||
workspace := t.TempDir()
|
workspace := t.TempDir()
|
||||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
|
|
||||||
_, err := ResolveSessionArtifact(paths, nil, ArtifactTranscriptTrimmed)
|
_, err := ResolveSessionArtifact(paths, nil, ArtifactTranscriptFinalTrimmed)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected error, got nil")
|
t.Fatal("expected error, got nil")
|
||||||
}
|
}
|
||||||
@@ -122,7 +227,7 @@ func TestResolveSessionArtifactMissingReturnsTypedError(t *testing.T) {
|
|||||||
func TestResolveSessionArtifactValidatesTranscriptShape(t *testing.T) {
|
func TestResolveSessionArtifactValidatesTranscriptShape(t *testing.T) {
|
||||||
workspace := t.TempDir()
|
workspace := t.TempDir()
|
||||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "processed.json")
|
canonicalPath := filepath.Join(paths.TranscriptsDir, "polished.json")
|
||||||
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
||||||
t.Fatalf("MkdirAll() error = %v", err)
|
t.Fatalf("MkdirAll() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -142,7 +247,7 @@ func TestResolveSessionArtifactValidatesTranscriptShape(t *testing.T) {
|
|||||||
func TestResolveSessionArtifactWithCatalogBuiltInBehaviorUnchanged(t *testing.T) {
|
func TestResolveSessionArtifactWithCatalogBuiltInBehaviorUnchanged(t *testing.T) {
|
||||||
workspace := t.TempDir()
|
workspace := t.TempDir()
|
||||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
canonicalPath := filepath.Join(paths.TranscriptsDir, "trimmed.json")
|
canonicalPath := filepath.Join(paths.TranscriptsDir, "final.trimmed.json")
|
||||||
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
||||||
t.Fatalf("MkdirAll() error = %v", err)
|
t.Fatalf("MkdirAll() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -150,7 +255,7 @@ func TestResolveSessionArtifactWithCatalogBuiltInBehaviorUnchanged(t *testing.T)
|
|||||||
t.Fatalf("WriteFile() error = %v", err)
|
t.Fatalf("WriteFile() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
resolved, err := ResolveSessionArtifactWithCatalog(paths, nil, ArtifactTranscriptTrimmed, NewArtifactCatalog())
|
resolved, err := ResolveSessionArtifactWithCatalog(paths, nil, ArtifactTranscriptFinalTrimmed, NewArtifactCatalog())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ResolveSessionArtifactWithCatalog() error = %v", err)
|
t.Fatalf("ResolveSessionArtifactWithCatalog() error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -266,3 +371,131 @@ func TestResolveSessionArtifactWithCatalogUnsupportedConfiguredSourceFails(t *te
|
|||||||
t.Fatalf("error = %q, want unsupported artifact source", err.Error())
|
t.Fatalf("error = %q, want unsupported artifact source", err.Error())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestResolvePreviousSessionArtifactWithCatalogPrefersManifestInputRecord(t *testing.T) {
|
||||||
|
workspace := t.TempDir()
|
||||||
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
|
|
||||||
|
manifestBackedPath := SessionPreviousArtifactPath(paths, "artifacts/session_recap.md")
|
||||||
|
fallbackPath := SessionPreviousArtifactPath(paths, "session_recap.md")
|
||||||
|
if err := os.MkdirAll(filepath.Dir(manifestBackedPath), 0o755); err != nil {
|
||||||
|
t.Fatalf("MkdirAll() error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(filepath.Dir(fallbackPath), 0o755); err != nil {
|
||||||
|
t.Fatalf("MkdirAll() error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(manifestBackedPath, []byte("recap from manifest input\n"), 0o644); err != nil {
|
||||||
|
t.Fatalf("WriteFile() error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(fallbackPath, []byte("recap fallback\n"), 0o644); err != nil {
|
||||||
|
t.Fatalf("WriteFile() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
catalog := NewArtifactCatalog()
|
||||||
|
if err := catalog.RegisterConfiguredArtifacts(
|
||||||
|
map[string]ConfiguredArtifactDefinition{
|
||||||
|
"session_recap": {Enabled: true, OutputPath: "artifacts/session_recap.md"},
|
||||||
|
},
|
||||||
|
nil,
|
||||||
|
); err != nil {
|
||||||
|
t.Fatalf("RegisterConfiguredArtifacts() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
m := manifest.New("session", time.Now().UTC())
|
||||||
|
m.Inputs = []manifest.InputRecord{
|
||||||
|
{Kind: "previous_artifact", Path: manifestBackedPath},
|
||||||
|
}
|
||||||
|
|
||||||
|
resolved, err := ResolvePreviousSessionArtifactWithCatalog(
|
||||||
|
paths,
|
||||||
|
m,
|
||||||
|
"narratio.previous_session.artifact.session_recap",
|
||||||
|
catalog,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ResolvePreviousSessionArtifactWithCatalog() error = %v", err)
|
||||||
|
}
|
||||||
|
if resolved.Path != manifestBackedPath {
|
||||||
|
t.Fatalf("resolved path = %q, want %q", resolved.Path, manifestBackedPath)
|
||||||
|
}
|
||||||
|
if resolved.Provenance != ArtifactProvenancePreviousCacheManifestInput {
|
||||||
|
t.Fatalf("provenance = %q, want %q", resolved.Provenance, ArtifactProvenancePreviousCacheManifestInput)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolvePreviousSessionArtifactWithCatalogFallsBackToPreparedCachePath(t *testing.T) {
|
||||||
|
workspace := t.TempDir()
|
||||||
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
|
|
||||||
|
fallbackPath := SessionPreviousArtifactPath(paths, "session_recap.md")
|
||||||
|
if err := os.MkdirAll(filepath.Dir(fallbackPath), 0o755); err != nil {
|
||||||
|
t.Fatalf("MkdirAll() error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(fallbackPath, []byte("recap fallback\n"), 0o644); err != nil {
|
||||||
|
t.Fatalf("WriteFile() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
catalog := NewArtifactCatalog()
|
||||||
|
if err := catalog.RegisterConfiguredArtifacts(
|
||||||
|
map[string]ConfiguredArtifactDefinition{
|
||||||
|
"session_recap": {Enabled: true, OutputPath: "artifacts/session_recap.md"},
|
||||||
|
},
|
||||||
|
nil,
|
||||||
|
); err != nil {
|
||||||
|
t.Fatalf("RegisterConfiguredArtifacts() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
resolved, err := ResolvePreviousSessionArtifactWithCatalog(
|
||||||
|
paths,
|
||||||
|
nil,
|
||||||
|
"narratio.previous_session.artifact.session_recap",
|
||||||
|
catalog,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ResolvePreviousSessionArtifactWithCatalog() error = %v", err)
|
||||||
|
}
|
||||||
|
if resolved.Path != fallbackPath {
|
||||||
|
t.Fatalf("resolved path = %q, want %q", resolved.Path, fallbackPath)
|
||||||
|
}
|
||||||
|
if resolved.Provenance != ArtifactProvenancePreviousCacheFilesystem {
|
||||||
|
t.Fatalf("provenance = %q, want %q", resolved.Provenance, ArtifactProvenancePreviousCacheFilesystem)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPreviousSessionCacheCandidatePathsStripsArtifactsPrefix(t *testing.T) {
|
||||||
|
workspace := t.TempDir()
|
||||||
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
|
|
||||||
|
got := previousSessionCacheCandidatePaths(paths, "artifacts/session_recap.md")
|
||||||
|
want := []string{filepath.Join(paths.PreviousArtifactsDir, "session_recap.md")}
|
||||||
|
if !reflect.DeepEqual(got, want) {
|
||||||
|
t.Fatalf("previousSessionCacheCandidatePaths() = %#v, want %#v", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolvePreviousSessionArtifactWithCatalogMissingReturnsTypedError(t *testing.T) {
|
||||||
|
workspace := t.TempDir()
|
||||||
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
|
catalog := NewArtifactCatalog()
|
||||||
|
if err := catalog.RegisterConfiguredArtifacts(
|
||||||
|
map[string]ConfiguredArtifactDefinition{
|
||||||
|
"session_recap": {Enabled: true, OutputPath: "artifacts/session_recap.md"},
|
||||||
|
},
|
||||||
|
nil,
|
||||||
|
); err != nil {
|
||||||
|
t.Fatalf("RegisterConfiguredArtifacts() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err := ResolvePreviousSessionArtifactWithCatalog(
|
||||||
|
paths,
|
||||||
|
nil,
|
||||||
|
"narratio.previous_session.artifact.session_recap",
|
||||||
|
catalog,
|
||||||
|
)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error, got nil")
|
||||||
|
}
|
||||||
|
if !errors.Is(err, ErrSessionArtifactNotFound) {
|
||||||
|
t.Fatalf("errors.Is(err, ErrSessionArtifactNotFound)=false; err=%v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -215,10 +215,10 @@ func (c *ArtifactCatalog) addEntry(entry CatalogEntry) error {
|
|||||||
|
|
||||||
func runtimeBuiltInArtifactIDs() []string {
|
func runtimeBuiltInArtifactIDs() []string {
|
||||||
return []string{
|
return []string{
|
||||||
ArtifactTranscriptMerged,
|
ArtifactTranscriptBase,
|
||||||
ArtifactTranscriptPolished,
|
ArtifactTranscriptPolished,
|
||||||
ArtifactTranscriptFull,
|
ArtifactTranscriptFinal,
|
||||||
ArtifactTranscriptTrimmed,
|
ArtifactTranscriptFinalTrimmed,
|
||||||
ArtifactBoundsSession,
|
ArtifactBoundsSession,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -8,9 +8,9 @@ func TestArtifactCatalogRegisterBuiltInsAndLookup(t *testing.T) {
|
|||||||
t.Fatalf("RegisterBuiltIns() error = %v", err)
|
t.Fatalf("RegisterBuiltIns() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
entry, ok := catalog.Lookup(ArtifactTranscriptFull)
|
entry, ok := catalog.Lookup(ArtifactTranscriptFinal)
|
||||||
if !ok {
|
if !ok {
|
||||||
t.Fatalf("Lookup(%q) ok = false, want true", ArtifactTranscriptFull)
|
t.Fatalf("Lookup(%q) ok = false, want true", ArtifactTranscriptFinal)
|
||||||
}
|
}
|
||||||
if !entry.Planned {
|
if !entry.Planned {
|
||||||
t.Fatalf("entry.Planned = false, want true")
|
t.Fatalf("entry.Planned = false, want true")
|
||||||
@@ -18,8 +18,8 @@ func TestArtifactCatalogRegisterBuiltInsAndLookup(t *testing.T) {
|
|||||||
if entry.Executable {
|
if entry.Executable {
|
||||||
t.Fatalf("entry.Executable = true, want false")
|
t.Fatalf("entry.Executable = true, want false")
|
||||||
}
|
}
|
||||||
if entry.CanonicalRelPath != "transcripts/normalized.json" {
|
if entry.CanonicalRelPath != "transcripts/final.json" {
|
||||||
t.Fatalf("entry.CanonicalRelPath = %q, want transcripts/normalized.json", entry.CanonicalRelPath)
|
t.Fatalf("entry.CanonicalRelPath = %q, want transcripts/final.json", entry.CanonicalRelPath)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -73,6 +73,8 @@ func (s *LocalStore) ensureLayout(paths SessionPaths) (SessionPaths, error) {
|
|||||||
paths.LogsDir,
|
paths.LogsDir,
|
||||||
paths.CurrentDir,
|
paths.CurrentDir,
|
||||||
paths.RunsDir,
|
paths.RunsDir,
|
||||||
|
paths.PreviousDir,
|
||||||
|
paths.PreviousArtifactsDir,
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, dir := range dirs {
|
for _, dir := range dirs {
|
||||||
|
|||||||
@@ -27,10 +27,15 @@ func TestEnsureLayoutCreatesExpectedDirectories(t *testing.T) {
|
|||||||
checkDirExists(t, paths.LogsDir)
|
checkDirExists(t, paths.LogsDir)
|
||||||
checkDirExists(t, paths.CurrentDir)
|
checkDirExists(t, paths.CurrentDir)
|
||||||
checkDirExists(t, paths.RunsDir)
|
checkDirExists(t, paths.RunsDir)
|
||||||
|
checkDirExists(t, paths.PreviousDir)
|
||||||
|
checkDirExists(t, paths.PreviousArtifactsDir)
|
||||||
|
|
||||||
if filepath.Base(paths.ManifestPath) != "manifest.json" {
|
if filepath.Base(paths.ManifestPath) != "manifest.json" {
|
||||||
t.Fatalf("ManifestPath = %q, want basename manifest.json", paths.ManifestPath)
|
t.Fatalf("ManifestPath = %q, want basename manifest.json", paths.ManifestPath)
|
||||||
}
|
}
|
||||||
|
if filepath.Base(paths.PreviousManifestPath) != "manifest.json" {
|
||||||
|
t.Fatalf("PreviousManifestPath = %q, want basename manifest.json", paths.PreviousManifestPath)
|
||||||
|
}
|
||||||
if filepath.Base(paths.LockPath) != ".lock" {
|
if filepath.Base(paths.LockPath) != ".lock" {
|
||||||
t.Fatalf("LockPath = %q, want basename .lock", paths.LockPath)
|
t.Fatalf("LockPath = %q, want basename .lock", paths.LockPath)
|
||||||
}
|
}
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user