Compare commits
5 Commits
23ed716450
...
ef1f650bc0
| Author | SHA1 | Date | |
|---|---|---|---|
| ef1f650bc0 | |||
| 0d02cb9fa0 | |||
| 0299b128cf | |||
| d723384888 | |||
| 54228055c8 |
16
README.md
16
README.md
@@ -1,22 +1,22 @@
|
|||||||
# narratio
|
# narratio
|
||||||
|
|
||||||
Narratio is a Go orchestration application that turns D&D session audio into polished transcripts and generated session artifacts.
|
Narratio is a stage-driven Go orchestrator for turning D&D session audio into polished transcripts and generated artifacts.
|
||||||
|
|
||||||
It coordinates transcription, merge/polish/normalize/trim processing, artifact generation, publish-stage uploads, and resumable run state in one operator workflow.
|
It runs a deterministic workflow across `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, `analyze`, and `publish`, with manifest-driven resume and restore support.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run 2026-04-04
|
narratio run 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
This command requires discoverable `pipeline.yml` and `session.yml` files (or explicit `--config` and `--session` flags).
|
This requires resolvable `pipeline.yml`, `campaign.yml`, and concrete `session.yml` (or explicit config flags).
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
- [Configuration](docs/config.md)
|
|
||||||
- [CLI Reference](docs/cli.md)
|
- [CLI Reference](docs/cli.md)
|
||||||
- [Operations and Recovery](docs/operations.md)
|
- [Configuration](docs/config.md)
|
||||||
|
- [Operations](docs/operations.md)
|
||||||
- [Troubleshooting](docs/troubleshooting.md)
|
- [Troubleshooting](docs/troubleshooting.md)
|
||||||
- [Development Guide](docs/development.md)
|
|
||||||
- [Architecture Principles](docs/architecture.md)
|
|
||||||
- [Internal Component Contracts](docs/internal/README.md)
|
- [Internal Component Contracts](docs/internal/README.md)
|
||||||
- [Config Examples](examples/)
|
- [Development Guide](docs/policy/development.md)
|
||||||
|
- [Architecture Principles](docs/policy/architecture.md)
|
||||||
|
- [Maintained Examples](examples/)
|
||||||
|
|||||||
180
docs/cli.md
180
docs/cli.md
@@ -1,4 +1,4 @@
|
|||||||
# CLI
|
# CLI Reference
|
||||||
|
|
||||||
## Shortest Useful Command
|
## Shortest Useful Command
|
||||||
|
|
||||||
@@ -6,18 +6,19 @@
|
|||||||
narratio run 2026-04-04
|
narratio run 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
This runs the full pipeline for the given session ID using default config discovery and campaign selection.
|
This runs the canonical full pipeline for session `2026-04-04`.
|
||||||
|
|
||||||
## Command Overview
|
## Command Overview
|
||||||
|
|
||||||
Top-level commands:
|
Top-level commands:
|
||||||
|
|
||||||
- `run <session_id>`: execute the pipeline.
|
- `run <session_id>`: run full stage order.
|
||||||
- `resume <session_id>`: continue from first non-succeeded stage.
|
- `resume <session_id>`: continue from first non-succeeded stage.
|
||||||
- `run-stage <stage> <session_id>`: execute exactly one stage.
|
- `run-stage <stage> <session_id>`: run one stage.
|
||||||
- `analyze <session_id>`: force-rerun analyze stage.
|
- `analyze <session_id>`: force-run analyze.
|
||||||
- `publish <session_id>`: force-rerun publish stage.
|
- `publish <session_id>`: force-run publish.
|
||||||
- `clean <session_id>|--all`: remove local workspace/spool state.
|
- `clean <session_id>` or `clean --all`: remove local work/spool state.
|
||||||
- `session <subcommand>`: session-scoped helper commands.
|
- `session <subcommand>`: session helper commands.
|
||||||
|
|
||||||
Session subcommands:
|
Session subcommands:
|
||||||
|
|
||||||
@@ -31,7 +32,8 @@ Session subcommands:
|
|||||||
- `session locks add <session_id> <source>`
|
- `session locks add <session_id> <source>`
|
||||||
- `session locks remove <session_id> <source>`
|
- `session locks remove <session_id> <source>`
|
||||||
|
|
||||||
## Common Flags
|
## Common Config Flags
|
||||||
|
|
||||||
Most session-aware commands accept:
|
Most session-aware commands accept:
|
||||||
|
|
||||||
- `--config <pipeline.yml>`
|
- `--config <pipeline.yml>`
|
||||||
@@ -40,30 +42,41 @@ Most session-aware commands accept:
|
|||||||
- `--session <session.yml>`
|
- `--session <session.yml>`
|
||||||
- `--previous-session-id <id>`
|
- `--previous-session-id <id>`
|
||||||
|
|
||||||
`--campaign` and `--campaign-file` are mutually exclusive.
|
Rules:
|
||||||
|
|
||||||
|
- `--campaign` and `--campaign-file` are mutually exclusive.
|
||||||
|
- `--session` is not used by `session init`.
|
||||||
|
- `clean --all` cannot be combined with campaign/session selectors.
|
||||||
|
|
||||||
## Command Reference
|
## Command Reference
|
||||||
|
|
||||||
### `run`
|
### `run`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run <session_id> [--force] [--artifacts <name[,name...]>] [...common flags]
|
narratio run <session_id> [--force] [--artifacts <name[,name...]>] [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Runs stages in canonical order and writes manifest state.
|
Behavior:
|
||||||
|
|
||||||
|
- evaluates full stage order;
|
||||||
|
- skips already-succeeded stages unless `--force` is set;
|
||||||
|
- writes session and run manifests.
|
||||||
|
|
||||||
### `resume`
|
### `resume`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio resume <session_id> [--force] [--artifacts <name[,name...]>] [...common flags]
|
narratio resume <session_id> [--force] [--artifacts <name[,name...]>] [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Starts at the first non-succeeded stage from the session manifest.
|
Behavior:
|
||||||
|
|
||||||
|
- when not forced, starts at first non-succeeded stage in manifest order;
|
||||||
|
- with `--force`, reevaluates the selected stage list as runnable.
|
||||||
|
|
||||||
### `run-stage`
|
### `run-stage`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run-stage <stage> <session_id> [--force] [--artifacts <name[,name...]>] [...common flags]
|
narratio run-stage <stage> <session_id> [--force] [--artifacts <name[,name...]>] [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Valid stage names:
|
Valid stage names:
|
||||||
@@ -78,123 +91,161 @@ Valid stage names:
|
|||||||
- `publish`
|
- `publish`
|
||||||
- `notify`
|
- `notify`
|
||||||
|
|
||||||
`--artifacts` is accepted only for `analyze` and `publish`.
|
Rules:
|
||||||
|
|
||||||
|
- `--artifacts` is accepted only for `analyze` and `publish` stage targets.
|
||||||
|
|
||||||
### `analyze`
|
### `analyze`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio analyze <session_id> [--artifacts <name[,name...]>] [...common flags]
|
narratio analyze <session_id> [--artifacts <name[,name...]>] [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Equivalent to `narratio run-stage analyze <session_id> --force`.
|
Equivalent to:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio run-stage analyze <session_id> --force [...common config flags]
|
||||||
|
```
|
||||||
|
|
||||||
### `publish`
|
### `publish`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio publish <session_id> [--artifacts <name[,name...]>] [...common flags]
|
narratio publish <session_id> [--artifacts <name[,name...]>] [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Equivalent to `narratio run-stage publish <session_id> --force`.
|
Equivalent to:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio run-stage publish <session_id> --force [...common config flags]
|
||||||
|
```
|
||||||
|
|
||||||
### `clean`
|
### `clean`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio clean <session_id> [--dry-run] [--clear-cache] [...common flags]
|
narratio clean <session_id> [--dry-run] [--clear-cache] [...common config flags]
|
||||||
narratio clean --all [--dry-run] [--clear-cache] [--config <pipeline.yml>]
|
narratio clean --all [--dry-run] [--clear-cache] [--config <pipeline.yml>]
|
||||||
```
|
```
|
||||||
|
|
||||||
- session mode deletes `{workspace.root}/work/{campaign}/{session_id}` and `{spool.root}/{campaign}/{session_id}`.
|
Behavior:
|
||||||
- `--all` deletes all session work and spool children.
|
|
||||||
- cache is preserved unless `--clear-cache` is passed.
|
- session mode removes:
|
||||||
|
- `{workspace.root}/work/{campaign}/{session_id}`
|
||||||
|
- `{spool.root}/{campaign}/{session_id}`
|
||||||
|
- `--all` removes:
|
||||||
|
- `{workspace.root}/work/*`
|
||||||
|
- direct children under `{spool.root}`
|
||||||
|
- cache remains unless `--clear-cache` is provided.
|
||||||
|
|
||||||
### `session plan`
|
### `session plan`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session plan <session_id> [--force] [...common flags]
|
narratio session plan <session_id> [--force] [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Validates config and session inputs, prepares workdir layout, and prints stage run/skip decisions.
|
Validates config, prepares local workdir layout, and prints run/skip decisions for each stage.
|
||||||
|
|
||||||
### `session validate`
|
### `session validate`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session validate <session_id> [...common flags]
|
narratio session validate <session_id> [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Read-only preflight checks for config, inputs, audio availability, previous-session requirements, publish outputs, and effective locks.
|
Read-only preflight checks for config validity, required inputs, audio mode, previous-session requirements, publish outputs, and effective locks.
|
||||||
|
|
||||||
### `session status`
|
### `session status`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session status <session_id> [...common flags]
|
narratio session status <session_id> [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Shows local manifest state, remote current state (when storage is configured), published-output availability, and effective locks.
|
Prints local manifest state and, when storage is available, remote current-state and published-output status.
|
||||||
|
|
||||||
### `session init`
|
### `session init`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session init <session_id> --output ./session.yml
|
narratio session init <session_id> --output ./session.yml [options]
|
||||||
narratio session init <session_id> --remote
|
narratio session init <session_id> --remote [options]
|
||||||
narratio session init <session_id> --remote --force
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Flags:
|
Required target selection:
|
||||||
|
|
||||||
- `--output <path>` or `--remote` (exactly one is required)
|
- exactly one of:
|
||||||
|
- `--output <path>`
|
||||||
|
- `--remote`
|
||||||
|
|
||||||
|
Options:
|
||||||
|
|
||||||
|
- `--config <pipeline.yml>`
|
||||||
|
- `--campaign <id>` or `--campaign-file <campaign.yml>`
|
||||||
- `--previous-session-id <id>`
|
- `--previous-session-id <id>`
|
||||||
- `--date <date>`
|
- `--date <YYYY-MM-DD>`
|
||||||
- `--title <title>`
|
- `--title <text>`
|
||||||
- `--audio-dir <path>`
|
- `--audio-dir <path>`
|
||||||
- `--audio-s3-prefix <prefix>`
|
- `--audio-s3-prefix <prefix>`
|
||||||
- `--force`
|
- `--force`
|
||||||
- common config/campaign flags
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- `--audio-dir` and `--audio-s3-prefix` are mutually exclusive.
|
||||||
|
- if campaign `session_template_file` is configured, `session init` renders it;
|
||||||
|
- generated session YAML must be concrete (no unresolved `{{ ... }}` placeholders).
|
||||||
|
|
||||||
### `session restore`
|
### `session restore`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session restore <session_id> [--dry-run] [--force] [--include-audio] [...common flags]
|
narratio session restore <session_id> [--dry-run] [--force] [--include-audio] [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Restores durable local session files from committed remote current state.
|
Behavior:
|
||||||
|
|
||||||
|
- discovers committed remote current state;
|
||||||
|
- plans local restores;
|
||||||
|
- writes `reports/restore-latest.json` on execution;
|
||||||
|
- blocks conflicting overwrites unless `--force` is set.
|
||||||
|
|
||||||
Default restore scope:
|
Default restore scope:
|
||||||
|
|
||||||
- `manifest.json`
|
- `manifest.json`
|
||||||
- `transcripts/**`
|
- `transcripts/**`
|
||||||
- `artifacts/**`
|
- `artifacts/**`
|
||||||
- `previous/**` when required by configured previous-session artifact inputs
|
- `previous/**` when required by configured previous-session inputs
|
||||||
|
|
||||||
`audio/**` is restored only when `--include-audio` is set.
|
`audio/**` is included only with `--include-audio`.
|
||||||
|
|
||||||
### `session artifacts`
|
### `session artifacts`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session artifacts <session_id> [--remote] [...common flags]
|
narratio session artifacts <session_id> [--remote] [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
Lists built-in sources, configured artifact sources, previous-session sources, publish output rules, and lock status. With `--remote`, includes remote published-state markers.
|
Lists effective built-in and configured artifact sources, publish rules, lock state, and optional remote published-state availability.
|
||||||
|
|
||||||
### `session locks`
|
### `session locks`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session locks <session_id> [...common flags]
|
narratio session locks <session_id> [...common config flags]
|
||||||
narratio session locks add <session_id> <source> [--reason <text>] [--force] [...common flags]
|
narratio session locks add <session_id> <source> [--reason <text>] [--force] [...common config flags]
|
||||||
narratio session locks remove <session_id> <source> [...common flags]
|
narratio session locks remove <session_id> <source> [...common config flags]
|
||||||
```
|
```
|
||||||
|
|
||||||
- list mode prints effective locks from static `pipeline.publish.locks` and remote `{session_prefix}/locks.yml`.
|
Behavior:
|
||||||
- add/remove mutate only the remote lock store.
|
|
||||||
- static pipeline locks cannot be removed by lock commands.
|
|
||||||
|
|
||||||
## `--artifacts` Rules
|
- list mode merges static `pipeline.publish.locks` with remote `{session_prefix}/locks.yml`;
|
||||||
|
- add/remove mutate only remote locks;
|
||||||
|
- static locks from pipeline config cannot be removed by CLI commands.
|
||||||
|
|
||||||
- accepted on `run`, `resume`, `run-stage`, `analyze`, and `publish`.
|
## `--artifacts` Selection Rules
|
||||||
- on `run-stage`, only valid for `analyze` and `publish`.
|
|
||||||
- filters configured analyze artifact execution.
|
- accepted on `run`, `resume`, `run-stage`, `analyze`, and `publish`;
|
||||||
- filters configured `pipeline.publish.outputs` entries for `narratio.artifact.<key>` sources.
|
- names must exist in `pipeline.scriptorium.artifacts`;
|
||||||
- does not suppress built-in transcript/bounds publish outputs.
|
- empty entries are invalid;
|
||||||
- does not imply `--force` for `run`, `resume`, or `run-stage`.
|
- repeated names are deduplicated.
|
||||||
|
|
||||||
|
Effects:
|
||||||
|
|
||||||
|
- filters analyze execution to selected configured artifacts;
|
||||||
|
- filters publish rules that source `narratio.artifact.<name>`;
|
||||||
|
- does not filter built-in transcript/bounds publish sources.
|
||||||
|
|
||||||
## Common Workflows
|
## Common Workflows
|
||||||
|
|
||||||
@@ -204,16 +255,16 @@ Run full pipeline:
|
|||||||
narratio run 2026-04-04
|
narratio run 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Run only selected analyze artifacts:
|
Dry-run restore plan:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run 2026-04-04 --artifacts session_recap,player_handout
|
narratio session restore 2026-04-04 --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
Force analyze only:
|
Generate a concrete session file from template/default structure:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio analyze 2026-04-04 --artifacts player_handout
|
narratio session init 2026-04-04 --output ./session.yml --date 2026-04-04 --title "Session 12"
|
||||||
```
|
```
|
||||||
|
|
||||||
Force publish only:
|
Force publish only:
|
||||||
@@ -221,10 +272,3 @@ Force publish only:
|
|||||||
```bash
|
```bash
|
||||||
narratio publish 2026-04-04
|
narratio publish 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Restore preview then apply:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio session restore 2026-04-04 --dry-run
|
|
||||||
narratio session restore 2026-04-04
|
|
||||||
```
|
|
||||||
|
|||||||
212
docs/config.md
212
docs/config.md
@@ -1,56 +1,56 @@
|
|||||||
# Configuration
|
# Configuration Reference
|
||||||
|
|
||||||
## Overview
|
## Purpose
|
||||||
Narratio loads three YAML files:
|
|
||||||
|
|
||||||
- `pipeline.yml`: pipeline-level runtime settings.
|
Narratio resolves three YAML documents:
|
||||||
- `campaign.yml`: stable campaign identity and campaign-level input defaults.
|
|
||||||
- `session.yml`: per-session metadata and input selection.
|
|
||||||
|
|
||||||
Commands that load and validate all three files include:
|
- `pipeline.yml`: pipeline/runtime settings
|
||||||
|
- `campaign.yml`: campaign identity and stable input defaults
|
||||||
|
- `session.yml`: session identity, metadata, and audio source selection
|
||||||
|
|
||||||
- `narratio run`
|
## Discovery and Selection
|
||||||
- `narratio resume`
|
|
||||||
- `narratio run-stage`
|
|
||||||
- `narratio analyze`
|
|
||||||
- `narratio publish`
|
|
||||||
- `narratio session plan`
|
|
||||||
- `narratio session status`
|
|
||||||
- `narratio session validate`
|
|
||||||
- `narratio session restore`
|
|
||||||
- `narratio session artifacts`
|
|
||||||
- `narratio session locks`
|
|
||||||
- `narratio clean <session_id>`
|
|
||||||
|
|
||||||
Validation behavior:
|
### `pipeline.yml`
|
||||||
|
|
||||||
- strict YAML decode is enabled (`KnownFields(true)`); unknown fields fail.
|
When `--config` is omitted, search order is:
|
||||||
- loaded `session.yml` files must be concrete YAML (no `{{ ... }}` placeholders).
|
|
||||||
- defaults are applied for optional pipeline fields.
|
|
||||||
- campaign/session identity mismatches fail load.
|
|
||||||
|
|
||||||
## File Discovery
|
|
||||||
Pipeline discovery order when `--config` is omitted:
|
|
||||||
|
|
||||||
1. `/usr/local/etc/narratio/pipeline.yml`
|
1. `/usr/local/etc/narratio/pipeline.yml`
|
||||||
2. `/etc/narratio/pipeline.yml`
|
2. `/etc/narratio/pipeline.yml`
|
||||||
|
|
||||||
Session discovery order when `--session` is omitted:
|
### `campaign.yml`
|
||||||
|
|
||||||
|
Selection rules:
|
||||||
|
|
||||||
|
- if `--campaign-file` is set, use that path;
|
||||||
|
- else if `--campaign <id>` is set, use `{pipeline.campaigns.root}/{id}/campaign.yml`;
|
||||||
|
- else use `{pipeline.campaigns.root}/{pipeline.campaigns.default_campaign_id}/campaign.yml`.
|
||||||
|
|
||||||
|
### `session.yml`
|
||||||
|
|
||||||
|
When `--session` is omitted, local search order is:
|
||||||
|
|
||||||
1. `/usr/local/etc/narratio/session.yml`
|
1. `/usr/local/etc/narratio/session.yml`
|
||||||
2. `/etc/narratio/session.yml`
|
2. `/etc/narratio/session.yml`
|
||||||
|
|
||||||
Campaign discovery when `--campaign-file` is omitted:
|
If local session discovery fails and a `session_id` is known, Narratio attempts remote session loading from:
|
||||||
|
|
||||||
- if `--campaign <id>` is set: `{pipeline.campaigns.root}/{id}/campaign.yml`
|
- `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`
|
||||||
- otherwise: `{pipeline.campaigns.root}/{pipeline.campaigns.default_campaign_id}/campaign.yml`
|
|
||||||
|
|
||||||
Remote `session.yml` fallback:
|
using configured object storage.
|
||||||
|
|
||||||
- if local session discovery fails and storage is configured, Narratio can load:
|
## Validation and Merge Rules
|
||||||
- `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`
|
|
||||||
|
- YAML decode is strict (`KnownFields(true)`): unknown fields fail load.
|
||||||
|
- Session files must be concrete; unresolved `{{ ... }}` placeholders fail load.
|
||||||
|
- Pipeline defaults are applied before validation.
|
||||||
|
- Campaign and session identities must agree.
|
||||||
|
- Stable files (`speakers_file`, `autocorrect_file`, `glossary_file`) resolve from session overrides when provided, otherwise from campaign defaults.
|
||||||
|
- Exactly one audio mode must be configured in session input:
|
||||||
|
- local (`audio_dir` or `audio_files`), or
|
||||||
|
- S3 (`audio_s3.prefix`).
|
||||||
|
|
||||||
|
## Minimal Working Configuration
|
||||||
|
|
||||||
## Minimal Working Config
|
|
||||||
`pipeline.yml`
|
`pipeline.yml`
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
@@ -58,7 +58,7 @@ campaigns:
|
|||||||
root: /usr/local/share/narratio/campaigns
|
root: /usr/local/share/narratio/campaigns
|
||||||
default_campaign_id: sample-campaign
|
default_campaign_id: sample-campaign
|
||||||
whisperx:
|
whisperx:
|
||||||
transcribe_url: "https://transcription.example.com/transcribe"
|
transcribe_url: https://transcription.example.com/transcribe
|
||||||
```
|
```
|
||||||
|
|
||||||
`campaign.yml`
|
`campaign.yml`
|
||||||
@@ -71,7 +71,7 @@ inputs:
|
|||||||
glossary_file: ./glossary.yml
|
glossary_file: ./glossary.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
`session.yml`
|
`session.yml` (local audio)
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
session_id: 2026-05-03
|
session_id: 2026-05-03
|
||||||
@@ -79,8 +79,16 @@ inputs:
|
|||||||
audio_dir: ./audio
|
audio_dir: ./audio
|
||||||
```
|
```
|
||||||
|
|
||||||
## Publish Config
|
## Secrets Handling
|
||||||
Top-level publish settings live at `pipeline.publish`.
|
|
||||||
|
- Do not place raw secrets in YAML.
|
||||||
|
- Use env var names in config (for example `pipeline.audita.llm_api_key_env`).
|
||||||
|
- Optionally load env files from `pipeline.secrets.env_dir`.
|
||||||
|
- Commands that need storage/auth load filesystem secrets before constructing adapters.
|
||||||
|
|
||||||
|
## Publish Configuration Summary
|
||||||
|
|
||||||
|
Publish rules live under `pipeline.publish`.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
publish:
|
publish:
|
||||||
@@ -95,26 +103,21 @@ publish:
|
|||||||
required: true
|
required: true
|
||||||
locks:
|
locks:
|
||||||
- source: narratio.artifact.session_recap
|
- source: narratio.artifact.session_recap
|
||||||
reason: Final recap was manually edited.
|
reason: manual post-publish edits
|
||||||
```
|
```
|
||||||
|
|
||||||
Rules:
|
Rules:
|
||||||
|
|
||||||
- `outputs[].source` is required.
|
- `outputs[].source` is required.
|
||||||
- `outputs[].dest` is optional; when omitted, Narratio derives destination from the source.
|
- `outputs[].dest` may be omitted when derivable from source.
|
||||||
- `outputs[].required` defaults to `true`.
|
- `outputs[].required` defaults to `true`.
|
||||||
- static `publish.locks` and remote `{session_prefix}/locks.yml` are merged; static locks win on duplicates.
|
- static locks (`pipeline.publish.locks`) merge with remote locks (`{session_prefix}/locks.yml`), with static locks taking precedence on duplicates.
|
||||||
- locks prevent overwrite of top-level published destinations.
|
|
||||||
|
|
||||||
Supported publish source families:
|
## Full Schema
|
||||||
|
|
||||||
- built-ins: `narratio.transcript.base`, `narratio.transcript.polished`, `narratio.transcript.final`, `narratio.transcript.final_trimmed`, `narratio.bounds.session`
|
|
||||||
- configured artifacts: `narratio.artifact.<artifact_key>`
|
|
||||||
|
|
||||||
## Full Reference
|
|
||||||
|
|
||||||
### Pipeline
|
### Pipeline
|
||||||
| Path | Type | Required | Default |
|
|
||||||
|
| Field | Type | Required | Default / Rule |
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
|
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
|
||||||
| `pipeline.workspace.cleanup_after_publish` | bool | No | `false` |
|
| `pipeline.workspace.cleanup_after_publish` | bool | No | `false` |
|
||||||
@@ -122,7 +125,7 @@ Supported publish source families:
|
|||||||
| `pipeline.campaigns.default_campaign_id` | string | No | empty |
|
| `pipeline.campaigns.default_campaign_id` | string | No | empty |
|
||||||
| `pipeline.secrets.env_dir` | string | No | empty |
|
| `pipeline.secrets.env_dir` | string | No | empty |
|
||||||
| `pipeline.storage.backend` | string | No | empty |
|
| `pipeline.storage.backend` | string | No | empty |
|
||||||
| `pipeline.storage.s3.bucket` | string | Conditional | empty |
|
| `pipeline.storage.s3.bucket` | string | Conditional | required for S3 session-audio and for publish upload when backend is `s3` |
|
||||||
| `pipeline.storage.s3.root_prefix` | string | No | `dnd` |
|
| `pipeline.storage.s3.root_prefix` | string | No | `dnd` |
|
||||||
| `pipeline.storage.s3.region` | string | No | empty |
|
| `pipeline.storage.s3.region` | string | No | empty |
|
||||||
| `pipeline.storage.s3.endpoint` | string | No | empty |
|
| `pipeline.storage.s3.endpoint` | string | No | empty |
|
||||||
@@ -135,21 +138,21 @@ Supported publish source families:
|
|||||||
| `pipeline.cache.s3_audio` | bool | No | `true` |
|
| `pipeline.cache.s3_audio` | bool | No | `true` |
|
||||||
| `pipeline.publish.enabled` | bool | No | `true` |
|
| `pipeline.publish.enabled` | bool | No | `true` |
|
||||||
| `pipeline.publish.upload_run` | bool | No | `true` |
|
| `pipeline.publish.upload_run` | bool | No | `true` |
|
||||||
| `pipeline.publish.outputs[]` | list | No | one final-trimmed output rule |
|
| `pipeline.publish.outputs[]` | list | No | defaults to final trimmed transcript output |
|
||||||
| `pipeline.publish.outputs[].source` | string | Yes (per rule) | none |
|
| `pipeline.publish.outputs[].source` | string | Yes (per rule) | must reference built-in or configured artifact source |
|
||||||
| `pipeline.publish.outputs[].dest` | string | No | derived from source |
|
| `pipeline.publish.outputs[].dest` | string | Conditional | derived if omitted and source supports derivation |
|
||||||
| `pipeline.publish.outputs[].required` | bool | No | `true` |
|
| `pipeline.publish.outputs[].required` | bool | No | `true` |
|
||||||
| `pipeline.publish.locks[]` | list | No | empty |
|
| `pipeline.publish.locks[]` | list | No | empty |
|
||||||
| `pipeline.publish.locks[].source` | string | Yes (per lock) | none |
|
| `pipeline.publish.locks[].source` | string | Yes (per lock) | must reference supported publish source |
|
||||||
| `pipeline.publish.locks[].reason` | string | No | empty |
|
| `pipeline.publish.locks[].reason` | string | No | empty |
|
||||||
| `pipeline.whisperx.transcribe_url` | string | Yes | none |
|
| `pipeline.whisperx.transcribe_url` | string | Yes | valid URL |
|
||||||
| `pipeline.whisperx.language` | string | No | `en` |
|
| `pipeline.whisperx.language` | string | No | `en` |
|
||||||
| `pipeline.whisperx.timeout` | duration string | No | `30m` |
|
| `pipeline.whisperx.timeout` | duration | No | `30m` |
|
||||||
| `pipeline.whisperx.retries` | int | No | `3` |
|
| `pipeline.whisperx.retries` | int | No | `3` |
|
||||||
| `pipeline.whisperx.retry_delay` | duration string | No | `2s` |
|
| `pipeline.whisperx.retry_delay` | duration | No | `2s` |
|
||||||
| `pipeline.whisperx.concurrency` | int | No | `2` |
|
| `pipeline.whisperx.concurrency` | int | No | `2` |
|
||||||
| `pipeline.seriatim.binary` | string | No | `seriatim` |
|
| `pipeline.seriatim.binary` | string | No | `seriatim` |
|
||||||
| `pipeline.seriatim.timeout` | duration string | No | `10m` |
|
| `pipeline.seriatim.timeout` | duration | No | `10m` |
|
||||||
| `pipeline.seriatim.output_schema` | string | No | `seriatim-intermediate` |
|
| `pipeline.seriatim.output_schema` | string | No | `seriatim-intermediate` |
|
||||||
| `pipeline.seriatim.coalesce_gap` | float | No | `3.0` |
|
| `pipeline.seriatim.coalesce_gap` | float | No | `3.0` |
|
||||||
| `pipeline.seriatim.report` | bool | No | `true` |
|
| `pipeline.seriatim.report` | bool | No | `true` |
|
||||||
@@ -158,7 +161,7 @@ Supported publish source families:
|
|||||||
| `pipeline.seriatim.env.backchannel_max_duration` | float | No | unset |
|
| `pipeline.seriatim.env.backchannel_max_duration` | float | No | unset |
|
||||||
| `pipeline.seriatim.env.filler_max_duration` | float | No | unset |
|
| `pipeline.seriatim.env.filler_max_duration` | float | No | unset |
|
||||||
| `pipeline.audita.binary` | string | No | `audita` |
|
| `pipeline.audita.binary` | string | No | `audita` |
|
||||||
| `pipeline.audita.timeout` | duration string | No | `3h` |
|
| `pipeline.audita.timeout` | duration | No | `3h` |
|
||||||
| `pipeline.audita.llm_api_key_env` | string | No | empty |
|
| `pipeline.audita.llm_api_key_env` | string | No | empty |
|
||||||
| `pipeline.audita.modules[]` | list[string] | No | empty |
|
| `pipeline.audita.modules[]` | list[string] | No | empty |
|
||||||
| `pipeline.audita.base_url` | string | No | empty |
|
| `pipeline.audita.base_url` | string | No | empty |
|
||||||
@@ -176,52 +179,85 @@ Supported publish source families:
|
|||||||
| `pipeline.normalize.output_schema` | string | No | `seriatim-intermediate` |
|
| `pipeline.normalize.output_schema` | string | No | `seriatim-intermediate` |
|
||||||
| `pipeline.normalize.report` | bool | No | `true` |
|
| `pipeline.normalize.report` | bool | No | `true` |
|
||||||
| `pipeline.trim.enabled` | bool | No | `false` |
|
| `pipeline.trim.enabled` | bool | No | `false` |
|
||||||
| `pipeline.trim.output_path` | string | Conditional | none |
|
| `pipeline.trim.output_path` | string | Conditional | required when trim enabled |
|
||||||
| `pipeline.trim.bounds.prompt_id` | string | Conditional | none |
|
| `pipeline.trim.bounds.prompt_id` | string | Conditional | required when trim enabled |
|
||||||
| `pipeline.trim.bounds.profile_id` | string | No | empty |
|
| `pipeline.trim.bounds.profile_id` | string | No | empty |
|
||||||
| `pipeline.trim.bounds.transcript_input_name` | string | Conditional | none |
|
| `pipeline.trim.bounds.transcript_input_name` | string | Conditional | required when trim enabled |
|
||||||
| `pipeline.trim.bounds.output_path` | string | Conditional | none |
|
| `pipeline.trim.bounds.output_path` | string | Conditional | required when trim enabled |
|
||||||
| `pipeline.trim.bounds.timeout` | duration string | No | `10m` |
|
| `pipeline.trim.bounds.timeout` | duration | No | `10m` |
|
||||||
| `pipeline.trim.bounds.render_debug` | bool | No | `false` |
|
| `pipeline.trim.bounds.render_debug` | bool | No | `false` |
|
||||||
| `pipeline.trim.bounds.render_output_path` | string | Conditional | none |
|
| `pipeline.trim.bounds.render_output_path` | string | Conditional | required when `render_debug` is true |
|
||||||
| `pipeline.trim.seriatim.report` | bool | No | `false` |
|
| `pipeline.trim.seriatim.report` | bool | No | `false` |
|
||||||
| `pipeline.scriptorium.binary` | string | No | `scriptorium` |
|
| `pipeline.scriptorium.binary` | string | No | `scriptorium` |
|
||||||
| `pipeline.scriptorium.config_path` | string | No | empty |
|
| `pipeline.scriptorium.config_path` | string | No | empty |
|
||||||
| `pipeline.scriptorium.timeout` | duration string | No | `10m` |
|
| `pipeline.scriptorium.timeout` | duration | No | `10m` |
|
||||||
| `pipeline.scriptorium.render_debug` | bool | No | `false` |
|
| `pipeline.scriptorium.render_debug` | bool | No | `false` |
|
||||||
| `pipeline.scriptorium.artifacts` | map | No | empty |
|
| `pipeline.scriptorium.artifacts` | map | No | empty |
|
||||||
| `pipeline.notification.backend` | string | No | empty |
|
| `pipeline.notification.backend` | string | No | empty |
|
||||||
| `pipeline.notification.recipient` | string | No | empty |
|
| `pipeline.notification.recipient` | string | No | empty |
|
||||||
| `pipeline.notification.timeout` | duration string | No | `30s` |
|
| `pipeline.notification.timeout` | duration | No | `30s` |
|
||||||
|
|
||||||
|
### Scriptorium Artifact Entries
|
||||||
|
|
||||||
|
For each `pipeline.scriptorium.artifacts.<name>`:
|
||||||
|
|
||||||
|
| Field | Type | Required | Rule |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `enabled` | bool | No | `false` if omitted |
|
||||||
|
| `depends_on[]` | list[string] | No | must reference configured artifact keys; no self-reference; enabled graph must be acyclic |
|
||||||
|
| `render_debug` | bool | No | per-artifact override |
|
||||||
|
| `prompt_id` | string | Conditional | required when artifact is enabled |
|
||||||
|
| `profile_id` | string | No | empty |
|
||||||
|
| `output_path` | string | Conditional | required when enabled; also required when referenced by publish/output/input rules |
|
||||||
|
| `timeout` | duration | No | artifact override |
|
||||||
|
| `inputs` | map | No | input key names must be non-empty |
|
||||||
|
| `vars` | map | No | values must be string or bool |
|
||||||
|
|
||||||
|
For each artifact input `pipeline.scriptorium.artifacts.<name>.inputs.<input_name>`:
|
||||||
|
|
||||||
|
| Field | Type | Required | Rule |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `source` | string | Yes | built-in runtime source, `narratio.artifact.<name>`, or `narratio.previous_session.artifact.<name>` |
|
||||||
|
| `artifact` | string | No | optional passthrough adapter field |
|
||||||
|
| `path` | string | No | optional passthrough adapter field |
|
||||||
|
| `required` | bool | No | optional input requirement |
|
||||||
|
|
||||||
### Campaign
|
### Campaign
|
||||||
| Path | Type | Required |
|
|
||||||
| --- | --- | --- |
|
| Field | Type | Required | Notes |
|
||||||
| `campaign_id` | string | Yes |
|
| --- | --- | --- | --- |
|
||||||
| `session_template_file` | string | No |
|
| `campaign_id` | string | Yes | canonical campaign identity |
|
||||||
| `inputs.speakers_file` | string | Yes |
|
| `session_template_file` | string | No | used by `session init` when set |
|
||||||
| `inputs.autocorrect_file` | string | Yes |
|
| `inputs.speakers_file` | string | Yes | stable input default |
|
||||||
| `inputs.glossary_file` | string | Yes |
|
| `inputs.autocorrect_file` | string | Yes | stable input default |
|
||||||
|
| `inputs.glossary_file` | string | Yes | stable input default |
|
||||||
|
|
||||||
### Session
|
### Session
|
||||||
| Path | Type | Required |
|
|
||||||
| --- | --- | --- |
|
| Field | Type | Required in session file | Notes |
|
||||||
| `session_id` | string | Yes |
|
| --- | --- | --- | --- |
|
||||||
| `previous_session_id` | string | No |
|
| `session_id` | string | Yes | must match CLI session target when provided |
|
||||||
| `campaign` | string | No |
|
| `previous_session_id` | string | No | must not equal `session_id` |
|
||||||
| `date` | string | No |
|
| `campaign` | string | No | filled from `campaign_id` during resolve if omitted |
|
||||||
| `title` | string | No |
|
| `date` | string | No | metadata |
|
||||||
| `inputs.speakers_file` | string | No |
|
| `title` | string | No | metadata |
|
||||||
| `inputs.autocorrect_file` | string | No |
|
| `inputs.speakers_file` | string | No | overrides campaign stable input |
|
||||||
| `inputs.glossary_file` | string | No |
|
| `inputs.autocorrect_file` | string | No | overrides campaign stable input |
|
||||||
| `inputs.audio_dir` | string | Conditional |
|
| `inputs.glossary_file` | string | No | overrides campaign stable input |
|
||||||
| `inputs.audio_files[]` | list[string] | Conditional |
|
| `inputs.audio_dir` | string | Conditional | local audio mode |
|
||||||
| `inputs.audio_s3.prefix` | string | Conditional |
|
| `inputs.audio_files[]` | list[string] | Conditional | local audio mode |
|
||||||
|
| `inputs.audio_s3.prefix` | string | Conditional | S3 audio mode |
|
||||||
|
|
||||||
|
Audio rules:
|
||||||
|
|
||||||
|
- configure local mode (`audio_dir` or `audio_files`) or S3 mode (`audio_s3.prefix`), not both.
|
||||||
|
|
||||||
## Maintained Examples
|
## Maintained Examples
|
||||||
|
|
||||||
- `examples/pipeline.minimal.yml`
|
- `examples/pipeline.minimal.yml`
|
||||||
- `examples/pipeline.production.yml`
|
- `examples/pipeline.production.yml`
|
||||||
- `examples/pipeline.full.annotated.yml`
|
- `examples/pipeline.full.annotated.yml`
|
||||||
- `examples/campaigns/sample-campaign/campaign.yml`
|
- `examples/campaigns/sample-campaign/campaign.yml`
|
||||||
- `examples/session.local-audio.yml`
|
- `examples/session.local-audio.yml`
|
||||||
- `examples/session.s3-audio.yml`
|
- `examples/session.s3-audio.yml`
|
||||||
|
- `examples/session.template.yml`
|
||||||
|
|||||||
@@ -1,15 +1,19 @@
|
|||||||
# Integration Documentation Index
|
# Integrations Index
|
||||||
|
|
||||||
## Audience
|
## Audience
|
||||||
Developers and LLM coding agents changing Narratio's external integration contracts.
|
Developers and coding agents changing Narratio's external integration boundaries.
|
||||||
|
|
||||||
## Scope
|
## Scope
|
||||||
Implemented-only reference notes for the external systems Narratio currently integrates with.
|
`docs/integrations/` is the implementation-level reference for downstream tool adapter contracts.
|
||||||
|
|
||||||
## Integration Docs
|
These docs cover what Narratio expects from external tools and what each adapter guarantees back to stage code.
|
||||||
- `audita.md`: Audita adapter invocation and validation contract.
|
|
||||||
- `seriatim.md`: Seriatim normalize/merge/trim adapter contract.
|
|
||||||
- `scriptorium.md`: Scriptorium run/render adapter contract.
|
|
||||||
|
|
||||||
## Canonical Owner
|
## Integration Contracts
|
||||||
`docs/integrations/` is the canonical home for external integration reference notes per `docs/documentation/policy.md`.
|
- `audita.md`: transcript polishing adapter (`audita process`).
|
||||||
|
- `seriatim.md`: merge/normalize/trim adapter (`seriatim`).
|
||||||
|
- `scriptorium.md`: artifact run/render adapter (`scriptorium run|render`).
|
||||||
|
|
||||||
|
## Related Canonical Docs
|
||||||
|
- `docs/config.md`: operator-facing configuration reference.
|
||||||
|
- `docs/internal/adapters.md`: shared adapter boundary and runner wiring.
|
||||||
|
- `docs/internal/stage-*.md`: stage-specific integration usage.
|
||||||
|
|||||||
@@ -1,66 +1,60 @@
|
|||||||
# Integration: audita
|
# Integration: Audita
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define Narratio's adapter contract for transcript polishing via Audita CLI subprocess execution.
|
Define the Audita adapter contract used by the `polish` stage.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Adapter Boundary
|
||||||
Inputs (`audita.PolishRequest`):
|
Interface:
|
||||||
- base transcript path
|
- `audita.Runner`
|
||||||
- glossary path
|
- method: `Run(ctx, PolishRequest) (PolishResult, error)`
|
||||||
- output polished transcript path
|
|
||||||
- optional report path (required when report enabled)
|
|
||||||
- work dir
|
|
||||||
- generated config path
|
|
||||||
- stdout/stderr log paths
|
|
||||||
- optional module/model/base URL and concurrency knobs
|
|
||||||
|
|
||||||
Outputs (`audita.PolishResult`):
|
Primary implementation:
|
||||||
- polished transcript path
|
- `internal/adapters/audita/SubprocessRunner`
|
||||||
- optional report path
|
|
||||||
- generated config path
|
|
||||||
- stdout/stderr log paths
|
|
||||||
- exit code, duration, invoked binary
|
|
||||||
- adapter metadata
|
|
||||||
|
|
||||||
## Boundaries
|
Execution mode:
|
||||||
Owns:
|
- subprocess invocation of `audita process`
|
||||||
- Deterministic CLI argument construction for `audita process`
|
|
||||||
- Environment bridging for API credentials
|
|
||||||
- Invocation config emission
|
|
||||||
- Output validation for polished transcript and report
|
|
||||||
|
|
||||||
Does not own:
|
## Request Contract
|
||||||
- Upstream/downstream stage orchestration
|
`PolishRequest` carries:
|
||||||
- Credential sourcing policy beyond required env-var presence check
|
- required transcript/glossary/output/work-dir paths;
|
||||||
|
- optional report path (required when report mode is enabled);
|
||||||
|
- generated config and stdout/stderr log paths;
|
||||||
|
- optional module/model/base-url/config/output-schema/concurrency settings.
|
||||||
|
|
||||||
## Config Fields Used
|
## Result Contract
|
||||||
Via `pipeline.audita.*` mapped in app/stage wiring:
|
`PolishResult` returns:
|
||||||
- `binary`, `timeout`, `llm_api_key_env`, `modules`, `base_url`, `model`
|
- processed transcript path;
|
||||||
- `transcript_description`, `config_path`, `output_schema`, `work_dir_retention`
|
- optional report path;
|
||||||
- `total_llm_concurrency`, `proposal_llm_concurrency`, `validation_model`, `validation_llm_concurrency`, `report`
|
- work dir and generated-config/log paths;
|
||||||
|
- exit code, duration, binary provenance;
|
||||||
|
- adapter metadata map.
|
||||||
|
|
||||||
## External Adapters Used
|
## Validation and Failure Semantics
|
||||||
- Shared subprocess helper (`internal/adapters/subprocess`) to run CLI and capture logs.
|
Construction fails for invalid static config values, including:
|
||||||
|
- empty binary;
|
||||||
|
- non-positive timeout;
|
||||||
|
- invalid base URL;
|
||||||
|
- invalid output schema;
|
||||||
|
- invalid work-dir retention value;
|
||||||
|
- invalid concurrency values.
|
||||||
|
|
||||||
## State and Manifest Behavior
|
Run fails for:
|
||||||
- No direct manifest writes.
|
- missing required request paths;
|
||||||
- Stage-level metadata records adapter provenance and credential-present signal.
|
- missing required credential env var when configured (`llm_api_key_env`);
|
||||||
- Generated invocation YAML is written when `GeneratedConfigPath` is provided.
|
- subprocess execution failure;
|
||||||
|
- invalid processed transcript JSON (`segments` array required);
|
||||||
|
- invalid report JSON when reporting is enabled.
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
Failure results still include output/log/config/exit metadata for diagnostics.
|
||||||
- Adapter has no skip/resume logic. Stage/runner controls this.
|
|
||||||
|
|
||||||
## Failure Behavior
|
## Deterministic Behavior
|
||||||
- Constructor validation fails on invalid binary/timeout/schema/concurrency/URL values.
|
- CLI args are built from runner config + request in a fixed order.
|
||||||
- Run fails on missing required paths, missing required credential env var, subprocess errors, invalid polished JSON shape, or invalid report JSON.
|
- Generated invocation YAML (`audita.generated.v1`) is emitted when requested.
|
||||||
- Failures preserve stdout/stderr paths in returned result metadata.
|
- Manifest writes are stage-owned; adapter itself is stateless.
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
## Config Mapping
|
||||||
- `internal/adapters/audita/subprocess_test.go`
|
Config fields consumed through runner/stage wiring are under `pipeline.audita.*`.
|
||||||
- `internal/adapters/audita/fake_test.go`
|
|
||||||
- `internal/stage/polish_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
Maintained example with Audita config:
|
||||||
- Polished output must be valid JSON with top-level `segments` array.
|
- `examples/pipeline.full.annotated.yml`
|
||||||
- When report is enabled, report output must be valid JSON.
|
- `examples/pipeline.production.yml`
|
||||||
- If `llm_api_key_env` is configured, credential must be present in environment.
|
|
||||||
|
|||||||
@@ -1,64 +1,66 @@
|
|||||||
# Integration: scriptorium
|
# Integration: Scriptorium
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define Narratio's adapter contract for Scriptorium artifact generation and render-debug subprocess invocations.
|
Define the Scriptorium adapter contract used by `analyze` and trim-bounds generation in `trim`.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Adapter Boundary
|
||||||
Inputs:
|
Interface:
|
||||||
- `RunArtifactRequest`: binary, config path, prompt/profile IDs, input map, vars map, timeout, output path, logs/config paths, optional API env and working dir
|
- `scriptorium.Runner`
|
||||||
- `RenderArtifactRequest`: same core fields for render mode
|
- methods:
|
||||||
|
- `RunArtifact(ctx, RunArtifactRequest)`
|
||||||
|
- `RenderArtifact(ctx, RenderArtifactRequest)`
|
||||||
|
|
||||||
Outputs (`ArtifactResult`):
|
Primary implementation:
|
||||||
- output path
|
- `internal/adapters/scriptorium/SubprocessRunner`
|
||||||
- stdout/stderr log paths
|
|
||||||
- generated config path
|
|
||||||
- exit code and duration
|
|
||||||
- command mode (`run` or `render`)
|
|
||||||
- prompt/profile provenance
|
|
||||||
- validation failure signal
|
|
||||||
- adapter metadata
|
|
||||||
|
|
||||||
## Boundaries
|
Execution modes:
|
||||||
Owns:
|
- `scriptorium run`
|
||||||
- Deterministic CLI arg construction for `scriptorium run` and `scriptorium render`
|
- `scriptorium render`
|
||||||
- Common request validation
|
|
||||||
- Invocation config emission
|
|
||||||
- Output existence/non-empty checks
|
|
||||||
- Validation-failure mapping for run exit code 2
|
|
||||||
|
|
||||||
Does not own:
|
## Request Contract
|
||||||
- Artifact selection policy (`analyze` stage)
|
Both request types carry:
|
||||||
- Bounds semantic validation (`trim` stage)
|
- binary/config/prompt/profile IDs;
|
||||||
|
- input map and vars map;
|
||||||
|
- output path;
|
||||||
|
- timeout;
|
||||||
|
- generated config + stdout/stderr log paths;
|
||||||
|
- optional API-key env var name;
|
||||||
|
- optional working directory.
|
||||||
|
|
||||||
## Config Fields Used
|
## Result Contract
|
||||||
Via `pipeline.scriptorium.*` and stage-level artifact config:
|
`ArtifactResult` returns:
|
||||||
- `binary`, `config_path`, `timeout`, `render_debug`
|
- output/log/generated-config paths;
|
||||||
- artifact-level `prompt_id`, `profile_id`, `timeout`, `inputs`, `vars`, `output_path`
|
- exit code and duration;
|
||||||
|
- command mode (`run` or `render`);
|
||||||
|
- prompt/profile provenance;
|
||||||
|
- `ValidationFailed` marker;
|
||||||
|
- metadata map.
|
||||||
|
|
||||||
## External Adapters Used
|
## Validation and Failure Semantics
|
||||||
- Shared subprocess helper (`internal/adapters/subprocess`).
|
Request validation fails for:
|
||||||
|
- missing binary, prompt id, or output path;
|
||||||
|
- non-positive timeout;
|
||||||
|
- empty input/var names;
|
||||||
|
- empty input path values;
|
||||||
|
- missing required credential env var when `APIKeyEnv` is set.
|
||||||
|
|
||||||
## State and Manifest Behavior
|
Run behavior:
|
||||||
- No direct manifest writes.
|
- subprocess errors propagate with context;
|
||||||
- Stage metadata records adapter outputs and command mode.
|
- `run` exit code `2` is mapped to `ValidationFailed=true`;
|
||||||
- Generated invocation YAML is written when requested.
|
- successful subprocess still fails if output file is missing or empty.
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
Render behavior:
|
||||||
- Adapter has no skip/resume logic. Stage/runner controls execution.
|
- subprocess errors propagate;
|
||||||
|
- output file must exist and be non-empty.
|
||||||
|
|
||||||
## Failure Behavior
|
## Deterministic Behavior
|
||||||
- Request validation fails for missing binary/prompt/output, invalid timeout, invalid input/var names, or missing required API env var.
|
- input and var maps are sorted into deterministic `--input` and `--var` CLI args.
|
||||||
- Subprocess errors bubble with command context.
|
- generated invocation YAML (`scriptorium.generated.v1`) is emitted when requested.
|
||||||
- `run` exit code 2 is treated as `ValidationFailed=true` and surfaced as error by calling stage.
|
- adapter is stateless and does not own artifact-selection policy.
|
||||||
- Successful subprocess still fails if output file is missing/empty.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
## Config Mapping
|
||||||
- `internal/adapters/scriptorium/subprocess_test.go`
|
Config fields consumed through runner/stage wiring are under `pipeline.scriptorium.*` plus per-artifact settings under `pipeline.scriptorium.artifacts.*`.
|
||||||
- `internal/adapters/scriptorium/fake_test.go`
|
|
||||||
- `internal/stage/analyze_test.go`
|
|
||||||
- `internal/stage/trim_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
Maintained examples with Scriptorium config:
|
||||||
- Both modes require explicit timeout > 0.
|
- `examples/pipeline.full.annotated.yml`
|
||||||
- Input/var maps are sorted into deterministic CLI argument order.
|
- `examples/pipeline.production.yml`
|
||||||
- Run-mode validation failures are represented explicitly, not silently skipped.
|
|
||||||
|
|||||||
@@ -1,60 +1,56 @@
|
|||||||
# Integration: seriatim
|
# Integration: Seriatim
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define Narratio's adapter contract for merge, normalize, and trim subprocess invocations of Seriatim.
|
Define the Seriatim adapter contract used by `merge`, `normalize`, and `trim`.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Adapter Boundary
|
||||||
Inputs:
|
Interface:
|
||||||
- `MergeRequest`: raw/per-speaker normalized transcript inputs, base output path, optional report, speaker/autocorrect paths, logs/config
|
- `seriatim.Runner`
|
||||||
- `NormalizeRequest`: input transcript, output path, schema, optional report, timeout/log/config
|
- methods:
|
||||||
- `TrimRequest`: input transcript, output path, keep selector, timeout/log/config
|
- `Run(ctx, MergeRequest)`
|
||||||
|
- `Normalize(ctx, NormalizeRequest)`
|
||||||
|
- `Trim(ctx, TrimRequest)`
|
||||||
|
|
||||||
Outputs:
|
Primary implementation:
|
||||||
- `MergeResult`, `NormalizeResult`, `TrimResult` with output paths, logs/config paths, exit code, duration, binary provenance, and metadata.
|
- `internal/adapters/seriatim/SubprocessRunner`
|
||||||
|
|
||||||
## Boundaries
|
Execution modes:
|
||||||
Owns:
|
- `seriatim merge`
|
||||||
- Validated deterministic CLI invocation construction
|
- `seriatim normalize`
|
||||||
- Optional env tuning propagation for merge
|
- `seriatim trim`
|
||||||
- Invocation config file emission
|
|
||||||
- JSON output validation
|
|
||||||
|
|
||||||
Does not own:
|
## Request/Result Contracts
|
||||||
- Transcript input selection/materialization logic (stage-owned)
|
- `MergeRequest`/`MergeResult`: multi-input merge to base transcript, optional report.
|
||||||
- Bounds computation (scriptorium/trim-stage-owned)
|
- `NormalizeRequest`/`NormalizeResult`: transcript normalization with explicit schema.
|
||||||
|
- `TrimRequest`/`TrimResult`: transcript trimming with required keep selector.
|
||||||
|
|
||||||
## Config Fields Used
|
Results include output/log/config paths, timing, exit code, and metadata.
|
||||||
Via `pipeline.seriatim.*` mapped in app/stage wiring:
|
|
||||||
- `binary`, `timeout`, `output_schema`, `coalesce_gap`, `report`
|
|
||||||
- `env.overlap_word_run_gap`
|
|
||||||
- `env.overlap_word_run_reorder_window`
|
|
||||||
- `env.backchannel_max_duration`
|
|
||||||
- `env.filler_max_duration`
|
|
||||||
|
|
||||||
## External Adapters Used
|
## Validation and Failure Semantics
|
||||||
- Shared subprocess helper (`internal/adapters/subprocess`).
|
Runner construction validates:
|
||||||
|
- binary presence;
|
||||||
|
- timeout > 0;
|
||||||
|
- supported output schema (`seriatim-minimal|seriatim-intermediate|seriatim-full`);
|
||||||
|
- non-negative coalesce gap.
|
||||||
|
|
||||||
## State and Manifest Behavior
|
Invocation fails on:
|
||||||
- No direct manifest writes.
|
- missing required request paths/inputs;
|
||||||
- Stage metadata consumes adapter result fields and preserves generated config/log references.
|
- invalid normalize schema override;
|
||||||
|
- subprocess failure;
|
||||||
|
- invalid JSON outputs;
|
||||||
|
- missing `segments` array for normalize/trim transcript outputs.
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
When report paths are provided/enabled, report files must parse as JSON.
|
||||||
- Adapter has no skip/resume logic. Runner controls stage execution.
|
|
||||||
|
|
||||||
## Failure Behavior
|
## Deterministic Behavior
|
||||||
- Constructor fails for invalid binary/timeout/output-schema/coalesce-gap.
|
- argument ordering is deterministic per command construction.
|
||||||
- Merge fails on missing output path/inputs/report path (if enabled), subprocess errors, invalid merged output JSON, invalid report JSON.
|
- merge env overrides are explicit (`SERIATIM_*`) and only emitted when configured.
|
||||||
- Normalize fails on missing input/output, invalid schema, subprocess errors, invalid final output JSON shape, invalid report JSON.
|
- generated invocation YAML (`seriatim.generated.v1`) is emitted when requested.
|
||||||
- Trim fails on missing input/output/keep selector, subprocess errors, invalid final-trimmed output JSON shape.
|
- adapter does not write manifests or choose stage inputs.
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
## Config Mapping
|
||||||
- `internal/adapters/seriatim/subprocess_test.go`
|
Config fields consumed through runner/stage wiring are under `pipeline.seriatim.*`.
|
||||||
- `internal/adapters/seriatim/fake_test.go`
|
|
||||||
- `internal/stage/merge_test.go`
|
|
||||||
- `internal/stage/normalize_test.go`
|
|
||||||
- `internal/stage/trim_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
Maintained examples with Seriatim config:
|
||||||
- Supported output schemas are limited to `seriatim-minimal`, `seriatim-intermediate`, `seriatim-full`.
|
- `examples/pipeline.full.annotated.yml`
|
||||||
- Final and final-trimmed outputs must include `segments` arrays.
|
- `examples/pipeline.production.yml`
|
||||||
- Merge/normalize/trim all route through deterministic subprocess invocation.
|
|
||||||
|
|||||||
@@ -1,29 +1,43 @@
|
|||||||
# Internal Documentation Index
|
# Internal Documentation Index
|
||||||
|
|
||||||
## Audience
|
## Audience
|
||||||
Developers and LLM coding agents changing Narratio internals.
|
Developers and coding agents changing Narratio internals.
|
||||||
|
|
||||||
## Scope
|
## Scope
|
||||||
Implementation-accurate contracts for workspace/state, manifests, stages, artifact resolution, adapter boundaries, and restore command behavior.
|
`docs/internal/` documents implemented internal contracts: stage boundaries, manifest/state behavior, artifact resolution, restore behavior, storage boundaries, and workspace invariants.
|
||||||
|
|
||||||
## Component Docs
|
User and operator behavior belongs in:
|
||||||
- `adapters.md`: external adapter map, runtime wiring, and boundary ownership.
|
- `docs/cli.md`
|
||||||
- `storage.md`: remote storage backend contracts and object-store invariants.
|
- `docs/config.md`
|
||||||
- `manifest.md`: session/run manifest schemas, lifecycle transitions, and persistence semantics.
|
- `docs/operations.md`
|
||||||
- `artifacts.md`: built-in artifact registry, runtime artifact catalog, and source-resolution behavior.
|
- `docs/troubleshooting.md`
|
||||||
- `workspace.md`: local state model, manifests, run-local layout, materialization, and cleanup invariants.
|
|
||||||
- `command-restore.md`: restore command discovery/planning/execution/reporting contract.
|
|
||||||
- `stage-prepare.md`: input materialization and provenance capture.
|
|
||||||
- `stage-transcribe.md`: WhisperX transcript generation.
|
|
||||||
- `stage-merge.md`: Seriatim normalization + merge.
|
|
||||||
- `stage-polish.md`: Audita transcript polishing.
|
|
||||||
- `stage-normalize.md`: post-polish normalization.
|
|
||||||
- `stage-trim.md`: bounds-driven transcript trimming.
|
|
||||||
- `stage-analyze.md`: dependency-ordered Scriptorium artifact generation for selected configured artifacts.
|
|
||||||
- `stage-publish.md`: publish upload and current-pointer commit contract.
|
|
||||||
|
|
||||||
## External Integration Notes
|
## Pipeline Stage Set
|
||||||
- `../integrations/README.md`: canonical location for external integration contracts (`audita.md`, `seriatim.md`, `scriptorium.md`).
|
Canonical stage order from `internal/stage.All()`:
|
||||||
|
1. `prepare`
|
||||||
|
2. `transcribe`
|
||||||
|
3. `merge`
|
||||||
|
4. `polish`
|
||||||
|
5. `normalize`
|
||||||
|
6. `trim`
|
||||||
|
7. `analyze`
|
||||||
|
8. `publish`
|
||||||
|
9. `notify` (placeholder)
|
||||||
|
|
||||||
## Canonical Owner
|
`notify` is currently a placeholder stage with optional notifier call behavior; it has no persisted pipeline outputs.
|
||||||
`docs/internal/` is the canonical home for implemented internals per `docs/documentation/policy.md`.
|
|
||||||
|
## Internal Component Docs
|
||||||
|
- `adapters.md`: external adapter boundaries and default runtime wiring.
|
||||||
|
- `artifacts.md`: canonical source IDs, runtime catalog behavior, and resolution rules.
|
||||||
|
- `manifest.md`: session and run manifest contracts.
|
||||||
|
- `storage.md`: object-store interface and S3 implementation behavior.
|
||||||
|
- `workspace.md`: local session layout, run-local layout, and cleanup guardrails.
|
||||||
|
- `command-restore.md`: restore discovery, planning, execution, and reporting.
|
||||||
|
- `stage-prepare.md`
|
||||||
|
- `stage-transcribe.md`
|
||||||
|
- `stage-merge.md`
|
||||||
|
- `stage-polish.md`
|
||||||
|
- `stage-normalize.md`
|
||||||
|
- `stage-trim.md`
|
||||||
|
- `stage-analyze.md`
|
||||||
|
- `stage-publish.md`
|
||||||
|
|||||||
@@ -1,38 +1,12 @@
|
|||||||
# Internal: Adapters
|
# Internal: Adapters
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Describe the external adapter boundaries used by Narratio stages and app orchestration, including default runtime wiring.
|
Define external integration boundaries and default adapter wiring used by app/stage orchestration.
|
||||||
|
|
||||||
## Inputs and outputs
|
## Adapter Boundaries
|
||||||
Inputs:
|
Narratio stage logic depends on adapter interfaces, not transport-specific details.
|
||||||
- Stage requests passed through adapter interfaces (for example transcription, merge/normalize/trim, polish, artifact generation, object-store operations, notifications).
|
|
||||||
- Resolved config values used to construct default adapters.
|
|
||||||
|
|
||||||
Outputs:
|
Primary adapters:
|
||||||
- Adapter-specific result structs (paths, metadata, status/attempt info, duration/exit details).
|
|
||||||
- Adapter errors returned to stage/app orchestration.
|
|
||||||
|
|
||||||
## Boundaries
|
|
||||||
Owns:
|
|
||||||
- Transport/process/SDK details at system boundaries (`HTTP`, subprocess CLI invocation, AWS SDK calls).
|
|
||||||
- Request/response contracts in `internal/adapters/*` packages.
|
|
||||||
|
|
||||||
Does not own:
|
|
||||||
- Stage sequencing, skip/force/resume decisions.
|
|
||||||
- Manifest transition logic.
|
|
||||||
- Canonical workspace path policy.
|
|
||||||
|
|
||||||
## Config fields used
|
|
||||||
Default wiring and adapter calls consume:
|
|
||||||
- `pipeline.whisperx.*`
|
|
||||||
- `pipeline.seriatim.*`
|
|
||||||
- `pipeline.audita.*`
|
|
||||||
- `pipeline.scriptorium.*`
|
|
||||||
- `pipeline.storage.*` and `pipeline.publish.*` (object-store construction/gating)
|
|
||||||
- `pipeline.notification.*` (sender boundary exists; placeholder behavior today)
|
|
||||||
|
|
||||||
## External adapters used
|
|
||||||
Runtime env boundary fields (`internal/stage.Env`):
|
|
||||||
- `whisperx.Client`
|
- `whisperx.Client`
|
||||||
- `seriatim.Runner`
|
- `seriatim.Runner`
|
||||||
- `audita.Runner`
|
- `audita.Runner`
|
||||||
@@ -40,39 +14,39 @@ Runtime env boundary fields (`internal/stage.Env`):
|
|||||||
- `storage.ObjectStore`
|
- `storage.ObjectStore`
|
||||||
- `notify.Sender`
|
- `notify.Sender`
|
||||||
|
|
||||||
Current execution usage:
|
Legacy compatibility boundary:
|
||||||
- Actively used by implemented stages: `WhisperX`, `Seriatim`, `Audita`, `Scriptorium`, `ObjectStore`, `Notifier`.
|
- `storage.Backend` remains in the storage adapter package and defaults to `NoopBackend`; current pipeline stages use `storage.ObjectStore`.
|
||||||
- Present but not used by implemented stage set: legacy `storage.Backend`.
|
|
||||||
|
|
||||||
Default construction in app runner:
|
## Ownership
|
||||||
- Auto-constructed when not injected: WhisperX HTTP client, Seriatim subprocess runner, Audita subprocess runner, Scriptorium subprocess runner, object store (only when needed), and `notify.NoopSender`.
|
Adapters own:
|
||||||
- Object-store construction goes through app command orchestration so configured filesystem secrets are loaded before the storage adapter is initialized.
|
- HTTP/subprocess/SDK argument and transport details.
|
||||||
- Callers can inject test/fake implementations through `app.RunOptions.Env`.
|
- Backend-specific request/response mapping.
|
||||||
|
|
||||||
## State and manifest behavior
|
Adapters do not own:
|
||||||
- Adapters do not directly mutate session/run manifests.
|
- stage ordering/skip/force logic;
|
||||||
- Stages and runner own manifest writes and stage status transitions.
|
- manifest transitions;
|
||||||
- Adapter outputs are persisted indirectly through stage result mapping (outputs/logs/generated configs/metadata).
|
- canonical path policy.
|
||||||
|
|
||||||
## Skip and resume behavior
|
## Default Wiring
|
||||||
- No adapter-level skip/resume semantics.
|
`internal/app/runner.go` initializes default adapters when not injected:
|
||||||
- Skip/resume/force behavior is decided by app runner using manifest stage state.
|
- WhisperX HTTP client from pipeline config.
|
||||||
|
- Seriatim subprocess runner.
|
||||||
|
- Audita subprocess runner.
|
||||||
|
- Scriptorium subprocess runner.
|
||||||
|
- Noop notifier (`notify.NoopSender`).
|
||||||
|
- Object store only when required by selected stages/config.
|
||||||
|
|
||||||
## Failure behavior
|
Object-store construction goes through `newCommandObjectStore`, which loads configured filesystem secrets before adapter initialization.
|
||||||
- Adapter constructors validate config-derived values and fail early on invalid required inputs.
|
|
||||||
- Adapter run-time failures are returned to stage code with boundary context and are recorded as stage failures by runner logic.
|
|
||||||
- Subprocess adapters preserve stdout/stderr and generated-config paths to aid diagnosis.
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
## Failure Semantics
|
||||||
|
- Constructor errors fail stage execution setup early.
|
||||||
|
- Runtime adapter errors propagate to stage code and then manifest failure handling.
|
||||||
|
- Subprocess adapters persist stage logs/generated configs through stage-managed paths.
|
||||||
|
|
||||||
|
## Test Surfaces
|
||||||
- `internal/adapters/whisperx/http_test.go`
|
- `internal/adapters/whisperx/http_test.go`
|
||||||
- `internal/adapters/seriatim/subprocess_test.go`
|
- `internal/adapters/seriatim/subprocess_test.go`
|
||||||
- `internal/adapters/audita/subprocess_test.go`
|
- `internal/adapters/audita/subprocess_test.go`
|
||||||
- `internal/adapters/scriptorium/subprocess_test.go`
|
- `internal/adapters/scriptorium/subprocess_test.go`
|
||||||
- `internal/adapters/storage/*_test.go`
|
- `internal/adapters/storage/*_test.go`
|
||||||
- `internal/adapters/notify/fake_test.go`
|
|
||||||
- `internal/app/runner_test.go`
|
- `internal/app/runner_test.go`
|
||||||
|
|
||||||
## Architectural invariants
|
|
||||||
- Stage code depends on adapter interfaces, not transport-specific implementation types.
|
|
||||||
- External SDK-specific types remain inside adapter implementations.
|
|
||||||
- Default app wiring must remain deterministic and overrideable via injected env dependencies.
|
|
||||||
|
|||||||
@@ -1,106 +1,68 @@
|
|||||||
# Internal: Artifacts
|
# Internal: Artifacts
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define Narratio artifact identity, catalog, and source-resolution behavior for:
|
Define canonical artifact IDs, runtime catalog behavior, and source resolution rules for stage execution and publish output selection.
|
||||||
- built-in session artifacts;
|
|
||||||
- configured analyze artifacts;
|
|
||||||
- canonical previous-session artifact sources.
|
|
||||||
|
|
||||||
## Inputs and outputs
|
## Built-in Source IDs
|
||||||
Inputs:
|
- `narratio.transcript.base` -> `transcripts/base.json` (`merge`)
|
||||||
- configured input sources (`pipeline.scriptorium.artifacts.*.inputs.*.source`);
|
- `narratio.transcript.polished` -> `transcripts/polished.json` (`polish`)
|
||||||
- session paths and manifest inputs/outputs;
|
- `narratio.transcript.final` -> `transcripts/final.json` (`normalize`)
|
||||||
- runtime catalog state.
|
- `narratio.transcript.final_trimmed` -> `transcripts/final.trimmed.json` (`trim`)
|
||||||
|
- `narratio.bounds.session` -> `artifacts/session_bounds.json` (`trim`)
|
||||||
|
|
||||||
Outputs:
|
## Configured and Previous-Session Sources
|
||||||
- resolved artifact path + provenance (`ResolvedSessionArtifact`);
|
- Configured artifact source ID: `narratio.artifact.<artifact_key>`
|
||||||
- runtime catalog entries for built-ins and configured artifacts;
|
- Previous-session source ID: `narratio.previous_session.artifact.<artifact_key>`
|
||||||
- requirement sets for canonical previous-session inputs;
|
|
||||||
- canonical S3 session, run, current, session config, session locks, audio, and published output keys.
|
|
||||||
|
|
||||||
## Boundaries
|
Configured and previous-session source IDs are validated by strict regex rules.
|
||||||
Owns:
|
|
||||||
- built-in source registry and validation;
|
|
||||||
- configured artifact catalog identity (`narratio.artifact.<name>`);
|
|
||||||
- canonical previous-session source parsing and resolution;
|
|
||||||
- previous-session requirement collection (`CollectPreviousArtifactRequirements`).
|
|
||||||
|
|
||||||
Does not own:
|
## Runtime Catalog
|
||||||
- prepare-stage remote hydration;
|
`ArtifactCatalog` tracks:
|
||||||
- stage success/skip transitions;
|
- `planned`: source registered for run context.
|
||||||
- publish upload orchestration.
|
- `executable`: selected and enabled for analyze execution.
|
||||||
|
- `available`: local file exists and validated.
|
||||||
|
- `provenance`: availability source.
|
||||||
|
|
||||||
## Built-in IDs
|
Current provenance values:
|
||||||
| Artifact ID | Canonical file | Producer stage | Output kind |
|
|
||||||
| --- | --- | --- | --- |
|
|
||||||
| `narratio.transcript.base` | `transcripts/base.json` | `merge` | `transcript_base` |
|
|
||||||
| `narratio.transcript.polished` | `transcripts/polished.json` | `polish` | `transcript_polished` |
|
|
||||||
| `narratio.transcript.final` | `transcripts/final.json` | `normalize` | `transcript_final` |
|
|
||||||
| `narratio.transcript.final_trimmed` | `transcripts/final.trimmed.json` | `trim` | `transcript_final_trimmed` |
|
|
||||||
| `narratio.bounds.session` | `artifacts/session_bounds.json` | `trim` | `session_bounds` |
|
|
||||||
|
|
||||||
## Source families
|
|
||||||
- built-in: `narratio.transcript.*`, `narratio.bounds.session`
|
|
||||||
- configured artifact: `narratio.artifact.<artifact_key>`
|
|
||||||
- canonical previous-session artifact: `narratio.previous_session.artifact.<artifact_key>`
|
|
||||||
|
|
||||||
## S3 key helpers
|
|
||||||
- session prefix: `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/`
|
|
||||||
- session config: `{session_prefix}/session.yml`
|
|
||||||
- session lock store: `{session_prefix}/locks.yml`
|
|
||||||
- run prefix: `{session_prefix}/runs/{run_id}/`
|
|
||||||
- audio prefix: `{session_prefix}/{session.inputs.audio_s3.prefix}`
|
|
||||||
- current manifest: `{session_prefix}/current/manifest.json`
|
|
||||||
- current run pointer: `{session_prefix}/current/run_id.txt`
|
|
||||||
|
|
||||||
## Runtime catalog model
|
|
||||||
Catalog entries track:
|
|
||||||
- `planned`: source is registered for this run;
|
|
||||||
- `executable`: configured artifact is selected for analyze execution;
|
|
||||||
- `available`: usable local file exists (generated this run or reused from disk).
|
|
||||||
|
|
||||||
Configured artifact provenance values include:
|
|
||||||
- `generated.current_analyze_run`
|
- `generated.current_analyze_run`
|
||||||
- `filesystem.disabled_artifact_output`
|
- `filesystem.disabled_artifact_output`
|
||||||
|
|
||||||
Previous-session canonical provenance values include:
|
|
||||||
- `manifest.inputs.previous_cache`
|
- `manifest.inputs.previous_cache`
|
||||||
- `current_session.previous_cache`
|
- `current_session.previous_cache`
|
||||||
|
|
||||||
## Resolution behavior
|
## Resolution Rules
|
||||||
- Built-ins resolve via manifest producer outputs first, then canonical fallback paths.
|
Built-ins:
|
||||||
- Configured `narratio.artifact.<name>` sources resolve through catalog availability.
|
1. manifest producer outputs (when present)
|
||||||
- Canonical previous-session sources resolve to current-session `previous/` cache candidates derived from configured artifact canonical output paths.
|
2. canonical session path fallback
|
||||||
- Publish-relative configured artifact paths under `artifacts/` are cached without a redundant nested `artifacts/` segment.
|
|
||||||
- Previous-session canonical resolution prefers manifest-recorded input paths when present, then filesystem fallback under `previous/artifacts/**`.
|
|
||||||
|
|
||||||
## Previous-session requirement scanning
|
Configured sources (`narratio.artifact.*`):
|
||||||
|
- resolve only through runtime catalog availability.
|
||||||
|
|
||||||
|
Previous-session sources (`narratio.previous_session.artifact.*`):
|
||||||
|
- resolve only from local `previous/` cache state.
|
||||||
|
- prefer manifest-backed previous input paths.
|
||||||
|
- fallback to existing previous-cache filesystem paths.
|
||||||
|
|
||||||
|
Validation by content type:
|
||||||
|
- transcript built-ins: JSON with top-level `segments` array.
|
||||||
|
- bounds built-in: valid JSON.
|
||||||
|
- configured/previous-session artifact files: non-empty text file.
|
||||||
|
|
||||||
|
## Previous Requirement Collection
|
||||||
`CollectPreviousArtifactRequirements`:
|
`CollectPreviousArtifactRequirements`:
|
||||||
- scans enabled configured artifacts only;
|
- scans enabled configured artifacts only;
|
||||||
- includes canonical previous-session sources only;
|
- extracts only canonical previous-session sources;
|
||||||
- deduplicates by artifact key;
|
- deduplicates by artifact key;
|
||||||
- merges required/optional references (`required` wins);
|
- merges required/optional (required wins);
|
||||||
- records deterministic sorted source locations for diagnostics.
|
- returns deterministic ordering and source locations.
|
||||||
|
|
||||||
## Validation behavior
|
## Key Path Helpers
|
||||||
- transcript built-ins: JSON with top-level `segments` array;
|
`internal/artifacts/paths.go` defines canonical helpers for:
|
||||||
- bounds built-in: valid JSON;
|
- session/work/run paths;
|
||||||
- configured and previous-session artifact files: non-empty text content.
|
- previous-cache paths;
|
||||||
|
- spool/cache paths;
|
||||||
|
- S3 key layout helpers for session/run/current pointers.
|
||||||
|
|
||||||
## Failure behavior
|
## Invariants
|
||||||
- unsupported source or malformed canonical previous source: validation/resolution error;
|
- Source ID formats are stable contracts.
|
||||||
- known source unavailable: `ErrSessionArtifactNotFound`;
|
- Resolution is deterministic and manifest-aware.
|
||||||
- configured/previous canonical source without catalog: error;
|
- Previous-session source resolution does not call remote storage in `analyze`; remote hydration is `prepare` responsibility.
|
||||||
- resolved invalid file content: validation error.
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
|
||||||
- `internal/artifacts/artifact_resolver_test.go`
|
|
||||||
- `internal/artifacts/catalog_test.go`
|
|
||||||
- `internal/artifacts/previous_requirements_test.go`
|
|
||||||
- `internal/stage/prepare_previous_test.go`
|
|
||||||
- `internal/stage/analyze_test.go`
|
|
||||||
|
|
||||||
## Architectural invariants
|
|
||||||
- Built-in source IDs are static.
|
|
||||||
- Configured and previous-session source IDs are artifact-key based and validation-gated.
|
|
||||||
- Resolution behavior remains deterministic and manifest-aware.
|
|
||||||
|
|||||||
@@ -1,106 +1,65 @@
|
|||||||
# Internal: Command Restore
|
# Internal: Command Restore
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define the implemented `narratio session restore` contract: committed remote-state discovery, deterministic plan classification, safe file install semantics, and restore reporting.
|
Document the implemented `narratio session restore` command contract:
|
||||||
|
- committed remote current-state discovery;
|
||||||
|
- deterministic restore plan classification;
|
||||||
|
- safe local install semantics;
|
||||||
|
- durable restore reporting.
|
||||||
|
|
||||||
## Inputs and outputs
|
## Discovery Contract
|
||||||
Inputs:
|
Restore discovers remote committed state using:
|
||||||
- CLI syntax: `narratio session restore <session_id>`.
|
- `current/run_id.txt` (required, non-empty)
|
||||||
- CLI flags: `--config`, `--campaign`, `--campaign-file`, `--session`, `--previous-session-id`, `--dry-run`, `--force`, `--include-audio`.
|
- `current/manifest.json` (required, decodable)
|
||||||
- Resolved/validated `pipeline.yml` and `session.yml`.
|
|
||||||
- Configured remote object store.
|
|
||||||
- Remote committed current-state markers (`current/run_id.txt`, `current/manifest.json`).
|
|
||||||
|
|
||||||
Outputs:
|
Discovered manifest identity must match requested `session_id` and `campaign`.
|
||||||
- Dry-run summary to stdout (plan + counts).
|
|
||||||
- Non-dry-run completion summary to stdout.
|
|
||||||
- Local durable session files restored under canonical session root.
|
|
||||||
- Non-dry-run restore report at `reports/restore-latest.json`.
|
|
||||||
|
|
||||||
## Boundaries
|
## Plan Contract
|
||||||
Owns:
|
Planner actions:
|
||||||
- Restore command flag parsing and command wiring.
|
- `download`
|
||||||
- Remote current-state discovery and identity validation.
|
- `skip_same`
|
||||||
- Restore plan construction and conflict classification.
|
- `conflict`
|
||||||
- Restore execution for planned downloads.
|
|
||||||
- Restore report model and persistence.
|
|
||||||
|
|
||||||
Does not own:
|
Plan behavior:
|
||||||
- Stage execution orchestration (`run`, `resume`, `run-stage`).
|
- remote list scope is the resolved session prefix;
|
||||||
- Publish-stage behavior.
|
- mapping to local paths is traversal-safe;
|
||||||
- Storage transport implementation details (owned by storage adapters).
|
- actions are sorted deterministically by local relative path.
|
||||||
|
|
||||||
## Config fields used
|
Restore scope from current remote state:
|
||||||
- Config/session discovery and templating fields consumed by all commands.
|
- include `manifest.json`
|
||||||
- `pipeline.workspace.root` (local restore target root).
|
- include `transcripts/**`
|
||||||
- `pipeline.storage.*` (remote backend + publish identity derivation).
|
- include `artifacts/**`
|
||||||
- `pipeline.storage.s3.*` identity components used by session-prefix helpers.
|
- include `audio/**` only with `--include-audio`
|
||||||
- `pipeline.spool.root` for active audio downloads.
|
|
||||||
- `pipeline.cache.root` and `pipeline.cache.s3_audio` for reusable S3 audio cache.
|
|
||||||
- `session.session_id`
|
|
||||||
- `session.campaign`
|
|
||||||
|
|
||||||
## External adapters used
|
Explicit exclusions from current remote state mapping:
|
||||||
- `storage.ObjectStore` for `Exists`, `List`, `Download`.
|
- `current/**`
|
||||||
- `artifacts.Store` (`LocalStore`) for layout and session lock management.
|
- `runs/**`
|
||||||
- `manifest.LocalStore` for manifest decode/validation and identity checks.
|
- `logs/**`
|
||||||
|
- `reports/**`
|
||||||
|
- `config/**`
|
||||||
|
- `inputs/**`
|
||||||
|
- `previous/**`
|
||||||
|
|
||||||
## State and manifest behavior
|
Previous-cache restore files are planned separately through `previouscache.BuildPlan` when configured previous-session requirements exist.
|
||||||
- Restore is not a pipeline run and does not create a run manifest.
|
|
||||||
- Restore uses committed remote current state only:
|
|
||||||
- `current/run_id.txt` must exist and be non-empty.
|
|
||||||
- `current/manifest.json` must decode and match requested session/campaign.
|
|
||||||
- Non-dry-run writes restore files to canonical session paths.
|
|
||||||
- With `--include-audio`, restore uses the shared S3 audio cache for `audio/**` objects. Cache hits avoid object downloads; cache misses download through spool, install the work file, and populate cache.
|
|
||||||
- Manifest install behavior:
|
|
||||||
- validated before replacement.
|
|
||||||
- installed last among download actions.
|
|
||||||
- existing local manifest is preserved if restored manifest validation/install fails.
|
|
||||||
- Non-dry-run report persists summary/action status metadata in `reports/restore-latest.json`.
|
|
||||||
|
|
||||||
Restore path scope:
|
## Execution Contract
|
||||||
- includes:
|
- non-manifest downloads happen before manifest install;
|
||||||
- `manifest.json`
|
- `manifest.json` is installed last;
|
||||||
- `transcripts/**`
|
- downloads use sibling temp files + atomic rename;
|
||||||
- `artifacts/**`
|
- manifest replacement is validated before rename;
|
||||||
- `previous/**`
|
- failed installs do not roll back previously written files.
|
||||||
- `audio/**` only when `--include-audio` is set
|
|
||||||
- excludes:
|
|
||||||
- `runs/**`
|
|
||||||
- `logs/**`
|
|
||||||
- `reports/**`
|
|
||||||
- `config/**`
|
|
||||||
- `inputs/**`
|
|
||||||
- remote `current/**` pointer files as local restore targets
|
|
||||||
|
|
||||||
## Skip and resume behavior
|
Audio restore path:
|
||||||
- Restore does not participate in stage skip/resume decisions.
|
- uses `audio.MaterializeS3Audio`;
|
||||||
- Restore provides durable local state so subsequent stage commands can resume or rerun based on restored manifest state.
|
- integrates spool and S3 audio cache paths;
|
||||||
- Audio cache is outside the workspace and is reused across restore and prepare invocations.
|
- supports cache hit reuse without object redownload.
|
||||||
- Dry-run is read-only and returns plan output only.
|
|
||||||
|
|
||||||
## Failure behavior
|
## Reporting Contract
|
||||||
- Fails when storage backend is unavailable or publish identity cannot be resolved.
|
- dry-run: summary only (no writes).
|
||||||
- Fails when remote current pointer/manifest is missing or invalid.
|
- non-dry-run: writes `reports/restore-latest.json`.
|
||||||
- Fails when remote manifest identity mismatches requested campaign/session.
|
- report captures plan counts, action status, and execution failures.
|
||||||
- Fails on local conflicts unless `--force` is set.
|
|
||||||
- Fails fast on session lock acquisition conflict for non-dry-run execution.
|
|
||||||
- On execution failure, previously installed files remain; no rollback is performed.
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
## Invariants
|
||||||
- `internal/app/restore_test.go`
|
- restore uses only committed remote current state as authority.
|
||||||
- `internal/app/restore_discovery_test.go`
|
- `current/run_id.txt` is the remote commit marker.
|
||||||
- `internal/app/restore_plan_test.go`
|
- restore is a standalone command and does not run stages.
|
||||||
- `internal/app/restore_execution_test.go`
|
|
||||||
- `internal/app/restore_workflow_test.go`
|
|
||||||
- `internal/artifacts/archive_identity_test.go`
|
|
||||||
|
|
||||||
## Architectural invariants
|
|
||||||
- Restore relies on centralized path/key helpers (`internal/artifacts`) rather than ad hoc key building.
|
|
||||||
- `current/run_id.txt` is the remote commit marker; restore must not infer committed state from incidental files.
|
|
||||||
- Local path mapping is traversal-safe and constrained to session root.
|
|
||||||
- Restore scope is deterministic and path-classified:
|
|
||||||
- include `manifest.json`, `transcripts/**`, `artifacts/**`, `previous/**`
|
|
||||||
- include `audio/**` only with `--include-audio`
|
|
||||||
- exclude `runs/**`, `logs/**`, `reports/**`, `config/**`, `inputs/**`
|
|
||||||
- Command remains standalone; no implicit `run --restore` behavior.
|
|
||||||
|
|||||||
@@ -1,81 +1,57 @@
|
|||||||
# Internal: Manifest
|
# Internal: Manifest
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Describe Narratio's durable execution state model for session-level and run-level manifests, including lifecycle transitions and persistence behavior.
|
Define durable session state (`manifest.json`) and invocation state (`runs/{run_id}/manifest.json`) contracts.
|
||||||
|
|
||||||
## Inputs and outputs
|
## Session Manifest
|
||||||
Inputs:
|
Path:
|
||||||
- Session identity and run identity from app orchestration.
|
- `{workspace.root}/work/{campaign}/{session_id}/manifest.json`
|
||||||
- Stage transition events and stage result payloads.
|
|
||||||
|
|
||||||
Outputs:
|
Primary model (`manifest.Manifest`):
|
||||||
- Session manifest at `{workspace.root}/work/{campaign}/{session_id}/manifest.json`.
|
- identity (`session_id`, `campaign`, `run_id`)
|
||||||
- Run manifest at `{workspace.root}/work/{campaign}/{session_id}/runs/{run_id}/manifest.json`.
|
- local path metadata (`local_workdir`, `local_spool_dir`)
|
||||||
|
- remote identity metadata (`s3_bucket`, `s3_session_prefix`, `s3_run_prefix`)
|
||||||
|
- `inputs` records
|
||||||
|
- durable `artifacts` records
|
||||||
|
- per-stage `stages` map
|
||||||
|
|
||||||
## Boundaries
|
Stage status enum:
|
||||||
Owns:
|
- `pending`
|
||||||
- Manifest schemas (`Manifest`, `RunManifest`, stage records, error records, input/artifact records).
|
- `running`
|
||||||
- Stage status/action transition methods.
|
- `succeeded`
|
||||||
- Persistent store contract (`manifest.Store`) and local JSON store implementation.
|
- `failed`
|
||||||
|
- `skipped`
|
||||||
|
- `stale`
|
||||||
|
- `interrupted`
|
||||||
|
|
||||||
Does not own:
|
## Run Manifest
|
||||||
- Stage implementation details.
|
Path:
|
||||||
- Path construction policy outside manifest file persistence calls.
|
- `{workspace.root}/work/{campaign}/{session_id}/runs/{run_id}/manifest.json`
|
||||||
- CLI command behavior.
|
|
||||||
|
|
||||||
## Config fields used
|
Run model (`manifest.RunManifest`):
|
||||||
Manifest package itself does not read config directly.
|
- invocation identity and `force` flag
|
||||||
|
- requested stages
|
||||||
|
- per-stage action (`run` or `skip`)
|
||||||
|
- per-stage status
|
||||||
|
- overall run status (`running`, `succeeded`, `failed`)
|
||||||
|
|
||||||
Manifest identity fields are populated by app/stage orchestration from:
|
## Persistence Semantics
|
||||||
- `session.session_id`
|
`manifest.LocalStore`:
|
||||||
- `session.campaign`
|
- validates loaded documents;
|
||||||
- `pipeline.workspace.root`
|
- normalizes missing maps/stage records;
|
||||||
- `pipeline.storage.s3.*` (when publish/S3 identity is set)
|
- writes atomically via temp file + rename;
|
||||||
|
- updates `updated_at` on save.
|
||||||
|
|
||||||
## External adapters used
|
## Execution Semantics
|
||||||
- No external service adapters.
|
Runner updates both manifests per stage transition:
|
||||||
- Uses local filesystem for persistence via `manifest.LocalStore`.
|
- mark running
|
||||||
|
- mark succeeded/failed/skipped
|
||||||
|
- persist logs/generated config refs and metadata
|
||||||
|
|
||||||
## State and manifest behavior
|
Session manifest is the authoritative stage-progress ledger across invocations.
|
||||||
Session manifest model:
|
Run manifest is invocation-scoped audit state.
|
||||||
- Tracks durable per-session stage state and provenance (`pending`, `running`, `succeeded`, `failed`, `skipped`, `stale`, `interrupted`).
|
|
||||||
- Stores resolved inputs, durable artifacts, stage logs/config refs, and stage metadata.
|
|
||||||
|
|
||||||
Run manifest model:
|
## Invariants
|
||||||
- Tracks one invocation (`run_id`) with requested stages and force mode.
|
- stage resume/skip decisions are session-manifest driven.
|
||||||
- Tracks per-stage action (`run` or `skip`) and per-stage status.
|
- force reruns stale downstream succeeded stages.
|
||||||
- Tracks overall run status (`running`, `succeeded`, `failed`).
|
- run manifest does not replace session manifest as progress authority.
|
||||||
|
|
||||||
Persistence behavior:
|
|
||||||
- Load validates required identity/timestamp fields and normalizes maps/records.
|
|
||||||
- Save updates `updated_at` and writes JSON atomically (temp file + rename).
|
|
||||||
- Session and run manifests are saved incrementally before/after stage transitions.
|
|
||||||
|
|
||||||
Relationship during execution:
|
|
||||||
- Runner updates both manifests for every stage transition.
|
|
||||||
- Session manifest is the durable pipeline-progress ledger.
|
|
||||||
- Run manifest is invocation history and audit record.
|
|
||||||
- Analyze stage outputs are persisted as `kind=scriptorium_artifact` with `source_id=narratio.artifact.<name>` for configured artifact identity.
|
|
||||||
|
|
||||||
## Skip and resume behavior
|
|
||||||
- Resume and skip decisions are based on session-manifest stage statuses.
|
|
||||||
- `--force` reruns selected stages and marks downstream succeeded stages as `stale` in session manifest.
|
|
||||||
- Run manifest records whether each stage was executed or skipped in that invocation.
|
|
||||||
|
|
||||||
## Failure behavior
|
|
||||||
- Stage failure marks both manifests failed for that stage and records error messages/timestamps.
|
|
||||||
- Save failures are returned immediately and fail the command.
|
|
||||||
- Invalid/malformed manifest files fail load with explicit validation/decode errors.
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
|
||||||
- `internal/manifest/manifest_test.go`
|
|
||||||
- `internal/manifest/run_manifest_test.go`
|
|
||||||
- `internal/manifest/store_test.go`
|
|
||||||
- `internal/app/runner_test.go`
|
|
||||||
- `internal/app/run_control_test.go`
|
|
||||||
- `internal/app/resume_run_stage_test.go`
|
|
||||||
|
|
||||||
## Architectural invariants
|
|
||||||
- Session manifest is authoritative for stage progression across invocations.
|
|
||||||
- Run manifest is invocation-scoped and never replaces session manifest as progress authority.
|
|
||||||
- Manifest writes are atomic and deterministic (JSON + newline, temp rename pattern).
|
|
||||||
|
|||||||
@@ -1,80 +1,38 @@
|
|||||||
# Stage: analyze
|
# Stage: analyze
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Execute selected configured Scriptorium artifacts in deterministic dependency order and materialize successful outputs to canonical session artifact paths.
|
Execute selected configured Scriptorium artifacts in dependency order and materialize outputs.
|
||||||
|
|
||||||
## Inputs and outputs
|
## Inputs
|
||||||
Inputs:
|
- configured artifacts from `pipeline.scriptorium.artifacts`
|
||||||
- configured artifact definitions from `pipeline.scriptorium.artifacts`;
|
- optional selected artifact filter (`--artifacts`)
|
||||||
- selected artifact filter (`--artifacts`) when provided;
|
- built-in/configured/previous-session source references in artifact inputs
|
||||||
- resolved artifact sources from resolver/catalog.
|
|
||||||
|
|
||||||
Source types used by analyze:
|
Supported source families:
|
||||||
- built-ins: `narratio.transcript.*`, `narratio.bounds.session`;
|
- built-ins: `narratio.transcript.*`, `narratio.bounds.session`
|
||||||
- configured artifacts: `narratio.artifact.<artifact_key>`;
|
- configured artifacts: `narratio.artifact.<key>`
|
||||||
- canonical previous-session artifacts: `narratio.previous_session.artifact.<artifact_key>`.
|
- previous-session cache: `narratio.previous_session.artifact.<key>`
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- materialized configured artifact files at each configured `output_path`;
|
- one materialized output per executed configured artifact (`output_path`)
|
||||||
- stage metadata (`generated_artifacts`, `reused_artifacts`, selected/order info).
|
- stage metadata describing selected/generated/reused artifacts
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- skips with metadata when Scriptorium config is missing or no executable artifacts remain.
|
||||||
- runtime artifact catalog construction;
|
- builds runtime artifact catalog (built-ins + configured artifacts).
|
||||||
- selected-artifact planning and dependency ordering;
|
- marks non-executable configured artifacts as reusable when output files already exist.
|
||||||
- per-input resolution and required/optional handling;
|
- validates selected artifact dependency order (cycle-safe topo ordering).
|
||||||
- Scriptorium render/run invocation;
|
- resolves required/optional inputs per artifact source definition.
|
||||||
- run-local output generation and canonical materialization.
|
- resolves previous-session sources from local `previous/` cache only.
|
||||||
|
- runs optional render-debug, then artifact execution.
|
||||||
|
- validates non-empty output files and materializes canonical outputs.
|
||||||
|
|
||||||
Does not own:
|
## Failure Semantics
|
||||||
- prepare-time previous-session hydration;
|
- required missing configured/previous-session inputs fail.
|
||||||
- object-store access for previous-session sources;
|
- missing required previous-session source includes prepare rerun guidance.
|
||||||
- publish output rule behavior.
|
- dependency cycles or unavailable required dependencies fail.
|
||||||
|
- adapter validation failures fail stage.
|
||||||
|
|
||||||
## Config fields used
|
## Invariants
|
||||||
- `session.session_id`
|
- `analyze` performs no remote storage calls for previous-session source resolution.
|
||||||
- `session.campaign`
|
- output provenance and metadata are deterministic per execution.
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `pipeline.scriptorium.binary`
|
|
||||||
- `pipeline.scriptorium.config_path`
|
|
||||||
- `pipeline.scriptorium.timeout`
|
|
||||||
- `pipeline.scriptorium.render_debug`
|
|
||||||
- `pipeline.scriptorium.artifacts.<name>.*`
|
|
||||||
|
|
||||||
## External adapters used
|
|
||||||
- Scriptorium adapter:
|
|
||||||
- optional `RenderArtifact` when render-debug is enabled;
|
|
||||||
- `RunArtifact` for artifact generation.
|
|
||||||
|
|
||||||
## State and manifest behavior
|
|
||||||
- If Scriptorium config is absent, or no artifacts are executable after filtering, analyze returns success metadata with `skipped=true`.
|
|
||||||
- Builds runtime catalog with built-ins and configured `narratio.artifact.<name>` entries.
|
|
||||||
- Non-executable configured artifacts may still be marked available from existing canonical output files.
|
|
||||||
- Resolves canonical previous-session sources from local prepared `previous/` cache:
|
|
||||||
- prefers manifest-backed previous input paths when present;
|
|
||||||
- may fall back to current-session `previous/` filesystem paths.
|
|
||||||
- Analyze does not call object storage for canonical previous-session source resolution.
|
|
||||||
- Required canonical previous-session input missing:
|
|
||||||
- fails with guidance to run `narratio run-stage --force prepare`.
|
|
||||||
- Optional missing sources are omitted from adapter input paths.
|
|
||||||
|
|
||||||
## Skip and resume behavior
|
|
||||||
- Runner-level skip applies when analyze is already `succeeded` and `--force` is not set.
|
|
||||||
- Analyze is stage-scoped for resume; no per-artifact manifest resume state.
|
|
||||||
- `--artifacts` filters executable artifacts but does not imply force rerun.
|
|
||||||
|
|
||||||
## Failure behavior
|
|
||||||
- Fails on dependency-order violations, missing required inputs, resolver validation failures, adapter errors, and missing/empty generated outputs.
|
|
||||||
- Required unavailable configured artifact source (`narratio.artifact.<name>`) fails before invocation.
|
|
||||||
- Required canonical previous-session source fails with prepare-rerun guidance.
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
|
||||||
- `internal/stage/analyze_test.go`
|
|
||||||
- `internal/artifacts/catalog_test.go`
|
|
||||||
- `internal/artifacts/artifact_resolver_test.go`
|
|
||||||
- `internal/app/restore_workflow_test.go`
|
|
||||||
|
|
||||||
## Architectural invariants
|
|
||||||
- Canonical previous-session behavior is local-cache only during analyze.
|
|
||||||
- Generated outputs are validated and materialized before stage success is recorded.
|
|
||||||
- Resolver/catalog decisions stay deterministic and validation-gated.
|
|
||||||
|
|||||||
@@ -1,63 +1,25 @@
|
|||||||
# Stage: merge
|
# Stage: merge
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Normalize per-speaker raw transcripts and merge them into the base transcript via Seriatim.
|
Normalize raw transcript inputs and merge into base transcript via Seriatim.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
|
||||||
- `transcripts/raw/*.json`
|
- `transcripts/raw/*.json`
|
||||||
- `inputs/speakers.yml`
|
- `inputs/speakers.yml`
|
||||||
- `inputs/autocorrect.yml`
|
- `inputs/autocorrect.yml`
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- `transcripts/base.json`
|
- `transcripts/base.json`
|
||||||
- optional `artifacts/seriatim.report.json` (when report enabled)
|
- optional `artifacts/seriatim.report.json`
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- discovers and validates raw transcript inputs.
|
||||||
- Raw transcript discovery/validation
|
- normalizes each raw transcript (`seriatim.Normalize`) into run-local scratch output.
|
||||||
- Per-input normalize calls to Seriatim
|
- merges normalized inputs (`seriatim.Run`) into base transcript.
|
||||||
- Final merge call to Seriatim
|
- validates merged transcript and optional report JSON.
|
||||||
- Run-local log/config/report path wiring
|
- materializes canonical outputs and records stage logs/generated configs.
|
||||||
- Materialization of base/report outputs to canonical paths
|
|
||||||
|
|
||||||
Does not own:
|
## Invariants
|
||||||
- Transcript polishing or downstream artifact generation
|
- merge always consumes normalized forms of raw inputs.
|
||||||
|
- base transcript must validate before stage success.
|
||||||
## Config Fields Used
|
- report output is config-gated.
|
||||||
- `session.session_id`
|
|
||||||
- `session.campaign`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `pipeline.seriatim.binary`
|
|
||||||
- `pipeline.seriatim.timeout`
|
|
||||||
- `pipeline.seriatim.output_schema`
|
|
||||||
- `pipeline.seriatim.coalesce_gap`
|
|
||||||
- `pipeline.seriatim.report`
|
|
||||||
- `pipeline.seriatim.env.*`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- Seriatim adapter:
|
|
||||||
- `Normalize` for each raw input
|
|
||||||
- `Run` for final merge
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Reads transcript inputs from transcribe stage outputs in manifest when present; falls back to canonical raw directory.
|
|
||||||
- Writes run-local outputs/logs/config under `runs/{run_id}/merge/...` when enabled.
|
|
||||||
- Materializes canonical base transcript and optional report.
|
|
||||||
- Records normalized-input provenance and adapter metadata in stage metadata.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies when already succeeded and not forced.
|
|
||||||
- Forced rerun of this or upstream stages can stale downstream succeeded stages via runner invalidation.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on missing/invalid raw transcripts, missing speakers/autocorrect files, normalize failure, merge failure, invalid base output JSON, or invalid report JSON when enabled.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/merge_test.go`
|
|
||||||
- `internal/adapters/seriatim/subprocess_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Merge consumes normalized forms of each raw transcript.
|
|
||||||
- Base transcript must validate before materialization.
|
|
||||||
- Report output is optional and gated by config.
|
|
||||||
|
|||||||
@@ -1,56 +1,22 @@
|
|||||||
# Stage: normalize
|
# Stage: normalize
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Normalize the polished transcript into the full final transcript and optionally emit a normalize report.
|
Normalize polished transcript into final transcript using Seriatim.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
|
||||||
- `transcripts/polished.json`
|
- `transcripts/polished.json`
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- `transcripts/final.json` (or configured normalize output path)
|
- `transcripts/final.json` (or configured normalize output path)
|
||||||
- optional `artifacts/seriatim.normalize.report.json`
|
- optional `artifacts/seriatim.normalize.report.json`
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- resolves polished transcript from manifest outputs/canonical fallback.
|
||||||
- Polished transcript discovery/validation
|
- applies `pipeline.normalize` config or default normalize config.
|
||||||
- Normalize request construction and invocation
|
- runs Seriatim normalize with configured timeout/binary.
|
||||||
- Optional normalize report wiring
|
- validates normalized transcript and optional report.
|
||||||
- Promotion of final transcript and optional report
|
- materializes canonical outputs and records logs/generated configs.
|
||||||
|
|
||||||
Does not own:
|
## Invariants
|
||||||
- Bounds detection or segment trimming
|
- final transcript must validate as processed transcript JSON (`segments` array).
|
||||||
|
- normalize defaults are applied when `pipeline.normalize` is unset.
|
||||||
## Config Fields Used
|
|
||||||
- `session.session_id`
|
|
||||||
- `session.campaign`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `pipeline.normalize.output_path`
|
|
||||||
- `pipeline.normalize.output_schema`
|
|
||||||
- `pipeline.normalize.report`
|
|
||||||
- `pipeline.seriatim.binary`
|
|
||||||
- `pipeline.seriatim.timeout`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- Seriatim adapter (`Normalize`).
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Reads polished transcript from polish outputs in manifest when present; falls back to canonical path.
|
|
||||||
- Uses run-local output/report/log/config paths when run layout is enabled.
|
|
||||||
- Promotes canonical final transcript and optional normalize report.
|
|
||||||
- Records adapter/result metadata including source path selection.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies when already succeeded and not forced.
|
|
||||||
- Forced reruns can stale downstream succeeded stages.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on missing/invalid polished transcript, adapter error, invalid final output, or invalid report output when report enabled.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/normalize_test.go`
|
|
||||||
- `internal/adapters/seriatim/subprocess_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Final output must validate as transcript-compatible JSON (`segments` array required).
|
|
||||||
- Default normalize config is applied when `pipeline.normalize` is unset.
|
|
||||||
|
|||||||
@@ -1,69 +1,23 @@
|
|||||||
# Stage: polish
|
# Stage: polish
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Polish the base transcript with Audita and produce a polished transcript for downstream normalization/analyze.
|
Run Audita polishing on base transcript and produce polished transcript.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
|
||||||
- `transcripts/base.json`
|
- `transcripts/base.json`
|
||||||
- `inputs/glossary.yml`
|
- `inputs/glossary.yml`
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- `transcripts/polished.json`
|
- `transcripts/polished.json`
|
||||||
- optional `artifacts/audita.report.json` (when report enabled)
|
- optional `artifacts/audita.report.json`
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- resolves base transcript from merge outputs/canonical fallback.
|
||||||
- Base transcript discovery/validation
|
- invokes Audita with configured model/module/runtime options.
|
||||||
- Audita invocation request construction
|
- validates processed transcript structure (`segments` array required).
|
||||||
- Run-local logs/config/work-dir/report wiring
|
- validates optional report JSON.
|
||||||
- Promotion of polished transcript and optional report
|
- materializes canonical outputs; records logs/generated config and adapter metadata.
|
||||||
|
|
||||||
Does not own:
|
## Invariants
|
||||||
- Upstream merge normalization
|
- polished transcript schema validation is mandatory.
|
||||||
- Downstream normalize/trim/analyze logic
|
- report output is config-gated.
|
||||||
|
|
||||||
## Config Fields Used
|
|
||||||
- `session.session_id`
|
|
||||||
- `session.campaign`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `pipeline.audita.binary`
|
|
||||||
- `pipeline.audita.timeout`
|
|
||||||
- `pipeline.audita.llm_api_key_env`
|
|
||||||
- `pipeline.audita.modules`
|
|
||||||
- `pipeline.audita.base_url`
|
|
||||||
- `pipeline.audita.model`
|
|
||||||
- `pipeline.audita.transcript_description`
|
|
||||||
- `pipeline.audita.config_path`
|
|
||||||
- `pipeline.audita.output_schema`
|
|
||||||
- `pipeline.audita.work_dir_retention`
|
|
||||||
- `pipeline.audita.total_llm_concurrency`
|
|
||||||
- `pipeline.audita.proposal_llm_concurrency`
|
|
||||||
- `pipeline.audita.validation_model`
|
|
||||||
- `pipeline.audita.validation_llm_concurrency`
|
|
||||||
- `pipeline.audita.report`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- Audita adapter (`env.Audita.Run`).
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Reads base transcript from merge manifest outputs when available; falls back to canonical base path.
|
|
||||||
- Uses run-local output/report/log/config/scratch paths when run layout is enabled.
|
|
||||||
- Promotes canonical `transcripts/polished.json` and optional report.
|
|
||||||
- Records adapter invocation metadata, credential presence signal, and output provenance in stage metadata.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies when already succeeded and not forced.
|
|
||||||
- Forced rerun can stale downstream succeeded stages via runner invalidation.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on missing/invalid base transcript, missing glossary, adapter error, invalid polished output shape (`segments` array required), or invalid report JSON when enabled.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/polish_test.go`
|
|
||||||
- `internal/adapters/audita/subprocess_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Polished transcript must contain a top-level `segments` array.
|
|
||||||
- Report behavior is strictly config-gated.
|
|
||||||
- Stage output canonicalization always ends at `transcripts/polished.json`.
|
|
||||||
|
|||||||
@@ -1,124 +1,42 @@
|
|||||||
# Stage: prepare
|
# Stage: prepare
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Materialize canonical current-session input state and provenance before downstream stages run.
|
Materialize canonical current-session inputs before processing stages.
|
||||||
|
|
||||||
Prepare owns:
|
## Inputs
|
||||||
- local input file materialization (`inputs/**`);
|
- resolved `campaign.yml`, `session.yml`, and pipeline config
|
||||||
- audio input materialization (`audio/**`);
|
- stable input files (`speakers`, `autocorrect`, `glossary`)
|
||||||
- previous-session cache hydration (`previous/**`) for canonical previous-session artifact sources.
|
|
||||||
|
|
||||||
## Inputs and outputs
|
|
||||||
Inputs:
|
|
||||||
- resolved config/campaign/session (`pipeline.yml`, `campaign.yml`, `session.yml`);
|
|
||||||
- remote session provenance when `session.yml` was loaded from S3;
|
|
||||||
- campaign or session input files (`speakers`, `autocorrect`, `glossary`);
|
|
||||||
- audio source:
|
- audio source:
|
||||||
- local: `session.inputs.audio_dir` or `session.inputs.audio_files`;
|
- local `audio_dir`/`audio_files`, or
|
||||||
- S3: `session.inputs.audio_s3.prefix`;
|
- S3 `audio_s3.prefix`
|
||||||
- configured enabled Scriptorium artifact inputs (for previous-session requirement scanning);
|
- enabled configured artifact input requirements for previous-session sources
|
||||||
- remote previous-session current publish state when previous hydration is required.
|
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- `inputs/campaign.yml`;
|
- `inputs/campaign.yml`
|
||||||
- `inputs/session.yml`;
|
- `inputs/session.yml`
|
||||||
- `inputs/pipeline.resolved.yml`;
|
- `inputs/pipeline.resolved.yml`
|
||||||
- `inputs/speakers.yml`;
|
- `inputs/speakers.yml`
|
||||||
- `inputs/autocorrect.yml`;
|
- `inputs/autocorrect.yml`
|
||||||
- `inputs/glossary.yml`;
|
- `inputs/glossary.yml`
|
||||||
- `audio/*.flac` in canonical session `audio/`;
|
- `audio/*.flac`
|
||||||
- optional `previous/manifest.json`;
|
- optional `previous/manifest.json`
|
||||||
- optional `previous/artifacts/**`;
|
- optional `previous/artifacts/**`
|
||||||
- deterministic `manifest.Inputs` records with checksums and provenance metadata.
|
- deterministic `manifest.inputs` entries (checksums + provenance)
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- validates required config/store state.
|
||||||
- input path resolution and materialization;
|
- enforces local audio vs S3 audio mutual exclusivity.
|
||||||
- S3 audio list/download/copy flow;
|
- materializes S3 audio through spool/cache-aware logic.
|
||||||
- previous-session artifact requirement collection from enabled configured artifacts;
|
- scans enabled configured artifact inputs for `narratio.previous_session.artifact.*` requirements.
|
||||||
- previous cache lifecycle when requirements exist (clear and rehydrate managed `previous/` state).
|
- when previous requirements exist:
|
||||||
|
|
||||||
Does not own:
|
|
||||||
- transcript or artifact generation;
|
|
||||||
- analyze-stage source resolution;
|
|
||||||
- publish commit behavior.
|
|
||||||
|
|
||||||
## Config fields used
|
|
||||||
- `session.session_id`
|
|
||||||
- `session.previous_session_id`
|
|
||||||
- `session.campaign`
|
|
||||||
- `session.inputs.speakers_file`
|
|
||||||
- `session.inputs.autocorrect_file`
|
|
||||||
- `session.inputs.glossary_file`
|
|
||||||
- `session.inputs.audio_dir`
|
|
||||||
- `session.inputs.audio_files`
|
|
||||||
- `session.inputs.audio_s3.prefix`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `pipeline.spool.root`
|
|
||||||
- `pipeline.cache.root`
|
|
||||||
- `pipeline.cache.s3_audio`
|
|
||||||
- `pipeline.storage.s3.bucket`
|
|
||||||
- `pipeline.storage.s3.root_prefix`
|
|
||||||
- `pipeline.scriptorium.artifacts.<name>.enabled`
|
|
||||||
- `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source`
|
|
||||||
- `pipeline.scriptorium.artifacts.<name>.inputs.<key>.required`
|
|
||||||
- `campaign.campaign_id`
|
|
||||||
- `campaign.inputs.speakers_file`
|
|
||||||
- `campaign.inputs.autocorrect_file`
|
|
||||||
- `campaign.inputs.glossary_file`
|
|
||||||
|
|
||||||
## External adapters used
|
|
||||||
- `storage.ObjectStore` for:
|
|
||||||
- S3 audio listing/downloads;
|
|
||||||
- previous-session current pointer/manifest/artifact object checks and downloads.
|
|
||||||
|
|
||||||
## State and manifest behavior
|
|
||||||
- Ensures workspace layout exists.
|
|
||||||
- Materializes canonical input files and audio files.
|
|
||||||
- For S3 audio, uses run-scoped spool for active downloads and durable cache for reusable audio files; cache hits copy directly to work audio without downloading the object again.
|
|
||||||
- Records `inputs/session.yml` provenance as local `session_config` or remote `session_config.s3`.
|
|
||||||
- Resolves campaign-provided stable input paths relative to `campaign.yml`.
|
|
||||||
- Resolves session-provided stable input overrides relative to `session.yml`.
|
|
||||||
- Scans enabled configured artifact inputs for canonical sources:
|
|
||||||
- `narratio.previous_session.artifact.<artifact_key>`
|
|
||||||
- If one or more canonical previous-session requirements exist:
|
|
||||||
- clears managed `previous/` state;
|
- clears managed `previous/` state;
|
||||||
- hydrates required/optional previous artifacts from the configured previous session’s committed publish current state;
|
- builds previous-cache remote plan;
|
||||||
- writes `previous/manifest.json` and hydrated `previous/artifacts/**`;
|
- downloads previous manifest/artifacts;
|
||||||
- stores publish-relative artifact paths such as `artifacts/session_recap.md` as `previous/artifacts/session_recap.md`, not `previous/artifacts/artifacts/session_recap.md`;
|
- records previous inputs in `manifest.inputs`.
|
||||||
- records hydrated previous inputs in `manifest.Inputs` with source `previous_session_publish.current`.
|
|
||||||
- If no canonical previous-session requirements exist, prepare does not manage `previous/`.
|
|
||||||
- `manifest.Inputs` is sorted deterministically by `(kind, path)`.
|
|
||||||
- S3 audio `manifest.Inputs` retain S3 provenance and include `cache_path`; `spool_path` is present only when the current prepare invocation downloaded the file.
|
|
||||||
|
|
||||||
## Required and optional previous-session behavior
|
Required previous-session inputs fail when unavailable; optional missing inputs are skipped.
|
||||||
- `previous_session_id` unset:
|
|
||||||
- if any referenced previous artifact is required: fail;
|
|
||||||
- if all referenced previous artifacts are optional: continue and omit them.
|
|
||||||
- Previous session publish current pointer or manifest missing:
|
|
||||||
- if any referenced previous artifact is required: fail;
|
|
||||||
- if all referenced previous artifacts are optional: continue and omit missing ones.
|
|
||||||
- Missing required previous artifact object: fail.
|
|
||||||
- Missing optional previous artifact object: omit.
|
|
||||||
- Downloaded previous artifacts must validate as non-empty files.
|
|
||||||
|
|
||||||
## Skip and resume behavior
|
## Invariants
|
||||||
- Runner-level skip remains authoritative:
|
- only `prepare` hydrates canonical `previous/` cache state.
|
||||||
- if `prepare` already succeeded and run is not forced, `prepare` does not run and no hydration/download occurs.
|
- managed previous artifacts are stored under `previous/artifacts/**` without duplicate `artifacts/artifacts/` nesting.
|
||||||
- If `prepare` runs (including with `--force`), it owns managed `previous/` state for canonical previous-session inputs.
|
- `manifest.inputs` ordering is deterministic (`kind`, `path`).
|
||||||
|
|
||||||
## Failure behavior
|
|
||||||
- Fails on missing required input files, invalid audio-source combinations, empty/duplicate audio inputs, missing object store for S3 modes, and remote access/download/validation errors.
|
|
||||||
- For required canonical previous-session inputs, analyze-time missing-input guidance is to rerun:
|
|
||||||
- `narratio run-stage --force prepare`
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
|
||||||
- `internal/stage/prepare_test.go`
|
|
||||||
- `internal/stage/prepare_previous_test.go`
|
|
||||||
- `internal/artifacts/previous_requirements_test.go`
|
|
||||||
- `internal/app/runner_test.go`
|
|
||||||
|
|
||||||
## Architectural invariants
|
|
||||||
- `audio_dir`/`audio_files` and `audio_s3` are mutually exclusive.
|
|
||||||
- Storage keys are computed by callers using path helpers; storage adapter receives explicit keys.
|
|
||||||
- `prepare` is the only stage that hydrates canonical previous-session cache state.
|
|
||||||
|
|||||||
@@ -1,88 +1,44 @@
|
|||||||
# Stage: publish
|
# Stage: publish
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Publish durable run/session state to object storage, then atomically advance remote current state.
|
Upload run/session outputs to object storage and atomically advance remote current state.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
- successful prerequisite stages: `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, `analyze`
|
||||||
- session manifest and prerequisite stage records
|
- run root `runs/{run_id}/**`
|
||||||
- run root contents under `runs/{run_id}/`
|
- publish output rules (`pipeline.publish.outputs`)
|
||||||
- publish output rules with artifact `source` IDs and publish `dest` paths (`pipeline.publish.outputs`)
|
- effective publish locks (static + remote merged lock set)
|
||||||
- effective source-based publish locks from static config and remote session lock store
|
- local `previous/**` files when present
|
||||||
- session-level `previous/**` cache files when present
|
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- uploaded run files under `{session_prefix}/runs/{run_id}/...`
|
- uploaded run files under remote `runs/{run_id}/...` (excluding `audio/**`)
|
||||||
- uploaded published outputs under `{session_prefix}/...`
|
- uploaded selected publish outputs under session prefix
|
||||||
- uploaded session previous-cache files under `{session_prefix}/previous/...` when present
|
- uploaded `previous/**` files under session prefix when present
|
||||||
- `{session_prefix}/current/manifest.json`
|
- uploaded `current/manifest.json`
|
||||||
- `{session_prefix}/current/run_id.txt` written last
|
- uploaded `current/run_id.txt` written last
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- stage can self-skip when publish disabled or run upload disabled.
|
||||||
- publish enable/disable gate behavior
|
- validates prerequisite stage success and object-store availability.
|
||||||
- prerequisite stage success enforcement
|
- collects deterministic run file list plus run `manifest.json`.
|
||||||
- run file collection and upload (excluding `audio/`)
|
- resolves publish output sources through runtime artifact catalog and manifest-aware resolution.
|
||||||
- publish output rule resolution and upload
|
- selected artifact filter applies to configured artifact sources only.
|
||||||
- publish lock enforcement
|
- locked outputs are skipped intentionally (including required ones).
|
||||||
- session previous-cache file collection/upload
|
- optional missing outputs are skipped; required missing unlocked outputs fail.
|
||||||
- commit pointer publish order
|
- writes remote current manifest before current run pointer.
|
||||||
|
|
||||||
Does not own:
|
## Metadata Signals
|
||||||
- stage execution before publish
|
Includes counts/lists for:
|
||||||
- post-publish local cleanup policy execution (handled by app cleanup logic)
|
- run uploads
|
||||||
|
- published output uploads
|
||||||
|
- previous uploads
|
||||||
|
- skipped optional outputs
|
||||||
|
- skipped unselected outputs
|
||||||
|
- locked outputs
|
||||||
|
- current-state key paths
|
||||||
|
- `current_pointer_written`
|
||||||
|
|
||||||
## Config Fields Used
|
## Invariants
|
||||||
- `pipeline.publish.enabled`
|
- `current/run_id.txt` is the remote commit marker and is written last.
|
||||||
- `pipeline.publish.upload_run`
|
- run upload excludes `audio/**`.
|
||||||
- `pipeline.publish.outputs`
|
- publish locks are not overridden by `--force`.
|
||||||
- `pipeline.publish.locks`
|
|
||||||
- `{session_prefix}/locks.yml` loaded by app orchestration before publish execution
|
|
||||||
- `pipeline.storage.s3.bucket`
|
|
||||||
- `pipeline.storage.s3.root_prefix`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `session.campaign`
|
|
||||||
- `session.session_id`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- Object storage backend (`env.ObjectStore`) for upload/list primitives.
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Requires `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, and `analyze` status `succeeded`.
|
|
||||||
- Resolves bucket/prefix from manifest identity first, then config fallback.
|
|
||||||
- Uploads session `previous/**` files as durable session state when the local `previous/` directory exists.
|
|
||||||
- Skips top-level published output uploads for effective locked sources; run-local materialized outputs remain unchanged.
|
|
||||||
- When selected configured artifact keys are supplied, skips publish rules for unselected `narratio.artifact.<key>` sources; built-in transcript and bounds outputs still publish.
|
|
||||||
- Effective locks are the union of `pipeline.publish.locks` and remote `{session_prefix}/locks.yml`; static pipeline locks win on duplicate sources.
|
|
||||||
- Writes metadata including:
|
|
||||||
- upload counts/paths
|
|
||||||
- `previous_files_uploaded` and `previous_uploaded_paths`
|
|
||||||
- `published_files_uploaded` and `published_paths`
|
|
||||||
- `skipped_optional_outputs`
|
|
||||||
- `skipped_unselected_outputs`
|
|
||||||
- `locked_output_count` and `locked_outputs`
|
|
||||||
- `current_manifest_key`
|
|
||||||
- `current_run_id_key`
|
|
||||||
- `current_pointer_written`
|
|
||||||
- On skipped publish path, returns metadata with `skipped=true` and pointer not written.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Stage may self-skip (metadata skip) when publish disabled or run upload disabled.
|
|
||||||
- Runner-level skip also applies for previously succeeded stage unless forced.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on missing prerequisite success, missing object store when required, missing run root, missing unlocked required output source, upload failures, or pointer write failures.
|
|
||||||
- Locked required outputs are intentional skips and do not fail publish.
|
|
||||||
- Pointer semantics are fail-safe: `current/run_id.txt` is not written if prior required uploads fail.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/archive_test.go`
|
|
||||||
- `internal/app/post_archive_cleanup_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Run upload excludes `audio/` subtree.
|
|
||||||
- Session `previous/**` is publishable durable input/provenance state, not run-local output.
|
|
||||||
- Ordinary `--force` does not override publish locks.
|
|
||||||
- Malformed or unreadable remote lock store fails publish-capable execution before output uploads.
|
|
||||||
- `current/manifest.json` uploads before `current/run_id.txt`.
|
|
||||||
- `current/run_id.txt` is the remote publish commit marker.
|
|
||||||
|
|||||||
@@ -1,58 +1,22 @@
|
|||||||
# Stage: transcribe
|
# Stage: transcribe
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Generate per-speaker raw transcripts from prepared audio using WhisperX.
|
Generate raw per-speaker transcripts from prepared audio using WhisperX.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
- `audio/*.flac` from `prepare`
|
||||||
- `audio/*.flac` prepared by `prepare`
|
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- `transcripts/raw/<speaker>.json` for each input audio file
|
- `transcripts/raw/<speaker>.json`
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
- discovers prepared audio from manifest inputs or canonical audio directory.
|
||||||
- Discovering prepared audio inputs
|
- derives speaker ID from `.flac` basename.
|
||||||
- Deriving speaker ids from audio basenames
|
- runs WhisperX with configured concurrency/retry settings.
|
||||||
- Parallel WhisperX invocation with bounded concurrency
|
- validates each output as JSON.
|
||||||
- Validating produced JSON and materializing run-local outputs
|
- writes run-local outputs then materializes canonical transcript outputs.
|
||||||
|
|
||||||
Does not own:
|
## Invariants
|
||||||
- Transcript merge/polish/normalize/trim/analyze
|
- speaker basenames must be unique.
|
||||||
|
- output path returned by adapter must match requested output path.
|
||||||
## Config Fields Used
|
- each successful output is validated before stage success.
|
||||||
- `session.session_id`
|
|
||||||
- `session.campaign`
|
|
||||||
- `pipeline.workspace.root`
|
|
||||||
- `pipeline.whisperx.transcribe_url`
|
|
||||||
- `pipeline.whisperx.language`
|
|
||||||
- `pipeline.whisperx.timeout`
|
|
||||||
- `pipeline.whisperx.retries`
|
|
||||||
- `pipeline.whisperx.retry_delay`
|
|
||||||
- `pipeline.whisperx.concurrency`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- WhisperX adapter (`env.WhisperX.Transcribe`).
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Uses run-local output paths under `runs/{run_id}/transcribe/outputs/...` when run layout is enabled.
|
|
||||||
- Validates each generated transcript JSON before materialization.
|
|
||||||
- Materializes canonical outputs to `transcripts/raw/*.json`.
|
|
||||||
- Records per-file metadata (attempts/status/duration/output path) in stage metadata.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies for previously succeeded stage unless forced.
|
|
||||||
- On forced upstream reruns, downstream succeeded stages can be marked `stale` by runner logic.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails if no prepared audio exists, duplicate speaker basenames are detected, adapter output path mismatches expected path, any output JSON is invalid, or one worker fails.
|
|
||||||
- Cancels in-flight workers after first terminal error.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/transcribe_test.go`
|
|
||||||
- `internal/app/whisperx_wiring_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Speaker identity is derived from `.flac` basename and must be unique.
|
|
||||||
- Every successful speaker output must be valid JSON before materialization.
|
|
||||||
- Canonical raw transcript set is the only supported merge input surface.
|
|
||||||
|
|||||||
@@ -1,75 +1,28 @@
|
|||||||
# Stage: trim
|
# Stage: trim
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Optionally trim the final transcript to session bounds; always produce a durable final-trimmed transcript.
|
Produce a final-trimmed transcript; optionally generate bounds-driven trim.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Inputs
|
||||||
Inputs:
|
|
||||||
- `transcripts/final.json`
|
- `transcripts/final.json`
|
||||||
|
|
||||||
Outputs:
|
## Outputs
|
||||||
- `transcripts/final.trimmed.json` (or configured trim output path)
|
- `transcripts/final.trimmed.json` (or configured trim output path)
|
||||||
- when trim enabled: `artifacts/session_bounds.json`
|
- when trim enabled: `artifacts/session_bounds.json`
|
||||||
|
|
||||||
## Boundaries
|
## Key Behavior
|
||||||
Owns:
|
When `trim.enabled=false`:
|
||||||
- Trim-enabled switch behavior
|
- copies normalized transcript to trimmed output.
|
||||||
- Bounds generation via Scriptorium artifact run
|
|
||||||
- Bounds validation against final transcript
|
|
||||||
- Keep-selector derivation and Seriatim trim invocation
|
|
||||||
- Copy-through behavior when disabled or bounds indicate unchanged transcript
|
|
||||||
|
|
||||||
Does not own:
|
When `trim.enabled=true`:
|
||||||
- Upstream normalization
|
- runs Scriptorium bounds artifact generation;
|
||||||
- Downstream artifact analysis
|
- optionally runs render-debug output generation;
|
||||||
|
- validates bounds payload against transcript;
|
||||||
|
- derives keep selector;
|
||||||
|
- either copies unchanged transcript or runs Seriatim trim;
|
||||||
|
- validates trimmed transcript and materializes bounds output.
|
||||||
|
|
||||||
## Config Fields Used
|
## Invariants
|
||||||
- `session.session_id`
|
- normalized transcript is required input.
|
||||||
- `session.campaign`
|
- bounds output exists only in enabled trim path.
|
||||||
- `pipeline.workspace.root`
|
- render-debug output is diagnostic and not a declared stage output.
|
||||||
- `pipeline.trim.enabled`
|
|
||||||
- `pipeline.trim.output_path`
|
|
||||||
- `pipeline.trim.bounds.prompt_id`
|
|
||||||
- `pipeline.trim.bounds.profile_id`
|
|
||||||
- `pipeline.trim.bounds.timeout`
|
|
||||||
- `pipeline.trim.bounds.output_path`
|
|
||||||
- `pipeline.trim.bounds.transcript_input_name`
|
|
||||||
- `pipeline.trim.bounds.render_debug`
|
|
||||||
- `pipeline.trim.bounds.render_output_path`
|
|
||||||
- `pipeline.seriatim.binary`
|
|
||||||
- `pipeline.seriatim.timeout`
|
|
||||||
- `pipeline.scriptorium.binary`
|
|
||||||
- `pipeline.scriptorium.config_path`
|
|
||||||
- `pipeline.scriptorium.timeout`
|
|
||||||
|
|
||||||
## External Adapters Used
|
|
||||||
- Scriptorium adapter:
|
|
||||||
- optional `RenderArtifact` for bounds debug render
|
|
||||||
- `RunArtifact` for bounds output
|
|
||||||
- Seriatim adapter:
|
|
||||||
- `Trim` when bounds indicate trimming is required
|
|
||||||
|
|
||||||
## State and Manifest Behavior
|
|
||||||
- Reads final transcript from normalize manifest outputs when available; falls back to canonical path.
|
|
||||||
- Uses run-local outputs/logs/reports/config/scratch paths when run layout is enabled.
|
|
||||||
- Materializes canonical final-trimmed transcript and session bounds when trim is enabled.
|
|
||||||
- Records bounds diagnostics, trim action, keep selector, and adapter metadata.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
|
||||||
- Runner-level skip applies when already succeeded and not forced.
|
|
||||||
- Forced reruns can stale downstream succeeded stages.
|
|
||||||
- When `trim.enabled=false`, stage still succeeds by copying final to final-trimmed output.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
- Fails on missing/invalid final transcript.
|
|
||||||
- With trim enabled, fails on missing adapters/config, bounds generation/validation errors, invalid bounds JSON, invalid range/segment ids, trim adapter failures, or invalid final-trimmed output.
|
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/stage/trim_test.go`
|
|
||||||
- `internal/adapters/scriptorium/subprocess_test.go`
|
|
||||||
- `internal/adapters/seriatim/subprocess_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Trim never falls back to polished transcript; final transcript is required input.
|
|
||||||
- `session_bounds` output exists only for enabled trim path.
|
|
||||||
- Render-debug artifacts are diagnostics and not declared stage outputs.
|
|
||||||
|
|||||||
@@ -1,75 +1,37 @@
|
|||||||
# Internal: Storage
|
# Internal: Storage
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Document Narratio's remote storage backend contracts and implementations under `internal/adapters/storage`.
|
Document remote object-store contracts and S3 implementation behavior.
|
||||||
|
|
||||||
## Inputs and outputs
|
## Primary Contract
|
||||||
Inputs:
|
`storage.ObjectStore` interface:
|
||||||
- Resolved storage config (`pipeline.storage.*`).
|
- `List(ctx, prefix)`
|
||||||
- Already-loaded environment variables for configured S3 credentials.
|
- `Download(ctx, key, localPath)`
|
||||||
- Bucket-relative object keys and local file paths from app/stage orchestration.
|
- `Upload(ctx, localPath, key, opts)`
|
||||||
|
- `Exists(ctx, key)`
|
||||||
|
|
||||||
Outputs:
|
Key invariant:
|
||||||
- Listed/downloaded/uploaded object metadata (`ObjectInfo`).
|
- callers pass full bucket-relative keys;
|
||||||
- Existence checks and storage-layer errors.
|
- storage implementations do not infer campaign/session/run prefixes.
|
||||||
|
|
||||||
## Boundaries
|
## Configuration
|
||||||
Owns:
|
`NewObjectStoreFromConfig` currently supports S3-backed stores from `pipeline.storage.*` config.
|
||||||
- Remote object-store interface and implementation details.
|
|
||||||
- S3 client wiring and API calls.
|
|
||||||
- Object key normalization and upload/download/list primitives.
|
|
||||||
|
|
||||||
Does not own:
|
S3 constructor behavior:
|
||||||
- Session/run prefix semantics.
|
- requires configured bucket;
|
||||||
- Archive commit order semantics.
|
- uses region/endpoint/path-style options when set;
|
||||||
- Manifest updates.
|
- resolves credentials from configured env var names (with defaults).
|
||||||
- Filesystem secret loading from `pipeline.secrets.env_dir`.
|
|
||||||
|
|
||||||
## Config fields used
|
## S3 Backend Behavior
|
||||||
- `pipeline.storage.backend`
|
- normalizes object keys.
|
||||||
- `pipeline.storage.s3.bucket`
|
- `List` paginates and returns normalized `ObjectInfo`.
|
||||||
- `pipeline.storage.s3.region`
|
- `Download` writes local files with parent directory creation.
|
||||||
- `pipeline.storage.s3.endpoint`
|
- `Upload` streams local file and returns remote metadata.
|
||||||
- `pipeline.storage.s3.force_path_style`
|
- `Exists` maps not-found responses to `false`.
|
||||||
- `pipeline.storage.s3.access_key_id_env`
|
|
||||||
- `pipeline.storage.s3.secret_access_key_env`
|
|
||||||
|
|
||||||
## External adapters used
|
## Legacy Compatibility Interface
|
||||||
Storage package contracts:
|
`storage.Backend` (with `ArchiveRequest`) remains as compatibility surface with `NoopBackend`; it is not used by current stage execution.
|
||||||
- `ObjectStore` (active remote object-store boundary): `List`, `Download`, `Upload`, `Exists`.
|
|
||||||
- `Backend` (legacy compatibility boundary): currently implemented with `NoopBackend` only.
|
|
||||||
|
|
||||||
Implementations:
|
## Invariants
|
||||||
- `S3Backend`: AWS SDK-backed `ObjectStore` implementation.
|
- storage layer is stateless regarding manifest/stage progression.
|
||||||
- `FakeBackend`: deterministic test `ObjectStore` and compatibility backend.
|
- publish ordering semantics are owned by stage/app code, not storage adapters.
|
||||||
- `NoopBackend`: deterministic no-op compatibility backend for wiring/tests.
|
|
||||||
|
|
||||||
## State and manifest behavior
|
|
||||||
- Storage implementations are stateless with respect to manifest/session lifecycle.
|
|
||||||
- Caller supplies fully-qualified bucket-relative keys.
|
|
||||||
- Storage layer does not infer campaign/session/run/root-prefix semantics.
|
|
||||||
- Caller controls publish ordering; storage layer executes individual operations in the order invoked.
|
|
||||||
|
|
||||||
## Skip and resume behavior
|
|
||||||
- No storage-level skip/resume behavior.
|
|
||||||
- Skip/resume decisions are made by stage/app logic before storage calls occur.
|
|
||||||
|
|
||||||
## Failure behavior
|
|
||||||
- `NewObjectStoreFromConfig` fails when no remote backend is configured or required S3 config is missing.
|
|
||||||
- `S3Backend` constructor fails when required bucket is missing or AWS client setup fails.
|
|
||||||
- App command orchestration loads configured filesystem secrets before calling the object-store factory.
|
|
||||||
- CRUD operations return contextual errors (including not-found behavior via `Exists`).
|
|
||||||
- Key normalization is applied before operations (`\\` to `/`, leading slash trimmed).
|
|
||||||
- Remote session loading uses `List` to find the exact `session.yml` key and `Download` to materialize it to a local temp file.
|
|
||||||
|
|
||||||
## Tests to inspect before changing
|
|
||||||
- `internal/adapters/storage/factory_test.go`
|
|
||||||
- `internal/adapters/storage/s3_backend_test.go`
|
|
||||||
- `internal/adapters/storage/fake_test.go`
|
|
||||||
- `internal/adapters/storage/keys_test.go`
|
|
||||||
- `internal/adapters/storage/archive.go` + consumers in stage tests (`prepare`, `publish`)
|
|
||||||
|
|
||||||
## Architectural invariants
|
|
||||||
- Callers pass full bucket-relative keys.
|
|
||||||
- Storage backends must not prepend or infer narratio prefixes.
|
|
||||||
- Remote transport details remain isolated to storage adapter implementations.
|
|
||||||
|
|||||||
@@ -1,78 +1,57 @@
|
|||||||
# Workspace internals
|
# Internal: Workspace
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define the local durable and run-local workspace model used by stages, manifests, resume, and publish.
|
Define local session layout, run-local stage layout, and cleanup guardrails.
|
||||||
|
|
||||||
## Inputs and Outputs
|
## Canonical Session Layout
|
||||||
Inputs:
|
Session root:
|
||||||
- `pipeline.workspace.root`
|
- `{workspace.root}/work/{campaign}/{session_id}`
|
||||||
- `session.campaign`
|
|
||||||
- `session.session_id`
|
|
||||||
- generated `run_id`
|
|
||||||
|
|
||||||
Outputs:
|
Core directories/files:
|
||||||
- Session manifest at `{workspace.root}/work/{campaign}/{session_id}/manifest.json`
|
- `inputs/`
|
||||||
- Run manifest at `{workspace.root}/work/{campaign}/{session_id}/runs/{run_id}/manifest.json`
|
- `audio/`
|
||||||
- Canonical durable session directories and run-local stage trees
|
- `transcripts/`
|
||||||
|
- `artifacts/`
|
||||||
|
- `reports/`
|
||||||
|
- `logs/`
|
||||||
|
- `config/`
|
||||||
|
- `current/`
|
||||||
|
- `runs/`
|
||||||
|
- `previous/`
|
||||||
|
- `manifest.json`
|
||||||
|
- `.lock`
|
||||||
|
|
||||||
## Boundaries
|
`previous/` reserved files:
|
||||||
Owns:
|
- `previous/manifest.json`
|
||||||
- Session-level path layout (`inputs/`, `audio/`, `transcripts/`, `artifacts/`, `reports/`, `logs/`, `config/`, `current/`, `runs/`, `previous/`)
|
- `previous/artifacts/**`
|
||||||
- `previous/manifest.json` and `previous/artifacts/**` are reserved for previous-session cache state materialized by `prepare` or `restore`
|
|
||||||
- Run-local stage sandbox layout under `runs/{run_id}/{stage}/`
|
|
||||||
- Session lock acquisition/release (`.lock`)
|
|
||||||
|
|
||||||
Does not own:
|
## Run-Local Stage Layout
|
||||||
- Stage business logic
|
When run context is available, stages use:
|
||||||
- Remote publish semantics (documented in `stage-publish.md`)
|
- `runs/{run_id}/{stage}/outputs/`
|
||||||
- CLI argument parsing
|
- `runs/{run_id}/{stage}/logs/`
|
||||||
|
- `runs/{run_id}/{stage}/reports/`
|
||||||
|
- `runs/{run_id}/{stage}/config/`
|
||||||
|
- `runs/{run_id}/{stage}/scratch/`
|
||||||
|
|
||||||
## Config Fields Used
|
Run-local outputs are materialized back into canonical session paths before stage success.
|
||||||
- `pipeline.workspace.root`
|
`previous/**` writes are never redirected to run-local output paths.
|
||||||
- `pipeline.workspace.cleanup_after_publish`
|
|
||||||
- `pipeline.spool.root`
|
|
||||||
- `pipeline.spool.delete_audio_after_publish`
|
|
||||||
- `pipeline.cache.root`
|
|
||||||
- `pipeline.cache.s3_audio`
|
|
||||||
- `session.campaign`
|
|
||||||
- `session.session_id`
|
|
||||||
|
|
||||||
## External Adapters Used
|
## Locking
|
||||||
None directly in this subsystem. Stages may use object storage adapters and then write local outputs into this layout.
|
`artifacts.LocalStore` enforces single-writer session lock via `.lock` file (`ErrLockConflict` on contention).
|
||||||
|
|
||||||
## State and Manifest Behavior
|
## Cleanup Semantics
|
||||||
- Session state is persisted in the session manifest (`manifest.Manifest`).
|
Automatic post-publish cleanup (`runPostArchiveCleanup`):
|
||||||
- Invocation history is persisted per run in run manifests under `runs/{run_id}/manifest.json`.
|
- only runs when publish actually executed and succeeded;
|
||||||
- During each run, stage outputs are often written run-local first (`runs/{run_id}/{stage}/outputs/...`) and then materialized to canonical session paths after stage success.
|
- requires `uploaded=true` and `current_pointer_written=true` metadata;
|
||||||
- `manifest.Artifacts` entries record `ProducerRunID` for durable outputs.
|
- respects `pipeline.spool.delete_audio_after_publish` and `pipeline.workspace.cleanup_after_publish`;
|
||||||
- For S3 audio sessions, `prepare` records work/cache paths, S3 provenance, and spool path when the invocation downloaded the object.
|
- refuses unsafe deletes (root delete, out-of-root delete, symlink paths).
|
||||||
- `previous/**` is reconstructed from configured previous-session requirements; restore uses the previous session's committed current publish state rather than treating current-session stored `previous/**` as authoritative.
|
|
||||||
- Durable cache state under `pipeline.cache.root` is not workspace state and is preserved by default by `narratio clean`.
|
|
||||||
- `narratio clean <id>` removes the session work root and session spool root.
|
|
||||||
- `narratio clean --all` removes all local session work under `workspace.root/work` and spool children under `spool.root`.
|
|
||||||
- `narratio clean --clear-cache` is the explicit opt-in for deleting matching S3 audio cache entries.
|
|
||||||
|
|
||||||
## Skip and Resume Behavior
|
Manual clean command:
|
||||||
- Skip/resume decisions are made in `internal/app` (`run_control.go`, `resume.go`) using stage status in the session manifest.
|
- `clean <session_id>` removes session work and spool subtree.
|
||||||
- `--force` reruns selected stages and marks downstream previously-succeeded stages as `stale`.
|
- `clean --all` removes all workspace work and spool children.
|
||||||
- Workspace layout is idempotent (`EnsureLayoutFor`) and reused across runs.
|
- durable cache is preserved unless `--clear-cache` is requested.
|
||||||
|
|
||||||
## Failure Behavior
|
## Invariants
|
||||||
- Failures preserve manifests and run-local files for inspection.
|
- campaign-aware session root is mandatory.
|
||||||
- Lock conflicts fail fast via `ErrLockConflict`.
|
- manifest-driven stage state is durable across runs.
|
||||||
- Cleanup can fail post-publish; failure is recorded in publish stage metadata and returned by the run.
|
- cleanup guardrails prevent destructive root/out-of-scope deletion.
|
||||||
|
|
||||||
## Tests to Inspect Before Changing
|
|
||||||
- `internal/artifacts/local_test.go`
|
|
||||||
- `internal/stage/run_local_test.go`
|
|
||||||
- `internal/app/run_control_test.go`
|
|
||||||
- `internal/app/resume_run_stage_test.go`
|
|
||||||
- `internal/app/post_archive_cleanup_test.go`
|
|
||||||
|
|
||||||
## Architectural Invariants
|
|
||||||
- Session root is campaign-aware: `{workspace.root}/work/{campaign}/{session_id}`.
|
|
||||||
- Run roots are always nested: `runs/{run_id}` under the session root.
|
|
||||||
- Run-local output materialization must end in canonical session paths.
|
|
||||||
- `previous/**` is session-durable state and must not be treated as run-local output scratch state.
|
|
||||||
- Automatic post-publish cleanup only targets run-scoped directories and must never delete configured root directories.
|
|
||||||
- Manual `clean` may delete session-scoped directories or the `workspace.root/work` directory, but it must preserve configured root directories and reject unsafe targets.
|
|
||||||
|
|||||||
@@ -1,72 +1,156 @@
|
|||||||
# Operations
|
# Operations Guide
|
||||||
|
|
||||||
This guide covers the implemented operator lifecycle for Narratio.
|
Operator workflow for running, recovering, and publishing Narratio sessions.
|
||||||
|
|
||||||
For field-level settings, see [docs/config.md](./config.md). For syntax/flags, see [docs/cli.md](./cli.md).
|
For command syntax, see [docs/cli.md](./cli.md). For field-level config, see [docs/config.md](./config.md).
|
||||||
|
|
||||||
## Normal Workflow
|
## Standard Session Workflow
|
||||||
|
|
||||||
1. Ensure `pipeline.yml`, `campaign.yml`, and `session.yml` are available.
|
1. Select pipeline/campaign/session config.
|
||||||
2. Ensure session audio is available (local `audio_dir`/`audio_files` or S3 prefix).
|
2. Validate session readiness:
|
||||||
3. Run:
|
|
||||||
|
```bash
|
||||||
|
narratio session validate 2026-04-04
|
||||||
|
```
|
||||||
|
|
||||||
|
3. (Optional) inspect stage decisions:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session plan 2026-04-04
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Run the pipeline:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run 2026-04-04
|
narratio run 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
4. Inspect status:
|
5. Check state:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session status 2026-04-04
|
narratio session status 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Campaign and Session Selection
|
||||||
|
|
||||||
|
Campaign selection priority:
|
||||||
|
|
||||||
|
- `--campaign-file`
|
||||||
|
- `--campaign`
|
||||||
|
- `pipeline.campaigns.default_campaign_id`
|
||||||
|
|
||||||
|
Session source priority:
|
||||||
|
|
||||||
|
- `--session`
|
||||||
|
- local default search paths
|
||||||
|
- remote session object (S3) when local session file is not found and storage is configured
|
||||||
|
|
||||||
|
## Session Initialization
|
||||||
|
|
||||||
|
Use `session init` to generate a concrete session file for local or remote use.
|
||||||
|
|
||||||
|
Local file:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session init 2026-04-04 --output ./session.yml --date 2026-04-04 --title "Session 12"
|
||||||
|
```
|
||||||
|
|
||||||
|
Remote session object:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session init 2026-04-04 --remote --force
|
||||||
|
```
|
||||||
|
|
||||||
|
If `campaign.yml` sets `session_template_file`, `session init` renders it. Template variables must resolve to concrete values.
|
||||||
|
|
||||||
|
## Stage Execution and Resume Behavior
|
||||||
|
|
||||||
|
Canonical stage order:
|
||||||
|
|
||||||
|
1. `prepare`
|
||||||
|
2. `transcribe`
|
||||||
|
3. `merge`
|
||||||
|
4. `polish`
|
||||||
|
5. `normalize`
|
||||||
|
6. `trim`
|
||||||
|
7. `analyze`
|
||||||
|
8. `publish`
|
||||||
|
9. `notify`
|
||||||
|
|
||||||
|
Execution rules:
|
||||||
|
|
||||||
|
- succeeded stages are skipped unless `--force` is set;
|
||||||
|
- `resume` starts at first non-succeeded stage;
|
||||||
|
- force rerunning a succeeded upstream stage marks succeeded downstream stages as `stale`.
|
||||||
|
|
||||||
|
Single-stage execution:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio run-stage normalize 2026-04-04 --force
|
||||||
|
```
|
||||||
|
|
||||||
|
## Artifact Selection
|
||||||
|
|
||||||
|
`--artifacts` can be used on `run`, `resume`, `run-stage`, `analyze`, and `publish`.
|
||||||
|
|
||||||
|
Selection behavior:
|
||||||
|
|
||||||
|
- validates names against `pipeline.scriptorium.artifacts`;
|
||||||
|
- filters analyze execution to selected configured artifacts;
|
||||||
|
- filters publish rules for `narratio.artifact.<name>` sources only;
|
||||||
|
- does not suppress built-in transcript or bounds publish sources.
|
||||||
|
|
||||||
## Publish Workflow
|
## Publish Workflow
|
||||||
|
|
||||||
Publish is the stage that commits remote current state.
|
Run publish only:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio publish 2026-04-04
|
narratio publish 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Equivalent command:
|
Equivalent:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run-stage publish 2026-04-04 --force
|
narratio run-stage publish 2026-04-04 --force
|
||||||
```
|
```
|
||||||
|
|
||||||
Publish uploads:
|
Publish commit model:
|
||||||
|
|
||||||
- run history files under `{session_prefix}/runs/{run_id}/` (excluding `audio/`)
|
- uploads run files under `{session_prefix}/runs/{run_id}/`;
|
||||||
- configured published outputs from `pipeline.publish.outputs`
|
- uploads configured published outputs;
|
||||||
- `previous/**` cache files when present
|
- uploads `previous/**` cache files when present;
|
||||||
- `current/manifest.json`
|
- writes `current/manifest.json`;
|
||||||
- `current/run_id.txt` last
|
- writes `current/run_id.txt` last.
|
||||||
|
|
||||||
`current/run_id.txt` is the remote commit marker.
|
`current/run_id.txt` is the remote current-state commit marker.
|
||||||
|
|
||||||
## Published Outputs and Locks
|
## Publish Locks
|
||||||
|
|
||||||
Published output behavior:
|
Lock sources:
|
||||||
|
|
||||||
- outputs are source-based rules in `pipeline.publish.outputs`.
|
- static locks in `pipeline.publish.locks`
|
||||||
- required missing unlocked sources fail publish.
|
- mutable remote locks in `{session_prefix}/locks.yml`
|
||||||
- optional missing unlocked sources are skipped.
|
|
||||||
- selected artifacts (`--artifacts`) only filter configured `narratio.artifact.<key>` output rules.
|
|
||||||
- built-in transcript and bounds output rules are not filtered by `--artifacts`.
|
|
||||||
|
|
||||||
Lock behavior:
|
Effective lock rules:
|
||||||
|
|
||||||
- static locks: `pipeline.publish.locks`.
|
- static and remote locks are merged;
|
||||||
- mutable locks: `{session_prefix}/locks.yml`.
|
- static locks win on source collisions;
|
||||||
- effective lock set is static + mutable; static wins on duplicate sources.
|
- locked outputs are intentional skips;
|
||||||
- locked outputs are intentional skips and do not fail publish.
|
- lock add/remove commands mutate only remote lock state.
|
||||||
- lock commands mutate only remote mutable locks.
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session locks 2026-04-04
|
||||||
|
narratio session locks add 2026-04-04 narratio.artifact.session_recap --reason "manual edits" --force
|
||||||
|
narratio session locks remove 2026-04-04 narratio.artifact.session_recap
|
||||||
|
```
|
||||||
|
|
||||||
## Restore Workflow
|
## Restore Workflow
|
||||||
|
|
||||||
Use restore when local durable session state is missing/stale and committed remote current state is authoritative.
|
Use restore when local durable session state is missing or stale and remote committed current state is authoritative.
|
||||||
|
|
||||||
Preview:
|
Dry run:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session restore 2026-04-04 --dry-run
|
narratio session restore 2026-04-04 --dry-run
|
||||||
@@ -83,21 +167,22 @@ Default restore scope:
|
|||||||
- `manifest.json`
|
- `manifest.json`
|
||||||
- `transcripts/**`
|
- `transcripts/**`
|
||||||
- `artifacts/**`
|
- `artifacts/**`
|
||||||
- `previous/**` when required by configured previous-session artifact inputs
|
- `previous/**` when needed by configured previous-session artifact inputs
|
||||||
|
|
||||||
Optional:
|
Optional:
|
||||||
|
|
||||||
- add `--include-audio` to restore `audio/**`.
|
- `--include-audio` to include `audio/**`
|
||||||
|
- `--force` to overwrite local conflicts
|
||||||
|
|
||||||
Restore reads committed current state only (`current/run_id.txt`, `current/manifest.json`).
|
Restore writes an execution report at `reports/restore-latest.json`.
|
||||||
|
|
||||||
## Workspace and State Layout
|
## Local State Layout
|
||||||
|
|
||||||
Session root:
|
Session root:
|
||||||
|
|
||||||
- `{workspace.root}/work/{campaign}/{session_id}/`
|
- `{workspace.root}/work/{campaign}/{session_id}`
|
||||||
|
|
||||||
Durable session state:
|
Durable session paths:
|
||||||
|
|
||||||
- `manifest.json`
|
- `manifest.json`
|
||||||
- `inputs/**`
|
- `inputs/**`
|
||||||
@@ -110,61 +195,55 @@ Durable session state:
|
|||||||
- `config/**`
|
- `config/**`
|
||||||
- `runs/**`
|
- `runs/**`
|
||||||
|
|
||||||
Run-local stage layout:
|
Run-local layout:
|
||||||
|
|
||||||
- `runs/{run_id}/{stage}/outputs|logs|reports|config|scratch`
|
- `runs/{run_id}/{stage}/outputs`
|
||||||
|
- `runs/{run_id}/{stage}/logs`
|
||||||
|
- `runs/{run_id}/{stage}/reports`
|
||||||
|
- `runs/{run_id}/{stage}/config`
|
||||||
|
- `runs/{run_id}/{stage}/scratch`
|
||||||
|
|
||||||
Stages typically write run-local outputs first, then materialize canonical session outputs on success.
|
Spool layout (runtime/transient):
|
||||||
|
|
||||||
## Resume and Force Rules
|
- `{spool.root}/{campaign}/{session_id}/{run_id}/...`
|
||||||
|
- restore audio spool under `{spool.root}/{campaign}/{session_id}/restore/audio`
|
||||||
|
|
||||||
- `run` and `run-stage` skip succeeded stages unless `--force` is set.
|
Cache layout (durable S3 audio cache):
|
||||||
- `resume` starts at the first non-succeeded stage.
|
|
||||||
- force-rerunning an upstream succeeded stage marks downstream succeeded stages as `stale`.
|
- `{cache.root}/s3/{bucket}/...`
|
||||||
- `--force` does not bypass publish locks.
|
|
||||||
|
|
||||||
## Cleanup
|
## Cleanup
|
||||||
|
|
||||||
Automatic post-publish cleanup is considered only when publish executes successfully and commits current state.
|
Session-scoped cleanup:
|
||||||
|
|
||||||
Config toggles:
|
|
||||||
|
|
||||||
- `pipeline.spool.delete_audio_after_publish=true`
|
|
||||||
- `pipeline.workspace.cleanup_after_publish=true`
|
|
||||||
|
|
||||||
Manual cleanup:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio clean 2026-04-04
|
narratio clean 2026-04-04
|
||||||
|
```
|
||||||
|
|
||||||
|
Global cleanup:
|
||||||
|
|
||||||
|
```bash
|
||||||
narratio clean --all
|
narratio clean --all
|
||||||
```
|
```
|
||||||
|
|
||||||
Cache is preserved by default. Use `--clear-cache` to remove matching S3 audio cache entries.
|
Dry-run and cache variants:
|
||||||
|
|
||||||
## Failure and Recovery
|
|
||||||
|
|
||||||
After stage failure, Narratio keeps manifests and run-local files for inspection.
|
|
||||||
|
|
||||||
Standard recovery flow:
|
|
||||||
|
|
||||||
1. inspect status:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session status 2026-04-04
|
narratio clean 2026-04-04 --dry-run --clear-cache
|
||||||
|
narratio clean --all --dry-run --clear-cache
|
||||||
```
|
```
|
||||||
|
|
||||||
2. if needed, inspect restore plan:
|
Rules:
|
||||||
|
|
||||||
```bash
|
- `clean` deletes work/spool session state;
|
||||||
narratio session restore 2026-04-04 --dry-run
|
- cache is preserved unless `--clear-cache` is set;
|
||||||
```
|
- automatic post-publish cleanup is gated by successful publish commit plus:
|
||||||
|
- `pipeline.spool.delete_audio_after_publish=true`
|
||||||
3. fix root cause.
|
- `pipeline.workspace.cleanup_after_publish=true`
|
||||||
4. continue with `resume`, or rerun a stage with `--force` then `resume`.
|
|
||||||
|
|
||||||
## Operational Caveats
|
## Operational Caveats
|
||||||
|
|
||||||
- local and S3 audio modes are mutually exclusive.
|
- Local and S3 audio modes are mutually exclusive.
|
||||||
- publish requires prerequisite stages through `analyze` to be `succeeded`.
|
- Publish requires prerequisite stages through analyze to be succeeded.
|
||||||
- restore requires configured object storage and committed current state.
|
- Restore requires configured object storage and committed remote current state.
|
||||||
- `session status` and `session artifacts --remote` both report remote published-output availability when storage is configured.
|
- Storage-backed commands load filesystem secrets before object-store initialization.
|
||||||
|
|||||||
105
docs/roadmap/documentation-stage1-audit.md
Normal file
105
docs/roadmap/documentation-stage1-audit.md
Normal file
@@ -0,0 +1,105 @@
|
|||||||
|
# Documentation Pass: Stage 1 Audit
|
||||||
|
|
||||||
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
|
## Scope Reviewed
|
||||||
|
|
||||||
|
- All non-policy documentation files under `docs/`
|
||||||
|
- `README.md`
|
||||||
|
- Documentation references to maintained `examples/` files
|
||||||
|
- Documentation-related expectations in tests under `internal/**`
|
||||||
|
|
||||||
|
## File Inventory and Canonical Scope
|
||||||
|
|
||||||
|
| File | Intended audience | Canonical scope (per policy) | Primary source-of-truth anchors |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `README.md` | Users, operators | Project orientation and links | `cmd/narratio`, `internal/app/commands.go`, docs index files |
|
||||||
|
| `docs/cli.md` | Users, operators | CLI syntax, flags, command workflows | `internal/app/*.go`, `internal/app/*_test.go` |
|
||||||
|
| `docs/config.md` | Operators, advanced users | Config discovery, schema, defaults, examples | `internal/config/*.go`, `internal/config/*_test.go`, `examples/*` |
|
||||||
|
| `docs/operations.md` | Operators | Run/resume/publish/restore/cleanup workflows | `internal/app/runner.go`, `internal/app/restore*.go`, `internal/stage/archive.go`, `internal/artifacts/*.go` |
|
||||||
|
| `docs/troubleshooting.md` | Operators | Failure diagnosis and safe fixes | `internal/app`, `internal/stage`, related tests |
|
||||||
|
| `docs/internal/README.md` | Developers, LLM coding agents | Internal docs index and scope boundaries | `docs/internal/*.md`, policy docs |
|
||||||
|
| `docs/internal/adapters.md` | Developers, LLM coding agents | Adapter boundaries and ownership | `internal/adapters/*`, `internal/stage/*` |
|
||||||
|
| `docs/internal/artifacts.md` | Developers, LLM coding agents | Artifact catalog and source resolution contracts | `internal/artifacts/*`, `internal/stage/analyze.go`, `internal/stage/prepare_previous.go` |
|
||||||
|
| `docs/internal/command-restore.md` | Developers, LLM coding agents | Restore command architecture and contracts | `internal/app/restore*.go`, `internal/app/restore*_test.go` |
|
||||||
|
| `docs/internal/manifest.md` | Developers, LLM coding agents | Session/run manifest contracts and transitions | `internal/manifest/*`, `internal/app/runner.go`, `internal/stage/*` |
|
||||||
|
| `docs/internal/stage-prepare.md` | Developers, LLM coding agents | Prepare stage IO and invariants | `internal/stage/prepare.go`, `internal/stage/prepare*_test.go` |
|
||||||
|
| `docs/internal/stage-transcribe.md` | Developers, LLM coding agents | Transcribe stage IO and invariants | `internal/stage/transcribe.go`, `internal/stage/transcribe_test.go` |
|
||||||
|
| `docs/internal/stage-merge.md` | Developers, LLM coding agents | Merge stage IO and invariants | `internal/stage/merge.go`, `internal/stage/merge_test.go` |
|
||||||
|
| `docs/internal/stage-polish.md` | Developers, LLM coding agents | Polish stage IO and invariants | `internal/stage/polish.go`, `internal/stage/polish_test.go` |
|
||||||
|
| `docs/internal/stage-normalize.md` | Developers, LLM coding agents | Normalize stage IO and invariants | `internal/stage/normalize.go`, `internal/stage/normalize_test.go` |
|
||||||
|
| `docs/internal/stage-trim.md` | Developers, LLM coding agents | Trim stage IO and invariants | `internal/stage/trim.go`, `internal/stage/trim_test.go` |
|
||||||
|
| `docs/internal/stage-analyze.md` | Developers, LLM coding agents | Analyze stage artifact execution and selection | `internal/stage/analyze.go`, `internal/stage/analyze_test.go` |
|
||||||
|
| `docs/internal/stage-publish.md` | Developers, LLM coding agents | Publish-stage commit/upload invariants | `internal/stage/archive.go`, `internal/stage/archive_test.go` |
|
||||||
|
| `docs/internal/storage.md` | Developers, LLM coding agents | Storage adapter contracts and semantics | `internal/adapters/storage/*`, `internal/app/object_store.go` |
|
||||||
|
| `docs/internal/workspace.md` | Developers, LLM coding agents | Local workspace/session/run path model | `internal/artifacts/*`, `internal/app/runner.go`, `internal/stage/run_local.go` |
|
||||||
|
| `docs/integrations/README.md` | Developers, LLM coding agents | Integration docs index | `docs/integrations/*.md` |
|
||||||
|
| `docs/integrations/audita.md` | Developers, integration maintainers | Audita adapter contract | `internal/adapters/audita/*`, `internal/stage/polish.go` |
|
||||||
|
| `docs/integrations/seriatim.md` | Developers, integration maintainers | Seriatim adapter contract | `internal/adapters/seriatim/*`, `internal/stage/merge.go`, `internal/stage/normalize.go`, `internal/stage/trim.go` |
|
||||||
|
| `docs/integrations/scriptorium.md` | Developers, integration maintainers | Scriptorium adapter contract | `internal/adapters/scriptorium/*`, `internal/stage/analyze.go`, `internal/stage/trim.go` |
|
||||||
|
| `docs/roadmap/documentation.md` | Developers, maintainers | Planning and implementation sequencing for documentation pass | N/A (planning artifact) |
|
||||||
|
| `docs/roadmap/documentation-stage1-audit.md` | Developers, maintainers | Stage-1 inventory and source-of-truth audit record | N/A (planning artifact) |
|
||||||
|
|
||||||
|
## Source-of-Truth Mapping Summary
|
||||||
|
|
||||||
|
- CLI behaviors and command names are grounded in `internal/app/commands.go` and command handlers in `internal/app/*.go`.
|
||||||
|
- Stage order and canonical stage names are grounded in `internal/stage/placeholders.go` (`prepare` -> `transcribe` -> `merge` -> `polish` -> `normalize` -> `trim` -> `analyze` -> `publish` -> `notify`).
|
||||||
|
- Publish behavior and current-pointer commit semantics are grounded in `internal/stage/archive.go`.
|
||||||
|
- Config schema/defaults/validation are grounded in `internal/config/*`.
|
||||||
|
- Local/remote paths, publish keys, and workspace layout are grounded in `internal/artifacts/*`.
|
||||||
|
- Restore behavior and report contracts are grounded in `internal/app/restore*.go`.
|
||||||
|
- Maintained examples and schema compatibility are grounded in `examples/*` plus `internal/config/load_validate_test.go` (`TestExamplesLoadAndValidate`).
|
||||||
|
|
||||||
|
## Findings
|
||||||
|
|
||||||
|
### Broken or stale references
|
||||||
|
|
||||||
|
1. `README.md` linked to non-existent files:
|
||||||
|
- `docs/development.md`
|
||||||
|
- `docs/architecture.md`
|
||||||
|
2. `docs/internal/README.md` and `docs/integrations/README.md` linked to non-existent path:
|
||||||
|
- `docs/documentation/policy.md`
|
||||||
|
|
||||||
|
Stage-1 fix applied:
|
||||||
|
- Updated those links to existing policy docs under `docs/policy/`.
|
||||||
|
|
||||||
|
### Stale terminology sweep
|
||||||
|
|
||||||
|
Sweep terms used: `archive`, `promote`, `promoted`, `promote_artifacts`, `run-stage archive`.
|
||||||
|
|
||||||
|
Findings:
|
||||||
|
- User-facing docs in scope did not show obvious stale command examples requiring immediate correction.
|
||||||
|
- Internal code and tests still contain historical `archive` identifiers while user-facing command/stage naming is `publish` (for example, `internal/stage/archive.go` type names). This is acceptable for now but should be normalized deliberately, not incidentally.
|
||||||
|
|
||||||
|
Stage-1 fix applied:
|
||||||
|
- Updated clearly stale publish-related wording in test expectation messages/comments:
|
||||||
|
- `internal/app/commands_test.go`
|
||||||
|
- `internal/app/operator_helpers_test.go`
|
||||||
|
|
||||||
|
### Example path validation
|
||||||
|
|
||||||
|
- All `examples/...` paths referenced from non-policy docs resolve to existing files.
|
||||||
|
- `internal/config/load_validate_test.go` includes `TestExamplesLoadAndValidate` and points to current example files.
|
||||||
|
|
||||||
|
### Roadmap leakage into current-behavior docs
|
||||||
|
|
||||||
|
- No obvious roadmap-only behavior leakage found in non-roadmap docs during this sweep.
|
||||||
|
|
||||||
|
### Duplicate content and scope drift
|
||||||
|
|
||||||
|
- No severe duplication requiring immediate rewrite in this stage.
|
||||||
|
- Existing docs still need full content rewrite for 1.0 readiness in later stages (user/operator first, then internal/integrations), as planned.
|
||||||
|
|
||||||
|
### Canonical-home inconsistency to resolve in rewrite stages
|
||||||
|
|
||||||
|
- Policy canonical-home language names `docs/architecture.md` and `docs/development.md`, while current repository stores those policy documents under `docs/policy/`.
|
||||||
|
- Stage 1 preserves repository behavior by fixing broken links to existing files. Later rewrite stages should converge canonical-home paths and references consistently across docs.
|
||||||
|
|
||||||
|
## Stage-1 Completion Check
|
||||||
|
|
||||||
|
Completed for this stage:
|
||||||
|
- Full non-policy file inventory with audience and scope mapping.
|
||||||
|
- Source-of-truth crosswalk to code/tests.
|
||||||
|
- Stale-term, link, and example-path sweeps.
|
||||||
|
- Documentation-related stale test wording corrections.
|
||||||
|
- Minimal fixes only; broad rewrites intentionally deferred.
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# Roadmap: 1.0 Documentation Pass
|
# Roadmap: 1.0 Documentation Pass
|
||||||
|
|
||||||
Status: Planned
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
## Goal
|
## Goal
|
||||||
|
|
||||||
@@ -43,7 +43,7 @@ Out of scope:
|
|||||||
|
|
||||||
### Stage 1: Inventory and Source-of-Truth Audit
|
### Stage 1: Inventory and Source-of-Truth Audit
|
||||||
|
|
||||||
Status: Planned
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
Create a file-by-file inventory of all non-policy docs before rewriting.
|
Create a file-by-file inventory of all non-policy docs before rewriting.
|
||||||
|
|
||||||
@@ -73,7 +73,7 @@ Acceptance criteria:
|
|||||||
|
|
||||||
### Stage 2: User and Operator Docs
|
### Stage 2: User and Operator Docs
|
||||||
|
|
||||||
Status: Planned
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
Rewrite the user-facing and operator-facing docs first.
|
Rewrite the user-facing and operator-facing docs first.
|
||||||
|
|
||||||
@@ -109,7 +109,7 @@ Acceptance criteria:
|
|||||||
|
|
||||||
### Stage 3: Internal Developer Docs
|
### Stage 3: Internal Developer Docs
|
||||||
|
|
||||||
Status: Planned
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
Rewrite implemented internal component docs after public docs stabilize.
|
Rewrite implemented internal component docs after public docs stabilize.
|
||||||
|
|
||||||
@@ -141,7 +141,7 @@ Acceptance criteria:
|
|||||||
|
|
||||||
### Stage 4: Integrations and Examples
|
### Stage 4: Integrations and Examples
|
||||||
|
|
||||||
Status: Planned
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
Review integration docs and maintained examples after core docs are rewritten.
|
Review integration docs and maintained examples after core docs are rewritten.
|
||||||
|
|
||||||
@@ -168,7 +168,7 @@ Acceptance criteria:
|
|||||||
|
|
||||||
### Stage 5: Roadmap Cleanup and Final Sweep
|
### Stage 5: Roadmap Cleanup and Final Sweep
|
||||||
|
|
||||||
Status: Planned
|
Status: Completed (2026-05-23)
|
||||||
|
|
||||||
Clean up roadmap state and run final documentation validation.
|
Clean up roadmap state and run final documentation validation.
|
||||||
|
|
||||||
|
|||||||
@@ -1,35 +1,58 @@
|
|||||||
# Troubleshooting
|
# Troubleshooting
|
||||||
|
|
||||||
## Purpose
|
Operational diagnosis guide for common Narratio failures.
|
||||||
Canonical operator troubleshooting guide for recurring Narratio failures.
|
|
||||||
|
|
||||||
## Config discovery failure
|
## Config file not found
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- command fails because `pipeline.yml`, `campaign.yml`, or `session.yml` was not found.
|
|
||||||
|
|
||||||
Likely cause:
|
- command fails to resolve `pipeline.yml`, `campaign.yml`, or `session.yml`.
|
||||||
- missing files in discovery paths.
|
|
||||||
- missing/incorrect campaign selection.
|
Likely causes:
|
||||||
- local file exists but was not passed explicitly.
|
|
||||||
|
- missing files in default search paths;
|
||||||
|
- wrong campaign selection;
|
||||||
|
- omitted explicit flags.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml
|
narratio session plan 2026-04-04
|
||||||
ls -l /usr/local/etc/narratio/session.yml /etc/narratio/session.yml
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- pass explicit `--config`, `--campaign` or `--campaign-file`, and `--session`.
|
- pass explicit `--config`, `--campaign` or `--campaign-file`, and `--session`.
|
||||||
|
|
||||||
## Templated session file rejected
|
## Session template placeholders rejected
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- load fails because `session.yml` must be concrete.
|
|
||||||
|
- load error says session file must be concrete or contains `{{ ... }}` placeholders.
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
- template placeholders (`{{ ... }}`) still present in loaded session config.
|
|
||||||
|
- using template content as runtime session config.
|
||||||
|
|
||||||
|
Diagnostics:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session validate 2026-04-04 --session /path/session.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
Safe fix:
|
||||||
|
|
||||||
|
- generate concrete session YAML with `narratio session init`.
|
||||||
|
|
||||||
|
## Strict decode or schema validation failure
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
- unknown field / invalid value error during config load.
|
||||||
|
|
||||||
|
Likely cause:
|
||||||
|
|
||||||
|
- stale field name, typo, invalid enum, or invalid duration/path format.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
@@ -38,82 +61,79 @@ narratio session plan 2026-04-04 --config /path/pipeline.yml --campaign-file /pa
|
|||||||
```
|
```
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
- generate concrete session YAML via `narratio session init`.
|
|
||||||
|
|
||||||
## Strict decode or validation failure
|
- align config with [docs/config.md](./config.md) and maintained files under `examples/`.
|
||||||
|
|
||||||
|
## Audio mode conflict
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- unknown field or invalid value error during config load.
|
|
||||||
|
- validation fails on session audio configuration.
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
- typo, stale field name, or invalid value.
|
|
||||||
|
- configured both local and S3 session audio inputs.
|
||||||
Diagnostics:
|
|
||||||
|
Safe fix:
|
||||||
```bash
|
|
||||||
narratio session plan 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
- use local mode (`audio_dir` or `audio_files`) or S3 mode (`audio_s3.prefix`), not both.
|
||||||
```
|
|
||||||
|
## `--artifacts` selection error
|
||||||
Safe fix:
|
|
||||||
- align config with [docs/config.md](./config.md) and maintained examples.
|
Symptom:
|
||||||
|
|
||||||
## `--artifacts` selection failure
|
- unknown artifact key or invalid `--artifacts` usage.
|
||||||
|
|
||||||
Symptom:
|
Likely causes:
|
||||||
- command fails on unknown/invalid selected artifact key.
|
|
||||||
|
- key not defined in `pipeline.scriptorium.artifacts`;
|
||||||
Likely cause:
|
- empty list entry (for example trailing comma);
|
||||||
- artifact key not defined in `pipeline.scriptorium.artifacts`.
|
- `run-stage` used with non-`analyze`/`publish` target.
|
||||||
- empty token in `--artifacts` input.
|
|
||||||
|
Safe fix:
|
||||||
Safe fix:
|
|
||||||
- use only configured artifact keys.
|
- provide only configured keys and use `--artifacts` with supported commands/stages.
|
||||||
|
|
||||||
## `run-stage --artifacts` unsupported stage
|
## Previous-session artifact input missing
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- `run-stage` rejects `--artifacts` for the selected stage.
|
|
||||||
|
- prepare/analyze fails due to missing required previous-session artifact cache input.
|
||||||
Likely cause:
|
|
||||||
- `--artifacts` used with a stage other than `analyze` or `publish`.
|
Likely causes:
|
||||||
|
|
||||||
Safe fix:
|
- missing `session.previous_session_id`;
|
||||||
- use `--artifacts` only with `run-stage analyze ...` or `run-stage publish ...`.
|
- previous artifact not restored/published for source session.
|
||||||
|
|
||||||
## Previous-session input unavailable
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
- analyze fails on required previous-session artifact input.
|
|
||||||
|
|
||||||
Likely cause:
|
|
||||||
- `previous/**` cache not hydrated for this session.
|
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
narratio session validate 2026-04-04
|
||||||
narratio session status 2026-04-04
|
narratio session status 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run-stage prepare 2026-04-04 --force
|
narratio session restore 2026-04-04
|
||||||
```
|
```
|
||||||
|
|
||||||
Or rehydrate from remote current state:
|
or rerun prepare after correcting session config:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session restore 2026-04-04
|
narratio run-stage prepare 2026-04-04 --force
|
||||||
```
|
```
|
||||||
|
|
||||||
## Session lock conflict (`.lock`)
|
## Session lock conflict (`.lock`)
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- command fails with lock conflict.
|
|
||||||
|
|
||||||
Likely cause:
|
- command fails acquiring session lock.
|
||||||
- another process is running for the same session.
|
|
||||||
- stale lock file from interrupted command.
|
Likely causes:
|
||||||
|
|
||||||
|
- another process is running for the same session;
|
||||||
|
- stale lock left by interrupted process.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
@@ -123,16 +143,19 @@ ps aux | grep narratio
|
|||||||
```
|
```
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
- wait for active process; remove stale lock only if no process is active.
|
|
||||||
|
|
||||||
## Restore current pointer/manifest missing
|
- wait for active process completion;
|
||||||
|
- remove stale lock only after confirming no live process owns it.
|
||||||
|
|
||||||
|
## Restore conflict without `--force`
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- restore fails reading remote current state.
|
|
||||||
|
- restore fails with conflict count.
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
- publish commit did not complete.
|
|
||||||
- `current/run_id.txt` or `current/manifest.json` is missing.
|
- local durable files differ from remote restore sources.
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
@@ -141,83 +164,110 @@ narratio session restore 2026-04-04 --dry-run
|
|||||||
```
|
```
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
- republish from a healthy local session state.
|
|
||||||
|
|
||||||
## Restore conflict without `--force`
|
- review conflicts;
|
||||||
|
- rerun with `--force` only when remote state should overwrite local.
|
||||||
|
|
||||||
|
## Restore current-state discovery failure
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
- restore reports conflict and exits.
|
|
||||||
|
|
||||||
Likely cause:
|
- restore cannot find current pointer or current manifest.
|
||||||
- local durable file differs from remote restore source.
|
|
||||||
|
|
||||||
Safe fix:
|
Likely causes:
|
||||||
- inspect with `--dry-run`.
|
|
||||||
- rerun with `--force` only when remote should overwrite local.
|
|
||||||
|
|
||||||
## Secrets or credentials failure
|
- no committed publish current state;
|
||||||
|
- storage credentials or connectivity failure.
|
||||||
Symptom:
|
|
||||||
- startup fails loading secrets dir, or storage/tool auth fails at runtime.
|
|
||||||
|
|
||||||
Likely cause:
|
|
||||||
- invalid `pipeline.secrets.env_dir`.
|
|
||||||
- missing credential env vars.
|
|
||||||
|
|
||||||
Diagnostics:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ls -la /path/to/secrets_dir
|
|
||||||
env | grep -E 'AUDITA|OBJECT_STORAGE|AWS|SCRIPTORIUM'
|
|
||||||
```
|
|
||||||
|
|
||||||
Safe fix:
|
|
||||||
- fix path/permissions/env vars; keep secret values out of YAML.
|
|
||||||
|
|
||||||
## S3 audio prepare failure
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
- prepare fails in S3 mode (list/download/no files/backend error).
|
|
||||||
|
|
||||||
Likely cause:
|
|
||||||
- bad `session.inputs.audio_s3.prefix`.
|
|
||||||
- no `.flac` objects at prefix.
|
|
||||||
- bad storage credentials/config.
|
|
||||||
- mixed local+S3 audio config.
|
|
||||||
|
|
||||||
Diagnostics:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio run-stage prepare 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
|
||||||
```
|
|
||||||
|
|
||||||
Safe fix:
|
|
||||||
- configure exactly one audio mode and verify storage access.
|
|
||||||
|
|
||||||
## Publish output or current-pointer failure
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
- publish fails on required output source missing, upload error, or commit-marker write failure.
|
|
||||||
|
|
||||||
Likely cause:
|
|
||||||
- required source file not produced.
|
|
||||||
- storage upload failed before `current/run_id.txt` write.
|
|
||||||
|
|
||||||
Diagnostics:
|
Diagnostics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio session status 2026-04-04
|
narratio session status 2026-04-04
|
||||||
narratio run-stage publish 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
narratio session restore 2026-04-04 --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
- rerun upstream stages to regenerate required outputs.
|
|
||||||
- adjust `pipeline.publish.outputs` source/dest rules.
|
|
||||||
- retry after storage issue is fixed.
|
|
||||||
|
|
||||||
## Helpful Links
|
- resolve storage/auth issue;
|
||||||
|
- republish from healthy local state if current pointer is missing.
|
||||||
|
|
||||||
|
## Publish output failure
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
- publish fails on missing required source, upload error, or commit write.
|
||||||
|
|
||||||
|
Likely causes:
|
||||||
|
|
||||||
|
- required source file not produced;
|
||||||
|
- lock/state expectations mismatch;
|
||||||
|
- remote storage failure.
|
||||||
|
|
||||||
|
Diagnostics:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session artifacts 2026-04-04 --remote
|
||||||
|
narratio session status 2026-04-04
|
||||||
|
narratio run-stage publish 2026-04-04 --force
|
||||||
|
```
|
||||||
|
|
||||||
|
Safe fix:
|
||||||
|
|
||||||
|
- regenerate missing sources by rerunning prerequisite stages;
|
||||||
|
- correct publish source/destination rules;
|
||||||
|
- retry after storage failure is resolved.
|
||||||
|
|
||||||
|
## Secrets or storage credential failure
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
- object-store command fails at initialization/auth.
|
||||||
|
|
||||||
|
Likely causes:
|
||||||
|
|
||||||
|
- invalid `pipeline.secrets.env_dir`;
|
||||||
|
- missing credential environment variables;
|
||||||
|
- invalid S3 endpoint/bucket settings.
|
||||||
|
|
||||||
|
Diagnostics:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ls -la /path/to/secrets_dir
|
||||||
|
env | grep -E 'OBJECT_STORAGE|AWS|AUDITA|SCRIPTORIUM'
|
||||||
|
```
|
||||||
|
|
||||||
|
Safe fix:
|
||||||
|
|
||||||
|
- correct secret-file path and permissions;
|
||||||
|
- provide required env vars;
|
||||||
|
- keep secret values out of YAML.
|
||||||
|
|
||||||
|
## S3 audio prepare failure
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
- prepare fails listing/downloading session S3 audio.
|
||||||
|
|
||||||
|
Likely causes:
|
||||||
|
|
||||||
|
- incorrect `session.inputs.audio_s3.prefix`;
|
||||||
|
- no matching `.flac` objects;
|
||||||
|
- storage connectivity or permissions failure.
|
||||||
|
|
||||||
|
Diagnostics:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio run-stage prepare 2026-04-04 --force
|
||||||
|
```
|
||||||
|
|
||||||
|
Safe fix:
|
||||||
|
|
||||||
|
- verify prefix contents and storage access;
|
||||||
|
- keep session audio mode consistent.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
- [docs/config.md](./config.md)
|
|
||||||
- [docs/cli.md](./cli.md)
|
- [docs/cli.md](./cli.md)
|
||||||
|
- [docs/config.md](./config.md)
|
||||||
- [docs/operations.md](./operations.md)
|
- [docs/operations.md](./operations.md)
|
||||||
- [docs/internal/stage-publish.md](./internal/stage-publish.md)
|
- [docs/internal/stage-publish.md](./internal/stage-publish.md)
|
||||||
|
|||||||
@@ -119,7 +119,7 @@ func TestExecuteRunStageArchiveAliasFails(t *testing.T) {
|
|||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
if !strings.Contains(stderr.String(), `unknown stage "archive"`) {
|
if !strings.Contains(stderr.String(), `unknown stage "archive"`) {
|
||||||
t.Fatalf("stderr = %q, want unknown archive stage error", stderr.String())
|
t.Fatalf("stderr = %q, want unknown stage alias error", stderr.String())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -851,13 +851,13 @@ func TestExecuteStatusReportsRemoteArtifactCatalogErrorsWithoutFailing(t *testin
|
|||||||
}
|
}
|
||||||
out := stdout.String()
|
out := stdout.String()
|
||||||
if !strings.Contains(out, "Remote publish: missing or unavailable:") {
|
if !strings.Contains(out, "Remote publish: missing or unavailable:") {
|
||||||
t.Fatalf("stdout = %q, want remote archive unavailable state", out)
|
t.Fatalf("stdout = %q, want remote publish unavailable state", out)
|
||||||
}
|
}
|
||||||
if !strings.Contains(out, "Remote outputs:") || !strings.Contains(out, "narratio.transcript.final_trimmed remote=error") {
|
if !strings.Contains(out, "Remote outputs:") || !strings.Contains(out, "narratio.transcript.final_trimmed remote=error") {
|
||||||
t.Fatalf("stdout = %q, want remote output error state", out)
|
t.Fatalf("stdout = %q, want remote output error state", out)
|
||||||
}
|
}
|
||||||
if !strings.Contains(out, "Publish locks: error:") {
|
if !strings.Contains(out, "Publish locks: error:") {
|
||||||
t.Fatalf("stdout = %q, want archive locks error", out)
|
t.Fatalf("stdout = %q, want publish locks error", out)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -872,7 +872,7 @@ func TestExecuteArchiveLoadsRemoteLocks(t *testing.T) {
|
|||||||
|
|
||||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||||
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze"} {
|
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze"} {
|
||||||
// The archive stage only checks the manifest statuses and source files.
|
// The publish stage only checks the manifest statuses and source files.
|
||||||
_ = stageName
|
_ = stageName
|
||||||
}
|
}
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.trimmed.json"), `{"segments":[]}`)
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.trimmed.json"), `{"segments":[]}`)
|
||||||
|
|||||||
Reference in New Issue
Block a user