Compare commits
26 Commits
ef1f650bc0
...
v1.1.1
| Author | SHA1 | Date | |
|---|---|---|---|
| c6632d5576 | |||
| ffc07922c7 | |||
| f3310d4d16 | |||
| 88cee96d8d | |||
| 2fece10215 | |||
| 0658f2f642 | |||
| a51228c803 | |||
| 4491fb5ccd | |||
| 30b905765c | |||
| 03eac70881 | |||
| 0f7e6b979f | |||
| c366912586 | |||
| 9fe44cd00d | |||
| 094b0d2532 | |||
| 98649f4d81 | |||
| 8a559efd5b | |||
| 72deccb4e2 | |||
| 5620fc5bcf | |||
| be57e675e0 | |||
| 3971443831 | |||
| a6b0c33e9f | |||
| 96b886e711 | |||
| 7d584ee6cd | |||
| 572a112c31 | |||
| ea87c335d6 | |||
| 7169ff04df |
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
Narratio is a stage-driven Go orchestrator for turning D&D session audio into polished transcripts and generated artifacts.
|
Narratio is a stage-driven Go orchestrator for turning D&D session audio into polished transcripts and generated artifacts.
|
||||||
|
|
||||||
It runs a deterministic workflow across `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, `analyze`, and `publish`, with manifest-driven resume and restore support.
|
It runs a deterministic workflow across `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, `render`, `analyze`, and `publish`, with manifest-driven continuation and restore support.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
narratio run 2026-04-04
|
narratio run 2026-04-04
|
||||||
|
|||||||
37
docs/cli.md
37
docs/cli.md
@@ -13,7 +13,6 @@ This runs the canonical full pipeline for session `2026-04-04`.
|
|||||||
Top-level commands:
|
Top-level commands:
|
||||||
|
|
||||||
- `run <session_id>`: run full stage order.
|
- `run <session_id>`: run full stage order.
|
||||||
- `resume <session_id>`: continue from first non-succeeded stage.
|
|
||||||
- `run-stage <stage> <session_id>`: run one stage.
|
- `run-stage <stage> <session_id>`: run one stage.
|
||||||
- `analyze <session_id>`: force-run analyze.
|
- `analyze <session_id>`: force-run analyze.
|
||||||
- `publish <session_id>`: force-run publish.
|
- `publish <session_id>`: force-run publish.
|
||||||
@@ -40,14 +39,31 @@ Most session-aware commands accept:
|
|||||||
- `--campaign <id>`
|
- `--campaign <id>`
|
||||||
- `--campaign-file <campaign.yml>`
|
- `--campaign-file <campaign.yml>`
|
||||||
- `--session <session.yml>`
|
- `--session <session.yml>`
|
||||||
- `--previous-session-id <id>`
|
- `--session-id <session_id>`
|
||||||
|
- `--previous-session-id <session_id>`
|
||||||
|
|
||||||
Rules:
|
Rules:
|
||||||
|
|
||||||
- `--campaign` and `--campaign-file` are mutually exclusive.
|
- `--campaign` and `--campaign-file` are mutually exclusive.
|
||||||
- `--session` is not used by `session init`.
|
- `--session` is not used by `session init`.
|
||||||
|
- if both positional `<session_id>` and `--session-id` are provided, values must match.
|
||||||
- `clean --all` cannot be combined with campaign/session selectors.
|
- `clean --all` cannot be combined with campaign/session selectors.
|
||||||
|
|
||||||
|
## Session ID Input Rules
|
||||||
|
|
||||||
|
Session-aware commands accept one of these forms:
|
||||||
|
|
||||||
|
- positional session ID: `... <session_id>`
|
||||||
|
- compatibility flag: `... --session-id <session_id>`
|
||||||
|
|
||||||
|
When both are present, command parsing requires an exact match.
|
||||||
|
|
||||||
|
Commands with additional positionals keep their command-specific order:
|
||||||
|
|
||||||
|
- `run-stage <stage> <session_id>` or `run-stage <stage> --session-id <session_id>`
|
||||||
|
- `session locks add <session_id> <source>` or `session locks add --session-id <session_id> <source>`
|
||||||
|
- `session locks remove <session_id> <source>` or `session locks remove --session-id <session_id> <source>`
|
||||||
|
|
||||||
## Command Reference
|
## Command Reference
|
||||||
|
|
||||||
### `run`
|
### `run`
|
||||||
@@ -60,19 +76,9 @@ Behavior:
|
|||||||
|
|
||||||
- evaluates full stage order;
|
- evaluates full stage order;
|
||||||
- skips already-succeeded stages unless `--force` is set;
|
- skips already-succeeded stages unless `--force` is set;
|
||||||
|
- continues interrupted or partially completed sessions by running non-succeeded stages;
|
||||||
- writes session and run manifests.
|
- writes session and run manifests.
|
||||||
|
|
||||||
### `resume`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio resume <session_id> [--force] [--artifacts <name[,name...]>] [...common config flags]
|
|
||||||
```
|
|
||||||
|
|
||||||
Behavior:
|
|
||||||
|
|
||||||
- when not forced, starts at first non-succeeded stage in manifest order;
|
|
||||||
- with `--force`, reevaluates the selected stage list as runnable.
|
|
||||||
|
|
||||||
### `run-stage`
|
### `run-stage`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -87,6 +93,7 @@ Valid stage names:
|
|||||||
- `polish`
|
- `polish`
|
||||||
- `normalize`
|
- `normalize`
|
||||||
- `trim`
|
- `trim`
|
||||||
|
- `render`
|
||||||
- `analyze`
|
- `analyze`
|
||||||
- `publish`
|
- `publish`
|
||||||
- `notify`
|
- `notify`
|
||||||
@@ -187,7 +194,7 @@ Options:
|
|||||||
Rules:
|
Rules:
|
||||||
|
|
||||||
- `--audio-dir` and `--audio-s3-prefix` are mutually exclusive.
|
- `--audio-dir` and `--audio-s3-prefix` are mutually exclusive.
|
||||||
- if campaign `session_template_file` is configured, `session init` renders it;
|
- if campaign `session_template_file` is configured, `session init` renders it.
|
||||||
- generated session YAML must be concrete (no unresolved `{{ ... }}` placeholders).
|
- generated session YAML must be concrete (no unresolved `{{ ... }}` placeholders).
|
||||||
|
|
||||||
### `session restore`
|
### `session restore`
|
||||||
@@ -236,7 +243,7 @@ Behavior:
|
|||||||
|
|
||||||
## `--artifacts` Selection Rules
|
## `--artifacts` Selection Rules
|
||||||
|
|
||||||
- accepted on `run`, `resume`, `run-stage`, `analyze`, and `publish`;
|
- accepted on `run`, `run-stage`, `analyze`, and `publish`;
|
||||||
- names must exist in `pipeline.scriptorium.artifacts`;
|
- names must exist in `pipeline.scriptorium.artifacts`;
|
||||||
- empty entries are invalid;
|
- empty entries are invalid;
|
||||||
- repeated names are deduplicated.
|
- repeated names are deduplicated.
|
||||||
|
|||||||
@@ -98,6 +98,12 @@ publish:
|
|||||||
- source: narratio.transcript.final_trimmed
|
- source: narratio.transcript.final_trimmed
|
||||||
dest: transcripts/final.trimmed.json
|
dest: transcripts/final.trimmed.json
|
||||||
required: true
|
required: true
|
||||||
|
- source: narratio.transcript.final_markdown
|
||||||
|
dest: transcripts/final.md
|
||||||
|
required: true
|
||||||
|
- source: narratio.transcript.final_trimmed_markdown
|
||||||
|
dest: transcripts/final.trimmed.md
|
||||||
|
required: true
|
||||||
- source: narratio.artifact.session_recap
|
- source: narratio.artifact.session_recap
|
||||||
dest: artifacts/session_recap.md
|
dest: artifacts/session_recap.md
|
||||||
required: true
|
required: true
|
||||||
@@ -138,7 +144,7 @@ Rules:
|
|||||||
| `pipeline.cache.s3_audio` | bool | No | `true` |
|
| `pipeline.cache.s3_audio` | bool | No | `true` |
|
||||||
| `pipeline.publish.enabled` | bool | No | `true` |
|
| `pipeline.publish.enabled` | bool | No | `true` |
|
||||||
| `pipeline.publish.upload_run` | bool | No | `true` |
|
| `pipeline.publish.upload_run` | bool | No | `true` |
|
||||||
| `pipeline.publish.outputs[]` | list | No | defaults to final trimmed transcript output |
|
| `pipeline.publish.outputs[]` | list | No | defaults to final trimmed JSON plus final and final-trimmed Markdown outputs |
|
||||||
| `pipeline.publish.outputs[].source` | string | Yes (per rule) | must reference built-in or configured artifact source |
|
| `pipeline.publish.outputs[].source` | string | Yes (per rule) | must reference built-in or configured artifact source |
|
||||||
| `pipeline.publish.outputs[].dest` | string | Conditional | derived if omitted and source supports derivation |
|
| `pipeline.publish.outputs[].dest` | string | Conditional | derived if omitted and source supports derivation |
|
||||||
| `pipeline.publish.outputs[].required` | bool | No | `true` |
|
| `pipeline.publish.outputs[].required` | bool | No | `true` |
|
||||||
@@ -188,6 +194,12 @@ Rules:
|
|||||||
| `pipeline.trim.bounds.render_debug` | bool | No | `false` |
|
| `pipeline.trim.bounds.render_debug` | bool | No | `false` |
|
||||||
| `pipeline.trim.bounds.render_output_path` | string | Conditional | required when `render_debug` is true |
|
| `pipeline.trim.bounds.render_output_path` | string | Conditional | required when `render_debug` is true |
|
||||||
| `pipeline.trim.seriatim.report` | bool | No | `false` |
|
| `pipeline.trim.seriatim.report` | bool | No | `false` |
|
||||||
|
| `pipeline.render.enabled` | bool | No | `true` |
|
||||||
|
| `pipeline.render.format` | string | No | `markdown` (only supported value) |
|
||||||
|
| `pipeline.render.title` | string | No | empty (falls back to `session.title` when set) |
|
||||||
|
| `pipeline.render.include_timestamps` | bool | No | `true` |
|
||||||
|
| `pipeline.render.include_segment_ids` | bool | No | `true` |
|
||||||
|
| `pipeline.render.include_metadata` | bool | No | `false` |
|
||||||
| `pipeline.scriptorium.binary` | string | No | `scriptorium` |
|
| `pipeline.scriptorium.binary` | string | No | `scriptorium` |
|
||||||
| `pipeline.scriptorium.config_path` | string | No | empty |
|
| `pipeline.scriptorium.config_path` | string | No | empty |
|
||||||
| `pipeline.scriptorium.timeout` | duration | No | `10m` |
|
| `pipeline.scriptorium.timeout` | duration | No | `10m` |
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ These docs cover what Narratio expects from external tools and what each adapter
|
|||||||
|
|
||||||
## Integration Contracts
|
## Integration Contracts
|
||||||
- `audita.md`: transcript polishing adapter (`audita process`).
|
- `audita.md`: transcript polishing adapter (`audita process`).
|
||||||
- `seriatim.md`: merge/normalize/trim adapter (`seriatim`).
|
- `seriatim.md`: merge/normalize/trim/render adapter (`seriatim`).
|
||||||
- `scriptorium.md`: artifact run/render adapter (`scriptorium run|render`).
|
- `scriptorium.md`: artifact run/render adapter (`scriptorium run|render`).
|
||||||
|
|
||||||
## Related Canonical Docs
|
## Related Canonical Docs
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
# Integration: Seriatim
|
# Integration: Seriatim
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define the Seriatim adapter contract used by `merge`, `normalize`, and `trim`.
|
Define the Seriatim adapter contract used by `merge`, `normalize`, `trim`, and `render`.
|
||||||
|
|
||||||
## Adapter Boundary
|
## Adapter Boundary
|
||||||
Interface:
|
Interface:
|
||||||
@@ -10,6 +10,7 @@ Interface:
|
|||||||
- `Run(ctx, MergeRequest)`
|
- `Run(ctx, MergeRequest)`
|
||||||
- `Normalize(ctx, NormalizeRequest)`
|
- `Normalize(ctx, NormalizeRequest)`
|
||||||
- `Trim(ctx, TrimRequest)`
|
- `Trim(ctx, TrimRequest)`
|
||||||
|
- `Render(ctx, RenderRequest)`
|
||||||
|
|
||||||
Primary implementation:
|
Primary implementation:
|
||||||
- `internal/adapters/seriatim/SubprocessRunner`
|
- `internal/adapters/seriatim/SubprocessRunner`
|
||||||
@@ -18,11 +19,13 @@ Execution modes:
|
|||||||
- `seriatim merge`
|
- `seriatim merge`
|
||||||
- `seriatim normalize`
|
- `seriatim normalize`
|
||||||
- `seriatim trim`
|
- `seriatim trim`
|
||||||
|
- `seriatim render`
|
||||||
|
|
||||||
## Request/Result Contracts
|
## Request/Result Contracts
|
||||||
- `MergeRequest`/`MergeResult`: multi-input merge to base transcript, optional report.
|
- `MergeRequest`/`MergeResult`: multi-input merge to base transcript, optional report.
|
||||||
- `NormalizeRequest`/`NormalizeResult`: transcript normalization with explicit schema.
|
- `NormalizeRequest`/`NormalizeResult`: transcript normalization with explicit schema.
|
||||||
- `TrimRequest`/`TrimResult`: transcript trimming with required keep selector.
|
- `TrimRequest`/`TrimResult`: transcript trimming with required keep selector.
|
||||||
|
- `RenderRequest`/`RenderResult`: transcript-to-markdown rendering with explicit format and render booleans.
|
||||||
|
|
||||||
Results include output/log/config paths, timing, exit code, and metadata.
|
Results include output/log/config paths, timing, exit code, and metadata.
|
||||||
|
|
||||||
@@ -36,9 +39,11 @@ Runner construction validates:
|
|||||||
Invocation fails on:
|
Invocation fails on:
|
||||||
- missing required request paths/inputs;
|
- missing required request paths/inputs;
|
||||||
- invalid normalize schema override;
|
- invalid normalize schema override;
|
||||||
|
- unsupported render format;
|
||||||
- subprocess failure;
|
- subprocess failure;
|
||||||
- invalid JSON outputs;
|
- invalid JSON outputs for merge/normalize/trim;
|
||||||
- missing `segments` array for normalize/trim transcript outputs.
|
- missing `segments` array for normalize/trim transcript outputs;
|
||||||
|
- empty render output files.
|
||||||
|
|
||||||
When report paths are provided/enabled, report files must parse as JSON.
|
When report paths are provided/enabled, report files must parse as JSON.
|
||||||
|
|
||||||
@@ -49,7 +54,7 @@ When report paths are provided/enabled, report files must parse as JSON.
|
|||||||
- adapter does not write manifests or choose stage inputs.
|
- adapter does not write manifests or choose stage inputs.
|
||||||
|
|
||||||
## Config Mapping
|
## Config Mapping
|
||||||
Config fields consumed through runner/stage wiring are under `pipeline.seriatim.*`.
|
Config fields consumed through runner/stage wiring are under `pipeline.seriatim.*` and `pipeline.render.*`.
|
||||||
|
|
||||||
Maintained examples with Seriatim config:
|
Maintained examples with Seriatim config:
|
||||||
- `examples/pipeline.full.annotated.yml`
|
- `examples/pipeline.full.annotated.yml`
|
||||||
|
|||||||
@@ -20,9 +20,10 @@ Canonical stage order from `internal/stage.All()`:
|
|||||||
4. `polish`
|
4. `polish`
|
||||||
5. `normalize`
|
5. `normalize`
|
||||||
6. `trim`
|
6. `trim`
|
||||||
7. `analyze`
|
7. `render`
|
||||||
8. `publish`
|
8. `analyze`
|
||||||
9. `notify` (placeholder)
|
9. `publish`
|
||||||
|
10. `notify` (placeholder)
|
||||||
|
|
||||||
`notify` is currently a placeholder stage with optional notifier call behavior; it has no persisted pipeline outputs.
|
`notify` is currently a placeholder stage with optional notifier call behavior; it has no persisted pipeline outputs.
|
||||||
|
|
||||||
@@ -39,5 +40,6 @@ Canonical stage order from `internal/stage.All()`:
|
|||||||
- `stage-polish.md`
|
- `stage-polish.md`
|
||||||
- `stage-normalize.md`
|
- `stage-normalize.md`
|
||||||
- `stage-trim.md`
|
- `stage-trim.md`
|
||||||
|
- `stage-render.md`
|
||||||
- `stage-analyze.md`
|
- `stage-analyze.md`
|
||||||
- `stage-publish.md`
|
- `stage-publish.md`
|
||||||
|
|||||||
@@ -14,9 +14,6 @@ Primary adapters:
|
|||||||
- `storage.ObjectStore`
|
- `storage.ObjectStore`
|
||||||
- `notify.Sender`
|
- `notify.Sender`
|
||||||
|
|
||||||
Legacy compatibility boundary:
|
|
||||||
- `storage.Backend` remains in the storage adapter package and defaults to `NoopBackend`; current pipeline stages use `storage.ObjectStore`.
|
|
||||||
|
|
||||||
## Ownership
|
## Ownership
|
||||||
Adapters own:
|
Adapters own:
|
||||||
- HTTP/subprocess/SDK argument and transport details.
|
- HTTP/subprocess/SDK argument and transport details.
|
||||||
|
|||||||
@@ -1,68 +1,115 @@
|
|||||||
# Internal: Artifacts
|
# Internal: Artifacts
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Define canonical artifact IDs, runtime catalog behavior, and source resolution rules for stage execution and publish output selection.
|
Define canonical artifact IDs, runtime catalog behavior, source resolution rules, and shared current-state mechanics used by app and previous-cache code.
|
||||||
|
|
||||||
## Built-in Source IDs
|
## Built-in Source IDs
|
||||||
|
|
||||||
- `narratio.transcript.base` -> `transcripts/base.json` (`merge`)
|
- `narratio.transcript.base` -> `transcripts/base.json` (`merge`)
|
||||||
- `narratio.transcript.polished` -> `transcripts/polished.json` (`polish`)
|
- `narratio.transcript.polished` -> `transcripts/polished.json` (`polish`)
|
||||||
- `narratio.transcript.final` -> `transcripts/final.json` (`normalize`)
|
- `narratio.transcript.final` -> `transcripts/final.json` (`normalize`)
|
||||||
- `narratio.transcript.final_trimmed` -> `transcripts/final.trimmed.json` (`trim`)
|
- `narratio.transcript.final_trimmed` -> `transcripts/final.trimmed.json` (`trim`)
|
||||||
|
- `narratio.transcript.final_markdown` -> `transcripts/final.md` (`render`)
|
||||||
|
- `narratio.transcript.final_trimmed_markdown` -> `transcripts/final.trimmed.md` (`render`)
|
||||||
- `narratio.bounds.session` -> `artifacts/session_bounds.json` (`trim`)
|
- `narratio.bounds.session` -> `artifacts/session_bounds.json` (`trim`)
|
||||||
|
|
||||||
## Configured and Previous-Session Sources
|
## Configured and Previous-Session Sources
|
||||||
- Configured artifact source ID: `narratio.artifact.<artifact_key>`
|
|
||||||
- Previous-session source ID: `narratio.previous_session.artifact.<artifact_key>`
|
|
||||||
|
|
||||||
Configured and previous-session source IDs are validated by strict regex rules.
|
- configured source ID format: `narratio.artifact.<artifact_key>`
|
||||||
|
- previous-session source ID format: `narratio.previous_session.artifact.<artifact_key>`
|
||||||
|
|
||||||
|
Both formats are validated by strict source-policy rules.
|
||||||
|
|
||||||
## Runtime Catalog
|
## Runtime Catalog
|
||||||
|
|
||||||
`ArtifactCatalog` tracks:
|
`ArtifactCatalog` tracks:
|
||||||
- `planned`: source registered for run context.
|
|
||||||
- `executable`: selected and enabled for analyze execution.
|
- `planned`: source registered for run context;
|
||||||
- `available`: local file exists and validated.
|
- `executable`: selected and enabled for analyze execution;
|
||||||
|
- `available`: local file exists and validates;
|
||||||
- `provenance`: availability source.
|
- `provenance`: availability source.
|
||||||
|
|
||||||
Current provenance values:
|
Current provenance values:
|
||||||
|
|
||||||
- `generated.current_analyze_run`
|
- `generated.current_analyze_run`
|
||||||
- `filesystem.disabled_artifact_output`
|
- `filesystem.disabled_artifact_output`
|
||||||
- `manifest.inputs.previous_cache`
|
- `manifest.inputs.previous_cache`
|
||||||
- `current_session.previous_cache`
|
- `current_session.previous_cache`
|
||||||
|
|
||||||
## Resolution Rules
|
## Resolution Rules
|
||||||
|
|
||||||
Built-ins:
|
Built-ins:
|
||||||
|
|
||||||
1. manifest producer outputs (when present)
|
1. manifest producer outputs (when present)
|
||||||
2. canonical session path fallback
|
2. canonical session-path fallback
|
||||||
|
|
||||||
Configured sources (`narratio.artifact.*`):
|
Configured sources (`narratio.artifact.*`):
|
||||||
|
|
||||||
- resolve only through runtime catalog availability.
|
- resolve only through runtime catalog availability.
|
||||||
|
|
||||||
Previous-session sources (`narratio.previous_session.artifact.*`):
|
Previous-session sources (`narratio.previous_session.artifact.*`):
|
||||||
- resolve only from local `previous/` cache state.
|
|
||||||
- prefer manifest-backed previous input paths.
|
- resolve only from local `previous/` cache state;
|
||||||
|
- prefer manifest-backed previous-input paths;
|
||||||
- fallback to existing previous-cache filesystem paths.
|
- fallback to existing previous-cache filesystem paths.
|
||||||
|
|
||||||
Validation by content type:
|
Validation by content type:
|
||||||
- transcript built-ins: JSON with top-level `segments` array.
|
|
||||||
- bounds built-in: valid JSON.
|
- transcript JSON built-ins: JSON with top-level `segments` array;
|
||||||
|
- transcript Markdown built-ins: non-empty text file;
|
||||||
|
- bounds built-in: valid JSON;
|
||||||
- configured/previous-session artifact files: non-empty text file.
|
- configured/previous-session artifact files: non-empty text file.
|
||||||
|
|
||||||
## Previous Requirement Collection
|
## Previous Requirement Collection
|
||||||
|
|
||||||
`CollectPreviousArtifactRequirements`:
|
`CollectPreviousArtifactRequirements`:
|
||||||
|
|
||||||
- scans enabled configured artifacts only;
|
- scans enabled configured artifacts only;
|
||||||
- extracts only canonical previous-session sources;
|
- extracts only canonical previous-session sources;
|
||||||
- deduplicates by artifact key;
|
- deduplicates by artifact key;
|
||||||
- merges required/optional (required wins);
|
- merges required and optional references (required wins);
|
||||||
- returns deterministic ordering and source locations.
|
- returns deterministic ordering and source locations.
|
||||||
|
|
||||||
|
## Current-State Helpers
|
||||||
|
|
||||||
|
Artifacts package owns shared remote current-state loading mechanics used by restore, status/validate checks, and previous-cache planning.
|
||||||
|
|
||||||
|
Core helpers:
|
||||||
|
|
||||||
|
- `LoadCurrentRunPointer`
|
||||||
|
- `LoadCurrentManifest`
|
||||||
|
- `LoadCurrentState`
|
||||||
|
- `ValidateCurrentStateIdentity`
|
||||||
|
|
||||||
|
Typed missing-state errors:
|
||||||
|
|
||||||
|
- `CurrentRunPointerMissingError` (`ErrCurrentRunPointerMissing`)
|
||||||
|
- `CurrentManifestMissingError` (`ErrCurrentManifestMissing`)
|
||||||
|
|
||||||
|
Identity validation supports caller-provided expectations:
|
||||||
|
|
||||||
|
- expected campaign;
|
||||||
|
- expected session ID;
|
||||||
|
- expected run ID, or pointer/manifest run-ID consistency check.
|
||||||
|
|
||||||
|
Caller policy is intentionally outside artifacts helpers:
|
||||||
|
|
||||||
|
- some callers fail on missing current state;
|
||||||
|
- some callers downgrade missing state to status/findings;
|
||||||
|
- some callers skip optional behavior when state is missing.
|
||||||
|
|
||||||
## Key Path Helpers
|
## Key Path Helpers
|
||||||
`internal/artifacts/paths.go` defines canonical helpers for:
|
|
||||||
|
`internal/artifacts/paths.go` and S3-key helpers define canonical helpers for:
|
||||||
|
|
||||||
- session/work/run paths;
|
- session/work/run paths;
|
||||||
- previous-cache paths;
|
- previous-cache paths;
|
||||||
- spool/cache paths;
|
- spool/cache paths;
|
||||||
- S3 key layout helpers for session/run/current pointers.
|
- S3 session/run/current-state key layout.
|
||||||
|
|
||||||
## Invariants
|
## Invariants
|
||||||
- Source ID formats are stable contracts.
|
|
||||||
- Resolution is deterministic and manifest-aware.
|
- source ID formats are stable contracts;
|
||||||
- Previous-session source resolution does not call remote storage in `analyze`; remote hydration is `prepare` responsibility.
|
- artifact resolution is deterministic and manifest-aware;
|
||||||
|
- previous-session source resolution in `analyze` is local-only;
|
||||||
|
- remote current-state key construction remains centralized in artifacts helpers.
|
||||||
|
|||||||
@@ -1,65 +1,84 @@
|
|||||||
# Internal: Command Restore
|
# Internal: Command Restore
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
Document the implemented `narratio session restore` command contract:
|
Define the implemented `narratio session restore` command contract:
|
||||||
|
|
||||||
- committed remote current-state discovery;
|
- committed remote current-state discovery;
|
||||||
- deterministic restore plan classification;
|
- deterministic restore planning;
|
||||||
- safe local install semantics;
|
- safe local install semantics;
|
||||||
- durable restore reporting.
|
- durable restore reporting.
|
||||||
|
|
||||||
## Discovery Contract
|
## Discovery Contract
|
||||||
Restore discovers remote committed state using:
|
|
||||||
- `current/run_id.txt` (required, non-empty)
|
|
||||||
- `current/manifest.json` (required, decodable)
|
|
||||||
|
|
||||||
Discovered manifest identity must match requested `session_id` and `campaign`.
|
Restore resolves remote committed state from the session publish current pointers:
|
||||||
|
|
||||||
## Plan Contract
|
- `current/run_id.txt` (required, non-empty);
|
||||||
Planner actions:
|
- `current/manifest.json` (required, decodable).
|
||||||
- `download`
|
|
||||||
- `skip_same`
|
Current-state discovery uses shared artifacts-level mechanics and validates identity against the resolved request config:
|
||||||
- `conflict`
|
|
||||||
|
- campaign must match;
|
||||||
|
- session ID must match.
|
||||||
|
|
||||||
|
Restore treats any missing or invalid remote current state as a command error.
|
||||||
|
|
||||||
|
## Planning Contract
|
||||||
|
|
||||||
|
Restore planner action kinds:
|
||||||
|
|
||||||
|
- `download`;
|
||||||
|
- `skip_same`;
|
||||||
|
- `conflict`.
|
||||||
|
|
||||||
|
Planner behavior:
|
||||||
|
|
||||||
Plan behavior:
|
|
||||||
- remote list scope is the resolved session prefix;
|
- remote list scope is the resolved session prefix;
|
||||||
- mapping to local paths is traversal-safe;
|
- remote-to-local mapping is traversal-safe;
|
||||||
- actions are sorted deterministically by local relative path.
|
- actions are sorted deterministically by local relative path.
|
||||||
|
|
||||||
Restore scope from current remote state:
|
Restore scope from current remote state:
|
||||||
- include `manifest.json`
|
|
||||||
- include `transcripts/**`
|
- include `manifest.json`;
|
||||||
- include `artifacts/**`
|
- include `transcripts/**`;
|
||||||
- include `audio/**` only with `--include-audio`
|
- include `artifacts/**`;
|
||||||
|
- include `audio/**` only with `--include-audio`.
|
||||||
|
|
||||||
Explicit exclusions from current remote state mapping:
|
Explicit exclusions from current remote state mapping:
|
||||||
- `current/**`
|
|
||||||
- `runs/**`
|
|
||||||
- `logs/**`
|
|
||||||
- `reports/**`
|
|
||||||
- `config/**`
|
|
||||||
- `inputs/**`
|
|
||||||
- `previous/**`
|
|
||||||
|
|
||||||
Previous-cache restore files are planned separately through `previouscache.BuildPlan` when configured previous-session requirements exist.
|
- `current/**`;
|
||||||
|
- `runs/**`;
|
||||||
|
- `logs/**`;
|
||||||
|
- `reports/**`;
|
||||||
|
- `config/**`;
|
||||||
|
- `inputs/**`;
|
||||||
|
- `previous/**`.
|
||||||
|
|
||||||
|
Previous-cache files are planned separately through `previouscache.BuildPlan` when configured previous-session requirements exist.
|
||||||
|
|
||||||
## Execution Contract
|
## Execution Contract
|
||||||
|
|
||||||
|
Execution order and safety:
|
||||||
|
|
||||||
- non-manifest downloads happen before manifest install;
|
- non-manifest downloads happen before manifest install;
|
||||||
- `manifest.json` is installed last;
|
- `manifest.json` installs last;
|
||||||
- downloads use sibling temp files + atomic rename;
|
- downloads use sibling temp files plus atomic rename;
|
||||||
- manifest replacement is validated before rename;
|
- manifest replacement is validated before rename;
|
||||||
- failed installs do not roll back previously written files.
|
- failed installs do not roll back files already written in the same execution.
|
||||||
|
|
||||||
Audio restore path:
|
Audio restore path:
|
||||||
|
|
||||||
- uses `audio.MaterializeS3Audio`;
|
- uses `audio.MaterializeS3Audio`;
|
||||||
- integrates spool and S3 audio cache paths;
|
- integrates spool and S3 audio cache paths;
|
||||||
- supports cache hit reuse without object redownload.
|
- supports cache-hit reuse without object redownload.
|
||||||
|
|
||||||
## Reporting Contract
|
## Reporting Contract
|
||||||
- dry-run: summary only (no writes).
|
|
||||||
|
- `--dry-run`: prints summary only; no local writes.
|
||||||
- non-dry-run: writes `reports/restore-latest.json`.
|
- non-dry-run: writes `reports/restore-latest.json`.
|
||||||
- report captures plan counts, action status, and execution failures.
|
- report includes plan counts, per-action status, and execution failures.
|
||||||
|
|
||||||
## Invariants
|
## Invariants
|
||||||
- restore uses only committed remote current state as authority.
|
|
||||||
- `current/run_id.txt` is the remote commit marker.
|
- restore uses committed remote current state as authority;
|
||||||
- restore is a standalone command and does not run stages.
|
- `current/run_id.txt` is the remote publish commit marker;
|
||||||
|
- restore does not execute pipeline stages.
|
||||||
|
|||||||
@@ -30,6 +30,7 @@ Supported source families:
|
|||||||
## Failure Semantics
|
## Failure Semantics
|
||||||
- required missing configured/previous-session inputs fail.
|
- required missing configured/previous-session inputs fail.
|
||||||
- missing required previous-session source includes prepare rerun guidance.
|
- missing required previous-session source includes prepare rerun guidance.
|
||||||
|
- missing required `narratio.transcript.final_markdown` or `narratio.transcript.final_trimmed_markdown` inputs includes render rerun guidance.
|
||||||
- dependency cycles or unavailable required dependencies fail.
|
- dependency cycles or unavailable required dependencies fail.
|
||||||
- adapter validation failures fail stage.
|
- adapter validation failures fail stage.
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,7 @@
|
|||||||
Upload run/session outputs to object storage and atomically advance remote current state.
|
Upload run/session outputs to object storage and atomically advance remote current state.
|
||||||
|
|
||||||
## Inputs
|
## Inputs
|
||||||
- successful prerequisite stages: `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, `analyze`
|
- successful prerequisite stages: `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, `render`, `analyze`
|
||||||
- run root `runs/{run_id}/**`
|
- run root `runs/{run_id}/**`
|
||||||
- publish output rules (`pipeline.publish.outputs`)
|
- publish output rules (`pipeline.publish.outputs`)
|
||||||
- effective publish locks (static + remote merged lock set)
|
- effective publish locks (static + remote merged lock set)
|
||||||
|
|||||||
29
docs/internal/stage-render.md
Normal file
29
docs/internal/stage-render.md
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
# Stage: render
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
Render Markdown transcript artifacts from normalized JSON transcripts via Seriatim.
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
- `narratio.transcript.final` (`transcripts/final.json`)
|
||||||
|
- `narratio.transcript.final_trimmed` (`transcripts/final.trimmed.json`)
|
||||||
|
|
||||||
|
## Outputs
|
||||||
|
- `narratio.transcript.final_markdown` -> `transcripts/final.md`
|
||||||
|
- `narratio.transcript.final_trimmed_markdown` -> `transcripts/final.trimmed.md`
|
||||||
|
|
||||||
|
## Key Behavior
|
||||||
|
- uses `pipeline.render` settings (enabled/format/title/booleans).
|
||||||
|
- resolves inputs manifest-first, then canonical fallback.
|
||||||
|
- writes run-local outputs first, then materializes canonical session outputs.
|
||||||
|
- records input provenance, output paths, adapter metadata, logs, and generated config refs.
|
||||||
|
- skips with stage metadata when `pipeline.render.enabled=false`.
|
||||||
|
|
||||||
|
## Failure Semantics
|
||||||
|
- missing normalized input fails with normalize rerun guidance.
|
||||||
|
- missing trimmed input fails with trim rerun guidance.
|
||||||
|
- adapter/subprocess failure fails stage.
|
||||||
|
- empty render output files fail validation.
|
||||||
|
|
||||||
|
## Invariants
|
||||||
|
- only `format: markdown` is supported.
|
||||||
|
- render stage owns production of built-in Markdown transcript sources.
|
||||||
@@ -29,9 +29,6 @@ S3 constructor behavior:
|
|||||||
- `Upload` streams local file and returns remote metadata.
|
- `Upload` streams local file and returns remote metadata.
|
||||||
- `Exists` maps not-found responses to `false`.
|
- `Exists` maps not-found responses to `false`.
|
||||||
|
|
||||||
## Legacy Compatibility Interface
|
|
||||||
`storage.Backend` (with `ArchiveRequest`) remains as compatibility surface with `NoopBackend`; it is not used by current stage execution.
|
|
||||||
|
|
||||||
## Invariants
|
## Invariants
|
||||||
- storage layer is stateless regarding manifest/stage progression.
|
- storage layer is stateless regarding manifest/stage progression.
|
||||||
- publish ordering semantics are owned by stage/app code, not storage adapters.
|
- publish ordering semantics are owned by stage/app code, not storage adapters.
|
||||||
|
|||||||
@@ -40,7 +40,7 @@ Run-local outputs are materialized back into canonical session paths before stag
|
|||||||
`artifacts.LocalStore` enforces single-writer session lock via `.lock` file (`ErrLockConflict` on contention).
|
`artifacts.LocalStore` enforces single-writer session lock via `.lock` file (`ErrLockConflict` on contention).
|
||||||
|
|
||||||
## Cleanup Semantics
|
## Cleanup Semantics
|
||||||
Automatic post-publish cleanup (`runPostArchiveCleanup`):
|
Automatic post-publish cleanup:
|
||||||
- only runs when publish actually executed and succeeded;
|
- only runs when publish actually executed and succeeded;
|
||||||
- requires `uploaded=true` and `current_pointer_written=true` metadata;
|
- requires `uploaded=true` and `current_pointer_written=true` metadata;
|
||||||
- respects `pipeline.spool.delete_audio_after_publish` and `pipeline.workspace.cleanup_after_publish`;
|
- respects `pipeline.spool.delete_audio_after_publish` and `pipeline.workspace.cleanup_after_publish`;
|
||||||
|
|||||||
@@ -4,33 +4,6 @@ Operator workflow for running, recovering, and publishing Narratio sessions.
|
|||||||
|
|
||||||
For command syntax, see [docs/cli.md](./cli.md). For field-level config, see [docs/config.md](./config.md).
|
For command syntax, see [docs/cli.md](./cli.md). For field-level config, see [docs/config.md](./config.md).
|
||||||
|
|
||||||
## Standard Session Workflow
|
|
||||||
|
|
||||||
1. Select pipeline/campaign/session config.
|
|
||||||
2. Validate session readiness:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio session validate 2026-04-04
|
|
||||||
```
|
|
||||||
|
|
||||||
3. (Optional) inspect stage decisions:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio session plan 2026-04-04
|
|
||||||
```
|
|
||||||
|
|
||||||
4. Run the pipeline:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio run 2026-04-04
|
|
||||||
```
|
|
||||||
|
|
||||||
5. Check state:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
narratio session status 2026-04-04
|
|
||||||
```
|
|
||||||
|
|
||||||
## Campaign and Session Selection
|
## Campaign and Session Selection
|
||||||
|
|
||||||
Campaign selection priority:
|
Campaign selection priority:
|
||||||
@@ -63,7 +36,34 @@ narratio session init 2026-04-04 --remote --force
|
|||||||
|
|
||||||
If `campaign.yml` sets `session_template_file`, `session init` renders it. Template variables must resolve to concrete values.
|
If `campaign.yml` sets `session_template_file`, `session init` renders it. Template variables must resolve to concrete values.
|
||||||
|
|
||||||
## Stage Execution and Resume Behavior
|
## Standard Session Workflow
|
||||||
|
|
||||||
|
1. Select pipeline/campaign/session config.
|
||||||
|
2. Validate session readiness:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session validate 2026-04-04
|
||||||
|
```
|
||||||
|
|
||||||
|
3. (Optional) inspect stage decisions:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session plan 2026-04-04
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Run the pipeline:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio run 2026-04-04
|
||||||
|
```
|
||||||
|
|
||||||
|
5. Check state:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session status 2026-04-04
|
||||||
|
```
|
||||||
|
|
||||||
|
## Stage Execution and Continuation Behavior
|
||||||
|
|
||||||
Canonical stage order:
|
Canonical stage order:
|
||||||
|
|
||||||
@@ -73,14 +73,15 @@ Canonical stage order:
|
|||||||
4. `polish`
|
4. `polish`
|
||||||
5. `normalize`
|
5. `normalize`
|
||||||
6. `trim`
|
6. `trim`
|
||||||
7. `analyze`
|
7. `render`
|
||||||
8. `publish`
|
8. `analyze`
|
||||||
9. `notify`
|
9. `publish`
|
||||||
|
10. `notify`
|
||||||
|
|
||||||
Execution rules:
|
Execution rules:
|
||||||
|
|
||||||
- succeeded stages are skipped unless `--force` is set;
|
- succeeded stages are skipped unless `--force` is set;
|
||||||
- `resume` starts at first non-succeeded stage;
|
- `run` continues interrupted or partially completed sessions by running non-succeeded stages;
|
||||||
- force rerunning a succeeded upstream stage marks succeeded downstream stages as `stale`.
|
- force rerunning a succeeded upstream stage marks succeeded downstream stages as `stale`.
|
||||||
|
|
||||||
Single-stage execution:
|
Single-stage execution:
|
||||||
@@ -91,7 +92,7 @@ narratio run-stage normalize 2026-04-04 --force
|
|||||||
|
|
||||||
## Artifact Selection
|
## Artifact Selection
|
||||||
|
|
||||||
`--artifacts` can be used on `run`, `resume`, `run-stage`, `analyze`, and `publish`.
|
`--artifacts` can be used on `run`, `run-stage`, `analyze`, and `publish`.
|
||||||
|
|
||||||
Selection behavior:
|
Selection behavior:
|
||||||
|
|
||||||
@@ -244,6 +245,7 @@ Rules:
|
|||||||
## Operational Caveats
|
## Operational Caveats
|
||||||
|
|
||||||
- Local and S3 audio modes are mutually exclusive.
|
- Local and S3 audio modes are mutually exclusive.
|
||||||
- Publish requires prerequisite stages through analyze to be succeeded.
|
- Publish requires prerequisite stages through `render` and `analyze` to be succeeded.
|
||||||
|
- Markdown publish defaults require render outputs (`transcripts/final.md` and `transcripts/final.trimmed.md`).
|
||||||
- Restore requires configured object storage and committed remote current state.
|
- Restore requires configured object storage and committed remote current state.
|
||||||
- Storage-backed commands load filesystem secrets before object-store initialization.
|
- Storage-backed commands load filesystem secrets before object-store initialization.
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ Canonical contributor workflow and engineering conventions for implemented Narra
|
|||||||
## Repository layout
|
## Repository layout
|
||||||
|
|
||||||
- `cmd/narratio/`: CLI entrypoint.
|
- `cmd/narratio/`: CLI entrypoint.
|
||||||
- `internal/app/`: command handlers, plan/run/resume orchestration, cleanup gates, secrets loading.
|
- `internal/app/`: command handlers, run/stage orchestration, cleanup gates, secrets loading.
|
||||||
- `internal/config/`: strict YAML loading, defaults, and validation.
|
- `internal/config/`: strict YAML loading, defaults, and validation.
|
||||||
- `internal/stage/`: stage implementations and stage registry/order.
|
- `internal/stage/`: stage implementations and stage registry/order.
|
||||||
- `internal/adapters/`: external boundary adapters (WhisperX, Seriatim, Audita, Scriptorium, storage, notify).
|
- `internal/adapters/`: external boundary adapters (WhisperX, Seriatim, Audita, Scriptorium, storage, notify).
|
||||||
|
|||||||
@@ -1,105 +0,0 @@
|
|||||||
# Documentation Pass: Stage 1 Audit
|
|
||||||
|
|
||||||
Status: Completed (2026-05-23)
|
|
||||||
|
|
||||||
## Scope Reviewed
|
|
||||||
|
|
||||||
- All non-policy documentation files under `docs/`
|
|
||||||
- `README.md`
|
|
||||||
- Documentation references to maintained `examples/` files
|
|
||||||
- Documentation-related expectations in tests under `internal/**`
|
|
||||||
|
|
||||||
## File Inventory and Canonical Scope
|
|
||||||
|
|
||||||
| File | Intended audience | Canonical scope (per policy) | Primary source-of-truth anchors |
|
|
||||||
| --- | --- | --- | --- |
|
|
||||||
| `README.md` | Users, operators | Project orientation and links | `cmd/narratio`, `internal/app/commands.go`, docs index files |
|
|
||||||
| `docs/cli.md` | Users, operators | CLI syntax, flags, command workflows | `internal/app/*.go`, `internal/app/*_test.go` |
|
|
||||||
| `docs/config.md` | Operators, advanced users | Config discovery, schema, defaults, examples | `internal/config/*.go`, `internal/config/*_test.go`, `examples/*` |
|
|
||||||
| `docs/operations.md` | Operators | Run/resume/publish/restore/cleanup workflows | `internal/app/runner.go`, `internal/app/restore*.go`, `internal/stage/archive.go`, `internal/artifacts/*.go` |
|
|
||||||
| `docs/troubleshooting.md` | Operators | Failure diagnosis and safe fixes | `internal/app`, `internal/stage`, related tests |
|
|
||||||
| `docs/internal/README.md` | Developers, LLM coding agents | Internal docs index and scope boundaries | `docs/internal/*.md`, policy docs |
|
|
||||||
| `docs/internal/adapters.md` | Developers, LLM coding agents | Adapter boundaries and ownership | `internal/adapters/*`, `internal/stage/*` |
|
|
||||||
| `docs/internal/artifacts.md` | Developers, LLM coding agents | Artifact catalog and source resolution contracts | `internal/artifacts/*`, `internal/stage/analyze.go`, `internal/stage/prepare_previous.go` |
|
|
||||||
| `docs/internal/command-restore.md` | Developers, LLM coding agents | Restore command architecture and contracts | `internal/app/restore*.go`, `internal/app/restore*_test.go` |
|
|
||||||
| `docs/internal/manifest.md` | Developers, LLM coding agents | Session/run manifest contracts and transitions | `internal/manifest/*`, `internal/app/runner.go`, `internal/stage/*` |
|
|
||||||
| `docs/internal/stage-prepare.md` | Developers, LLM coding agents | Prepare stage IO and invariants | `internal/stage/prepare.go`, `internal/stage/prepare*_test.go` |
|
|
||||||
| `docs/internal/stage-transcribe.md` | Developers, LLM coding agents | Transcribe stage IO and invariants | `internal/stage/transcribe.go`, `internal/stage/transcribe_test.go` |
|
|
||||||
| `docs/internal/stage-merge.md` | Developers, LLM coding agents | Merge stage IO and invariants | `internal/stage/merge.go`, `internal/stage/merge_test.go` |
|
|
||||||
| `docs/internal/stage-polish.md` | Developers, LLM coding agents | Polish stage IO and invariants | `internal/stage/polish.go`, `internal/stage/polish_test.go` |
|
|
||||||
| `docs/internal/stage-normalize.md` | Developers, LLM coding agents | Normalize stage IO and invariants | `internal/stage/normalize.go`, `internal/stage/normalize_test.go` |
|
|
||||||
| `docs/internal/stage-trim.md` | Developers, LLM coding agents | Trim stage IO and invariants | `internal/stage/trim.go`, `internal/stage/trim_test.go` |
|
|
||||||
| `docs/internal/stage-analyze.md` | Developers, LLM coding agents | Analyze stage artifact execution and selection | `internal/stage/analyze.go`, `internal/stage/analyze_test.go` |
|
|
||||||
| `docs/internal/stage-publish.md` | Developers, LLM coding agents | Publish-stage commit/upload invariants | `internal/stage/archive.go`, `internal/stage/archive_test.go` |
|
|
||||||
| `docs/internal/storage.md` | Developers, LLM coding agents | Storage adapter contracts and semantics | `internal/adapters/storage/*`, `internal/app/object_store.go` |
|
|
||||||
| `docs/internal/workspace.md` | Developers, LLM coding agents | Local workspace/session/run path model | `internal/artifacts/*`, `internal/app/runner.go`, `internal/stage/run_local.go` |
|
|
||||||
| `docs/integrations/README.md` | Developers, LLM coding agents | Integration docs index | `docs/integrations/*.md` |
|
|
||||||
| `docs/integrations/audita.md` | Developers, integration maintainers | Audita adapter contract | `internal/adapters/audita/*`, `internal/stage/polish.go` |
|
|
||||||
| `docs/integrations/seriatim.md` | Developers, integration maintainers | Seriatim adapter contract | `internal/adapters/seriatim/*`, `internal/stage/merge.go`, `internal/stage/normalize.go`, `internal/stage/trim.go` |
|
|
||||||
| `docs/integrations/scriptorium.md` | Developers, integration maintainers | Scriptorium adapter contract | `internal/adapters/scriptorium/*`, `internal/stage/analyze.go`, `internal/stage/trim.go` |
|
|
||||||
| `docs/roadmap/documentation.md` | Developers, maintainers | Planning and implementation sequencing for documentation pass | N/A (planning artifact) |
|
|
||||||
| `docs/roadmap/documentation-stage1-audit.md` | Developers, maintainers | Stage-1 inventory and source-of-truth audit record | N/A (planning artifact) |
|
|
||||||
|
|
||||||
## Source-of-Truth Mapping Summary
|
|
||||||
|
|
||||||
- CLI behaviors and command names are grounded in `internal/app/commands.go` and command handlers in `internal/app/*.go`.
|
|
||||||
- Stage order and canonical stage names are grounded in `internal/stage/placeholders.go` (`prepare` -> `transcribe` -> `merge` -> `polish` -> `normalize` -> `trim` -> `analyze` -> `publish` -> `notify`).
|
|
||||||
- Publish behavior and current-pointer commit semantics are grounded in `internal/stage/archive.go`.
|
|
||||||
- Config schema/defaults/validation are grounded in `internal/config/*`.
|
|
||||||
- Local/remote paths, publish keys, and workspace layout are grounded in `internal/artifacts/*`.
|
|
||||||
- Restore behavior and report contracts are grounded in `internal/app/restore*.go`.
|
|
||||||
- Maintained examples and schema compatibility are grounded in `examples/*` plus `internal/config/load_validate_test.go` (`TestExamplesLoadAndValidate`).
|
|
||||||
|
|
||||||
## Findings
|
|
||||||
|
|
||||||
### Broken or stale references
|
|
||||||
|
|
||||||
1. `README.md` linked to non-existent files:
|
|
||||||
- `docs/development.md`
|
|
||||||
- `docs/architecture.md`
|
|
||||||
2. `docs/internal/README.md` and `docs/integrations/README.md` linked to non-existent path:
|
|
||||||
- `docs/documentation/policy.md`
|
|
||||||
|
|
||||||
Stage-1 fix applied:
|
|
||||||
- Updated those links to existing policy docs under `docs/policy/`.
|
|
||||||
|
|
||||||
### Stale terminology sweep
|
|
||||||
|
|
||||||
Sweep terms used: `archive`, `promote`, `promoted`, `promote_artifacts`, `run-stage archive`.
|
|
||||||
|
|
||||||
Findings:
|
|
||||||
- User-facing docs in scope did not show obvious stale command examples requiring immediate correction.
|
|
||||||
- Internal code and tests still contain historical `archive` identifiers while user-facing command/stage naming is `publish` (for example, `internal/stage/archive.go` type names). This is acceptable for now but should be normalized deliberately, not incidentally.
|
|
||||||
|
|
||||||
Stage-1 fix applied:
|
|
||||||
- Updated clearly stale publish-related wording in test expectation messages/comments:
|
|
||||||
- `internal/app/commands_test.go`
|
|
||||||
- `internal/app/operator_helpers_test.go`
|
|
||||||
|
|
||||||
### Example path validation
|
|
||||||
|
|
||||||
- All `examples/...` paths referenced from non-policy docs resolve to existing files.
|
|
||||||
- `internal/config/load_validate_test.go` includes `TestExamplesLoadAndValidate` and points to current example files.
|
|
||||||
|
|
||||||
### Roadmap leakage into current-behavior docs
|
|
||||||
|
|
||||||
- No obvious roadmap-only behavior leakage found in non-roadmap docs during this sweep.
|
|
||||||
|
|
||||||
### Duplicate content and scope drift
|
|
||||||
|
|
||||||
- No severe duplication requiring immediate rewrite in this stage.
|
|
||||||
- Existing docs still need full content rewrite for 1.0 readiness in later stages (user/operator first, then internal/integrations), as planned.
|
|
||||||
|
|
||||||
### Canonical-home inconsistency to resolve in rewrite stages
|
|
||||||
|
|
||||||
- Policy canonical-home language names `docs/architecture.md` and `docs/development.md`, while current repository stores those policy documents under `docs/policy/`.
|
|
||||||
- Stage 1 preserves repository behavior by fixing broken links to existing files. Later rewrite stages should converge canonical-home paths and references consistently across docs.
|
|
||||||
|
|
||||||
## Stage-1 Completion Check
|
|
||||||
|
|
||||||
Completed for this stage:
|
|
||||||
- Full non-policy file inventory with audience and scope mapping.
|
|
||||||
- Source-of-truth crosswalk to code/tests.
|
|
||||||
- Stale-term, link, and example-path sweeps.
|
|
||||||
- Documentation-related stale test wording corrections.
|
|
||||||
- Minimal fixes only; broad rewrites intentionally deferred.
|
|
||||||
@@ -1,237 +0,0 @@
|
|||||||
# Roadmap: 1.0 Documentation Pass
|
|
||||||
|
|
||||||
Status: Completed (2026-05-23)
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Prepare Narratio documentation for a 1.0 release by reviewing and rewriting
|
|
||||||
every non-policy document under `docs/` against the implemented codebase.
|
|
||||||
|
|
||||||
The finished documentation set should be accurate, concise, complete for its
|
|
||||||
audience, and compliant with:
|
|
||||||
|
|
||||||
- `docs/policy/documentation.md`
|
|
||||||
- `docs/policy/architecture.md`
|
|
||||||
- `docs/policy/development.md`
|
|
||||||
|
|
||||||
Do not modify files under `docs/policy/` during this pass.
|
|
||||||
|
|
||||||
Current behavior belongs in canonical docs. Future, planned, aspirational, or
|
|
||||||
unimplemented behavior belongs only under `docs/roadmap/`.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- `docs/*.md`
|
|
||||||
- `docs/internal/*.md`
|
|
||||||
- `docs/integrations/*.md`
|
|
||||||
- `docs/roadmap/*.md`
|
|
||||||
- documentation references to files under `examples/`
|
|
||||||
- test expectation fixes when the documentation review exposes stale or
|
|
||||||
incorrect doc/example/path expectations
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- product/runtime code changes
|
|
||||||
- feature implementation
|
|
||||||
- edits under `docs/policy/`
|
|
||||||
- adding roadmap behavior to current-behavior docs before that behavior is
|
|
||||||
implemented
|
|
||||||
|
|
||||||
## Implementation Stages
|
|
||||||
|
|
||||||
### Stage 1: Inventory and Source-of-Truth Audit
|
|
||||||
|
|
||||||
Status: Completed (2026-05-23)
|
|
||||||
|
|
||||||
Create a file-by-file inventory of all non-policy docs before rewriting.
|
|
||||||
|
|
||||||
Implementation requirements:
|
|
||||||
|
|
||||||
- List every non-policy documentation file and assign its intended audience.
|
|
||||||
- Identify each document's canonical scope using `docs/policy/documentation.md`.
|
|
||||||
- Compare docs against the current code and tests, especially:
|
|
||||||
- `internal/app`
|
|
||||||
- `internal/config`
|
|
||||||
- `internal/stage`
|
|
||||||
- `internal/artifacts`
|
|
||||||
- `examples`
|
|
||||||
- relevant tests under `internal/**`
|
|
||||||
- Record stale terminology, broken links, stale example paths, duplicate
|
|
||||||
content, and roadmap-only behavior that leaked into current-behavior docs.
|
|
||||||
- Record stale test expectations related to docs, examples, paths, or command
|
|
||||||
text.
|
|
||||||
- Do not rewrite content in this stage except obvious broken links or test
|
|
||||||
corrections needed to make documentation validation meaningful.
|
|
||||||
|
|
||||||
Acceptance criteria:
|
|
||||||
|
|
||||||
- The rewrite has a concrete file inventory and source-of-truth map.
|
|
||||||
- The team knows which docs are canonical and which should link elsewhere.
|
|
||||||
- Known stale terms and broken references are identified before broad edits.
|
|
||||||
|
|
||||||
### Stage 2: User and Operator Docs
|
|
||||||
|
|
||||||
Status: Completed (2026-05-23)
|
|
||||||
|
|
||||||
Rewrite the user-facing and operator-facing docs first.
|
|
||||||
|
|
||||||
Implementation requirements:
|
|
||||||
|
|
||||||
- Rewrite these docs as fresh, concise current-behavior references:
|
|
||||||
- `README.md`, if present
|
|
||||||
- `docs/cli.md`
|
|
||||||
- `docs/config.md`
|
|
||||||
- `docs/operations.md`
|
|
||||||
- `docs/troubleshooting.md`
|
|
||||||
- Verify every command, flag, config field, discovery rule, path, and workflow
|
|
||||||
against implemented behavior.
|
|
||||||
- Cover implemented 1.0 behavior, including:
|
|
||||||
- campaign registry selection;
|
|
||||||
- concrete session loading and template-driven `session init`;
|
|
||||||
- session-oriented helper commands;
|
|
||||||
- clean, restore, analyze, and publish workflows;
|
|
||||||
- artifact selection behavior;
|
|
||||||
- locks and published output behavior;
|
|
||||||
- workspace, spool, and cache behavior;
|
|
||||||
- secrets loading and S3-backed operation.
|
|
||||||
- Keep examples short and link to maintained files under `examples/` instead
|
|
||||||
of duplicating large config blocks.
|
|
||||||
- Fix tests only when they assert stale doc paths, example paths, command
|
|
||||||
names, or current-behavior text.
|
|
||||||
|
|
||||||
Acceptance criteria:
|
|
||||||
|
|
||||||
- User/operator docs are task-oriented and match actual CLI/config behavior.
|
|
||||||
- Current-behavior docs do not depend on roadmaps for normal usage.
|
|
||||||
- No current-behavior doc describes unimplemented roadmap items.
|
|
||||||
|
|
||||||
### Stage 3: Internal Developer Docs
|
|
||||||
|
|
||||||
Status: Completed (2026-05-23)
|
|
||||||
|
|
||||||
Rewrite implemented internal component docs after public docs stabilize.
|
|
||||||
|
|
||||||
Implementation requirements:
|
|
||||||
|
|
||||||
- Rewrite:
|
|
||||||
- `docs/internal/README.md`
|
|
||||||
- `docs/internal/adapters.md`
|
|
||||||
- `docs/internal/artifacts.md`
|
|
||||||
- `docs/internal/command-restore.md`
|
|
||||||
- `docs/internal/manifest.md`
|
|
||||||
- `docs/internal/stage-*.md`
|
|
||||||
- `docs/internal/storage.md`
|
|
||||||
- `docs/internal/workspace.md`
|
|
||||||
- Verify stage docs against current stage names, stage ordering, manifest
|
|
||||||
records, declared inputs/outputs, adapters, path helpers, storage behavior,
|
|
||||||
publish/current-state behavior, restore behavior, cache behavior, and
|
|
||||||
workspace cleanup.
|
|
||||||
- Keep implementation details in `docs/internal/`, not in user-facing docs.
|
|
||||||
- Avoid turning internal docs into duplicate config or CLI references; link to
|
|
||||||
canonical docs when needed.
|
|
||||||
|
|
||||||
Acceptance criteria:
|
|
||||||
|
|
||||||
- Internal docs are accurate enough for developers and LLM coding agents to
|
|
||||||
change the system safely.
|
|
||||||
- Stage and adapter boundaries match `docs/policy/architecture.md`.
|
|
||||||
- Manifest, path, storage, and publish invariants are explicit and current.
|
|
||||||
|
|
||||||
### Stage 4: Integrations and Examples
|
|
||||||
|
|
||||||
Status: Completed (2026-05-23)
|
|
||||||
|
|
||||||
Review integration docs and maintained examples after core docs are rewritten.
|
|
||||||
|
|
||||||
Implementation requirements:
|
|
||||||
|
|
||||||
- Rewrite:
|
|
||||||
- `docs/integrations/README.md`
|
|
||||||
- `docs/integrations/audita.md`
|
|
||||||
- `docs/integrations/scriptorium.md`
|
|
||||||
- `docs/integrations/seriatim.md`
|
|
||||||
- Verify integration docs against current adapter contracts and expected
|
|
||||||
downstream tool behavior.
|
|
||||||
- Confirm every referenced example file exists.
|
|
||||||
- Confirm examples match current schema and command usage.
|
|
||||||
- Run or rely on example validation tests.
|
|
||||||
- Fix tests when they reference moved, renamed, or intentionally retired
|
|
||||||
examples.
|
|
||||||
|
|
||||||
Acceptance criteria:
|
|
||||||
|
|
||||||
- Integration docs describe only implemented adapter expectations.
|
|
||||||
- Maintained examples are valid, secret-free, and linked from canonical docs.
|
|
||||||
- Example validation tests reflect the documented example set.
|
|
||||||
|
|
||||||
### Stage 5: Roadmap Cleanup and Final Sweep
|
|
||||||
|
|
||||||
Status: Completed (2026-05-23)
|
|
||||||
|
|
||||||
Clean up roadmap state and run final documentation validation.
|
|
||||||
|
|
||||||
Implementation requirements:
|
|
||||||
|
|
||||||
- Review `docs/roadmap/**` for implemented items that should be marked
|
|
||||||
implemented, retired, or left planned.
|
|
||||||
- Keep historical and planned behavior in roadmaps only.
|
|
||||||
- Run final link/path/term sweeps.
|
|
||||||
- Run validation commands:
|
|
||||||
- `go test ./internal/config -run TestExamplesLoadAndValidate -v`
|
|
||||||
- `go test ./internal/app -run TestExecute -v`
|
|
||||||
- `go test ./...`
|
|
||||||
|
|
||||||
Acceptance criteria:
|
|
||||||
|
|
||||||
- All non-policy docs are current for the 1.0 release.
|
|
||||||
- Roadmaps do not serve as required user/operator documentation.
|
|
||||||
- Tests pass after allowed documentation-related test expectation fixes.
|
|
||||||
|
|
||||||
## Required Checks
|
|
||||||
|
|
||||||
Run searches for stale terminology and references during the pass.
|
|
||||||
|
|
||||||
Stale terminology:
|
|
||||||
|
|
||||||
- `archive`
|
|
||||||
- `promote`
|
|
||||||
- `promoted`
|
|
||||||
- `promote_artifacts`
|
|
||||||
- legacy campaign path/discovery language
|
|
||||||
- old transcript names and paths
|
|
||||||
- removed CLI commands or aliases
|
|
||||||
|
|
||||||
Broken or stale references:
|
|
||||||
|
|
||||||
- missing local doc links;
|
|
||||||
- stale `examples/` paths;
|
|
||||||
- stale internal doc filenames;
|
|
||||||
- references to `docs/policy/**` as editable targets;
|
|
||||||
- command examples that no longer match the CLI.
|
|
||||||
|
|
||||||
Policy checks:
|
|
||||||
|
|
||||||
- Current-behavior docs mention only implemented behavior.
|
|
||||||
- Planned behavior appears only under `docs/roadmap/`.
|
|
||||||
- Docs do not expose raw secrets or recommend storing secrets in config.
|
|
||||||
- Docs use canonical homes:
|
|
||||||
- `docs/config.md` for config schema;
|
|
||||||
- `docs/cli.md` for command syntax;
|
|
||||||
- `docs/operations.md` for operator workflows;
|
|
||||||
- `docs/troubleshooting.md` for failure diagnosis;
|
|
||||||
- `docs/internal/` for implementation contracts;
|
|
||||||
- `docs/integrations/` for downstream tool integration notes;
|
|
||||||
- `docs/roadmap/` for future work.
|
|
||||||
|
|
||||||
## Assumptions
|
|
||||||
|
|
||||||
- `docs/policy/**` is read-only for this documentation pass.
|
|
||||||
- This pass is for 1.0 release readiness, not feature implementation.
|
|
||||||
- Product and runtime code changes are out of scope.
|
|
||||||
- Test fixes are in scope when they correct stale documentation, example, path,
|
|
||||||
command, or current-behavior expectations uncovered during the review.
|
|
||||||
- Roadmap files may remain as planning and historical records.
|
|
||||||
- Current-behavior docs must be sufficient for normal use without requiring
|
|
||||||
readers to consult roadmaps.
|
|
||||||
@@ -217,6 +217,33 @@ Safe fix:
|
|||||||
- correct publish source/destination rules;
|
- correct publish source/destination rules;
|
||||||
- retry after storage failure is resolved.
|
- retry after storage failure is resolved.
|
||||||
|
|
||||||
|
## Render markdown source missing
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
- analyze or publish fails because `narratio.transcript.final_markdown` or `narratio.transcript.final_trimmed_markdown` is unavailable.
|
||||||
|
|
||||||
|
Likely causes:
|
||||||
|
|
||||||
|
- render stage was not executed after transcript changes;
|
||||||
|
- render stage failed before producing canonical markdown outputs.
|
||||||
|
|
||||||
|
Diagnostics:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio session status 2026-04-04
|
||||||
|
narratio run-stage render 2026-04-04 --force
|
||||||
|
```
|
||||||
|
|
||||||
|
Safe fix:
|
||||||
|
|
||||||
|
- rerun render and then retry downstream stage(s):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
narratio run-stage render 2026-04-04 --force
|
||||||
|
narratio run-stage analyze 2026-04-04 --force
|
||||||
|
```
|
||||||
|
|
||||||
## Secrets or storage credential failure
|
## Secrets or storage credential failure
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
|
|||||||
@@ -48,6 +48,12 @@ publish:
|
|||||||
- source: narratio.transcript.final_trimmed
|
- source: narratio.transcript.final_trimmed
|
||||||
dest: transcripts/final.trimmed.json
|
dest: transcripts/final.trimmed.json
|
||||||
required: true
|
required: true
|
||||||
|
- source: narratio.transcript.final_markdown
|
||||||
|
dest: transcripts/final.md
|
||||||
|
required: true
|
||||||
|
- source: narratio.transcript.final_trimmed_markdown
|
||||||
|
dest: transcripts/final.trimmed.md
|
||||||
|
required: true
|
||||||
- source: narratio.artifact.session_recap
|
- source: narratio.artifact.session_recap
|
||||||
dest: artifacts/session_recap.md
|
dest: artifacts/session_recap.md
|
||||||
required: true
|
required: true
|
||||||
|
|||||||
@@ -26,6 +26,12 @@ publish:
|
|||||||
- source: narratio.transcript.final_trimmed
|
- source: narratio.transcript.final_trimmed
|
||||||
dest: transcripts/final.trimmed.json
|
dest: transcripts/final.trimmed.json
|
||||||
required: true
|
required: true
|
||||||
|
- source: narratio.transcript.final_markdown
|
||||||
|
dest: transcripts/final.md
|
||||||
|
required: true
|
||||||
|
- source: narratio.transcript.final_trimmed_markdown
|
||||||
|
dest: transcripts/final.trimmed.md
|
||||||
|
required: true
|
||||||
- source: narratio.artifact.session_recap
|
- source: narratio.artifact.session_recap
|
||||||
dest: artifacts/session_recap.md
|
dest: artifacts/session_recap.md
|
||||||
required: true
|
required: true
|
||||||
|
|||||||
@@ -68,6 +68,26 @@ func (n *NoopRunner) Normalize(ctx context.Context, req NormalizeRequest) (Norma
|
|||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Render returns the requested output path with placeholder metadata.
|
||||||
|
func (n *NoopRunner) Render(ctx context.Context, req RenderRequest) (RenderResult, error) {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return RenderResult{}, err
|
||||||
|
}
|
||||||
|
if err := materializeRenderPlaceholders(req); err != nil {
|
||||||
|
return RenderResult{}, err
|
||||||
|
}
|
||||||
|
return RenderResult{
|
||||||
|
OutputRenderedPath: req.OutputRenderedPath,
|
||||||
|
StdoutLogPath: req.StdoutLogPath,
|
||||||
|
StderrLogPath: req.StderrLogPath,
|
||||||
|
GeneratedConfigPath: req.GeneratedConfigPath,
|
||||||
|
InvokedBinary: "noop",
|
||||||
|
Format: req.Format,
|
||||||
|
Title: req.Title,
|
||||||
|
Metadata: map[string]any{"placeholder": true},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
// FakeRunner captures merge requests and returns deterministic responses.
|
// FakeRunner captures merge requests and returns deterministic responses.
|
||||||
type FakeRunner struct {
|
type FakeRunner struct {
|
||||||
Requests []MergeRequest
|
Requests []MergeRequest
|
||||||
@@ -79,6 +99,9 @@ type FakeRunner struct {
|
|||||||
TrimRequests []TrimRequest
|
TrimRequests []TrimRequest
|
||||||
TrimErr error
|
TrimErr error
|
||||||
TrimResult TrimResult
|
TrimResult TrimResult
|
||||||
|
RenderRequests []RenderRequest
|
||||||
|
RenderErr error
|
||||||
|
RenderResult RenderResult
|
||||||
}
|
}
|
||||||
|
|
||||||
// Run records request and returns configured response.
|
// Run records request and returns configured response.
|
||||||
@@ -195,6 +218,46 @@ func (f *FakeRunner) Normalize(ctx context.Context, req NormalizeRequest) (Norma
|
|||||||
return res, nil
|
return res, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Render records request and returns configured response.
|
||||||
|
func (f *FakeRunner) Render(ctx context.Context, req RenderRequest) (RenderResult, error) {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return RenderResult{}, err
|
||||||
|
}
|
||||||
|
f.RenderRequests = append(f.RenderRequests, req)
|
||||||
|
if f.RenderErr != nil {
|
||||||
|
return RenderResult{}, f.RenderErr
|
||||||
|
}
|
||||||
|
if err := materializeRenderPlaceholders(req); err != nil {
|
||||||
|
return RenderResult{}, err
|
||||||
|
}
|
||||||
|
res := f.RenderResult
|
||||||
|
if res.OutputRenderedPath == "" {
|
||||||
|
res.OutputRenderedPath = req.OutputRenderedPath
|
||||||
|
}
|
||||||
|
if res.StdoutLogPath == "" {
|
||||||
|
res.StdoutLogPath = req.StdoutLogPath
|
||||||
|
}
|
||||||
|
if res.StderrLogPath == "" {
|
||||||
|
res.StderrLogPath = req.StderrLogPath
|
||||||
|
}
|
||||||
|
if res.GeneratedConfigPath == "" {
|
||||||
|
res.GeneratedConfigPath = req.GeneratedConfigPath
|
||||||
|
}
|
||||||
|
if res.InvokedBinary == "" {
|
||||||
|
res.InvokedBinary = "fake"
|
||||||
|
}
|
||||||
|
if res.Format == "" {
|
||||||
|
res.Format = req.Format
|
||||||
|
}
|
||||||
|
if res.Title == "" {
|
||||||
|
res.Title = req.Title
|
||||||
|
}
|
||||||
|
if res.Metadata == nil {
|
||||||
|
res.Metadata = map[string]any{"fake": true}
|
||||||
|
}
|
||||||
|
return res, nil
|
||||||
|
}
|
||||||
|
|
||||||
func materializePlaceholders(req MergeRequest) error {
|
func materializePlaceholders(req MergeRequest) error {
|
||||||
if req.OutputMergedTranscriptPath != "" {
|
if req.OutputMergedTranscriptPath != "" {
|
||||||
if err := subprocess.WriteFileAtomic(req.OutputMergedTranscriptPath, []byte(`{"schema":"seriatim.intermediate.v1","segments":[]}`), 0o644); err != nil {
|
if err := subprocess.WriteFileAtomic(req.OutputMergedTranscriptPath, []byte(`{"schema":"seriatim.intermediate.v1","segments":[]}`), 0o644); err != nil {
|
||||||
@@ -301,3 +364,39 @@ func materializeNormalizePlaceholders(req NormalizeRequest) error {
|
|||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func materializeRenderPlaceholders(req RenderRequest) error {
|
||||||
|
if req.OutputRenderedPath != "" {
|
||||||
|
if err := subprocess.WriteFileAtomic(req.OutputRenderedPath, []byte("# Transcript\n\nRendered markdown placeholder.\n"), 0o644); err != nil {
|
||||||
|
return fmt.Errorf("write rendered transcript %q: %w", req.OutputRenderedPath, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if req.GeneratedConfigPath != "" {
|
||||||
|
payload := map[string]any{
|
||||||
|
"schema": "seriatim.generated.v1",
|
||||||
|
"placeholder": true,
|
||||||
|
"command": "render",
|
||||||
|
"input_path": req.InputTranscriptPath,
|
||||||
|
"output_path": req.OutputRenderedPath,
|
||||||
|
"format": req.Format,
|
||||||
|
"title": req.Title,
|
||||||
|
"include_timestamps": req.IncludeTimestamps,
|
||||||
|
"include_segment_ids": req.IncludeSegmentIDs,
|
||||||
|
"include_metadata": req.IncludeMetadata,
|
||||||
|
}
|
||||||
|
if err := subprocess.WriteYAMLAtomic(req.GeneratedConfigPath, payload, 0o644); err != nil {
|
||||||
|
return fmt.Errorf("write generated config %q: %w", req.GeneratedConfigPath, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if req.StdoutLogPath != "" {
|
||||||
|
if err := subprocess.WriteFileAtomic(req.StdoutLogPath, []byte("seriatim noop/fake render stdout placeholder\n"), 0o644); err != nil {
|
||||||
|
return fmt.Errorf("write stdout log %q: %w", req.StdoutLogPath, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if req.StderrLogPath != "" {
|
||||||
|
if err := subprocess.WriteFileAtomic(req.StderrLogPath, []byte("seriatim noop/fake render stderr placeholder\n"), 0o644); err != nil {
|
||||||
|
return fmt.Errorf("write stderr log %q: %w", req.StderrLogPath, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -148,3 +148,58 @@ func TestFakeRunnerNormalizeError(t *testing.T) {
|
|||||||
t.Fatal("expected error, got nil")
|
t.Fatal("expected error, got nil")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestFakeRunnerRenderCapturesRequestAndReturnsPath(t *testing.T) {
|
||||||
|
fake := &FakeRunner{}
|
||||||
|
dir := t.TempDir()
|
||||||
|
req := RenderRequest{
|
||||||
|
GeneratedConfigPath: filepath.Join(dir, "config", "seriatim.render.yml"),
|
||||||
|
InputTranscriptPath: filepath.Join(dir, "transcripts", "final.trimmed.json"),
|
||||||
|
OutputRenderedPath: filepath.Join(dir, "transcripts", "final.trimmed.md"),
|
||||||
|
Format: "markdown",
|
||||||
|
Title: "Session render",
|
||||||
|
IncludeTimestamps: true,
|
||||||
|
IncludeSegmentIDs: false,
|
||||||
|
IncludeMetadata: true,
|
||||||
|
StdoutLogPath: filepath.Join(dir, "logs", "seriatim.render.stdout.log"),
|
||||||
|
StderrLogPath: filepath.Join(dir, "logs", "seriatim.render.stderr.log"),
|
||||||
|
}
|
||||||
|
|
||||||
|
res, err := fake.Render(context.Background(), req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Render() error = %v", err)
|
||||||
|
}
|
||||||
|
if len(fake.RenderRequests) != 1 || fake.RenderRequests[0].GeneratedConfigPath == "" {
|
||||||
|
t.Fatalf("render requests = %#v, want captured request", fake.RenderRequests)
|
||||||
|
}
|
||||||
|
if res.OutputRenderedPath != req.OutputRenderedPath {
|
||||||
|
t.Fatalf("rendered path = %q, want %q", res.OutputRenderedPath, req.OutputRenderedPath)
|
||||||
|
}
|
||||||
|
if res.Format != req.Format {
|
||||||
|
t.Fatalf("format = %q, want %q", res.Format, req.Format)
|
||||||
|
}
|
||||||
|
if res.Title != req.Title {
|
||||||
|
t.Fatalf("title = %q, want %q", res.Title, req.Title)
|
||||||
|
}
|
||||||
|
|
||||||
|
cfgData, err := os.ReadFile(req.GeneratedConfigPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read generated config: %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(string(cfgData), "command: render") {
|
||||||
|
t.Fatalf("generated config = %q, want render command marker", string(cfgData))
|
||||||
|
}
|
||||||
|
for _, path := range []string{req.StdoutLogPath, req.StderrLogPath, req.OutputRenderedPath} {
|
||||||
|
if _, err := os.Stat(path); err != nil {
|
||||||
|
t.Fatalf("expected file %q to exist: %v", path, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFakeRunnerRenderError(t *testing.T) {
|
||||||
|
fake := &FakeRunner{RenderErr: errors.New("boom")}
|
||||||
|
_, err := fake.Render(context.Background(), RenderRequest{})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error, got nil")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
// Package seriatim declares the adapter contract for transcript merge/normalize/trim execution.
|
// Package seriatim declares the adapter contract for transcript merge/normalize/trim/render execution.
|
||||||
package seriatim
|
package seriatim
|
||||||
|
|
||||||
import (
|
import (
|
||||||
@@ -6,11 +6,12 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Runner is the adapter boundary for seriatim merge/normalize/trim invocations.
|
// Runner is the adapter boundary for seriatim merge/normalize/trim/render invocations.
|
||||||
type Runner interface {
|
type Runner interface {
|
||||||
Run(ctx context.Context, req MergeRequest) (MergeResult, error)
|
Run(ctx context.Context, req MergeRequest) (MergeResult, error)
|
||||||
Normalize(ctx context.Context, req NormalizeRequest) (NormalizeResult, error)
|
Normalize(ctx context.Context, req NormalizeRequest) (NormalizeResult, error)
|
||||||
Trim(ctx context.Context, req TrimRequest) (TrimResult, error)
|
Trim(ctx context.Context, req TrimRequest) (TrimResult, error)
|
||||||
|
Render(ctx context.Context, req RenderRequest) (RenderResult, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
// MergeRequest describes a seriatim merge invocation.
|
// MergeRequest describes a seriatim merge invocation.
|
||||||
@@ -90,3 +91,33 @@ type TrimResult struct {
|
|||||||
KeepSelector string
|
KeepSelector string
|
||||||
Metadata map[string]any
|
Metadata map[string]any
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RenderRequest describes a seriatim render invocation.
|
||||||
|
type RenderRequest struct {
|
||||||
|
Binary string
|
||||||
|
InputTranscriptPath string
|
||||||
|
OutputRenderedPath string
|
||||||
|
Format string
|
||||||
|
Title string
|
||||||
|
IncludeTimestamps bool
|
||||||
|
IncludeSegmentIDs bool
|
||||||
|
IncludeMetadata bool
|
||||||
|
StdoutLogPath string
|
||||||
|
StderrLogPath string
|
||||||
|
GeneratedConfigPath string
|
||||||
|
Timeout time.Duration
|
||||||
|
}
|
||||||
|
|
||||||
|
// RenderResult describes a render output.
|
||||||
|
type RenderResult struct {
|
||||||
|
OutputRenderedPath string
|
||||||
|
StdoutLogPath string
|
||||||
|
StderrLogPath string
|
||||||
|
GeneratedConfigPath string
|
||||||
|
ExitCode int
|
||||||
|
Duration time.Duration
|
||||||
|
InvokedBinary string
|
||||||
|
Format string
|
||||||
|
Title string
|
||||||
|
Metadata map[string]any
|
||||||
|
}
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import (
|
|||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/subprocess"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/subprocess"
|
||||||
)
|
)
|
||||||
@@ -384,6 +385,96 @@ func (r *SubprocessRunner) Normalize(ctx context.Context, req NormalizeRequest)
|
|||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Render executes Seriatim render with deterministic flags and validates non-empty text output.
|
||||||
|
func (r *SubprocessRunner) Render(ctx context.Context, req RenderRequest) (RenderResult, error) {
|
||||||
|
if r == nil {
|
||||||
|
return RenderResult{}, fmt.Errorf("seriatim subprocess runner is nil")
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(req.InputTranscriptPath) == "" {
|
||||||
|
return RenderResult{}, fmt.Errorf("seriatim render input path is required")
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(req.OutputRenderedPath) == "" {
|
||||||
|
return RenderResult{}, fmt.Errorf("seriatim render output path is required")
|
||||||
|
}
|
||||||
|
format := strings.TrimSpace(req.Format)
|
||||||
|
if format == "" {
|
||||||
|
format = "markdown"
|
||||||
|
}
|
||||||
|
if format != "markdown" {
|
||||||
|
return RenderResult{}, fmt.Errorf("seriatim render format %q is unsupported", req.Format)
|
||||||
|
}
|
||||||
|
|
||||||
|
binary := r.binary
|
||||||
|
if strings.TrimSpace(req.Binary) != "" {
|
||||||
|
binary = strings.TrimSpace(req.Binary)
|
||||||
|
}
|
||||||
|
|
||||||
|
timeout := r.timeout
|
||||||
|
if req.Timeout < 0 {
|
||||||
|
return RenderResult{}, fmt.Errorf("seriatim render timeout must be >= 0")
|
||||||
|
}
|
||||||
|
if req.Timeout > 0 {
|
||||||
|
timeout = req.Timeout
|
||||||
|
}
|
||||||
|
|
||||||
|
args := buildRenderArgs(req, format)
|
||||||
|
if req.GeneratedConfigPath != "" {
|
||||||
|
if err := writeRenderInvocationConfig(req, args, binary, timeout, format); err != nil {
|
||||||
|
return RenderResult{}, fmt.Errorf("write seriatim render invocation config %q: %w", req.GeneratedConfigPath, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
runRes, err := subprocess.Run(ctx, subprocess.RunRequest{
|
||||||
|
Executable: binary,
|
||||||
|
Args: args,
|
||||||
|
Timeout: timeout,
|
||||||
|
StdoutLogPath: req.StdoutLogPath,
|
||||||
|
StderrLogPath: req.StderrLogPath,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return RenderResult{
|
||||||
|
OutputRenderedPath: req.OutputRenderedPath,
|
||||||
|
StdoutLogPath: req.StdoutLogPath,
|
||||||
|
StderrLogPath: req.StderrLogPath,
|
||||||
|
GeneratedConfigPath: req.GeneratedConfigPath,
|
||||||
|
ExitCode: runRes.ExitCode,
|
||||||
|
Duration: runRes.Duration,
|
||||||
|
InvokedBinary: binary,
|
||||||
|
Format: format,
|
||||||
|
Title: req.Title,
|
||||||
|
}, fmt.Errorf("run seriatim render (binary=%q): %w", binary, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := validateNonEmptyTextFile(req.OutputRenderedPath); err != nil {
|
||||||
|
return RenderResult{
|
||||||
|
OutputRenderedPath: req.OutputRenderedPath,
|
||||||
|
StdoutLogPath: req.StdoutLogPath,
|
||||||
|
StderrLogPath: req.StderrLogPath,
|
||||||
|
GeneratedConfigPath: req.GeneratedConfigPath,
|
||||||
|
ExitCode: runRes.ExitCode,
|
||||||
|
Duration: runRes.Duration,
|
||||||
|
InvokedBinary: binary,
|
||||||
|
Format: format,
|
||||||
|
Title: req.Title,
|
||||||
|
}, fmt.Errorf("validate seriatim rendered output %q: %w", req.OutputRenderedPath, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return RenderResult{
|
||||||
|
OutputRenderedPath: req.OutputRenderedPath,
|
||||||
|
StdoutLogPath: req.StdoutLogPath,
|
||||||
|
StderrLogPath: req.StderrLogPath,
|
||||||
|
GeneratedConfigPath: req.GeneratedConfigPath,
|
||||||
|
ExitCode: runRes.ExitCode,
|
||||||
|
Duration: runRes.Duration,
|
||||||
|
InvokedBinary: binary,
|
||||||
|
Format: format,
|
||||||
|
Title: req.Title,
|
||||||
|
Metadata: map[string]any{
|
||||||
|
"adapter": "seriatim_subprocess",
|
||||||
|
},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
func (r *SubprocessRunner) buildMergeArgs(req MergeRequest) []string {
|
func (r *SubprocessRunner) buildMergeArgs(req MergeRequest) []string {
|
||||||
args := []string{"merge"}
|
args := []string{"merge"}
|
||||||
|
|
||||||
@@ -480,6 +571,22 @@ func buildNormalizeArgs(req NormalizeRequest, outputSchema string) []string {
|
|||||||
return args
|
return args
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func buildRenderArgs(req RenderRequest, format string) []string {
|
||||||
|
args := []string{
|
||||||
|
"render",
|
||||||
|
"--input-file", req.InputTranscriptPath,
|
||||||
|
"--output-file", req.OutputRenderedPath,
|
||||||
|
"--format", format,
|
||||||
|
"--include-timestamps=" + strconv.FormatBool(req.IncludeTimestamps),
|
||||||
|
"--include-segment-ids=" + strconv.FormatBool(req.IncludeSegmentIDs),
|
||||||
|
"--include-metadata=" + strconv.FormatBool(req.IncludeMetadata),
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(req.Title) != "" {
|
||||||
|
args = append(args, "--title", req.Title)
|
||||||
|
}
|
||||||
|
return args
|
||||||
|
}
|
||||||
|
|
||||||
func writeTrimInvocationConfig(req TrimRequest, args []string, binary string, timeout time.Duration) error {
|
func writeTrimInvocationConfig(req TrimRequest, args []string, binary string, timeout time.Duration) error {
|
||||||
payload := map[string]any{
|
payload := map[string]any{
|
||||||
"schema": "seriatim.generated.v1",
|
"schema": "seriatim.generated.v1",
|
||||||
@@ -509,6 +616,24 @@ func writeNormalizeInvocationConfig(req NormalizeRequest, args []string, binary
|
|||||||
return subprocess.WriteYAMLAtomic(req.GeneratedConfigPath, payload, 0o644)
|
return subprocess.WriteYAMLAtomic(req.GeneratedConfigPath, payload, 0o644)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func writeRenderInvocationConfig(req RenderRequest, args []string, binary string, timeout time.Duration, format string) error {
|
||||||
|
payload := map[string]any{
|
||||||
|
"schema": "seriatim.generated.v1",
|
||||||
|
"command": "render",
|
||||||
|
"binary": binary,
|
||||||
|
"args": args,
|
||||||
|
"timeout": timeout.String(),
|
||||||
|
"input_path": req.InputTranscriptPath,
|
||||||
|
"output_path": req.OutputRenderedPath,
|
||||||
|
"format": format,
|
||||||
|
"title": req.Title,
|
||||||
|
"include_timestamps": req.IncludeTimestamps,
|
||||||
|
"include_segment_ids": req.IncludeSegmentIDs,
|
||||||
|
"include_metadata": req.IncludeMetadata,
|
||||||
|
}
|
||||||
|
return subprocess.WriteYAMLAtomic(req.GeneratedConfigPath, payload, 0o644)
|
||||||
|
}
|
||||||
|
|
||||||
func validateJSONFile(path string) error {
|
func validateJSONFile(path string) error {
|
||||||
data, err := os.ReadFile(path)
|
data, err := os.ReadFile(path)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -541,3 +666,20 @@ func validateJSONFileWithSegments(path string) error {
|
|||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func validateNonEmptyTextFile(path string) error {
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("read file: %w", err)
|
||||||
|
}
|
||||||
|
if len(data) == 0 {
|
||||||
|
return fmt.Errorf("file is empty")
|
||||||
|
}
|
||||||
|
if !utf8.Valid(data) {
|
||||||
|
return fmt.Errorf("file is not valid utf-8 text")
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(string(data)) == "" {
|
||||||
|
return fmt.Errorf("file has no non-whitespace content")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -569,6 +569,156 @@ func TestSubprocessRunnerNormalizeInvalidReportJSONFails(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestSubprocessRunnerRenderSuccessInvocationAndProvenance(t *testing.T) {
|
||||||
|
if runtime.GOOS == "windows" {
|
||||||
|
t.Skip("helper wrapper script uses /bin/sh")
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Setenv("GO_WANT_SERIATIM_HELPER", "1")
|
||||||
|
t.Setenv("SERIATIM_HELPER_MODE", "render_success")
|
||||||
|
recordPath := filepath.Join(t.TempDir(), "record.json")
|
||||||
|
t.Setenv("SERIATIM_HELPER_RECORD_PATH", recordPath)
|
||||||
|
|
||||||
|
wrapper := writeHelperWrapper(t)
|
||||||
|
runner := mustRunner(t, wrapper, false)
|
||||||
|
req := renderReqForTest(t)
|
||||||
|
|
||||||
|
res, err := runner.Render(context.Background(), req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Render() error = %v", err)
|
||||||
|
}
|
||||||
|
if res.OutputRenderedPath != req.OutputRenderedPath {
|
||||||
|
t.Fatalf("OutputRenderedPath = %q, want %q", res.OutputRenderedPath, req.OutputRenderedPath)
|
||||||
|
}
|
||||||
|
if res.Format != req.Format {
|
||||||
|
t.Fatalf("Format = %q, want %q", res.Format, req.Format)
|
||||||
|
}
|
||||||
|
if res.Title != req.Title {
|
||||||
|
t.Fatalf("Title = %q, want %q", res.Title, req.Title)
|
||||||
|
}
|
||||||
|
if res.InvokedBinary != wrapper {
|
||||||
|
t.Fatalf("InvokedBinary = %q, want %q", res.InvokedBinary, wrapper)
|
||||||
|
}
|
||||||
|
if res.ExitCode != 0 {
|
||||||
|
t.Fatalf("ExitCode = %d, want 0", res.ExitCode)
|
||||||
|
}
|
||||||
|
if res.Duration <= 0 {
|
||||||
|
t.Fatalf("Duration = %s, want >0", res.Duration)
|
||||||
|
}
|
||||||
|
if res.Metadata == nil || res.Metadata["adapter"] != "seriatim_subprocess" {
|
||||||
|
t.Fatalf("Metadata = %#v, want adapter marker", res.Metadata)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := os.Stat(req.OutputRenderedPath); err != nil {
|
||||||
|
t.Fatalf("rendered output missing: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(req.StdoutLogPath); err != nil {
|
||||||
|
t.Fatalf("stdout log missing: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(req.StderrLogPath); err != nil {
|
||||||
|
t.Fatalf("stderr log missing: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(req.GeneratedConfigPath); err != nil {
|
||||||
|
t.Fatalf("generated config missing: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rec := readHelperRecord(t, recordPath)
|
||||||
|
wantArgs := []string{
|
||||||
|
"render",
|
||||||
|
"--input-file", req.InputTranscriptPath,
|
||||||
|
"--output-file", req.OutputRenderedPath,
|
||||||
|
"--format", req.Format,
|
||||||
|
"--include-timestamps=true",
|
||||||
|
"--include-segment-ids=true",
|
||||||
|
"--include-metadata=false",
|
||||||
|
"--title", req.Title,
|
||||||
|
}
|
||||||
|
if strings.Join(rec.Args, "\n") != strings.Join(wantArgs, "\n") {
|
||||||
|
t.Fatalf("args = %#v, want %#v", rec.Args, wantArgs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubprocessRunnerRenderWithoutTitleOmitsTitleArg(t *testing.T) {
|
||||||
|
if runtime.GOOS == "windows" {
|
||||||
|
t.Skip("helper wrapper script uses /bin/sh")
|
||||||
|
}
|
||||||
|
t.Setenv("GO_WANT_SERIATIM_HELPER", "1")
|
||||||
|
t.Setenv("SERIATIM_HELPER_MODE", "render_success")
|
||||||
|
recordPath := filepath.Join(t.TempDir(), "record.json")
|
||||||
|
t.Setenv("SERIATIM_HELPER_RECORD_PATH", recordPath)
|
||||||
|
|
||||||
|
runner := mustRunner(t, writeHelperWrapper(t), false)
|
||||||
|
req := renderReqForTest(t)
|
||||||
|
req.Title = ""
|
||||||
|
if _, err := runner.Render(context.Background(), req); err != nil {
|
||||||
|
t.Fatalf("Render() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rec := readHelperRecord(t, recordPath)
|
||||||
|
for i := 0; i < len(rec.Args); i++ {
|
||||||
|
if rec.Args[i] == "--title" {
|
||||||
|
t.Fatalf("args = %#v, did not expect --title", rec.Args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubprocessRunnerRenderSubprocessFailure(t *testing.T) {
|
||||||
|
if runtime.GOOS == "windows" {
|
||||||
|
t.Skip("helper wrapper script uses /bin/sh")
|
||||||
|
}
|
||||||
|
t.Setenv("GO_WANT_SERIATIM_HELPER", "1")
|
||||||
|
t.Setenv("SERIATIM_HELPER_MODE", "fail")
|
||||||
|
t.Setenv("SERIATIM_HELPER_RECORD_PATH", filepath.Join(t.TempDir(), "record.json"))
|
||||||
|
|
||||||
|
runner := mustRunner(t, writeHelperWrapper(t), false)
|
||||||
|
req := renderReqForTest(t)
|
||||||
|
_, err := runner.Render(context.Background(), req)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("Render() error = nil, want non-nil")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "run seriatim render") {
|
||||||
|
t.Fatalf("error = %q, want subprocess context", err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubprocessRunnerRenderMissingOutputFails(t *testing.T) {
|
||||||
|
if runtime.GOOS == "windows" {
|
||||||
|
t.Skip("helper wrapper script uses /bin/sh")
|
||||||
|
}
|
||||||
|
t.Setenv("GO_WANT_SERIATIM_HELPER", "1")
|
||||||
|
t.Setenv("SERIATIM_HELPER_MODE", "missing_output")
|
||||||
|
t.Setenv("SERIATIM_HELPER_RECORD_PATH", filepath.Join(t.TempDir(), "record.json"))
|
||||||
|
|
||||||
|
runner := mustRunner(t, writeHelperWrapper(t), false)
|
||||||
|
req := renderReqForTest(t)
|
||||||
|
_, err := runner.Render(context.Background(), req)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("Render() error = nil, want non-nil")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "validate seriatim rendered output") {
|
||||||
|
t.Fatalf("error = %q, want output validation context", err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubprocessRunnerRenderEmptyOutputFails(t *testing.T) {
|
||||||
|
if runtime.GOOS == "windows" {
|
||||||
|
t.Skip("helper wrapper script uses /bin/sh")
|
||||||
|
}
|
||||||
|
t.Setenv("GO_WANT_SERIATIM_HELPER", "1")
|
||||||
|
t.Setenv("SERIATIM_HELPER_MODE", "render_empty_output")
|
||||||
|
t.Setenv("SERIATIM_HELPER_RECORD_PATH", filepath.Join(t.TempDir(), "record.json"))
|
||||||
|
|
||||||
|
runner := mustRunner(t, writeHelperWrapper(t), false)
|
||||||
|
req := renderReqForTest(t)
|
||||||
|
_, err := runner.Render(context.Background(), req)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("Render() error = nil, want non-nil")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "file is empty") {
|
||||||
|
t.Fatalf("error = %q, want empty-file validation", err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestSubprocessRunnerConstructorValidation(t *testing.T) {
|
func TestSubprocessRunnerConstructorValidation(t *testing.T) {
|
||||||
_, err := NewSubprocessRunnerFromConfigValues("", "10m", "seriatim-intermediate", nil, true, EnvConfig{})
|
_, err := NewSubprocessRunnerFromConfigValues("", "10m", "seriatim-intermediate", nil, true, EnvConfig{})
|
||||||
if err == nil {
|
if err == nil {
|
||||||
@@ -702,6 +852,14 @@ func TestSeriatimSubprocessHelper(t *testing.T) {
|
|||||||
case "normalize_report_missing":
|
case "normalize_report_missing":
|
||||||
writeSeriatimHelperFile(outputPath, `{"schema":"seriatim.intermediate.v1","segments":[]}`)
|
writeSeriatimHelperFile(outputPath, `{"schema":"seriatim.intermediate.v1","segments":[]}`)
|
||||||
os.Exit(0)
|
os.Exit(0)
|
||||||
|
case "render_success":
|
||||||
|
writeSeriatimHelperFile(outputPath, "# Rendered transcript\n\nHello.\n")
|
||||||
|
_, _ = os.Stdout.WriteString("seriatim helper render stdout\n")
|
||||||
|
_, _ = os.Stderr.WriteString("seriatim helper render stderr\n")
|
||||||
|
os.Exit(0)
|
||||||
|
case "render_empty_output":
|
||||||
|
writeSeriatimHelperFile(outputPath, "")
|
||||||
|
os.Exit(0)
|
||||||
default:
|
default:
|
||||||
_, _ = os.Stderr.WriteString(fmt.Sprintf("unknown helper mode %q\n", mode))
|
_, _ = os.Stderr.WriteString(fmt.Sprintf("unknown helper mode %q\n", mode))
|
||||||
os.Exit(2)
|
os.Exit(2)
|
||||||
@@ -777,6 +935,25 @@ func normalizeReqForTest(t *testing.T, withReport bool) NormalizeRequest {
|
|||||||
return req
|
return req
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func renderReqForTest(t *testing.T) RenderRequest {
|
||||||
|
t.Helper()
|
||||||
|
dir := t.TempDir()
|
||||||
|
input := filepath.Join(dir, "final.trimmed.json")
|
||||||
|
writeSeriatimFile(t, input, `{"schema":"seriatim.intermediate.v1","segments":[]}`)
|
||||||
|
return RenderRequest{
|
||||||
|
InputTranscriptPath: input,
|
||||||
|
OutputRenderedPath: filepath.Join(dir, "final.trimmed.md"),
|
||||||
|
Format: "markdown",
|
||||||
|
Title: "Session 42",
|
||||||
|
IncludeTimestamps: true,
|
||||||
|
IncludeSegmentIDs: true,
|
||||||
|
IncludeMetadata: false,
|
||||||
|
GeneratedConfigPath: filepath.Join(dir, "seriatim.render.generated.yml"),
|
||||||
|
StdoutLogPath: filepath.Join(dir, "seriatim.render.stdout.log"),
|
||||||
|
StderrLogPath: filepath.Join(dir, "seriatim.render.stderr.log"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func mustRunner(t *testing.T, binary string, report bool) *SubprocessRunner {
|
func mustRunner(t *testing.T, binary string, report bool) *SubprocessRunner {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
coalesce := 3.0
|
coalesce := 3.0
|
||||||
|
|||||||
@@ -1,31 +0,0 @@
|
|||||||
// Package storage declares archive/storage backend adapter boundaries.
|
|
||||||
package storage
|
|
||||||
|
|
||||||
import "context"
|
|
||||||
|
|
||||||
// TODO: implement remote storage/archive backends (S3/SFTP/etc.).
|
|
||||||
|
|
||||||
// Backend is the adapter boundary for archive/storage operations.
|
|
||||||
type Backend interface {
|
|
||||||
Archive(ctx context.Context, req ArchiveRequest) (ArchiveResult, error)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ArchiveItem describes one item to archive.
|
|
||||||
type ArchiveItem struct {
|
|
||||||
Kind string
|
|
||||||
LocalPath string
|
|
||||||
RemoteKey string
|
|
||||||
}
|
|
||||||
|
|
||||||
// ArchiveRequest describes one archive operation.
|
|
||||||
type ArchiveRequest struct {
|
|
||||||
SessionID string
|
|
||||||
ManifestPath string
|
|
||||||
Items []ArchiveItem
|
|
||||||
}
|
|
||||||
|
|
||||||
// ArchiveResult describes archive operation output.
|
|
||||||
type ArchiveResult struct {
|
|
||||||
Archived []ArchiveItem
|
|
||||||
Metadata map[string]any
|
|
||||||
}
|
|
||||||
@@ -10,23 +10,8 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
// NoopBackend is a deterministic no-op archive/storage adapter.
|
// FakeBackend provides a deterministic in-memory object store for tests.
|
||||||
type NoopBackend struct{}
|
|
||||||
|
|
||||||
// Archive returns the requested items as archived with placeholder metadata.
|
|
||||||
func (n *NoopBackend) Archive(ctx context.Context, req ArchiveRequest) (ArchiveResult, error) {
|
|
||||||
if err := ctx.Err(); err != nil {
|
|
||||||
return ArchiveResult{}, err
|
|
||||||
}
|
|
||||||
return ArchiveResult{Archived: append([]ArchiveItem(nil), req.Items...), Metadata: map[string]any{"placeholder": true}}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// FakeBackend captures archive requests and returns deterministic responses.
|
|
||||||
type FakeBackend struct {
|
type FakeBackend struct {
|
||||||
Requests []ArchiveRequest
|
|
||||||
Err error
|
|
||||||
Result ArchiveResult
|
|
||||||
|
|
||||||
Objects map[string]FakeObject
|
Objects map[string]FakeObject
|
||||||
Uploads []FakeUploadCall
|
Uploads []FakeUploadCall
|
||||||
Downloads []FakeDownloadCall
|
Downloads []FakeDownloadCall
|
||||||
@@ -50,25 +35,6 @@ type FakeDownloadCall struct {
|
|||||||
LocalPath string
|
LocalPath string
|
||||||
}
|
}
|
||||||
|
|
||||||
// Archive records request and returns configured response.
|
|
||||||
func (f *FakeBackend) Archive(ctx context.Context, req ArchiveRequest) (ArchiveResult, error) {
|
|
||||||
if err := ctx.Err(); err != nil {
|
|
||||||
return ArchiveResult{}, err
|
|
||||||
}
|
|
||||||
f.Requests = append(f.Requests, req)
|
|
||||||
if f.Err != nil {
|
|
||||||
return ArchiveResult{}, f.Err
|
|
||||||
}
|
|
||||||
res := f.Result
|
|
||||||
if res.Archived == nil {
|
|
||||||
res.Archived = append([]ArchiveItem(nil), req.Items...)
|
|
||||||
}
|
|
||||||
if res.Metadata == nil {
|
|
||||||
res.Metadata = map[string]any{"fake": true}
|
|
||||||
}
|
|
||||||
return res, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// FakeObject is a deterministic fake object-store record.
|
// FakeObject is a deterministic fake object-store record.
|
||||||
type FakeObject struct {
|
type FakeObject struct {
|
||||||
Key string
|
Key string
|
||||||
|
|||||||
@@ -9,30 +9,6 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestFakeBackendCapturesRequestAndReturnsItems(t *testing.T) {
|
|
||||||
fake := &FakeBackend{}
|
|
||||||
req := ArchiveRequest{SessionID: "s1", Items: []ArchiveItem{{Kind: "artifact", LocalPath: "artifacts/log.md"}}}
|
|
||||||
|
|
||||||
res, err := fake.Archive(context.Background(), req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Archive() error = %v", err)
|
|
||||||
}
|
|
||||||
if len(fake.Requests) != 1 || fake.Requests[0].SessionID != "s1" {
|
|
||||||
t.Fatalf("requests = %#v, want captured request", fake.Requests)
|
|
||||||
}
|
|
||||||
if len(res.Archived) != 1 {
|
|
||||||
t.Fatalf("archived len = %d, want 1", len(res.Archived))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestFakeBackendError(t *testing.T) {
|
|
||||||
fake := &FakeBackend{Err: errors.New("boom")}
|
|
||||||
_, err := fake.Archive(context.Background(), ArchiveRequest{})
|
|
||||||
if err == nil {
|
|
||||||
t.Fatal("expected error, got nil")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestFakeBackendListPrefixFiltering(t *testing.T) {
|
func TestFakeBackendListPrefixFiltering(t *testing.T) {
|
||||||
fake := &FakeBackend{}
|
fake := &FakeBackend{}
|
||||||
fake.SeedObject(FakeObject{Key: "dnd/campaigns/forsaken/audio/a.flac", Data: []byte("a")})
|
fake.SeedObject(FakeObject{Key: "dnd/campaigns/forsaken/audio/a.flac", Data: []byte("a")})
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ObjectStore is a remote object storage boundary used by future prepare/archive work.
|
// ObjectStore is a remote object storage boundary used by prepare, restore, and publish work.
|
||||||
//
|
//
|
||||||
// Key invariant:
|
// Key invariant:
|
||||||
// callers pass full bucket-relative object keys. Backend implementations do not
|
// callers pass full bucket-relative object keys. Backend implementations do not
|
||||||
|
|||||||
36
internal/adapters/storage/temp_download.go
Normal file
36
internal/adapters/storage/temp_download.go
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
package storage
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// DownloadObjectToTemp downloads an object into a temporary file and returns
|
||||||
|
// the cleaned local path.
|
||||||
|
func DownloadObjectToTemp(ctx context.Context, store ObjectStore, key, pattern string) (string, error) {
|
||||||
|
if store == nil {
|
||||||
|
return "", fmt.Errorf("object store is required")
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(pattern) == "" {
|
||||||
|
return "", fmt.Errorf("temp file pattern is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
tmp, err := os.CreateTemp("", pattern)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("create temp file: %w", err)
|
||||||
|
}
|
||||||
|
path := tmp.Name()
|
||||||
|
if err := tmp.Close(); err != nil {
|
||||||
|
_ = os.Remove(path)
|
||||||
|
return "", fmt.Errorf("close temp file: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := store.Download(ctx, key, path); err != nil {
|
||||||
|
_ = os.Remove(path)
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return filepath.Clean(path), nil
|
||||||
|
}
|
||||||
69
internal/adapters/storage/temp_download_test.go
Normal file
69
internal/adapters/storage/temp_download_test.go
Normal file
@@ -0,0 +1,69 @@
|
|||||||
|
package storage
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestDownloadObjectToTempSuccess(t *testing.T) {
|
||||||
|
store := &FakeBackend{}
|
||||||
|
store.SeedObject(FakeObject{Key: "sessions/a/current/run_id.txt", Data: []byte("run-123\n")})
|
||||||
|
|
||||||
|
path, err := DownloadObjectToTemp(context.Background(), store, "sessions/a/current/run_id.txt", "narratio-test-*.txt")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("DownloadObjectToTemp() error = %v", err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { _ = os.Remove(path) })
|
||||||
|
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReadFile() error = %v", err)
|
||||||
|
}
|
||||||
|
if string(data) != "run-123\n" {
|
||||||
|
t.Fatalf("downloaded data = %q, want %q", string(data), "run-123\n")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDownloadObjectToTempFailedDownloadRemovesTempFile(t *testing.T) {
|
||||||
|
sentinel := errors.New("download failed")
|
||||||
|
store := &FakeBackend{DownloadErr: sentinel}
|
||||||
|
pattern := "narratio-test-fail-*.txt"
|
||||||
|
before, err := filepath.Glob(filepath.Join(os.TempDir(), "narratio-test-fail-*.txt"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Glob(before) error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
path, err := DownloadObjectToTemp(context.Background(), store, "sessions/a/current/run_id.txt", pattern)
|
||||||
|
if !errors.Is(err, sentinel) {
|
||||||
|
t.Fatalf("DownloadObjectToTemp() error = %v, want %v", err, sentinel)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(path) != "" {
|
||||||
|
t.Fatalf("DownloadObjectToTemp() path = %q, want empty on failure", path)
|
||||||
|
}
|
||||||
|
after, err := filepath.Glob(filepath.Join(os.TempDir(), "narratio-test-fail-*.txt"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Glob(after) error = %v", err)
|
||||||
|
}
|
||||||
|
if len(after) != len(before) {
|
||||||
|
t.Fatalf("temp file count changed after failed download: before=%d after=%d", len(before), len(after))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDownloadObjectToTempCallerContextWrappingPreservesCause(t *testing.T) {
|
||||||
|
sentinel := errors.New("object missing")
|
||||||
|
store := &FakeBackend{DownloadErr: sentinel}
|
||||||
|
|
||||||
|
_, err := DownloadObjectToTemp(context.Background(), store, "sessions/a/current/run_id.txt", "narratio-test-*.txt")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("DownloadObjectToTemp() error = nil, want error")
|
||||||
|
}
|
||||||
|
err = fmt.Errorf("download run pointer failed: %w", err)
|
||||||
|
if !errors.Is(err, sentinel) {
|
||||||
|
t.Fatalf("wrapped error does not preserve sentinel cause: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -145,7 +145,7 @@ func TestHTTPClientDoesNotRetryOnNonRetryableStatus(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestHTTPClientInvalidJSONFailsAndDoesNotPromote(t *testing.T) {
|
func TestHTTPClientInvalidJSONFailsAndDoesNotInstallOutput(t *testing.T) {
|
||||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
_, _ = w.Write([]byte(`not-json`))
|
_, _ = w.Write([]byte(`not-json`))
|
||||||
}))
|
}))
|
||||||
|
|||||||
@@ -33,7 +33,7 @@ func TestExecuteRunStageArtifactsUnsupportedStageFails(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteRunStageArchivePropagatesSelectedArtifacts(t *testing.T) {
|
func TestExecuteRunStagePublishPropagatesSelectedArtifacts(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
@@ -120,14 +120,14 @@ func TestRunStageArtifactsDoesNotImplyForce(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestResumeArtifactsWithSucceededAnalyzeSkipsUnlessForced(t *testing.T) {
|
func TestRunArtifactsWithSucceededAnalyzeSkipsUnlessForced(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
seed := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
seed := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
||||||
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "analyze", "publish", "notify"} {
|
||||||
seed.MarkStageSucceeded(stageName, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
seed.MarkStageSucceeded(stageName, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
||||||
}
|
}
|
||||||
if err := store.Save(context.Background(), manifestPath, seed); err != nil {
|
if err := store.Save(context.Background(), manifestPath, seed); err != nil {
|
||||||
@@ -135,16 +135,16 @@ func TestResumeArtifactsWithSucceededAnalyzeSkipsUnlessForced(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Resume(
|
err := Run(
|
||||||
context.Background(),
|
context.Background(),
|
||||||
[]string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
[]string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--artifacts", "session_recap"},
|
||||||
&out,
|
&out,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Resume() error = %v", err)
|
t.Fatalf("Run() error = %v", err)
|
||||||
}
|
}
|
||||||
if !strings.Contains(out.String(), "has no remaining stages") {
|
if !strings.Contains(out.String(), "executed=0 skipped=10") {
|
||||||
t.Fatalf("output = %q, want no remaining stages", out.String())
|
t.Fatalf("output = %q, want all stages skipped", out.String())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -281,7 +281,7 @@ func TestExecuteAnalyzeMissingConfigUsesRunStageLoadingPath(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecutePublishForceRunsArchive(t *testing.T) {
|
func TestExecutePublishForceRunsPublish(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
|
||||||
|
|||||||
@@ -16,7 +16,6 @@ import (
|
|||||||
// Clean removes local workspace/spool state while preserving durable cache
|
// Clean removes local workspace/spool state while preserving durable cache
|
||||||
// state unless cache cleanup is explicitly requested.
|
// state unless cache cleanup is explicitly requested.
|
||||||
func Clean(ctx context.Context, args []string, out io.Writer) error {
|
func Clean(ctx context.Context, args []string, out io.Writer) error {
|
||||||
positionalSessionID, args := pullLeadingSessionID(args)
|
|
||||||
fs := flag.NewFlagSet("clean", flag.ContinueOnError)
|
fs := flag.NewFlagSet("clean", flag.ContinueOnError)
|
||||||
fs.SetOutput(io.Discard)
|
fs.SetOutput(io.Discard)
|
||||||
var flags commonConfigFlags
|
var flags commonConfigFlags
|
||||||
@@ -27,20 +26,8 @@ func Clean(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
fs.BoolVar(&all, "all", false, "clean all local session work/spool state")
|
fs.BoolVar(&all, "all", false, "clean all local session work/spool state")
|
||||||
fs.BoolVar(&dryRun, "dry-run", false, "print cleanup targets without deleting")
|
fs.BoolVar(&dryRun, "dry-run", false, "print cleanup targets without deleting")
|
||||||
fs.BoolVar(&clearCache, "clear-cache", false, "also clear durable S3 audio cache entries")
|
fs.BoolVar(&clearCache, "clear-cache", false, "also clear durable S3 audio cache entries")
|
||||||
if err := fs.Parse(args); err != nil {
|
if err := parseSessionAwareFlags("clean", fs, args, &flags.sessionID); err != nil {
|
||||||
return fmt.Errorf("clean: invalid flags: %w", err)
|
return err
|
||||||
}
|
|
||||||
if positionalSessionID == "" {
|
|
||||||
if err := applyParsedSessionIDArg("clean", fs, &flags.sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
if fs.NArg() != 0 {
|
|
||||||
return fmt.Errorf("clean: unexpected positional arguments")
|
|
||||||
}
|
|
||||||
if err := applyPositionalSessionID("clean", positionalSessionID, &flags.sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
if all {
|
if all {
|
||||||
return cleanAllLocal(flags, dryRun, clearCache, out)
|
return cleanAllLocal(flags, dryRun, clearCache, out)
|
||||||
@@ -182,26 +169,12 @@ func reportCleanRootChildren(out io.Writer, root, policy string, dryRun bool) er
|
|||||||
}
|
}
|
||||||
|
|
||||||
func cleanableRootChildren(root, policy string) (string, []string, error) {
|
func cleanableRootChildren(root, policy string) (string, []string, error) {
|
||||||
cleanRoot := strings.TrimSpace(root)
|
rootAbs, exists, err := validateCleanRoot(root, policy)
|
||||||
if cleanRoot == "" {
|
|
||||||
return "", nil, fmt.Errorf("cleanup policy %s: root path is required", policy)
|
|
||||||
}
|
|
||||||
rootAbs, err := filepath.Abs(cleanRoot)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", nil, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
return "", nil, err
|
||||||
}
|
}
|
||||||
info, err := os.Lstat(rootAbs)
|
if !exists {
|
||||||
if err != nil {
|
return rootAbs, nil, nil
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return rootAbs, nil, nil
|
|
||||||
}
|
|
||||||
return "", nil, fmt.Errorf("cleanup policy %s: stat root %q: %w", policy, rootAbs, err)
|
|
||||||
}
|
|
||||||
if info.Mode()&os.ModeSymlink != 0 {
|
|
||||||
return "", nil, fmt.Errorf("cleanup policy %s: refusing to clean symlink root %q", policy, rootAbs)
|
|
||||||
}
|
|
||||||
if !info.IsDir() {
|
|
||||||
return "", nil, fmt.Errorf("cleanup policy %s: root %q is not a directory", policy, rootAbs)
|
|
||||||
}
|
}
|
||||||
entries, err := os.ReadDir(rootAbs)
|
entries, err := os.ReadDir(rootAbs)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -300,46 +273,7 @@ func reportCleanScopedFile(out io.Writer, root, target, policy string, dryRun bo
|
|||||||
}
|
}
|
||||||
|
|
||||||
func validateScopedFile(root, target, policy string) (scopedDir, error) {
|
func validateScopedFile(root, target, policy string) (scopedDir, error) {
|
||||||
cleanRoot := strings.TrimSpace(root)
|
return validateScopedTarget(root, target, policy, false)
|
||||||
cleanTarget := strings.TrimSpace(target)
|
|
||||||
if cleanRoot == "" {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: root path is required", policy)
|
|
||||||
}
|
|
||||||
if cleanTarget == "" {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: target path is required", policy)
|
|
||||||
}
|
|
||||||
rootAbs, err := filepath.Abs(cleanRoot)
|
|
||||||
if err != nil {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
|
||||||
}
|
|
||||||
targetAbs, err := filepath.Abs(cleanTarget)
|
|
||||||
if err != nil {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
|
|
||||||
}
|
|
||||||
rel, err := filepath.Rel(rootAbs, targetAbs)
|
|
||||||
if err != nil {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
|
|
||||||
}
|
|
||||||
if rel == "." {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
|
|
||||||
}
|
|
||||||
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
|
|
||||||
}
|
|
||||||
info, err := os.Lstat(targetAbs)
|
|
||||||
if err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: false}, nil
|
|
||||||
}
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
|
|
||||||
}
|
|
||||||
if info.Mode()&os.ModeSymlink != 0 {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
|
|
||||||
}
|
|
||||||
if info.IsDir() {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is a directory", policy, targetAbs)
|
|
||||||
}
|
|
||||||
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: true}, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func cleanIsFlac(path string) bool {
|
func cleanIsFlac(path string) bool {
|
||||||
|
|||||||
82
internal/app/cleanup_targets.go
Normal file
82
internal/app/cleanup_targets.go
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
func validateScopedTarget(root, target, policy string, requireDir bool) (scopedDir, error) {
|
||||||
|
cleanRoot := strings.TrimSpace(root)
|
||||||
|
cleanTarget := strings.TrimSpace(target)
|
||||||
|
if cleanRoot == "" {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: root path is required", policy)
|
||||||
|
}
|
||||||
|
if cleanTarget == "" {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: target path is required", policy)
|
||||||
|
}
|
||||||
|
|
||||||
|
rootAbs, err := filepath.Abs(cleanRoot)
|
||||||
|
if err != nil {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
||||||
|
}
|
||||||
|
targetAbs, err := filepath.Abs(cleanTarget)
|
||||||
|
if err != nil {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rel, err := filepath.Rel(rootAbs, targetAbs)
|
||||||
|
if err != nil {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
|
||||||
|
}
|
||||||
|
if rel == "." {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
|
||||||
|
}
|
||||||
|
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
|
||||||
|
}
|
||||||
|
|
||||||
|
info, err := os.Lstat(targetAbs)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: false}, nil
|
||||||
|
}
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
|
||||||
|
}
|
||||||
|
if info.Mode()&os.ModeSymlink != 0 {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
|
||||||
|
}
|
||||||
|
if requireDir && !info.IsDir() {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is not a directory", policy, targetAbs)
|
||||||
|
}
|
||||||
|
if !requireDir && info.IsDir() {
|
||||||
|
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is a directory", policy, targetAbs)
|
||||||
|
}
|
||||||
|
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateCleanRoot(root, policy string) (string, bool, error) {
|
||||||
|
cleanRoot := strings.TrimSpace(root)
|
||||||
|
if cleanRoot == "" {
|
||||||
|
return "", false, fmt.Errorf("cleanup policy %s: root path is required", policy)
|
||||||
|
}
|
||||||
|
rootAbs, err := filepath.Abs(cleanRoot)
|
||||||
|
if err != nil {
|
||||||
|
return "", false, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
||||||
|
}
|
||||||
|
info, err := os.Lstat(rootAbs)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return rootAbs, false, nil
|
||||||
|
}
|
||||||
|
return "", false, fmt.Errorf("cleanup policy %s: stat root %q: %w", policy, rootAbs, err)
|
||||||
|
}
|
||||||
|
if info.Mode()&os.ModeSymlink != 0 {
|
||||||
|
return "", false, fmt.Errorf("cleanup policy %s: refusing to clean symlink root %q", policy, rootAbs)
|
||||||
|
}
|
||||||
|
if !info.IsDir() {
|
||||||
|
return "", false, fmt.Errorf("cleanup policy %s: root %q is not a directory", policy, rootAbs)
|
||||||
|
}
|
||||||
|
return rootAbs, true, nil
|
||||||
|
}
|
||||||
103
internal/app/cleanup_targets_test.go
Normal file
103
internal/app/cleanup_targets_test.go
Normal file
@@ -0,0 +1,103 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestCleanValidateScopedDirAndFile(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
dirTarget := filepath.Join(root, "runs", "run-1")
|
||||||
|
fileTarget := filepath.Join(root, "cache", "a.flac")
|
||||||
|
if err := os.MkdirAll(dirTarget, 0o755); err != nil {
|
||||||
|
t.Fatalf("MkdirAll(dirTarget) error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(filepath.Dir(fileTarget), 0o755); err != nil {
|
||||||
|
t.Fatalf("MkdirAll(file parent) error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(fileTarget, []byte("audio"), 0o644); err != nil {
|
||||||
|
t.Fatalf("WriteFile(fileTarget) error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := validateScopedDir(root, dirTarget, "test.dir"); err != nil {
|
||||||
|
t.Fatalf("validateScopedDir() error = %v", err)
|
||||||
|
}
|
||||||
|
if _, err := validateScopedFile(root, fileTarget, "test.file"); err != nil {
|
||||||
|
t.Fatalf("validateScopedFile() error = %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCleanValidateScopedTargetSafetyRules(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
outside := t.TempDir()
|
||||||
|
target := filepath.Join(root, "runs", "run-1")
|
||||||
|
if err := os.MkdirAll(target, 0o755); err != nil {
|
||||||
|
t.Fatalf("MkdirAll(target) error = %v", err)
|
||||||
|
}
|
||||||
|
fileTarget := filepath.Join(root, "cache", "a.flac")
|
||||||
|
if err := os.MkdirAll(filepath.Dir(fileTarget), 0o755); err != nil {
|
||||||
|
t.Fatalf("MkdirAll(file parent) error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(fileTarget, []byte("audio"), 0o644); err != nil {
|
||||||
|
t.Fatalf("WriteFile(fileTarget) error = %v", err)
|
||||||
|
}
|
||||||
|
symlinkTarget := filepath.Join(root, "symlink")
|
||||||
|
if err := os.Symlink(target, symlinkTarget); err != nil {
|
||||||
|
t.Fatalf("Symlink() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := validateScopedDir(root, root, "test.root"); err == nil || !strings.Contains(err.Error(), "refusing to delete root directory") {
|
||||||
|
t.Fatalf("validateScopedDir(root) error = %v, want root deletion rejection", err)
|
||||||
|
}
|
||||||
|
if _, err := validateScopedDir(root, filepath.Join(outside, "x"), "test.outside"); err == nil || !strings.Contains(err.Error(), "outside root") {
|
||||||
|
t.Fatalf("validateScopedDir(outside) error = %v, want outside-root rejection", err)
|
||||||
|
}
|
||||||
|
if _, err := validateScopedDir(root, fileTarget, "test.file-as-dir"); err == nil || !strings.Contains(err.Error(), "is not a directory") {
|
||||||
|
t.Fatalf("validateScopedDir(file) error = %v, want not-a-directory rejection", err)
|
||||||
|
}
|
||||||
|
if _, err := validateScopedFile(root, target, "test.dir-as-file"); err == nil || !strings.Contains(err.Error(), "is a directory") {
|
||||||
|
t.Fatalf("validateScopedFile(dir) error = %v, want is-a-directory rejection", err)
|
||||||
|
}
|
||||||
|
if _, err := validateScopedDir(root, symlinkTarget, "test.symlink"); err == nil || !strings.Contains(err.Error(), "refusing to delete symlink path") {
|
||||||
|
t.Fatalf("validateScopedDir(symlink) error = %v, want symlink rejection", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCleanableRootChildrenRejectsSymlinkChild(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
realChild := filepath.Join(root, "runs")
|
||||||
|
if err := os.MkdirAll(realChild, 0o755); err != nil {
|
||||||
|
t.Fatalf("MkdirAll(realChild) error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink(realChild, filepath.Join(root, "link")); err != nil {
|
||||||
|
t.Fatalf("Symlink() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, _, err := cleanableRootChildren(root, "test.root.children")
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "refusing to delete symlink path") {
|
||||||
|
t.Fatalf("cleanableRootChildren() error = %v, want symlink rejection", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCleanValidateScopedTargetMissing(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
missingDir := filepath.Join(root, "runs", "missing")
|
||||||
|
got, err := validateScopedDir(root, missingDir, "test.missing")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("validateScopedDir(missing) error = %v", err)
|
||||||
|
}
|
||||||
|
if got.Exists {
|
||||||
|
t.Fatalf("validateScopedDir(missing).Exists = true, want false")
|
||||||
|
}
|
||||||
|
|
||||||
|
missingFile := filepath.Join(root, "cache", "missing.flac")
|
||||||
|
got, err = validateScopedFile(root, missingFile, "test.missing.file")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("validateScopedFile(missing) error = %v", err)
|
||||||
|
}
|
||||||
|
if got.Exists {
|
||||||
|
t.Fatalf("validateScopedFile(missing).Exists = true, want false")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -7,7 +7,7 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
)
|
)
|
||||||
|
|
||||||
var supportedCommands = []string{"run", "run-stage", "resume", "analyze", "publish", "clean", "session"}
|
var supportedCommands = []string{"run", "run-stage", "analyze", "publish", "clean", "session"}
|
||||||
|
|
||||||
// Execute dispatches CLI commands and returns a process exit code.
|
// Execute dispatches CLI commands and returns a process exit code.
|
||||||
func Execute(args []string, stdout, stderr io.Writer) int {
|
func Execute(args []string, stdout, stderr io.Writer) int {
|
||||||
@@ -24,8 +24,6 @@ func Execute(args []string, stdout, stderr io.Writer) int {
|
|||||||
switch cmd {
|
switch cmd {
|
||||||
case "run":
|
case "run":
|
||||||
err = Run(ctx, cmdArgs, stdout)
|
err = Run(ctx, cmdArgs, stdout)
|
||||||
case "resume":
|
|
||||||
err = Resume(ctx, cmdArgs, stdout)
|
|
||||||
case "run-stage":
|
case "run-stage":
|
||||||
err = RunStage(ctx, cmdArgs, stdout)
|
err = RunStage(ctx, cmdArgs, stdout)
|
||||||
case "analyze":
|
case "analyze":
|
||||||
|
|||||||
@@ -31,10 +31,9 @@ func TestExecuteValidCommands(t *testing.T) {
|
|||||||
args []string
|
args []string
|
||||||
wantOut string
|
wantOut string
|
||||||
}{
|
}{
|
||||||
{name: "run", args: []string{"run", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "narratio run: session 2026-05-03; executed=9 skipped=0; manifest="},
|
{name: "run", args: []string{"run", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "narratio run: session 2026-05-03; executed=10 skipped=0; manifest="},
|
||||||
{name: "session plan", args: []string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "prepare: skip\ntranscribe: skip\nmerge: skip\npolish: skip\nnormalize: skip\ntrim: skip\nanalyze: skip\npublish: skip\nnotify: skip"},
|
{name: "session plan", args: []string{"session", "plan", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "prepare: skip\ntranscribe: skip\nmerge: skip\npolish: skip\nnormalize: skip\ntrim: skip\nrender: skip\nanalyze: skip\npublish: skip\nnotify: skip"},
|
||||||
{name: "session status", args: []string{"session", "status", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "Session: 2026-05-03"},
|
{name: "session status", args: []string{"session", "status", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "Session: 2026-05-03"},
|
||||||
{name: "resume", args: []string{"resume", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "narratio resume: session 2026-05-03 has no remaining stages"},
|
|
||||||
{name: "run-stage", args: []string{"run-stage", "polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "narratio run-stage: stage=polish executed=0 skipped=1 force=false; manifest="},
|
{name: "run-stage", args: []string{"run-stage", "polish", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, wantOut: "narratio run-stage: stage=polish executed=0 skipped=1 force=false; manifest="},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -66,7 +65,7 @@ func TestExecuteMissingRequiredFlags(t *testing.T) {
|
|||||||
{name: "run missing session", args: []string{"run"}, want: "run: session_id is required"},
|
{name: "run missing session", args: []string{"run"}, want: "run: session_id is required"},
|
||||||
{name: "plan old top-level removed", args: []string{"plan"}, want: `unknown command: "plan"`},
|
{name: "plan old top-level removed", args: []string{"plan"}, want: `unknown command: "plan"`},
|
||||||
{name: "status old top-level removed", args: []string{"status"}, want: `unknown command: "status"`},
|
{name: "status old top-level removed", args: []string{"status"}, want: `unknown command: "status"`},
|
||||||
{name: "resume missing session", args: []string{"resume"}, want: "resume: session_id is required"},
|
{name: "resume removed", args: []string{"resume"}, want: `unknown command: "resume"`},
|
||||||
{name: "run-stage missing name", args: []string{"run-stage", "--config", "a", "--session", "b"}, want: "run-stage: expected stage name and session_id"},
|
{name: "run-stage missing name", args: []string{"run-stage", "--config", "a", "--session", "b"}, want: "run-stage: expected stage name and session_id"},
|
||||||
{name: "run-stage missing session", args: []string{"run-stage", "polish"}, want: "run-stage: expected stage name and session_id"},
|
{name: "run-stage missing session", args: []string{"run-stage", "polish"}, want: "run-stage: expected stage name and session_id"},
|
||||||
{name: "run missing config uses defaults", args: []string{"run", "2026-05-03", "--session", "session.yml"}, want: "run: no pipeline config path provided and no default pipeline config found; searched:"},
|
{name: "run missing config uses defaults", args: []string{"run", "2026-05-03", "--session", "session.yml"}, want: "run: no pipeline config path provided and no default pipeline config found; searched:"},
|
||||||
@@ -332,7 +331,7 @@ func TestExecuteUsesDefaultPipelineConfigPathWhenConfigFlagOmitted(t *testing.T)
|
|||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
if !strings.Contains(stdout.String(), "narratio run: session 2026-05-03; executed=9 skipped=0; manifest=") {
|
if !strings.Contains(stdout.String(), "narratio run: session 2026-05-03; executed=10 skipped=0; manifest=") {
|
||||||
t.Fatalf("stdout = %q, want successful run output", stdout.String())
|
t.Fatalf("stdout = %q, want successful run output", stdout.String())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
@@ -59,7 +58,7 @@ func loadCommandConfig(ctx context.Context, pipelineFlag, campaignFlag, campaign
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, missingSessionConfigError(discoveredSession.Searched, err.Error())
|
return nil, missingSessionConfigError(discoveredSession.Searched, err.Error())
|
||||||
}
|
}
|
||||||
sessionTempPath, err := downloadRemoteSessionConfig(ctx, store, remoteKey)
|
sessionTempPath, err := storage.DownloadObjectToTemp(ctx, store, remoteKey, "narratio-session-*.yml")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, missingSessionConfigError(discoveredSession.Searched, fmt.Sprintf("remote session %q download failed: %v", remoteKey, err))
|
return nil, missingSessionConfigError(discoveredSession.Searched, fmt.Sprintf("remote session %q download failed: %v", remoteKey, err))
|
||||||
}
|
}
|
||||||
@@ -134,21 +133,6 @@ func findRemoteSessionConfig(ctx context.Context, store storage.ObjectStore, ses
|
|||||||
return storage.ObjectInfo{}, fmt.Errorf("remote session %q not found", remoteKey)
|
return storage.ObjectInfo{}, fmt.Errorf("remote session %q not found", remoteKey)
|
||||||
}
|
}
|
||||||
|
|
||||||
func downloadRemoteSessionConfig(ctx context.Context, store storage.ObjectStore, remoteKey string) (string, error) {
|
|
||||||
f, err := os.CreateTemp("", "narratio-session-*.yml")
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("create temp file: %w", err)
|
|
||||||
}
|
|
||||||
path := f.Name()
|
|
||||||
if err := f.Close(); err != nil {
|
|
||||||
return "", fmt.Errorf("close temp file %q: %w", path, err)
|
|
||||||
}
|
|
||||||
if err := store.Download(ctx, remoteKey, path); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return filepath.Clean(path), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func s3BucketName(cfg *config.PipelineConfig) string {
|
func s3BucketName(cfg *config.PipelineConfig) string {
|
||||||
if cfg == nil || cfg.Storage.S3 == nil {
|
if cfg == nil || cfg.Storage.S3 == nil {
|
||||||
return ""
|
return ""
|
||||||
|
|||||||
131
internal/app/operator_artifact_rendering.go
Normal file
131
internal/app/operator_artifact_rendering.go
Normal file
@@ -0,0 +1,131 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifactpolicy"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
func buildHelperArtifactCatalog(cfg *config.Config) (*artifacts.ArtifactCatalog, error) {
|
||||||
|
catalog := artifacts.NewArtifactCatalog()
|
||||||
|
if err := catalog.RegisterBuiltIns(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
configured := map[string]artifacts.ConfiguredArtifactDefinition{}
|
||||||
|
if cfg.Pipeline.Scriptorium != nil {
|
||||||
|
for key, item := range cfg.Pipeline.Scriptorium.Artifacts {
|
||||||
|
configured[key] = artifacts.ConfiguredArtifactDefinition{Enabled: item.Enabled, OutputPath: item.OutputPath}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := catalog.RegisterConfiguredArtifacts(configured, nil); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return catalog, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeArtifactList(out io.Writer, cfg *config.Config, catalog *artifacts.ArtifactCatalog, locks *effectiveLocks, publishedRemoteState map[string]string) {
|
||||||
|
lockSet := lockSourceSet(locks.All)
|
||||||
|
fmt.Fprintln(out, "Built-in:")
|
||||||
|
for _, transcript := range artifacts.RuntimeTranscriptArtifacts() {
|
||||||
|
writeArtifactLine(out, transcript.SourceID, lockSet)
|
||||||
|
}
|
||||||
|
writeArtifactLine(out, artifacts.ArtifactBoundsSession, lockSet)
|
||||||
|
fmt.Fprintln(out, "Configured:")
|
||||||
|
for _, entry := range catalog.ListConfigured() {
|
||||||
|
writeArtifactLine(out, entry.SourceID, lockSet)
|
||||||
|
}
|
||||||
|
fmt.Fprintln(out, "Previous-session:")
|
||||||
|
for _, req := range artifacts.CollectPreviousArtifactRequirements(configuredScriptoriumArtifacts(cfg)) {
|
||||||
|
fmt.Fprintf(out, "- %s required=%t\n", artifactpolicy.PreviousSessionSourceID(req.Name), req.Required)
|
||||||
|
}
|
||||||
|
fmt.Fprintln(out, "Published:")
|
||||||
|
for _, rule := range cfg.Pipeline.Publish.Outputs {
|
||||||
|
writePublishedOutputLine(out, rule, catalog, lockSet, publishedRemoteState)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeArtifactLine(out io.Writer, source string, lockSet map[string]config.PublishLockRule) {
|
||||||
|
parts := []string{source}
|
||||||
|
if _, ok := lockSet[source]; ok {
|
||||||
|
parts = append(parts, "locked")
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "- %s\n", strings.Join(parts, " "))
|
||||||
|
}
|
||||||
|
|
||||||
|
func writePublishedOutputLine(out io.Writer, rule config.PublishOutputRule, catalog *artifacts.ArtifactCatalog, lockSet map[string]config.PublishLockRule, remoteState map[string]string) {
|
||||||
|
source := strings.TrimSpace(rule.Source)
|
||||||
|
parts := []string{source}
|
||||||
|
if _, ok := lockSet[source]; ok {
|
||||||
|
parts = append(parts, "locked")
|
||||||
|
}
|
||||||
|
dest, showDest, err := helperPublishedOutputDest(rule, catalog)
|
||||||
|
if err != nil {
|
||||||
|
parts = append(parts, "remote=error")
|
||||||
|
fmt.Fprintf(out, "- %s\n", strings.Join(parts, " "))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if showDest {
|
||||||
|
parts = append(parts, "dest="+dest)
|
||||||
|
}
|
||||||
|
if state := remoteState[publishedOutputRemoteStateKey(source, dest)]; state != "" {
|
||||||
|
parts = append(parts, state)
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "- %s\n", strings.Join(parts, " "))
|
||||||
|
}
|
||||||
|
|
||||||
|
func remotePublishedOutputAvailability(ctx context.Context, cfg *config.Config, store storage.ObjectStore, catalog *artifacts.ArtifactCatalog) map[string]string {
|
||||||
|
out := map[string]string{}
|
||||||
|
sessionPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
for _, rule := range cfg.Pipeline.Publish.Outputs {
|
||||||
|
source := strings.TrimSpace(rule.Source)
|
||||||
|
dest, _, err := helperPublishedOutputDest(rule, catalog)
|
||||||
|
if err != nil {
|
||||||
|
out[publishedOutputRemoteStateKey(source, "")] = "remote=error"
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
key := artifacts.S3PublishedOutputKey(sessionPrefix, dest)
|
||||||
|
if exists, err := store.Exists(ctx, key); err == nil && exists {
|
||||||
|
out[publishedOutputRemoteStateKey(source, dest)] = "remote=published"
|
||||||
|
} else if err != nil {
|
||||||
|
out[publishedOutputRemoteStateKey(source, dest)] = "remote=error"
|
||||||
|
} else {
|
||||||
|
out[publishedOutputRemoteStateKey(source, dest)] = "remote=missing"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func helperPublishedOutputDest(rule config.PublishOutputRule, catalog *artifacts.ArtifactCatalog) (string, bool, error) {
|
||||||
|
source := strings.TrimSpace(rule.Source)
|
||||||
|
normalized, err := artifactpolicy.ResolvePublishedDestination(source, rule.Dest, helperConfiguredOutputPathMap(catalog))
|
||||||
|
if err != nil {
|
||||||
|
return "", false, err
|
||||||
|
}
|
||||||
|
entry, ok := catalog.Lookup(source)
|
||||||
|
showDest := !ok || strings.TrimSpace(entry.CanonicalRelPath) != normalized
|
||||||
|
return normalized, showDest, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func helperConfiguredOutputPathMap(catalog *artifacts.ArtifactCatalog) map[string]string {
|
||||||
|
out := map[string]string{}
|
||||||
|
if catalog == nil {
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
for _, entry := range catalog.ListConfigured() {
|
||||||
|
if strings.TrimSpace(entry.ConfiguredKey) == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out[entry.ConfiguredKey] = strings.TrimSpace(entry.CanonicalRelPath)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func publishedOutputRemoteStateKey(source, dest string) string {
|
||||||
|
return strings.TrimSpace(source) + "\x00" + strings.TrimSpace(dest)
|
||||||
|
}
|
||||||
39
internal/app/operator_artifacts_list.go
Normal file
39
internal/app/operator_artifacts_list.go
Normal file
@@ -0,0 +1,39 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ArtifactsList lists effective artifact sources.
|
||||||
|
func ArtifactsList(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
fs := flag.NewFlagSet("artifacts list", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(io.Discard)
|
||||||
|
var flags commonConfigFlags
|
||||||
|
var remote bool
|
||||||
|
addCommonConfigFlags(fs, &flags)
|
||||||
|
fs.BoolVar(&remote, "remote", false, "inspect remote publish availability")
|
||||||
|
if err := parseSessionAwareFlags("artifacts list", fs, args, &flags.sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
|
return fmt.Errorf("artifacts list: session_id is required")
|
||||||
|
}
|
||||||
|
cfg, store, locks, _, err := loadHelperContext(ctx, flags, remote)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("artifacts list: %w", err)
|
||||||
|
}
|
||||||
|
catalog, err := buildHelperArtifactCatalog(cfg)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("artifacts list: %w", err)
|
||||||
|
}
|
||||||
|
publishedRemoteState := map[string]string{}
|
||||||
|
if remote && store != nil {
|
||||||
|
publishedRemoteState = remotePublishedOutputAvailability(ctx, cfg, store, catalog)
|
||||||
|
}
|
||||||
|
writeArtifactList(out, cfg, catalog, locks, publishedRemoteState)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
140
internal/app/operator_findings.go
Normal file
140
internal/app/operator_findings.go
Normal file
@@ -0,0 +1,140 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
|
)
|
||||||
|
|
||||||
|
type finding struct {
|
||||||
|
Severity string
|
||||||
|
Category string
|
||||||
|
Message string
|
||||||
|
}
|
||||||
|
|
||||||
|
type findingError struct {
|
||||||
|
count int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e findingError) Error() string {
|
||||||
|
return fmt.Sprintf("%d validation error(s)", e.count)
|
||||||
|
}
|
||||||
|
|
||||||
|
func renderFindings(out io.Writer, campaign, sessionID string, findings []finding) error {
|
||||||
|
if campaign != "" || sessionID != "" {
|
||||||
|
fmt.Fprintf(out, "Campaign: %s\n", campaign)
|
||||||
|
fmt.Fprintf(out, "Session: %s\n\n", sessionID)
|
||||||
|
}
|
||||||
|
errorsCount := 0
|
||||||
|
for _, f := range findings {
|
||||||
|
if f.Severity == "ERROR" {
|
||||||
|
errorsCount++
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "%-5s %-10s %s\n", f.Severity, f.Category, f.Message)
|
||||||
|
}
|
||||||
|
if errorsCount > 0 {
|
||||||
|
return findingError{count: errorsCount}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func okFinding(category, msg string) finding { return finding{"OK", category, msg} }
|
||||||
|
func infoFinding(category, msg string) finding { return finding{"INFO", category, msg} }
|
||||||
|
func warnFinding(category, msg string) finding { return finding{"WARN", category, msg} }
|
||||||
|
func errorFinding(category, msg string) finding { return finding{"ERROR", category, msg} }
|
||||||
|
|
||||||
|
func sessionSourceSummary(cfg *config.Config) string {
|
||||||
|
source := cfg.SessionSource.Source
|
||||||
|
if source == "" {
|
||||||
|
source = "session_config"
|
||||||
|
}
|
||||||
|
if cfg.SessionSource.S3Key != "" {
|
||||||
|
return source + " " + cfg.SessionSource.S3Key
|
||||||
|
}
|
||||||
|
return source + " " + cfg.SessionPath
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateStableInputFindings(cfg *config.Config) []finding {
|
||||||
|
checks := inspectStableInputs(cfg)
|
||||||
|
out := make([]finding, 0, len(checks))
|
||||||
|
for _, check := range checks {
|
||||||
|
if check.Err != nil {
|
||||||
|
msg := check.Name + ": " + check.Err.Error()
|
||||||
|
if strings.TrimSpace(check.Path) != "" {
|
||||||
|
msg = fmt.Sprintf("%s missing: %v", check.Name, check.Err)
|
||||||
|
}
|
||||||
|
out = append(out, errorFinding("inputs", msg))
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out = append(out, okFinding("inputs", check.Name+": "+check.Path))
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveHelperConfigRelativePath(input config.ResolvedInputFile) (string, error) {
|
||||||
|
if strings.TrimSpace(input.ConfigPath) == "" {
|
||||||
|
return "", fmt.Errorf("source config path is required")
|
||||||
|
}
|
||||||
|
path := strings.TrimSpace(input.Path)
|
||||||
|
if path == "" {
|
||||||
|
return "", fmt.Errorf("path is required")
|
||||||
|
}
|
||||||
|
if filepath.IsAbs(path) {
|
||||||
|
return filepath.Clean(path), nil
|
||||||
|
}
|
||||||
|
return filepath.Clean(filepath.Join(filepath.Dir(input.ConfigPath), path)), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateLocalAudioFindings(cfg *config.Config) []finding {
|
||||||
|
check := inspectLocalAudioPresence(cfg)
|
||||||
|
if !check.Checked {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if check.Err != nil {
|
||||||
|
return []finding{errorFinding("audio", check.Err.Error())}
|
||||||
|
}
|
||||||
|
return []finding{okFinding("audio", fmt.Sprintf("%d local audio file(s)", len(check.Paths)))}
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateRemoteAudioFinding(ctx context.Context, cfg *config.Config, store storage.ObjectStore) finding {
|
||||||
|
check := inspectRemoteAudioPresence(ctx, cfg, store)
|
||||||
|
if check.Err != nil {
|
||||||
|
return errorFinding("audio", check.Err.Error())
|
||||||
|
}
|
||||||
|
return okFinding("audio", fmt.Sprintf("%d remote .flac object(s)", len(check.Keys)))
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadLocalManifest(ctx context.Context, path string) (*manifest.Manifest, error) {
|
||||||
|
if _, err := os.Stat(path); err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
store := &manifest.LocalStore{}
|
||||||
|
return store.Load(ctx, path)
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeStageStatuses(out io.Writer, m *manifest.Manifest) {
|
||||||
|
if m == nil || len(m.Stages) == 0 {
|
||||||
|
fmt.Fprintln(out, "stages: no stages recorded")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
fmt.Fprintln(out, "stages:")
|
||||||
|
names := make([]string, 0, len(m.Stages))
|
||||||
|
for name := range m.Stages {
|
||||||
|
names = append(names, name)
|
||||||
|
}
|
||||||
|
sort.Strings(names)
|
||||||
|
for _, name := range names {
|
||||||
|
fmt.Fprintf(out, "- %s: %s\n", name, m.Stages[name].Status)
|
||||||
|
}
|
||||||
|
}
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -593,10 +593,57 @@ func TestExecuteLocksRequireSessionID(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestExecuteLocksMutationRejectsSessionIDMismatch(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
args []string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "add mismatch",
|
||||||
|
args: []string{
|
||||||
|
"session", "locks", "add", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||||
|
"--session-id", "2026-05-04",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "remove mismatch",
|
||||||
|
args: []string{
|
||||||
|
"session", "locks", "remove", "2026-05-03", "narratio.transcript.final_trimmed",
|
||||||
|
"--session-id", "2026-05-04",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute(tt.args, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "does not match expected session id") {
|
||||||
|
t.Fatalf("stderr = %q, want session-id mismatch guidance", stderr.String())
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestExecuteLocksCannotModifyStaticLocks(t *testing.T) {
|
func TestExecuteLocksCannotModifyStaticLocks(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
addStaticArchiveLockToPipelineConfig(t, pipelinePath, "narratio.transcript.final_trimmed")
|
addStaticPublishLockToPipelineConfig(t, pipelinePath, "narratio.transcript.final_trimmed")
|
||||||
fake := &storage.FakeBackend{}
|
fake := &storage.FakeBackend{}
|
||||||
var storeInitCalls int
|
var storeInitCalls int
|
||||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
@@ -683,10 +730,10 @@ func addSessionTemplateToCampaign(t *testing.T, campaignPath, templateFile strin
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteArtifactsListRemoteReportsPromotedAvailability(t *testing.T) {
|
func TestExecuteArtifactsListRemoteReportsPublishedAvailability(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
addArchivePromotionsToPipeline(t, pipelinePath, `
|
addPublishOutputsToPipeline(t, pipelinePath, `
|
||||||
outputs:
|
outputs:
|
||||||
- source: narratio.transcript.final_trimmed
|
- source: narratio.transcript.final_trimmed
|
||||||
dest: transcripts/final.trimmed.json
|
dest: transcripts/final.trimmed.json
|
||||||
@@ -714,14 +761,14 @@ func TestExecuteArtifactsListRemoteReportsPromotedAvailability(t *testing.T) {
|
|||||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
if !strings.Contains(stdout.String(), "narratio.transcript.final_trimmed remote=published") {
|
if !strings.Contains(stdout.String(), "narratio.transcript.final_trimmed remote=published") {
|
||||||
t.Fatalf("stdout = %q, want promoted remote availability", stdout.String())
|
t.Fatalf("stdout = %q, want published remote availability", stdout.String())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteArtifactsListRemoteUsesPromotionDestinations(t *testing.T) {
|
func TestExecuteArtifactsListRemoteUsesPublishOutputDestinations(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
addArchivePromotionsToPipeline(t, pipelinePath, `
|
addPublishOutputsToPipeline(t, pipelinePath, `
|
||||||
outputs:
|
outputs:
|
||||||
- source: narratio.transcript.final
|
- source: narratio.transcript.final
|
||||||
dest: transcripts/full.json
|
dest: transcripts/full.json
|
||||||
@@ -771,7 +818,7 @@ func TestExecuteArtifactsListRemoteUsesPromotionDestinations(t *testing.T) {
|
|||||||
func TestExecuteStatusReportsRemoteArtifactCatalog(t *testing.T) {
|
func TestExecuteStatusReportsRemoteArtifactCatalog(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
addArchivePromotionsToPipeline(t, pipelinePath, `
|
addPublishOutputsToPipeline(t, pipelinePath, `
|
||||||
outputs:
|
outputs:
|
||||||
- source: narratio.transcript.final_trimmed
|
- source: narratio.transcript.final_trimmed
|
||||||
dest: transcripts/final.trimmed.json
|
dest: transcripts/final.trimmed.json
|
||||||
@@ -782,7 +829,7 @@ func TestExecuteStatusReportsRemoteArtifactCatalog(t *testing.T) {
|
|||||||
`)
|
`)
|
||||||
fake := &storage.FakeBackend{}
|
fake := &storage.FakeBackend{}
|
||||||
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
|
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
|
||||||
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
manifestKey, runIDKey := artifacts.ResolveCurrentStateKeys(sessionPrefix)
|
||||||
trimmedKey := artifacts.S3PublishedOutputKey(sessionPrefix, "transcripts/final.trimmed.json")
|
trimmedKey := artifacts.S3PublishedOutputKey(sessionPrefix, "transcripts/final.trimmed.json")
|
||||||
fullKey := artifacts.S3PublishedOutputKey(sessionPrefix, "transcripts/full.json")
|
fullKey := artifacts.S3PublishedOutputKey(sessionPrefix, "transcripts/full.json")
|
||||||
lockKey := artifacts.S3SessionLocksKey(sessionPrefix)
|
lockKey := artifacts.S3SessionLocksKey(sessionPrefix)
|
||||||
@@ -828,7 +875,7 @@ func TestExecuteStatusReportsRemoteArtifactCatalog(t *testing.T) {
|
|||||||
func TestExecuteStatusReportsRemoteArtifactCatalogErrorsWithoutFailing(t *testing.T) {
|
func TestExecuteStatusReportsRemoteArtifactCatalogErrorsWithoutFailing(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
addArchivePromotionsToPipeline(t, pipelinePath, `
|
addPublishOutputsToPipeline(t, pipelinePath, `
|
||||||
outputs:
|
outputs:
|
||||||
- source: narratio.transcript.final_trimmed
|
- source: narratio.transcript.final_trimmed
|
||||||
dest: transcripts/final.trimmed.json
|
dest: transcripts/final.trimmed.json
|
||||||
@@ -861,9 +908,88 @@ func TestExecuteStatusReportsRemoteArtifactCatalogErrorsWithoutFailing(t *testin
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteArchiveLoadsRemoteLocks(t *testing.T) {
|
func TestExecuteStatusReportsMissingRemoteCurrentStateWithoutFailing(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidArchiveConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "status", "2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "Remote publish: missing or unavailable: remote current run pointer missing") {
|
||||||
|
t.Fatalf("stdout = %q, want missing remote current-state line", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteStatusReportsPreviousStateReadinessWithoutFailing(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
replaceInFileOrFatal(t, pipelinePath, "source: narratio.artifact.session_recap", "source: narratio.previous_session.artifact.session_recap")
|
||||||
|
replaceInFileOrFatal(t, sessionPath, "session_id: 2026-05-03\n", "session_id: 2026-05-03\nprevious_session_id: 2026-04-26\n")
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "status", "2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "Previous-session artifacts: unavailable: remote current run pointer missing") {
|
||||||
|
t.Fatalf("stdout = %q, want previous readiness unavailable line", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionValidateReportsPreviousStateFindingAndReturnsFindingError(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
|
||||||
|
replaceInFileOrFatal(t, pipelinePath, "source: narratio.artifact.session_recap", "source: narratio.previous_session.artifact.session_recap")
|
||||||
|
replaceInFileOrFatal(t, sessionPath, "session_id: 2026-05-03\n", "session_id: 2026-05-03\nprevious_session_id: 2026-04-26\n")
|
||||||
|
fake := &storage.FakeBackend{}
|
||||||
|
var storeInitCalls int
|
||||||
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "validate", "2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code == 0 {
|
||||||
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "ERROR previous") {
|
||||||
|
t.Fatalf("stdout = %q, want previous finding error", stdout.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "remote current run pointer missing") {
|
||||||
|
t.Fatalf("stdout = %q, want missing run pointer finding", stdout.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "validation error(s)") {
|
||||||
|
t.Fatalf("stderr = %q, want finding error summary", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecutePublishLoadsRemoteLocks(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidPublishRunConfigFiles(t, workspaceRoot)
|
||||||
fake := &storage.FakeBackend{}
|
fake := &storage.FakeBackend{}
|
||||||
lockKey := artifacts.S3SessionLocksKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"))
|
lockKey := artifacts.S3SessionLocksKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"))
|
||||||
fake.SeedObject(storage.FakeObject{Key: lockKey, Data: []byte("locks:\n - source: narratio.transcript.final_trimmed\n reason: remote review\n")})
|
fake.SeedObject(storage.FakeObject{Key: lockKey, Data: []byte("locks:\n - source: narratio.transcript.final_trimmed\n reason: remote review\n")})
|
||||||
@@ -871,11 +997,13 @@ func TestExecuteArchiveLoadsRemoteLocks(t *testing.T) {
|
|||||||
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
|
||||||
|
|
||||||
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||||
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze"} {
|
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "analyze"} {
|
||||||
// The publish stage only checks the manifest statuses and source files.
|
// The publish stage only checks the manifest statuses and source files.
|
||||||
_ = stageName
|
_ = stageName
|
||||||
}
|
}
|
||||||
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.trimmed.json"), `{"segments":[]}`)
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.trimmed.json"), `{"segments":[]}`)
|
||||||
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.md"), "# final\n")
|
||||||
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.trimmed.md"), "# final trimmed\n")
|
||||||
|
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
@@ -883,28 +1011,43 @@ func TestExecuteArchiveLoadsRemoteLocks(t *testing.T) {
|
|||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
promotedKey := artifacts.S3PublishedOutputKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"), "transcripts/final.trimmed.json")
|
publishedKey := artifacts.S3PublishedOutputKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"), "transcripts/final.trimmed.json")
|
||||||
if _, ok := fake.Objects[promotedKey]; ok {
|
if _, ok := fake.Objects[publishedKey]; ok {
|
||||||
t.Fatalf("locked promoted key %q was uploaded", promotedKey)
|
t.Fatalf("locked published key %q was uploaded", publishedKey)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func addArchivePromotionsToPipeline(t *testing.T, pipelinePath, archiveYAML string) {
|
func addPublishOutputsToPipeline(t *testing.T, pipelinePath, publishYAML string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
data, err := os.ReadFile(pipelinePath)
|
data, err := os.ReadFile(pipelinePath)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("read pipeline: %v", err)
|
t.Fatalf("read pipeline: %v", err)
|
||||||
}
|
}
|
||||||
updated := strings.Replace(string(data), " upload_run: false\n", " upload_run: false\n"+archiveYAML, 1)
|
updated := strings.Replace(string(data), " upload_run: false\n", " upload_run: false\n"+publishYAML, 1)
|
||||||
if updated == string(data) {
|
if updated == string(data) {
|
||||||
t.Fatalf("pipeline %q did not contain archive upload_run marker", pipelinePath)
|
t.Fatalf("pipeline %q did not contain publish upload_run marker", pipelinePath)
|
||||||
}
|
}
|
||||||
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
|
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
|
||||||
t.Fatalf("write pipeline: %v", err)
|
t.Fatalf("write pipeline: %v", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func writeValidArchiveConfigFiles(t *testing.T, workspaceRoot string) (string, string, string) {
|
func replaceInFileOrFatal(t *testing.T, path, old, new string) {
|
||||||
|
t.Helper()
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read %s: %v", path, err)
|
||||||
|
}
|
||||||
|
updated := strings.Replace(string(data), old, new, 1)
|
||||||
|
if updated == string(data) {
|
||||||
|
t.Fatalf("%s did not contain %q", path, old)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(path, []byte(updated), 0o644); err != nil {
|
||||||
|
t.Fatalf("write %s: %v", path, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeValidPublishRunConfigFiles(t *testing.T, workspaceRoot string) (string, string, string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
data, err := os.ReadFile(pipelinePath)
|
data, err := os.ReadFile(pipelinePath)
|
||||||
@@ -924,7 +1067,7 @@ func writeValidArchiveConfigFiles(t *testing.T, workspaceRoot string) (string, s
|
|||||||
m := manifest.New("2026-05-03", nowUTC())
|
m := manifest.New("2026-05-03", nowUTC())
|
||||||
m.Campaign = "sample-campaign"
|
m.Campaign = "sample-campaign"
|
||||||
m.RunID = "20260521T160000Z-test"
|
m.RunID = "20260521T160000Z-test"
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "analyze"} {
|
||||||
m.MarkStageSucceeded(name, nowUTC(), nil)
|
m.MarkStageSucceeded(name, nowUTC(), nil)
|
||||||
}
|
}
|
||||||
path := artifacts.SessionManifestPathForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
path := artifacts.SessionManifestPathForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
@@ -941,7 +1084,7 @@ func writeValidArchiveConfigFiles(t *testing.T, workspaceRoot string) (string, s
|
|||||||
return pipelinePath, campaignPath, sessionPath
|
return pipelinePath, campaignPath, sessionPath
|
||||||
}
|
}
|
||||||
|
|
||||||
func addStaticArchiveLockToPipelineConfig(t *testing.T, pipelinePath, source string) {
|
func addStaticPublishLockToPipelineConfig(t *testing.T, pipelinePath, source string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
data, err := os.ReadFile(pipelinePath)
|
data, err := os.ReadFile(pipelinePath)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -954,7 +1097,7 @@ func addStaticArchiveLockToPipelineConfig(t *testing.T, pipelinePath, source str
|
|||||||
1,
|
1,
|
||||||
)
|
)
|
||||||
if updated == string(data) {
|
if updated == string(data) {
|
||||||
t.Fatalf("archive section not found in pipeline config")
|
t.Fatalf("publish section not found in pipeline config")
|
||||||
}
|
}
|
||||||
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
|
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
|
||||||
t.Fatalf("write pipeline: %v", err)
|
t.Fatalf("write pipeline: %v", err)
|
||||||
|
|||||||
274
internal/app/operator_inspection.go
Normal file
274
internal/app/operator_inspection.go
Normal file
@@ -0,0 +1,274 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
type stableInputCheck struct {
|
||||||
|
Name string
|
||||||
|
Path string
|
||||||
|
Err error
|
||||||
|
}
|
||||||
|
|
||||||
|
type localAudioCheck struct {
|
||||||
|
Checked bool
|
||||||
|
Paths []string
|
||||||
|
Err error
|
||||||
|
}
|
||||||
|
|
||||||
|
type remoteAudioCheck struct {
|
||||||
|
Checked bool
|
||||||
|
Prefix string
|
||||||
|
Keys []string
|
||||||
|
Err error
|
||||||
|
}
|
||||||
|
|
||||||
|
type previousArtifactReadiness struct {
|
||||||
|
Requirements []artifacts.PreviousArtifactRequirement
|
||||||
|
MissingID bool
|
||||||
|
Err error
|
||||||
|
}
|
||||||
|
|
||||||
|
type remoteCurrentStateCheck struct {
|
||||||
|
State *RemoteCurrentState
|
||||||
|
Err error
|
||||||
|
}
|
||||||
|
|
||||||
|
type effectiveLocksCheck struct {
|
||||||
|
Locks *effectiveLocks
|
||||||
|
Err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func inspectStableInputs(cfg *config.Config) []stableInputCheck {
|
||||||
|
items := []struct {
|
||||||
|
name string
|
||||||
|
in config.ResolvedInputFile
|
||||||
|
}{
|
||||||
|
{name: "speakers", in: cfg.StableInputs.SpeakersFile},
|
||||||
|
{name: "autocorrect", in: cfg.StableInputs.AutocorrectFile},
|
||||||
|
{name: "glossary", in: cfg.StableInputs.GlossaryFile},
|
||||||
|
}
|
||||||
|
out := make([]stableInputCheck, 0, len(items))
|
||||||
|
for _, item := range items {
|
||||||
|
path, err := resolveHelperConfigRelativePath(item.in)
|
||||||
|
if err != nil {
|
||||||
|
out = append(out, stableInputCheck{Name: item.name, Err: err})
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(path); err != nil {
|
||||||
|
out = append(out, stableInputCheck{Name: item.name, Path: path, Err: err})
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out = append(out, stableInputCheck{Name: item.name, Path: path})
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func inspectLocalAudioPresence(cfg *config.Config) localAudioCheck {
|
||||||
|
if cfg.Session.Inputs.AudioS3 != nil {
|
||||||
|
return localAudioCheck{}
|
||||||
|
}
|
||||||
|
|
||||||
|
sessionDir := filepath.Dir(cfg.SessionPath)
|
||||||
|
resolved, err := resolveLocalInspectionAudioPaths(sessionDir, cfg.Session.Inputs)
|
||||||
|
if err != nil {
|
||||||
|
return localAudioCheck{Checked: true, Err: err}
|
||||||
|
}
|
||||||
|
return localAudioCheck{
|
||||||
|
Checked: true,
|
||||||
|
Paths: resolved,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func inspectRemoteAudioPresence(ctx context.Context, cfg *config.Config, store storage.ObjectStore) remoteAudioCheck {
|
||||||
|
if cfg.Session.Inputs.AudioS3 == nil {
|
||||||
|
return remoteAudioCheck{}
|
||||||
|
}
|
||||||
|
if store == nil {
|
||||||
|
return remoteAudioCheck{Checked: true, Err: fmt.Errorf("storage backend is required for remote audio checks")}
|
||||||
|
}
|
||||||
|
|
||||||
|
sessionPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
audioPrefix := artifacts.S3AudioPrefix(sessionPrefix, cfg.Session.Inputs.AudioS3.Prefix)
|
||||||
|
objects, err := store.List(ctx, audioPrefix)
|
||||||
|
if err != nil {
|
||||||
|
return remoteAudioCheck{Checked: true, Prefix: audioPrefix, Err: err}
|
||||||
|
}
|
||||||
|
|
||||||
|
keys := make([]string, 0, len(objects))
|
||||||
|
seenBase := map[string]string{}
|
||||||
|
for _, obj := range objects {
|
||||||
|
key := strings.TrimSpace(obj.Key)
|
||||||
|
if key == "" || strings.HasSuffix(key, "/") || !isInspectionFlacPath(key) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
base := path.Base(key)
|
||||||
|
if prev, exists := seenBase[base]; exists && prev != key {
|
||||||
|
return remoteAudioCheck{
|
||||||
|
Checked: true,
|
||||||
|
Prefix: audioPrefix,
|
||||||
|
Err: fmt.Errorf("duplicate s3 audio basename %q from %q and %q", base, prev, key),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
seenBase[base] = key
|
||||||
|
keys = append(keys, key)
|
||||||
|
}
|
||||||
|
sort.Strings(keys)
|
||||||
|
if len(keys) == 0 {
|
||||||
|
return remoteAudioCheck{
|
||||||
|
Checked: true,
|
||||||
|
Prefix: audioPrefix,
|
||||||
|
Err: fmt.Errorf("no .flac files found under s3 audio prefix %q", audioPrefix),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return remoteAudioCheck{
|
||||||
|
Checked: true,
|
||||||
|
Prefix: audioPrefix,
|
||||||
|
Keys: keys,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func inspectPreviousArtifactReadiness(
|
||||||
|
ctx context.Context,
|
||||||
|
cfg *config.Config,
|
||||||
|
store storage.ObjectStore,
|
||||||
|
requirements []artifacts.PreviousArtifactRequirement,
|
||||||
|
) previousArtifactReadiness {
|
||||||
|
out := previousArtifactReadiness{
|
||||||
|
Requirements: append([]artifacts.PreviousArtifactRequirement(nil), requirements...),
|
||||||
|
}
|
||||||
|
if len(requirements) == 0 {
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(cfg.Session.PreviousSessionID) == "" {
|
||||||
|
out.MissingID = true
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
if store == nil {
|
||||||
|
out.Err = fmt.Errorf("previous-session artifacts cannot be checked because storage is unavailable")
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
prefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.PreviousSessionID)
|
||||||
|
if _, err := artifacts.LoadCurrentState(ctx, store, prefix, artifacts.CurrentStateValidation{
|
||||||
|
ExpectedSessionID: strings.TrimSpace(cfg.Session.PreviousSessionID),
|
||||||
|
ExpectedCampaign: strings.TrimSpace(cfg.Session.Campaign),
|
||||||
|
ValidateRunID: true,
|
||||||
|
}); err != nil {
|
||||||
|
out.Err = fmt.Errorf("remote %v", err)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func inspectRemoteCurrentState(ctx context.Context, cfg *config.Config, store storage.ObjectStore) remoteCurrentStateCheck {
|
||||||
|
if store == nil {
|
||||||
|
return remoteCurrentStateCheck{}
|
||||||
|
}
|
||||||
|
current, err := discoverRemoteCurrentStateFn(ctx, cfg, store)
|
||||||
|
if err != nil {
|
||||||
|
return remoteCurrentStateCheck{Err: err}
|
||||||
|
}
|
||||||
|
return remoteCurrentStateCheck{State: current}
|
||||||
|
}
|
||||||
|
|
||||||
|
func inspectEffectiveLocks(ctx context.Context, cfg *config.Config, store storage.ObjectStore) effectiveLocksCheck {
|
||||||
|
locks, err := loadEffectiveLocks(ctx, cfg, store)
|
||||||
|
if err != nil {
|
||||||
|
return effectiveLocksCheck{Err: err}
|
||||||
|
}
|
||||||
|
return effectiveLocksCheck{Locks: locks}
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveLocalInspectionAudioPaths(sessionDir string, inputs config.SessionInputsConfig) ([]string, error) {
|
||||||
|
if len(inputs.AudioFiles) > 0 {
|
||||||
|
out := make([]string, 0, len(inputs.AudioFiles))
|
||||||
|
seenBase := map[string]string{}
|
||||||
|
for _, item := range inputs.AudioFiles {
|
||||||
|
resolved, err := resolveInspectionPath(sessionDir, item)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !isInspectionFlacPath(resolved) {
|
||||||
|
return nil, fmt.Errorf("audio file %q must have .flac extension", resolved)
|
||||||
|
}
|
||||||
|
if err := requireInspectionFile(resolved, "audio file"); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
base := filepath.Base(resolved)
|
||||||
|
if prev, exists := seenBase[base]; exists && prev != resolved {
|
||||||
|
return nil, fmt.Errorf("duplicate audio basename %q from %q and %q", base, prev, resolved)
|
||||||
|
}
|
||||||
|
seenBase[base] = resolved
|
||||||
|
out = append(out, resolved)
|
||||||
|
}
|
||||||
|
sort.Strings(out)
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
audioDir, err := resolveInspectionPath(sessionDir, inputs.AudioDir)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
entries, err := os.ReadDir(audioDir)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read audio directory %q: %w", audioDir, err)
|
||||||
|
}
|
||||||
|
out := make([]string, 0, len(entries))
|
||||||
|
for _, entry := range entries {
|
||||||
|
if entry.IsDir() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
full := filepath.Join(audioDir, entry.Name())
|
||||||
|
if !isInspectionFlacPath(full) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := requireInspectionFile(full, "audio file"); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, full)
|
||||||
|
}
|
||||||
|
if len(out) == 0 {
|
||||||
|
return nil, fmt.Errorf("no .flac files found in audio directory %q", audioDir)
|
||||||
|
}
|
||||||
|
sort.Strings(out)
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveInspectionPath(baseDir, inputPath string) (string, error) {
|
||||||
|
pathValue := strings.TrimSpace(inputPath)
|
||||||
|
if pathValue == "" {
|
||||||
|
return "", fmt.Errorf("path is required")
|
||||||
|
}
|
||||||
|
if filepath.IsAbs(pathValue) {
|
||||||
|
return filepath.Clean(pathValue), nil
|
||||||
|
}
|
||||||
|
return filepath.Clean(filepath.Join(baseDir, pathValue)), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func requireInspectionFile(path, label string) error {
|
||||||
|
info, err := os.Stat(path)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return fmt.Errorf("%s %q does not exist", label, path)
|
||||||
|
}
|
||||||
|
return fmt.Errorf("stat %s %q: %w", label, path, err)
|
||||||
|
}
|
||||||
|
if info.IsDir() {
|
||||||
|
return fmt.Errorf("%s %q is a directory", label, path)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func isInspectionFlacPath(path string) bool {
|
||||||
|
return strings.EqualFold(filepath.Ext(strings.TrimSpace(path)), ".flac")
|
||||||
|
}
|
||||||
172
internal/app/operator_locks.go
Normal file
172
internal/app/operator_locks.go
Normal file
@@ -0,0 +1,172 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Locks dispatches publish lock list and mutation helpers.
|
||||||
|
func Locks(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
if len(args) > 0 && !strings.HasPrefix(args[0], "-") {
|
||||||
|
switch args[0] {
|
||||||
|
case "add":
|
||||||
|
return LocksAdd(ctx, args[1:], out)
|
||||||
|
case "remove":
|
||||||
|
return LocksRemove(ctx, args[1:], out)
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("locks: unknown subcommand %q", args[0])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return LocksList(ctx, args, out)
|
||||||
|
}
|
||||||
|
|
||||||
|
// LocksList lists effective publish locks.
|
||||||
|
func LocksList(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
fs := flag.NewFlagSet("locks", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(io.Discard)
|
||||||
|
var flags commonConfigFlags
|
||||||
|
addCommonConfigFlags(fs, &flags)
|
||||||
|
if err := parseSessionAwareFlags("locks", fs, args, &flags.sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
|
return fmt.Errorf("locks: session_id is required")
|
||||||
|
}
|
||||||
|
cfg, _, locks, _, err := loadHelperContext(ctx, flags, true)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("locks: %w", err)
|
||||||
|
}
|
||||||
|
writeLocks(out, cfg, locks)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// LocksAdd adds or updates one remote lock.
|
||||||
|
func LocksAdd(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
fs := flag.NewFlagSet("locks add", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(io.Discard)
|
||||||
|
var flags commonConfigFlags
|
||||||
|
var reason string
|
||||||
|
var force bool
|
||||||
|
addCommonConfigFlags(fs, &flags)
|
||||||
|
fs.StringVar(&reason, "reason", "", "lock reason")
|
||||||
|
fs.BoolVar(&force, "force", false, "update existing remote lock")
|
||||||
|
source, err := parseSessionIDAndOnePositionalArg("locks add", "source id", fs, args, &flags.sessionID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
|
return fmt.Errorf("locks add: session_id is required")
|
||||||
|
}
|
||||||
|
cfg, store, locks, _, err := loadHelperContext(ctx, flags, true)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("locks add: %w", err)
|
||||||
|
}
|
||||||
|
if _, err := config.ValidatePublishLockRules([]config.PublishLockRule{{Source: source}}, cfg.Pipeline.Scriptorium, "locks add"); err != nil {
|
||||||
|
return fmt.Errorf("locks add: %w", err)
|
||||||
|
}
|
||||||
|
if _, ok := lockSourceSet(locks.Static)[source]; ok {
|
||||||
|
return fmt.Errorf("locks add: source %q is locked by pipeline config and cannot be modified remotely", source)
|
||||||
|
}
|
||||||
|
remoteSet := lockSourceSet(locks.Remote)
|
||||||
|
if _, exists := remoteSet[source]; exists && !force {
|
||||||
|
return fmt.Errorf("locks add: remote lock for %q already exists; pass --force to update", source)
|
||||||
|
}
|
||||||
|
remoteSet[source] = config.PublishLockRule{Source: source, Reason: strings.TrimSpace(reason)}
|
||||||
|
remoteLocks := lockMapValues(remoteSet)
|
||||||
|
if _, err := config.ValidatePublishLockRules(remoteLocks, cfg.Pipeline.Scriptorium, "locks"); err != nil {
|
||||||
|
return fmt.Errorf("locks add: %w", err)
|
||||||
|
}
|
||||||
|
if err := uploadRemoteLockStore(ctx, store, locks.Key, &config.PublishLockStore{Locks: remoteLocks}); err != nil {
|
||||||
|
return fmt.Errorf("locks add: %w", err)
|
||||||
|
}
|
||||||
|
_, err = fmt.Fprintf(out, "narratio session locks add: locked %s\n", source)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// LocksRemove removes one remote lock.
|
||||||
|
func LocksRemove(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
fs := flag.NewFlagSet("locks remove", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(io.Discard)
|
||||||
|
var flags commonConfigFlags
|
||||||
|
addCommonConfigFlags(fs, &flags)
|
||||||
|
source, err := parseSessionIDAndOnePositionalArg("locks remove", "source id", fs, args, &flags.sessionID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
|
return fmt.Errorf("locks remove: session_id is required")
|
||||||
|
}
|
||||||
|
cfg, store, locks, _, err := loadHelperContext(ctx, flags, true)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("locks remove: %w", err)
|
||||||
|
}
|
||||||
|
if _, err := config.ValidatePublishLockRules([]config.PublishLockRule{{Source: source}}, cfg.Pipeline.Scriptorium, "locks remove"); err != nil {
|
||||||
|
return fmt.Errorf("locks remove: %w", err)
|
||||||
|
}
|
||||||
|
remoteSet := lockSourceSet(locks.Remote)
|
||||||
|
if _, ok := remoteSet[source]; !ok {
|
||||||
|
if _, static := lockSourceSet(locks.Static)[source]; static {
|
||||||
|
return fmt.Errorf("locks remove: source %q is locked by pipeline config and cannot be unlocked remotely", source)
|
||||||
|
}
|
||||||
|
return fmt.Errorf("locks remove: remote lock for %q does not exist", source)
|
||||||
|
}
|
||||||
|
delete(remoteSet, source)
|
||||||
|
remoteLocks := lockMapValues(remoteSet)
|
||||||
|
if err := uploadRemoteLockStore(ctx, store, locks.Key, &config.PublishLockStore{Locks: remoteLocks}); err != nil {
|
||||||
|
return fmt.Errorf("locks remove: %w", err)
|
||||||
|
}
|
||||||
|
_, err = fmt.Fprintf(out, "narratio session locks remove: unlocked %s\n", source)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeLocks(out io.Writer, cfg *config.Config, locks *effectiveLocks) {
|
||||||
|
if locks == nil || len(locks.All) == 0 {
|
||||||
|
fmt.Fprintln(out, "Publish locks: none")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
fmt.Fprintln(out, "Publish locks:")
|
||||||
|
published := map[string]config.PublishOutputRule{}
|
||||||
|
if cfg != nil && cfg.Pipeline != nil && cfg.Pipeline.Publish != nil {
|
||||||
|
for _, rule := range cfg.Pipeline.Publish.Outputs {
|
||||||
|
published[strings.TrimSpace(rule.Source)] = rule
|
||||||
|
}
|
||||||
|
}
|
||||||
|
staticSet := lockSourceSet(locks.Static)
|
||||||
|
for _, lock := range locks.All {
|
||||||
|
origin := "remote"
|
||||||
|
if _, ok := staticSet[lock.Source]; ok {
|
||||||
|
origin = "pipeline"
|
||||||
|
}
|
||||||
|
promo := "not-published"
|
||||||
|
if _, ok := published[lock.Source]; ok {
|
||||||
|
promo = "published"
|
||||||
|
}
|
||||||
|
reason := strings.TrimSpace(lock.Reason)
|
||||||
|
if reason == "" {
|
||||||
|
reason = "(no reason)"
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "- %s origin=%s %s reason=%s\n", lock.Source, origin, promo, reason)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func lockMapValues(in map[string]config.PublishLockRule) []config.PublishLockRule {
|
||||||
|
keys := make([]string, 0, len(in))
|
||||||
|
for key := range in {
|
||||||
|
keys = append(keys, key)
|
||||||
|
}
|
||||||
|
sort.Strings(keys)
|
||||||
|
out := make([]config.PublishLockRule, 0, len(keys))
|
||||||
|
for _, key := range keys {
|
||||||
|
item := in[key]
|
||||||
|
item.Source = key
|
||||||
|
item.Reason = strings.TrimSpace(item.Reason)
|
||||||
|
out = append(out, item)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
268
internal/app/operator_session_init.go
Normal file
268
internal/app/operator_session_init.go
Normal file
@@ -0,0 +1,268 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"regexp"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
"gopkg.in/yaml.v3"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SessionInit creates a local or remote session.yml skeleton.
|
||||||
|
func SessionInit(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
fs := flag.NewFlagSet("session init", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(io.Discard)
|
||||||
|
var pipelinePath, campaignPath, campaignFilePath, sessionID, previousSessionID, date, title, output, audioS3Prefix, audioDir string
|
||||||
|
var remote, force bool
|
||||||
|
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
||||||
|
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
||||||
|
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
||||||
|
fs.StringVar(&sessionID, "session-id", "", "session identifier")
|
||||||
|
fs.StringVar(&previousSessionID, "previous-session-id", "", "previous session identifier")
|
||||||
|
fs.StringVar(&date, "date", "", "session date")
|
||||||
|
fs.StringVar(&title, "title", "", "session title")
|
||||||
|
fs.StringVar(&output, "output", "", "local output session.yml path")
|
||||||
|
fs.StringVar(&audioS3Prefix, "audio-s3-prefix", "", "session audio S3 prefix")
|
||||||
|
fs.StringVar(&audioDir, "audio-dir", "", "local audio directory")
|
||||||
|
fs.BoolVar(&remote, "remote", false, "write session.yml to S3 session prefix")
|
||||||
|
fs.BoolVar(&force, "force", false, "overwrite existing target")
|
||||||
|
if err := parseSessionAwareFlags("session init", fs, args, &sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(sessionID) == "" {
|
||||||
|
return fmt.Errorf("session init: session_id is required")
|
||||||
|
}
|
||||||
|
if (strings.TrimSpace(output) == "") == !remote {
|
||||||
|
return fmt.Errorf("session init: specify exactly one target: --output <path> or --remote")
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(audioDir) != "" && strings.TrimSpace(audioS3Prefix) != "" {
|
||||||
|
return fmt.Errorf("session init: --audio-dir and --audio-s3-prefix are mutually exclusive")
|
||||||
|
}
|
||||||
|
|
||||||
|
base, err := loadPipelineCampaignConfig(pipelinePath, campaignPath, campaignFilePath)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("session init: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
input := sessionInitInput{
|
||||||
|
Campaign: config.CampaignID(base.Campaign),
|
||||||
|
CampaignPath: base.CampaignPath,
|
||||||
|
TemplateFile: base.Campaign.SessionTemplateFile,
|
||||||
|
SessionID: sessionID,
|
||||||
|
PreviousSessionID: previousSessionID,
|
||||||
|
Date: date,
|
||||||
|
Title: title,
|
||||||
|
AudioS3Prefix: audioS3Prefix,
|
||||||
|
AudioDir: audioDir,
|
||||||
|
}
|
||||||
|
data, err := buildSessionInitYAML(input)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("session init: %w", err)
|
||||||
|
}
|
||||||
|
label := strings.TrimSpace(output)
|
||||||
|
if label == "" {
|
||||||
|
label = "remote session.yml"
|
||||||
|
}
|
||||||
|
sessionCfg, err := config.LoadSessionBytesWithOptions(label, data, config.SessionLoadOptions{
|
||||||
|
SessionID: sessionID,
|
||||||
|
PreviousSessionID: previousSessionID,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("session init: %w", err)
|
||||||
|
}
|
||||||
|
cfg, err := config.Resolve(base.PipelinePath, base.Pipeline, base.CampaignPath, base.Campaign, label, sessionCfg, config.SessionSource{Source: "session_config", LocalPath: label})
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("session init: %w", err)
|
||||||
|
}
|
||||||
|
if err := config.Validate(cfg); err != nil {
|
||||||
|
return fmt.Errorf("session init: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !remote {
|
||||||
|
if err := writeLocalFile(output, data, force); err != nil {
|
||||||
|
return fmt.Errorf("session init: %w", err)
|
||||||
|
}
|
||||||
|
_, err := fmt.Fprintf(out, "narratio session init: wrote %s\n", filepath.Clean(output))
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
store, err := newCommandObjectStore(ctx, cfg, nil)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("session init: %w", err)
|
||||||
|
}
|
||||||
|
sessionPrefix := artifacts.S3SessionPrefix(base.Pipeline.Storage.S3.RootPrefix, config.CampaignID(base.Campaign), sessionID)
|
||||||
|
key := artifacts.S3SessionConfigKey(sessionPrefix)
|
||||||
|
exists, err := store.Exists(ctx, key)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("session init: check remote session %q: %w", key, err)
|
||||||
|
}
|
||||||
|
if exists && !force {
|
||||||
|
return fmt.Errorf("session init: remote session %q already exists; pass --force to overwrite", key)
|
||||||
|
}
|
||||||
|
tmp, err := os.CreateTemp("", "narratio-session-init-*.yml")
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("session init: create temp file: %w", err)
|
||||||
|
}
|
||||||
|
tmpPath := tmp.Name()
|
||||||
|
defer func() { _ = os.Remove(tmpPath) }()
|
||||||
|
if _, err := tmp.Write(data); err != nil {
|
||||||
|
_ = tmp.Close()
|
||||||
|
return fmt.Errorf("session init: write temp file: %w", err)
|
||||||
|
}
|
||||||
|
if err := tmp.Close(); err != nil {
|
||||||
|
return fmt.Errorf("session init: close temp file: %w", err)
|
||||||
|
}
|
||||||
|
if _, err := store.Upload(ctx, tmpPath, key, storage.UploadOptions{ContentType: "application/x-yaml; charset=utf-8"}); err != nil {
|
||||||
|
return fmt.Errorf("session init: upload remote session %q: %w", key, err)
|
||||||
|
}
|
||||||
|
_, err = fmt.Fprintf(out, "narratio session init: wrote s3://%s/%s\n", s3BucketName(base.Pipeline), key)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildSessionYAML(campaign, sessionID, previousSessionID, date, title, audioS3Prefix, audioDir string) ([]byte, error) {
|
||||||
|
if strings.TrimSpace(date) == "" && regexp.MustCompile(`^\d{4}-\d{2}-\d{2}$`).MatchString(strings.TrimSpace(sessionID)) {
|
||||||
|
date = strings.TrimSpace(sessionID)
|
||||||
|
}
|
||||||
|
type audioS3 struct {
|
||||||
|
Prefix string `yaml:"prefix"`
|
||||||
|
}
|
||||||
|
type inputs struct {
|
||||||
|
AudioDir string `yaml:"audio_dir,omitempty"`
|
||||||
|
AudioS3 *audioS3 `yaml:"audio_s3,omitempty"`
|
||||||
|
}
|
||||||
|
type sessionYAML struct {
|
||||||
|
Campaign string `yaml:"campaign"`
|
||||||
|
SessionID string `yaml:"session_id"`
|
||||||
|
PreviousSessionID string `yaml:"previous_session_id,omitempty"`
|
||||||
|
Date string `yaml:"date,omitempty"`
|
||||||
|
Title string `yaml:"title,omitempty"`
|
||||||
|
Inputs inputs `yaml:"inputs"`
|
||||||
|
}
|
||||||
|
in := inputs{AudioDir: strings.TrimSpace(audioDir)}
|
||||||
|
if in.AudioDir == "" {
|
||||||
|
prefix := strings.TrimSpace(audioS3Prefix)
|
||||||
|
if prefix == "" {
|
||||||
|
prefix = "audio/"
|
||||||
|
}
|
||||||
|
in.AudioS3 = &audioS3{Prefix: prefix}
|
||||||
|
}
|
||||||
|
data, err := yaml.Marshal(sessionYAML{
|
||||||
|
Campaign: strings.TrimSpace(campaign),
|
||||||
|
SessionID: strings.TrimSpace(sessionID),
|
||||||
|
PreviousSessionID: strings.TrimSpace(previousSessionID),
|
||||||
|
Date: strings.TrimSpace(date),
|
||||||
|
Title: strings.TrimSpace(title),
|
||||||
|
Inputs: in,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return data, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type sessionInitInput struct {
|
||||||
|
Campaign string
|
||||||
|
CampaignPath string
|
||||||
|
TemplateFile string
|
||||||
|
SessionID string
|
||||||
|
PreviousSessionID string
|
||||||
|
Date string
|
||||||
|
Title string
|
||||||
|
AudioS3Prefix string
|
||||||
|
AudioDir string
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildSessionInitYAML(in sessionInitInput) ([]byte, error) {
|
||||||
|
if strings.TrimSpace(in.TemplateFile) == "" {
|
||||||
|
return buildSessionYAML(in.Campaign, in.SessionID, in.PreviousSessionID, in.Date, in.Title, in.AudioS3Prefix, in.AudioDir)
|
||||||
|
}
|
||||||
|
templatePath := resolveSessionInitTemplatePath(in.CampaignPath, in.TemplateFile)
|
||||||
|
templateBytes, err := os.ReadFile(templatePath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read session template %q: %w", templatePath, err)
|
||||||
|
}
|
||||||
|
rendered, err := renderSessionInitTemplate(string(templateBytes), in)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("render session template %q: %w", templatePath, err)
|
||||||
|
}
|
||||||
|
return []byte(rendered), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveSessionInitTemplatePath(campaignPath, templateFile string) string {
|
||||||
|
templateFile = strings.TrimSpace(templateFile)
|
||||||
|
if filepath.IsAbs(templateFile) {
|
||||||
|
return filepath.Clean(templateFile)
|
||||||
|
}
|
||||||
|
return filepath.Clean(filepath.Join(filepath.Dir(campaignPath), templateFile))
|
||||||
|
}
|
||||||
|
|
||||||
|
var sessionInitTemplatePattern = regexp.MustCompile(`\{\{\s*([a-zA-Z_][a-zA-Z0-9_]*)\s*\}\}`)
|
||||||
|
|
||||||
|
func renderSessionInitTemplate(content string, in sessionInitInput) (string, error) {
|
||||||
|
values := map[string]string{
|
||||||
|
"session_id": strings.TrimSpace(in.SessionID),
|
||||||
|
"previous_session_id": strings.TrimSpace(in.PreviousSessionID),
|
||||||
|
"date": strings.TrimSpace(in.Date),
|
||||||
|
"title": strings.TrimSpace(in.Title),
|
||||||
|
"audio_s3_prefix": strings.TrimSpace(in.AudioS3Prefix),
|
||||||
|
"audio_dir": strings.TrimSpace(in.AudioDir),
|
||||||
|
}
|
||||||
|
used := map[string]struct{}{}
|
||||||
|
unknown := map[string]struct{}{}
|
||||||
|
missing := map[string]struct{}{}
|
||||||
|
rendered := sessionInitTemplatePattern.ReplaceAllStringFunc(content, func(match string) string {
|
||||||
|
parts := sessionInitTemplatePattern.FindStringSubmatch(match)
|
||||||
|
if len(parts) < 2 {
|
||||||
|
return match
|
||||||
|
}
|
||||||
|
name := parts[1]
|
||||||
|
value, ok := values[name]
|
||||||
|
if !ok {
|
||||||
|
unknown[name] = struct{}{}
|
||||||
|
return match
|
||||||
|
}
|
||||||
|
used[name] = struct{}{}
|
||||||
|
if value == "" {
|
||||||
|
missing[name] = struct{}{}
|
||||||
|
return match
|
||||||
|
}
|
||||||
|
return value
|
||||||
|
})
|
||||||
|
if len(unknown) > 0 {
|
||||||
|
return "", fmt.Errorf("unsupported template variable(s): %s", sortedStringSet(unknown))
|
||||||
|
}
|
||||||
|
if len(missing) > 0 {
|
||||||
|
return "", fmt.Errorf("missing required template variable value(s): %s", sortedStringSet(missing))
|
||||||
|
}
|
||||||
|
unused := map[string]struct{}{}
|
||||||
|
for _, name := range []string{"previous_session_id", "date", "title", "audio_s3_prefix", "audio_dir"} {
|
||||||
|
if values[name] == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, ok := used[name]; !ok {
|
||||||
|
unused[name] = struct{}{}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(unused) > 0 {
|
||||||
|
return "", fmt.Errorf("unused template variable value(s): %s", sortedStringSet(unused))
|
||||||
|
}
|
||||||
|
return rendered, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func sortedStringSet(set map[string]struct{}) string {
|
||||||
|
items := make([]string, 0, len(set))
|
||||||
|
for item := range set {
|
||||||
|
items = append(items, item)
|
||||||
|
}
|
||||||
|
sort.Strings(items)
|
||||||
|
return strings.Join(items, ", ")
|
||||||
|
}
|
||||||
84
internal/app/operator_session_validate.go
Normal file
84
internal/app/operator_session_validate.go
Normal file
@@ -0,0 +1,84 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SessionValidate performs a read-only session preflight.
|
||||||
|
func SessionValidate(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
fs := flag.NewFlagSet("session validate", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(io.Discard)
|
||||||
|
var flags commonConfigFlags
|
||||||
|
addCommonConfigFlags(fs, &flags)
|
||||||
|
if err := parseSessionAwareFlags("session validate", fs, args, &flags.sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
|
return fmt.Errorf("session validate: session_id is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
findings := []finding{}
|
||||||
|
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
|
||||||
|
if err != nil {
|
||||||
|
findings = append(findings, errorFinding("config", err.Error()))
|
||||||
|
return renderFindings(out, "", "", findings)
|
||||||
|
}
|
||||||
|
if err := config.Validate(cfg); err != nil {
|
||||||
|
findings = append(findings, errorFinding("config", err.Error()))
|
||||||
|
} else {
|
||||||
|
findings = append(findings, okFinding("config", "resolved pipeline, campaign, and session config"))
|
||||||
|
}
|
||||||
|
findings = append(findings, okFinding("session", fmt.Sprintf("session source: %s", sessionSourceSummary(cfg))))
|
||||||
|
|
||||||
|
paths := artifacts.NewLocalStore(cfg.Pipeline.Workspace.Root).SessionPathsFor(cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
findings = append(findings, validateStableInputFindings(cfg)...)
|
||||||
|
findings = append(findings, validateLocalAudioFindings(cfg)...)
|
||||||
|
|
||||||
|
store, storeErr := objectStoreIfConfigured(ctx, cfg)
|
||||||
|
if storeErr != nil {
|
||||||
|
findings = append(findings, errorFinding("storage", storeErr.Error()))
|
||||||
|
}
|
||||||
|
if cfg.Session.Inputs.AudioS3 != nil {
|
||||||
|
if storeErr != nil {
|
||||||
|
findings = append(findings, errorFinding("audio", "remote audio cannot be checked because storage is unavailable"))
|
||||||
|
} else {
|
||||||
|
findings = append(findings, validateRemoteAudioFinding(ctx, cfg, store))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
requirements := artifacts.CollectPreviousArtifactRequirements(configuredScriptoriumArtifacts(cfg))
|
||||||
|
previous := inspectPreviousArtifactReadiness(ctx, cfg, store, requirements)
|
||||||
|
if len(previous.Requirements) == 0 {
|
||||||
|
findings = append(findings, okFinding("previous", "no previous-session artifacts required"))
|
||||||
|
} else if previous.MissingID {
|
||||||
|
findings = append(findings, errorFinding("previous", "previous_session_id is required by configured previous-session artifacts"))
|
||||||
|
} else if previous.Err != nil {
|
||||||
|
findings = append(findings, errorFinding("previous", previous.Err.Error()))
|
||||||
|
} else {
|
||||||
|
for _, req := range previous.Requirements {
|
||||||
|
findings = append(findings, okFinding("previous", fmt.Sprintf("%s required=%t", req.Name, req.Required)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
locks := inspectEffectiveLocks(ctx, cfg, store)
|
||||||
|
if locks.Err != nil {
|
||||||
|
findings = append(findings, errorFinding("locks", locks.Err.Error()))
|
||||||
|
} else if len(locks.Locks.All) == 0 {
|
||||||
|
findings = append(findings, okFinding("locks", "no effective publish locks"))
|
||||||
|
} else {
|
||||||
|
for _, lock := range locks.Locks.All {
|
||||||
|
findings = append(findings, warnFinding("locks", fmt.Sprintf("%s locked: %s", lock.Source, strings.TrimSpace(lock.Reason))))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if paths.ManifestPath != "" {
|
||||||
|
findings = append(findings, infoFinding("workspace", "manifest path: "+paths.ManifestPath))
|
||||||
|
}
|
||||||
|
return renderFindings(out, cfg.Session.Campaign, cfg.Session.SessionID, findings)
|
||||||
|
}
|
||||||
166
internal/app/operator_status.go
Normal file
166
internal/app/operator_status.go
Normal file
@@ -0,0 +1,166 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Status reports effective local/remote session state.
|
||||||
|
func Status(ctx context.Context, args []string, out io.Writer) error {
|
||||||
|
fs := flag.NewFlagSet("status", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(io.Discard)
|
||||||
|
var flags commonConfigFlags
|
||||||
|
addCommonConfigFlags(fs, &flags)
|
||||||
|
if err := parseSessionAwareFlags("status", fs, args, &flags.sessionID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
|
return fmt.Errorf("status: session_id is required")
|
||||||
|
}
|
||||||
|
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("status: %w", err)
|
||||||
|
}
|
||||||
|
if err := config.Validate(cfg); err != nil {
|
||||||
|
return fmt.Errorf("status: %w", err)
|
||||||
|
}
|
||||||
|
paths := artifacts.NewLocalStore(cfg.Pipeline.Workspace.Root).SessionPathsFor(cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
|
fmt.Fprintf(out, "Session: %s\n", cfg.Session.SessionID)
|
||||||
|
fmt.Fprintf(out, "Campaign: %s\n", cfg.Session.Campaign)
|
||||||
|
fmt.Fprintf(out, "Workspace: %s\n", paths.Root)
|
||||||
|
fmt.Fprintf(out, "Session config: %s\n", sessionSourceSummary(cfg))
|
||||||
|
writeStatusStableInputs(out, inspectStableInputs(cfg))
|
||||||
|
writeStatusLocalAudio(out, inspectLocalAudioPresence(cfg))
|
||||||
|
|
||||||
|
if m, err := loadLocalManifest(ctx, paths.ManifestPath); err != nil {
|
||||||
|
fmt.Fprintf(out, "Local manifest: error: %v\n", err)
|
||||||
|
} else if m == nil {
|
||||||
|
fmt.Fprintln(out, "Local manifest: missing")
|
||||||
|
} else {
|
||||||
|
fmt.Fprintf(out, "Local manifest: %s\n", paths.ManifestPath)
|
||||||
|
writeStageStatuses(out, m)
|
||||||
|
}
|
||||||
|
|
||||||
|
store, storeErr := objectStoreIfConfigured(ctx, cfg)
|
||||||
|
if storeErr != nil {
|
||||||
|
fmt.Fprintf(out, "Remote publish: unavailable: %v\n", storeErr)
|
||||||
|
} else if store != nil {
|
||||||
|
current := inspectRemoteCurrentState(ctx, cfg, store)
|
||||||
|
if current.Err != nil {
|
||||||
|
fmt.Fprintf(out, "Remote publish: missing or unavailable: %v\n", current.Err)
|
||||||
|
} else {
|
||||||
|
fmt.Fprintf(out, "Remote publish: current run %s\n", current.State.RunID)
|
||||||
|
fmt.Fprintf(out, "Remote manifest: %s\n", current.State.CurrentManifestKey)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
writeStatusRemoteAudio(ctx, out, cfg, store, storeErr)
|
||||||
|
writeStatusPreviousArtifacts(out, inspectPreviousArtifactReadiness(
|
||||||
|
ctx,
|
||||||
|
cfg,
|
||||||
|
store,
|
||||||
|
artifacts.CollectPreviousArtifactRequirements(configuredScriptoriumArtifacts(cfg)),
|
||||||
|
))
|
||||||
|
|
||||||
|
lockChecks := inspectEffectiveLocks(ctx, cfg, store)
|
||||||
|
locks := lockChecks.Locks
|
||||||
|
lockErr := lockChecks.Err
|
||||||
|
if catalog, catalogErr := buildHelperArtifactCatalog(cfg); catalogErr != nil {
|
||||||
|
fmt.Fprintf(out, "Remote outputs: error: %v\n", catalogErr)
|
||||||
|
} else if storeErr == nil {
|
||||||
|
catalogLocks := locks
|
||||||
|
if lockErr != nil {
|
||||||
|
catalogLocks = &effectiveLocks{
|
||||||
|
Static: staticPublishLocks(cfg),
|
||||||
|
All: staticPublishLocks(cfg),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
publishedRemoteState := map[string]string{}
|
||||||
|
if store != nil {
|
||||||
|
publishedRemoteState = remotePublishedOutputAvailability(ctx, cfg, store, catalog)
|
||||||
|
}
|
||||||
|
fmt.Fprintln(out, "Remote outputs:")
|
||||||
|
writeArtifactList(out, cfg, catalog, catalogLocks, publishedRemoteState)
|
||||||
|
}
|
||||||
|
if lockErr != nil {
|
||||||
|
fmt.Fprintf(out, "Publish locks: error: %v\n", lockErr)
|
||||||
|
} else {
|
||||||
|
writeLocks(out, cfg, locks)
|
||||||
|
}
|
||||||
|
fmt.Fprintln(out, "Next actions:")
|
||||||
|
fmt.Fprintf(out, "- narratio session validate %s\n", cfg.Session.SessionID)
|
||||||
|
fmt.Fprintf(out, "- narratio session restore %s --dry-run\n", cfg.Session.SessionID)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeStatusStableInputs(out io.Writer, checks []stableInputCheck) {
|
||||||
|
if len(checks) == 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, check := range checks {
|
||||||
|
if check.Err != nil {
|
||||||
|
if strings.TrimSpace(check.Path) != "" {
|
||||||
|
fmt.Fprintf(out, "Stable input %s: unavailable: %v\n", check.Name, check.Err)
|
||||||
|
} else {
|
||||||
|
fmt.Fprintf(out, "Stable input %s: unavailable: %s\n", check.Name, check.Err.Error())
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "Stable input %s: %s\n", check.Name, check.Path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeStatusLocalAudio(out io.Writer, check localAudioCheck) {
|
||||||
|
if !check.Checked {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if check.Err != nil {
|
||||||
|
fmt.Fprintf(out, "Local audio: unavailable: %v\n", check.Err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "Local audio: %d file(s)\n", len(check.Paths))
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeStatusRemoteAudio(ctx context.Context, out io.Writer, cfg *config.Config, store storage.ObjectStore, storeErr error) {
|
||||||
|
if cfg.Session.Inputs.AudioS3 == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if storeErr != nil {
|
||||||
|
fmt.Fprintf(out, "Remote audio: unavailable: %v\n", storeErr)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
check := inspectRemoteAudioPresence(ctx, cfg, store)
|
||||||
|
if check.Err != nil {
|
||||||
|
fmt.Fprintf(out, "Remote audio: unavailable: %v\n", check.Err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "Remote audio: %d .flac object(s)\n", len(check.Keys))
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeStatusPreviousArtifacts(out io.Writer, readiness previousArtifactReadiness) {
|
||||||
|
if len(readiness.Requirements) == 0 {
|
||||||
|
fmt.Fprintln(out, "Previous-session artifacts: not required")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if readiness.MissingID {
|
||||||
|
fmt.Fprintln(out, "Previous-session artifacts: unavailable: previous_session_id is required by configured previous-session artifacts")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if readiness.Err != nil {
|
||||||
|
fmt.Fprintf(out, "Previous-session artifacts: unavailable: %v\n", readiness.Err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
names := make([]string, 0, len(readiness.Requirements))
|
||||||
|
for _, req := range readiness.Requirements {
|
||||||
|
names = append(names, fmt.Sprintf("%s(required=%t)", req.Name, req.Required))
|
||||||
|
}
|
||||||
|
sort.Strings(names)
|
||||||
|
fmt.Fprintf(out, "Previous-session artifacts: ready: %s\n", strings.Join(names, ", "))
|
||||||
|
}
|
||||||
@@ -7,7 +7,6 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"os"
|
"os"
|
||||||
"strings"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
@@ -17,46 +16,21 @@ import (
|
|||||||
|
|
||||||
// Plan validates configuration, prepares the local workdir, and prints stage order.
|
// Plan validates configuration, prepares the local workdir, and prints stage order.
|
||||||
func Plan(ctx context.Context, args []string, out io.Writer) error {
|
func Plan(ctx context.Context, args []string, out io.Writer) error {
|
||||||
positionalSessionID, args := pullLeadingSessionID(args)
|
|
||||||
fs := flag.NewFlagSet("plan", flag.ContinueOnError)
|
fs := flag.NewFlagSet("plan", flag.ContinueOnError)
|
||||||
fs.SetOutput(io.Discard)
|
fs.SetOutput(io.Discard)
|
||||||
|
|
||||||
var pipelinePath string
|
var flags commonConfigFlags
|
||||||
var campaignPath string
|
|
||||||
var campaignFilePath string
|
|
||||||
var sessionPath string
|
|
||||||
var sessionID string
|
|
||||||
var previousSessionID string
|
|
||||||
var force bool
|
var force bool
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
addCommonConfigFlags(fs, &flags)
|
||||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
|
||||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
|
||||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
|
||||||
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
if err := parseSessionAwareFlags("plan", fs, args, &flags.sessionID); err != nil {
|
||||||
return fmt.Errorf("plan: invalid flags: %w", err)
|
return err
|
||||||
}
|
}
|
||||||
if positionalSessionID == "" {
|
if flags.sessionID == "" {
|
||||||
if err := applyParsedSessionIDArg("plan", fs, &sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
if fs.NArg() != 0 {
|
|
||||||
return fmt.Errorf("plan: unexpected positional arguments")
|
|
||||||
}
|
|
||||||
if err := applyPositionalSessionID("plan", positionalSessionID, &sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if strings.TrimSpace(sessionID) == "" {
|
|
||||||
return fmt.Errorf("plan: session_id is required")
|
return fmt.Errorf("plan: session_id is required")
|
||||||
}
|
}
|
||||||
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
|
||||||
SessionID: sessionID,
|
|
||||||
PreviousSessionID: previousSessionID,
|
|
||||||
})
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("plan: %w", err)
|
return fmt.Errorf("plan: %w", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -27,12 +27,12 @@ func TestPlanCreatesAndReusesWorkdir(t *testing.T) {
|
|||||||
if !strings.Contains(got, "narratio session plan: workdir prepared at") {
|
if !strings.Contains(got, "narratio session plan: workdir prepared at") {
|
||||||
t.Fatalf("first output = %q, want workdir prepared", got)
|
t.Fatalf("first output = %q, want workdir prepared", got)
|
||||||
}
|
}
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "analyze", "publish", "notify"} {
|
||||||
if !strings.Contains(got, name+": run") {
|
if !strings.Contains(got, name+": run") {
|
||||||
t.Fatalf("first output = %q, missing stage %q", got, name)
|
t.Fatalf("first output = %q, missing stage %q", got, name)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if !strings.Contains(got, "totals: run=9 skip=0") {
|
if !strings.Contains(got, "totals: run=10 skip=0") {
|
||||||
t.Fatalf("first output = %q, want totals", got)
|
t.Fatalf("first output = %q, want totals", got)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -84,8 +84,8 @@ func TestPlanShowsRunAndSkipFromManifest(t *testing.T) {
|
|||||||
if !strings.Contains(got, "trim: run") {
|
if !strings.Contains(got, "trim: run") {
|
||||||
t.Fatalf("output = %q, want trim run", got)
|
t.Fatalf("output = %q, want trim run", got)
|
||||||
}
|
}
|
||||||
if !strings.Contains(got, "totals: run=7 skip=2") {
|
if !strings.Contains(got, "totals: run=8 skip=2") {
|
||||||
t.Fatalf("output = %q, want totals run=7 skip=2", got)
|
t.Fatalf("output = %q, want totals run=8 skip=2", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ import "testing"
|
|||||||
|
|
||||||
func TestBuildFullPlanOrder(t *testing.T) {
|
func TestBuildFullPlanOrder(t *testing.T) {
|
||||||
got := BuildFullPlan()
|
got := BuildFullPlan()
|
||||||
want := []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"}
|
want := []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "analyze", "publish", "notify"}
|
||||||
if len(got) != len(want) {
|
if len(got) != len(want) {
|
||||||
t.Fatalf("len(plan) = %d, want %d", len(got), len(want))
|
t.Fatalf("len(plan) = %d, want %d", len(got), len(want))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
)
|
)
|
||||||
|
|
||||||
func runPostArchiveCleanup(ctx context.Context, env *Env, manifestPath string, m *manifest.Manifest, executed []string) error {
|
func runPostPublishCleanup(ctx context.Context, env *Env, manifestPath string, m *manifest.Manifest, executed []string) error {
|
||||||
if env == nil || env.Config == nil || env.Config.Pipeline == nil || m == nil {
|
if env == nil || env.Config == nil || env.Config.Pipeline == nil || m == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -23,7 +23,7 @@ func runPostArchiveCleanup(ctx context.Context, env *Env, manifestPath string, m
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
sr := archiveStageRecordForCleanup(m, executed)
|
sr := publishStageRecordForCleanup(m, executed)
|
||||||
if sr == nil {
|
if sr == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -33,7 +33,7 @@ func runPostArchiveCleanup(ctx context.Context, env *Env, manifestPath string, m
|
|||||||
sr.Metadata["spool_cleanup_requested"] = spoolRequested
|
sr.Metadata["spool_cleanup_requested"] = spoolRequested
|
||||||
sr.Metadata["workdir_cleanup_requested"] = workRequested
|
sr.Metadata["workdir_cleanup_requested"] = workRequested
|
||||||
|
|
||||||
eligible, reason := archiveCleanupEligible(env.Config, sr)
|
eligible, reason := publishCleanupEligible(env.Config, sr)
|
||||||
if !eligible {
|
if !eligible {
|
||||||
sr.Metadata["cleanup_skipped"] = true
|
sr.Metadata["cleanup_skipped"] = true
|
||||||
sr.Metadata["cleanup_skipped_reason"] = reason
|
sr.Metadata["cleanup_skipped_reason"] = reason
|
||||||
@@ -96,7 +96,7 @@ func runPostArchiveCleanup(ctx context.Context, env *Env, manifestPath string, m
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func archiveStageRecordForCleanup(m *manifest.Manifest, executed []string) *manifest.StageRecord {
|
func publishStageRecordForCleanup(m *manifest.Manifest, executed []string) *manifest.StageRecord {
|
||||||
if m == nil {
|
if m == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -117,7 +117,7 @@ func archiveStageRecordForCleanup(m *manifest.Manifest, executed []string) *mani
|
|||||||
return sr
|
return sr
|
||||||
}
|
}
|
||||||
|
|
||||||
func archiveCleanupEligible(cfg *config.Config, sr *manifest.StageRecord) (bool, string) {
|
func publishCleanupEligible(cfg *config.Config, sr *manifest.StageRecord) (bool, string) {
|
||||||
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Publish == nil {
|
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Publish == nil {
|
||||||
return false, "publish configuration is missing"
|
return false, "publish configuration is missing"
|
||||||
}
|
}
|
||||||
@@ -174,49 +174,7 @@ func removeRunScopedDir(root, target, policy string) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func validateScopedDir(root, target, policy string) (scopedDir, error) {
|
func validateScopedDir(root, target, policy string) (scopedDir, error) {
|
||||||
cleanRoot := strings.TrimSpace(root)
|
return validateScopedTarget(root, target, policy, true)
|
||||||
cleanTarget := strings.TrimSpace(target)
|
|
||||||
if cleanRoot == "" {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: root path is required", policy)
|
|
||||||
}
|
|
||||||
if cleanTarget == "" {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: target path is required", policy)
|
|
||||||
}
|
|
||||||
|
|
||||||
rootAbs, err := filepath.Abs(cleanRoot)
|
|
||||||
if err != nil {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
|
|
||||||
}
|
|
||||||
targetAbs, err := filepath.Abs(cleanTarget)
|
|
||||||
if err != nil {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
rel, err := filepath.Rel(rootAbs, targetAbs)
|
|
||||||
if err != nil {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
|
|
||||||
}
|
|
||||||
if rel == "." {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
|
|
||||||
}
|
|
||||||
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
|
|
||||||
}
|
|
||||||
|
|
||||||
info, err := os.Lstat(targetAbs)
|
|
||||||
if err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: false}, nil
|
|
||||||
}
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
|
|
||||||
}
|
|
||||||
if info.Mode()&os.ModeSymlink != 0 {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
|
|
||||||
}
|
|
||||||
if !info.IsDir() {
|
|
||||||
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is not a directory", policy, targetAbs)
|
|
||||||
}
|
|
||||||
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: true}, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func asString(v any) string {
|
func asString(v any) string {
|
||||||
@@ -16,13 +16,13 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/narratio/internal/stage"
|
"gitea.maximumdirect.net/eric/narratio/internal/stage"
|
||||||
)
|
)
|
||||||
|
|
||||||
type archiveSuccessStage struct {
|
type publishSuccessStage struct {
|
||||||
metadata map[string]any
|
metadata map[string]any
|
||||||
}
|
}
|
||||||
|
|
||||||
func (archiveSuccessStage) Name() string { return "publish" }
|
func (publishSuccessStage) Name() string { return "publish" }
|
||||||
func (archiveSuccessStage) Declares() stage.IODecl { return stage.IODecl{} }
|
func (publishSuccessStage) Declares() stage.IODecl { return stage.IODecl{} }
|
||||||
func (s archiveSuccessStage) Run(_ context.Context, _ *stage.Env, _ *manifest.Manifest) (*stage.StageResult, error) {
|
func (s publishSuccessStage) Run(_ context.Context, _ *stage.Env, _ *manifest.Manifest) (*stage.StageResult, error) {
|
||||||
md := map[string]any{
|
md := map[string]any{
|
||||||
"stage": "publish",
|
"stage": "publish",
|
||||||
"uploaded": true,
|
"uploaded": true,
|
||||||
@@ -43,12 +43,12 @@ func (notifyFailStage) Run(_ context.Context, _ *stage.Env, _ *manifest.Manifest
|
|||||||
return nil, errors.New("notify failed")
|
return nil, errors.New("notify failed")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupDisabledKeepsLocalDirs(t *testing.T) {
|
func TestPostPublishCleanupDisabledKeepsLocalDirs(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = false
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = false
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = false
|
cfg.Pipeline.Workspace.CleanupAfterPublish = false
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -57,12 +57,12 @@ func TestPostArchiveCleanupDisabledKeepsLocalDirs(t *testing.T) {
|
|||||||
assertExists(t, seed.localSourceAudio)
|
assertExists(t, seed.localSourceAudio)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupSpoolOnly(t *testing.T) {
|
func TestPostPublishCleanupSpoolOnly(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = false
|
cfg.Pipeline.Workspace.CleanupAfterPublish = false
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -71,12 +71,12 @@ func TestPostArchiveCleanupSpoolOnly(t *testing.T) {
|
|||||||
assertExists(t, seed.localSourceAudio)
|
assertExists(t, seed.localSourceAudio)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupWorkdirOnly(t *testing.T) {
|
func TestPostPublishCleanupWorkdirOnly(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = false
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = false
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -87,12 +87,12 @@ func TestPostArchiveCleanupWorkdirOnly(t *testing.T) {
|
|||||||
assertExists(t, seed.spoolAudioDir)
|
assertExists(t, seed.spoolAudioDir)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupBothPolicies(t *testing.T) {
|
func TestPostPublishCleanupBothPolicies(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -102,12 +102,12 @@ func TestPostArchiveCleanupBothPolicies(t *testing.T) {
|
|||||||
assertExists(t, seed.previousCachePath)
|
assertExists(t, seed.previousCachePath)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenArchiveFails(t *testing.T) {
|
func TestPostPublishCleanupNotRunWhenPublishFails(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
_, err := executeStages(context.Background(), cfg, []stage.Stage{failingStage{name: "publish", err: errors.New("archive failed")}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
_, err := executeStages(context.Background(), cfg, []stage.Stage{failingStage{name: "publish", err: errors.New("publish failed")}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
||||||
if err == nil || !strings.Contains(err.Error(), "stage \"publish\" failed") {
|
if err == nil || !strings.Contains(err.Error(), "stage \"publish\" failed") {
|
||||||
t.Fatalf("executeStages() error = %v, want publish failure", err)
|
t.Fatalf("executeStages() error = %v, want publish failure", err)
|
||||||
}
|
}
|
||||||
@@ -116,12 +116,12 @@ func TestPostArchiveCleanupNotRunWhenArchiveFails(t *testing.T) {
|
|||||||
assertExists(t, seed.runWorkDir)
|
assertExists(t, seed.runWorkDir)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenArchiveSkipped(t *testing.T) {
|
func TestPostPublishCleanupNotRunWhenPublishSkipped(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{metadata: map[string]any{"skipped": true}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{metadata: map[string]any{"skipped": true}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -129,12 +129,12 @@ func TestPostArchiveCleanupNotRunWhenArchiveSkipped(t *testing.T) {
|
|||||||
assertExists(t, seed.runWorkDir)
|
assertExists(t, seed.runWorkDir)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenCurrentPointerMissing(t *testing.T) {
|
func TestPostPublishCleanupNotRunWhenCurrentPointerMissing(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{metadata: map[string]any{"current_pointer_written": false}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{metadata: map[string]any{"current_pointer_written": false}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -142,13 +142,13 @@ func TestPostArchiveCleanupNotRunWhenCurrentPointerMissing(t *testing.T) {
|
|||||||
assertExists(t, seed.runWorkDir)
|
assertExists(t, seed.runWorkDir)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenArchiveUploadDisabled(t *testing.T) {
|
func TestPostPublishCleanupNotRunWhenPublishUploadDisabled(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
cfg.Pipeline.Publish.UploadRun = boolPtr(false)
|
cfg.Pipeline.Publish.UploadRun = boolPtr(false)
|
||||||
|
|
||||||
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -156,12 +156,12 @@ func TestPostArchiveCleanupNotRunWhenArchiveUploadDisabled(t *testing.T) {
|
|||||||
assertExists(t, seed.runWorkDir)
|
assertExists(t, seed.runWorkDir)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupWaitsUntilAllStagesSucceed(t *testing.T) {
|
func TestPostPublishCleanupWaitsUntilAllStagesSucceed(t *testing.T) {
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
|
|
||||||
_, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}, notifyFailStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
_, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}, notifyFailStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
||||||
if err == nil || !strings.Contains(err.Error(), "stage \"notify\" failed") {
|
if err == nil || !strings.Contains(err.Error(), "stage \"notify\" failed") {
|
||||||
t.Fatalf("executeStages() error = %v, want notify failure", err)
|
t.Fatalf("executeStages() error = %v, want notify failure", err)
|
||||||
}
|
}
|
||||||
@@ -170,7 +170,7 @@ func TestPostArchiveCleanupWaitsUntilAllStagesSucceed(t *testing.T) {
|
|||||||
assertExists(t, seed.runWorkDir)
|
assertExists(t, seed.runWorkDir)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupFailsOnUnsafePath(t *testing.T) {
|
func TestPostPublishCleanupFailsOnUnsafePath(t *testing.T) {
|
||||||
cfg, _ := cleanupFixtureConfig(t)
|
cfg, _ := cleanupFixtureConfig(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = false
|
cfg.Pipeline.Workspace.CleanupAfterPublish = false
|
||||||
@@ -186,25 +186,25 @@ func TestPostArchiveCleanupFailsOnUnsafePath(t *testing.T) {
|
|||||||
t.Fatalf("Save() error = %v", err)
|
t.Fatalf("Save() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
_, err = executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
||||||
if err == nil || !strings.Contains(err.Error(), "refusing to delete path outside root") {
|
if err == nil || !strings.Contains(err.Error(), "refusing to delete path outside root") {
|
||||||
t.Fatalf("executeStages() error = %v, want safe-path failure", err)
|
t.Fatalf("executeStages() error = %v, want safe-path failure", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenPromotionIsMissing(t *testing.T) {
|
func TestPostPublishCleanupNotRunWhenOutputIsMissing(t *testing.T) {
|
||||||
cfg, seed, runID := archiveStageCleanupFixture(t)
|
cfg, seed, runID := publishStageCleanupFixture(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
cfg.Pipeline.Publish.Outputs = []config.PublishOutputRule{
|
cfg.Pipeline.Publish.Outputs = []config.PublishOutputRule{
|
||||||
{Source: "narratio.transcript.base", Dest: "transcripts/base.json", Required: boolPtr(true)},
|
{Source: "narratio.transcript.base", Dest: "transcripts/base.json", Required: boolPtr(true)},
|
||||||
}
|
}
|
||||||
|
|
||||||
archiveStageImpl, err := stage.Select("publish")
|
publishStageImpl, err := stage.Select("publish")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Select(publish) error = %v", err)
|
t.Fatalf("Select(publish) error = %v", err)
|
||||||
}
|
}
|
||||||
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
_, err = executeStages(context.Background(), cfg, []stage.Stage{publishStageImpl}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
|
||||||
if err == nil || !strings.Contains(err.Error(), "required output source unavailable") {
|
if err == nil || !strings.Contains(err.Error(), "required output source unavailable") {
|
||||||
t.Fatalf("executeStages() error = %v, want required output source unavailable failure", err)
|
t.Fatalf("executeStages() error = %v, want required output source unavailable failure", err)
|
||||||
}
|
}
|
||||||
@@ -215,17 +215,17 @@ func TestPostArchiveCleanupNotRunWhenPromotionIsMissing(t *testing.T) {
|
|||||||
assertExists(t, artifacts.SessionRunRootForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID, runID))
|
assertExists(t, artifacts.SessionRunRootForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID, runID))
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenCurrentManifestUploadFails(t *testing.T) {
|
func TestPostPublishCleanupNotRunWhenCurrentManifestUploadFails(t *testing.T) {
|
||||||
cfg, seed, _ := archiveStageCleanupFixture(t)
|
cfg, seed, _ := publishStageCleanupFixture(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
failKey := seed.sessionPrefix + "current/manifest.json"
|
failKey := seed.sessionPrefix + "current/manifest.json"
|
||||||
|
|
||||||
archiveStageImpl, err := stage.Select("publish")
|
publishStageImpl, err := stage.Select("publish")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Select(publish) error = %v", err)
|
t.Fatalf("Select(publish) error = %v", err)
|
||||||
}
|
}
|
||||||
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{
|
_, err = executeStages(context.Background(), cfg, []stage.Stage{publishStageImpl}, RunOptions{
|
||||||
Env: &Env{ObjectStore: &failKeyStore{delegate: &storage.FakeBackend{}, failKey: failKey}},
|
Env: &Env{ObjectStore: &failKeyStore{delegate: &storage.FakeBackend{}, failKey: failKey}},
|
||||||
})
|
})
|
||||||
if err == nil || !strings.Contains(err.Error(), "current manifest") {
|
if err == nil || !strings.Contains(err.Error(), "current manifest") {
|
||||||
@@ -236,17 +236,17 @@ func TestPostArchiveCleanupNotRunWhenCurrentManifestUploadFails(t *testing.T) {
|
|||||||
assertExists(t, seed.runWorkDir)
|
assertExists(t, seed.runWorkDir)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPostArchiveCleanupNotRunWhenCurrentPointerUploadFails(t *testing.T) {
|
func TestPostPublishCleanupNotRunWhenCurrentPointerUploadFails(t *testing.T) {
|
||||||
cfg, seed, _ := archiveStageCleanupFixture(t)
|
cfg, seed, _ := publishStageCleanupFixture(t)
|
||||||
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
|
||||||
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
cfg.Pipeline.Workspace.CleanupAfterPublish = true
|
||||||
failKey := seed.sessionPrefix + "current/run_id.txt"
|
failKey := seed.sessionPrefix + "current/run_id.txt"
|
||||||
|
|
||||||
archiveStageImpl, err := stage.Select("publish")
|
publishStageImpl, err := stage.Select("publish")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Select(publish) error = %v", err)
|
t.Fatalf("Select(publish) error = %v", err)
|
||||||
}
|
}
|
||||||
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{
|
_, err = executeStages(context.Background(), cfg, []stage.Stage{publishStageImpl}, RunOptions{
|
||||||
Env: &Env{ObjectStore: &failKeyStore{delegate: &storage.FakeBackend{}, failKey: failKey}},
|
Env: &Env{ObjectStore: &failKeyStore{delegate: &storage.FakeBackend{}, failKey: failKey}},
|
||||||
})
|
})
|
||||||
if err == nil || !strings.Contains(err.Error(), "current run pointer") {
|
if err == nil || !strings.Contains(err.Error(), "current run pointer") {
|
||||||
@@ -320,7 +320,7 @@ func cleanupFixtureConfig(t *testing.T) (*config.Config, cleanupSeed) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func archiveStageCleanupFixture(t *testing.T) (*config.Config, cleanupSeed, string) {
|
func publishStageCleanupFixture(t *testing.T) (*config.Config, cleanupSeed, string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
cfg, seed := cleanupFixtureConfig(t)
|
cfg, seed := cleanupFixtureConfig(t)
|
||||||
@@ -344,7 +344,7 @@ func archiveStageCleanupFixture(t *testing.T) (*config.Config, cleanupSeed, stri
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
writeArchiveFixtureRunFiles(
|
writePublishFixtureRunFiles(
|
||||||
t,
|
t,
|
||||||
seed.runWorkDir,
|
seed.runWorkDir,
|
||||||
artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID),
|
artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID),
|
||||||
@@ -355,7 +355,7 @@ func archiveStageCleanupFixture(t *testing.T) (*config.Config, cleanupSeed, stri
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Load() error = %v", err)
|
t.Fatalf("Load() error = %v", err)
|
||||||
}
|
}
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "analyze"} {
|
||||||
seedManifest.MarkStageSucceeded(name, time.Now().UTC(), nil)
|
seedManifest.MarkStageSucceeded(name, time.Now().UTC(), nil)
|
||||||
}
|
}
|
||||||
seedManifest.S3SessionPrefix = artifacts.S3SessionPrefix("dnd", cfg.Session.Campaign, cfg.Session.SessionID)
|
seedManifest.S3SessionPrefix = artifacts.S3SessionPrefix("dnd", cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
@@ -367,7 +367,7 @@ func archiveStageCleanupFixture(t *testing.T) (*config.Config, cleanupSeed, stri
|
|||||||
return cfg, seed, runID
|
return cfg, seed, runID
|
||||||
}
|
}
|
||||||
|
|
||||||
func writeArchiveFixtureRunFiles(t *testing.T, runWorkDir, sessionRoot string) {
|
func writePublishFixtureRunFiles(t *testing.T, runWorkDir, sessionRoot string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "prepare", "inputs", "session.yml"), "session_id: 2026-05-03\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "prepare", "inputs", "session.yml"), "session_id: 2026-05-03\n")
|
||||||
mustWriteFile(t, filepath.Join(runWorkDir, "transcribe", "outputs", "transcripts", "raw", "speaker.json"), "{}\n")
|
mustWriteFile(t, filepath.Join(runWorkDir, "transcribe", "outputs", "transcripts", "raw", "speaker.json"), "{}\n")
|
||||||
@@ -46,7 +46,7 @@ func loadRemoteLockStore(ctx context.Context, cfg *config.Config, store storage.
|
|||||||
if !exists {
|
if !exists {
|
||||||
return &config.PublishLockStore{}, key, nil
|
return &config.PublishLockStore{}, key, nil
|
||||||
}
|
}
|
||||||
tmp, err := downloadObjectToTemp(ctx, store, key, "narratio-locks-*.yml")
|
tmp, err := storage.DownloadObjectToTemp(ctx, store, key, "narratio-locks-*.yml")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, key, fmt.Errorf("download remote locks %q: %w", key, err)
|
return nil, key, fmt.Errorf("download remote locks %q: %w", key, err)
|
||||||
}
|
}
|
||||||
@@ -63,7 +63,7 @@ func loadRemoteLockStore(ctx context.Context, cfg *config.Config, store storage.
|
|||||||
}
|
}
|
||||||
|
|
||||||
func loadEffectiveLocks(ctx context.Context, cfg *config.Config, store storage.ObjectStore) (*effectiveLocks, error) {
|
func loadEffectiveLocks(ctx context.Context, cfg *config.Config, store storage.ObjectStore) (*effectiveLocks, error) {
|
||||||
staticLocks := staticArchiveLocks(cfg)
|
staticLocks := staticPublishLocks(cfg)
|
||||||
if store == nil {
|
if store == nil {
|
||||||
return &effectiveLocks{
|
return &effectiveLocks{
|
||||||
Static: staticLocks,
|
Static: staticLocks,
|
||||||
@@ -83,7 +83,7 @@ func loadEffectiveLocks(ctx context.Context, cfg *config.Config, store storage.O
|
|||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func staticArchiveLocks(cfg *config.Config) []config.PublishLockRule {
|
func staticPublishLocks(cfg *config.Config) []config.PublishLockRule {
|
||||||
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Publish == nil {
|
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Publish == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -27,23 +27,14 @@ func Restore(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
fs := flag.NewFlagSet("restore", flag.ContinueOnError)
|
fs := flag.NewFlagSet("restore", flag.ContinueOnError)
|
||||||
fs.SetOutput(out)
|
fs.SetOutput(out)
|
||||||
|
|
||||||
var pipelinePath string
|
var flags commonConfigFlags
|
||||||
var campaignPath string
|
|
||||||
var campaignFilePath string
|
|
||||||
var sessionPath string
|
|
||||||
var sessionID string
|
|
||||||
var previousSessionID string
|
|
||||||
var dryRun bool
|
var dryRun bool
|
||||||
var force bool
|
var force bool
|
||||||
var includeAudio bool
|
var includeAudio bool
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
addCommonConfigFlags(fs, &flags)
|
||||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
|
||||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
|
||||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
|
||||||
fs.BoolVar(&dryRun, "dry-run", false, "plan restore actions without writing local files")
|
fs.BoolVar(&dryRun, "dry-run", false, "plan restore actions without writing local files")
|
||||||
fs.BoolVar(&force, "force", false, "overwrite local conflicts with remote state")
|
fs.BoolVar(&force, "force", false, "overwrite local conflicts with remote state")
|
||||||
fs.BoolVar(&includeAudio, "include-audio", false, "include archived session-level audio objects")
|
fs.BoolVar(&includeAudio, "include-audio", false, "include remote session-level audio objects")
|
||||||
fs.Usage = func() {
|
fs.Usage = func() {
|
||||||
_, _ = fmt.Fprintln(out, "Usage: narratio session restore <session_id> [--config <path>] [--campaign <id>] [--campaign-file <path>] [--session <path>] [--previous-session-id <value>] [--dry-run] [--force] [--include-audio]")
|
_, _ = fmt.Fprintln(out, "Usage: narratio session restore <session_id> [--config <path>] [--campaign <id>] [--campaign-file <path>] [--session <path>] [--previous-session-id <value>] [--dry-run] [--force] [--include-audio]")
|
||||||
_, _ = fmt.Fprintln(out)
|
_, _ = fmt.Fprintln(out)
|
||||||
@@ -57,25 +48,13 @@ func Restore(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
}
|
}
|
||||||
return fmt.Errorf("restore: invalid flags: %w", err)
|
return fmt.Errorf("restore: invalid flags: %w", err)
|
||||||
}
|
}
|
||||||
if positionalSessionID == "" {
|
if err := resolveParsedSessionID("restore", positionalSessionID, fs, &flags.sessionID); err != nil {
|
||||||
if err := applyParsedSessionIDArg("restore", fs, &sessionID); err != nil {
|
return err
|
||||||
return err
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
if fs.NArg() != 0 {
|
|
||||||
return fmt.Errorf("restore: unexpected positional arguments")
|
|
||||||
}
|
|
||||||
if err := applyPositionalSessionID("restore", positionalSessionID, &sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
if strings.TrimSpace(sessionID) == "" {
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
return fmt.Errorf("restore: session_id is required")
|
return fmt.Errorf("restore: session_id is required")
|
||||||
}
|
}
|
||||||
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
|
||||||
SessionID: sessionID,
|
|
||||||
PreviousSessionID: previousSessionID,
|
|
||||||
})
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("restore: %w", err)
|
return fmt.Errorf("restore: %w", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,7 +3,6 @@ package app
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
@@ -12,7 +11,7 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// RemoteCurrentState captures discovered committed remote archive state for one session.
|
// RemoteCurrentState captures discovered committed remote published current state for one session.
|
||||||
type RemoteCurrentState struct {
|
type RemoteCurrentState struct {
|
||||||
Bucket string
|
Bucket string
|
||||||
SessionPrefix string
|
SessionPrefix string
|
||||||
@@ -32,80 +31,24 @@ func discoverRemoteCurrentState(ctx context.Context, cfg *config.Config, store s
|
|||||||
return nil, fmt.Errorf("remote object store is required")
|
return nil, fmt.Errorf("remote object store is required")
|
||||||
}
|
}
|
||||||
|
|
||||||
bucket := artifacts.ResolveArchiveBucket(cfg, nil)
|
bucket := artifacts.ResolvePublishBucket(cfg, nil)
|
||||||
if strings.TrimSpace(bucket) == "" {
|
if strings.TrimSpace(bucket) == "" {
|
||||||
return nil, fmt.Errorf("archive bucket is required")
|
return nil, fmt.Errorf("publish bucket is required")
|
||||||
}
|
}
|
||||||
sessionPrefix, err := artifacts.ResolveArchiveSessionPrefix(cfg, nil)
|
sessionPrefix, err := artifacts.ResolvePublishSessionPrefix(cfg, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("resolve archive session prefix: %w", err)
|
return nil, fmt.Errorf("resolve publish session prefix: %w", err)
|
||||||
}
|
|
||||||
currentManifestKey, currentRunIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
|
||||||
|
|
||||||
exists, err := store.Exists(ctx, currentRunIDKey)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("check remote current run pointer %q: %w", currentRunIDKey, err)
|
|
||||||
}
|
|
||||||
if !exists {
|
|
||||||
return nil, fmt.Errorf("remote current run pointer missing: %q", currentRunIDKey)
|
|
||||||
}
|
|
||||||
|
|
||||||
runIDPath, err := downloadObjectToTemp(ctx, store, currentRunIDKey, "narratio-restore-current-run-id-*.txt")
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("download remote current run pointer %q: %w", currentRunIDKey, err)
|
|
||||||
}
|
|
||||||
defer func() { _ = os.Remove(runIDPath) }()
|
|
||||||
|
|
||||||
runIDData, err := os.ReadFile(runIDPath)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("read downloaded run pointer %q: %w", currentRunIDKey, err)
|
|
||||||
}
|
|
||||||
runID := strings.TrimSpace(string(runIDData))
|
|
||||||
if runID == "" {
|
|
||||||
return nil, fmt.Errorf("remote current run pointer %q is empty", currentRunIDKey)
|
|
||||||
}
|
|
||||||
|
|
||||||
exists, err = store.Exists(ctx, currentManifestKey)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("check remote current manifest %q: %w", currentManifestKey, err)
|
|
||||||
}
|
|
||||||
if !exists {
|
|
||||||
return nil, fmt.Errorf("remote current manifest missing: %q", currentManifestKey)
|
|
||||||
}
|
|
||||||
|
|
||||||
manifestPath, err := downloadObjectToTemp(ctx, store, currentManifestKey, "narratio-restore-current-manifest-*.json")
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("download remote current manifest %q: %w", currentManifestKey, err)
|
|
||||||
}
|
|
||||||
defer func() { _ = os.Remove(manifestPath) }()
|
|
||||||
|
|
||||||
manifestStore := &manifest.LocalStore{}
|
|
||||||
remoteManifest, err := manifestStore.Load(ctx, manifestPath)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("remote current manifest decode failed: %w", err)
|
|
||||||
}
|
}
|
||||||
|
currentManifestKey, currentRunIDKey := artifacts.ResolveCurrentStateKeys(sessionPrefix)
|
||||||
|
|
||||||
requestedSession := strings.TrimSpace(cfg.Session.SessionID)
|
requestedSession := strings.TrimSpace(cfg.Session.SessionID)
|
||||||
requestedCampaign := strings.TrimSpace(cfg.Session.Campaign)
|
requestedCampaign := strings.TrimSpace(cfg.Session.Campaign)
|
||||||
manifestSession := strings.TrimSpace(remoteManifest.SessionID)
|
current, err := artifacts.LoadCurrentState(ctx, store, sessionPrefix, artifacts.CurrentStateValidation{
|
||||||
manifestCampaign := strings.TrimSpace(remoteManifest.Campaign)
|
ExpectedSessionID: requestedSession,
|
||||||
|
ExpectedCampaign: requestedCampaign,
|
||||||
if manifestSession != requestedSession {
|
})
|
||||||
return nil, fmt.Errorf(
|
if err != nil {
|
||||||
"remote current manifest session_id %q does not match requested session_id %q",
|
return nil, fmt.Errorf("remote %w", err)
|
||||||
manifestSession,
|
|
||||||
requestedSession,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
if manifestCampaign == "" {
|
|
||||||
return nil, fmt.Errorf("remote current manifest campaign is required")
|
|
||||||
}
|
|
||||||
if manifestCampaign != requestedCampaign {
|
|
||||||
return nil, fmt.Errorf(
|
|
||||||
"remote current manifest campaign %q does not match requested campaign %q",
|
|
||||||
manifestCampaign,
|
|
||||||
requestedCampaign,
|
|
||||||
)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return &RemoteCurrentState{
|
return &RemoteCurrentState{
|
||||||
@@ -113,27 +56,9 @@ func discoverRemoteCurrentState(ctx context.Context, cfg *config.Config, store s
|
|||||||
SessionPrefix: sessionPrefix,
|
SessionPrefix: sessionPrefix,
|
||||||
CurrentRunIDKey: currentRunIDKey,
|
CurrentRunIDKey: currentRunIDKey,
|
||||||
CurrentManifestKey: currentManifestKey,
|
CurrentManifestKey: currentManifestKey,
|
||||||
RunID: runID,
|
RunID: current.RunID,
|
||||||
SessionID: manifestSession,
|
SessionID: strings.TrimSpace(current.Manifest.SessionID),
|
||||||
Campaign: manifestCampaign,
|
Campaign: strings.TrimSpace(current.Manifest.Campaign),
|
||||||
Manifest: remoteManifest,
|
Manifest: current.Manifest,
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func downloadObjectToTemp(ctx context.Context, store storage.ObjectStore, key, pattern string) (string, error) {
|
|
||||||
tmp, err := os.CreateTemp("", pattern)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("create temp file: %w", err)
|
|
||||||
}
|
|
||||||
path := tmp.Name()
|
|
||||||
if err := tmp.Close(); err != nil {
|
|
||||||
_ = os.Remove(path)
|
|
||||||
return "", fmt.Errorf("close temp file: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := store.Download(ctx, key, path); err != nil {
|
|
||||||
_ = os.Remove(path)
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return path, nil
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -102,7 +102,7 @@ func TestDiscoverRemoteCurrentStateSessionMismatchFails(t *testing.T) {
|
|||||||
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, "wrong-session", cfg.Session.Campaign)})
|
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, "wrong-session", cfg.Session.Campaign)})
|
||||||
|
|
||||||
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
||||||
if err == nil || !strings.Contains(err.Error(), "does not match requested session_id") {
|
if err == nil || !strings.Contains(err.Error(), "does not match expected session_id") {
|
||||||
t.Fatalf("error = %v, want session mismatch failure", err)
|
t.Fatalf("error = %v, want session mismatch failure", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -116,7 +116,7 @@ func TestDiscoverRemoteCurrentStateCampaignMismatchFails(t *testing.T) {
|
|||||||
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, cfg.Session.SessionID, "wrong-campaign")})
|
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, cfg.Session.SessionID, "wrong-campaign")})
|
||||||
|
|
||||||
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
|
||||||
if err == nil || !strings.Contains(err.Error(), "does not match requested campaign") {
|
if err == nil || !strings.Contains(err.Error(), "does not match expected campaign") {
|
||||||
t.Fatalf("error = %v, want campaign mismatch failure", err)
|
t.Fatalf("error = %v, want campaign mismatch failure", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -208,7 +208,7 @@ func restoreDiscoveryConfig() *config.Config {
|
|||||||
|
|
||||||
func restoreDiscoveryKeys(cfg *config.Config) (sessionPrefix, manifestKey, runIDKey string) {
|
func restoreDiscoveryKeys(cfg *config.Config) (sessionPrefix, manifestKey, runIDKey string) {
|
||||||
sessionPrefix = artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
|
sessionPrefix = artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
manifestKey, runIDKey = artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
manifestKey, runIDKey = artifacts.ResolveCurrentStateKeys(sessionPrefix)
|
||||||
return sessionPrefix, manifestKey, runIDKey
|
return sessionPrefix, manifestKey, runIDKey
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/audio"
|
"gitea.maximumdirect.net/eric/narratio/internal/audio"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/fileops"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -114,10 +115,7 @@ func executeRestoreDownloadAction(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if err := os.Chmod(tmpPath, 0o644); err != nil {
|
if err := fileops.InstallDownloadedTempFile(tmpPath, safeLocalPath, 0o644); err != nil {
|
||||||
return fmt.Errorf("set file permissions: %w", err)
|
|
||||||
}
|
|
||||||
if err := os.Rename(tmpPath, safeLocalPath); err != nil {
|
|
||||||
return fmt.Errorf("install file atomically: %w", err)
|
return fmt.Errorf("install file atomically: %w", err)
|
||||||
}
|
}
|
||||||
removeTmp = false
|
removeTmp = false
|
||||||
|
|||||||
@@ -39,7 +39,7 @@ func TestExecuteRestoreNonDryRunRestoresDurableFiles(t *testing.T) {
|
|||||||
if stderr.Len() != 0 {
|
if stderr.Len() != 0 {
|
||||||
t.Fatalf("stderr = %q, want empty", stderr.String())
|
t.Fatalf("stderr = %q, want empty", stderr.String())
|
||||||
}
|
}
|
||||||
if !strings.Contains(stdout.String(), "Restored session archive for sample-campaign/2026-05-03") {
|
if !strings.Contains(stdout.String(), "Restored session state for sample-campaign/2026-05-03") {
|
||||||
t.Fatalf("stdout = %q, want completion summary", stdout.String())
|
t.Fatalf("stdout = %q, want completion summary", stdout.String())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -427,7 +427,7 @@ func seedRestoreCommittedState(t *testing.T, fake *storage.FakeBackend, pipeline
|
|||||||
}
|
}
|
||||||
|
|
||||||
sessionPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
|
sessionPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
manifestKey, runIDKey := artifacts.ResolveCurrentStateKeys(sessionPrefix)
|
||||||
|
|
||||||
seedRestoreObject(fake, runIDKey, []byte("20260519T010203Z-a1b2c3d4\n"))
|
seedRestoreObject(fake, runIDKey, []byte("20260519T010203Z-a1b2c3d4\n"))
|
||||||
seedRestoreObject(fake, manifestKey, restoreManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign))
|
seedRestoreObject(fake, manifestKey, restoreManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign))
|
||||||
@@ -465,7 +465,7 @@ func seedRestorePreviousCurrent(t *testing.T, fake *storage.FakeBackend, cfg *co
|
|||||||
func seedRestorePreviousCurrentManifestOnly(t *testing.T, fake *storage.FakeBackend, cfg *config.Config) {
|
func seedRestorePreviousCurrentManifestOnly(t *testing.T, fake *storage.FakeBackend, cfg *config.Config) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
previousPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.PreviousSessionID)
|
previousPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.PreviousSessionID)
|
||||||
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(previousPrefix)
|
manifestKey, runIDKey := artifacts.ResolveCurrentStateKeys(previousPrefix)
|
||||||
previousRunID := "20260426T010203Z-a1b2c3d4"
|
previousRunID := "20260426T010203Z-a1b2c3d4"
|
||||||
seedRestoreObject(fake, runIDKey, []byte(previousRunID+"\n"))
|
seedRestoreObject(fake, runIDKey, []byte(previousRunID+"\n"))
|
||||||
|
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package app
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"os"
|
"os"
|
||||||
@@ -13,6 +14,7 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/pathsafe"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/previouscache"
|
"gitea.maximumdirect.net/eric/narratio/internal/previouscache"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -215,19 +217,17 @@ func joinWithinSessionRoot(sessionRoot, relative string) (string, error) {
|
|||||||
if strings.TrimSpace(sessionRoot) == "" {
|
if strings.TrimSpace(sessionRoot) == "" {
|
||||||
return "", fmt.Errorf("session root is required")
|
return "", fmt.Errorf("session root is required")
|
||||||
}
|
}
|
||||||
cleanRel := path.Clean(strings.TrimSpace(relative))
|
joined, err := pathsafe.JoinSlashRelativeUnderRoot(sessionRoot, filepath.ToSlash(strings.TrimSpace(relative)))
|
||||||
if cleanRel == "." || cleanRel == "" {
|
if err != nil {
|
||||||
return "", fmt.Errorf("relative path is required")
|
if errors.Is(err, pathsafe.ErrRelativePathRequired) {
|
||||||
|
return "", fmt.Errorf("relative path is required")
|
||||||
|
}
|
||||||
|
if errors.Is(err, pathsafe.ErrRelativePathEscape) || errors.Is(err, pathsafe.ErrRelativePathAbsolute) {
|
||||||
|
return "", fmt.Errorf("relative path escapes session root")
|
||||||
|
}
|
||||||
|
return "", fmt.Errorf("join relative path under session root: %w", err)
|
||||||
}
|
}
|
||||||
if cleanRel == ".." || strings.HasPrefix(cleanRel, "../") || strings.HasPrefix(cleanRel, "/") {
|
return joined, nil
|
||||||
return "", fmt.Errorf("relative path escapes session root")
|
|
||||||
}
|
|
||||||
abs := filepath.Clean(filepath.Join(sessionRoot, filepath.FromSlash(cleanRel)))
|
|
||||||
root := filepath.Clean(sessionRoot)
|
|
||||||
if abs != root && !strings.HasPrefix(abs, root+string(filepath.Separator)) {
|
|
||||||
return "", fmt.Errorf("resolved local path escapes session root")
|
|
||||||
}
|
|
||||||
return abs, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func buildPreviousCacheRestoreActions(
|
func buildPreviousCacheRestoreActions(
|
||||||
@@ -338,7 +338,7 @@ func classifyRestoreAction(
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return RestoreAction{}, fmt.Errorf("checksum local file: %w", err)
|
return RestoreAction{}, fmt.Errorf("checksum local file: %w", err)
|
||||||
}
|
}
|
||||||
remotePath, err := downloadObjectToTemp(ctx, store, action.RemoteKey, "narratio-restore-plan-remote-*.tmp")
|
remotePath, err := storage.DownloadObjectToTemp(ctx, store, action.RemoteKey, "narratio-restore-plan-remote-*.tmp")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return RestoreAction{}, fmt.Errorf("download remote object: %w", err)
|
return RestoreAction{}, fmt.Errorf("download remote object: %w", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ func TestRestorePlanDefaultScope(t *testing.T) {
|
|||||||
seedRestoreObject(store, current.SessionPrefix+"artifacts/session_recap.md", []byte("# recap\n"))
|
seedRestoreObject(store, current.SessionPrefix+"artifacts/session_recap.md", []byte("# recap\n"))
|
||||||
seedRestoreObject(store, current.SessionPrefix+"audio/alice.flac", []byte("audio"))
|
seedRestoreObject(store, current.SessionPrefix+"audio/alice.flac", []byte("audio"))
|
||||||
seedRestoreObject(store, current.SessionPrefix+"runs/20260519T010203Z-a1b2/manifest.json", []byte("{}"))
|
seedRestoreObject(store, current.SessionPrefix+"runs/20260519T010203Z-a1b2/manifest.json", []byte("{}"))
|
||||||
seedRestoreObject(store, current.SessionPrefix+"logs/archive.log", []byte("log"))
|
seedRestoreObject(store, current.SessionPrefix+"logs/publish.log", []byte("log"))
|
||||||
|
|
||||||
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
plan, err := buildRestorePlan(context.Background(), cfg, current, store, RestorePlanOptions{})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -325,7 +325,7 @@ func configureRestorePlanPreviousRequirement(cfg *config.Config, required bool)
|
|||||||
func restorePlanCurrentState(t *testing.T, cfg *config.Config) *RemoteCurrentState {
|
func restorePlanCurrentState(t *testing.T, cfg *config.Config) *RemoteCurrentState {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
sessionPrefix := artifacts.S3SessionPrefix("dnd", cfg.Session.Campaign, cfg.Session.SessionID)
|
sessionPrefix := artifacts.S3SessionPrefix("dnd", cfg.Session.Campaign, cfg.Session.SessionID)
|
||||||
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
manifestKey, runIDKey := artifacts.ResolveCurrentStateKeys(sessionPrefix)
|
||||||
return &RemoteCurrentState{
|
return &RemoteCurrentState{
|
||||||
Bucket: "test-bucket",
|
Bucket: "test-bucket",
|
||||||
SessionPrefix: sessionPrefix,
|
SessionPrefix: sessionPrefix,
|
||||||
|
|||||||
@@ -201,7 +201,7 @@ func writeRestoreSuccessSummary(out io.Writer, report *RestoreReport) error {
|
|||||||
if report == nil {
|
if report == nil {
|
||||||
return fmt.Errorf("restore report is required")
|
return fmt.Errorf("restore report is required")
|
||||||
}
|
}
|
||||||
if _, err := fmt.Fprintf(out, "Restored session archive for %s/%s\n", report.Campaign, report.SessionID); err != nil {
|
if _, err := fmt.Fprintf(out, "Restored session state for %s/%s\n", report.Campaign, report.SessionID); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if _, err := fmt.Fprintf(out, "Remote run: %s\n", report.RunID); err != nil {
|
if _, err := fmt.Fprintf(out, "Remote run: %s\n", report.RunID); err != nil {
|
||||||
|
|||||||
@@ -369,7 +369,7 @@ func TestExecuteRestoreNonDryRunForceExecutesPlan(t *testing.T) {
|
|||||||
if code != 0 {
|
if code != 0 {
|
||||||
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
}
|
}
|
||||||
if !strings.Contains(stdout.String(), "Restored session archive for sample-campaign/2026-05-03") {
|
if !strings.Contains(stdout.String(), "Restored session state for sample-campaign/2026-05-03") {
|
||||||
t.Fatalf("stdout = %q, want completion summary", stdout.String())
|
t.Fatalf("stdout = %q, want completion summary", stdout.String())
|
||||||
}
|
}
|
||||||
if stderr.Len() != 0 {
|
if stderr.Len() != 0 {
|
||||||
|
|||||||
@@ -1,128 +0,0 @@
|
|||||||
package app
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"flag"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"strings"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
|
||||||
)
|
|
||||||
|
|
||||||
// Resume continues execution from the first non-succeeded stage in the manifest.
|
|
||||||
func Resume(ctx context.Context, args []string, out io.Writer) error {
|
|
||||||
positionalSessionID, args := pullLeadingSessionID(args)
|
|
||||||
fs := flag.NewFlagSet("resume", flag.ContinueOnError)
|
|
||||||
fs.SetOutput(io.Discard)
|
|
||||||
|
|
||||||
var pipelinePath string
|
|
||||||
var campaignPath string
|
|
||||||
var campaignFilePath string
|
|
||||||
var sessionPath string
|
|
||||||
var sessionID string
|
|
||||||
var previousSessionID string
|
|
||||||
var force bool
|
|
||||||
var selectedArtifacts artifactSelectionFlag
|
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
|
||||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
|
||||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
|
||||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
|
||||||
fs.BoolVar(&force, "force", false, "force stage execution")
|
|
||||||
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute and publish (comma-separated or repeatable)")
|
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
|
||||||
return fmt.Errorf("resume: invalid flags: %w", err)
|
|
||||||
}
|
|
||||||
if positionalSessionID == "" {
|
|
||||||
if err := applyParsedSessionIDArg("resume", fs, &sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
if fs.NArg() != 0 {
|
|
||||||
return fmt.Errorf("resume: unexpected positional arguments")
|
|
||||||
}
|
|
||||||
if err := applyPositionalSessionID("resume", positionalSessionID, &sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if strings.TrimSpace(sessionID) == "" {
|
|
||||||
return fmt.Errorf("resume: session_id is required")
|
|
||||||
}
|
|
||||||
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
|
||||||
SessionID: sessionID,
|
|
||||||
PreviousSessionID: previousSessionID,
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("resume: %w", err)
|
|
||||||
}
|
|
||||||
if err := config.Validate(cfg); err != nil {
|
|
||||||
return fmt.Errorf("resume: %w", err)
|
|
||||||
}
|
|
||||||
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("resume: invalid --artifacts: %w", err)
|
|
||||||
}
|
|
||||||
if err := validateSelectedArtifacts(cfg, normalizedArtifacts); err != nil {
|
|
||||||
return fmt.Errorf("resume: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
full := BuildFullPlan()
|
|
||||||
selected := full
|
|
||||||
if !force {
|
|
||||||
m, err := loadManifestIfPresent(ctx, cfg)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("resume: %w", err)
|
|
||||||
}
|
|
||||||
if m != nil {
|
|
||||||
start := firstNonSucceededIndex(full, m)
|
|
||||||
if start >= len(full) {
|
|
||||||
_, err := fmt.Fprintf(out, "narratio resume: session %s has no remaining stages\n", cfg.Session.SessionID)
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
selected = full[start:]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
summary, err := executeStagesFn(ctx, cfg, selected, RunOptions{
|
|
||||||
Force: force,
|
|
||||||
SelectedArtifacts: normalizedArtifacts,
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("resume: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = fmt.Fprintf(
|
|
||||||
out,
|
|
||||||
"narratio resume: session %s; executed=%d skipped=%d; manifest=%s\n",
|
|
||||||
summary.SessionID,
|
|
||||||
len(summary.Executed),
|
|
||||||
len(summary.Skipped),
|
|
||||||
summary.ManifestPath,
|
|
||||||
)
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
func loadManifestIfPresent(ctx context.Context, cfg *config.Config) (*manifest.Manifest, error) {
|
|
||||||
path := artifacts.SessionManifestPathForCampaign(
|
|
||||||
cfg.Pipeline.Workspace.Root,
|
|
||||||
cfg.Session.Campaign,
|
|
||||||
cfg.Session.SessionID,
|
|
||||||
)
|
|
||||||
exists, err := fileExists(path)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("check manifest %q: %w", path, err)
|
|
||||||
}
|
|
||||||
if !exists {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
store := &manifest.LocalStore{}
|
|
||||||
m, err := store.Load(ctx, path)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("load manifest %q: %w", path, err)
|
|
||||||
}
|
|
||||||
return m, nil
|
|
||||||
}
|
|
||||||
@@ -5,55 +5,29 @@ import (
|
|||||||
"flag"
|
"flag"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"strings"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Run executes the pipeline plan and persists manifest state.
|
// Run executes the pipeline plan and persists manifest state.
|
||||||
func Run(ctx context.Context, args []string, out io.Writer) error {
|
func Run(ctx context.Context, args []string, out io.Writer) error {
|
||||||
positionalSessionID, args := pullLeadingSessionID(args)
|
|
||||||
fs := flag.NewFlagSet("run", flag.ContinueOnError)
|
fs := flag.NewFlagSet("run", flag.ContinueOnError)
|
||||||
fs.SetOutput(io.Discard)
|
fs.SetOutput(io.Discard)
|
||||||
|
|
||||||
var pipelinePath string
|
var flags commonConfigFlags
|
||||||
var campaignPath string
|
|
||||||
var campaignFilePath string
|
|
||||||
var sessionPath string
|
|
||||||
var sessionID string
|
|
||||||
var previousSessionID string
|
|
||||||
var force bool
|
var force bool
|
||||||
var selectedArtifacts artifactSelectionFlag
|
var selectedArtifacts artifactSelectionFlag
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
addCommonConfigFlags(fs, &flags)
|
||||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
|
||||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
|
||||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
|
||||||
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
||||||
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute and publish (comma-separated or repeatable)")
|
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute and publish (comma-separated or repeatable)")
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
if err := parseSessionAwareFlags("run", fs, args, &flags.sessionID); err != nil {
|
||||||
return fmt.Errorf("run: invalid flags: %w", err)
|
return err
|
||||||
}
|
}
|
||||||
if positionalSessionID == "" {
|
if flags.sessionID == "" {
|
||||||
if err := applyParsedSessionIDArg("run", fs, &sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
if fs.NArg() != 0 {
|
|
||||||
return fmt.Errorf("run: unexpected positional arguments")
|
|
||||||
}
|
|
||||||
if err := applyPositionalSessionID("run", positionalSessionID, &sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if strings.TrimSpace(sessionID) == "" {
|
|
||||||
return fmt.Errorf("run: session_id is required")
|
return fmt.Errorf("run: session_id is required")
|
||||||
}
|
}
|
||||||
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
|
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
|
||||||
SessionID: sessionID,
|
|
||||||
PreviousSessionID: previousSessionID,
|
|
||||||
})
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("run: %w", err)
|
return fmt.Errorf("run: %w", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,8 +1,12 @@
|
|||||||
package app
|
package app
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/stage"
|
"gitea.maximumdirect.net/eric/narratio/internal/stage"
|
||||||
)
|
)
|
||||||
@@ -43,13 +47,25 @@ func stageSucceeded(m *manifest.Manifest, name string) bool {
|
|||||||
return sr != nil && sr.Status == manifest.StatusSucceeded
|
return sr != nil && sr.Status == manifest.StatusSucceeded
|
||||||
}
|
}
|
||||||
|
|
||||||
func firstNonSucceededIndex(stages []stage.Stage, m *manifest.Manifest) int {
|
func loadManifestIfPresent(ctx context.Context, cfg *config.Config) (*manifest.Manifest, error) {
|
||||||
for i, s := range stages {
|
path := artifacts.SessionManifestPathForCampaign(
|
||||||
if !stageSucceeded(m, s.Name()) {
|
cfg.Pipeline.Workspace.Root,
|
||||||
return i
|
cfg.Session.Campaign,
|
||||||
}
|
cfg.Session.SessionID,
|
||||||
|
)
|
||||||
|
exists, err := fileExists(path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("check manifest %q: %w", path, err)
|
||||||
}
|
}
|
||||||
return len(stages)
|
if !exists {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
store := &manifest.LocalStore{}
|
||||||
|
m, err := store.Load(ctx, path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("load manifest %q: %w", path, err)
|
||||||
|
}
|
||||||
|
return m, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func canonicalStageNames() []string {
|
func canonicalStageNames() []string {
|
||||||
|
|||||||
@@ -8,18 +8,6 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestFirstNonSucceededIndex(t *testing.T) {
|
|
||||||
stages := BuildFullPlan()
|
|
||||||
m := manifest.New("2026-05-03", time.Now().UTC())
|
|
||||||
m.MarkStageSucceeded("prepare", time.Now().UTC(), nil)
|
|
||||||
m.MarkStageSucceeded("transcribe", time.Now().UTC(), nil)
|
|
||||||
|
|
||||||
got := firstNonSucceededIndex(stages, m)
|
|
||||||
if got != 2 {
|
|
||||||
t.Fatalf("firstNonSucceededIndex() = %d, want 2", got)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestDecideStageActions(t *testing.T) {
|
func TestDecideStageActions(t *testing.T) {
|
||||||
stages := BuildFullPlan()[:2]
|
stages := BuildFullPlan()[:2]
|
||||||
m := manifest.New("2026-05-03", time.Now().UTC())
|
m := manifest.New("2026-05-03", time.Now().UTC())
|
||||||
@@ -44,7 +32,7 @@ func TestDecideStageActions(t *testing.T) {
|
|||||||
|
|
||||||
func TestDownstreamStageNames(t *testing.T) {
|
func TestDownstreamStageNames(t *testing.T) {
|
||||||
got := downstreamStageNames("polish")
|
got := downstreamStageNames("polish")
|
||||||
want := []string{"normalize", "trim", "analyze", "publish", "notify"}
|
want := []string{"normalize", "trim", "render", "analyze", "publish", "notify"}
|
||||||
if !reflect.DeepEqual(got, want) {
|
if !reflect.DeepEqual(got, want) {
|
||||||
t.Fatalf("downstreamStageNames(polish) = %#v, want %#v", got, want)
|
t.Fatalf("downstreamStageNames(polish) = %#v, want %#v", got, want)
|
||||||
}
|
}
|
||||||
@@ -64,12 +52,13 @@ func TestInvalidateDownstreamSucceededStages(t *testing.T) {
|
|||||||
m.MarkStageSucceeded("polish", now, nil)
|
m.MarkStageSucceeded("polish", now, nil)
|
||||||
m.MarkStageSucceeded("normalize", now, nil)
|
m.MarkStageSucceeded("normalize", now, nil)
|
||||||
m.MarkStageSucceeded("trim", now, nil)
|
m.MarkStageSucceeded("trim", now, nil)
|
||||||
|
m.MarkStageSucceeded("render", now, nil)
|
||||||
m.MarkStageFailed("analyze", now, "analysis failed")
|
m.MarkStageFailed("analyze", now, "analysis failed")
|
||||||
m.MarkStageSucceeded("publish", now, nil)
|
m.MarkStageSucceeded("publish", now, nil)
|
||||||
m.MarkStageSucceeded("notify", now, nil)
|
m.MarkStageSucceeded("notify", now, nil)
|
||||||
|
|
||||||
got := invalidateDownstreamSucceededStages(m, "polish", now.Add(1*time.Second))
|
got := invalidateDownstreamSucceededStages(m, "polish", now.Add(1*time.Second))
|
||||||
want := []string{"normalize", "trim", "publish", "notify"}
|
want := []string{"normalize", "trim", "render", "publish", "notify"}
|
||||||
if !reflect.DeepEqual(got, want) {
|
if !reflect.DeepEqual(got, want) {
|
||||||
t.Fatalf("invalidateDownstreamSucceededStages() = %#v, want %#v", got, want)
|
t.Fatalf("invalidateDownstreamSucceededStages() = %#v, want %#v", got, want)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -23,19 +23,10 @@ func RunStage(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
fs := flag.NewFlagSet("run-stage", flag.ContinueOnError)
|
fs := flag.NewFlagSet("run-stage", flag.ContinueOnError)
|
||||||
fs.SetOutput(io.Discard)
|
fs.SetOutput(io.Discard)
|
||||||
|
|
||||||
var pipelinePath string
|
var flags commonConfigFlags
|
||||||
var campaignPath string
|
|
||||||
var campaignFilePath string
|
|
||||||
var sessionPath string
|
|
||||||
var sessionID string
|
|
||||||
var previousSessionID string
|
|
||||||
var force bool
|
var force bool
|
||||||
var selectedArtifacts artifactSelectionFlag
|
var selectedArtifacts artifactSelectionFlag
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
addCommonConfigFlags(fs, &flags)
|
||||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
|
||||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
|
||||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
|
||||||
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
|
||||||
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute or publish (comma-separated or repeatable)")
|
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute or publish (comma-separated or repeatable)")
|
||||||
|
|
||||||
@@ -47,16 +38,21 @@ func RunStage(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
case 2:
|
case 2:
|
||||||
stageName = strings.TrimSpace(fs.Arg(0))
|
stageName = strings.TrimSpace(fs.Arg(0))
|
||||||
positionalSessionID = strings.TrimSpace(fs.Arg(1))
|
positionalSessionID = strings.TrimSpace(fs.Arg(1))
|
||||||
|
case 1:
|
||||||
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
|
return fmt.Errorf("run-stage: expected stage name and session_id")
|
||||||
|
}
|
||||||
|
stageName = strings.TrimSpace(fs.Arg(0))
|
||||||
default:
|
default:
|
||||||
return fmt.Errorf("run-stage: expected stage name and session_id")
|
return fmt.Errorf("run-stage: expected stage name and session_id")
|
||||||
}
|
}
|
||||||
} else if fs.NArg() != 0 {
|
} else if fs.NArg() != 0 {
|
||||||
return fmt.Errorf("run-stage: unexpected positional arguments")
|
return fmt.Errorf("run-stage: unexpected positional arguments")
|
||||||
}
|
}
|
||||||
if err := applyPositionalSessionID("run-stage", positionalSessionID, &sessionID); err != nil {
|
if err := applyPositionalSessionID("run-stage", positionalSessionID, &flags.sessionID); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if strings.TrimSpace(sessionID) == "" {
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
return fmt.Errorf("run-stage: session_id is required")
|
return fmt.Errorf("run-stage: session_id is required")
|
||||||
}
|
}
|
||||||
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
||||||
@@ -70,12 +66,12 @@ func RunStage(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
||||||
CommandName: "run-stage",
|
CommandName: "run-stage",
|
||||||
StageName: stageName,
|
StageName: stageName,
|
||||||
PipelinePath: pipelinePath,
|
PipelinePath: flags.pipelinePath,
|
||||||
CampaignPath: campaignPath,
|
CampaignPath: flags.campaignPath,
|
||||||
CampaignFilePath: campaignFilePath,
|
CampaignFilePath: flags.campaignFilePath,
|
||||||
SessionPath: sessionPath,
|
SessionPath: flags.sessionPath,
|
||||||
SessionID: sessionID,
|
SessionID: flags.sessionID,
|
||||||
PreviousSessionID: previousSessionID,
|
PreviousSessionID: flags.previousSessionID,
|
||||||
Force: force,
|
Force: force,
|
||||||
SelectedArtifacts: normalizedArtifacts,
|
SelectedArtifacts: normalizedArtifacts,
|
||||||
})
|
})
|
||||||
@@ -97,40 +93,18 @@ func RunStage(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
|
|
||||||
// Analyze force-runs the analyze stage.
|
// Analyze force-runs the analyze stage.
|
||||||
func Analyze(ctx context.Context, args []string, out io.Writer) error {
|
func Analyze(ctx context.Context, args []string, out io.Writer) error {
|
||||||
positionalSessionID, args := pullLeadingSessionID(args)
|
|
||||||
fs := flag.NewFlagSet("analyze", flag.ContinueOnError)
|
fs := flag.NewFlagSet("analyze", flag.ContinueOnError)
|
||||||
fs.SetOutput(io.Discard)
|
fs.SetOutput(io.Discard)
|
||||||
|
|
||||||
var pipelinePath string
|
var flags commonConfigFlags
|
||||||
var campaignPath string
|
|
||||||
var campaignFilePath string
|
|
||||||
var sessionPath string
|
|
||||||
var sessionID string
|
|
||||||
var previousSessionID string
|
|
||||||
var selectedArtifacts artifactSelectionFlag
|
var selectedArtifacts artifactSelectionFlag
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
addCommonConfigFlags(fs, &flags)
|
||||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
|
||||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
|
||||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
|
||||||
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute during analyze (comma-separated or repeatable)")
|
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute during analyze (comma-separated or repeatable)")
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
if err := parseSessionAwareFlags("analyze", fs, args, &flags.sessionID); err != nil {
|
||||||
return fmt.Errorf("analyze: invalid flags: %w", err)
|
return err
|
||||||
}
|
}
|
||||||
if positionalSessionID == "" {
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
if err := applyParsedSessionIDArg("analyze", fs, &sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
if fs.NArg() != 0 {
|
|
||||||
return fmt.Errorf("analyze: unexpected positional arguments")
|
|
||||||
}
|
|
||||||
if err := applyPositionalSessionID("analyze", positionalSessionID, &sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if strings.TrimSpace(sessionID) == "" {
|
|
||||||
return fmt.Errorf("analyze: session_id is required")
|
return fmt.Errorf("analyze: session_id is required")
|
||||||
}
|
}
|
||||||
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
||||||
@@ -141,12 +115,12 @@ func Analyze(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
||||||
CommandName: "analyze",
|
CommandName: "analyze",
|
||||||
StageName: "analyze",
|
StageName: "analyze",
|
||||||
PipelinePath: pipelinePath,
|
PipelinePath: flags.pipelinePath,
|
||||||
CampaignPath: campaignPath,
|
CampaignPath: flags.campaignPath,
|
||||||
CampaignFilePath: campaignFilePath,
|
CampaignFilePath: flags.campaignFilePath,
|
||||||
SessionPath: sessionPath,
|
SessionPath: flags.sessionPath,
|
||||||
SessionID: sessionID,
|
SessionID: flags.sessionID,
|
||||||
PreviousSessionID: previousSessionID,
|
PreviousSessionID: flags.previousSessionID,
|
||||||
Force: true,
|
Force: true,
|
||||||
SelectedArtifacts: normalizedArtifacts,
|
SelectedArtifacts: normalizedArtifacts,
|
||||||
})
|
})
|
||||||
@@ -166,40 +140,18 @@ func Analyze(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
|
|
||||||
// Publish force-runs the publish stage.
|
// Publish force-runs the publish stage.
|
||||||
func Publish(ctx context.Context, args []string, out io.Writer) error {
|
func Publish(ctx context.Context, args []string, out io.Writer) error {
|
||||||
positionalSessionID, args := pullLeadingSessionID(args)
|
|
||||||
fs := flag.NewFlagSet("publish", flag.ContinueOnError)
|
fs := flag.NewFlagSet("publish", flag.ContinueOnError)
|
||||||
fs.SetOutput(io.Discard)
|
fs.SetOutput(io.Discard)
|
||||||
|
|
||||||
var pipelinePath string
|
var flags commonConfigFlags
|
||||||
var campaignPath string
|
|
||||||
var campaignFilePath string
|
|
||||||
var sessionPath string
|
|
||||||
var sessionID string
|
|
||||||
var previousSessionID string
|
|
||||||
var selectedArtifacts artifactSelectionFlag
|
var selectedArtifacts artifactSelectionFlag
|
||||||
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
|
addCommonConfigFlags(fs, &flags)
|
||||||
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
|
|
||||||
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
|
|
||||||
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
|
|
||||||
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
|
|
||||||
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to publish (comma-separated or repeatable)")
|
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to publish (comma-separated or repeatable)")
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
if err := parseSessionAwareFlags("publish", fs, args, &flags.sessionID); err != nil {
|
||||||
return fmt.Errorf("publish: invalid flags: %w", err)
|
return err
|
||||||
}
|
}
|
||||||
if positionalSessionID == "" {
|
if strings.TrimSpace(flags.sessionID) == "" {
|
||||||
if err := applyParsedSessionIDArg("publish", fs, &sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
if fs.NArg() != 0 {
|
|
||||||
return fmt.Errorf("publish: unexpected positional arguments")
|
|
||||||
}
|
|
||||||
if err := applyPositionalSessionID("publish", positionalSessionID, &sessionID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if strings.TrimSpace(sessionID) == "" {
|
|
||||||
return fmt.Errorf("publish: session_id is required")
|
return fmt.Errorf("publish: session_id is required")
|
||||||
}
|
}
|
||||||
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
normalizedArtifacts, err := selectedArtifacts.Normalize()
|
||||||
@@ -210,12 +162,12 @@ func Publish(ctx context.Context, args []string, out io.Writer) error {
|
|||||||
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
summary, err := runSingleStageCommand(ctx, singleStageCommand{
|
||||||
CommandName: "publish",
|
CommandName: "publish",
|
||||||
StageName: "publish",
|
StageName: "publish",
|
||||||
PipelinePath: pipelinePath,
|
PipelinePath: flags.pipelinePath,
|
||||||
CampaignPath: campaignPath,
|
CampaignPath: flags.campaignPath,
|
||||||
CampaignFilePath: campaignFilePath,
|
CampaignFilePath: flags.campaignFilePath,
|
||||||
SessionPath: sessionPath,
|
SessionPath: flags.sessionPath,
|
||||||
SessionID: sessionID,
|
SessionID: flags.sessionID,
|
||||||
PreviousSessionID: previousSessionID,
|
PreviousSessionID: flags.previousSessionID,
|
||||||
Force: true,
|
Force: true,
|
||||||
SelectedArtifacts: normalizedArtifacts,
|
SelectedArtifacts: normalizedArtifacts,
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestResumeStartsAfterCompletedStages(t *testing.T) {
|
func TestRunContinuesAfterCompletedStages(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
@@ -32,12 +32,12 @@ func TestResumeStartsAfterCompletedStages(t *testing.T) {
|
|||||||
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
mustWriteTestFile(t, filepath.Join(workRoot, "inputs", "glossary.yml"), "terms: []\n")
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
err := Run(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Resume() error = %v", err)
|
t.Fatalf("Run() error = %v", err)
|
||||||
}
|
}
|
||||||
if !strings.Contains(out.String(), "executed=7 skipped=0") {
|
if !strings.Contains(out.String(), "executed=8 skipped=2") {
|
||||||
t.Fatalf("output = %q, want executed=7 skipped=0", out.String())
|
t.Fatalf("output = %q, want executed=8 skipped=2", out.String())
|
||||||
}
|
}
|
||||||
|
|
||||||
loaded, err := store.Load(context.Background(), manifestPath)
|
loaded, err := store.Load(context.Background(), manifestPath)
|
||||||
@@ -49,14 +49,14 @@ func TestResumeStartsAfterCompletedStages(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestResumeNoRemainingStages(t *testing.T) {
|
func TestRunNoRemainingStagesRecordsSkippedStages(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
m := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
m := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "analyze", "publish", "notify"} {
|
||||||
m.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
m.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
||||||
}
|
}
|
||||||
if err := store.Save(context.Background(), manifestPath, m); err != nil {
|
if err := store.Save(context.Background(), manifestPath, m); err != nil {
|
||||||
@@ -64,20 +64,20 @@ func TestResumeNoRemainingStages(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
err := Run(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Resume() error = %v", err)
|
t.Fatalf("Run() error = %v", err)
|
||||||
}
|
}
|
||||||
if !strings.Contains(out.String(), "has no remaining stages") {
|
if !strings.Contains(out.String(), "executed=0 skipped=10") {
|
||||||
t.Fatalf("output = %q, want no remaining stages", out.String())
|
t.Fatalf("output = %q, want executed=0 skipped=10", out.String())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestResumeForceRerunsSucceeded(t *testing.T) {
|
func TestRunForceRerunsSucceeded(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
w.Header().Set("Content-Type", "application/json")
|
w.Header().Set("Content-Type", "application/json")
|
||||||
_, _ = w.Write([]byte(`{"source":"resume-force-test","segments":[{"speaker":"alice"}]}`))
|
_, _ = w.Write([]byte(`{"source":"run-force-test","segments":[{"speaker":"alice"}]}`))
|
||||||
}))
|
}))
|
||||||
defer srv.Close()
|
defer srv.Close()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot, srv.URL)
|
||||||
@@ -85,7 +85,7 @@ func TestResumeForceRerunsSucceeded(t *testing.T) {
|
|||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
m := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
m := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "analyze", "publish", "notify"} {
|
||||||
m.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
m.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
||||||
}
|
}
|
||||||
if err := store.Save(context.Background(), manifestPath, m); err != nil {
|
if err := store.Save(context.Background(), manifestPath, m); err != nil {
|
||||||
@@ -93,11 +93,11 @@ func TestResumeForceRerunsSucceeded(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
err := Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &out)
|
err := Run(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath, "--force"}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Resume() error = %v", err)
|
t.Fatalf("Run() error = %v", err)
|
||||||
}
|
}
|
||||||
if !strings.Contains(out.String(), "executed=9 skipped=0") {
|
if !strings.Contains(out.String(), "executed=10 skipped=0") {
|
||||||
t.Fatalf("output = %q, want forced full rerun", out.String())
|
t.Fatalf("output = %q, want forced full rerun", out.String())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -166,7 +166,7 @@ func TestRunStageSkipAndForce(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing.T) {
|
func TestRunStageForceMarksDownstreamStaleAndRunContinuesFromStale(t *testing.T) {
|
||||||
workspaceRoot := t.TempDir()
|
workspaceRoot := t.TempDir()
|
||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
@@ -176,7 +176,7 @@ func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing
|
|||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
seed := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
seed := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "analyze", "publish", "notify"} {
|
||||||
seed.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
seed.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
||||||
}
|
}
|
||||||
if err := store.Save(context.Background(), manifestPath, seed); err != nil {
|
if err := store.Save(context.Background(), manifestPath, seed); err != nil {
|
||||||
@@ -196,19 +196,19 @@ func TestRunStageForceMarksDownstreamStaleAndResumeContinuesFromStale(t *testing
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("load manifest after force: %v", err)
|
t.Fatalf("load manifest after force: %v", err)
|
||||||
}
|
}
|
||||||
for _, name := range []string{"normalize", "trim", "analyze", "publish", "notify"} {
|
for _, name := range []string{"normalize", "trim", "render", "analyze", "publish", "notify"} {
|
||||||
if afterForce.Stages[name] == nil || afterForce.Stages[name].Status != manifest.StatusStale {
|
if afterForce.Stages[name] == nil || afterForce.Stages[name].Status != manifest.StatusStale {
|
||||||
t.Fatalf("stage %q = %#v, want stale", name, afterForce.Stages[name])
|
t.Fatalf("stage %q = %#v, want stale", name, afterForce.Stages[name])
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
out.Reset()
|
out.Reset()
|
||||||
err = Resume(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
err = Run(context.Background(), []string{"2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Resume() error = %v", err)
|
t.Fatalf("Run() error = %v", err)
|
||||||
}
|
}
|
||||||
if !strings.Contains(out.String(), "executed=5 skipped=0") {
|
if !strings.Contains(out.String(), "executed=6 skipped=4") {
|
||||||
t.Fatalf("output = %q, want resume to execute normalize..notify", out.String())
|
t.Fatalf("output = %q, want run to execute stale downstream stages", out.String())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -266,3 +266,30 @@ func TestRunStageNormalizeExecutes(t *testing.T) {
|
|||||||
t.Fatalf("normalize stage = %#v, want succeeded", m.Stages["normalize"])
|
t.Fatalf("normalize stage = %#v, want succeeded", m.Stages["normalize"])
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRunStageRenderExecutes(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
manifestPath := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03", "manifest.json")
|
||||||
|
workRoot := filepath.Join(workspaceRoot, "work", "sample-campaign", "2026-05-03")
|
||||||
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.json"), `{"segments":[{"id":1}]}`)
|
||||||
|
mustWriteTestFile(t, filepath.Join(workRoot, "transcripts", "final.trimmed.json"), `{"segments":[{"id":2}]}`)
|
||||||
|
|
||||||
|
var out bytes.Buffer
|
||||||
|
err := RunStage(context.Background(), []string{"render", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath}, &out)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("RunStage(render) error = %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(out.String(), "stage=render executed=1 skipped=0") {
|
||||||
|
t.Fatalf("output = %q, want stage=render executed", out.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
store := &manifest.LocalStore{}
|
||||||
|
m, err := store.Load(context.Background(), manifestPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("load manifest: %v", err)
|
||||||
|
}
|
||||||
|
if m.Stages["render"] == nil || m.Stages["render"].Status != manifest.StatusSucceeded {
|
||||||
|
t.Fatalf("render stage = %#v, want succeeded", m.Stages["render"])
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -11,8 +11,8 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/notify"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/notify"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/scriptorium"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/scriptorium"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/seriatim"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/seriatim"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/whisperx"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/whisperx"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifactmodel"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/logging"
|
"gitea.maximumdirect.net/eric/narratio/internal/logging"
|
||||||
@@ -83,9 +83,6 @@ func executeStages(ctx context.Context, cfg *config.Config, stages []stage.Stage
|
|||||||
if env.Scriptorium == nil {
|
if env.Scriptorium == nil {
|
||||||
env.Scriptorium = scriptorium.NewSubprocessRunner()
|
env.Scriptorium = scriptorium.NewSubprocessRunner()
|
||||||
}
|
}
|
||||||
if env.Storage == nil {
|
|
||||||
env.Storage = &storage.NoopBackend{}
|
|
||||||
}
|
|
||||||
if env.ObjectStore == nil && needsObjectStoreForRun(env.Config, stages) {
|
if env.ObjectStore == nil && needsObjectStoreForRun(env.Config, stages) {
|
||||||
objectStore, err := newCommandObjectStore(ctx, env.Config, nil)
|
objectStore, err := newCommandObjectStore(ctx, env.Config, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -96,7 +93,7 @@ func executeStages(ctx context.Context, cfg *config.Config, stages []stage.Stage
|
|||||||
if needsRemoteLocksForRun(env.Config, stages) {
|
if needsRemoteLocksForRun(env.Config, stages) {
|
||||||
locks, err := loadEffectiveLocks(ctx, env.Config, env.ObjectStore)
|
locks, err := loadEffectiveLocks(ctx, env.Config, env.ObjectStore)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("load remote archive locks: %w", err)
|
return nil, fmt.Errorf("load remote publish locks: %w", err)
|
||||||
}
|
}
|
||||||
applyEffectiveLocks(env.Config, locks.All)
|
applyEffectiveLocks(env.Config, locks.All)
|
||||||
}
|
}
|
||||||
@@ -234,14 +231,14 @@ func executeStages(ctx context.Context, cfg *config.Config, stages []stage.Stage
|
|||||||
env.Logger.Info("stage succeeded", "stage", s.Name())
|
env.Logger.Info("stage succeeded", "stage", s.Name())
|
||||||
}
|
}
|
||||||
|
|
||||||
if err := runPostArchiveCleanup(ctx, env, manifestPath, m, executed); err != nil {
|
if err := runPostPublishCleanup(ctx, env, manifestPath, m, executed); err != nil {
|
||||||
failedAt := nowUTC()
|
failedAt := nowUTC()
|
||||||
runManifest.MarkFailed(failedAt, err.Error())
|
runManifest.MarkFailed(failedAt, err.Error())
|
||||||
syncRunManifestIdentityFromSession(m, runManifest)
|
syncRunManifestIdentityFromSession(m, runManifest)
|
||||||
if saveErr := runManifestStore.SaveRun(ctx, runManifestPath, runManifest); saveErr != nil {
|
if saveErr := runManifestStore.SaveRun(ctx, runManifestPath, runManifest); saveErr != nil {
|
||||||
return nil, fmt.Errorf("post-archive cleanup failed (%v) and run-manifest save failed (%v)", err, saveErr)
|
return nil, fmt.Errorf("post-publish cleanup failed (%v) and run-manifest save failed (%v)", err, saveErr)
|
||||||
}
|
}
|
||||||
return nil, fmt.Errorf("post-archive cleanup: %w", err)
|
return nil, fmt.Errorf("post-publish cleanup: %w", err)
|
||||||
}
|
}
|
||||||
completedAt := nowUTC()
|
completedAt := nowUTC()
|
||||||
runManifest.MarkSucceeded(completedAt)
|
runManifest.MarkSucceeded(completedAt)
|
||||||
@@ -414,6 +411,8 @@ func mapResultOutputs(stageName string, result *stage.StageResult, runID string)
|
|||||||
if stageName == "analyze" {
|
if stageName == "analyze" {
|
||||||
sourceID = artifacts.ConfiguredArtifactSourceID(ref.Kind)
|
sourceID = artifacts.ConfiguredArtifactSourceID(ref.Kind)
|
||||||
kind = "scriptorium_artifact"
|
kind = "scriptorium_artifact"
|
||||||
|
} else {
|
||||||
|
sourceID = sourceIDForOutputKind(kind)
|
||||||
}
|
}
|
||||||
out = append(out, manifest.ArtifactRecord{
|
out = append(out, manifest.ArtifactRecord{
|
||||||
Kind: kind,
|
Kind: kind,
|
||||||
@@ -428,6 +427,22 @@ func mapResultOutputs(stageName string, result *stage.StageResult, runID string)
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func sourceIDForOutputKind(kind string) string {
|
||||||
|
trimmed := strings.TrimSpace(kind)
|
||||||
|
if trimmed == "" {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
if trimmed == "session_bounds" {
|
||||||
|
return artifacts.ArtifactBoundsSession
|
||||||
|
}
|
||||||
|
for _, spec := range artifactmodel.RuntimeTranscriptArtifacts() {
|
||||||
|
if spec.OutputKind == trimmed {
|
||||||
|
return spec.SourceID
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
func applyStageResultToManifest(m *manifest.Manifest, stageName string, result *stage.StageResult) {
|
func applyStageResultToManifest(m *manifest.Manifest, stageName string, result *stage.StageResult) {
|
||||||
if m == nil || result == nil {
|
if m == nil || result == nil {
|
||||||
return
|
return
|
||||||
|
|||||||
@@ -248,7 +248,7 @@ func TestNeedsObjectStoreForRunPrepareWithPreviousRequirements(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testing.T) {
|
func TestExecuteStagesPublishSkipsRequiredUnselectedConfiguredOutput(t *testing.T) {
|
||||||
cfg := testConfig(t)
|
cfg := testConfig(t)
|
||||||
cfg.Pipeline.Storage.S3 = &config.StorageS3Config{
|
cfg.Pipeline.Storage.S3 = &config.StorageS3Config{
|
||||||
Bucket: "my-dnd-archive",
|
Bucket: "my-dnd-archive",
|
||||||
@@ -273,7 +273,7 @@ func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testi
|
|||||||
manifestPath := manifestPathFor(cfg)
|
manifestPath := manifestPathFor(cfg)
|
||||||
seed := manifest.New(cfg.Session.SessionID, time.Now().UTC())
|
seed := manifest.New(cfg.Session.SessionID, time.Now().UTC())
|
||||||
seed.Campaign = cfg.Session.Campaign
|
seed.Campaign = cfg.Session.Campaign
|
||||||
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim"} {
|
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render"} {
|
||||||
seed.MarkStageSucceeded(stageName, time.Now().UTC(), nil)
|
seed.MarkStageSucceeded(stageName, time.Now().UTC(), nil)
|
||||||
}
|
}
|
||||||
if err := os.MkdirAll(filepath.Dir(manifestPath), 0o755); err != nil {
|
if err := os.MkdirAll(filepath.Dir(manifestPath), 0o755); err != nil {
|
||||||
@@ -283,7 +283,7 @@ func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testi
|
|||||||
t.Fatalf("Save manifest error = %v", err)
|
t.Fatalf("Save manifest error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
archiveStageImpl, err := stage.Select("publish")
|
publishStageImpl, err := stage.Select("publish")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Select(publish) error = %v", err)
|
t.Fatalf("Select(publish) error = %v", err)
|
||||||
}
|
}
|
||||||
@@ -293,7 +293,7 @@ func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testi
|
|||||||
cfg,
|
cfg,
|
||||||
[]stage.Stage{
|
[]stage.Stage{
|
||||||
selectedAnalyzeArtifactStage{expected: []string{"player_handout"}},
|
selectedAnalyzeArtifactStage{expected: []string{"player_handout"}},
|
||||||
archiveStageImpl,
|
publishStageImpl,
|
||||||
},
|
},
|
||||||
RunOptions{
|
RunOptions{
|
||||||
SelectedArtifacts: []string{"player_handout"},
|
SelectedArtifacts: []string{"player_handout"},
|
||||||
@@ -304,7 +304,7 @@ func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testi
|
|||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
}
|
}
|
||||||
if len(summary.Executed) != 2 || summary.Executed[0] != "analyze" || summary.Executed[1] != "publish" {
|
if len(summary.Executed) != 2 || summary.Executed[0] != "analyze" || summary.Executed[1] != "publish" {
|
||||||
t.Fatalf("executed = %#v, want analyze and archive", summary.Executed)
|
t.Fatalf("executed = %#v, want analyze and publish", summary.Executed)
|
||||||
}
|
}
|
||||||
|
|
||||||
loadedManifest, err := store.Load(context.Background(), summary.ManifestPath)
|
loadedManifest, err := store.Load(context.Background(), summary.ManifestPath)
|
||||||
@@ -321,7 +321,7 @@ func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testi
|
|||||||
t.Fatalf("skipped item = %#v, want object", skipped[0])
|
t.Fatalf("skipped item = %#v, want object", skipped[0])
|
||||||
}
|
}
|
||||||
if item["source"] != "narratio.artifact.session_recap" || item["dest"] != "artifacts/session_recap.md" || item["required"] != true {
|
if item["source"] != "narratio.artifact.session_recap" || item["dest"] != "artifacts/session_recap.md" || item["required"] != true {
|
||||||
t.Fatalf("skipped item = %#v, want required session_recap promotion", item)
|
t.Fatalf("skipped item = %#v, want required session_recap published output", item)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -332,8 +332,8 @@ func TestExecuteStagesPlaceholderSuccessUpdatesManifest(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("executeStages() error = %v", err)
|
t.Fatalf("executeStages() error = %v", err)
|
||||||
}
|
}
|
||||||
if len(summary.StageNames) != 9 || len(summary.Executed) != 9 || len(summary.Skipped) != 0 {
|
if len(summary.StageNames) != 10 || len(summary.Executed) != 10 || len(summary.Skipped) != 0 {
|
||||||
t.Fatalf("summary = %#v, want all 9 executed", summary)
|
t.Fatalf("summary = %#v, want all 10 executed", summary)
|
||||||
}
|
}
|
||||||
|
|
||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
@@ -342,7 +342,7 @@ func TestExecuteStagesPlaceholderSuccessUpdatesManifest(t *testing.T) {
|
|||||||
t.Fatalf("Load manifest error = %v", err)
|
t.Fatalf("Load manifest error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze", "publish", "notify"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "analyze", "publish", "notify"} {
|
||||||
sr := m.Stages[name]
|
sr := m.Stages[name]
|
||||||
if sr == nil {
|
if sr == nil {
|
||||||
t.Fatalf("missing stage record %q", name)
|
t.Fatalf("missing stage record %q", name)
|
||||||
@@ -425,12 +425,21 @@ func TestExecuteStagesPlaceholderSuccessUpdatesManifest(t *testing.T) {
|
|||||||
}
|
}
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
if name == "render" {
|
||||||
|
if sr.Metadata == nil || sr.Metadata["stage"] != "render" {
|
||||||
|
t.Fatalf("render metadata missing stage=render: %#v", sr.Metadata)
|
||||||
|
}
|
||||||
|
if len(sr.Outputs) == 0 {
|
||||||
|
t.Fatalf("render outputs missing")
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
if name == "publish" {
|
if name == "publish" {
|
||||||
if sr.Metadata == nil || sr.Metadata["stage"] != "publish" {
|
if sr.Metadata == nil || sr.Metadata["stage"] != "publish" {
|
||||||
t.Fatalf("archive metadata missing stage=publish: %#v", sr.Metadata)
|
t.Fatalf("publish metadata missing stage=publish: %#v", sr.Metadata)
|
||||||
}
|
}
|
||||||
if sr.Metadata["skipped"] != true {
|
if sr.Metadata["skipped"] != true {
|
||||||
t.Fatalf("archive metadata missing skipped=true for test config without archive section: %#v", sr.Metadata)
|
t.Fatalf("publish metadata missing skipped=true for test config without publish section: %#v", sr.Metadata)
|
||||||
}
|
}
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
@@ -522,7 +531,7 @@ func TestExecuteStagesForceSuccessInvalidatesDownstreamSucceededStages(t *testin
|
|||||||
store := &manifest.LocalStore{}
|
store := &manifest.LocalStore{}
|
||||||
|
|
||||||
existing := manifest.New(cfg.Session.SessionID, time.Date(2026, 5, 3, 1, 0, 0, 0, time.UTC))
|
existing := manifest.New(cfg.Session.SessionID, time.Date(2026, 5, 3, 1, 0, 0, 0, time.UTC))
|
||||||
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "publish", "notify"} {
|
for _, stageName := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "publish", "notify"} {
|
||||||
existing.MarkStageSucceeded(stageName, time.Date(2026, 5, 3, 1, 1, 0, 0, time.UTC), nil)
|
existing.MarkStageSucceeded(stageName, time.Date(2026, 5, 3, 1, 1, 0, 0, time.UTC), nil)
|
||||||
}
|
}
|
||||||
existing.MarkStageFailed("analyze", time.Date(2026, 5, 3, 1, 1, 0, 0, time.UTC), "previous analyze failure")
|
existing.MarkStageFailed("analyze", time.Date(2026, 5, 3, 1, 1, 0, 0, time.UTC), "previous analyze failure")
|
||||||
@@ -553,7 +562,7 @@ func TestExecuteStagesForceSuccessInvalidatesDownstreamSucceededStages(t *testin
|
|||||||
if loaded.Stages["polish"] == nil || loaded.Stages["polish"].Status != manifest.StatusSucceeded {
|
if loaded.Stages["polish"] == nil || loaded.Stages["polish"].Status != manifest.StatusSucceeded {
|
||||||
t.Fatalf("polish status = %#v, want succeeded", loaded.Stages["polish"])
|
t.Fatalf("polish status = %#v, want succeeded", loaded.Stages["polish"])
|
||||||
}
|
}
|
||||||
for _, stageName := range []string{"normalize", "trim", "publish", "notify"} {
|
for _, stageName := range []string{"normalize", "trim", "render", "publish", "notify"} {
|
||||||
if loaded.Stages[stageName] == nil || loaded.Stages[stageName].Status != manifest.StatusStale {
|
if loaded.Stages[stageName] == nil || loaded.Stages[stageName].Status != manifest.StatusStale {
|
||||||
t.Fatalf("%s status = %#v, want stale", stageName, loaded.Stages[stageName])
|
t.Fatalf("%s status = %#v, want stale", stageName, loaded.Stages[stageName])
|
||||||
}
|
}
|
||||||
@@ -729,7 +738,7 @@ func TestExecuteStagesRunManifestRecordsSkippedStage(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteStagesRunLocalArtifactsAndCanonicalPromotion(t *testing.T) {
|
func TestExecuteStagesRunLocalArtifactsAndCanonicalSync(t *testing.T) {
|
||||||
cfg := testConfig(t)
|
cfg := testConfig(t)
|
||||||
stages := []stage.Stage{
|
stages := []stage.Stage{
|
||||||
BuildFullPlan()[0], // prepare
|
BuildFullPlan()[0], // prepare
|
||||||
@@ -778,7 +787,7 @@ func TestExecuteStagesRunLocalArtifactsAndCanonicalPromotion(t *testing.T) {
|
|||||||
}
|
}
|
||||||
for _, p := range canonicalChecks {
|
for _, p := range canonicalChecks {
|
||||||
if _, statErr := os.Stat(p); statErr != nil {
|
if _, statErr := os.Stat(p); statErr != nil {
|
||||||
t.Fatalf("canonical promoted artifact missing at %q: %v", p, statErr)
|
t.Fatalf("canonical published artifact missing at %q: %v", p, statErr)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -864,7 +873,7 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
|
|||||||
{name: "merge", env: &Env{Seriatim: &seriatim.FakeRunner{Err: errors.New("merge fail")}}},
|
{name: "merge", env: &Env{Seriatim: &seriatim.FakeRunner{Err: errors.New("merge fail")}}},
|
||||||
{name: "polish", env: &Env{Audita: &audita.FakeRunner{Err: errors.New("polish fail")}}},
|
{name: "polish", env: &Env{Audita: &audita.FakeRunner{Err: errors.New("polish fail")}}},
|
||||||
{name: "analyze", env: &Env{Scriptorium: &scriptorium.FakeRunner{RunErr: errors.New("analyze fail")}}},
|
{name: "analyze", env: &Env{Scriptorium: &scriptorium.FakeRunner{RunErr: errors.New("analyze fail")}}},
|
||||||
{name: "publish", env: &Env{ObjectStore: &storage.FakeBackend{UploadErr: errors.New("archive fail")}}},
|
{name: "publish", env: &Env{ObjectStore: &storage.FakeBackend{UploadErr: errors.New("publish fail")}}},
|
||||||
{name: "notify", env: &Env{Notifier: ¬ify.FakeSender{Err: errors.New("notify fail")}}},
|
{name: "notify", env: &Env{Notifier: ¬ify.FakeSender{Err: errors.New("notify fail")}}},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -961,13 +970,13 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
|
|||||||
runID := "20260516T010203Z-0a1b2c3d"
|
runID := "20260516T010203Z-0a1b2c3d"
|
||||||
runWorkDir := filepath.Join(cfg.Pipeline.Workspace.Root, "work", cfg.Session.Campaign, cfg.Session.SessionID, runID)
|
runWorkDir := filepath.Join(cfg.Pipeline.Workspace.Root, "work", cfg.Session.Campaign, cfg.Session.SessionID, runID)
|
||||||
if err := os.MkdirAll(filepath.Join(runWorkDir, "inputs"), 0o755); err != nil {
|
if err := os.MkdirAll(filepath.Join(runWorkDir, "inputs"), 0o755); err != nil {
|
||||||
t.Fatalf("mkdir archive inputs dir: %v", err)
|
t.Fatalf("mkdir publish inputs dir: %v", err)
|
||||||
}
|
}
|
||||||
if err := os.WriteFile(filepath.Join(runWorkDir, "inputs", "session.yml"), []byte("session_id: 2026-05-03\n"), 0o644); err != nil {
|
if err := os.WriteFile(filepath.Join(runWorkDir, "inputs", "session.yml"), []byte("session_id: 2026-05-03\n"), 0o644); err != nil {
|
||||||
t.Fatalf("write archive fixture session.yml: %v", err)
|
t.Fatalf("write publish fixture session.yml: %v", err)
|
||||||
}
|
}
|
||||||
if err := os.WriteFile(filepath.Join(runWorkDir, "manifest.json"), []byte("{}\n"), 0o644); err != nil {
|
if err := os.WriteFile(filepath.Join(runWorkDir, "manifest.json"), []byte("{}\n"), 0o644); err != nil {
|
||||||
t.Fatalf("write archive fixture manifest.json: %v", err)
|
t.Fatalf("write publish fixture manifest.json: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
seed := manifest.New(cfg.Session.SessionID, time.Now().UTC())
|
seed := manifest.New(cfg.Session.SessionID, time.Now().UTC())
|
||||||
@@ -977,11 +986,11 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
|
|||||||
seed.S3Bucket = "my-dnd-archive"
|
seed.S3Bucket = "my-dnd-archive"
|
||||||
seed.S3SessionPrefix = "dnd/campaigns/" + cfg.Session.Campaign + "/sessions/" + cfg.Session.SessionID + "/"
|
seed.S3SessionPrefix = "dnd/campaigns/" + cfg.Session.Campaign + "/sessions/" + cfg.Session.SessionID + "/"
|
||||||
seed.S3RunPrefix = seed.S3SessionPrefix + "runs/" + runID + "/"
|
seed.S3RunPrefix = seed.S3SessionPrefix + "runs/" + runID + "/"
|
||||||
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "analyze"} {
|
for _, name := range []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render", "analyze"} {
|
||||||
seed.MarkStageSucceeded(name, time.Now().UTC(), nil)
|
seed.MarkStageSucceeded(name, time.Now().UTC(), nil)
|
||||||
}
|
}
|
||||||
if err := tc.env.ManifestStore.Save(context.Background(), manifestPathFor(cfg), seed); err != nil {
|
if err := tc.env.ManifestStore.Save(context.Background(), manifestPathFor(cfg), seed); err != nil {
|
||||||
t.Fatalf("seed archive manifest: %v", err)
|
t.Fatalf("seed publish manifest: %v", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -41,3 +41,55 @@ func applyParsedSessionIDArg(command string, fs *flag.FlagSet, sessionID *string
|
|||||||
return fmt.Errorf("%s: unexpected positional arguments", command)
|
return fmt.Errorf("%s: unexpected positional arguments", command)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func resolveParsedSessionID(command, positionalSessionID string, fs *flag.FlagSet, sessionID *string) error {
|
||||||
|
if strings.TrimSpace(positionalSessionID) == "" {
|
||||||
|
return applyParsedSessionIDArg(command, fs, sessionID)
|
||||||
|
}
|
||||||
|
if fs.NArg() != 0 {
|
||||||
|
return fmt.Errorf("%s: unexpected positional arguments", command)
|
||||||
|
}
|
||||||
|
return applyPositionalSessionID(command, positionalSessionID, sessionID)
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseSessionAwareFlags(command string, fs *flag.FlagSet, args []string, sessionID *string) error {
|
||||||
|
positionalSessionID, args := pullLeadingSessionID(args)
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return fmt.Errorf("%s: invalid flags: %w", command, err)
|
||||||
|
}
|
||||||
|
return resolveParsedSessionID(command, positionalSessionID, fs, sessionID)
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseSessionIDAndOnePositionalArg(command, argName string, fs *flag.FlagSet, args []string, sessionID *string) (string, error) {
|
||||||
|
var positionalSessionID string
|
||||||
|
value := ""
|
||||||
|
if len(args) >= 2 && !isCLIFlagToken(args[0]) && !isCLIFlagToken(args[1]) {
|
||||||
|
positionalSessionID = strings.TrimSpace(args[0])
|
||||||
|
value = strings.TrimSpace(args[1])
|
||||||
|
args = append([]string(nil), args[2:]...)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return "", fmt.Errorf("%s: invalid flags: %w", command, err)
|
||||||
|
}
|
||||||
|
if value == "" {
|
||||||
|
switch fs.NArg() {
|
||||||
|
case 2:
|
||||||
|
positionalSessionID = strings.TrimSpace(fs.Arg(0))
|
||||||
|
value = strings.TrimSpace(fs.Arg(1))
|
||||||
|
case 1:
|
||||||
|
if strings.TrimSpace(*sessionID) == "" {
|
||||||
|
return "", fmt.Errorf("%s: expected session_id and %s", command, argName)
|
||||||
|
}
|
||||||
|
value = strings.TrimSpace(fs.Arg(0))
|
||||||
|
default:
|
||||||
|
return "", fmt.Errorf("%s: expected session_id and %s", command, argName)
|
||||||
|
}
|
||||||
|
} else if fs.NArg() != 0 {
|
||||||
|
return "", fmt.Errorf("%s: unexpected positional arguments", command)
|
||||||
|
}
|
||||||
|
if err := applyPositionalSessionID(command, positionalSessionID, sessionID); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return value, nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -71,15 +71,36 @@ func TestExecutePositionalSessionIDMismatchFails(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExecuteSessionIDFlagFails(t *testing.T) {
|
func TestExecuteSessionIDFlagMismatchFails(t *testing.T) {
|
||||||
var stdout bytes.Buffer
|
var stdout bytes.Buffer
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
code := Execute([]string{"session", "status", "2026-05-03", "--session-id", "2026-05-04"}, &stdout, &stderr)
|
code := Execute([]string{"session", "status", "2026-05-03", "--session-id", "2026-05-04"}, &stdout, &stderr)
|
||||||
if code == 0 {
|
if code == 0 {
|
||||||
t.Fatal("exit code = 0, want non-zero")
|
t.Fatal("exit code = 0, want non-zero")
|
||||||
}
|
}
|
||||||
if !strings.Contains(stderr.String(), "flag provided but not defined: -session-id") {
|
if !strings.Contains(stderr.String(), "does not match expected session id") {
|
||||||
t.Fatalf("stderr = %q, want invalid --session-id flag", stderr.String())
|
t.Fatalf("stderr = %q, want positional/flag mismatch", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecuteSessionIDFlagAcceptedWithoutPositional(t *testing.T) {
|
||||||
|
workspaceRoot := t.TempDir()
|
||||||
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
|
|
||||||
|
var stdout bytes.Buffer
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Execute([]string{
|
||||||
|
"session", "status",
|
||||||
|
"--session-id", "2026-05-03",
|
||||||
|
"--config", pipelinePath,
|
||||||
|
"--campaign-file", campaignPath,
|
||||||
|
"--session", sessionPath,
|
||||||
|
}, &stdout, &stderr)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(stdout.String(), "Session: 2026-05-03") {
|
||||||
|
t.Fatalf("stdout = %q, want status output", stdout.String())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -141,8 +162,8 @@ func TestExecuteWorkflowCommandsAcceptPositionalSessionID(t *testing.T) {
|
|||||||
wantForce bool
|
wantForce bool
|
||||||
}{
|
}{
|
||||||
{
|
{
|
||||||
name: "resume",
|
name: "run",
|
||||||
args: []string{"resume", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
args: []string{"run", "2026-05-03", "--config", pipelinePath, "--campaign-file", campaignPath, "--session", sessionPath},
|
||||||
wantStage: "prepare",
|
wantStage: "prepare",
|
||||||
wantForce: false,
|
wantForce: false,
|
||||||
},
|
},
|
||||||
@@ -212,7 +233,7 @@ func TestExecuteSessionSubcommandsAcceptPositionalSessionID(t *testing.T) {
|
|||||||
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
|
||||||
fake := &storage.FakeBackend{}
|
fake := &storage.FakeBackend{}
|
||||||
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
|
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
|
||||||
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
|
manifestKey, runIDKey := artifacts.ResolveCurrentStateKeys(sessionPrefix)
|
||||||
fake.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
fake.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||||
fake.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, "2026-05-03", "sample-campaign")})
|
fake.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, "2026-05-03", "sample-campaign")})
|
||||||
var storeInitCalls int
|
var storeInitCalls int
|
||||||
|
|||||||
@@ -3,24 +3,30 @@ package artifactmodel
|
|||||||
import "strings"
|
import "strings"
|
||||||
|
|
||||||
const (
|
const (
|
||||||
SourceTranscriptBase = "narratio.transcript.base"
|
SourceTranscriptBase = "narratio.transcript.base"
|
||||||
SourceTranscriptPolished = "narratio.transcript.polished"
|
SourceTranscriptPolished = "narratio.transcript.polished"
|
||||||
SourceTranscriptFinal = "narratio.transcript.final"
|
SourceTranscriptFinal = "narratio.transcript.final"
|
||||||
SourceTranscriptFinalTrimmed = "narratio.transcript.final_trimmed"
|
SourceTranscriptFinalTrimmed = "narratio.transcript.final_trimmed"
|
||||||
|
SourceTranscriptFinalMarkdown = "narratio.transcript.final_markdown"
|
||||||
|
SourceTranscriptFinalTrimmedMarkdown = "narratio.transcript.final_trimmed_markdown"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
TranscriptPathBase = "transcripts/base.json"
|
TranscriptPathBase = "transcripts/base.json"
|
||||||
TranscriptPathPolished = "transcripts/polished.json"
|
TranscriptPathPolished = "transcripts/polished.json"
|
||||||
TranscriptPathFinal = "transcripts/final.json"
|
TranscriptPathFinal = "transcripts/final.json"
|
||||||
TranscriptPathFinalTrimmed = "transcripts/final.trimmed.json"
|
TranscriptPathFinalTrimmed = "transcripts/final.trimmed.json"
|
||||||
|
TranscriptPathFinalMarkdown = "transcripts/final.md"
|
||||||
|
TranscriptPathFinalTrimmedMarkdown = "transcripts/final.trimmed.md"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
TranscriptOutputKindBase = "transcript_base"
|
TranscriptOutputKindBase = "transcript_base"
|
||||||
TranscriptOutputKindPolished = "transcript_polished"
|
TranscriptOutputKindPolished = "transcript_polished"
|
||||||
TranscriptOutputKindFinal = "transcript_final"
|
TranscriptOutputKindFinal = "transcript_final"
|
||||||
TranscriptOutputKindFinalTrimmed = "transcript_final_trimmed"
|
TranscriptOutputKindFinalTrimmed = "transcript_final_trimmed"
|
||||||
|
TranscriptOutputKindFinalMarkdown = "transcript_final_markdown"
|
||||||
|
TranscriptOutputKindFinalTrimmedMarkdown = "transcript_final_trimmed_markdown"
|
||||||
)
|
)
|
||||||
|
|
||||||
// TranscriptArtifactSpec describes one built-in transcript artifact mapping.
|
// TranscriptArtifactSpec describes one built-in transcript artifact mapping.
|
||||||
@@ -56,6 +62,18 @@ var runtimeTranscriptArtifacts = []TranscriptArtifactSpec{
|
|||||||
ProducerStage: "trim",
|
ProducerStage: "trim",
|
||||||
OutputKind: TranscriptOutputKindFinalTrimmed,
|
OutputKind: TranscriptOutputKindFinalTrimmed,
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptFinalMarkdown,
|
||||||
|
CanonicalRelPath: TranscriptPathFinalMarkdown,
|
||||||
|
ProducerStage: "render",
|
||||||
|
OutputKind: TranscriptOutputKindFinalMarkdown,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptFinalTrimmedMarkdown,
|
||||||
|
CanonicalRelPath: TranscriptPathFinalTrimmedMarkdown,
|
||||||
|
ProducerStage: "render",
|
||||||
|
OutputKind: TranscriptOutputKindFinalTrimmedMarkdown,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
// RuntimeTranscriptArtifacts returns transcript mappings in pipeline order.
|
// RuntimeTranscriptArtifacts returns transcript mappings in pipeline order.
|
||||||
|
|||||||
60
internal/artifactmodel/transcripts_test.go
Normal file
60
internal/artifactmodel/transcripts_test.go
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
package artifactmodel
|
||||||
|
|
||||||
|
import (
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRuntimeTranscriptArtifactsIncludesMarkdownOutputs(t *testing.T) {
|
||||||
|
want := []TranscriptArtifactSpec{
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptBase,
|
||||||
|
CanonicalRelPath: TranscriptPathBase,
|
||||||
|
ProducerStage: "merge",
|
||||||
|
OutputKind: TranscriptOutputKindBase,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptPolished,
|
||||||
|
CanonicalRelPath: TranscriptPathPolished,
|
||||||
|
ProducerStage: "polish",
|
||||||
|
OutputKind: TranscriptOutputKindPolished,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptFinal,
|
||||||
|
CanonicalRelPath: TranscriptPathFinal,
|
||||||
|
ProducerStage: "normalize",
|
||||||
|
OutputKind: TranscriptOutputKindFinal,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptFinalTrimmed,
|
||||||
|
CanonicalRelPath: TranscriptPathFinalTrimmed,
|
||||||
|
ProducerStage: "trim",
|
||||||
|
OutputKind: TranscriptOutputKindFinalTrimmed,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptFinalMarkdown,
|
||||||
|
CanonicalRelPath: TranscriptPathFinalMarkdown,
|
||||||
|
ProducerStage: "render",
|
||||||
|
OutputKind: TranscriptOutputKindFinalMarkdown,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceID: SourceTranscriptFinalTrimmedMarkdown,
|
||||||
|
CanonicalRelPath: TranscriptPathFinalTrimmedMarkdown,
|
||||||
|
ProducerStage: "render",
|
||||||
|
OutputKind: TranscriptOutputKindFinalTrimmedMarkdown,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
got := RuntimeTranscriptArtifacts()
|
||||||
|
if !reflect.DeepEqual(got, want) {
|
||||||
|
t.Fatalf("RuntimeTranscriptArtifacts() = %#v, want %#v", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLookupRuntimeTranscriptArtifactFindsMarkdownOutputs(t *testing.T) {
|
||||||
|
for _, source := range []string{SourceTranscriptFinalMarkdown, SourceTranscriptFinalTrimmedMarkdown} {
|
||||||
|
if _, ok := LookupRuntimeTranscriptArtifact(source); !ok {
|
||||||
|
t.Fatalf("LookupRuntimeTranscriptArtifact(%q) ok=false, want true", source)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
237
internal/artifactpolicy/policy.go
Normal file
237
internal/artifactpolicy/policy.go
Normal file
@@ -0,0 +1,237 @@
|
|||||||
|
package artifactpolicy
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifactmodel"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/pathsafe"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
SourceBoundsSession = "narratio.bounds.session"
|
||||||
|
|
||||||
|
configuredSourcePrefix = "narratio.artifact."
|
||||||
|
previousConfiguredSrcPrefix = "narratio.previous_session.artifact."
|
||||||
|
)
|
||||||
|
|
||||||
|
var configuredSourceRE = regexp.MustCompile(`^narratio\.artifact\.([a-z][a-z0-9_]*)$`)
|
||||||
|
var previousSourceRE = regexp.MustCompile(`^narratio\.previous_session\.artifact\.([a-z][a-z0-9_]*)$`)
|
||||||
|
|
||||||
|
var (
|
||||||
|
ErrUnsupportedScriptoriumInputSource = errors.New("unsupported scriptorium input source")
|
||||||
|
ErrInvalidPreviousSessionSource = errors.New("invalid previous-session source format")
|
||||||
|
)
|
||||||
|
|
||||||
|
type SourceKind string
|
||||||
|
|
||||||
|
const (
|
||||||
|
SourceKindBuiltIn SourceKind = "built_in"
|
||||||
|
SourceKindConfiguredArtifact SourceKind = "configured_artifact"
|
||||||
|
SourceKindPreviousArtifact SourceKind = "previous_session_configured_artifact"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Source describes one normalized artifact source identifier.
|
||||||
|
type Source struct {
|
||||||
|
ID string
|
||||||
|
Kind SourceKind
|
||||||
|
ConfiguredKey string
|
||||||
|
}
|
||||||
|
|
||||||
|
// ScriptoriumInputSourceDescriptor describes one validated Scriptorium input source.
|
||||||
|
type ScriptoriumInputSourceDescriptor struct {
|
||||||
|
Source Source
|
||||||
|
PreviousSession *PreviousSessionSourceDescriptor
|
||||||
|
}
|
||||||
|
|
||||||
|
// PreviousSessionSourceDescriptor describes one canonical previous-session input source.
|
||||||
|
type PreviousSessionSourceDescriptor struct {
|
||||||
|
SourceID string
|
||||||
|
ConfiguredKey string
|
||||||
|
ConfiguredSourceID string
|
||||||
|
}
|
||||||
|
|
||||||
|
// UnknownConfiguredArtifactError reports a source that references an undefined configured artifact key.
|
||||||
|
type UnknownConfiguredArtifactError struct {
|
||||||
|
ConfiguredKey string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *UnknownConfiguredArtifactError) Error() string {
|
||||||
|
return fmt.Sprintf("references unknown artifact %q", e.ConfiguredKey)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ConfiguredSourceID converts a configured artifact key into source id form.
|
||||||
|
func ConfiguredSourceID(key string) string {
|
||||||
|
return configuredSourcePrefix + strings.TrimSpace(key)
|
||||||
|
}
|
||||||
|
|
||||||
|
// PreviousSessionSourceID converts a configured artifact key into previous-session source id form.
|
||||||
|
func PreviousSessionSourceID(key string) string {
|
||||||
|
return previousConfiguredSrcPrefix + strings.TrimSpace(key)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParseConfiguredSource extracts configured key from narratio.artifact.<key>.
|
||||||
|
func ParseConfiguredSource(source string) (string, bool) {
|
||||||
|
matches := configuredSourceRE.FindStringSubmatch(strings.TrimSpace(source))
|
||||||
|
if len(matches) != 2 {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return matches[1], true
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParsePreviousSessionSource extracts configured key from narratio.previous_session.artifact.<key>.
|
||||||
|
func ParsePreviousSessionSource(source string) (string, bool) {
|
||||||
|
matches := previousSourceRE.FindStringSubmatch(strings.TrimSpace(source))
|
||||||
|
if len(matches) != 2 {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return matches[1], true
|
||||||
|
}
|
||||||
|
|
||||||
|
// ClassifySource classifies a source id as built-in, configured, or previous-session configured.
|
||||||
|
func ClassifySource(source string) (Source, error) {
|
||||||
|
trimmed := strings.TrimSpace(source)
|
||||||
|
if trimmed == "" {
|
||||||
|
return Source{}, fmt.Errorf("artifact source is required")
|
||||||
|
}
|
||||||
|
if _, ok := artifactmodel.LookupRuntimeTranscriptArtifact(trimmed); ok {
|
||||||
|
return Source{ID: trimmed, Kind: SourceKindBuiltIn}, nil
|
||||||
|
}
|
||||||
|
if trimmed == SourceBoundsSession {
|
||||||
|
return Source{ID: trimmed, Kind: SourceKindBuiltIn}, nil
|
||||||
|
}
|
||||||
|
if key, ok := ParseConfiguredSource(trimmed); ok {
|
||||||
|
return Source{ID: trimmed, Kind: SourceKindConfiguredArtifact, ConfiguredKey: key}, nil
|
||||||
|
}
|
||||||
|
if key, ok := ParsePreviousSessionSource(trimmed); ok {
|
||||||
|
return Source{ID: trimmed, Kind: SourceKindPreviousArtifact, ConfiguredKey: key}, nil
|
||||||
|
}
|
||||||
|
return Source{}, fmt.Errorf("unsupported artifact source %q", source)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DescribeScriptoriumInputSource classifies one input source and returns descriptor
|
||||||
|
// metadata used by config validation, analyze input resolution, and previous-cache planning.
|
||||||
|
func DescribeScriptoriumInputSource(source string) (ScriptoriumInputSourceDescriptor, error) {
|
||||||
|
trimmed := strings.TrimSpace(source)
|
||||||
|
if trimmed == "" {
|
||||||
|
return ScriptoriumInputSourceDescriptor{}, ErrUnsupportedScriptoriumInputSource
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(trimmed, "narratio.previous_session.artifact") {
|
||||||
|
descriptor, err := DescribePreviousSessionSource(trimmed)
|
||||||
|
if err != nil {
|
||||||
|
return ScriptoriumInputSourceDescriptor{}, err
|
||||||
|
}
|
||||||
|
return ScriptoriumInputSourceDescriptor{
|
||||||
|
Source: Source{
|
||||||
|
ID: descriptor.SourceID,
|
||||||
|
Kind: SourceKindPreviousArtifact,
|
||||||
|
ConfiguredKey: descriptor.ConfiguredKey,
|
||||||
|
},
|
||||||
|
PreviousSession: &descriptor,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
classified, err := ClassifySource(trimmed)
|
||||||
|
if err != nil {
|
||||||
|
return ScriptoriumInputSourceDescriptor{}, ErrUnsupportedScriptoriumInputSource
|
||||||
|
}
|
||||||
|
return ScriptoriumInputSourceDescriptor{Source: classified}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DescribePreviousSessionSource validates a canonical previous-session source id
|
||||||
|
// and returns both previous and configured-source vocabulary descriptors.
|
||||||
|
func DescribePreviousSessionSource(source string) (PreviousSessionSourceDescriptor, error) {
|
||||||
|
configuredKey, ok := ParsePreviousSessionSource(source)
|
||||||
|
if !ok {
|
||||||
|
return PreviousSessionSourceDescriptor{}, ErrInvalidPreviousSessionSource
|
||||||
|
}
|
||||||
|
return PreviousSessionSourceDescriptor{
|
||||||
|
SourceID: PreviousSessionSourceID(configuredKey),
|
||||||
|
ConfiguredKey: configuredKey,
|
||||||
|
ConfiguredSourceID: ConfiguredSourceID(configuredKey),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// PreviousSessionSourceDescriptorForConfiguredKey derives a previous-session source descriptor
|
||||||
|
// from a configured artifact key.
|
||||||
|
func PreviousSessionSourceDescriptorForConfiguredKey(configuredKey string) (PreviousSessionSourceDescriptor, error) {
|
||||||
|
return DescribePreviousSessionSource(PreviousSessionSourceID(configuredKey))
|
||||||
|
}
|
||||||
|
|
||||||
|
// ValidateInputConfiguredReference checks that configured/previous-session sources
|
||||||
|
// reference configured artifacts known to the current Scriptorium config.
|
||||||
|
func ValidateInputConfiguredReference(
|
||||||
|
descriptor ScriptoriumInputSourceDescriptor,
|
||||||
|
configured map[string]struct{},
|
||||||
|
) error {
|
||||||
|
switch descriptor.Source.Kind {
|
||||||
|
case SourceKindConfiguredArtifact, SourceKindPreviousArtifact:
|
||||||
|
if _, ok := configured[descriptor.Source.ConfiguredKey]; !ok {
|
||||||
|
return &UnknownConfiguredArtifactError{ConfiguredKey: descriptor.Source.ConfiguredKey}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ValidatePublishSource validates that a source is publish-compatible and references a known configured artifact.
|
||||||
|
func ValidatePublishSource(source string, configured map[string]string) (Source, error) {
|
||||||
|
classified, err := ClassifySource(source)
|
||||||
|
if err != nil {
|
||||||
|
return Source{}, fmt.Errorf("must be a built-in source id or narratio.artifact.<name>")
|
||||||
|
}
|
||||||
|
if classified.Kind == SourceKindPreviousArtifact {
|
||||||
|
return Source{}, fmt.Errorf("must be a built-in source id or narratio.artifact.<name>")
|
||||||
|
}
|
||||||
|
if classified.Kind == SourceKindConfiguredArtifact {
|
||||||
|
if configured == nil {
|
||||||
|
return Source{}, fmt.Errorf("configured artifact %q is not defined in pipeline.scriptorium.artifacts", classified.ConfiguredKey)
|
||||||
|
}
|
||||||
|
if _, ok := configured[classified.ConfiguredKey]; !ok {
|
||||||
|
return Source{}, fmt.Errorf("configured artifact %q is not defined in pipeline.scriptorium.artifacts", classified.ConfiguredKey)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return classified, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeriveDefaultPublishedDestination returns the default publish destination for one source.
|
||||||
|
func DeriveDefaultPublishedDestination(source Source, configured map[string]string) (string, error) {
|
||||||
|
switch source.Kind {
|
||||||
|
case SourceKindBuiltIn:
|
||||||
|
if spec, ok := artifactmodel.LookupRuntimeTranscriptArtifact(source.ID); ok {
|
||||||
|
return pathsafe.NormalizeRelativeDestination(spec.CanonicalRelPath)
|
||||||
|
}
|
||||||
|
if source.ID == SourceBoundsSession {
|
||||||
|
return pathsafe.NormalizeRelativeDestination("artifacts/session_bounds.json")
|
||||||
|
}
|
||||||
|
return "", fmt.Errorf("unsupported built-in source %q", source.ID)
|
||||||
|
case SourceKindConfiguredArtifact:
|
||||||
|
if configured == nil {
|
||||||
|
return "", fmt.Errorf("configured artifact %q is not defined in pipeline.scriptorium.artifacts", source.ConfiguredKey)
|
||||||
|
}
|
||||||
|
outputPath, ok := configured[source.ConfiguredKey]
|
||||||
|
if !ok {
|
||||||
|
return "", fmt.Errorf("configured artifact %q is not defined in pipeline.scriptorium.artifacts", source.ConfiguredKey)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(outputPath) == "" {
|
||||||
|
return "", fmt.Errorf("pipeline.scriptorium.artifacts.%s.output_path is empty", source.ConfiguredKey)
|
||||||
|
}
|
||||||
|
return pathsafe.NormalizeRelativeDestination(outputPath)
|
||||||
|
default:
|
||||||
|
return "", fmt.Errorf("publish destination cannot be derived from source %q", source.ID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResolvePublishedDestination validates and normalizes an explicit destination,
|
||||||
|
// or derives one when omitted.
|
||||||
|
func ResolvePublishedDestination(sourceID, explicitDest string, configured map[string]string) (string, error) {
|
||||||
|
source, err := ValidatePublishSource(sourceID, configured)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(explicitDest) != "" {
|
||||||
|
return pathsafe.NormalizeRelativeDestination(explicitDest)
|
||||||
|
}
|
||||||
|
return DeriveDefaultPublishedDestination(source, configured)
|
||||||
|
}
|
||||||
200
internal/artifactpolicy/policy_test.go
Normal file
200
internal/artifactpolicy/policy_test.go
Normal file
@@ -0,0 +1,200 @@
|
|||||||
|
package artifactpolicy
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestClassifySource(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
source string
|
||||||
|
wantKind SourceKind
|
||||||
|
wantKey string
|
||||||
|
wantErrLike string
|
||||||
|
}{
|
||||||
|
{name: "built in transcript", source: "narratio.transcript.final_trimmed", wantKind: SourceKindBuiltIn},
|
||||||
|
{name: "built in bounds", source: "narratio.bounds.session", wantKind: SourceKindBuiltIn},
|
||||||
|
{name: "configured artifact", source: "narratio.artifact.session_recap", wantKind: SourceKindConfiguredArtifact, wantKey: "session_recap"},
|
||||||
|
{name: "previous session configured", source: "narratio.previous_session.artifact.session_recap", wantKind: SourceKindPreviousArtifact, wantKey: "session_recap"},
|
||||||
|
{name: "unsupported", source: "narratio.unknown", wantErrLike: "unsupported artifact source"},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
got, err := ClassifySource(tt.source)
|
||||||
|
if tt.wantErrLike != "" {
|
||||||
|
if err == nil || !strings.Contains(err.Error(), tt.wantErrLike) {
|
||||||
|
t.Fatalf("ClassifySource() error = %v, want like %q", err, tt.wantErrLike)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ClassifySource() error = %v", err)
|
||||||
|
}
|
||||||
|
if got.Kind != tt.wantKind {
|
||||||
|
t.Fatalf("ClassifySource().Kind = %q, want %q", got.Kind, tt.wantKind)
|
||||||
|
}
|
||||||
|
if got.ConfiguredKey != tt.wantKey {
|
||||||
|
t.Fatalf("ClassifySource().ConfiguredKey = %q, want %q", got.ConfiguredKey, tt.wantKey)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestValidatePublishSource(t *testing.T) {
|
||||||
|
configured := map[string]string{"session_recap": "artifacts/session_recap.md"}
|
||||||
|
if _, err := ValidatePublishSource("narratio.artifact.session_recap", configured); err != nil {
|
||||||
|
t.Fatalf("ValidatePublishSource(configured) error = %v", err)
|
||||||
|
}
|
||||||
|
if _, err := ValidatePublishSource("narratio.previous_session.artifact.session_recap", configured); err == nil {
|
||||||
|
t.Fatal("ValidatePublishSource(previous) error = nil, want error")
|
||||||
|
}
|
||||||
|
if _, err := ValidatePublishSource("narratio.artifact.missing", configured); err == nil {
|
||||||
|
t.Fatal("ValidatePublishSource(missing configured) error = nil, want error")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolvePublishedDestination(t *testing.T) {
|
||||||
|
configured := map[string]string{"session_recap": "artifacts/session_recap.md"}
|
||||||
|
|
||||||
|
got, err := ResolvePublishedDestination("narratio.transcript.final_trimmed", "", configured)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ResolvePublishedDestination(built-in) error = %v", err)
|
||||||
|
}
|
||||||
|
if got != "transcripts/final.trimmed.json" {
|
||||||
|
t.Fatalf("built-in destination = %q, want transcripts/final.trimmed.json", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
got, err = ResolvePublishedDestination("narratio.transcript.final_markdown", "", configured)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ResolvePublishedDestination(markdown built-in) error = %v", err)
|
||||||
|
}
|
||||||
|
if got != "transcripts/final.md" {
|
||||||
|
t.Fatalf("markdown built-in destination = %q, want transcripts/final.md", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
got, err = ResolvePublishedDestination("narratio.artifact.session_recap", "", configured)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ResolvePublishedDestination(configured) error = %v", err)
|
||||||
|
}
|
||||||
|
if got != "artifacts/session_recap.md" {
|
||||||
|
t.Fatalf("configured destination = %q, want artifacts/session_recap.md", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
got, err = ResolvePublishedDestination("narratio.transcript.final_trimmed", "published/../published/final.json", configured)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ResolvePublishedDestination(explicit) error = %v", err)
|
||||||
|
}
|
||||||
|
if got != "published/final.json" {
|
||||||
|
t.Fatalf("explicit destination = %q, want published/final.json", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolvePublishedDestinationRejectsTraversal(t *testing.T) {
|
||||||
|
configured := map[string]string{"session_recap": "artifacts/session_recap.md"}
|
||||||
|
_, err := ResolvePublishedDestination("narratio.transcript.final_trimmed", "../escape.txt", configured)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("ResolvePublishedDestination() error = nil, want traversal rejection")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDescribeScriptoriumInputSource(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
source string
|
||||||
|
wantKind SourceKind
|
||||||
|
wantKey string
|
||||||
|
wantPrev bool
|
||||||
|
wantErr error
|
||||||
|
wantErrLike string
|
||||||
|
}{
|
||||||
|
{name: "built in", source: "narratio.transcript.final_trimmed", wantKind: SourceKindBuiltIn},
|
||||||
|
{name: "built in markdown", source: "narratio.transcript.final_markdown", wantKind: SourceKindBuiltIn},
|
||||||
|
{name: "configured", source: "narratio.artifact.session_recap", wantKind: SourceKindConfiguredArtifact, wantKey: "session_recap"},
|
||||||
|
{name: "previous", source: "narratio.previous_session.artifact.session_recap", wantKind: SourceKindPreviousArtifact, wantKey: "session_recap", wantPrev: true},
|
||||||
|
{name: "invalid previous", source: "narratio.previous_session.artifact.", wantErr: ErrInvalidPreviousSessionSource},
|
||||||
|
{name: "unsupported", source: "narratio.unknown", wantErr: ErrUnsupportedScriptoriumInputSource},
|
||||||
|
{name: "empty", source: " ", wantErr: ErrUnsupportedScriptoriumInputSource},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
got, err := DescribeScriptoriumInputSource(tt.source)
|
||||||
|
if tt.wantErr != nil {
|
||||||
|
if !errors.Is(err, tt.wantErr) {
|
||||||
|
t.Fatalf("DescribeScriptoriumInputSource() error = %v, want %v", err, tt.wantErr)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if tt.wantErrLike != "" {
|
||||||
|
if err == nil || !strings.Contains(err.Error(), tt.wantErrLike) {
|
||||||
|
t.Fatalf("DescribeScriptoriumInputSource() error = %v, want like %q", err, tt.wantErrLike)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("DescribeScriptoriumInputSource() error = %v", err)
|
||||||
|
}
|
||||||
|
if got.Source.Kind != tt.wantKind {
|
||||||
|
t.Fatalf("DescribeScriptoriumInputSource().Source.Kind = %q, want %q", got.Source.Kind, tt.wantKind)
|
||||||
|
}
|
||||||
|
if got.Source.ConfiguredKey != tt.wantKey {
|
||||||
|
t.Fatalf("DescribeScriptoriumInputSource().Source.ConfiguredKey = %q, want %q", got.Source.ConfiguredKey, tt.wantKey)
|
||||||
|
}
|
||||||
|
if tt.wantPrev && got.PreviousSession == nil {
|
||||||
|
t.Fatal("DescribeScriptoriumInputSource().PreviousSession = nil, want descriptor")
|
||||||
|
}
|
||||||
|
if !tt.wantPrev && got.PreviousSession != nil {
|
||||||
|
t.Fatalf("DescribeScriptoriumInputSource().PreviousSession = %#v, want nil", got.PreviousSession)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestValidateInputConfiguredReference(t *testing.T) {
|
||||||
|
configured := map[string]struct{}{"session_recap": {}}
|
||||||
|
|
||||||
|
desc, err := DescribeScriptoriumInputSource("narratio.artifact.session_recap")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("DescribeScriptoriumInputSource(configured) error = %v", err)
|
||||||
|
}
|
||||||
|
if err := ValidateInputConfiguredReference(desc, configured); err != nil {
|
||||||
|
t.Fatalf("ValidateInputConfiguredReference(configured) error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
prevDesc, err := DescribeScriptoriumInputSource("narratio.previous_session.artifact.session_recap")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("DescribeScriptoriumInputSource(previous) error = %v", err)
|
||||||
|
}
|
||||||
|
if err := ValidateInputConfiguredReference(prevDesc, configured); err != nil {
|
||||||
|
t.Fatalf("ValidateInputConfiguredReference(previous) error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
missingDesc, err := DescribeScriptoriumInputSource("narratio.artifact.quest_log")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("DescribeScriptoriumInputSource(missing configured) error = %v", err)
|
||||||
|
}
|
||||||
|
err = ValidateInputConfiguredReference(missingDesc, configured)
|
||||||
|
var unknown *UnknownConfiguredArtifactError
|
||||||
|
if !errors.As(err, &unknown) || unknown.ConfiguredKey != "quest_log" {
|
||||||
|
t.Fatalf("ValidateInputConfiguredReference(missing configured) error = %v, want UnknownConfiguredArtifactError(quest_log)", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPreviousSessionSourceDescriptorForConfiguredKey(t *testing.T) {
|
||||||
|
got, err := PreviousSessionSourceDescriptorForConfiguredKey("session_recap")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("PreviousSessionSourceDescriptorForConfiguredKey() error = %v", err)
|
||||||
|
}
|
||||||
|
if got.SourceID != "narratio.previous_session.artifact.session_recap" {
|
||||||
|
t.Fatalf("SourceID = %q, want narratio.previous_session.artifact.session_recap", got.SourceID)
|
||||||
|
}
|
||||||
|
if got.ConfiguredSourceID != "narratio.artifact.session_recap" {
|
||||||
|
t.Fatalf("ConfiguredSourceID = %q, want narratio.artifact.session_recap", got.ConfiguredSourceID)
|
||||||
|
}
|
||||||
|
if got.ConfiguredKey != "session_recap" {
|
||||||
|
t.Fatalf("ConfiguredKey = %q, want session_recap", got.ConfiguredKey)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -6,42 +6,46 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"regexp"
|
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifactmodel"
|
"gitea.maximumdirect.net/eric/narratio/internal/artifactmodel"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifactpolicy"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
ArtifactTranscriptBase = artifactmodel.SourceTranscriptBase
|
ArtifactTranscriptBase = artifactmodel.SourceTranscriptBase
|
||||||
ArtifactTranscriptPolished = artifactmodel.SourceTranscriptPolished
|
ArtifactTranscriptPolished = artifactmodel.SourceTranscriptPolished
|
||||||
ArtifactTranscriptFinal = artifactmodel.SourceTranscriptFinal
|
ArtifactTranscriptFinal = artifactmodel.SourceTranscriptFinal
|
||||||
ArtifactTranscriptFinalTrimmed = artifactmodel.SourceTranscriptFinalTrimmed
|
ArtifactTranscriptFinalTrimmed = artifactmodel.SourceTranscriptFinalTrimmed
|
||||||
ArtifactBoundsSession = "narratio.bounds.session"
|
ArtifactTranscriptFinalMarkdown = artifactmodel.SourceTranscriptFinalMarkdown
|
||||||
|
ArtifactTranscriptFinalTrimmedMarkdown = artifactmodel.SourceTranscriptFinalTrimmedMarkdown
|
||||||
|
ArtifactBoundsSession = "narratio.bounds.session"
|
||||||
|
|
||||||
ArtifactProvenancePreviousCacheManifestInput = "manifest.inputs.previous_cache"
|
ArtifactProvenancePreviousCacheManifestInput = "manifest.inputs.previous_cache"
|
||||||
ArtifactProvenancePreviousCacheFilesystem = "current_session.previous_cache"
|
ArtifactProvenancePreviousCacheFilesystem = "current_session.previous_cache"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
TranscriptPathBase = artifactmodel.TranscriptPathBase
|
TranscriptPathBase = artifactmodel.TranscriptPathBase
|
||||||
TranscriptPathPolished = artifactmodel.TranscriptPathPolished
|
TranscriptPathPolished = artifactmodel.TranscriptPathPolished
|
||||||
TranscriptPathFinal = artifactmodel.TranscriptPathFinal
|
TranscriptPathFinal = artifactmodel.TranscriptPathFinal
|
||||||
TranscriptPathFinalTrimmed = artifactmodel.TranscriptPathFinalTrimmed
|
TranscriptPathFinalTrimmed = artifactmodel.TranscriptPathFinalTrimmed
|
||||||
|
TranscriptPathFinalMarkdown = artifactmodel.TranscriptPathFinalMarkdown
|
||||||
|
TranscriptPathFinalTrimmedMarkdown = artifactmodel.TranscriptPathFinalTrimmedMarkdown
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
TranscriptOutputKindBase = artifactmodel.TranscriptOutputKindBase
|
TranscriptOutputKindBase = artifactmodel.TranscriptOutputKindBase
|
||||||
TranscriptOutputKindPolished = artifactmodel.TranscriptOutputKindPolished
|
TranscriptOutputKindPolished = artifactmodel.TranscriptOutputKindPolished
|
||||||
TranscriptOutputKindFinal = artifactmodel.TranscriptOutputKindFinal
|
TranscriptOutputKindFinal = artifactmodel.TranscriptOutputKindFinal
|
||||||
TranscriptOutputKindFinalTrimmed = artifactmodel.TranscriptOutputKindFinalTrimmed
|
TranscriptOutputKindFinalTrimmed = artifactmodel.TranscriptOutputKindFinalTrimmed
|
||||||
|
TranscriptOutputKindFinalMarkdown = artifactmodel.TranscriptOutputKindFinalMarkdown
|
||||||
|
TranscriptOutputKindFinalTrimmedMarkdown = artifactmodel.TranscriptOutputKindFinalTrimmedMarkdown
|
||||||
)
|
)
|
||||||
|
|
||||||
// ErrSessionArtifactNotFound is returned when no readable artifact exists for a known ID.
|
// ErrSessionArtifactNotFound is returned when no readable artifact exists for a known ID.
|
||||||
var ErrSessionArtifactNotFound = errors.New("session artifact not found")
|
var ErrSessionArtifactNotFound = errors.New("session artifact not found")
|
||||||
var configuredArtifactSourceRE = regexp.MustCompile(`^narratio\.artifact\.([a-z][a-z0-9_]*)$`)
|
|
||||||
var previousSessionArtifactSourceRE = regexp.MustCompile(`^narratio\.previous_session\.artifact\.([a-z][a-z0-9_]*)$`)
|
|
||||||
|
|
||||||
type artifactContentKind string
|
type artifactContentKind string
|
||||||
|
|
||||||
@@ -69,7 +73,7 @@ func buildArtifactRegistry() map[string]artifactSpec {
|
|||||||
CanonicalRelPath: transcript.CanonicalRelPath,
|
CanonicalRelPath: transcript.CanonicalRelPath,
|
||||||
ProducerStage: transcript.ProducerStage,
|
ProducerStage: transcript.ProducerStage,
|
||||||
OutputKind: transcript.OutputKind,
|
OutputKind: transcript.OutputKind,
|
||||||
ContentKind: contentTranscriptJSON,
|
ContentKind: transcriptContentKind(transcript),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
registry[ArtifactBoundsSession] = artifactSpec{
|
registry[ArtifactBoundsSession] = artifactSpec{
|
||||||
@@ -82,6 +86,15 @@ func buildArtifactRegistry() map[string]artifactSpec {
|
|||||||
return registry
|
return registry
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func transcriptContentKind(transcript TranscriptArtifactSpec) artifactContentKind {
|
||||||
|
switch transcript.SourceID {
|
||||||
|
case ArtifactTranscriptFinalMarkdown, ArtifactTranscriptFinalTrimmedMarkdown:
|
||||||
|
return contentText
|
||||||
|
default:
|
||||||
|
return contentTranscriptJSON
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ResolvedSessionArtifact describes one session-level artifact lookup result.
|
// ResolvedSessionArtifact describes one session-level artifact lookup result.
|
||||||
type ResolvedSessionArtifact struct {
|
type ResolvedSessionArtifact struct {
|
||||||
ID string
|
ID string
|
||||||
@@ -107,43 +120,39 @@ func (e *SessionArtifactNotFoundError) Unwrap() error {
|
|||||||
|
|
||||||
// NormalizeSessionArtifactSource validates canonical artifact IDs.
|
// NormalizeSessionArtifactSource validates canonical artifact IDs.
|
||||||
func NormalizeSessionArtifactSource(source string) (string, error) {
|
func NormalizeSessionArtifactSource(source string) (string, error) {
|
||||||
normalized := strings.TrimSpace(source)
|
classified, err := artifactpolicy.ClassifySource(source)
|
||||||
if normalized == "" {
|
if err != nil {
|
||||||
return "", fmt.Errorf("artifact source is required")
|
|
||||||
}
|
|
||||||
if _, ok := artifactRegistry[normalized]; !ok {
|
|
||||||
return "", fmt.Errorf("unsupported artifact source %q", source)
|
return "", fmt.Errorf("unsupported artifact source %q", source)
|
||||||
}
|
}
|
||||||
return normalized, nil
|
if classified.Kind != artifactpolicy.SourceKindBuiltIn {
|
||||||
|
return "", fmt.Errorf("unsupported artifact source %q", source)
|
||||||
|
}
|
||||||
|
if _, ok := artifactRegistry[classified.ID]; !ok {
|
||||||
|
return "", fmt.Errorf("unsupported artifact source %q", source)
|
||||||
|
}
|
||||||
|
return classified.ID, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// IsConfiguredArtifactSource returns true when source is narratio.artifact.<name>.
|
// IsConfiguredArtifactSource returns true when source is narratio.artifact.<name>.
|
||||||
func IsConfiguredArtifactSource(source string) bool {
|
func IsConfiguredArtifactSource(source string) bool {
|
||||||
return configuredArtifactSourceRE.MatchString(strings.TrimSpace(source))
|
_, ok := artifactpolicy.ParseConfiguredSource(source)
|
||||||
|
return ok
|
||||||
}
|
}
|
||||||
|
|
||||||
// ConfiguredArtifactName extracts <name> from narratio.artifact.<name>.
|
// ConfiguredArtifactName extracts <name> from narratio.artifact.<name>.
|
||||||
func ConfiguredArtifactName(source string) (string, bool) {
|
func ConfiguredArtifactName(source string) (string, bool) {
|
||||||
matches := configuredArtifactSourceRE.FindStringSubmatch(strings.TrimSpace(source))
|
return artifactpolicy.ParseConfiguredSource(source)
|
||||||
if len(matches) != 2 {
|
|
||||||
return "", false
|
|
||||||
}
|
|
||||||
return matches[1], true
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// IsPreviousSessionArtifactSource returns true when source is narratio.previous_session.artifact.<name>.
|
// IsPreviousSessionArtifactSource returns true when source is narratio.previous_session.artifact.<name>.
|
||||||
func IsPreviousSessionArtifactSource(source string) bool {
|
func IsPreviousSessionArtifactSource(source string) bool {
|
||||||
_, ok := PreviousSessionArtifactName(source)
|
_, ok := artifactpolicy.ParsePreviousSessionSource(source)
|
||||||
return ok
|
return ok
|
||||||
}
|
}
|
||||||
|
|
||||||
// PreviousSessionArtifactName extracts <name> from narratio.previous_session.artifact.<name>.
|
// PreviousSessionArtifactName extracts <name> from narratio.previous_session.artifact.<name>.
|
||||||
func PreviousSessionArtifactName(source string) (string, bool) {
|
func PreviousSessionArtifactName(source string) (string, bool) {
|
||||||
matches := previousSessionArtifactSourceRE.FindStringSubmatch(strings.TrimSpace(source))
|
return artifactpolicy.ParsePreviousSessionSource(source)
|
||||||
if len(matches) != 2 {
|
|
||||||
return "", false
|
|
||||||
}
|
|
||||||
return matches[1], true
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// ResolveSessionArtifact resolves a symbolic source to a readable local session artifact path.
|
// ResolveSessionArtifact resolves a symbolic source to a readable local session artifact path.
|
||||||
|
|||||||
@@ -211,6 +211,29 @@ func TestResolveSessionArtifactFallsBackToCanonicalPath(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestResolveSessionArtifactFallsBackToCanonicalMarkdownPath(t *testing.T) {
|
||||||
|
workspace := t.TempDir()
|
||||||
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
|
canonicalPath := filepath.Join(paths.TranscriptsDir, "final.md")
|
||||||
|
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
||||||
|
t.Fatalf("MkdirAll() error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(canonicalPath, []byte("# Final transcript\n"), 0o644); err != nil {
|
||||||
|
t.Fatalf("WriteFile() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
resolved, err := ResolveSessionArtifact(paths, nil, ArtifactTranscriptFinalMarkdown)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ResolveSessionArtifact() error = %v", err)
|
||||||
|
}
|
||||||
|
if resolved.Path != canonicalPath {
|
||||||
|
t.Fatalf("resolved path = %q, want %q", resolved.Path, canonicalPath)
|
||||||
|
}
|
||||||
|
if resolved.Provenance != "fallback.canonical_path" {
|
||||||
|
t.Fatalf("provenance = %q, want fallback.canonical_path", resolved.Provenance)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestResolveSessionArtifactMissingReturnsTypedError(t *testing.T) {
|
func TestResolveSessionArtifactMissingReturnsTypedError(t *testing.T) {
|
||||||
workspace := t.TempDir()
|
workspace := t.TempDir()
|
||||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
@@ -244,6 +267,26 @@ func TestResolveSessionArtifactValidatesTranscriptShape(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestResolveSessionArtifactRejectsEmptyMarkdownContent(t *testing.T) {
|
||||||
|
workspace := t.TempDir()
|
||||||
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
|
canonicalPath := filepath.Join(paths.TranscriptsDir, "final.trimmed.md")
|
||||||
|
if err := os.MkdirAll(filepath.Dir(canonicalPath), 0o755); err != nil {
|
||||||
|
t.Fatalf("MkdirAll() error = %v", err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(canonicalPath, []byte{}, 0o644); err != nil {
|
||||||
|
t.Fatalf("WriteFile() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err := ResolveSessionArtifact(paths, nil, ArtifactTranscriptFinalTrimmedMarkdown)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error, got nil")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "file is empty") {
|
||||||
|
t.Fatalf("error = %q, want empty file validation", err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestResolveSessionArtifactWithCatalogBuiltInBehaviorUnchanged(t *testing.T) {
|
func TestResolveSessionArtifactWithCatalogBuiltInBehaviorUnchanged(t *testing.T) {
|
||||||
workspace := t.TempDir()
|
workspace := t.TempDir()
|
||||||
paths := buildSessionPaths(workspace, "campaign", "session")
|
paths := buildSessionPaths(workspace, "campaign", "session")
|
||||||
|
|||||||
@@ -4,6 +4,8 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"sort"
|
"sort"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifactpolicy"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -47,7 +49,7 @@ func NewArtifactCatalog() *ArtifactCatalog {
|
|||||||
|
|
||||||
// ConfiguredArtifactSourceID converts a configured artifact key into canonical source ID.
|
// ConfiguredArtifactSourceID converts a configured artifact key into canonical source ID.
|
||||||
func ConfiguredArtifactSourceID(key string) string {
|
func ConfiguredArtifactSourceID(key string) string {
|
||||||
return "narratio.artifact." + strings.TrimSpace(key)
|
return artifactpolicy.ConfiguredSourceID(key)
|
||||||
}
|
}
|
||||||
|
|
||||||
// RegisterBuiltIns registers built-in source definitions used by runtime artifact resolution.
|
// RegisterBuiltIns registers built-in source definitions used by runtime artifact resolution.
|
||||||
@@ -214,11 +216,10 @@ func (c *ArtifactCatalog) addEntry(entry CatalogEntry) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func runtimeBuiltInArtifactIDs() []string {
|
func runtimeBuiltInArtifactIDs() []string {
|
||||||
return []string{
|
ids := make([]string, 0, len(RuntimeTranscriptArtifacts())+1)
|
||||||
ArtifactTranscriptBase,
|
for _, transcript := range RuntimeTranscriptArtifacts() {
|
||||||
ArtifactTranscriptPolished,
|
ids = append(ids, transcript.SourceID)
|
||||||
ArtifactTranscriptFinal,
|
|
||||||
ArtifactTranscriptFinalTrimmed,
|
|
||||||
ArtifactBoundsSession,
|
|
||||||
}
|
}
|
||||||
|
ids = append(ids, ArtifactBoundsSession)
|
||||||
|
return ids
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -23,6 +23,26 @@ func TestArtifactCatalogRegisterBuiltInsAndLookup(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestArtifactCatalogRegisterBuiltInsIncludesMarkdownSources(t *testing.T) {
|
||||||
|
catalog := NewArtifactCatalog()
|
||||||
|
if err := catalog.RegisterBuiltIns(); err != nil {
|
||||||
|
t.Fatalf("RegisterBuiltIns() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, sourceID := range []string{
|
||||||
|
ArtifactTranscriptFinalMarkdown,
|
||||||
|
ArtifactTranscriptFinalTrimmedMarkdown,
|
||||||
|
} {
|
||||||
|
entry, ok := catalog.Lookup(sourceID)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("Lookup(%q) ok=false, want true", sourceID)
|
||||||
|
}
|
||||||
|
if !entry.Planned {
|
||||||
|
t.Fatalf("%s planned=false, want true", sourceID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestArtifactCatalogRegisterConfiguredArtifactsDefaultsToEnabled(t *testing.T) {
|
func TestArtifactCatalogRegisterConfiguredArtifactsDefaultsToEnabled(t *testing.T) {
|
||||||
catalog := NewArtifactCatalog()
|
catalog := NewArtifactCatalog()
|
||||||
if err := catalog.RegisterConfiguredArtifacts(map[string]ConfiguredArtifactDefinition{
|
if err := catalog.RegisterConfiguredArtifacts(map[string]ConfiguredArtifactDefinition{
|
||||||
|
|||||||
203
internal/artifacts/current_state.go
Normal file
203
internal/artifacts/current_state.go
Normal file
@@ -0,0 +1,203 @@
|
|||||||
|
package artifacts
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
ErrCurrentRunPointerMissing = errors.New("current run pointer missing")
|
||||||
|
ErrCurrentManifestMissing = errors.New("current manifest missing")
|
||||||
|
)
|
||||||
|
|
||||||
|
type CurrentRunPointerMissingError struct {
|
||||||
|
Key string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *CurrentRunPointerMissingError) Error() string {
|
||||||
|
return fmt.Sprintf("%s: %q", ErrCurrentRunPointerMissing, e.Key)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *CurrentRunPointerMissingError) Unwrap() error {
|
||||||
|
return ErrCurrentRunPointerMissing
|
||||||
|
}
|
||||||
|
|
||||||
|
type CurrentManifestMissingError struct {
|
||||||
|
Key string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *CurrentManifestMissingError) Error() string {
|
||||||
|
return fmt.Sprintf("%s: %q", ErrCurrentManifestMissing, e.Key)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *CurrentManifestMissingError) Unwrap() error {
|
||||||
|
return ErrCurrentManifestMissing
|
||||||
|
}
|
||||||
|
|
||||||
|
type CurrentState struct {
|
||||||
|
SessionPrefix string
|
||||||
|
CurrentRunIDKey string
|
||||||
|
CurrentManifestKey string
|
||||||
|
RunID string
|
||||||
|
Manifest *manifest.Manifest
|
||||||
|
}
|
||||||
|
|
||||||
|
type CurrentStateValidation struct {
|
||||||
|
ExpectedCampaign string
|
||||||
|
ExpectedSessionID string
|
||||||
|
ExpectedRunID string
|
||||||
|
ValidateRunID bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func LoadCurrentRunPointer(ctx context.Context, store storage.ObjectStore, currentRunIDKey string) (string, error) {
|
||||||
|
if store == nil {
|
||||||
|
return "", fmt.Errorf("object store is required")
|
||||||
|
}
|
||||||
|
key := strings.TrimSpace(currentRunIDKey)
|
||||||
|
if key == "" {
|
||||||
|
return "", fmt.Errorf("current run pointer key is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
exists, err := store.Exists(ctx, key)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("check current run pointer %q: %w", key, err)
|
||||||
|
}
|
||||||
|
if !exists {
|
||||||
|
return "", &CurrentRunPointerMissingError{Key: key}
|
||||||
|
}
|
||||||
|
|
||||||
|
localPath, err := storage.DownloadObjectToTemp(ctx, store, key, "narratio-current-run-id-*.txt")
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("download current run pointer %q: %w", key, err)
|
||||||
|
}
|
||||||
|
defer func() { _ = os.Remove(localPath) }()
|
||||||
|
|
||||||
|
data, err := os.ReadFile(localPath)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("read downloaded current run pointer %q: %w", key, err)
|
||||||
|
}
|
||||||
|
runID := strings.TrimSpace(string(data))
|
||||||
|
if runID == "" {
|
||||||
|
return "", fmt.Errorf("current run pointer %q is empty", key)
|
||||||
|
}
|
||||||
|
return runID, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func LoadCurrentManifest(ctx context.Context, store storage.ObjectStore, currentManifestKey string) (*manifest.Manifest, error) {
|
||||||
|
if store == nil {
|
||||||
|
return nil, fmt.Errorf("object store is required")
|
||||||
|
}
|
||||||
|
key := strings.TrimSpace(currentManifestKey)
|
||||||
|
if key == "" {
|
||||||
|
return nil, fmt.Errorf("current manifest key is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
exists, err := store.Exists(ctx, key)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("check current manifest %q: %w", key, err)
|
||||||
|
}
|
||||||
|
if !exists {
|
||||||
|
return nil, &CurrentManifestMissingError{Key: key}
|
||||||
|
}
|
||||||
|
|
||||||
|
localPath, err := storage.DownloadObjectToTemp(ctx, store, key, "narratio-current-manifest-*.json")
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("download current manifest %q: %w", key, err)
|
||||||
|
}
|
||||||
|
defer func() { _ = os.Remove(localPath) }()
|
||||||
|
|
||||||
|
manifestStore := &manifest.LocalStore{}
|
||||||
|
m, err := manifestStore.Load(ctx, localPath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("current manifest decode failed: %w", err)
|
||||||
|
}
|
||||||
|
return m, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func LoadCurrentState(
|
||||||
|
ctx context.Context,
|
||||||
|
store storage.ObjectStore,
|
||||||
|
sessionPrefix string,
|
||||||
|
validation CurrentStateValidation,
|
||||||
|
) (*CurrentState, error) {
|
||||||
|
prefix := strings.TrimSpace(sessionPrefix)
|
||||||
|
if prefix == "" {
|
||||||
|
return nil, fmt.Errorf("session prefix is required")
|
||||||
|
}
|
||||||
|
currentManifestKey, currentRunIDKey := ResolveCurrentStateKeys(prefix)
|
||||||
|
runID, err := LoadCurrentRunPointer(ctx, store, currentRunIDKey)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
m, err := LoadCurrentManifest(ctx, store, currentManifestKey)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
state := &CurrentState{
|
||||||
|
SessionPrefix: prefix,
|
||||||
|
CurrentRunIDKey: currentRunIDKey,
|
||||||
|
CurrentManifestKey: currentManifestKey,
|
||||||
|
RunID: runID,
|
||||||
|
Manifest: m,
|
||||||
|
}
|
||||||
|
if err := ValidateCurrentStateIdentity(state, validation); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return state, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func ValidateCurrentStateIdentity(state *CurrentState, validation CurrentStateValidation) error {
|
||||||
|
if state == nil || state.Manifest == nil {
|
||||||
|
return fmt.Errorf("current state with manifest is required")
|
||||||
|
}
|
||||||
|
expectedSessionID := strings.TrimSpace(validation.ExpectedSessionID)
|
||||||
|
expectedCampaign := strings.TrimSpace(validation.ExpectedCampaign)
|
||||||
|
expectedRunID := strings.TrimSpace(validation.ExpectedRunID)
|
||||||
|
manifestSessionID := strings.TrimSpace(state.Manifest.SessionID)
|
||||||
|
manifestCampaign := strings.TrimSpace(state.Manifest.Campaign)
|
||||||
|
manifestRunID := strings.TrimSpace(state.Manifest.RunID)
|
||||||
|
|
||||||
|
if expectedSessionID != "" && manifestSessionID != expectedSessionID {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"current manifest session_id %q does not match expected session_id %q",
|
||||||
|
manifestSessionID,
|
||||||
|
expectedSessionID,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if expectedCampaign != "" {
|
||||||
|
if manifestCampaign == "" {
|
||||||
|
return fmt.Errorf("current manifest campaign is required")
|
||||||
|
}
|
||||||
|
if manifestCampaign != expectedCampaign {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"current manifest campaign %q does not match expected campaign %q",
|
||||||
|
manifestCampaign,
|
||||||
|
expectedCampaign,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if expectedRunID == "" && validation.ValidateRunID {
|
||||||
|
expectedRunID = strings.TrimSpace(state.RunID)
|
||||||
|
}
|
||||||
|
if expectedRunID != "" {
|
||||||
|
if manifestRunID == "" {
|
||||||
|
return fmt.Errorf("current manifest run_id is required")
|
||||||
|
}
|
||||||
|
if manifestRunID != expectedRunID {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"current run pointer %q references run %q but current manifest run_id is %q",
|
||||||
|
state.CurrentRunIDKey,
|
||||||
|
expectedRunID,
|
||||||
|
manifestRunID,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
140
internal/artifacts/current_state_test.go
Normal file
140
internal/artifacts/current_state_test.go
Normal file
@@ -0,0 +1,140 @@
|
|||||||
|
package artifacts
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestLoadCurrentStateMissingRunPointer(t *testing.T) {
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("LoadCurrentState() error = nil, want missing run pointer error")
|
||||||
|
}
|
||||||
|
var missing *CurrentRunPointerMissingError
|
||||||
|
if !errors.As(err, &missing) {
|
||||||
|
t.Fatalf("errors.As(err, *CurrentRunPointerMissingError) = false; err=%v", err)
|
||||||
|
}
|
||||||
|
if !errors.Is(err, ErrCurrentRunPointerMissing) {
|
||||||
|
t.Fatalf("errors.Is(err, ErrCurrentRunPointerMissing) = false; err=%v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadCurrentStateMissingManifest(t *testing.T) {
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
_, _, runIDKey := testCurrentStateKeys()
|
||||||
|
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||||
|
|
||||||
|
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("LoadCurrentState() error = nil, want missing manifest error")
|
||||||
|
}
|
||||||
|
var missing *CurrentManifestMissingError
|
||||||
|
if !errors.As(err, &missing) {
|
||||||
|
t.Fatalf("errors.As(err, *CurrentManifestMissingError) = false; err=%v", err)
|
||||||
|
}
|
||||||
|
if !errors.Is(err, ErrCurrentManifestMissing) {
|
||||||
|
t.Fatalf("errors.Is(err, ErrCurrentManifestMissing) = false; err=%v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadCurrentStateEmptyRunPointerFails(t *testing.T) {
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
_, manifestKey, runIDKey := testCurrentStateKeys()
|
||||||
|
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte(" \n\t")})
|
||||||
|
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: testManifestJSON(t, "2026-05-03", "sample-campaign", "20260519T010203Z-a1b2c3d4")})
|
||||||
|
|
||||||
|
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "is empty") {
|
||||||
|
t.Fatalf("error = %v, want empty run pointer failure", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadCurrentStateMalformedManifestFails(t *testing.T) {
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
_, manifestKey, runIDKey := testCurrentStateKeys()
|
||||||
|
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||||
|
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: []byte("{invalid json")})
|
||||||
|
|
||||||
|
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "current manifest decode failed") {
|
||||||
|
t.Fatalf("error = %v, want manifest decode failure", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadCurrentStateCampaignMismatchFails(t *testing.T) {
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
seedCurrentState(t, store, "2026-05-03", "wrong-campaign", "20260519T010203Z-a1b2c3d4")
|
||||||
|
|
||||||
|
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{
|
||||||
|
ExpectedCampaign: "sample-campaign",
|
||||||
|
})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "does not match expected campaign") {
|
||||||
|
t.Fatalf("error = %v, want campaign mismatch failure", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadCurrentStateSessionMismatchFails(t *testing.T) {
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
seedCurrentState(t, store, "wrong-session", "sample-campaign", "20260519T010203Z-a1b2c3d4")
|
||||||
|
|
||||||
|
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{
|
||||||
|
ExpectedSessionID: "2026-05-03",
|
||||||
|
})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "does not match expected session_id") {
|
||||||
|
t.Fatalf("error = %v, want session mismatch failure", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadCurrentStateRunIDMismatchFails(t *testing.T) {
|
||||||
|
store := &storage.FakeBackend{}
|
||||||
|
seedCurrentState(t, store, "2026-05-03", "sample-campaign", "different-run-id")
|
||||||
|
|
||||||
|
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{
|
||||||
|
ValidateRunID: true,
|
||||||
|
})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "current manifest run_id") {
|
||||||
|
t.Fatalf("error = %v, want run mismatch failure", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testCurrentSessionPrefix() string {
|
||||||
|
return S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testCurrentStateKeys() (sessionPrefix, manifestKey, runIDKey string) {
|
||||||
|
sessionPrefix = testCurrentSessionPrefix()
|
||||||
|
manifestKey, runIDKey = ResolveCurrentStateKeys(sessionPrefix)
|
||||||
|
return sessionPrefix, manifestKey, runIDKey
|
||||||
|
}
|
||||||
|
|
||||||
|
func seedCurrentState(t *testing.T, store *storage.FakeBackend, sessionID, campaign, manifestRunID string) {
|
||||||
|
t.Helper()
|
||||||
|
_, manifestKey, runIDKey := testCurrentStateKeys()
|
||||||
|
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
|
||||||
|
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: testManifestJSON(t, sessionID, campaign, manifestRunID)})
|
||||||
|
}
|
||||||
|
|
||||||
|
func testManifestJSON(t *testing.T, sessionID, campaign, runID string) []byte {
|
||||||
|
t.Helper()
|
||||||
|
now := time.Date(2026, 5, 19, 23, 0, 0, 0, time.UTC).Format(time.RFC3339Nano)
|
||||||
|
payload := map[string]any{
|
||||||
|
"session_id": sessionID,
|
||||||
|
"campaign": campaign,
|
||||||
|
"run_id": runID,
|
||||||
|
"created_at": now,
|
||||||
|
"updated_at": now,
|
||||||
|
"stages": map[string]any{},
|
||||||
|
}
|
||||||
|
data, err := json.Marshal(payload)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("marshal manifest payload: %v", err)
|
||||||
|
}
|
||||||
|
return append(data, '\n')
|
||||||
|
}
|
||||||
@@ -9,6 +9,9 @@ import (
|
|||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/fileops"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/pathsafe"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ErrLockConflict is returned when a session lock already exists.
|
// ErrLockConflict is returned when a session lock already exists.
|
||||||
@@ -101,7 +104,7 @@ func (s *LocalStore) copyInputWithPaths(paths SessionPaths, sessionID, srcPath,
|
|||||||
return Ref{}, fmt.Errorf("copy input: %w", err)
|
return Ref{}, fmt.Errorf("copy input: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if err := copyFileAtomic(srcPath, destAbs, 0o644); err != nil {
|
if err := fileops.CopyFileAtomic(srcPath, destAbs, 0o644); err != nil {
|
||||||
return Ref{}, fmt.Errorf("copy input %q -> %q: %w", srcPath, destAbs, err)
|
return Ref{}, fmt.Errorf("copy input %q -> %q: %w", srcPath, destAbs, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -145,45 +148,9 @@ func (s *LocalStore) WriteFileAtomic(path string, data []byte, perm os.FileMode)
|
|||||||
if strings.TrimSpace(path) == "" {
|
if strings.TrimSpace(path) == "" {
|
||||||
return fmt.Errorf("write file atomic: path is required")
|
return fmt.Errorf("write file atomic: path is required")
|
||||||
}
|
}
|
||||||
|
if err := fileops.WriteFileAtomic(path, data, perm); err != nil {
|
||||||
dir := filepath.Dir(path)
|
return fmt.Errorf("write file atomic: %w", err)
|
||||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
|
||||||
return fmt.Errorf("write file atomic: create parent dir %q: %w", dir, err)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
base := filepath.Base(path)
|
|
||||||
tmp, err := os.CreateTemp(dir, "."+base+".tmp-*")
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("write file atomic: create temp file: %w", err)
|
|
||||||
}
|
|
||||||
tmpName := tmp.Name()
|
|
||||||
removeTmp := true
|
|
||||||
defer func() {
|
|
||||||
if removeTmp {
|
|
||||||
_ = os.Remove(tmpName)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
if _, err := tmp.Write(data); err != nil {
|
|
||||||
_ = tmp.Close()
|
|
||||||
return fmt.Errorf("write file atomic: write temp file: %w", err)
|
|
||||||
}
|
|
||||||
if err := tmp.Sync(); err != nil {
|
|
||||||
_ = tmp.Close()
|
|
||||||
return fmt.Errorf("write file atomic: sync temp file: %w", err)
|
|
||||||
}
|
|
||||||
if err := tmp.Close(); err != nil {
|
|
||||||
return fmt.Errorf("write file atomic: close temp file: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.Chmod(tmpName, perm); err != nil {
|
|
||||||
return fmt.Errorf("write file atomic: chmod temp file: %w", err)
|
|
||||||
}
|
|
||||||
if err := os.Rename(tmpName, path); err != nil {
|
|
||||||
return fmt.Errorf("write file atomic: rename temp file: %w", err)
|
|
||||||
}
|
|
||||||
removeTmp = false
|
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -256,62 +223,21 @@ func (s *LocalStore) ReleaseSessionLock(lock *LockHandle) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func resolveInRoot(root, relative string) (string, error) {
|
func resolveInRoot(root, relative string) (string, error) {
|
||||||
rel := filepath.Clean(relative)
|
joined, err := pathsafe.JoinSlashRelativeUnderRoot(root, filepath.ToSlash(relative))
|
||||||
if rel == "." || rel == "" {
|
if err != nil {
|
||||||
|
switch {
|
||||||
|
case errors.Is(err, pathsafe.ErrRelativePathRequired):
|
||||||
|
return "", fmt.Errorf("relative destination path is required")
|
||||||
|
case errors.Is(err, pathsafe.ErrRelativePathAbsolute):
|
||||||
|
return "", fmt.Errorf("relative destination must not be absolute: %q", relative)
|
||||||
|
case errors.Is(err, pathsafe.ErrRelativePathEscape):
|
||||||
|
return "", fmt.Errorf("relative destination escapes root: %q", relative)
|
||||||
|
default:
|
||||||
|
return "", fmt.Errorf("resolve destination in root: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(joined) == "" {
|
||||||
return "", fmt.Errorf("relative destination path is required")
|
return "", fmt.Errorf("relative destination path is required")
|
||||||
}
|
}
|
||||||
if filepath.IsAbs(rel) {
|
return joined, nil
|
||||||
return "", fmt.Errorf("relative destination must not be absolute: %q", relative)
|
|
||||||
}
|
|
||||||
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
|
||||||
return "", fmt.Errorf("relative destination escapes root: %q", relative)
|
|
||||||
}
|
|
||||||
return filepath.Join(root, rel), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func copyFileAtomic(srcPath, dstPath string, perm os.FileMode) error {
|
|
||||||
src, err := os.Open(srcPath)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer src.Close()
|
|
||||||
|
|
||||||
if err := os.MkdirAll(filepath.Dir(dstPath), 0o755); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
dir := filepath.Dir(dstPath)
|
|
||||||
base := filepath.Base(dstPath)
|
|
||||||
tmp, err := os.CreateTemp(dir, "."+base+".tmp-*")
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
tmpName := tmp.Name()
|
|
||||||
removeTmp := true
|
|
||||||
defer func() {
|
|
||||||
if removeTmp {
|
|
||||||
_ = os.Remove(tmpName)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
if _, err := io.Copy(tmp, src); err != nil {
|
|
||||||
_ = tmp.Close()
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := tmp.Sync(); err != nil {
|
|
||||||
_ = tmp.Close()
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := tmp.Close(); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := os.Chmod(tmpName, perm); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := os.Rename(tmpName, dstPath); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
removeTmp = false
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -75,9 +75,9 @@ func TestSessionPreviousPathsForCampaign(t *testing.T) {
|
|||||||
t.Fatalf("SessionPreviousArtifactPathForCampaign() = %q, want %q", artifactPath, wantArtifactPath)
|
t.Fatalf("SessionPreviousArtifactPathForCampaign() = %q, want %q", artifactPath, wantArtifactPath)
|
||||||
}
|
}
|
||||||
|
|
||||||
archiveRelativeArtifactPath := SessionPreviousArtifactPathForCampaign(root, "forsaken", "2026-04-19", "artifacts/session_recap.md")
|
previousRelativeArtifactPath := SessionPreviousArtifactPathForCampaign(root, "forsaken", "2026-04-19", "artifacts/session_recap.md")
|
||||||
if archiveRelativeArtifactPath != wantArtifactPath {
|
if previousRelativeArtifactPath != wantArtifactPath {
|
||||||
t.Fatalf("SessionPreviousArtifactPathForCampaign(archive-relative) = %q, want %q", archiveRelativeArtifactPath, wantArtifactPath)
|
t.Fatalf("SessionPreviousArtifactPathForCampaign(previous-relative) = %q, want %q", previousRelativeArtifactPath, wantArtifactPath)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -102,7 +102,7 @@ func TestSessionPreviousPathsFromSessionPaths(t *testing.T) {
|
|||||||
got = SessionPreviousArtifactPath(paths, "artifacts/session_recap.md")
|
got = SessionPreviousArtifactPath(paths, "artifacts/session_recap.md")
|
||||||
want = filepath.Join(paths.PreviousArtifactsDir, "session_recap.md")
|
want = filepath.Join(paths.PreviousArtifactsDir, "session_recap.md")
|
||||||
if got != want {
|
if got != want {
|
||||||
t.Fatalf("SessionPreviousArtifactPath(archive-relative) = %q, want %q", got, want)
|
t.Fatalf("SessionPreviousArtifactPath(previous-relative) = %q, want %q", got, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"sort"
|
"sort"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/artifactpolicy"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -36,10 +37,11 @@ func CollectPreviousArtifactRequirements(
|
|||||||
inputNames := sortedScriptoriumInputKeys(artifactCfg.Inputs)
|
inputNames := sortedScriptoriumInputKeys(artifactCfg.Inputs)
|
||||||
for _, inputName := range inputNames {
|
for _, inputName := range inputNames {
|
||||||
inputCfg := artifactCfg.Inputs[inputName]
|
inputCfg := artifactCfg.Inputs[inputName]
|
||||||
previousName, ok := PreviousSessionArtifactName(inputCfg.Source)
|
descriptor, err := artifactpolicy.DescribeScriptoriumInputSource(inputCfg.Source)
|
||||||
if !ok {
|
if err != nil || descriptor.PreviousSession == nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
previousName := descriptor.PreviousSession.ConfiguredKey
|
||||||
|
|
||||||
location := fmt.Sprintf(
|
location := fmt.Sprintf(
|
||||||
"pipeline.scriptorium.artifacts.%s.inputs.%s.source",
|
"pipeline.scriptorium.artifacts.%s.inputs.%s.source",
|
||||||
|
|||||||
@@ -8,8 +8,8 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ResolveArchiveBucket resolves archive bucket identity with manifest-first precedence.
|
// ResolvePublishBucket resolves publish bucket identity with manifest-first precedence.
|
||||||
func ResolveArchiveBucket(cfg *config.Config, m *manifest.Manifest) string {
|
func ResolvePublishBucket(cfg *config.Config, m *manifest.Manifest) string {
|
||||||
if m != nil && strings.TrimSpace(m.S3Bucket) != "" {
|
if m != nil && strings.TrimSpace(m.S3Bucket) != "" {
|
||||||
return strings.TrimSpace(m.S3Bucket)
|
return strings.TrimSpace(m.S3Bucket)
|
||||||
}
|
}
|
||||||
@@ -19,8 +19,8 @@ func ResolveArchiveBucket(cfg *config.Config, m *manifest.Manifest) string {
|
|||||||
return strings.TrimSpace(cfg.Pipeline.Storage.S3.Bucket)
|
return strings.TrimSpace(cfg.Pipeline.Storage.S3.Bucket)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ResolveArchiveSessionPrefix resolves archive session prefix with manifest-first precedence.
|
// ResolvePublishSessionPrefix resolves publish session prefix with manifest-first precedence.
|
||||||
func ResolveArchiveSessionPrefix(cfg *config.Config, m *manifest.Manifest) (string, error) {
|
func ResolvePublishSessionPrefix(cfg *config.Config, m *manifest.Manifest) (string, error) {
|
||||||
if m != nil && strings.TrimSpace(m.S3SessionPrefix) != "" {
|
if m != nil && strings.TrimSpace(m.S3SessionPrefix) != "" {
|
||||||
return strings.TrimSpace(m.S3SessionPrefix), nil
|
return strings.TrimSpace(m.S3SessionPrefix), nil
|
||||||
}
|
}
|
||||||
@@ -47,8 +47,8 @@ func ResolveArchiveSessionPrefix(cfg *config.Config, m *manifest.Manifest) (stri
|
|||||||
return sessionPrefix, nil
|
return sessionPrefix, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// ResolveArchiveRunPrefix resolves archive run prefix with manifest-first precedence.
|
// ResolvePublishRunPrefix resolves publish run prefix with manifest-first precedence.
|
||||||
func ResolveArchiveRunPrefix(cfg *config.Config, m *manifest.Manifest) (string, error) {
|
func ResolvePublishRunPrefix(cfg *config.Config, m *manifest.Manifest) (string, error) {
|
||||||
if m != nil {
|
if m != nil {
|
||||||
runPrefix := strings.TrimSpace(m.S3RunPrefix)
|
runPrefix := strings.TrimSpace(m.S3RunPrefix)
|
||||||
if runPrefix != "" {
|
if runPrefix != "" {
|
||||||
@@ -56,7 +56,7 @@ func ResolveArchiveRunPrefix(cfg *config.Config, m *manifest.Manifest) (string,
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
sessionPrefix, err := ResolveArchiveSessionPrefix(cfg, m)
|
sessionPrefix, err := ResolvePublishSessionPrefix(cfg, m)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", err
|
return "", err
|
||||||
}
|
}
|
||||||
@@ -71,7 +71,7 @@ func ResolveArchiveRunPrefix(cfg *config.Config, m *manifest.Manifest) (string,
|
|||||||
return S3RunPrefix(sessionPrefix, runID), nil
|
return S3RunPrefix(sessionPrefix, runID), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// ResolveArchiveCurrentStateKeys returns current pointer keys for a session prefix.
|
// ResolveCurrentStateKeys returns current pointer keys for a session prefix.
|
||||||
func ResolveArchiveCurrentStateKeys(sessionPrefix string) (manifestKey, runIDKey string) {
|
func ResolveCurrentStateKeys(sessionPrefix string) (manifestKey, runIDKey string) {
|
||||||
return S3CurrentManifestKey(sessionPrefix), S3CurrentRunPointerKey(sessionPrefix)
|
return S3CurrentManifestKey(sessionPrefix), S3CurrentRunPointerKey(sessionPrefix)
|
||||||
}
|
}
|
||||||
@@ -8,7 +8,7 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestResolveArchiveBucketPrefersManifestThenConfig(t *testing.T) {
|
func TestResolvePublishBucketPrefersManifestThenConfig(t *testing.T) {
|
||||||
cfg := &config.Config{
|
cfg := &config.Config{
|
||||||
Pipeline: &config.PipelineConfig{
|
Pipeline: &config.PipelineConfig{
|
||||||
Storage: config.StorageConfig{
|
Storage: config.StorageConfig{
|
||||||
@@ -17,15 +17,15 @@ func TestResolveArchiveBucketPrefersManifestThenConfig(t *testing.T) {
|
|||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
if got := ResolveArchiveBucket(cfg, &manifest.Manifest{S3Bucket: "manifest-bucket"}); got != "manifest-bucket" {
|
if got := ResolvePublishBucket(cfg, &manifest.Manifest{S3Bucket: "manifest-bucket"}); got != "manifest-bucket" {
|
||||||
t.Fatalf("bucket = %q, want manifest-bucket", got)
|
t.Fatalf("bucket = %q, want manifest-bucket", got)
|
||||||
}
|
}
|
||||||
if got := ResolveArchiveBucket(cfg, &manifest.Manifest{}); got != "cfg-bucket" {
|
if got := ResolvePublishBucket(cfg, &manifest.Manifest{}); got != "cfg-bucket" {
|
||||||
t.Fatalf("bucket = %q, want cfg-bucket", got)
|
t.Fatalf("bucket = %q, want cfg-bucket", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestResolveArchiveSessionPrefixPrefersManifestThenConfig(t *testing.T) {
|
func TestResolvePublishSessionPrefixPrefersManifestThenConfig(t *testing.T) {
|
||||||
cfg := &config.Config{
|
cfg := &config.Config{
|
||||||
Pipeline: &config.PipelineConfig{
|
Pipeline: &config.PipelineConfig{
|
||||||
Storage: config.StorageConfig{
|
Storage: config.StorageConfig{
|
||||||
@@ -39,17 +39,17 @@ func TestResolveArchiveSessionPrefixPrefersManifestThenConfig(t *testing.T) {
|
|||||||
}
|
}
|
||||||
m := &manifest.Manifest{S3SessionPrefix: "manifest/session/prefix/"}
|
m := &manifest.Manifest{S3SessionPrefix: "manifest/session/prefix/"}
|
||||||
|
|
||||||
got, err := ResolveArchiveSessionPrefix(cfg, m)
|
got, err := ResolvePublishSessionPrefix(cfg, m)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ResolveArchiveSessionPrefix() error = %v", err)
|
t.Fatalf("ResolvePublishSessionPrefix() error = %v", err)
|
||||||
}
|
}
|
||||||
if got != "manifest/session/prefix/" {
|
if got != "manifest/session/prefix/" {
|
||||||
t.Fatalf("session prefix = %q, want manifest/session/prefix/", got)
|
t.Fatalf("session prefix = %q, want manifest/session/prefix/", got)
|
||||||
}
|
}
|
||||||
|
|
||||||
got, err = ResolveArchiveSessionPrefix(cfg, &manifest.Manifest{})
|
got, err = ResolvePublishSessionPrefix(cfg, &manifest.Manifest{})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ResolveArchiveSessionPrefix() error = %v", err)
|
t.Fatalf("ResolvePublishSessionPrefix() error = %v", err)
|
||||||
}
|
}
|
||||||
want := "dnd/campaigns/forsaken/sessions/2026-04-19/"
|
want := "dnd/campaigns/forsaken/sessions/2026-04-19/"
|
||||||
if got != want {
|
if got != want {
|
||||||
@@ -57,7 +57,7 @@ func TestResolveArchiveSessionPrefixPrefersManifestThenConfig(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestResolveArchiveRunPrefixPrefersManifestThenDerived(t *testing.T) {
|
func TestResolvePublishRunPrefixPrefersManifestThenDerived(t *testing.T) {
|
||||||
cfg := &config.Config{
|
cfg := &config.Config{
|
||||||
Pipeline: &config.PipelineConfig{
|
Pipeline: &config.PipelineConfig{
|
||||||
Storage: config.StorageConfig{
|
Storage: config.StorageConfig{
|
||||||
@@ -74,9 +74,9 @@ func TestResolveArchiveRunPrefixPrefersManifestThenDerived(t *testing.T) {
|
|||||||
RunID: "20260516T010203Z-1a2b3c4d",
|
RunID: "20260516T010203Z-1a2b3c4d",
|
||||||
S3RunPrefix: "manifest/run/prefix/",
|
S3RunPrefix: "manifest/run/prefix/",
|
||||||
}
|
}
|
||||||
got, err := ResolveArchiveRunPrefix(cfg, m)
|
got, err := ResolvePublishRunPrefix(cfg, m)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ResolveArchiveRunPrefix() error = %v", err)
|
t.Fatalf("ResolvePublishRunPrefix() error = %v", err)
|
||||||
}
|
}
|
||||||
if got != "manifest/run/prefix/" {
|
if got != "manifest/run/prefix/" {
|
||||||
t.Fatalf("run prefix = %q, want manifest/run/prefix/", got)
|
t.Fatalf("run prefix = %q, want manifest/run/prefix/", got)
|
||||||
@@ -85,9 +85,9 @@ func TestResolveArchiveRunPrefixPrefersManifestThenDerived(t *testing.T) {
|
|||||||
m = &manifest.Manifest{
|
m = &manifest.Manifest{
|
||||||
RunID: "20260516T010203Z-1a2b3c4d",
|
RunID: "20260516T010203Z-1a2b3c4d",
|
||||||
}
|
}
|
||||||
got, err = ResolveArchiveRunPrefix(cfg, m)
|
got, err = ResolvePublishRunPrefix(cfg, m)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ResolveArchiveRunPrefix() error = %v", err)
|
t.Fatalf("ResolvePublishRunPrefix() error = %v", err)
|
||||||
}
|
}
|
||||||
want := "dnd/campaigns/forsaken/sessions/2026-04-19/runs/20260516T010203Z-1a2b3c4d/"
|
want := "dnd/campaigns/forsaken/sessions/2026-04-19/runs/20260516T010203Z-1a2b3c4d/"
|
||||||
if got != want {
|
if got != want {
|
||||||
@@ -95,12 +95,12 @@ func TestResolveArchiveRunPrefixPrefersManifestThenDerived(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestResolveArchiveIdentityErrorsAreDeterministic(t *testing.T) {
|
func TestResolvePublishIdentityErrorsAreDeterministic(t *testing.T) {
|
||||||
cfgNoS3 := &config.Config{
|
cfgNoS3 := &config.Config{
|
||||||
Pipeline: &config.PipelineConfig{},
|
Pipeline: &config.PipelineConfig{},
|
||||||
Session: &config.SessionConfig{SessionID: "2026-04-19", Campaign: "forsaken"},
|
Session: &config.SessionConfig{SessionID: "2026-04-19", Campaign: "forsaken"},
|
||||||
}
|
}
|
||||||
_, err := ResolveArchiveSessionPrefix(cfgNoS3, &manifest.Manifest{})
|
_, err := ResolvePublishSessionPrefix(cfgNoS3, &manifest.Manifest{})
|
||||||
if err == nil || !strings.Contains(err.Error(), "pipeline.storage.s3 configuration is required") {
|
if err == nil || !strings.Contains(err.Error(), "pipeline.storage.s3 configuration is required") {
|
||||||
t.Fatalf("error = %v, want missing storage.s3", err)
|
t.Fatalf("error = %v, want missing storage.s3", err)
|
||||||
}
|
}
|
||||||
@@ -113,14 +113,14 @@ func TestResolveArchiveIdentityErrorsAreDeterministic(t *testing.T) {
|
|||||||
},
|
},
|
||||||
Session: &config.SessionConfig{SessionID: "2026-04-19", Campaign: "forsaken"},
|
Session: &config.SessionConfig{SessionID: "2026-04-19", Campaign: "forsaken"},
|
||||||
}
|
}
|
||||||
_, err = ResolveArchiveRunPrefix(cfg, &manifest.Manifest{})
|
_, err = ResolvePublishRunPrefix(cfg, &manifest.Manifest{})
|
||||||
if err == nil || !strings.Contains(err.Error(), "run id is required") {
|
if err == nil || !strings.Contains(err.Error(), "run id is required") {
|
||||||
t.Fatalf("error = %v, want missing run id", err)
|
t.Fatalf("error = %v, want missing run id", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestResolveArchiveCurrentStateKeys(t *testing.T) {
|
func TestResolveCurrentStateKeys(t *testing.T) {
|
||||||
manifestKey, runIDKey := ResolveArchiveCurrentStateKeys("dnd/campaigns/forsaken/sessions/2026-04-19/")
|
manifestKey, runIDKey := ResolveCurrentStateKeys("dnd/campaigns/forsaken/sessions/2026-04-19/")
|
||||||
if manifestKey != "dnd/campaigns/forsaken/sessions/2026-04-19/current/manifest.json" {
|
if manifestKey != "dnd/campaigns/forsaken/sessions/2026-04-19/current/manifest.json" {
|
||||||
t.Fatalf("manifest key = %q", manifestKey)
|
t.Fatalf("manifest key = %q", manifestKey)
|
||||||
}
|
}
|
||||||
@@ -43,9 +43,9 @@ func TestS3KeyConstruction(t *testing.T) {
|
|||||||
t.Fatalf("manifest key = %q", manifestKey)
|
t.Fatalf("manifest key = %q", manifestKey)
|
||||||
}
|
}
|
||||||
|
|
||||||
promoted := S3PublishedOutputKey(sessionPrefix, "transcripts/final.trimmed.json")
|
publishedKey := S3PublishedOutputKey(sessionPrefix, "transcripts/final.trimmed.json")
|
||||||
if promoted != "dnd/campaigns/forsaken/sessions/2026-04-19/transcripts/final.trimmed.json" {
|
if publishedKey != "dnd/campaigns/forsaken/sessions/2026-04-19/transcripts/final.trimmed.json" {
|
||||||
t.Fatalf("promoted key = %q", promoted)
|
t.Fatalf("published key = %q", publishedKey)
|
||||||
}
|
}
|
||||||
|
|
||||||
runRelative := S3RunRelativeDestinationKey(runPrefix, `logs\whisperx.stdout.log`)
|
runRelative := S3RunRelativeDestinationKey(runPrefix, `logs\whisperx.stdout.log`)
|
||||||
|
|||||||
@@ -31,6 +31,18 @@ func TestRuntimeTranscriptArtifacts(t *testing.T) {
|
|||||||
ProducerStage: "trim",
|
ProducerStage: "trim",
|
||||||
OutputKind: TranscriptOutputKindFinalTrimmed,
|
OutputKind: TranscriptOutputKindFinalTrimmed,
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
SourceID: ArtifactTranscriptFinalMarkdown,
|
||||||
|
CanonicalRelPath: TranscriptPathFinalMarkdown,
|
||||||
|
ProducerStage: "render",
|
||||||
|
OutputKind: TranscriptOutputKindFinalMarkdown,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceID: ArtifactTranscriptFinalTrimmedMarkdown,
|
||||||
|
CanonicalRelPath: TranscriptPathFinalTrimmedMarkdown,
|
||||||
|
ProducerStage: "render",
|
||||||
|
OutputKind: TranscriptOutputKindFinalTrimmedMarkdown,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
got := RuntimeTranscriptArtifacts()
|
got := RuntimeTranscriptArtifacts()
|
||||||
@@ -75,6 +87,18 @@ func TestPlannedTranscriptArtifacts(t *testing.T) {
|
|||||||
ProducerStage: "trim",
|
ProducerStage: "trim",
|
||||||
OutputKind: TranscriptOutputKindFinalTrimmed,
|
OutputKind: TranscriptOutputKindFinalTrimmed,
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
SourceID: ArtifactTranscriptFinalMarkdown,
|
||||||
|
CanonicalRelPath: TranscriptPathFinalMarkdown,
|
||||||
|
ProducerStage: "render",
|
||||||
|
OutputKind: TranscriptOutputKindFinalMarkdown,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
SourceID: ArtifactTranscriptFinalTrimmedMarkdown,
|
||||||
|
CanonicalRelPath: TranscriptPathFinalTrimmedMarkdown,
|
||||||
|
ProducerStage: "render",
|
||||||
|
OutputKind: TranscriptOutputKindFinalTrimmedMarkdown,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
got := PlannedTranscriptArtifacts()
|
got := PlannedTranscriptArtifacts()
|
||||||
@@ -113,10 +137,14 @@ func TestRuntimeArtifactRegistryUsesTranscriptSpecs(t *testing.T) {
|
|||||||
if !ok {
|
if !ok {
|
||||||
t.Fatalf("artifactRegistry missing %q", transcript.SourceID)
|
t.Fatalf("artifactRegistry missing %q", transcript.SourceID)
|
||||||
}
|
}
|
||||||
|
wantContentKind := contentTranscriptJSON
|
||||||
|
if transcript.SourceID == ArtifactTranscriptFinalMarkdown || transcript.SourceID == ArtifactTranscriptFinalTrimmedMarkdown {
|
||||||
|
wantContentKind = contentText
|
||||||
|
}
|
||||||
if spec.CanonicalRelPath != transcript.CanonicalRelPath ||
|
if spec.CanonicalRelPath != transcript.CanonicalRelPath ||
|
||||||
spec.ProducerStage != transcript.ProducerStage ||
|
spec.ProducerStage != transcript.ProducerStage ||
|
||||||
spec.OutputKind != transcript.OutputKind ||
|
spec.OutputKind != transcript.OutputKind ||
|
||||||
spec.ContentKind != contentTranscriptJSON {
|
spec.ContentKind != wantContentKind {
|
||||||
t.Fatalf("artifactRegistry[%q] = %#v, want transcript spec %#v", transcript.SourceID, spec, transcript)
|
t.Fatalf("artifactRegistry[%q] = %#v, want transcript spec %#v", transcript.SourceID, spec, transcript)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,16 +2,14 @@ package audio
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"crypto/sha256"
|
|
||||||
"encoding/hex"
|
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
|
||||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/narratio/internal/fileops"
|
||||||
)
|
)
|
||||||
|
|
||||||
// S3MaterializeRequest describes one S3-backed audio materialization.
|
// S3MaterializeRequest describes one S3-backed audio materialization.
|
||||||
@@ -61,7 +59,7 @@ func MaterializeS3Audio(ctx context.Context, req S3MaterializeRequest) (S3Materi
|
|||||||
if ok, err := validCachedAudio(cachePath, req.Object.Size); err != nil {
|
if ok, err := validCachedAudio(cachePath, req.Object.Size); err != nil {
|
||||||
return S3MaterializeResult{}, err
|
return S3MaterializeResult{}, err
|
||||||
} else if ok {
|
} else if ok {
|
||||||
checksum, err := copyFileAtomicWithChecksum(cachePath, req.DestPath, 0o644)
|
checksum, err := fileops.CopyFileAtomicWithChecksum(cachePath, req.DestPath, 0o644)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return S3MaterializeResult{}, fmt.Errorf("materialize cached audio %q: %w", cachePath, err)
|
return S3MaterializeResult{}, fmt.Errorf("materialize cached audio %q: %w", cachePath, err)
|
||||||
}
|
}
|
||||||
@@ -82,7 +80,7 @@ func MaterializeS3Audio(ctx context.Context, req S3MaterializeRequest) (S3Materi
|
|||||||
return S3MaterializeResult{}, fmt.Errorf("validate downloaded audio %q: %w", spoolPath, err)
|
return S3MaterializeResult{}, fmt.Errorf("validate downloaded audio %q: %w", spoolPath, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
checksum, err := copyFileAtomicWithChecksum(spoolPath, req.DestPath, 0o644)
|
checksum, err := fileops.CopyFileAtomicWithChecksum(spoolPath, req.DestPath, 0o644)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return S3MaterializeResult{}, fmt.Errorf("materialize downloaded audio %q: %w", filepath.Base(req.DestPath), err)
|
return S3MaterializeResult{}, fmt.Errorf("materialize downloaded audio %q: %w", filepath.Base(req.DestPath), err)
|
||||||
}
|
}
|
||||||
@@ -91,7 +89,7 @@ func MaterializeS3Audio(ctx context.Context, req S3MaterializeRequest) (S3Materi
|
|||||||
result.Downloaded = true
|
result.Downloaded = true
|
||||||
|
|
||||||
if result.CachePath != "" {
|
if result.CachePath != "" {
|
||||||
if _, err := copyFileAtomicWithChecksum(spoolPath, result.CachePath, 0o644); err != nil {
|
if _, err := fileops.CopyFileAtomicWithChecksum(spoolPath, result.CachePath, 0o644); err != nil {
|
||||||
return S3MaterializeResult{}, fmt.Errorf("populate audio cache %q: %w", result.CachePath, err)
|
return S3MaterializeResult{}, fmt.Errorf("populate audio cache %q: %w", result.CachePath, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -164,60 +162,9 @@ func downloadObjectAtomic(ctx context.Context, store storage.ObjectStore, key, d
|
|||||||
if err := store.Download(ctx, key, tmpPath); err != nil {
|
if err := store.Download(ctx, key, tmpPath); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if err := os.Chmod(tmpPath, 0o644); err != nil {
|
if err := fileops.InstallDownloadedTempFile(tmpPath, destPath, 0o644); err != nil {
|
||||||
return fmt.Errorf("set temp file permissions: %w", err)
|
return err
|
||||||
}
|
|
||||||
if err := os.Rename(tmpPath, destPath); err != nil {
|
|
||||||
return fmt.Errorf("install downloaded file: %w", err)
|
|
||||||
}
|
}
|
||||||
removeTmp = false
|
removeTmp = false
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func copyFileAtomicWithChecksum(src, dst string, perm os.FileMode) (string, error) {
|
|
||||||
if strings.TrimSpace(src) == "" || strings.TrimSpace(dst) == "" {
|
|
||||||
return "", fmt.Errorf("source and destination paths are required")
|
|
||||||
}
|
|
||||||
in, err := os.Open(src)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
defer func() { _ = in.Close() }()
|
|
||||||
|
|
||||||
if err := os.MkdirAll(filepath.Dir(dst), 0o755); err != nil {
|
|
||||||
return "", fmt.Errorf("create destination directory: %w", err)
|
|
||||||
}
|
|
||||||
base := filepath.Base(dst)
|
|
||||||
tmp, err := os.CreateTemp(filepath.Dir(dst), "."+base+".tmp-*")
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("create temp file: %w", err)
|
|
||||||
}
|
|
||||||
tmpPath := tmp.Name()
|
|
||||||
removeTmp := true
|
|
||||||
defer func() {
|
|
||||||
if removeTmp {
|
|
||||||
_ = os.Remove(tmpPath)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
digest := sha256.New()
|
|
||||||
if _, err := io.Copy(io.MultiWriter(tmp, digest), in); err != nil {
|
|
||||||
_ = tmp.Close()
|
|
||||||
return "", fmt.Errorf("copy file: %w", err)
|
|
||||||
}
|
|
||||||
if err := tmp.Sync(); err != nil {
|
|
||||||
_ = tmp.Close()
|
|
||||||
return "", fmt.Errorf("sync temp file: %w", err)
|
|
||||||
}
|
|
||||||
if err := tmp.Close(); err != nil {
|
|
||||||
return "", fmt.Errorf("close temp file: %w", err)
|
|
||||||
}
|
|
||||||
if err := os.Chmod(tmpPath, perm); err != nil {
|
|
||||||
return "", fmt.Errorf("chmod temp file: %w", err)
|
|
||||||
}
|
|
||||||
if err := os.Rename(tmpPath, dst); err != nil {
|
|
||||||
return "", fmt.Errorf("install temp file: %w", err)
|
|
||||||
}
|
|
||||||
removeTmp = false
|
|
||||||
return hex.EncodeToString(digest.Sum(nil)), nil
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -28,6 +28,7 @@ type PipelineConfig struct {
|
|||||||
Audita AuditaConfig `yaml:"audita"`
|
Audita AuditaConfig `yaml:"audita"`
|
||||||
Normalize *NormalizeConfig `yaml:"normalize"`
|
Normalize *NormalizeConfig `yaml:"normalize"`
|
||||||
Trim *TrimConfig `yaml:"trim"`
|
Trim *TrimConfig `yaml:"trim"`
|
||||||
|
Render *RenderConfig `yaml:"render"`
|
||||||
Scriptorium *ScriptoriumConfig `yaml:"scriptorium"`
|
Scriptorium *ScriptoriumConfig `yaml:"scriptorium"`
|
||||||
Notification NotificationConfig `yaml:"notification"`
|
Notification NotificationConfig `yaml:"notification"`
|
||||||
}
|
}
|
||||||
@@ -209,6 +210,16 @@ type TrimSeriatimConfig struct {
|
|||||||
Report *bool `yaml:"report"`
|
Report *bool `yaml:"report"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RenderConfig configures render-stage output formatting behavior.
|
||||||
|
type RenderConfig struct {
|
||||||
|
Enabled *bool `yaml:"enabled"`
|
||||||
|
Format string `yaml:"format"`
|
||||||
|
Title string `yaml:"title"`
|
||||||
|
IncludeTimestamps *bool `yaml:"include_timestamps"`
|
||||||
|
IncludeSegmentIDs *bool `yaml:"include_segment_ids"`
|
||||||
|
IncludeMetadata bool `yaml:"include_metadata"`
|
||||||
|
}
|
||||||
|
|
||||||
// ScriptoriumConfig configures Scriptorium-backed artifact generation.
|
// ScriptoriumConfig configures Scriptorium-backed artifact generation.
|
||||||
type ScriptoriumConfig struct {
|
type ScriptoriumConfig struct {
|
||||||
Binary string `yaml:"binary"`
|
Binary string `yaml:"binary"`
|
||||||
|
|||||||
@@ -40,6 +40,12 @@ const (
|
|||||||
|
|
||||||
DefaultTrimBoundsTimeout = "10m"
|
DefaultTrimBoundsTimeout = "10m"
|
||||||
DefaultTrimSeriatimReport = false
|
DefaultTrimSeriatimReport = false
|
||||||
|
DefaultRenderEnabled = true
|
||||||
|
DefaultRenderFormat = "markdown"
|
||||||
|
DefaultRenderTitle = ""
|
||||||
|
DefaultRenderTimestamps = true
|
||||||
|
DefaultRenderSegmentIDs = true
|
||||||
|
DefaultRenderMetadata = false
|
||||||
|
|
||||||
DefaultNormalizeOutputPath = artifactmodel.TranscriptPathFinal
|
DefaultNormalizeOutputPath = artifactmodel.TranscriptPathFinal
|
||||||
DefaultNormalizeOutputSchema = "seriatim-intermediate"
|
DefaultNormalizeOutputSchema = "seriatim-intermediate"
|
||||||
@@ -80,6 +86,8 @@ const (
|
|||||||
// Callers should copy this slice before mutating.
|
// Callers should copy this slice before mutating.
|
||||||
var DefaultPublishOutputs = []PublishOutputRule{
|
var DefaultPublishOutputs = []PublishOutputRule{
|
||||||
{Source: artifactmodel.SourceTranscriptFinalTrimmed, Dest: PathTranscriptFinalTrimmed},
|
{Source: artifactmodel.SourceTranscriptFinalTrimmed, Dest: PathTranscriptFinalTrimmed},
|
||||||
|
{Source: artifactmodel.SourceTranscriptFinalMarkdown, Dest: artifactmodel.TranscriptPathFinalMarkdown},
|
||||||
|
{Source: artifactmodel.SourceTranscriptFinalTrimmedMarkdown, Dest: artifactmodel.TranscriptPathFinalTrimmedMarkdown},
|
||||||
}
|
}
|
||||||
|
|
||||||
// DefaultPipelineConfigSearchPaths defines the default search order for
|
// DefaultPipelineConfigSearchPaths defines the default search order for
|
||||||
|
|||||||
@@ -337,6 +337,7 @@ func applyPipelineDefaults(cfg *PipelineConfig) {
|
|||||||
}
|
}
|
||||||
applyNormalizeDefaults(cfg.Normalize)
|
applyNormalizeDefaults(cfg.Normalize)
|
||||||
applyTrimDefaults(cfg.Trim)
|
applyTrimDefaults(cfg.Trim)
|
||||||
|
applyRenderDefaults(&cfg.Render)
|
||||||
applyScriptoriumDefaults(cfg.Scriptorium)
|
applyScriptoriumDefaults(cfg.Scriptorium)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -507,6 +508,30 @@ func applyTrimDefaults(cfg *TrimConfig) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func applyRenderDefaults(cfg **RenderConfig) {
|
||||||
|
if cfg == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if *cfg == nil {
|
||||||
|
*cfg = &RenderConfig{}
|
||||||
|
}
|
||||||
|
if (*cfg).Enabled == nil {
|
||||||
|
(*cfg).Enabled = boolPtr(DefaultRenderEnabled)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace((*cfg).Format) == "" {
|
||||||
|
(*cfg).Format = DefaultRenderFormat
|
||||||
|
}
|
||||||
|
if strings.TrimSpace((*cfg).Title) == "" {
|
||||||
|
(*cfg).Title = DefaultRenderTitle
|
||||||
|
}
|
||||||
|
if (*cfg).IncludeTimestamps == nil {
|
||||||
|
(*cfg).IncludeTimestamps = boolPtr(DefaultRenderTimestamps)
|
||||||
|
}
|
||||||
|
if (*cfg).IncludeSegmentIDs == nil {
|
||||||
|
(*cfg).IncludeSegmentIDs = boolPtr(DefaultRenderSegmentIDs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func applyNormalizeDefaults(cfg *NormalizeConfig) {
|
func applyNormalizeDefaults(cfg *NormalizeConfig) {
|
||||||
if cfg == nil {
|
if cfg == nil {
|
||||||
return
|
return
|
||||||
|
|||||||
140
internal/config/render_test.go
Normal file
140
internal/config/render_test.go
Normal file
@@ -0,0 +1,140 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRenderLoadAndValidate(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
renderYAML string
|
||||||
|
wantLoadErr string
|
||||||
|
wantValidateErr string
|
||||||
|
assert func(t *testing.T, cfg *Config)
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "render defaults when omitted",
|
||||||
|
renderYAML: "",
|
||||||
|
assert: func(t *testing.T, cfg *Config) {
|
||||||
|
t.Helper()
|
||||||
|
if cfg.Pipeline.Render == nil {
|
||||||
|
t.Fatal("render config should be present via defaults")
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Render.Enabled == nil || !*cfg.Pipeline.Render.Enabled {
|
||||||
|
t.Fatalf("render.enabled = %#v, want true", cfg.Pipeline.Render.Enabled)
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Render.Format != "markdown" {
|
||||||
|
t.Fatalf("render.format = %q, want markdown", cfg.Pipeline.Render.Format)
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Render.IncludeTimestamps == nil || !*cfg.Pipeline.Render.IncludeTimestamps {
|
||||||
|
t.Fatalf("render.include_timestamps = %#v, want true", cfg.Pipeline.Render.IncludeTimestamps)
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Render.IncludeSegmentIDs == nil || !*cfg.Pipeline.Render.IncludeSegmentIDs {
|
||||||
|
t.Fatalf("render.include_segment_ids = %#v, want true", cfg.Pipeline.Render.IncludeSegmentIDs)
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Render.IncludeMetadata {
|
||||||
|
t.Fatalf("render.include_metadata = true, want false")
|
||||||
|
}
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "valid explicit render config",
|
||||||
|
renderYAML: `render:
|
||||||
|
enabled: false
|
||||||
|
format: markdown
|
||||||
|
title: Session Render
|
||||||
|
include_timestamps: false
|
||||||
|
include_segment_ids: true
|
||||||
|
include_metadata: true
|
||||||
|
`,
|
||||||
|
assert: func(t *testing.T, cfg *Config) {
|
||||||
|
t.Helper()
|
||||||
|
if cfg.Pipeline.Render == nil {
|
||||||
|
t.Fatal("render config should be present")
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Render.Enabled == nil || *cfg.Pipeline.Render.Enabled {
|
||||||
|
t.Fatalf("render.enabled = %#v, want false", cfg.Pipeline.Render.Enabled)
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Render.IncludeTimestamps == nil || *cfg.Pipeline.Render.IncludeTimestamps {
|
||||||
|
t.Fatalf("render.include_timestamps = %#v, want false", cfg.Pipeline.Render.IncludeTimestamps)
|
||||||
|
}
|
||||||
|
if cfg.Pipeline.Render.IncludeSegmentIDs == nil || !*cfg.Pipeline.Render.IncludeSegmentIDs {
|
||||||
|
t.Fatalf("render.include_segment_ids = %#v, want true", cfg.Pipeline.Render.IncludeSegmentIDs)
|
||||||
|
}
|
||||||
|
if !cfg.Pipeline.Render.IncludeMetadata {
|
||||||
|
t.Fatalf("render.include_metadata = false, want true")
|
||||||
|
}
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "explicit segment ids false overrides default",
|
||||||
|
renderYAML: `render:
|
||||||
|
include_segment_ids: false
|
||||||
|
`,
|
||||||
|
assert: func(t *testing.T, cfg *Config) {
|
||||||
|
t.Helper()
|
||||||
|
if cfg.Pipeline.Render.IncludeSegmentIDs == nil || *cfg.Pipeline.Render.IncludeSegmentIDs {
|
||||||
|
t.Fatalf("render.include_segment_ids = %#v, want false", cfg.Pipeline.Render.IncludeSegmentIDs)
|
||||||
|
}
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "invalid render format fails",
|
||||||
|
renderYAML: `render:
|
||||||
|
format: html
|
||||||
|
`,
|
||||||
|
wantValidateErr: "pipeline.render.format must be markdown",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "unknown render field fails strict decoding",
|
||||||
|
renderYAML: `render:
|
||||||
|
format: markdown
|
||||||
|
unknown: true
|
||||||
|
`,
|
||||||
|
wantLoadErr: "strict decode failed",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
pipelineYAML := testPipelineBaseYAML
|
||||||
|
if tt.renderYAML != "" {
|
||||||
|
pipelineYAML += "\n" + tt.renderYAML
|
||||||
|
}
|
||||||
|
pipelinePath, sessionPath := writeConfigFiles(t, pipelineYAML, testSessionBaseYAML)
|
||||||
|
|
||||||
|
cfg, err := Load(pipelinePath, sessionPath)
|
||||||
|
if tt.wantLoadErr != "" {
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("expected load error containing %q, got nil", tt.wantLoadErr)
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), tt.wantLoadErr) {
|
||||||
|
t.Fatalf("load error = %q, want to contain %q", err.Error(), tt.wantLoadErr)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Load() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if tt.assert != nil {
|
||||||
|
tt.assert(t, cfg)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = Validate(cfg)
|
||||||
|
if tt.wantValidateErr != "" {
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("expected validation error containing %q, got nil", tt.wantValidateErr)
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), tt.wantValidateErr) {
|
||||||
|
t.Fatalf("validation error = %q, want to contain %q", err.Error(), tt.wantValidateErr)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Validate() error = %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -200,6 +200,21 @@ func TestScriptoriumLoadAndValidate(t *testing.T) {
|
|||||||
transcript:
|
transcript:
|
||||||
source: narratio.transcript.final_trimmed
|
source: narratio.transcript.final_trimmed
|
||||||
required: true
|
required: true
|
||||||
|
`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "markdown built in artifact source is accepted",
|
||||||
|
scriptoriumYAML: `scriptorium:
|
||||||
|
binary: scriptorium
|
||||||
|
artifacts:
|
||||||
|
session_recap:
|
||||||
|
enabled: true
|
||||||
|
prompt_id: dnd.session_recap
|
||||||
|
output_path: artifacts/session_recap.md
|
||||||
|
inputs:
|
||||||
|
transcript_markdown:
|
||||||
|
source: narratio.transcript.final_markdown
|
||||||
|
required: true
|
||||||
`,
|
`,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -153,7 +153,7 @@ storage:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestSpoolAndArchiveDefaults(t *testing.T) {
|
func TestSpoolAndPublishDefaults(t *testing.T) {
|
||||||
pipelinePath, sessionPath := writeConfigFiles(t, testPipelineBaseYAML, testSessionBaseYAML)
|
pipelinePath, sessionPath := writeConfigFiles(t, testPipelineBaseYAML, testSessionBaseYAML)
|
||||||
|
|
||||||
cfg, err := Load(pipelinePath, sessionPath)
|
cfg, err := Load(pipelinePath, sessionPath)
|
||||||
@@ -171,30 +171,41 @@ func TestSpoolAndArchiveDefaults(t *testing.T) {
|
|||||||
t.Fatalf("workspace.cleanup_after_publish = true, want false")
|
t.Fatalf("workspace.cleanup_after_publish = true, want false")
|
||||||
}
|
}
|
||||||
if cfg.Pipeline.Publish == nil {
|
if cfg.Pipeline.Publish == nil {
|
||||||
t.Fatal("archive should be initialized by defaults")
|
t.Fatal("publish should be initialized by defaults")
|
||||||
}
|
}
|
||||||
if cfg.Pipeline.Publish.Enabled == nil || !*cfg.Pipeline.Publish.Enabled {
|
if cfg.Pipeline.Publish.Enabled == nil || !*cfg.Pipeline.Publish.Enabled {
|
||||||
t.Fatalf("archive.enabled = %#v, want true", cfg.Pipeline.Publish.Enabled)
|
t.Fatalf("publish.enabled = %#v, want true", cfg.Pipeline.Publish.Enabled)
|
||||||
}
|
}
|
||||||
if cfg.Pipeline.Publish.UploadRun == nil || !*cfg.Pipeline.Publish.UploadRun {
|
if cfg.Pipeline.Publish.UploadRun == nil || !*cfg.Pipeline.Publish.UploadRun {
|
||||||
t.Fatalf("archive.upload_run = %#v, want true", cfg.Pipeline.Publish.UploadRun)
|
t.Fatalf("publish.upload_run = %#v, want true", cfg.Pipeline.Publish.UploadRun)
|
||||||
}
|
}
|
||||||
if len(cfg.Pipeline.Publish.Outputs) != 1 {
|
if len(cfg.Pipeline.Publish.Outputs) != 3 {
|
||||||
t.Fatalf("publish.outputs len = %d, want 1 default", len(cfg.Pipeline.Publish.Outputs))
|
t.Fatalf("publish.outputs len = %d, want 3 defaults", len(cfg.Pipeline.Publish.Outputs))
|
||||||
}
|
}
|
||||||
item := cfg.Pipeline.Publish.Outputs[0]
|
wantBySource := map[string]string{
|
||||||
if item.Required == nil || !*item.Required {
|
"narratio.transcript.final_trimmed": "transcripts/final.trimmed.json",
|
||||||
t.Fatalf("publish.outputs[0].required = %#v, want true", item.Required)
|
"narratio.transcript.final_markdown": "transcripts/final.md",
|
||||||
|
"narratio.transcript.final_trimmed_markdown": "transcripts/final.trimmed.md",
|
||||||
}
|
}
|
||||||
if item.Source != "narratio.transcript.final_trimmed" {
|
for i, item := range cfg.Pipeline.Publish.Outputs {
|
||||||
t.Fatalf("publish.outputs[0].source = %q, want narratio.transcript.final_trimmed", item.Source)
|
if item.Required == nil || !*item.Required {
|
||||||
|
t.Fatalf("publish.outputs[%d].required = %#v, want true", i, item.Required)
|
||||||
|
}
|
||||||
|
wantDest, ok := wantBySource[item.Source]
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("publish.outputs[%d].source = %q, want known default source", i, item.Source)
|
||||||
|
}
|
||||||
|
if item.Dest != wantDest {
|
||||||
|
t.Fatalf("publish.outputs[%d].dest = %q, want %q", i, item.Dest, wantDest)
|
||||||
|
}
|
||||||
|
delete(wantBySource, item.Source)
|
||||||
}
|
}
|
||||||
if item.Dest != "transcripts/final.trimmed.json" {
|
if len(wantBySource) != 0 {
|
||||||
t.Fatalf("publish.outputs[0].dest = %q, want transcripts/final.trimmed.json", item.Dest)
|
t.Fatalf("missing default publish outputs for sources: %#v", wantBySource)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestArchivePromotionValidation(t *testing.T) {
|
func TestPublishOutputValidation(t *testing.T) {
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
ruleYML string
|
ruleYML string
|
||||||
@@ -278,7 +289,7 @@ publish:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestArchivePromotionLegacyTranscriptSourcesRejected(t *testing.T) {
|
func TestPublishOutputLegacyTranscriptSourcesRejected(t *testing.T) {
|
||||||
legacyTranscriptSources := []string{
|
legacyTranscriptSources := []string{
|
||||||
"narratio.transcript." + "merged",
|
"narratio.transcript." + "merged",
|
||||||
"narratio.transcript." + "full",
|
"narratio.transcript." + "full",
|
||||||
@@ -307,7 +318,7 @@ publish:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestArchivePromotionDerivesDestinationWhenOmitted(t *testing.T) {
|
func TestPublishOutputDerivesDestinationWhenOmitted(t *testing.T) {
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
pipelineYML string
|
pipelineYML string
|
||||||
@@ -337,6 +348,15 @@ publish:
|
|||||||
`,
|
`,
|
||||||
wantDest: "artifacts/session_recap.md",
|
wantDest: "artifacts/session_recap.md",
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
name: "markdown built in derives canonical destination",
|
||||||
|
pipelineYML: testPipelineBaseYAML + `
|
||||||
|
publish:
|
||||||
|
outputs:
|
||||||
|
- source: narratio.transcript.final_markdown
|
||||||
|
`,
|
||||||
|
wantDest: "transcripts/final.md",
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tt := range tests {
|
for _, tt := range tests {
|
||||||
@@ -359,7 +379,7 @@ publish:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestArchiveLockValidation(t *testing.T) {
|
func TestPublishLockValidation(t *testing.T) {
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
pipelineYML string
|
pipelineYML string
|
||||||
@@ -439,7 +459,7 @@ publish:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestArchiveLockLegacyTranscriptSourcesRejected(t *testing.T) {
|
func TestPublishLockLegacyTranscriptSourcesRejected(t *testing.T) {
|
||||||
legacyTranscriptSources := []string{
|
legacyTranscriptSources := []string{
|
||||||
"narratio.transcript." + "merged",
|
"narratio.transcript." + "merged",
|
||||||
"narratio.transcript." + "full",
|
"narratio.transcript." + "full",
|
||||||
@@ -467,7 +487,7 @@ publish:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestArchiveLockUnknownFieldFailsStrictDecode(t *testing.T) {
|
func TestPublishLockUnknownFieldFailsStrictDecode(t *testing.T) {
|
||||||
pipelineYAML := testPipelineBaseYAML + `
|
pipelineYAML := testPipelineBaseYAML + `
|
||||||
publish:
|
publish:
|
||||||
locks:
|
locks:
|
||||||
@@ -481,7 +501,7 @@ publish:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestArchiveLegacyFromToFailsStrictDecode(t *testing.T) {
|
func TestPublishLegacyFromToFailsStrictDecode(t *testing.T) {
|
||||||
pipelineYAML := testPipelineBaseYAML + `
|
pipelineYAML := testPipelineBaseYAML + `
|
||||||
publish:
|
publish:
|
||||||
outputs:
|
outputs:
|
||||||
@@ -495,7 +515,7 @@ publish:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestArchiveLockStoreBytesStrictDecodeAndValidation(t *testing.T) {
|
func TestPublishLockStoreBytesStrictDecodeAndValidation(t *testing.T) {
|
||||||
store, err := LoadPublishLockStoreBytes("locks.yml", []byte(`locks:
|
store, err := LoadPublishLockStoreBytes("locks.yml", []byte(`locks:
|
||||||
- source: narratio.transcript.final_trimmed
|
- source: narratio.transcript.final_trimmed
|
||||||
reason: reviewed
|
reason: reviewed
|
||||||
@@ -524,7 +544,7 @@ func TestArchiveLockStoreBytesStrictDecodeAndValidation(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestMergeArchiveLockRulesStaticWins(t *testing.T) {
|
func TestMergePublishLockRulesStaticWins(t *testing.T) {
|
||||||
merged := MergePublishLockRules(
|
merged := MergePublishLockRules(
|
||||||
[]PublishLockRule{{Source: "narratio.transcript.final_trimmed", Reason: "static"}},
|
[]PublishLockRule{{Source: "narratio.transcript.final_trimmed", Reason: "static"}},
|
||||||
[]PublishLockRule{
|
[]PublishLockRule{
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user