Add archive storage path configuration

This commit is contained in:
2026-05-16 14:11:59 +00:00
parent 58c6ab2d54
commit 0454296c81
23 changed files with 950 additions and 22 deletions

View File

@@ -22,10 +22,23 @@ Implemented:
- real `trim` stage producing `transcripts/trimmed.json`
- real `analyze` stage for initial `session_recap` generation
- optional Scriptorium render diagnostics (`render_debug`) before production run
- storage/archive configuration and validation foundations for:
- `pipeline.storage.s3`
- `pipeline.spool`
- `pipeline.archive` promotion rules
- `session.inputs.audio_s3`
- run identity and path-model foundations:
- run ID generation (`YYYYMMDDTHHMMSSZ-xxxxxxxx`)
- S3 session/run/current key builders
- campaign/session/run local work/spool path helpers
- manifest run/path identity fields (`campaign`, `run_id`, local and S3 prefixes)
Still placeholder/future:
- `archive` stage behavior
- real S3 storage backend (list/download/upload/exists)
- prepare-stage S3 audio download behavior
- archive-stage S3 upload/promotion behavior
- `notify` stage behavior
- additional Scriptorium artifact types beyond `session_recap`
- artifact-to-artifact workflows beyond the initial single-artifact implementation
@@ -110,6 +123,40 @@ Optional pipeline secrets directory:
- if configured, unreadable/missing `env_dir` fails command execution early
- relative `env_dir` values are resolved from current working directory
Storage and archive foundations:
- `pipeline.storage.s3` is available for modeling S3 coordinates:
- `bucket`
- `root_prefix` (default `dnd`)
- `region`
- `endpoint`
- `force_path_style` (default `false`)
- `pipeline.spool.root` defaults to `/var/spool/narratio`
- `pipeline.spool.delete_audio_after_archive` defaults to `false` (cleanup behavior not implemented yet)
- `pipeline.archive` is optional and defaults to:
- `enabled: true`
- `upload_run: true`
- default `promote_artifacts`:
- `transcripts/trimmed.json`
- `artifacts/session_recap.md`
- archive promotion rules enforce safe relative paths:
- `from` and `to` are required
- absolute paths are rejected
- traversal segments such as `..` are rejected
Session input foundations:
- `session.campaign` is required
- local audio remains supported through `session.inputs.audio_dir` or `session.inputs.audio_files`
- optional S3 audio input shape is modeled as `session.inputs.audio_s3.prefix`
- `audio_dir`/`audio_files` and `audio_s3` are mutually exclusive
- S3 input execution (object listing/downloading) is not implemented yet
Cross-config validation scope:
- `pipeline.storage.s3.bucket` is required only when an S3-dependent feature is explicitly configured (for current foundations, that includes `session.inputs.audio_s3`, and archive upload intent when using `storage.backend: s3`)
- no AWS credentials are stored in Narratio config; credential resolution remains an external runtime concern
`pipeline.scriptorium` is optional. Existing pipelines without Scriptorium continue to work.
`pipeline.trim` is optional. Existing pipelines without trim config continue to work.