Add archive storage path configuration
This commit is contained in:
@@ -22,10 +22,23 @@ Implemented:
|
||||
- real `trim` stage producing `transcripts/trimmed.json`
|
||||
- real `analyze` stage for initial `session_recap` generation
|
||||
- optional Scriptorium render diagnostics (`render_debug`) before production run
|
||||
- storage/archive configuration and validation foundations for:
|
||||
- `pipeline.storage.s3`
|
||||
- `pipeline.spool`
|
||||
- `pipeline.archive` promotion rules
|
||||
- `session.inputs.audio_s3`
|
||||
- run identity and path-model foundations:
|
||||
- run ID generation (`YYYYMMDDTHHMMSSZ-xxxxxxxx`)
|
||||
- S3 session/run/current key builders
|
||||
- campaign/session/run local work/spool path helpers
|
||||
- manifest run/path identity fields (`campaign`, `run_id`, local and S3 prefixes)
|
||||
|
||||
Still placeholder/future:
|
||||
|
||||
- `archive` stage behavior
|
||||
- real S3 storage backend (list/download/upload/exists)
|
||||
- prepare-stage S3 audio download behavior
|
||||
- archive-stage S3 upload/promotion behavior
|
||||
- `notify` stage behavior
|
||||
- additional Scriptorium artifact types beyond `session_recap`
|
||||
- artifact-to-artifact workflows beyond the initial single-artifact implementation
|
||||
@@ -110,6 +123,40 @@ Optional pipeline secrets directory:
|
||||
- if configured, unreadable/missing `env_dir` fails command execution early
|
||||
- relative `env_dir` values are resolved from current working directory
|
||||
|
||||
Storage and archive foundations:
|
||||
|
||||
- `pipeline.storage.s3` is available for modeling S3 coordinates:
|
||||
- `bucket`
|
||||
- `root_prefix` (default `dnd`)
|
||||
- `region`
|
||||
- `endpoint`
|
||||
- `force_path_style` (default `false`)
|
||||
- `pipeline.spool.root` defaults to `/var/spool/narratio`
|
||||
- `pipeline.spool.delete_audio_after_archive` defaults to `false` (cleanup behavior not implemented yet)
|
||||
- `pipeline.archive` is optional and defaults to:
|
||||
- `enabled: true`
|
||||
- `upload_run: true`
|
||||
- default `promote_artifacts`:
|
||||
- `transcripts/trimmed.json`
|
||||
- `artifacts/session_recap.md`
|
||||
- archive promotion rules enforce safe relative paths:
|
||||
- `from` and `to` are required
|
||||
- absolute paths are rejected
|
||||
- traversal segments such as `..` are rejected
|
||||
|
||||
Session input foundations:
|
||||
|
||||
- `session.campaign` is required
|
||||
- local audio remains supported through `session.inputs.audio_dir` or `session.inputs.audio_files`
|
||||
- optional S3 audio input shape is modeled as `session.inputs.audio_s3.prefix`
|
||||
- `audio_dir`/`audio_files` and `audio_s3` are mutually exclusive
|
||||
- S3 input execution (object listing/downloading) is not implemented yet
|
||||
|
||||
Cross-config validation scope:
|
||||
|
||||
- `pipeline.storage.s3.bucket` is required only when an S3-dependent feature is explicitly configured (for current foundations, that includes `session.inputs.audio_s3`, and archive upload intent when using `storage.backend: s3`)
|
||||
- no AWS credentials are stored in Narratio config; credential resolution remains an external runtime concern
|
||||
|
||||
`pipeline.scriptorium` is optional. Existing pipelines without Scriptorium continue to work.
|
||||
|
||||
`pipeline.trim` is optional. Existing pipelines without trim config continue to work.
|
||||
|
||||
Reference in New Issue
Block a user