85 lines
2.4 KiB
Markdown
85 lines
2.4 KiB
Markdown
# S3 Audio Input
|
|
|
|
This document describes implemented S3 audio input behavior in `prepare`.
|
|
|
|
## Scope
|
|
|
|
Implemented:
|
|
|
|
- `prepare` can acquire source audio from S3 when `session.inputs.audio_s3.prefix` is configured.
|
|
- object listing and download go through the storage backend abstraction.
|
|
- tests use fake storage; no live S3 service is required for test runs.
|
|
|
|
Not implemented:
|
|
|
|
- uploads of failed runs
|
|
|
|
## Required Configuration
|
|
|
|
`pipeline.yml`:
|
|
|
|
- `storage.s3.bucket` must be set when S3 audio input is used.
|
|
- `storage.s3.root_prefix` defaults to `dnd`.
|
|
- `storage.s3.access_key_id_env` defaults to `OBJECT_STORAGE_KEY_ID`.
|
|
- `storage.s3.secret_access_key_env` defaults to `OBJECT_STORAGE_KEY`.
|
|
- `spool.root` defaults to `/var/spool/narratio`.
|
|
|
|
`session.yml`:
|
|
|
|
- configure `session.campaign` and `session.session_id`.
|
|
- configure `session.inputs.audio_s3.prefix` for S3 audio input.
|
|
- do not configure `inputs.audio_dir` or `inputs.audio_files` at the same time as `inputs.audio_s3`.
|
|
|
|
## Prefix Shape
|
|
|
|
Session S3 root:
|
|
|
|
`{root_prefix}/campaigns/{campaign}/sessions/{session_id}/`
|
|
|
|
Audio prefix:
|
|
|
|
`{session_root}/{audio_s3.prefix}`
|
|
|
|
Example:
|
|
|
|
`dnd/campaigns/forsaken/sessions/2026-04-19/audio/`
|
|
|
|
Audio files must already exist in S3 before running Narratio.
|
|
|
|
## Prepare Behavior
|
|
|
|
When `inputs.audio_s3.prefix` is configured, `prepare`:
|
|
|
|
1. lists objects under the computed S3 audio prefix
|
|
2. filters to `.flac` objects
|
|
3. fails when no `.flac` objects are found
|
|
4. downloads selected objects to spool audio:
|
|
- `{spool.root}/{campaign}/{session_id}/{run_id}/audio/`
|
|
5. materializes audio into workdir audio:
|
|
- `{workspace.root}/work/{campaign}/{session_id}/{run_id}/audio/`
|
|
6. records input provenance in the manifest (bucket, key, metadata, local paths, checksum)
|
|
|
|
Notes:
|
|
|
|
- `.flac` filtering is case-insensitive.
|
|
- ETag is recorded as provider metadata only and is not treated as a checksum.
|
|
|
|
## Local Audio Development
|
|
|
|
Local audio workflows remain supported:
|
|
|
|
- `inputs.audio_dir`
|
|
- `inputs.audio_files`
|
|
|
|
These options are mutually exclusive with `inputs.audio_s3`.
|
|
|
|
## Archive Boundary
|
|
|
|
Current archive behavior relevant to S3 audio input:
|
|
|
|
- successful runs are uploaded by archive under `runs/{run_id}/`
|
|
- configured promotions are uploaded to session-level destinations
|
|
- `current/manifest.json` and `current/run_id.txt` are published
|
|
- local source audio is not re-uploaded by default
|
|
- failed or incomplete runs are not uploaded
|