Add archive storage path configuration

This commit is contained in:
2026-05-16 14:11:59 +00:00
parent 58c6ab2d54
commit 0454296c81
23 changed files with 950 additions and 22 deletions

View File

@@ -79,6 +79,32 @@ current/run_id.txt:
written last as the effective S3 commit pointer
```
### 2.1 Implementation Status (2026-05-16)
Implemented in repository:
- storage/archive configuration and validation foundations:
- `storage.s3`
- `spool`
- `archive`
- promotion-rule safety checks
- `inputs.audio_s3` modeling
- run and path-model foundations:
- run ID generation (`YYYYMMDDTHHMMSSZ-xxxxxxxx`)
- S3 key builders for session/run/current/promoted destinations
- campaign/session/run local work and spool path helpers
- manifest run/path identity fields
- examples and tests for the above foundations
Not implemented yet:
- real S3 backend
- AWS SDK integration
- prepare-stage S3 list/download behavior
- archive-stage S3 upload behavior
- promotion uploads
- writing `current/manifest.json` and `current/run_id.txt` to S3
## 3. S3 Layout
The canonical S3 layout should be:
@@ -801,9 +827,9 @@ Test:
- `run-stage` can find or require a run ID according to final CLI policy
- multiple local runs are handled deterministically
## 15. Implementation Phases
## 15. Implementation Sequence
### Phase 1: Config and Path Model
### Config and Path Model (Implemented)
Implement:
@@ -823,7 +849,7 @@ Expected commit:
Add archive storage path configuration
```
### Phase 2: Storage Backend Interface and S3 Backend
### Storage Backend Interface and S3 Backend
Implement:
@@ -841,7 +867,7 @@ Expected commit:
Add S3 storage backend abstraction
```
### Phase 3: Prepare Stage S3 Audio Download
### Prepare Stage S3 Audio Download
Implement:
@@ -858,7 +884,7 @@ Expected commit:
Download S3 audio during prepare"
```
### Phase 4: Real Archive Stage Run Upload
### Real Archive Stage Run Upload
Implement:
@@ -873,7 +899,7 @@ Expected commit:
Upload successful run records to S3
```
### Phase 5: Promotion Rules and Current Pointer
### Promotion Rules and Current Pointer
Implement:
@@ -890,7 +916,7 @@ Expected commit:
Promote current session artifacts to S3
```
### Phase 6: Documentation and Examples
### Documentation and Examples
Update:
@@ -907,7 +933,7 @@ Expected commit:
Document S3 archive workflow
```
### Phase 7: Architectural Review
### Architectural Review
Review:

View File

@@ -0,0 +1,32 @@
# S3 Archive Foundations Runbook
This runbook documents the currently implemented storage/archive foundations and the boundaries of current behavior.
## Implemented Now
- config modeling for:
- `pipeline.storage.s3`
- `pipeline.spool`
- `pipeline.archive`
- `session.inputs.audio_s3`
- promotion rule validation for safe relative paths
- run ID generation and path/key helper functions
- manifest run/path identity fields
## Not Implemented Yet
- real S3 backend integration
- AWS SDK wiring
- prepare-stage S3 object listing or download
- archive-stage S3 upload or promotion writes
- writing `current/manifest.json` or `current/run_id.txt` in S3
## Operational Notes
- local audio workflows remain the active development path (`audio_dir` or `audio_files`)
- `audio_s3` and local audio config are mutually exclusive
- do not place AWS credentials in Narratio config files
## Next Implementation Target
Build the storage backend layer that can list/download/upload S3 objects through a fake-tested interface, then wire prepare/archive stages to use that backend.