Add archive storage path configuration
This commit is contained in:
@@ -79,6 +79,32 @@ current/run_id.txt:
|
||||
written last as the effective S3 commit pointer
|
||||
```
|
||||
|
||||
### 2.1 Implementation Status (2026-05-16)
|
||||
|
||||
Implemented in repository:
|
||||
|
||||
- storage/archive configuration and validation foundations:
|
||||
- `storage.s3`
|
||||
- `spool`
|
||||
- `archive`
|
||||
- promotion-rule safety checks
|
||||
- `inputs.audio_s3` modeling
|
||||
- run and path-model foundations:
|
||||
- run ID generation (`YYYYMMDDTHHMMSSZ-xxxxxxxx`)
|
||||
- S3 key builders for session/run/current/promoted destinations
|
||||
- campaign/session/run local work and spool path helpers
|
||||
- manifest run/path identity fields
|
||||
- examples and tests for the above foundations
|
||||
|
||||
Not implemented yet:
|
||||
|
||||
- real S3 backend
|
||||
- AWS SDK integration
|
||||
- prepare-stage S3 list/download behavior
|
||||
- archive-stage S3 upload behavior
|
||||
- promotion uploads
|
||||
- writing `current/manifest.json` and `current/run_id.txt` to S3
|
||||
|
||||
## 3. S3 Layout
|
||||
|
||||
The canonical S3 layout should be:
|
||||
@@ -801,9 +827,9 @@ Test:
|
||||
- `run-stage` can find or require a run ID according to final CLI policy
|
||||
- multiple local runs are handled deterministically
|
||||
|
||||
## 15. Implementation Phases
|
||||
## 15. Implementation Sequence
|
||||
|
||||
### Phase 1: Config and Path Model
|
||||
### Config and Path Model (Implemented)
|
||||
|
||||
Implement:
|
||||
|
||||
@@ -823,7 +849,7 @@ Expected commit:
|
||||
Add archive storage path configuration
|
||||
```
|
||||
|
||||
### Phase 2: Storage Backend Interface and S3 Backend
|
||||
### Storage Backend Interface and S3 Backend
|
||||
|
||||
Implement:
|
||||
|
||||
@@ -841,7 +867,7 @@ Expected commit:
|
||||
Add S3 storage backend abstraction
|
||||
```
|
||||
|
||||
### Phase 3: Prepare Stage S3 Audio Download
|
||||
### Prepare Stage S3 Audio Download
|
||||
|
||||
Implement:
|
||||
|
||||
@@ -858,7 +884,7 @@ Expected commit:
|
||||
Download S3 audio during prepare"
|
||||
```
|
||||
|
||||
### Phase 4: Real Archive Stage Run Upload
|
||||
### Real Archive Stage Run Upload
|
||||
|
||||
Implement:
|
||||
|
||||
@@ -873,7 +899,7 @@ Expected commit:
|
||||
Upload successful run records to S3
|
||||
```
|
||||
|
||||
### Phase 5: Promotion Rules and Current Pointer
|
||||
### Promotion Rules and Current Pointer
|
||||
|
||||
Implement:
|
||||
|
||||
@@ -890,7 +916,7 @@ Expected commit:
|
||||
Promote current session artifacts to S3
|
||||
```
|
||||
|
||||
### Phase 6: Documentation and Examples
|
||||
### Documentation and Examples
|
||||
|
||||
Update:
|
||||
|
||||
@@ -907,7 +933,7 @@ Expected commit:
|
||||
Document S3 archive workflow
|
||||
```
|
||||
|
||||
### Phase 7: Architectural Review
|
||||
### Architectural Review
|
||||
|
||||
Review:
|
||||
|
||||
|
||||
32
docs/runbooks/s3-archive-foundations.md
Normal file
32
docs/runbooks/s3-archive-foundations.md
Normal file
@@ -0,0 +1,32 @@
|
||||
# S3 Archive Foundations Runbook
|
||||
|
||||
This runbook documents the currently implemented storage/archive foundations and the boundaries of current behavior.
|
||||
|
||||
## Implemented Now
|
||||
|
||||
- config modeling for:
|
||||
- `pipeline.storage.s3`
|
||||
- `pipeline.spool`
|
||||
- `pipeline.archive`
|
||||
- `session.inputs.audio_s3`
|
||||
- promotion rule validation for safe relative paths
|
||||
- run ID generation and path/key helper functions
|
||||
- manifest run/path identity fields
|
||||
|
||||
## Not Implemented Yet
|
||||
|
||||
- real S3 backend integration
|
||||
- AWS SDK wiring
|
||||
- prepare-stage S3 object listing or download
|
||||
- archive-stage S3 upload or promotion writes
|
||||
- writing `current/manifest.json` or `current/run_id.txt` in S3
|
||||
|
||||
## Operational Notes
|
||||
|
||||
- local audio workflows remain the active development path (`audio_dir` or `audio_files`)
|
||||
- `audio_s3` and local audio config are mutually exclusive
|
||||
- do not place AWS credentials in Narratio config files
|
||||
|
||||
## Next Implementation Target
|
||||
|
||||
Build the storage backend layer that can list/download/upload S3 objects through a fake-tested interface, then wire prepare/archive stages to use that backend.
|
||||
Reference in New Issue
Block a user