Update documentation for the new analyze stage and artifact registry
This commit is contained in:
@@ -14,45 +14,42 @@ These commands load and validate both files before running:
|
||||
- `narratio resume`
|
||||
- `narratio run-stage`
|
||||
|
||||
Configuration behavior:
|
||||
Behavior:
|
||||
|
||||
- strict YAML decode is enabled (`KnownFields(true)`): unknown fields fail.
|
||||
- session templates are rendered before session YAML decode.
|
||||
- defaults are applied for many optional pipeline fields.
|
||||
- session templates render before session YAML decode.
|
||||
- defaults are applied for optional pipeline fields.
|
||||
- validation enforces required fields, value formats, and cross-field constraints.
|
||||
|
||||
## 2. Config file discovery
|
||||
|
||||
Pipeline config lookup for `run`, `plan`, `resume`, and `run-stage`:
|
||||
|
||||
- If `--config <path>` is provided, that explicit path is used.
|
||||
- If `--config` is omitted, Narratio searches in order:
|
||||
- If `--config <path>` is provided, that path is used.
|
||||
- If omitted, Narratio searches in order:
|
||||
1. `/usr/local/etc/narratio/pipeline.yml`
|
||||
2. `/etc/narratio/pipeline.yml`
|
||||
- The first existing file wins.
|
||||
- If none exist, the command fails with a searched-paths error.
|
||||
- First existing file wins.
|
||||
|
||||
## 3. Session file discovery and templating
|
||||
|
||||
Session config lookup for `run`, `plan`, `resume`, and `run-stage`:
|
||||
|
||||
- If `--session <path>` is provided, that explicit path is used.
|
||||
- If `--session` is omitted, Narratio searches in order:
|
||||
- If `--session <path>` is provided, that path is used.
|
||||
- If omitted, Narratio searches in order:
|
||||
1. `./session.yml`
|
||||
2. `/usr/local/etc/narratio/session.yml`
|
||||
3. `/etc/narratio/session.yml`
|
||||
- The first existing file wins.
|
||||
- If none exist, the command fails and asks you to pass `--session`.
|
||||
- First existing file wins.
|
||||
|
||||
Session templating:
|
||||
Template behavior:
|
||||
|
||||
- Supported placeholders:
|
||||
- `{{session_id}}`
|
||||
- `{{ session_id }}`
|
||||
- `--session-id <value>` supplies the template value.
|
||||
- Unresolved placeholders fail with a template-rendering error.
|
||||
- If `--session-id` is provided and rendered `session_id` differs, load fails with a mismatch error.
|
||||
- Strict YAML decode still applies after template rendering.
|
||||
- `--session-id <value>` supplies the placeholder value.
|
||||
- unresolved placeholders fail load.
|
||||
- if rendered `session_id` mismatches `--session-id`, load fails.
|
||||
|
||||
## 4. Minimal pipeline config
|
||||
|
||||
@@ -64,9 +61,8 @@ whisperx:
|
||||
Why this is sufficient:
|
||||
|
||||
- `whisperx.transcribe_url` is required.
|
||||
- `workspace.root` is optional and defaults to `/var/lib/narratio`.
|
||||
- Seriatim and Audita sections may be omitted; defaults are applied.
|
||||
- Archive, storage, spool, normalize, and other optional sections get defaults when omitted.
|
||||
- `workspace.root` defaults to `/var/lib/narratio`.
|
||||
- optional sections (`seriatim`, `audita`, `archive`, `scriptorium`, `trim`, `normalize`, etc.) receive defaults or stay inactive.
|
||||
|
||||
## 5. Minimal session template
|
||||
|
||||
@@ -119,18 +115,26 @@ archive:
|
||||
|
||||
whisperx:
|
||||
transcribe_url: "https://transcription.example.com/transcribe"
|
||||
|
||||
scriptorium:
|
||||
artifacts:
|
||||
session_recap:
|
||||
enabled: true
|
||||
prompt_id: dnd.session_recap
|
||||
output_path: artifacts/session_recap.md
|
||||
inputs:
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
required: true
|
||||
```
|
||||
|
||||
Operational notes:
|
||||
|
||||
- `workspace.cleanup_after_archive` controls run-scoped workspace cleanup after successful archive commit.
|
||||
- `spool.delete_audio_after_archive` controls run-scoped spool-audio cleanup after successful archive commit.
|
||||
- S3 archive/session-audio workflows require `storage.s3.bucket`.
|
||||
- archive promotion is explicit and path-based via `archive.promote_artifacts`.
|
||||
- Narratio does not auto-promote all generated analyze artifacts.
|
||||
|
||||
## 7. Full pipeline reference
|
||||
|
||||
Defaults listed here are effective runtime defaults after load.
|
||||
|
||||
| Path | Type | Required | Default |
|
||||
| --- | --- | --- | --- |
|
||||
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
|
||||
@@ -150,7 +154,7 @@ Defaults listed here are effective runtime defaults after load.
|
||||
| `pipeline.spool.delete_audio_after_archive` | bool | No | `false` |
|
||||
| `pipeline.archive.enabled` | bool | No | `true` |
|
||||
| `pipeline.archive.upload_run` | bool | No | `true` |
|
||||
| `pipeline.archive.promote_artifacts[]` | list | No | two default rules |
|
||||
| `pipeline.archive.promote_artifacts[]` | list | No | trimmed + session_recap rules |
|
||||
| `pipeline.archive.promote_artifacts[].from` | string | Yes (per rule) | none |
|
||||
| `pipeline.archive.promote_artifacts[].to` | string | Yes (per rule) | none |
|
||||
| `pipeline.archive.promote_artifacts[].required` | bool | No | `true` |
|
||||
@@ -203,6 +207,7 @@ Defaults listed here are effective runtime defaults after load.
|
||||
| `pipeline.scriptorium.render_debug` | bool | No | `false` |
|
||||
| `pipeline.scriptorium.artifacts` | map | No | empty |
|
||||
| `pipeline.scriptorium.artifacts.<name>.enabled` | bool | No | `false` |
|
||||
| `pipeline.scriptorium.artifacts.<name>.depends_on[]` | list[string] | No | empty |
|
||||
| `pipeline.scriptorium.artifacts.<name>.render_debug` | bool | No | unset |
|
||||
| `pipeline.scriptorium.artifacts.<name>.prompt_id` | string | Conditional | none |
|
||||
| `pipeline.scriptorium.artifacts.<name>.profile_id` | string | No | empty |
|
||||
@@ -221,6 +226,18 @@ Defaults listed here are effective runtime defaults after load.
|
||||
| `pipeline.notification.recipient` | string | No | empty |
|
||||
| `pipeline.notification.timeout` | duration string | No | empty |
|
||||
|
||||
Scriptorium artifact-key and dependency rules:
|
||||
|
||||
- artifact keys must match `^[a-z][a-z0-9_]*$`.
|
||||
- enabled artifacts require `prompt_id` and `output_path`.
|
||||
- `output_path` must be relative, traversal-safe, and under `artifacts/`.
|
||||
- configured artifact input sources use `narratio.artifact.<name>`.
|
||||
- if input source references `narratio.artifact.<name>`, artifact `<name>` must exist and must be listed in `depends_on`.
|
||||
- every `depends_on` entry must be a configured artifact key.
|
||||
- self-dependency is rejected.
|
||||
- enabled dependency cycles are rejected.
|
||||
- any artifact referenced by `depends_on` or `narratio.artifact.<name>` source must define `output_path` (even if not enabled).
|
||||
|
||||
Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:
|
||||
|
||||
- `previous_session_artifact`
|
||||
@@ -229,7 +246,7 @@ Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:
|
||||
- `narratio.transcript.full`
|
||||
- `narratio.transcript.trimmed`
|
||||
- `narratio.bounds.session`
|
||||
- `narratio.artifact.session_recap`
|
||||
- `narratio.artifact.<configured_artifact_key>`
|
||||
|
||||
## 8. Full session reference
|
||||
|
||||
@@ -248,11 +265,11 @@ Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:
|
||||
|
||||
Audio-source rule:
|
||||
|
||||
- You must configure exactly one audio source mode:
|
||||
- configure exactly one mode:
|
||||
- `audio_dir`, or
|
||||
- `audio_files` (at least one), or
|
||||
- `audio_s3.prefix`
|
||||
- `audio_s3` cannot be combined with `audio_dir` or `audio_files`.
|
||||
- `audio_s3` cannot be combined with local audio fields.
|
||||
|
||||
## 9. Secrets
|
||||
|
||||
@@ -261,22 +278,21 @@ Narratio supports filesystem-based secret injection via `pipeline.secrets.env_di
|
||||
Behavior:
|
||||
|
||||
- `env_dir` may be absolute or relative.
|
||||
- Relative `env_dir` is resolved from Narratio’s current working directory.
|
||||
- Each top-level file with a valid env-var filename (`[A-Za-z_][A-Za-z0-9_]*`) is loaded.
|
||||
- File contents become env-var values, with trailing `\n` / `\r\n` trimmed.
|
||||
- Existing process environment variables are preserved and not overwritten.
|
||||
- Invalid names and directories inside `env_dir` are skipped.
|
||||
- Missing/unreadable `env_dir` fails command execution.
|
||||
- relative `env_dir` resolves from current working directory.
|
||||
- files with valid env-var names (`[A-Za-z_][A-Za-z0-9_]*`) are loaded.
|
||||
- values are loaded from file contents with trailing newline trimming.
|
||||
- existing process env vars are preserved.
|
||||
- invalid names and subdirectories are skipped.
|
||||
- missing/unreadable `env_dir` fails command execution.
|
||||
|
||||
Guidance:
|
||||
|
||||
- Store secret values in secret files or pre-set environment variables.
|
||||
- Do not put secret values directly in `pipeline.yml` or `session.yml`.
|
||||
- Use config fields like `llm_api_key_env` and S3 credential env names to reference secret variable names, not secret data.
|
||||
- do not put secret values directly in YAML.
|
||||
- configure env var names in config and provide values via env/secrets files.
|
||||
|
||||
## 10. Examples
|
||||
|
||||
Maintained config examples:
|
||||
Maintained examples:
|
||||
|
||||
- `docs/examples/pipeline.minimal.yml`
|
||||
- `docs/examples/pipeline.production.yml`
|
||||
@@ -285,4 +301,4 @@ Maintained config examples:
|
||||
- `docs/examples/session.local-audio.yml`
|
||||
- `docs/examples/session.s3-audio.yml`
|
||||
|
||||
These examples are covered by configuration load/validate tests in `internal/config`.
|
||||
These examples are validated by `internal/config` tests.
|
||||
|
||||
Reference in New Issue
Block a user