Update documentation for the new analyze stage and artifact registry

This commit is contained in:
2026-05-19 19:42:28 -05:00
parent ebb21b9201
commit 574b1cde6c
11 changed files with 464 additions and 344 deletions

View File

@@ -14,45 +14,42 @@ These commands load and validate both files before running:
- `narratio resume`
- `narratio run-stage`
Configuration behavior:
Behavior:
- strict YAML decode is enabled (`KnownFields(true)`): unknown fields fail.
- session templates are rendered before session YAML decode.
- defaults are applied for many optional pipeline fields.
- session templates render before session YAML decode.
- defaults are applied for optional pipeline fields.
- validation enforces required fields, value formats, and cross-field constraints.
## 2. Config file discovery
Pipeline config lookup for `run`, `plan`, `resume`, and `run-stage`:
- If `--config <path>` is provided, that explicit path is used.
- If `--config` is omitted, Narratio searches in order:
- If `--config <path>` is provided, that path is used.
- If omitted, Narratio searches in order:
1. `/usr/local/etc/narratio/pipeline.yml`
2. `/etc/narratio/pipeline.yml`
- The first existing file wins.
- If none exist, the command fails with a searched-paths error.
- First existing file wins.
## 3. Session file discovery and templating
Session config lookup for `run`, `plan`, `resume`, and `run-stage`:
- If `--session <path>` is provided, that explicit path is used.
- If `--session` is omitted, Narratio searches in order:
- If `--session <path>` is provided, that path is used.
- If omitted, Narratio searches in order:
1. `./session.yml`
2. `/usr/local/etc/narratio/session.yml`
3. `/etc/narratio/session.yml`
- The first existing file wins.
- If none exist, the command fails and asks you to pass `--session`.
- First existing file wins.
Session templating:
Template behavior:
- Supported placeholders:
- `{{session_id}}`
- `{{ session_id }}`
- `--session-id <value>` supplies the template value.
- Unresolved placeholders fail with a template-rendering error.
- If `--session-id` is provided and rendered `session_id` differs, load fails with a mismatch error.
- Strict YAML decode still applies after template rendering.
- `--session-id <value>` supplies the placeholder value.
- unresolved placeholders fail load.
- if rendered `session_id` mismatches `--session-id`, load fails.
## 4. Minimal pipeline config
@@ -64,9 +61,8 @@ whisperx:
Why this is sufficient:
- `whisperx.transcribe_url` is required.
- `workspace.root` is optional and defaults to `/var/lib/narratio`.
- Seriatim and Audita sections may be omitted; defaults are applied.
- Archive, storage, spool, normalize, and other optional sections get defaults when omitted.
- `workspace.root` defaults to `/var/lib/narratio`.
- optional sections (`seriatim`, `audita`, `archive`, `scriptorium`, `trim`, `normalize`, etc.) receive defaults or stay inactive.
## 5. Minimal session template
@@ -119,18 +115,26 @@ archive:
whisperx:
transcribe_url: "https://transcription.example.com/transcribe"
scriptorium:
artifacts:
session_recap:
enabled: true
prompt_id: dnd.session_recap
output_path: artifacts/session_recap.md
inputs:
transcript:
source: narratio.transcript.trimmed
required: true
```
Operational notes:
- `workspace.cleanup_after_archive` controls run-scoped workspace cleanup after successful archive commit.
- `spool.delete_audio_after_archive` controls run-scoped spool-audio cleanup after successful archive commit.
- S3 archive/session-audio workflows require `storage.s3.bucket`.
- archive promotion is explicit and path-based via `archive.promote_artifacts`.
- Narratio does not auto-promote all generated analyze artifacts.
## 7. Full pipeline reference
Defaults listed here are effective runtime defaults after load.
| Path | Type | Required | Default |
| --- | --- | --- | --- |
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
@@ -150,7 +154,7 @@ Defaults listed here are effective runtime defaults after load.
| `pipeline.spool.delete_audio_after_archive` | bool | No | `false` |
| `pipeline.archive.enabled` | bool | No | `true` |
| `pipeline.archive.upload_run` | bool | No | `true` |
| `pipeline.archive.promote_artifacts[]` | list | No | two default rules |
| `pipeline.archive.promote_artifacts[]` | list | No | trimmed + session_recap rules |
| `pipeline.archive.promote_artifacts[].from` | string | Yes (per rule) | none |
| `pipeline.archive.promote_artifacts[].to` | string | Yes (per rule) | none |
| `pipeline.archive.promote_artifacts[].required` | bool | No | `true` |
@@ -203,6 +207,7 @@ Defaults listed here are effective runtime defaults after load.
| `pipeline.scriptorium.render_debug` | bool | No | `false` |
| `pipeline.scriptorium.artifacts` | map | No | empty |
| `pipeline.scriptorium.artifacts.<name>.enabled` | bool | No | `false` |
| `pipeline.scriptorium.artifacts.<name>.depends_on[]` | list[string] | No | empty |
| `pipeline.scriptorium.artifacts.<name>.render_debug` | bool | No | unset |
| `pipeline.scriptorium.artifacts.<name>.prompt_id` | string | Conditional | none |
| `pipeline.scriptorium.artifacts.<name>.profile_id` | string | No | empty |
@@ -221,6 +226,18 @@ Defaults listed here are effective runtime defaults after load.
| `pipeline.notification.recipient` | string | No | empty |
| `pipeline.notification.timeout` | duration string | No | empty |
Scriptorium artifact-key and dependency rules:
- artifact keys must match `^[a-z][a-z0-9_]*$`.
- enabled artifacts require `prompt_id` and `output_path`.
- `output_path` must be relative, traversal-safe, and under `artifacts/`.
- configured artifact input sources use `narratio.artifact.<name>`.
- if input source references `narratio.artifact.<name>`, artifact `<name>` must exist and must be listed in `depends_on`.
- every `depends_on` entry must be a configured artifact key.
- self-dependency is rejected.
- enabled dependency cycles are rejected.
- any artifact referenced by `depends_on` or `narratio.artifact.<name>` source must define `output_path` (even if not enabled).
Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:
- `previous_session_artifact`
@@ -229,7 +246,7 @@ Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:
- `narratio.transcript.full`
- `narratio.transcript.trimmed`
- `narratio.bounds.session`
- `narratio.artifact.session_recap`
- `narratio.artifact.<configured_artifact_key>`
## 8. Full session reference
@@ -248,11 +265,11 @@ Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:
Audio-source rule:
- You must configure exactly one audio source mode:
- configure exactly one mode:
- `audio_dir`, or
- `audio_files` (at least one), or
- `audio_s3.prefix`
- `audio_s3` cannot be combined with `audio_dir` or `audio_files`.
- `audio_s3` cannot be combined with local audio fields.
## 9. Secrets
@@ -261,22 +278,21 @@ Narratio supports filesystem-based secret injection via `pipeline.secrets.env_di
Behavior:
- `env_dir` may be absolute or relative.
- Relative `env_dir` is resolved from Narratios current working directory.
- Each top-level file with a valid env-var filename (`[A-Za-z_][A-Za-z0-9_]*`) is loaded.
- File contents become env-var values, with trailing `\n` / `\r\n` trimmed.
- Existing process environment variables are preserved and not overwritten.
- Invalid names and directories inside `env_dir` are skipped.
- Missing/unreadable `env_dir` fails command execution.
- relative `env_dir` resolves from current working directory.
- files with valid env-var names (`[A-Za-z_][A-Za-z0-9_]*`) are loaded.
- values are loaded from file contents with trailing newline trimming.
- existing process env vars are preserved.
- invalid names and subdirectories are skipped.
- missing/unreadable `env_dir` fails command execution.
Guidance:
- Store secret values in secret files or pre-set environment variables.
- Do not put secret values directly in `pipeline.yml` or `session.yml`.
- Use config fields like `llm_api_key_env` and S3 credential env names to reference secret variable names, not secret data.
- do not put secret values directly in YAML.
- configure env var names in config and provide values via env/secrets files.
## 10. Examples
Maintained config examples:
Maintained examples:
- `docs/examples/pipeline.minimal.yml`
- `docs/examples/pipeline.production.yml`
@@ -285,4 +301,4 @@ Maintained config examples:
- `docs/examples/session.local-audio.yml`
- `docs/examples/session.s3-audio.yml`
These examples are covered by configuration load/validate tests in `internal/config`.
These examples are validated by `internal/config` tests.