Normalize the default configuration discovery paths for all three config files, and update documentation and tests accordingly

This commit is contained in:
2026-05-21 09:55:56 -05:00
parent 7111edeca4
commit d8f58dce31
10 changed files with 120 additions and 81 deletions

View File

@@ -6,7 +6,9 @@
narratio run --session-id 2026-04-04
```
This command uses default discovery for `pipeline.yml`, `campaign.yml`, and local `session.yml`. If local session discovery misses and S3 storage is configured, `--session-id` can load remote `session.yml` from the canonical session prefix.
This command uses default system discovery for `pipeline.yml`, `campaign.yml`, and local `session.yml`. If local session discovery misses and S3 storage is configured, `--session-id` can load remote `session.yml` from the canonical session prefix.
Default discovery checks system config locations only. Pass `--config`, `--campaign`, and `--session` to use files from the current working directory.
## Command Overview
@@ -109,7 +111,7 @@ Success output:
- `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
Common failure cases:
- missing default config/campaign/session paths when flags omitted.
- missing system default config/campaign/session paths when flags omitted.
- missing local session plus missing/unavailable remote `session.yml`.
- invalid template/rendered session mismatch.
- unknown/invalid `--artifacts` value.

View File

@@ -28,7 +28,15 @@ Behavior:
## 2. Config file discovery
Pipeline config lookup for `run`, `plan`, `resume`, `run-stage`, and `restore`:
These commands use the same config discovery behavior:
- `narratio run`
- `narratio plan`
- `narratio resume`
- `narratio run-stage`
- `narratio restore`
Pipeline config lookup:
- if `--config <path>` is provided, that path is used.
- if omitted, Narratio searches in order:
@@ -36,30 +44,29 @@ Pipeline config lookup for `run`, `plan`, `resume`, `run-stage`, and `restore`:
2. `/etc/narratio/pipeline.yml`
- first existing file wins.
Campaign config lookup for `run`, `plan`, `resume`, `run-stage`, and `restore`:
Campaign config lookup:
- if `--campaign <path>` is provided, that path is used.
- if omitted, Narratio searches in order:
1. `./campaign.yml`
2. `/usr/local/etc/narratio/campaign.yml`
3. `/etc/narratio/campaign.yml`
1. `/usr/local/etc/narratio/campaign.yml`
2. `/etc/narratio/campaign.yml`
- first existing file wins.
## 3. Session file discovery and templating
Session config lookup for `run`, `plan`, `resume`, `run-stage`, and `restore`:
Session config lookup:
- if `--session <path>` is provided, that path is used.
- if `--session` is omitted, Narratio searches locally in order:
1. `./session.yml`
2. `/usr/local/etc/narratio/session.yml`
3. `/etc/narratio/session.yml`
1. `/usr/local/etc/narratio/session.yml`
2. `/etc/narratio/session.yml`
- first existing local file wins.
- if no local session file is found, `--session-id <value>` is present, storage is configured, and campaign identity is resolved, Narratio loads remote `session.yml` from:
- `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`
- local discovery always runs before remote fallback.
- local files in the current working directory are used only when passed explicitly, for example `--config ./pipeline.yml --campaign ./campaign.yml --session ./session.yml`.
Template behavior:
## 3. Session templating
Template behavior for local and remote `session.yml`:
- supported placeholders:
- `{{session_id}}`
@@ -72,7 +79,9 @@ Template behavior:
- if rendered `session_id` mismatches `--session-id`, load fails.
- if rendered `previous_session_id` mismatches `--previous-session-id`, load fails.
## 4. Minimal pipeline config
## 4. Minimal config set
### `pipeline.yml`
```yaml
whisperx:
@@ -85,9 +94,7 @@ Why this is sufficient:
- `workspace.root` defaults to `/var/lib/narratio`.
- optional sections (`seriatim`, `audita`, `archive`, `scriptorium`, `trim`, `normalize`, etc.) receive defaults or stay inactive.
## 5. Minimal session template
`campaign.yml`:
### `campaign.yml`
```yaml
campaign: sample-campaign
@@ -97,7 +104,12 @@ inputs:
glossary_file: ./glossary.yml
```
`session.yml`:
Why this is sufficient:
- `campaign` supplies the stable campaign identity.
- stable input files are required and resolve relative to `campaign.yml` when copied during `prepare`.
### `session.yml`
```yaml
session_id: "{{ session_id }}"
@@ -105,7 +117,14 @@ inputs:
audio_dir: ./audio
```
Usage:
Why this is sufficient:
- `session_id` is required and can be rendered from `--session-id`.
- `campaign` can be omitted because it is supplied by `campaign.yml`.
- stable input paths can be omitted because `campaign.yml` supplies defaults.
- local `audio_dir` resolves relative to `session.yml`.
Minimal local-file usage:
```bash
narratio run --config /path/to/pipeline.yml --campaign ./campaign.yml --session ./session.yml --session-id 2026-05-03
@@ -124,7 +143,9 @@ inputs:
narratio run --config /path/to/pipeline.yml --campaign ./campaign.yml --session ./session.yml --session-id 2026-05-03 --previous-session-id 2026-04-26
```
## 6. Production-oriented config
## 5. Production-oriented config set
### `pipeline.yml`
```yaml
workspace:
@@ -176,6 +197,42 @@ scriptorium:
required: false
```
### `campaign.yml`
```yaml
campaign: forsaken
inputs:
speakers_file: /srv/narratio/campaigns/forsaken/speakers.yml
autocorrect_file: /srv/narratio/campaigns/forsaken/autocorrect.yml
glossary_file: /srv/narratio/campaigns/forsaken/glossary.yml
```
### Local `session.yml`
```yaml
session_id: "{{ session_id }}"
previous_session_id: "{{ previous_session_id }}"
date: 2026-05-03
title: The Black Cabin
inputs:
audio_s3:
prefix: audio/
```
### S3-first session config
For S3-first operation, upload the same `session.yml` content to:
```text
{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml
```
Then run with explicit or discovered pipeline/campaign config and no `--session`:
```bash
narratio run --config /usr/local/etc/narratio/pipeline.yml --campaign /usr/local/etc/narratio/campaign.yml --session-id 2026-05-03 --previous-session-id 2026-04-26
```
Operational notes:
- archive promotion is explicit and source-based via `archive.promote_artifacts`.
@@ -184,7 +241,7 @@ Operational notes:
- Narratio does not auto-promote all generated analyze artifacts.
- `restore` reads the same config/campaign/session inputs and restore scope is bounded by committed archive current state.
## 7. Full pipeline reference
## 6. Full pipeline reference
| Path | Type | Required | Default |
| --- | --- | --- | --- |
@@ -336,7 +393,7 @@ Restore-related implications:
- restore remote identity requires archive S3 identity to resolve (`pipeline.storage.s3.bucket` and session prefix derivation inputs).
- restore scope considers committed current state and durable paths (`manifest.json`, `transcripts/**`, `artifacts/**`, `previous/**`, optional `audio/**`).
## 8. Full campaign reference
## 7. Full campaign reference
| Path | Type | Required | Default |
| --- | --- | --- | --- |
@@ -347,7 +404,7 @@ Restore-related implications:
Campaign input paths may be absolute or relative. Relative paths resolve from the directory containing `campaign.yml`.
## 9. Full session reference
## 8. Full session reference
| Path | Type | Required | Default |
| --- | --- | --- | --- |
@@ -378,7 +435,7 @@ Previous-session rule:
- if `session.previous_session_id` is set, it must not equal `session.session_id`.
- canonical previous-session sources (`narratio.previous_session.artifact.<name>`) are hydrated during `prepare` from archive current state when required by enabled configured artifacts.
## 10. Secrets
## 9. Secrets
Narratio supports filesystem-based secret injection via `pipeline.secrets.env_dir`.
@@ -397,7 +454,7 @@ Guidance:
- do not put secret values directly in YAML.
- configure env var names in config and provide values via env/secrets files.
## 11. Examples
## 10. Examples
Maintained examples:

View File

@@ -6,7 +6,7 @@ For field-level configuration, see [docs/config.md](./config.md). For full comma
## Normal workflow (S3-first path)
1. Upload `session.yml` to the configured session prefix, or keep a local `session.yml` available.
1. Upload `session.yml` to the configured session prefix, or pass a local `session.yml` explicitly.
2. Upload session `.flac` files to object storage under the configured session audio prefix.
3. Run Narratio:
@@ -19,7 +19,7 @@ narratio run --session-id 2026-04-04
- use `manifest=<path>` with `status` for inspection.
Notes:
- default config/campaign/session discovery applies unless `--config`, `--campaign`, and `--session` are passed.
- default config/campaign/session discovery checks system config locations unless `--config`, `--campaign`, and `--session` are passed.
- when local `session.yml` discovery misses, `--session-id` loads remote `session.yml` from `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`.
- S3 audio mode requires `session.inputs.audio_s3.prefix` and valid object-store access.

View File

@@ -17,11 +17,11 @@ The current codebase already settles several design choices:
- CLI commands use short noun flags: `--config`, `--session`, `--session-id`, `--previous-session-id`, `--force`, and `--artifacts`.
- `run-stage` uses flags before the positional stage name, for example:
narratio run-stage --session ./session.yml prepare
narratio run-stage --config ./pipeline.yml --campaign ./campaign.yml --session ./session.yml prepare
- `session.yml` is represented by `config.SessionConfig` and currently owns `session_id`, `previous_session_id`, `campaign`, `date`, `title`, and `inputs`.
- Strict YAML decoding is already implemented with `yaml.Decoder.KnownFields(true)`.
- Local session discovery is already ordered as `./session.yml`, `/usr/local/etc/narratio/session.yml`, then `/etc/narratio/session.yml`.
- Default local config discovery uses system paths under `/usr/local/etc/narratio/` and `/etc/narratio/`; working-directory files are used only when passed explicitly.
- The canonical S3 session prefix is already:
{root_prefix}/campaigns/{campaign}/sessions/{session_id}/
@@ -62,9 +62,8 @@ Examples:
Campaign config discovery order:
1. explicit `--campaign <path>`;
2. `./campaign.yml`;
3. `/usr/local/etc/narratio/campaign.yml`;
4. `/etc/narratio/campaign.yml`.
2. `/usr/local/etc/narratio/campaign.yml`;
3. `/etc/narratio/campaign.yml`.
Implement this in the same style as `resolvePipelineConfigPath` and `resolveSessionConfigPath`. Add default path constants and a search-path variable in `internal/config/defaults.go`.
@@ -129,7 +128,7 @@ Build this phase after `campaign.yml`, because campaign identity is required to
Session loading order:
1. If `--session <path>` is supplied, load that local file.
2. If `--session` is omitted, use existing local discovery: `./session.yml`, `/usr/local/etc/narratio/session.yml`, `/etc/narratio/session.yml`.
2. If `--session` is omitted, use existing local discovery: `/usr/local/etc/narratio/session.yml`, `/etc/narratio/session.yml`.
3. If no local session file is found, `--session-id` is present, storage is configured, and campaign identity is resolved, load remote `session.yml`.
4. If no local or remote session can be loaded, fail with a message that lists the local search paths and the remote key that was attempted when applicable.

View File

@@ -9,19 +9,19 @@ Symptom:
- `run`, `plan`, `resume`, `run-stage`, or `restore` fails with config/session not found.
Likely Cause:
- `pipeline.yml` or `session.yml` is missing from discovery paths.
- wrong working directory when relying on `./session.yml`.
- `pipeline.yml`, `campaign.yml`, or `session.yml` is missing from system discovery paths.
- a local working-directory config file was not passed explicitly.
Diagnostics:
```bash
pwd
ls -l ./session.yml
ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml
ls -l /usr/local/etc/narratio/campaign.yml /etc/narratio/campaign.yml
ls -l /usr/local/etc/narratio/session.yml /etc/narratio/session.yml
```
Safe Fix:
- pass explicit `--config` and `--session`.
- pass explicit `--config`, `--campaign`, and `--session`.
- or place files in documented discovery paths.
Links:
@@ -40,7 +40,7 @@ Likely Cause:
Diagnostics:
```bash
narratio plan --session ./session.yml --session-id 2026-04-04
narratio plan --config /path/to/pipeline.yml --campaign /path/to/campaign.yml --session ./session.yml --session-id 2026-04-04
```
Safe Fix:
@@ -62,7 +62,7 @@ Likely Cause:
Diagnostics:
```bash
narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04
narratio plan --config /path/to/pipeline.yml --campaign /path/to/campaign.yml --session /path/to/session.yml --session-id 2026-04-04
```
Safe Fix:
@@ -84,7 +84,7 @@ Likely Cause:
Diagnostics:
```bash
narratio run --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout
narratio run --config /path/to/pipeline.yml --campaign /path/to/campaign.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout
```
Safe Fix:
@@ -106,7 +106,7 @@ Likely Cause:
Diagnostics:
```bash
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts session_recap polish
narratio run-stage --config /path/to/pipeline.yml --campaign /path/to/campaign.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts session_recap polish
```
Safe Fix:
@@ -129,7 +129,7 @@ Likely Cause:
Diagnostics:
```bash
narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04
narratio plan --config /path/to/pipeline.yml --campaign /path/to/campaign.yml --session /path/to/session.yml --session-id 2026-04-04
```
Safe Fix:
@@ -154,7 +154,7 @@ Diagnostics:
```bash
narratio status --manifest /path/to/manifest.json
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout analyze
narratio run-stage --config /path/to/pipeline.yml --campaign /path/to/campaign.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout analyze
```
Safe Fix:
@@ -227,7 +227,7 @@ Likely Cause:
Diagnostics:
```bash
narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
narratio restore --config /path/to/pipeline.yml --campaign /path/to/campaign.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
```
Safe Fix:
@@ -250,7 +250,7 @@ Likely Cause:
Diagnostics:
```bash
narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
narratio restore --config /path/to/pipeline.yml --campaign /path/to/campaign.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
```
Safe Fix:
@@ -272,7 +272,7 @@ Likely Cause:
Diagnostics:
```bash
narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
narratio restore --config /path/to/pipeline.yml --campaign /path/to/campaign.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
```
Safe Fix:
@@ -342,7 +342,7 @@ Likely Cause:
Diagnostics:
```bash
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 prepare
narratio run-stage --config /path/to/pipeline.yml --campaign /path/to/campaign.yml --session /path/to/session.yml --session-id 2026-04-04 prepare
```
Safe Fix:
@@ -366,7 +366,7 @@ Diagnostics:
```bash
narratio status --manifest /path/to/manifest.json
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 archive
narratio run-stage --config /path/to/pipeline.yml --campaign /path/to/campaign.yml --session /path/to/session.yml --session-id 2026-04-04 archive
```
Safe Fix:

View File

@@ -36,7 +36,7 @@ func TestResolveCampaignConfigPathUsesFirstExistingDefault(t *testing.T) {
}
func TestResolveCampaignConfigPathErrorIncludesSearchedPaths(t *testing.T) {
_, err := resolveCampaignConfigPathWithCandidates("", []string{"./campaign.yml", "/usr/local/etc/narratio/campaign.yml", "/etc/narratio/campaign.yml"})
_, err := resolveCampaignConfigPathWithCandidates("", []string{"/usr/local/etc/narratio/campaign.yml", "/etc/narratio/campaign.yml"})
if err == nil {
t.Fatal("expected error, got nil")
}

View File

@@ -4,7 +4,6 @@ import (
"bytes"
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
@@ -66,16 +65,7 @@ func TestExecuteLocalSessionDiscoveryPrecedenceSkipsRemote(t *testing.T) {
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
fake := &storage.FakeBackend{}
var storeInitCalls int
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, config.DefaultSessionConfigSearchPaths)
originalWD, err := os.Getwd()
if err != nil {
t.Fatalf("Getwd(): %v", err)
}
if err := os.Chdir(filepath.Dir(sessionPath)); err != nil {
t.Fatalf("Chdir(%q): %v", filepath.Dir(sessionPath), err)
}
t.Cleanup(func() { _ = os.Chdir(originalWD) })
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
var stdout bytes.Buffer
var stderr bytes.Buffer

View File

@@ -4,14 +4,15 @@ import (
"bytes"
"context"
"os"
"path/filepath"
"strings"
"testing"
"gitea.maximumdirect.net/eric/narratio/internal/config"
)
func TestPlanUsesDiscoveredSessionTemplateWithSessionIDs(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, _, sessionPath := writeValidConfigFiles(t, workspaceRoot)
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
sessionTemplate := `session_id: "{{ session_id }}"
previous_session_id: "{{ previous_session_id }}"
@@ -26,19 +27,14 @@ inputs:
t.Fatalf("write session template: %v", err)
}
cwd := filepath.Dir(sessionPath)
originalWD, err := os.Getwd()
if err != nil {
t.Fatalf("Getwd(): %v", err)
}
if err := os.Chdir(cwd); err != nil {
t.Fatalf("Chdir(%q): %v", cwd, err)
}
t.Cleanup(func() { _ = os.Chdir(originalWD) })
origSessionDefaults := append([]string(nil), config.DefaultSessionConfigSearchPaths...)
config.DefaultSessionConfigSearchPaths = []string{sessionPath}
t.Cleanup(func() { config.DefaultSessionConfigSearchPaths = origSessionDefaults })
var out bytes.Buffer
if err := Plan(context.Background(), []string{
"--config", pipelinePath,
"--campaign", campaignPath,
"--session-id", "2026-04-04",
"--previous-session-id", "2026-03-28",
}, &out); err != nil {
@@ -111,7 +107,7 @@ func TestRunStageAcceptsSessionIDFlagAndParsesStageName(t *testing.T) {
}
func TestResolveSessionConfigPathErrorIncludesSearchedPaths(t *testing.T) {
_, err := resolveSessionConfigPathWithCandidates("", []string{"./session.yml", "/usr/local/etc/narratio/session.yml", "/etc/narratio/session.yml"})
_, err := resolveSessionConfigPathWithCandidates("", []string{"/usr/local/etc/narratio/session.yml", "/etc/narratio/session.yml"})
if err == nil {
t.Fatal("expected error, got nil")
}

View File

@@ -9,7 +9,6 @@ import (
func TestCampaignConfigDefaultSearchOrder(t *testing.T) {
want := []string{
"./campaign.yml",
"/usr/local/etc/narratio/campaign.yml",
"/etc/narratio/campaign.yml",
}

View File

@@ -1,14 +1,12 @@
package config
// Default filesystem locations for pipeline configuration lookup when --config
// is omitted. Order is highest to lowest precedence.
// Default filesystem locations for config lookup when config path flags are
// omitted. Order is highest to lowest precedence.
const (
DefaultPipelineConfigPathUsrLocal = "/usr/local/etc/narratio/pipeline.yml"
DefaultPipelineConfigPathEtc = "/etc/narratio/pipeline.yml"
DefaultCampaignConfigPathLocal = "./campaign.yml"
DefaultCampaignConfigPathUsrLocal = "/usr/local/etc/narratio/campaign.yml"
DefaultCampaignConfigPathEtc = "/etc/narratio/campaign.yml"
DefaultSessionConfigPathLocal = "./session.yml"
DefaultSessionConfigPathUsrLocal = "/usr/local/etc/narratio/session.yml"
DefaultSessionConfigPathEtc = "/etc/narratio/session.yml"
DefaultS3AccessKeyIDEnv = "OBJECT_STORAGE_KEY_ID"
@@ -97,7 +95,6 @@ var DefaultPipelineConfigSearchPaths = []string{
// Keep this in a variable so future defaults can be extended without changing
// call sites.
var DefaultCampaignConfigSearchPaths = []string{
DefaultCampaignConfigPathLocal,
DefaultCampaignConfigPathUsrLocal,
DefaultCampaignConfigPathEtc,
}
@@ -108,7 +105,6 @@ var DefaultCampaignConfigSearchPaths = []string{
// Keep this in a variable so future defaults can be extended without changing
// call sites.
var DefaultSessionConfigSearchPaths = []string{
DefaultSessionConfigPathLocal,
DefaultSessionConfigPathUsrLocal,
DefaultSessionConfigPathEtc,
}