7 Commits

58 changed files with 2776 additions and 2014 deletions

View File

@@ -40,14 +40,31 @@ Most session-aware commands accept:
- `--campaign <id>`
- `--campaign-file <campaign.yml>`
- `--session <session.yml>`
- `--previous-session-id <id>`
- `--session-id <session_id>`
- `--previous-session-id <session_id>`
Rules:
- `--campaign` and `--campaign-file` are mutually exclusive.
- `--session` is not used by `session init`.
- if both positional `<session_id>` and `--session-id` are provided, values must match.
- `clean --all` cannot be combined with campaign/session selectors.
## Session ID Input Rules
Session-aware commands accept one of these forms:
- positional session ID: `... <session_id>`
- compatibility flag: `... --session-id <session_id>`
When both are present, command parsing requires an exact match.
Commands with additional positionals keep their command-specific order:
- `run-stage <stage> <session_id>` or `run-stage <stage> --session-id <session_id>`
- `session locks add <session_id> <source>` or `session locks add --session-id <session_id> <source>`
- `session locks remove <session_id> <source>` or `session locks remove --session-id <session_id> <source>`
## Command Reference
### `run`
@@ -187,7 +204,7 @@ Options:
Rules:
- `--audio-dir` and `--audio-s3-prefix` are mutually exclusive.
- if campaign `session_template_file` is configured, `session init` renders it;
- if campaign `session_template_file` is configured, `session init` renders it.
- generated session YAML must be concrete (no unresolved `{{ ... }}` placeholders).
### `session restore`

View File

@@ -1,9 +1,10 @@
# Internal: Artifacts
## Purpose
Define canonical artifact IDs, runtime catalog behavior, and source resolution rules for stage execution and publish output selection.
Define canonical artifact IDs, runtime catalog behavior, source resolution rules, and shared current-state mechanics used by app and previous-cache code.
## Built-in Source IDs
- `narratio.transcript.base` -> `transcripts/base.json` (`merge`)
- `narratio.transcript.polished` -> `transcripts/polished.json` (`polish`)
- `narratio.transcript.final` -> `transcripts/final.json` (`normalize`)
@@ -11,58 +12,101 @@ Define canonical artifact IDs, runtime catalog behavior, and source resolution r
- `narratio.bounds.session` -> `artifacts/session_bounds.json` (`trim`)
## Configured and Previous-Session Sources
- Configured artifact source ID: `narratio.artifact.<artifact_key>`
- Previous-session source ID: `narratio.previous_session.artifact.<artifact_key>`
Configured and previous-session source IDs are validated by strict regex rules.
- configured source ID format: `narratio.artifact.<artifact_key>`
- previous-session source ID format: `narratio.previous_session.artifact.<artifact_key>`
Both formats are validated by strict source-policy rules.
## Runtime Catalog
`ArtifactCatalog` tracks:
- `planned`: source registered for run context.
- `executable`: selected and enabled for analyze execution.
- `available`: local file exists and validated.
- `planned`: source registered for run context;
- `executable`: selected and enabled for analyze execution;
- `available`: local file exists and validates;
- `provenance`: availability source.
Current provenance values:
- `generated.current_analyze_run`
- `filesystem.disabled_artifact_output`
- `manifest.inputs.previous_cache`
- `current_session.previous_cache`
## Resolution Rules
Built-ins:
1. manifest producer outputs (when present)
2. canonical session path fallback
2. canonical session-path fallback
Configured sources (`narratio.artifact.*`):
- resolve only through runtime catalog availability.
Previous-session sources (`narratio.previous_session.artifact.*`):
- resolve only from local `previous/` cache state.
- prefer manifest-backed previous input paths.
- resolve only from local `previous/` cache state;
- prefer manifest-backed previous-input paths;
- fallback to existing previous-cache filesystem paths.
Validation by content type:
- transcript built-ins: JSON with top-level `segments` array.
- bounds built-in: valid JSON.
- transcript built-ins: JSON with top-level `segments` array;
- bounds built-in: valid JSON;
- configured/previous-session artifact files: non-empty text file.
## Previous Requirement Collection
`CollectPreviousArtifactRequirements`:
- scans enabled configured artifacts only;
- extracts only canonical previous-session sources;
- deduplicates by artifact key;
- merges required/optional (required wins);
- merges required and optional references (required wins);
- returns deterministic ordering and source locations.
## Current-State Helpers
Artifacts package owns shared remote current-state loading mechanics used by restore, status/validate checks, and previous-cache planning.
Core helpers:
- `LoadCurrentRunPointer`
- `LoadCurrentManifest`
- `LoadCurrentState`
- `ValidateCurrentStateIdentity`
Typed missing-state errors:
- `CurrentRunPointerMissingError` (`ErrCurrentRunPointerMissing`)
- `CurrentManifestMissingError` (`ErrCurrentManifestMissing`)
Identity validation supports caller-provided expectations:
- expected campaign;
- expected session ID;
- expected run ID, or pointer/manifest run-ID consistency check.
Caller policy is intentionally outside artifacts helpers:
- some callers fail on missing current state;
- some callers downgrade missing state to status/findings;
- some callers skip optional behavior when state is missing.
## Key Path Helpers
`internal/artifacts/paths.go` defines canonical helpers for:
`internal/artifacts/paths.go` and S3-key helpers define canonical helpers for:
- session/work/run paths;
- previous-cache paths;
- spool/cache paths;
- S3 key layout helpers for session/run/current pointers.
- S3 session/run/current-state key layout.
## Invariants
- Source ID formats are stable contracts.
- Resolution is deterministic and manifest-aware.
- Previous-session source resolution does not call remote storage in `analyze`; remote hydration is `prepare` responsibility.
- source ID formats are stable contracts;
- artifact resolution is deterministic and manifest-aware;
- previous-session source resolution in `analyze` is local-only;
- remote current-state key construction remains centralized in artifacts helpers.

View File

@@ -1,65 +1,84 @@
# Internal: Command Restore
## Purpose
Document the implemented `narratio session restore` command contract:
Define the implemented `narratio session restore` command contract:
- committed remote current-state discovery;
- deterministic restore plan classification;
- deterministic restore planning;
- safe local install semantics;
- durable restore reporting.
## Discovery Contract
Restore discovers remote committed state using:
- `current/run_id.txt` (required, non-empty)
- `current/manifest.json` (required, decodable)
Discovered manifest identity must match requested `session_id` and `campaign`.
Restore resolves remote committed state from the session publish current pointers:
## Plan Contract
Planner actions:
- `download`
- `skip_same`
- `conflict`
- `current/run_id.txt` (required, non-empty);
- `current/manifest.json` (required, decodable).
Current-state discovery uses shared artifacts-level mechanics and validates identity against the resolved request config:
- campaign must match;
- session ID must match.
Restore treats any missing or invalid remote current state as a command error.
## Planning Contract
Restore planner action kinds:
- `download`;
- `skip_same`;
- `conflict`.
Planner behavior:
Plan behavior:
- remote list scope is the resolved session prefix;
- mapping to local paths is traversal-safe;
- remote-to-local mapping is traversal-safe;
- actions are sorted deterministically by local relative path.
Restore scope from current remote state:
- include `manifest.json`
- include `transcripts/**`
- include `artifacts/**`
- include `audio/**` only with `--include-audio`
- include `manifest.json`;
- include `transcripts/**`;
- include `artifacts/**`;
- include `audio/**` only with `--include-audio`.
Explicit exclusions from current remote state mapping:
- `current/**`
- `runs/**`
- `logs/**`
- `reports/**`
- `config/**`
- `inputs/**`
- `previous/**`
Previous-cache restore files are planned separately through `previouscache.BuildPlan` when configured previous-session requirements exist.
- `current/**`;
- `runs/**`;
- `logs/**`;
- `reports/**`;
- `config/**`;
- `inputs/**`;
- `previous/**`.
Previous-cache files are planned separately through `previouscache.BuildPlan` when configured previous-session requirements exist.
## Execution Contract
Execution order and safety:
- non-manifest downloads happen before manifest install;
- `manifest.json` is installed last;
- downloads use sibling temp files + atomic rename;
- `manifest.json` installs last;
- downloads use sibling temp files plus atomic rename;
- manifest replacement is validated before rename;
- failed installs do not roll back previously written files.
- failed installs do not roll back files already written in the same execution.
Audio restore path:
- uses `audio.MaterializeS3Audio`;
- integrates spool and S3 audio cache paths;
- supports cache hit reuse without object redownload.
- supports cache-hit reuse without object redownload.
## Reporting Contract
- dry-run: summary only (no writes).
- `--dry-run`: prints summary only; no local writes.
- non-dry-run: writes `reports/restore-latest.json`.
- report captures plan counts, action status, and execution failures.
- report includes plan counts, per-action status, and execution failures.
## Invariants
- restore uses only committed remote current state as authority.
- `current/run_id.txt` is the remote commit marker.
- restore is a standalone command and does not run stages.
- restore uses committed remote current state as authority;
- `current/run_id.txt` is the remote publish commit marker;
- restore does not execute pipeline stages.

View File

@@ -1,8 +1,8 @@
# Roadmap: Pre-1.0 Code Cleanup
Status: Planned
Status: Implemented (Stages 1-6 complete)
This roadmap turns the findings in `docs/roadmap/audit.md` into staged cleanup work for the 1.0 release. It is planning-only. Do not implement these refactors until a stage is explicitly selected for implementation.
This roadmap turns the findings in `docs/roadmap/audit.md` into staged cleanup work for the 1.0 release and records completion status for each selected stage.
The cleanup work must follow the policy documents under `docs/policy/`, especially these invariants:

View File

@@ -0,0 +1,36 @@
package storage
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
)
// DownloadObjectToTemp downloads an object into a temporary file and returns
// the cleaned local path.
func DownloadObjectToTemp(ctx context.Context, store ObjectStore, key, pattern string) (string, error) {
if store == nil {
return "", fmt.Errorf("object store is required")
}
if strings.TrimSpace(pattern) == "" {
return "", fmt.Errorf("temp file pattern is required")
}
tmp, err := os.CreateTemp("", pattern)
if err != nil {
return "", fmt.Errorf("create temp file: %w", err)
}
path := tmp.Name()
if err := tmp.Close(); err != nil {
_ = os.Remove(path)
return "", fmt.Errorf("close temp file: %w", err)
}
if err := store.Download(ctx, key, path); err != nil {
_ = os.Remove(path)
return "", err
}
return filepath.Clean(path), nil
}

View File

@@ -0,0 +1,69 @@
package storage
import (
"context"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
)
func TestDownloadObjectToTempSuccess(t *testing.T) {
store := &FakeBackend{}
store.SeedObject(FakeObject{Key: "sessions/a/current/run_id.txt", Data: []byte("run-123\n")})
path, err := DownloadObjectToTemp(context.Background(), store, "sessions/a/current/run_id.txt", "narratio-test-*.txt")
if err != nil {
t.Fatalf("DownloadObjectToTemp() error = %v", err)
}
t.Cleanup(func() { _ = os.Remove(path) })
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile() error = %v", err)
}
if string(data) != "run-123\n" {
t.Fatalf("downloaded data = %q, want %q", string(data), "run-123\n")
}
}
func TestDownloadObjectToTempFailedDownloadRemovesTempFile(t *testing.T) {
sentinel := errors.New("download failed")
store := &FakeBackend{DownloadErr: sentinel}
pattern := "narratio-test-fail-*.txt"
before, err := filepath.Glob(filepath.Join(os.TempDir(), "narratio-test-fail-*.txt"))
if err != nil {
t.Fatalf("Glob(before) error = %v", err)
}
path, err := DownloadObjectToTemp(context.Background(), store, "sessions/a/current/run_id.txt", pattern)
if !errors.Is(err, sentinel) {
t.Fatalf("DownloadObjectToTemp() error = %v, want %v", err, sentinel)
}
if strings.TrimSpace(path) != "" {
t.Fatalf("DownloadObjectToTemp() path = %q, want empty on failure", path)
}
after, err := filepath.Glob(filepath.Join(os.TempDir(), "narratio-test-fail-*.txt"))
if err != nil {
t.Fatalf("Glob(after) error = %v", err)
}
if len(after) != len(before) {
t.Fatalf("temp file count changed after failed download: before=%d after=%d", len(before), len(after))
}
}
func TestDownloadObjectToTempCallerContextWrappingPreservesCause(t *testing.T) {
sentinel := errors.New("object missing")
store := &FakeBackend{DownloadErr: sentinel}
_, err := DownloadObjectToTemp(context.Background(), store, "sessions/a/current/run_id.txt", "narratio-test-*.txt")
if err == nil {
t.Fatal("DownloadObjectToTemp() error = nil, want error")
}
err = fmt.Errorf("download run pointer failed: %w", err)
if !errors.Is(err, sentinel) {
t.Fatalf("wrapped error does not preserve sentinel cause: %v", err)
}
}

View File

@@ -33,7 +33,7 @@ func TestExecuteRunStageArtifactsUnsupportedStageFails(t *testing.T) {
}
}
func TestExecuteRunStageArchivePropagatesSelectedArtifacts(t *testing.T) {
func TestExecuteRunStagePublishPropagatesSelectedArtifacts(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
@@ -281,7 +281,7 @@ func TestExecuteAnalyzeMissingConfigUsesRunStageLoadingPath(t *testing.T) {
}
}
func TestExecutePublishForceRunsArchive(t *testing.T) {
func TestExecutePublishForceRunsPublish(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)

View File

@@ -16,7 +16,6 @@ import (
// Clean removes local workspace/spool state while preserving durable cache
// state unless cache cleanup is explicitly requested.
func Clean(ctx context.Context, args []string, out io.Writer) error {
positionalSessionID, args := pullLeadingSessionID(args)
fs := flag.NewFlagSet("clean", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var flags commonConfigFlags
@@ -27,20 +26,8 @@ func Clean(ctx context.Context, args []string, out io.Writer) error {
fs.BoolVar(&all, "all", false, "clean all local session work/spool state")
fs.BoolVar(&dryRun, "dry-run", false, "print cleanup targets without deleting")
fs.BoolVar(&clearCache, "clear-cache", false, "also clear durable S3 audio cache entries")
if err := fs.Parse(args); err != nil {
return fmt.Errorf("clean: invalid flags: %w", err)
}
if positionalSessionID == "" {
if err := applyParsedSessionIDArg("clean", fs, &flags.sessionID); err != nil {
return err
}
} else {
if fs.NArg() != 0 {
return fmt.Errorf("clean: unexpected positional arguments")
}
if err := applyPositionalSessionID("clean", positionalSessionID, &flags.sessionID); err != nil {
return err
}
if err := parseSessionAwareFlags("clean", fs, args, &flags.sessionID); err != nil {
return err
}
if all {
return cleanAllLocal(flags, dryRun, clearCache, out)
@@ -182,26 +169,12 @@ func reportCleanRootChildren(out io.Writer, root, policy string, dryRun bool) er
}
func cleanableRootChildren(root, policy string) (string, []string, error) {
cleanRoot := strings.TrimSpace(root)
if cleanRoot == "" {
return "", nil, fmt.Errorf("cleanup policy %s: root path is required", policy)
}
rootAbs, err := filepath.Abs(cleanRoot)
rootAbs, exists, err := validateCleanRoot(root, policy)
if err != nil {
return "", nil, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
return "", nil, err
}
info, err := os.Lstat(rootAbs)
if err != nil {
if os.IsNotExist(err) {
return rootAbs, nil, nil
}
return "", nil, fmt.Errorf("cleanup policy %s: stat root %q: %w", policy, rootAbs, err)
}
if info.Mode()&os.ModeSymlink != 0 {
return "", nil, fmt.Errorf("cleanup policy %s: refusing to clean symlink root %q", policy, rootAbs)
}
if !info.IsDir() {
return "", nil, fmt.Errorf("cleanup policy %s: root %q is not a directory", policy, rootAbs)
if !exists {
return rootAbs, nil, nil
}
entries, err := os.ReadDir(rootAbs)
if err != nil {
@@ -300,46 +273,7 @@ func reportCleanScopedFile(out io.Writer, root, target, policy string, dryRun bo
}
func validateScopedFile(root, target, policy string) (scopedDir, error) {
cleanRoot := strings.TrimSpace(root)
cleanTarget := strings.TrimSpace(target)
if cleanRoot == "" {
return scopedDir{}, fmt.Errorf("cleanup policy %s: root path is required", policy)
}
if cleanTarget == "" {
return scopedDir{}, fmt.Errorf("cleanup policy %s: target path is required", policy)
}
rootAbs, err := filepath.Abs(cleanRoot)
if err != nil {
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
}
targetAbs, err := filepath.Abs(cleanTarget)
if err != nil {
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
}
rel, err := filepath.Rel(rootAbs, targetAbs)
if err != nil {
return scopedDir{}, fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
}
if rel == "." {
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
}
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
}
info, err := os.Lstat(targetAbs)
if err != nil {
if os.IsNotExist(err) {
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: false}, nil
}
return scopedDir{}, fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
}
if info.Mode()&os.ModeSymlink != 0 {
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
}
if info.IsDir() {
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is a directory", policy, targetAbs)
}
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: true}, nil
return validateScopedTarget(root, target, policy, false)
}
func cleanIsFlac(path string) bool {

View File

@@ -0,0 +1,82 @@
package app
import (
"fmt"
"os"
"path/filepath"
"strings"
)
func validateScopedTarget(root, target, policy string, requireDir bool) (scopedDir, error) {
cleanRoot := strings.TrimSpace(root)
cleanTarget := strings.TrimSpace(target)
if cleanRoot == "" {
return scopedDir{}, fmt.Errorf("cleanup policy %s: root path is required", policy)
}
if cleanTarget == "" {
return scopedDir{}, fmt.Errorf("cleanup policy %s: target path is required", policy)
}
rootAbs, err := filepath.Abs(cleanRoot)
if err != nil {
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
}
targetAbs, err := filepath.Abs(cleanTarget)
if err != nil {
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
}
rel, err := filepath.Rel(rootAbs, targetAbs)
if err != nil {
return scopedDir{}, fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
}
if rel == "." {
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
}
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
}
info, err := os.Lstat(targetAbs)
if err != nil {
if os.IsNotExist(err) {
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: false}, nil
}
return scopedDir{}, fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
}
if info.Mode()&os.ModeSymlink != 0 {
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
}
if requireDir && !info.IsDir() {
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is not a directory", policy, targetAbs)
}
if !requireDir && info.IsDir() {
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is a directory", policy, targetAbs)
}
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: true}, nil
}
func validateCleanRoot(root, policy string) (string, bool, error) {
cleanRoot := strings.TrimSpace(root)
if cleanRoot == "" {
return "", false, fmt.Errorf("cleanup policy %s: root path is required", policy)
}
rootAbs, err := filepath.Abs(cleanRoot)
if err != nil {
return "", false, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
}
info, err := os.Lstat(rootAbs)
if err != nil {
if os.IsNotExist(err) {
return rootAbs, false, nil
}
return "", false, fmt.Errorf("cleanup policy %s: stat root %q: %w", policy, rootAbs, err)
}
if info.Mode()&os.ModeSymlink != 0 {
return "", false, fmt.Errorf("cleanup policy %s: refusing to clean symlink root %q", policy, rootAbs)
}
if !info.IsDir() {
return "", false, fmt.Errorf("cleanup policy %s: root %q is not a directory", policy, rootAbs)
}
return rootAbs, true, nil
}

View File

@@ -0,0 +1,103 @@
package app
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestCleanValidateScopedDirAndFile(t *testing.T) {
root := t.TempDir()
dirTarget := filepath.Join(root, "runs", "run-1")
fileTarget := filepath.Join(root, "cache", "a.flac")
if err := os.MkdirAll(dirTarget, 0o755); err != nil {
t.Fatalf("MkdirAll(dirTarget) error = %v", err)
}
if err := os.MkdirAll(filepath.Dir(fileTarget), 0o755); err != nil {
t.Fatalf("MkdirAll(file parent) error = %v", err)
}
if err := os.WriteFile(fileTarget, []byte("audio"), 0o644); err != nil {
t.Fatalf("WriteFile(fileTarget) error = %v", err)
}
if _, err := validateScopedDir(root, dirTarget, "test.dir"); err != nil {
t.Fatalf("validateScopedDir() error = %v", err)
}
if _, err := validateScopedFile(root, fileTarget, "test.file"); err != nil {
t.Fatalf("validateScopedFile() error = %v", err)
}
}
func TestCleanValidateScopedTargetSafetyRules(t *testing.T) {
root := t.TempDir()
outside := t.TempDir()
target := filepath.Join(root, "runs", "run-1")
if err := os.MkdirAll(target, 0o755); err != nil {
t.Fatalf("MkdirAll(target) error = %v", err)
}
fileTarget := filepath.Join(root, "cache", "a.flac")
if err := os.MkdirAll(filepath.Dir(fileTarget), 0o755); err != nil {
t.Fatalf("MkdirAll(file parent) error = %v", err)
}
if err := os.WriteFile(fileTarget, []byte("audio"), 0o644); err != nil {
t.Fatalf("WriteFile(fileTarget) error = %v", err)
}
symlinkTarget := filepath.Join(root, "symlink")
if err := os.Symlink(target, symlinkTarget); err != nil {
t.Fatalf("Symlink() error = %v", err)
}
if _, err := validateScopedDir(root, root, "test.root"); err == nil || !strings.Contains(err.Error(), "refusing to delete root directory") {
t.Fatalf("validateScopedDir(root) error = %v, want root deletion rejection", err)
}
if _, err := validateScopedDir(root, filepath.Join(outside, "x"), "test.outside"); err == nil || !strings.Contains(err.Error(), "outside root") {
t.Fatalf("validateScopedDir(outside) error = %v, want outside-root rejection", err)
}
if _, err := validateScopedDir(root, fileTarget, "test.file-as-dir"); err == nil || !strings.Contains(err.Error(), "is not a directory") {
t.Fatalf("validateScopedDir(file) error = %v, want not-a-directory rejection", err)
}
if _, err := validateScopedFile(root, target, "test.dir-as-file"); err == nil || !strings.Contains(err.Error(), "is a directory") {
t.Fatalf("validateScopedFile(dir) error = %v, want is-a-directory rejection", err)
}
if _, err := validateScopedDir(root, symlinkTarget, "test.symlink"); err == nil || !strings.Contains(err.Error(), "refusing to delete symlink path") {
t.Fatalf("validateScopedDir(symlink) error = %v, want symlink rejection", err)
}
}
func TestCleanableRootChildrenRejectsSymlinkChild(t *testing.T) {
root := t.TempDir()
realChild := filepath.Join(root, "runs")
if err := os.MkdirAll(realChild, 0o755); err != nil {
t.Fatalf("MkdirAll(realChild) error = %v", err)
}
if err := os.Symlink(realChild, filepath.Join(root, "link")); err != nil {
t.Fatalf("Symlink() error = %v", err)
}
_, _, err := cleanableRootChildren(root, "test.root.children")
if err == nil || !strings.Contains(err.Error(), "refusing to delete symlink path") {
t.Fatalf("cleanableRootChildren() error = %v, want symlink rejection", err)
}
}
func TestCleanValidateScopedTargetMissing(t *testing.T) {
root := t.TempDir()
missingDir := filepath.Join(root, "runs", "missing")
got, err := validateScopedDir(root, missingDir, "test.missing")
if err != nil {
t.Fatalf("validateScopedDir(missing) error = %v", err)
}
if got.Exists {
t.Fatalf("validateScopedDir(missing).Exists = true, want false")
}
missingFile := filepath.Join(root, "cache", "missing.flac")
got, err = validateScopedFile(root, missingFile, "test.missing.file")
if err != nil {
t.Fatalf("validateScopedFile(missing) error = %v", err)
}
if got.Exists {
t.Fatalf("validateScopedFile(missing).Exists = true, want false")
}
}

View File

@@ -0,0 +1,136 @@
package app
import (
"context"
"fmt"
"io"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
"gitea.maximumdirect.net/eric/narratio/internal/artifactpolicy"
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
"gitea.maximumdirect.net/eric/narratio/internal/config"
)
func buildHelperArtifactCatalog(cfg *config.Config) (*artifacts.ArtifactCatalog, error) {
catalog := artifacts.NewArtifactCatalog()
if err := catalog.RegisterBuiltIns(); err != nil {
return nil, err
}
configured := map[string]artifacts.ConfiguredArtifactDefinition{}
if cfg.Pipeline.Scriptorium != nil {
for key, item := range cfg.Pipeline.Scriptorium.Artifacts {
configured[key] = artifacts.ConfiguredArtifactDefinition{Enabled: item.Enabled, OutputPath: item.OutputPath}
}
}
if err := catalog.RegisterConfiguredArtifacts(configured, nil); err != nil {
return nil, err
}
return catalog, nil
}
func writeArtifactList(out io.Writer, cfg *config.Config, catalog *artifacts.ArtifactCatalog, locks *effectiveLocks, publishedRemoteState map[string]string) {
lockSet := lockSourceSet(locks.All)
fmt.Fprintln(out, "Built-in:")
for _, id := range []string{
artifacts.ArtifactTranscriptBase,
artifacts.ArtifactTranscriptPolished,
artifacts.ArtifactTranscriptFinal,
artifacts.ArtifactTranscriptFinalTrimmed,
artifacts.ArtifactBoundsSession,
} {
writeArtifactLine(out, id, lockSet)
}
fmt.Fprintln(out, "Configured:")
for _, entry := range catalog.ListConfigured() {
writeArtifactLine(out, entry.SourceID, lockSet)
}
fmt.Fprintln(out, "Previous-session:")
for _, req := range artifacts.CollectPreviousArtifactRequirements(configuredScriptoriumArtifacts(cfg)) {
fmt.Fprintf(out, "- %s required=%t\n", artifactpolicy.PreviousSessionSourceID(req.Name), req.Required)
}
fmt.Fprintln(out, "Published:")
for _, rule := range cfg.Pipeline.Publish.Outputs {
writePublishedOutputLine(out, rule, catalog, lockSet, publishedRemoteState)
}
}
func writeArtifactLine(out io.Writer, source string, lockSet map[string]config.PublishLockRule) {
parts := []string{source}
if _, ok := lockSet[source]; ok {
parts = append(parts, "locked")
}
fmt.Fprintf(out, "- %s\n", strings.Join(parts, " "))
}
func writePublishedOutputLine(out io.Writer, rule config.PublishOutputRule, catalog *artifacts.ArtifactCatalog, lockSet map[string]config.PublishLockRule, remoteState map[string]string) {
source := strings.TrimSpace(rule.Source)
parts := []string{source}
if _, ok := lockSet[source]; ok {
parts = append(parts, "locked")
}
dest, showDest, err := helperPublishedOutputDest(rule, catalog)
if err != nil {
parts = append(parts, "remote=error")
fmt.Fprintf(out, "- %s\n", strings.Join(parts, " "))
return
}
if showDest {
parts = append(parts, "dest="+dest)
}
if state := remoteState[publishedOutputRemoteStateKey(source, dest)]; state != "" {
parts = append(parts, state)
}
fmt.Fprintf(out, "- %s\n", strings.Join(parts, " "))
}
func remotePublishedOutputAvailability(ctx context.Context, cfg *config.Config, store storage.ObjectStore, catalog *artifacts.ArtifactCatalog) map[string]string {
out := map[string]string{}
sessionPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
for _, rule := range cfg.Pipeline.Publish.Outputs {
source := strings.TrimSpace(rule.Source)
dest, _, err := helperPublishedOutputDest(rule, catalog)
if err != nil {
out[publishedOutputRemoteStateKey(source, "")] = "remote=error"
continue
}
key := artifacts.S3PublishedOutputKey(sessionPrefix, dest)
if exists, err := store.Exists(ctx, key); err == nil && exists {
out[publishedOutputRemoteStateKey(source, dest)] = "remote=published"
} else if err != nil {
out[publishedOutputRemoteStateKey(source, dest)] = "remote=error"
} else {
out[publishedOutputRemoteStateKey(source, dest)] = "remote=missing"
}
}
return out
}
func helperPublishedOutputDest(rule config.PublishOutputRule, catalog *artifacts.ArtifactCatalog) (string, bool, error) {
source := strings.TrimSpace(rule.Source)
normalized, err := artifactpolicy.ResolvePublishedDestination(source, rule.Dest, helperConfiguredOutputPathMap(catalog))
if err != nil {
return "", false, err
}
entry, ok := catalog.Lookup(source)
showDest := !ok || strings.TrimSpace(entry.CanonicalRelPath) != normalized
return normalized, showDest, nil
}
func helperConfiguredOutputPathMap(catalog *artifacts.ArtifactCatalog) map[string]string {
out := map[string]string{}
if catalog == nil {
return out
}
for _, entry := range catalog.ListConfigured() {
if strings.TrimSpace(entry.ConfiguredKey) == "" {
continue
}
out[entry.ConfiguredKey] = strings.TrimSpace(entry.CanonicalRelPath)
}
return out
}
func publishedOutputRemoteStateKey(source, dest string) string {
return strings.TrimSpace(source) + "\x00" + strings.TrimSpace(dest)
}

View File

@@ -0,0 +1,39 @@
package app
import (
"context"
"flag"
"fmt"
"io"
"strings"
)
// ArtifactsList lists effective artifact sources.
func ArtifactsList(ctx context.Context, args []string, out io.Writer) error {
fs := flag.NewFlagSet("artifacts list", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var flags commonConfigFlags
var remote bool
addCommonConfigFlags(fs, &flags)
fs.BoolVar(&remote, "remote", false, "inspect remote publish availability")
if err := parseSessionAwareFlags("artifacts list", fs, args, &flags.sessionID); err != nil {
return err
}
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("artifacts list: session_id is required")
}
cfg, store, locks, _, err := loadHelperContext(ctx, flags, remote)
if err != nil {
return fmt.Errorf("artifacts list: %w", err)
}
catalog, err := buildHelperArtifactCatalog(cfg)
if err != nil {
return fmt.Errorf("artifacts list: %w", err)
}
publishedRemoteState := map[string]string{}
if remote && store != nil {
publishedRemoteState = remotePublishedOutputAvailability(ctx, cfg, store, catalog)
}
writeArtifactList(out, cfg, catalog, locks, publishedRemoteState)
return nil
}

View File

@@ -0,0 +1,203 @@
package app
import (
"context"
"fmt"
"io"
"os"
"path/filepath"
"sort"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
"gitea.maximumdirect.net/eric/narratio/internal/config"
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
)
type finding struct {
Severity string
Category string
Message string
}
type findingError struct {
count int
}
func (e findingError) Error() string {
return fmt.Sprintf("%d validation error(s)", e.count)
}
func renderFindings(out io.Writer, campaign, sessionID string, findings []finding) error {
if campaign != "" || sessionID != "" {
fmt.Fprintf(out, "Campaign: %s\n", campaign)
fmt.Fprintf(out, "Session: %s\n\n", sessionID)
}
errorsCount := 0
for _, f := range findings {
if f.Severity == "ERROR" {
errorsCount++
}
fmt.Fprintf(out, "%-5s %-10s %s\n", f.Severity, f.Category, f.Message)
}
if errorsCount > 0 {
return findingError{count: errorsCount}
}
return nil
}
func okFinding(category, msg string) finding { return finding{"OK", category, msg} }
func infoFinding(category, msg string) finding { return finding{"INFO", category, msg} }
func warnFinding(category, msg string) finding { return finding{"WARN", category, msg} }
func errorFinding(category, msg string) finding { return finding{"ERROR", category, msg} }
func sessionSourceSummary(cfg *config.Config) string {
source := cfg.SessionSource.Source
if source == "" {
source = "session_config"
}
if cfg.SessionSource.S3Key != "" {
return source + " " + cfg.SessionSource.S3Key
}
return source + " " + cfg.SessionPath
}
func validateStableInputFindings(cfg *config.Config) []finding {
items := []struct {
name string
in config.ResolvedInputFile
}{
{"speakers", cfg.StableInputs.SpeakersFile},
{"autocorrect", cfg.StableInputs.AutocorrectFile},
{"glossary", cfg.StableInputs.GlossaryFile},
}
out := make([]finding, 0, len(items))
for _, item := range items {
path, err := resolveHelperConfigRelativePath(item.in)
if err != nil {
out = append(out, errorFinding("inputs", item.name+": "+err.Error()))
continue
}
if _, err := os.Stat(path); err != nil {
out = append(out, errorFinding("inputs", fmt.Sprintf("%s missing: %v", item.name, err)))
} else {
out = append(out, okFinding("inputs", item.name+": "+path))
}
}
return out
}
func resolveHelperConfigRelativePath(input config.ResolvedInputFile) (string, error) {
if strings.TrimSpace(input.ConfigPath) == "" {
return "", fmt.Errorf("source config path is required")
}
path := strings.TrimSpace(input.Path)
if path == "" {
return "", fmt.Errorf("path is required")
}
if filepath.IsAbs(path) {
return filepath.Clean(path), nil
}
return filepath.Clean(filepath.Join(filepath.Dir(input.ConfigPath), path)), nil
}
func validateLocalAudioFindings(cfg *config.Config) []finding {
if cfg.Session.Inputs.AudioS3 != nil {
return nil
}
audioDir := strings.TrimSpace(cfg.Session.Inputs.AudioDir)
if audioDir == "" && len(cfg.Session.Inputs.AudioFiles) == 0 {
return []finding{errorFinding("audio", "audio_dir, audio_files, or audio_s3 is required")}
}
base := filepath.Dir(cfg.SessionPath)
paths := []string{}
if audioDir != "" {
dir := audioDir
if !filepath.IsAbs(dir) {
dir = filepath.Join(base, dir)
}
matches, err := filepath.Glob(filepath.Join(dir, "*.flac"))
if err != nil || len(matches) == 0 {
return []finding{errorFinding("audio", "no .flac files found in "+dir)}
}
paths = append(paths, matches...)
}
for _, file := range cfg.Session.Inputs.AudioFiles {
p := file
if !filepath.IsAbs(p) {
p = filepath.Join(base, p)
}
paths = append(paths, p)
}
for _, p := range paths {
if _, err := os.Stat(p); err != nil {
return []finding{errorFinding("audio", fmt.Sprintf("audio file missing: %v", err))}
}
}
return []finding{okFinding("audio", fmt.Sprintf("%d local audio file(s)", len(paths)))}
}
func validateRemoteAudioFinding(ctx context.Context, cfg *config.Config, store storage.ObjectStore) finding {
sessionPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
audioPrefix := artifacts.S3AudioPrefix(sessionPrefix, cfg.Session.Inputs.AudioS3.Prefix)
objects, err := store.List(ctx, audioPrefix)
if err != nil {
return errorFinding("audio", err.Error())
}
count := 0
for _, obj := range objects {
if strings.HasSuffix(strings.ToLower(obj.Key), ".flac") {
count++
}
}
if count == 0 {
return errorFinding("audio", "no remote .flac objects found under "+audioPrefix)
}
return okFinding("audio", fmt.Sprintf("%d remote .flac object(s)", count))
}
func validatePreviousArtifactFindings(ctx context.Context, cfg *config.Config, store storage.ObjectStore, requirements []artifacts.PreviousArtifactRequirement) []finding {
out := []finding{}
prefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.PreviousSessionID)
_, err := artifacts.LoadCurrentState(ctx, store, prefix, artifacts.CurrentStateValidation{
ExpectedSessionID: strings.TrimSpace(cfg.Session.PreviousSessionID),
ExpectedCampaign: strings.TrimSpace(cfg.Session.Campaign),
ValidateRunID: true,
})
if err != nil {
out = append(out, errorFinding("previous", fmt.Sprintf("remote %v", err)))
return out
}
for _, req := range requirements {
out = append(out, okFinding("previous", fmt.Sprintf("%s required=%t", req.Name, req.Required)))
}
return out
}
func loadLocalManifest(ctx context.Context, path string) (*manifest.Manifest, error) {
if _, err := os.Stat(path); err != nil {
if os.IsNotExist(err) {
return nil, nil
}
return nil, err
}
store := &manifest.LocalStore{}
return store.Load(ctx, path)
}
func writeStageStatuses(out io.Writer, m *manifest.Manifest) {
if m == nil || len(m.Stages) == 0 {
fmt.Fprintln(out, "stages: no stages recorded")
return
}
fmt.Fprintln(out, "stages:")
names := make([]string, 0, len(m.Stages))
for name := range m.Stages {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
fmt.Fprintf(out, "- %s: %s\n", name, m.Stages[name].Status)
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -596,7 +596,7 @@ func TestExecuteLocksRequireSessionID(t *testing.T) {
func TestExecuteLocksCannotModifyStaticLocks(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
addStaticArchiveLockToPipelineConfig(t, pipelinePath, "narratio.transcript.final_trimmed")
addStaticPublishLockToPipelineConfig(t, pipelinePath, "narratio.transcript.final_trimmed")
fake := &storage.FakeBackend{}
var storeInitCalls int
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
@@ -683,10 +683,10 @@ func addSessionTemplateToCampaign(t *testing.T, campaignPath, templateFile strin
}
}
func TestExecuteArtifactsListRemoteReportsPromotedAvailability(t *testing.T) {
func TestExecuteArtifactsListRemoteReportsPublishedAvailability(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
addArchivePromotionsToPipeline(t, pipelinePath, `
addPublishOutputsToPipeline(t, pipelinePath, `
outputs:
- source: narratio.transcript.final_trimmed
dest: transcripts/final.trimmed.json
@@ -714,14 +714,14 @@ func TestExecuteArtifactsListRemoteReportsPromotedAvailability(t *testing.T) {
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
}
if !strings.Contains(stdout.String(), "narratio.transcript.final_trimmed remote=published") {
t.Fatalf("stdout = %q, want promoted remote availability", stdout.String())
t.Fatalf("stdout = %q, want published remote availability", stdout.String())
}
}
func TestExecuteArtifactsListRemoteUsesPromotionDestinations(t *testing.T) {
func TestExecuteArtifactsListRemoteUsesPublishOutputDestinations(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
addArchivePromotionsToPipeline(t, pipelinePath, `
addPublishOutputsToPipeline(t, pipelinePath, `
outputs:
- source: narratio.transcript.final
dest: transcripts/full.json
@@ -771,7 +771,7 @@ func TestExecuteArtifactsListRemoteUsesPromotionDestinations(t *testing.T) {
func TestExecuteStatusReportsRemoteArtifactCatalog(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
addArchivePromotionsToPipeline(t, pipelinePath, `
addPublishOutputsToPipeline(t, pipelinePath, `
outputs:
- source: narratio.transcript.final_trimmed
dest: transcripts/final.trimmed.json
@@ -782,7 +782,7 @@ func TestExecuteStatusReportsRemoteArtifactCatalog(t *testing.T) {
`)
fake := &storage.FakeBackend{}
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
manifestKey, runIDKey := artifacts.ResolveCurrentStateKeys(sessionPrefix)
trimmedKey := artifacts.S3PublishedOutputKey(sessionPrefix, "transcripts/final.trimmed.json")
fullKey := artifacts.S3PublishedOutputKey(sessionPrefix, "transcripts/full.json")
lockKey := artifacts.S3SessionLocksKey(sessionPrefix)
@@ -828,7 +828,7 @@ func TestExecuteStatusReportsRemoteArtifactCatalog(t *testing.T) {
func TestExecuteStatusReportsRemoteArtifactCatalogErrorsWithoutFailing(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
addArchivePromotionsToPipeline(t, pipelinePath, `
addPublishOutputsToPipeline(t, pipelinePath, `
outputs:
- source: narratio.transcript.final_trimmed
dest: transcripts/final.trimmed.json
@@ -861,9 +861,63 @@ func TestExecuteStatusReportsRemoteArtifactCatalogErrorsWithoutFailing(t *testin
}
}
func TestExecuteArchiveLoadsRemoteLocks(t *testing.T) {
func TestExecuteStatusReportsMissingRemoteCurrentStateWithoutFailing(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, sessionPath := writeValidArchiveConfigFiles(t, workspaceRoot)
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
fake := &storage.FakeBackend{}
var storeInitCalls int
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
var stdout bytes.Buffer
var stderr bytes.Buffer
code := Execute([]string{
"session", "status", "2026-05-03",
"--config", pipelinePath,
"--campaign-file", campaignPath,
"--session", sessionPath,
}, &stdout, &stderr)
if code != 0 {
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
}
if !strings.Contains(stdout.String(), "Remote publish: missing or unavailable: remote current run pointer missing") {
t.Fatalf("stdout = %q, want missing remote current-state line", stdout.String())
}
}
func TestExecuteSessionValidateReportsPreviousStateFindingAndReturnsFindingError(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, sessionPath := writeValidConfigFilesWithScriptoriumArtifacts(t, workspaceRoot)
replaceInFileOrFatal(t, pipelinePath, "source: narratio.artifact.session_recap", "source: narratio.previous_session.artifact.session_recap")
replaceInFileOrFatal(t, sessionPath, "session_id: 2026-05-03\n", "session_id: 2026-05-03\nprevious_session_id: 2026-04-26\n")
fake := &storage.FakeBackend{}
var storeInitCalls int
restoreAppConfigTestGlobals(t, fake, &storeInitCalls, []string{sessionPath})
var stdout bytes.Buffer
var stderr bytes.Buffer
code := Execute([]string{
"session", "validate", "2026-05-03",
"--config", pipelinePath,
"--campaign-file", campaignPath,
"--session", sessionPath,
}, &stdout, &stderr)
if code == 0 {
t.Fatal("exit code = 0, want non-zero")
}
if !strings.Contains(stdout.String(), "ERROR previous") {
t.Fatalf("stdout = %q, want previous finding error", stdout.String())
}
if !strings.Contains(stdout.String(), "remote current run pointer missing") {
t.Fatalf("stdout = %q, want missing run pointer finding", stdout.String())
}
if !strings.Contains(stderr.String(), "validation error(s)") {
t.Fatalf("stderr = %q, want finding error summary", stderr.String())
}
}
func TestExecutePublishLoadsRemoteLocks(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, sessionPath := writeValidPublishRunConfigFiles(t, workspaceRoot)
fake := &storage.FakeBackend{}
lockKey := artifacts.S3SessionLocksKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"))
fake.SeedObject(storage.FakeObject{Key: lockKey, Data: []byte("locks:\n - source: narratio.transcript.final_trimmed\n reason: remote review\n")})
@@ -883,28 +937,43 @@ func TestExecuteArchiveLoadsRemoteLocks(t *testing.T) {
if code != 0 {
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
}
promotedKey := artifacts.S3PublishedOutputKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"), "transcripts/final.trimmed.json")
if _, ok := fake.Objects[promotedKey]; ok {
t.Fatalf("locked promoted key %q was uploaded", promotedKey)
publishedKey := artifacts.S3PublishedOutputKey(artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03"), "transcripts/final.trimmed.json")
if _, ok := fake.Objects[publishedKey]; ok {
t.Fatalf("locked published key %q was uploaded", publishedKey)
}
}
func addArchivePromotionsToPipeline(t *testing.T, pipelinePath, archiveYAML string) {
func addPublishOutputsToPipeline(t *testing.T, pipelinePath, publishYAML string) {
t.Helper()
data, err := os.ReadFile(pipelinePath)
if err != nil {
t.Fatalf("read pipeline: %v", err)
}
updated := strings.Replace(string(data), " upload_run: false\n", " upload_run: false\n"+archiveYAML, 1)
updated := strings.Replace(string(data), " upload_run: false\n", " upload_run: false\n"+publishYAML, 1)
if updated == string(data) {
t.Fatalf("pipeline %q did not contain archive upload_run marker", pipelinePath)
t.Fatalf("pipeline %q did not contain publish upload_run marker", pipelinePath)
}
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
t.Fatalf("write pipeline: %v", err)
}
}
func writeValidArchiveConfigFiles(t *testing.T, workspaceRoot string) (string, string, string) {
func replaceInFileOrFatal(t *testing.T, path, old, new string) {
t.Helper()
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
updated := strings.Replace(string(data), old, new, 1)
if updated == string(data) {
t.Fatalf("%s did not contain %q", path, old)
}
if err := os.WriteFile(path, []byte(updated), 0o644); err != nil {
t.Fatalf("write %s: %v", path, err)
}
}
func writeValidPublishRunConfigFiles(t *testing.T, workspaceRoot string) (string, string, string) {
t.Helper()
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
data, err := os.ReadFile(pipelinePath)
@@ -941,7 +1010,7 @@ func writeValidArchiveConfigFiles(t *testing.T, workspaceRoot string) (string, s
return pipelinePath, campaignPath, sessionPath
}
func addStaticArchiveLockToPipelineConfig(t *testing.T, pipelinePath, source string) {
func addStaticPublishLockToPipelineConfig(t *testing.T, pipelinePath, source string) {
t.Helper()
data, err := os.ReadFile(pipelinePath)
if err != nil {
@@ -954,7 +1023,7 @@ func addStaticArchiveLockToPipelineConfig(t *testing.T, pipelinePath, source str
1,
)
if updated == string(data) {
t.Fatalf("archive section not found in pipeline config")
t.Fatalf("publish section not found in pipeline config")
}
if err := os.WriteFile(pipelinePath, []byte(updated), 0o644); err != nil {
t.Fatalf("write pipeline: %v", err)

View File

@@ -0,0 +1,222 @@
package app
import (
"context"
"flag"
"fmt"
"io"
"sort"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/config"
)
// Locks dispatches publish lock list and mutation helpers.
func Locks(ctx context.Context, args []string, out io.Writer) error {
if len(args) > 0 && !strings.HasPrefix(args[0], "-") {
switch args[0] {
case "add":
return LocksAdd(ctx, args[1:], out)
case "remove":
return LocksRemove(ctx, args[1:], out)
default:
return fmt.Errorf("locks: unknown subcommand %q", args[0])
}
}
return LocksList(ctx, args, out)
}
// LocksList lists effective publish locks.
func LocksList(ctx context.Context, args []string, out io.Writer) error {
fs := flag.NewFlagSet("locks", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var flags commonConfigFlags
addCommonConfigFlags(fs, &flags)
if err := parseSessionAwareFlags("locks", fs, args, &flags.sessionID); err != nil {
return err
}
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("locks: session_id is required")
}
cfg, _, locks, _, err := loadHelperContext(ctx, flags, true)
if err != nil {
return fmt.Errorf("locks: %w", err)
}
writeLocks(out, cfg, locks)
return nil
}
// LocksAdd adds or updates one remote lock.
func LocksAdd(ctx context.Context, args []string, out io.Writer) error {
var positionalSessionID string
var source string
if len(args) >= 2 && !isCLIFlagToken(args[0]) && !isCLIFlagToken(args[1]) {
positionalSessionID = strings.TrimSpace(args[0])
source = strings.TrimSpace(args[1])
args = append([]string(nil), args[2:]...)
}
fs := flag.NewFlagSet("locks add", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var flags commonConfigFlags
var reason string
var force bool
addCommonConfigFlags(fs, &flags)
fs.StringVar(&reason, "reason", "", "lock reason")
fs.BoolVar(&force, "force", false, "update existing remote lock")
if err := fs.Parse(args); err != nil {
return fmt.Errorf("locks add: invalid flags: %w", err)
}
if source == "" {
switch fs.NArg() {
case 2:
positionalSessionID = strings.TrimSpace(fs.Arg(0))
source = strings.TrimSpace(fs.Arg(1))
case 1:
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("locks add: expected session_id and source id")
}
source = strings.TrimSpace(fs.Arg(0))
default:
return fmt.Errorf("locks add: expected session_id and source id")
}
} else if fs.NArg() != 0 {
return fmt.Errorf("locks add: unexpected positional arguments")
}
if err := applyPositionalSessionID("locks add", positionalSessionID, &flags.sessionID); err != nil {
return err
}
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("locks add: session_id is required")
}
cfg, store, locks, _, err := loadHelperContext(ctx, flags, true)
if err != nil {
return fmt.Errorf("locks add: %w", err)
}
if _, err := config.ValidatePublishLockRules([]config.PublishLockRule{{Source: source}}, cfg.Pipeline.Scriptorium, "locks add"); err != nil {
return fmt.Errorf("locks add: %w", err)
}
if _, ok := lockSourceSet(locks.Static)[source]; ok {
return fmt.Errorf("locks add: source %q is locked by pipeline config and cannot be modified remotely", source)
}
remoteSet := lockSourceSet(locks.Remote)
if _, exists := remoteSet[source]; exists && !force {
return fmt.Errorf("locks add: remote lock for %q already exists; pass --force to update", source)
}
remoteSet[source] = config.PublishLockRule{Source: source, Reason: strings.TrimSpace(reason)}
remoteLocks := lockMapValues(remoteSet)
if _, err := config.ValidatePublishLockRules(remoteLocks, cfg.Pipeline.Scriptorium, "locks"); err != nil {
return fmt.Errorf("locks add: %w", err)
}
if err := uploadRemoteLockStore(ctx, store, locks.Key, &config.PublishLockStore{Locks: remoteLocks}); err != nil {
return fmt.Errorf("locks add: %w", err)
}
_, err = fmt.Fprintf(out, "narratio session locks add: locked %s\n", source)
return err
}
// LocksRemove removes one remote lock.
func LocksRemove(ctx context.Context, args []string, out io.Writer) error {
var positionalSessionID string
var source string
if len(args) >= 2 && !isCLIFlagToken(args[0]) && !isCLIFlagToken(args[1]) {
positionalSessionID = strings.TrimSpace(args[0])
source = strings.TrimSpace(args[1])
args = append([]string(nil), args[2:]...)
}
fs := flag.NewFlagSet("locks remove", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var flags commonConfigFlags
addCommonConfigFlags(fs, &flags)
if err := fs.Parse(args); err != nil {
return fmt.Errorf("locks remove: invalid flags: %w", err)
}
if source == "" {
switch fs.NArg() {
case 2:
positionalSessionID = strings.TrimSpace(fs.Arg(0))
source = strings.TrimSpace(fs.Arg(1))
case 1:
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("locks remove: expected session_id and source id")
}
source = strings.TrimSpace(fs.Arg(0))
default:
return fmt.Errorf("locks remove: expected session_id and source id")
}
} else if fs.NArg() != 0 {
return fmt.Errorf("locks remove: unexpected positional arguments")
}
if err := applyPositionalSessionID("locks remove", positionalSessionID, &flags.sessionID); err != nil {
return err
}
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("locks remove: session_id is required")
}
cfg, store, locks, _, err := loadHelperContext(ctx, flags, true)
if err != nil {
return fmt.Errorf("locks remove: %w", err)
}
if _, err := config.ValidatePublishLockRules([]config.PublishLockRule{{Source: source}}, cfg.Pipeline.Scriptorium, "locks remove"); err != nil {
return fmt.Errorf("locks remove: %w", err)
}
remoteSet := lockSourceSet(locks.Remote)
if _, ok := remoteSet[source]; !ok {
if _, static := lockSourceSet(locks.Static)[source]; static {
return fmt.Errorf("locks remove: source %q is locked by pipeline config and cannot be unlocked remotely", source)
}
return fmt.Errorf("locks remove: remote lock for %q does not exist", source)
}
delete(remoteSet, source)
remoteLocks := lockMapValues(remoteSet)
if err := uploadRemoteLockStore(ctx, store, locks.Key, &config.PublishLockStore{Locks: remoteLocks}); err != nil {
return fmt.Errorf("locks remove: %w", err)
}
_, err = fmt.Fprintf(out, "narratio session locks remove: unlocked %s\n", source)
return err
}
func writeLocks(out io.Writer, cfg *config.Config, locks *effectiveLocks) {
if locks == nil || len(locks.All) == 0 {
fmt.Fprintln(out, "Publish locks: none")
return
}
fmt.Fprintln(out, "Publish locks:")
published := map[string]config.PublishOutputRule{}
if cfg != nil && cfg.Pipeline != nil && cfg.Pipeline.Publish != nil {
for _, rule := range cfg.Pipeline.Publish.Outputs {
published[strings.TrimSpace(rule.Source)] = rule
}
}
staticSet := lockSourceSet(locks.Static)
for _, lock := range locks.All {
origin := "remote"
if _, ok := staticSet[lock.Source]; ok {
origin = "pipeline"
}
promo := "not-published"
if _, ok := published[lock.Source]; ok {
promo = "published"
}
reason := strings.TrimSpace(lock.Reason)
if reason == "" {
reason = "(no reason)"
}
fmt.Fprintf(out, "- %s origin=%s %s reason=%s\n", lock.Source, origin, promo, reason)
}
}
func lockMapValues(in map[string]config.PublishLockRule) []config.PublishLockRule {
keys := make([]string, 0, len(in))
for key := range in {
keys = append(keys, key)
}
sort.Strings(keys)
out := make([]config.PublishLockRule, 0, len(keys))
for _, key := range keys {
item := in[key]
item.Source = key
item.Reason = strings.TrimSpace(item.Reason)
out = append(out, item)
}
return out
}

View File

@@ -0,0 +1,268 @@
package app
import (
"context"
"flag"
"fmt"
"io"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
"gitea.maximumdirect.net/eric/narratio/internal/config"
"gopkg.in/yaml.v3"
)
// SessionInit creates a local or remote session.yml skeleton.
func SessionInit(ctx context.Context, args []string, out io.Writer) error {
fs := flag.NewFlagSet("session init", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var pipelinePath, campaignPath, campaignFilePath, sessionID, previousSessionID, date, title, output, audioS3Prefix, audioDir string
var remote, force bool
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
fs.StringVar(&sessionID, "session-id", "", "session identifier")
fs.StringVar(&previousSessionID, "previous-session-id", "", "previous session identifier")
fs.StringVar(&date, "date", "", "session date")
fs.StringVar(&title, "title", "", "session title")
fs.StringVar(&output, "output", "", "local output session.yml path")
fs.StringVar(&audioS3Prefix, "audio-s3-prefix", "", "session audio S3 prefix")
fs.StringVar(&audioDir, "audio-dir", "", "local audio directory")
fs.BoolVar(&remote, "remote", false, "write session.yml to S3 session prefix")
fs.BoolVar(&force, "force", false, "overwrite existing target")
if err := parseSessionAwareFlags("session init", fs, args, &sessionID); err != nil {
return err
}
if strings.TrimSpace(sessionID) == "" {
return fmt.Errorf("session init: session_id is required")
}
if (strings.TrimSpace(output) == "") == !remote {
return fmt.Errorf("session init: specify exactly one target: --output <path> or --remote")
}
if strings.TrimSpace(audioDir) != "" && strings.TrimSpace(audioS3Prefix) != "" {
return fmt.Errorf("session init: --audio-dir and --audio-s3-prefix are mutually exclusive")
}
base, err := loadPipelineCampaignConfig(pipelinePath, campaignPath, campaignFilePath)
if err != nil {
return fmt.Errorf("session init: %w", err)
}
input := sessionInitInput{
Campaign: config.CampaignID(base.Campaign),
CampaignPath: base.CampaignPath,
TemplateFile: base.Campaign.SessionTemplateFile,
SessionID: sessionID,
PreviousSessionID: previousSessionID,
Date: date,
Title: title,
AudioS3Prefix: audioS3Prefix,
AudioDir: audioDir,
}
data, err := buildSessionInitYAML(input)
if err != nil {
return fmt.Errorf("session init: %w", err)
}
label := strings.TrimSpace(output)
if label == "" {
label = "remote session.yml"
}
sessionCfg, err := config.LoadSessionBytesWithOptions(label, data, config.SessionLoadOptions{
SessionID: sessionID,
PreviousSessionID: previousSessionID,
})
if err != nil {
return fmt.Errorf("session init: %w", err)
}
cfg, err := config.Resolve(base.PipelinePath, base.Pipeline, base.CampaignPath, base.Campaign, label, sessionCfg, config.SessionSource{Source: "session_config", LocalPath: label})
if err != nil {
return fmt.Errorf("session init: %w", err)
}
if err := config.Validate(cfg); err != nil {
return fmt.Errorf("session init: %w", err)
}
if !remote {
if err := writeLocalFile(output, data, force); err != nil {
return fmt.Errorf("session init: %w", err)
}
_, err := fmt.Fprintf(out, "narratio session init: wrote %s\n", filepath.Clean(output))
return err
}
store, err := newCommandObjectStore(ctx, cfg, nil)
if err != nil {
return fmt.Errorf("session init: %w", err)
}
sessionPrefix := artifacts.S3SessionPrefix(base.Pipeline.Storage.S3.RootPrefix, config.CampaignID(base.Campaign), sessionID)
key := artifacts.S3SessionConfigKey(sessionPrefix)
exists, err := store.Exists(ctx, key)
if err != nil {
return fmt.Errorf("session init: check remote session %q: %w", key, err)
}
if exists && !force {
return fmt.Errorf("session init: remote session %q already exists; pass --force to overwrite", key)
}
tmp, err := os.CreateTemp("", "narratio-session-init-*.yml")
if err != nil {
return fmt.Errorf("session init: create temp file: %w", err)
}
tmpPath := tmp.Name()
defer func() { _ = os.Remove(tmpPath) }()
if _, err := tmp.Write(data); err != nil {
_ = tmp.Close()
return fmt.Errorf("session init: write temp file: %w", err)
}
if err := tmp.Close(); err != nil {
return fmt.Errorf("session init: close temp file: %w", err)
}
if _, err := store.Upload(ctx, tmpPath, key, storage.UploadOptions{ContentType: "application/x-yaml; charset=utf-8"}); err != nil {
return fmt.Errorf("session init: upload remote session %q: %w", key, err)
}
_, err = fmt.Fprintf(out, "narratio session init: wrote s3://%s/%s\n", s3BucketName(base.Pipeline), key)
return err
}
func buildSessionYAML(campaign, sessionID, previousSessionID, date, title, audioS3Prefix, audioDir string) ([]byte, error) {
if strings.TrimSpace(date) == "" && regexp.MustCompile(`^\d{4}-\d{2}-\d{2}$`).MatchString(strings.TrimSpace(sessionID)) {
date = strings.TrimSpace(sessionID)
}
type audioS3 struct {
Prefix string `yaml:"prefix"`
}
type inputs struct {
AudioDir string `yaml:"audio_dir,omitempty"`
AudioS3 *audioS3 `yaml:"audio_s3,omitempty"`
}
type sessionYAML struct {
Campaign string `yaml:"campaign"`
SessionID string `yaml:"session_id"`
PreviousSessionID string `yaml:"previous_session_id,omitempty"`
Date string `yaml:"date,omitempty"`
Title string `yaml:"title,omitempty"`
Inputs inputs `yaml:"inputs"`
}
in := inputs{AudioDir: strings.TrimSpace(audioDir)}
if in.AudioDir == "" {
prefix := strings.TrimSpace(audioS3Prefix)
if prefix == "" {
prefix = "audio/"
}
in.AudioS3 = &audioS3{Prefix: prefix}
}
data, err := yaml.Marshal(sessionYAML{
Campaign: strings.TrimSpace(campaign),
SessionID: strings.TrimSpace(sessionID),
PreviousSessionID: strings.TrimSpace(previousSessionID),
Date: strings.TrimSpace(date),
Title: strings.TrimSpace(title),
Inputs: in,
})
if err != nil {
return nil, err
}
return data, nil
}
type sessionInitInput struct {
Campaign string
CampaignPath string
TemplateFile string
SessionID string
PreviousSessionID string
Date string
Title string
AudioS3Prefix string
AudioDir string
}
func buildSessionInitYAML(in sessionInitInput) ([]byte, error) {
if strings.TrimSpace(in.TemplateFile) == "" {
return buildSessionYAML(in.Campaign, in.SessionID, in.PreviousSessionID, in.Date, in.Title, in.AudioS3Prefix, in.AudioDir)
}
templatePath := resolveSessionInitTemplatePath(in.CampaignPath, in.TemplateFile)
templateBytes, err := os.ReadFile(templatePath)
if err != nil {
return nil, fmt.Errorf("read session template %q: %w", templatePath, err)
}
rendered, err := renderSessionInitTemplate(string(templateBytes), in)
if err != nil {
return nil, fmt.Errorf("render session template %q: %w", templatePath, err)
}
return []byte(rendered), nil
}
func resolveSessionInitTemplatePath(campaignPath, templateFile string) string {
templateFile = strings.TrimSpace(templateFile)
if filepath.IsAbs(templateFile) {
return filepath.Clean(templateFile)
}
return filepath.Clean(filepath.Join(filepath.Dir(campaignPath), templateFile))
}
var sessionInitTemplatePattern = regexp.MustCompile(`\{\{\s*([a-zA-Z_][a-zA-Z0-9_]*)\s*\}\}`)
func renderSessionInitTemplate(content string, in sessionInitInput) (string, error) {
values := map[string]string{
"session_id": strings.TrimSpace(in.SessionID),
"previous_session_id": strings.TrimSpace(in.PreviousSessionID),
"date": strings.TrimSpace(in.Date),
"title": strings.TrimSpace(in.Title),
"audio_s3_prefix": strings.TrimSpace(in.AudioS3Prefix),
"audio_dir": strings.TrimSpace(in.AudioDir),
}
used := map[string]struct{}{}
unknown := map[string]struct{}{}
missing := map[string]struct{}{}
rendered := sessionInitTemplatePattern.ReplaceAllStringFunc(content, func(match string) string {
parts := sessionInitTemplatePattern.FindStringSubmatch(match)
if len(parts) < 2 {
return match
}
name := parts[1]
value, ok := values[name]
if !ok {
unknown[name] = struct{}{}
return match
}
used[name] = struct{}{}
if value == "" {
missing[name] = struct{}{}
return match
}
return value
})
if len(unknown) > 0 {
return "", fmt.Errorf("unsupported template variable(s): %s", sortedStringSet(unknown))
}
if len(missing) > 0 {
return "", fmt.Errorf("missing required template variable value(s): %s", sortedStringSet(missing))
}
unused := map[string]struct{}{}
for _, name := range []string{"previous_session_id", "date", "title", "audio_s3_prefix", "audio_dir"} {
if values[name] == "" {
continue
}
if _, ok := used[name]; !ok {
unused[name] = struct{}{}
}
}
if len(unused) > 0 {
return "", fmt.Errorf("unused template variable value(s): %s", sortedStringSet(unused))
}
return rendered, nil
}
func sortedStringSet(set map[string]struct{}) string {
items := make([]string, 0, len(set))
for item := range set {
items = append(items, item)
}
sort.Strings(items)
return strings.Join(items, ", ")
}

View File

@@ -0,0 +1,81 @@
package app
import (
"context"
"flag"
"fmt"
"io"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
"gitea.maximumdirect.net/eric/narratio/internal/config"
)
// SessionValidate performs a read-only session preflight.
func SessionValidate(ctx context.Context, args []string, out io.Writer) error {
fs := flag.NewFlagSet("session validate", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var flags commonConfigFlags
addCommonConfigFlags(fs, &flags)
if err := parseSessionAwareFlags("session validate", fs, args, &flags.sessionID); err != nil {
return err
}
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("session validate: session_id is required")
}
findings := []finding{}
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
if err != nil {
findings = append(findings, errorFinding("config", err.Error()))
return renderFindings(out, "", "", findings)
}
if err := config.Validate(cfg); err != nil {
findings = append(findings, errorFinding("config", err.Error()))
} else {
findings = append(findings, okFinding("config", "resolved pipeline, campaign, and session config"))
}
findings = append(findings, okFinding("session", fmt.Sprintf("session source: %s", sessionSourceSummary(cfg))))
paths := artifacts.NewLocalStore(cfg.Pipeline.Workspace.Root).SessionPathsFor(cfg.Session.Campaign, cfg.Session.SessionID)
findings = append(findings, validateStableInputFindings(cfg)...)
findings = append(findings, validateLocalAudioFindings(cfg)...)
store, storeErr := objectStoreIfConfigured(ctx, cfg)
if storeErr != nil {
findings = append(findings, errorFinding("storage", storeErr.Error()))
}
if cfg.Session.Inputs.AudioS3 != nil {
if storeErr != nil {
findings = append(findings, errorFinding("audio", "remote audio cannot be checked because storage is unavailable"))
} else {
findings = append(findings, validateRemoteAudioFinding(ctx, cfg, store))
}
}
requirements := artifacts.CollectPreviousArtifactRequirements(configuredScriptoriumArtifacts(cfg))
if len(requirements) == 0 {
findings = append(findings, okFinding("previous", "no previous-session artifacts required"))
} else if strings.TrimSpace(cfg.Session.PreviousSessionID) == "" {
findings = append(findings, errorFinding("previous", "previous_session_id is required by configured previous-session artifacts"))
} else if storeErr != nil {
findings = append(findings, errorFinding("previous", "previous-session artifacts cannot be checked because storage is unavailable"))
} else {
findings = append(findings, validatePreviousArtifactFindings(ctx, cfg, store, requirements)...)
}
locks, lockErr := loadEffectiveLocks(ctx, cfg, store)
if lockErr != nil {
findings = append(findings, errorFinding("locks", lockErr.Error()))
} else if len(locks.All) == 0 {
findings = append(findings, okFinding("locks", "no effective publish locks"))
} else {
for _, lock := range locks.All {
findings = append(findings, warnFinding("locks", fmt.Sprintf("%s locked: %s", lock.Source, strings.TrimSpace(lock.Reason))))
}
}
if paths.ManifestPath != "" {
findings = append(findings, infoFinding("workspace", "manifest path: "+paths.ManifestPath))
}
return renderFindings(out, cfg.Session.Campaign, cfg.Session.SessionID, findings)
}

View File

@@ -0,0 +1,88 @@
package app
import (
"context"
"flag"
"fmt"
"io"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
"gitea.maximumdirect.net/eric/narratio/internal/config"
)
// Status reports effective local/remote session state.
func Status(ctx context.Context, args []string, out io.Writer) error {
fs := flag.NewFlagSet("status", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var flags commonConfigFlags
addCommonConfigFlags(fs, &flags)
if err := parseSessionAwareFlags("status", fs, args, &flags.sessionID); err != nil {
return err
}
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("status: session_id is required")
}
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
if err != nil {
return fmt.Errorf("status: %w", err)
}
if err := config.Validate(cfg); err != nil {
return fmt.Errorf("status: %w", err)
}
paths := artifacts.NewLocalStore(cfg.Pipeline.Workspace.Root).SessionPathsFor(cfg.Session.Campaign, cfg.Session.SessionID)
fmt.Fprintf(out, "Session: %s\n", cfg.Session.SessionID)
fmt.Fprintf(out, "Campaign: %s\n", cfg.Session.Campaign)
fmt.Fprintf(out, "Workspace: %s\n", paths.Root)
fmt.Fprintf(out, "Session config: %s\n", sessionSourceSummary(cfg))
if m, err := loadLocalManifest(ctx, paths.ManifestPath); err != nil {
fmt.Fprintf(out, "Local manifest: error: %v\n", err)
} else if m == nil {
fmt.Fprintln(out, "Local manifest: missing")
} else {
fmt.Fprintf(out, "Local manifest: %s\n", paths.ManifestPath)
writeStageStatuses(out, m)
}
store, storeErr := objectStoreIfConfigured(ctx, cfg)
if storeErr != nil {
fmt.Fprintf(out, "Remote publish: unavailable: %v\n", storeErr)
} else if store != nil {
current, err := discoverRemoteCurrentStateFn(ctx, cfg, store)
if err != nil {
fmt.Fprintf(out, "Remote publish: missing or unavailable: %v\n", err)
} else {
fmt.Fprintf(out, "Remote publish: current run %s\n", current.RunID)
fmt.Fprintf(out, "Remote manifest: %s\n", current.CurrentManifestKey)
}
}
locks, err := loadEffectiveLocks(ctx, cfg, store)
if catalog, catalogErr := buildHelperArtifactCatalog(cfg); catalogErr != nil {
fmt.Fprintf(out, "Remote outputs: error: %v\n", catalogErr)
} else if storeErr == nil {
catalogLocks := locks
if err != nil {
catalogLocks = &effectiveLocks{
Static: staticPublishLocks(cfg),
All: staticPublishLocks(cfg),
}
}
publishedRemoteState := map[string]string{}
if store != nil {
publishedRemoteState = remotePublishedOutputAvailability(ctx, cfg, store, catalog)
}
fmt.Fprintln(out, "Remote outputs:")
writeArtifactList(out, cfg, catalog, catalogLocks, publishedRemoteState)
}
if err != nil {
fmt.Fprintf(out, "Publish locks: error: %v\n", err)
} else {
writeLocks(out, cfg, locks)
}
fmt.Fprintln(out, "Next actions:")
fmt.Fprintf(out, "- narratio session validate %s\n", cfg.Session.SessionID)
fmt.Fprintf(out, "- narratio session restore %s --dry-run\n", cfg.Session.SessionID)
return nil
}

View File

@@ -7,7 +7,6 @@ import (
"io"
"log/slog"
"os"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
"gitea.maximumdirect.net/eric/narratio/internal/config"
@@ -17,46 +16,21 @@ import (
// Plan validates configuration, prepares the local workdir, and prints stage order.
func Plan(ctx context.Context, args []string, out io.Writer) error {
positionalSessionID, args := pullLeadingSessionID(args)
fs := flag.NewFlagSet("plan", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var pipelinePath string
var campaignPath string
var campaignFilePath string
var sessionPath string
var sessionID string
var previousSessionID string
var flags commonConfigFlags
var force bool
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
addCommonConfigFlags(fs, &flags)
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
if err := fs.Parse(args); err != nil {
return fmt.Errorf("plan: invalid flags: %w", err)
if err := parseSessionAwareFlags("plan", fs, args, &flags.sessionID); err != nil {
return err
}
if positionalSessionID == "" {
if err := applyParsedSessionIDArg("plan", fs, &sessionID); err != nil {
return err
}
} else {
if fs.NArg() != 0 {
return fmt.Errorf("plan: unexpected positional arguments")
}
if err := applyPositionalSessionID("plan", positionalSessionID, &sessionID); err != nil {
return err
}
}
if strings.TrimSpace(sessionID) == "" {
if flags.sessionID == "" {
return fmt.Errorf("plan: session_id is required")
}
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
SessionID: sessionID,
PreviousSessionID: previousSessionID,
})
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
if err != nil {
return fmt.Errorf("plan: %w", err)
}

View File

@@ -12,7 +12,7 @@ import (
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
)
func runPostArchiveCleanup(ctx context.Context, env *Env, manifestPath string, m *manifest.Manifest, executed []string) error {
func runPostPublishCleanup(ctx context.Context, env *Env, manifestPath string, m *manifest.Manifest, executed []string) error {
if env == nil || env.Config == nil || env.Config.Pipeline == nil || m == nil {
return nil
}
@@ -23,7 +23,7 @@ func runPostArchiveCleanup(ctx context.Context, env *Env, manifestPath string, m
return nil
}
sr := archiveStageRecordForCleanup(m, executed)
sr := publishStageRecordForCleanup(m, executed)
if sr == nil {
return nil
}
@@ -33,7 +33,7 @@ func runPostArchiveCleanup(ctx context.Context, env *Env, manifestPath string, m
sr.Metadata["spool_cleanup_requested"] = spoolRequested
sr.Metadata["workdir_cleanup_requested"] = workRequested
eligible, reason := archiveCleanupEligible(env.Config, sr)
eligible, reason := publishCleanupEligible(env.Config, sr)
if !eligible {
sr.Metadata["cleanup_skipped"] = true
sr.Metadata["cleanup_skipped_reason"] = reason
@@ -96,7 +96,7 @@ func runPostArchiveCleanup(ctx context.Context, env *Env, manifestPath string, m
return nil
}
func archiveStageRecordForCleanup(m *manifest.Manifest, executed []string) *manifest.StageRecord {
func publishStageRecordForCleanup(m *manifest.Manifest, executed []string) *manifest.StageRecord {
if m == nil {
return nil
}
@@ -117,7 +117,7 @@ func archiveStageRecordForCleanup(m *manifest.Manifest, executed []string) *mani
return sr
}
func archiveCleanupEligible(cfg *config.Config, sr *manifest.StageRecord) (bool, string) {
func publishCleanupEligible(cfg *config.Config, sr *manifest.StageRecord) (bool, string) {
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Publish == nil {
return false, "publish configuration is missing"
}
@@ -174,49 +174,7 @@ func removeRunScopedDir(root, target, policy string) error {
}
func validateScopedDir(root, target, policy string) (scopedDir, error) {
cleanRoot := strings.TrimSpace(root)
cleanTarget := strings.TrimSpace(target)
if cleanRoot == "" {
return scopedDir{}, fmt.Errorf("cleanup policy %s: root path is required", policy)
}
if cleanTarget == "" {
return scopedDir{}, fmt.Errorf("cleanup policy %s: target path is required", policy)
}
rootAbs, err := filepath.Abs(cleanRoot)
if err != nil {
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve root %q: %w", policy, cleanRoot, err)
}
targetAbs, err := filepath.Abs(cleanTarget)
if err != nil {
return scopedDir{}, fmt.Errorf("cleanup policy %s: resolve target %q: %w", policy, cleanTarget, err)
}
rel, err := filepath.Rel(rootAbs, targetAbs)
if err != nil {
return scopedDir{}, fmt.Errorf("cleanup policy %s: relative path from %q to %q: %w", policy, rootAbs, targetAbs, err)
}
if rel == "." {
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete root directory %q", policy, rootAbs)
}
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete path outside root: root=%q target=%q", policy, rootAbs, targetAbs)
}
info, err := os.Lstat(targetAbs)
if err != nil {
if os.IsNotExist(err) {
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: false}, nil
}
return scopedDir{}, fmt.Errorf("cleanup policy %s: stat target %q: %w", policy, targetAbs, err)
}
if info.Mode()&os.ModeSymlink != 0 {
return scopedDir{}, fmt.Errorf("cleanup policy %s: refusing to delete symlink path %q", policy, targetAbs)
}
if !info.IsDir() {
return scopedDir{}, fmt.Errorf("cleanup policy %s: target %q is not a directory", policy, targetAbs)
}
return scopedDir{RootAbs: rootAbs, TargetAbs: targetAbs, Exists: true}, nil
return validateScopedTarget(root, target, policy, true)
}
func asString(v any) string {

View File

@@ -16,13 +16,13 @@ import (
"gitea.maximumdirect.net/eric/narratio/internal/stage"
)
type archiveSuccessStage struct {
type publishSuccessStage struct {
metadata map[string]any
}
func (archiveSuccessStage) Name() string { return "publish" }
func (archiveSuccessStage) Declares() stage.IODecl { return stage.IODecl{} }
func (s archiveSuccessStage) Run(_ context.Context, _ *stage.Env, _ *manifest.Manifest) (*stage.StageResult, error) {
func (publishSuccessStage) Name() string { return "publish" }
func (publishSuccessStage) Declares() stage.IODecl { return stage.IODecl{} }
func (s publishSuccessStage) Run(_ context.Context, _ *stage.Env, _ *manifest.Manifest) (*stage.StageResult, error) {
md := map[string]any{
"stage": "publish",
"uploaded": true,
@@ -43,12 +43,12 @@ func (notifyFailStage) Run(_ context.Context, _ *stage.Env, _ *manifest.Manifest
return nil, errors.New("notify failed")
}
func TestPostArchiveCleanupDisabledKeepsLocalDirs(t *testing.T) {
func TestPostPublishCleanupDisabledKeepsLocalDirs(t *testing.T) {
cfg, seed := cleanupFixtureConfig(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = false
cfg.Pipeline.Workspace.CleanupAfterPublish = false
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
t.Fatalf("executeStages() error = %v", err)
}
@@ -57,12 +57,12 @@ func TestPostArchiveCleanupDisabledKeepsLocalDirs(t *testing.T) {
assertExists(t, seed.localSourceAudio)
}
func TestPostArchiveCleanupSpoolOnly(t *testing.T) {
func TestPostPublishCleanupSpoolOnly(t *testing.T) {
cfg, seed := cleanupFixtureConfig(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
cfg.Pipeline.Workspace.CleanupAfterPublish = false
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
t.Fatalf("executeStages() error = %v", err)
}
@@ -71,12 +71,12 @@ func TestPostArchiveCleanupSpoolOnly(t *testing.T) {
assertExists(t, seed.localSourceAudio)
}
func TestPostArchiveCleanupWorkdirOnly(t *testing.T) {
func TestPostPublishCleanupWorkdirOnly(t *testing.T) {
cfg, seed := cleanupFixtureConfig(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = false
cfg.Pipeline.Workspace.CleanupAfterPublish = true
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
t.Fatalf("executeStages() error = %v", err)
}
@@ -87,12 +87,12 @@ func TestPostArchiveCleanupWorkdirOnly(t *testing.T) {
assertExists(t, seed.spoolAudioDir)
}
func TestPostArchiveCleanupBothPolicies(t *testing.T) {
func TestPostPublishCleanupBothPolicies(t *testing.T) {
cfg, seed := cleanupFixtureConfig(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
cfg.Pipeline.Workspace.CleanupAfterPublish = true
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
t.Fatalf("executeStages() error = %v", err)
}
@@ -102,12 +102,12 @@ func TestPostArchiveCleanupBothPolicies(t *testing.T) {
assertExists(t, seed.previousCachePath)
}
func TestPostArchiveCleanupNotRunWhenArchiveFails(t *testing.T) {
func TestPostPublishCleanupNotRunWhenPublishFails(t *testing.T) {
cfg, seed := cleanupFixtureConfig(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
cfg.Pipeline.Workspace.CleanupAfterPublish = true
_, err := executeStages(context.Background(), cfg, []stage.Stage{failingStage{name: "publish", err: errors.New("archive failed")}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
_, err := executeStages(context.Background(), cfg, []stage.Stage{failingStage{name: "publish", err: errors.New("publish failed")}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
if err == nil || !strings.Contains(err.Error(), "stage \"publish\" failed") {
t.Fatalf("executeStages() error = %v, want publish failure", err)
}
@@ -116,12 +116,12 @@ func TestPostArchiveCleanupNotRunWhenArchiveFails(t *testing.T) {
assertExists(t, seed.runWorkDir)
}
func TestPostArchiveCleanupNotRunWhenArchiveSkipped(t *testing.T) {
func TestPostPublishCleanupNotRunWhenPublishSkipped(t *testing.T) {
cfg, seed := cleanupFixtureConfig(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
cfg.Pipeline.Workspace.CleanupAfterPublish = true
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{metadata: map[string]any{"skipped": true}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{metadata: map[string]any{"skipped": true}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
t.Fatalf("executeStages() error = %v", err)
}
@@ -129,12 +129,12 @@ func TestPostArchiveCleanupNotRunWhenArchiveSkipped(t *testing.T) {
assertExists(t, seed.runWorkDir)
}
func TestPostArchiveCleanupNotRunWhenCurrentPointerMissing(t *testing.T) {
func TestPostPublishCleanupNotRunWhenCurrentPointerMissing(t *testing.T) {
cfg, seed := cleanupFixtureConfig(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
cfg.Pipeline.Workspace.CleanupAfterPublish = true
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{metadata: map[string]any{"current_pointer_written": false}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{metadata: map[string]any{"current_pointer_written": false}}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
t.Fatalf("executeStages() error = %v", err)
}
@@ -142,13 +142,13 @@ func TestPostArchiveCleanupNotRunWhenCurrentPointerMissing(t *testing.T) {
assertExists(t, seed.runWorkDir)
}
func TestPostArchiveCleanupNotRunWhenArchiveUploadDisabled(t *testing.T) {
func TestPostPublishCleanupNotRunWhenPublishUploadDisabled(t *testing.T) {
cfg, seed := cleanupFixtureConfig(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
cfg.Pipeline.Workspace.CleanupAfterPublish = true
cfg.Pipeline.Publish.UploadRun = boolPtr(false)
if _, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
if _, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}}); err != nil {
t.Fatalf("executeStages() error = %v", err)
}
@@ -156,12 +156,12 @@ func TestPostArchiveCleanupNotRunWhenArchiveUploadDisabled(t *testing.T) {
assertExists(t, seed.runWorkDir)
}
func TestPostArchiveCleanupWaitsUntilAllStagesSucceed(t *testing.T) {
func TestPostPublishCleanupWaitsUntilAllStagesSucceed(t *testing.T) {
cfg, seed := cleanupFixtureConfig(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
cfg.Pipeline.Workspace.CleanupAfterPublish = true
_, err := executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}, notifyFailStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
_, err := executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}, notifyFailStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
if err == nil || !strings.Contains(err.Error(), "stage \"notify\" failed") {
t.Fatalf("executeStages() error = %v, want notify failure", err)
}
@@ -170,7 +170,7 @@ func TestPostArchiveCleanupWaitsUntilAllStagesSucceed(t *testing.T) {
assertExists(t, seed.runWorkDir)
}
func TestPostArchiveCleanupFailsOnUnsafePath(t *testing.T) {
func TestPostPublishCleanupFailsOnUnsafePath(t *testing.T) {
cfg, _ := cleanupFixtureConfig(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
cfg.Pipeline.Workspace.CleanupAfterPublish = false
@@ -186,25 +186,25 @@ func TestPostArchiveCleanupFailsOnUnsafePath(t *testing.T) {
t.Fatalf("Save() error = %v", err)
}
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
_, err = executeStages(context.Background(), cfg, []stage.Stage{publishSuccessStage{}}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
if err == nil || !strings.Contains(err.Error(), "refusing to delete path outside root") {
t.Fatalf("executeStages() error = %v, want safe-path failure", err)
}
}
func TestPostArchiveCleanupNotRunWhenPromotionIsMissing(t *testing.T) {
cfg, seed, runID := archiveStageCleanupFixture(t)
func TestPostPublishCleanupNotRunWhenOutputIsMissing(t *testing.T) {
cfg, seed, runID := publishStageCleanupFixture(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
cfg.Pipeline.Workspace.CleanupAfterPublish = true
cfg.Pipeline.Publish.Outputs = []config.PublishOutputRule{
{Source: "narratio.transcript.base", Dest: "transcripts/base.json", Required: boolPtr(true)},
}
archiveStageImpl, err := stage.Select("publish")
publishStageImpl, err := stage.Select("publish")
if err != nil {
t.Fatalf("Select(publish) error = %v", err)
}
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
_, err = executeStages(context.Background(), cfg, []stage.Stage{publishStageImpl}, RunOptions{Env: &Env{ObjectStore: &storage.FakeBackend{}}})
if err == nil || !strings.Contains(err.Error(), "required output source unavailable") {
t.Fatalf("executeStages() error = %v, want required output source unavailable failure", err)
}
@@ -215,17 +215,17 @@ func TestPostArchiveCleanupNotRunWhenPromotionIsMissing(t *testing.T) {
assertExists(t, artifacts.SessionRunRootForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID, runID))
}
func TestPostArchiveCleanupNotRunWhenCurrentManifestUploadFails(t *testing.T) {
cfg, seed, _ := archiveStageCleanupFixture(t)
func TestPostPublishCleanupNotRunWhenCurrentManifestUploadFails(t *testing.T) {
cfg, seed, _ := publishStageCleanupFixture(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
cfg.Pipeline.Workspace.CleanupAfterPublish = true
failKey := seed.sessionPrefix + "current/manifest.json"
archiveStageImpl, err := stage.Select("publish")
publishStageImpl, err := stage.Select("publish")
if err != nil {
t.Fatalf("Select(publish) error = %v", err)
}
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{
_, err = executeStages(context.Background(), cfg, []stage.Stage{publishStageImpl}, RunOptions{
Env: &Env{ObjectStore: &failKeyStore{delegate: &storage.FakeBackend{}, failKey: failKey}},
})
if err == nil || !strings.Contains(err.Error(), "current manifest") {
@@ -236,17 +236,17 @@ func TestPostArchiveCleanupNotRunWhenCurrentManifestUploadFails(t *testing.T) {
assertExists(t, seed.runWorkDir)
}
func TestPostArchiveCleanupNotRunWhenCurrentPointerUploadFails(t *testing.T) {
cfg, seed, _ := archiveStageCleanupFixture(t)
func TestPostPublishCleanupNotRunWhenCurrentPointerUploadFails(t *testing.T) {
cfg, seed, _ := publishStageCleanupFixture(t)
cfg.Pipeline.Spool.DeleteAudioAfterPublish = true
cfg.Pipeline.Workspace.CleanupAfterPublish = true
failKey := seed.sessionPrefix + "current/run_id.txt"
archiveStageImpl, err := stage.Select("publish")
publishStageImpl, err := stage.Select("publish")
if err != nil {
t.Fatalf("Select(publish) error = %v", err)
}
_, err = executeStages(context.Background(), cfg, []stage.Stage{archiveStageImpl}, RunOptions{
_, err = executeStages(context.Background(), cfg, []stage.Stage{publishStageImpl}, RunOptions{
Env: &Env{ObjectStore: &failKeyStore{delegate: &storage.FakeBackend{}, failKey: failKey}},
})
if err == nil || !strings.Contains(err.Error(), "current run pointer") {
@@ -320,7 +320,7 @@ func cleanupFixtureConfig(t *testing.T) (*config.Config, cleanupSeed) {
}
}
func archiveStageCleanupFixture(t *testing.T) (*config.Config, cleanupSeed, string) {
func publishStageCleanupFixture(t *testing.T) (*config.Config, cleanupSeed, string) {
t.Helper()
cfg, seed := cleanupFixtureConfig(t)
@@ -344,7 +344,7 @@ func archiveStageCleanupFixture(t *testing.T) (*config.Config, cleanupSeed, stri
},
},
}
writeArchiveFixtureRunFiles(
writePublishFixtureRunFiles(
t,
seed.runWorkDir,
artifacts.SessionWorkDirForCampaign(cfg.Pipeline.Workspace.Root, cfg.Session.Campaign, cfg.Session.SessionID),
@@ -367,7 +367,7 @@ func archiveStageCleanupFixture(t *testing.T) (*config.Config, cleanupSeed, stri
return cfg, seed, runID
}
func writeArchiveFixtureRunFiles(t *testing.T, runWorkDir, sessionRoot string) {
func writePublishFixtureRunFiles(t *testing.T, runWorkDir, sessionRoot string) {
t.Helper()
mustWriteFile(t, filepath.Join(runWorkDir, "prepare", "inputs", "session.yml"), "session_id: 2026-05-03\n")
mustWriteFile(t, filepath.Join(runWorkDir, "transcribe", "outputs", "transcripts", "raw", "speaker.json"), "{}\n")

View File

@@ -46,7 +46,7 @@ func loadRemoteLockStore(ctx context.Context, cfg *config.Config, store storage.
if !exists {
return &config.PublishLockStore{}, key, nil
}
tmp, err := downloadObjectToTemp(ctx, store, key, "narratio-locks-*.yml")
tmp, err := storage.DownloadObjectToTemp(ctx, store, key, "narratio-locks-*.yml")
if err != nil {
return nil, key, fmt.Errorf("download remote locks %q: %w", key, err)
}
@@ -63,7 +63,7 @@ func loadRemoteLockStore(ctx context.Context, cfg *config.Config, store storage.
}
func loadEffectiveLocks(ctx context.Context, cfg *config.Config, store storage.ObjectStore) (*effectiveLocks, error) {
staticLocks := staticArchiveLocks(cfg)
staticLocks := staticPublishLocks(cfg)
if store == nil {
return &effectiveLocks{
Static: staticLocks,
@@ -83,7 +83,7 @@ func loadEffectiveLocks(ctx context.Context, cfg *config.Config, store storage.O
}, nil
}
func staticArchiveLocks(cfg *config.Config) []config.PublishLockRule {
func staticPublishLocks(cfg *config.Config) []config.PublishLockRule {
if cfg == nil || cfg.Pipeline == nil || cfg.Pipeline.Publish == nil {
return nil
}

View File

@@ -27,20 +27,11 @@ func Restore(ctx context.Context, args []string, out io.Writer) error {
fs := flag.NewFlagSet("restore", flag.ContinueOnError)
fs.SetOutput(out)
var pipelinePath string
var campaignPath string
var campaignFilePath string
var sessionPath string
var sessionID string
var previousSessionID string
var flags commonConfigFlags
var dryRun bool
var force bool
var includeAudio bool
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
addCommonConfigFlags(fs, &flags)
fs.BoolVar(&dryRun, "dry-run", false, "plan restore actions without writing local files")
fs.BoolVar(&force, "force", false, "overwrite local conflicts with remote state")
fs.BoolVar(&includeAudio, "include-audio", false, "include archived session-level audio objects")
@@ -57,25 +48,13 @@ func Restore(ctx context.Context, args []string, out io.Writer) error {
}
return fmt.Errorf("restore: invalid flags: %w", err)
}
if positionalSessionID == "" {
if err := applyParsedSessionIDArg("restore", fs, &sessionID); err != nil {
return err
}
} else {
if fs.NArg() != 0 {
return fmt.Errorf("restore: unexpected positional arguments")
}
if err := applyPositionalSessionID("restore", positionalSessionID, &sessionID); err != nil {
return err
}
if err := resolveParsedSessionID("restore", positionalSessionID, fs, &flags.sessionID); err != nil {
return err
}
if strings.TrimSpace(sessionID) == "" {
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("restore: session_id is required")
}
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
SessionID: sessionID,
PreviousSessionID: previousSessionID,
})
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
if err != nil {
return fmt.Errorf("restore: %w", err)
}

View File

@@ -3,7 +3,6 @@ package app
import (
"context"
"fmt"
"os"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
@@ -12,7 +11,7 @@ import (
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
)
// RemoteCurrentState captures discovered committed remote archive state for one session.
// RemoteCurrentState captures discovered committed remote published current state for one session.
type RemoteCurrentState struct {
Bucket string
SessionPrefix string
@@ -32,80 +31,24 @@ func discoverRemoteCurrentState(ctx context.Context, cfg *config.Config, store s
return nil, fmt.Errorf("remote object store is required")
}
bucket := artifacts.ResolveArchiveBucket(cfg, nil)
bucket := artifacts.ResolvePublishBucket(cfg, nil)
if strings.TrimSpace(bucket) == "" {
return nil, fmt.Errorf("archive bucket is required")
return nil, fmt.Errorf("publish bucket is required")
}
sessionPrefix, err := artifacts.ResolveArchiveSessionPrefix(cfg, nil)
sessionPrefix, err := artifacts.ResolvePublishSessionPrefix(cfg, nil)
if err != nil {
return nil, fmt.Errorf("resolve archive session prefix: %w", err)
}
currentManifestKey, currentRunIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
exists, err := store.Exists(ctx, currentRunIDKey)
if err != nil {
return nil, fmt.Errorf("check remote current run pointer %q: %w", currentRunIDKey, err)
}
if !exists {
return nil, fmt.Errorf("remote current run pointer missing: %q", currentRunIDKey)
}
runIDPath, err := downloadObjectToTemp(ctx, store, currentRunIDKey, "narratio-restore-current-run-id-*.txt")
if err != nil {
return nil, fmt.Errorf("download remote current run pointer %q: %w", currentRunIDKey, err)
}
defer func() { _ = os.Remove(runIDPath) }()
runIDData, err := os.ReadFile(runIDPath)
if err != nil {
return nil, fmt.Errorf("read downloaded run pointer %q: %w", currentRunIDKey, err)
}
runID := strings.TrimSpace(string(runIDData))
if runID == "" {
return nil, fmt.Errorf("remote current run pointer %q is empty", currentRunIDKey)
}
exists, err = store.Exists(ctx, currentManifestKey)
if err != nil {
return nil, fmt.Errorf("check remote current manifest %q: %w", currentManifestKey, err)
}
if !exists {
return nil, fmt.Errorf("remote current manifest missing: %q", currentManifestKey)
}
manifestPath, err := downloadObjectToTemp(ctx, store, currentManifestKey, "narratio-restore-current-manifest-*.json")
if err != nil {
return nil, fmt.Errorf("download remote current manifest %q: %w", currentManifestKey, err)
}
defer func() { _ = os.Remove(manifestPath) }()
manifestStore := &manifest.LocalStore{}
remoteManifest, err := manifestStore.Load(ctx, manifestPath)
if err != nil {
return nil, fmt.Errorf("remote current manifest decode failed: %w", err)
return nil, fmt.Errorf("resolve publish session prefix: %w", err)
}
currentManifestKey, currentRunIDKey := artifacts.ResolveCurrentStateKeys(sessionPrefix)
requestedSession := strings.TrimSpace(cfg.Session.SessionID)
requestedCampaign := strings.TrimSpace(cfg.Session.Campaign)
manifestSession := strings.TrimSpace(remoteManifest.SessionID)
manifestCampaign := strings.TrimSpace(remoteManifest.Campaign)
if manifestSession != requestedSession {
return nil, fmt.Errorf(
"remote current manifest session_id %q does not match requested session_id %q",
manifestSession,
requestedSession,
)
}
if manifestCampaign == "" {
return nil, fmt.Errorf("remote current manifest campaign is required")
}
if manifestCampaign != requestedCampaign {
return nil, fmt.Errorf(
"remote current manifest campaign %q does not match requested campaign %q",
manifestCampaign,
requestedCampaign,
)
current, err := artifacts.LoadCurrentState(ctx, store, sessionPrefix, artifacts.CurrentStateValidation{
ExpectedSessionID: requestedSession,
ExpectedCampaign: requestedCampaign,
})
if err != nil {
return nil, fmt.Errorf("remote %w", err)
}
return &RemoteCurrentState{
@@ -113,27 +56,9 @@ func discoverRemoteCurrentState(ctx context.Context, cfg *config.Config, store s
SessionPrefix: sessionPrefix,
CurrentRunIDKey: currentRunIDKey,
CurrentManifestKey: currentManifestKey,
RunID: runID,
SessionID: manifestSession,
Campaign: manifestCampaign,
Manifest: remoteManifest,
RunID: current.RunID,
SessionID: strings.TrimSpace(current.Manifest.SessionID),
Campaign: strings.TrimSpace(current.Manifest.Campaign),
Manifest: current.Manifest,
}, nil
}
func downloadObjectToTemp(ctx context.Context, store storage.ObjectStore, key, pattern string) (string, error) {
tmp, err := os.CreateTemp("", pattern)
if err != nil {
return "", fmt.Errorf("create temp file: %w", err)
}
path := tmp.Name()
if err := tmp.Close(); err != nil {
_ = os.Remove(path)
return "", fmt.Errorf("close temp file: %w", err)
}
if err := store.Download(ctx, key, path); err != nil {
_ = os.Remove(path)
return "", err
}
return path, nil
}

View File

@@ -102,7 +102,7 @@ func TestDiscoverRemoteCurrentStateSessionMismatchFails(t *testing.T) {
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, "wrong-session", cfg.Session.Campaign)})
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
if err == nil || !strings.Contains(err.Error(), "does not match requested session_id") {
if err == nil || !strings.Contains(err.Error(), "does not match expected session_id") {
t.Fatalf("error = %v, want session mismatch failure", err)
}
}
@@ -116,7 +116,7 @@ func TestDiscoverRemoteCurrentStateCampaignMismatchFails(t *testing.T) {
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, cfg.Session.SessionID, "wrong-campaign")})
_, err := discoverRemoteCurrentState(context.Background(), cfg, store)
if err == nil || !strings.Contains(err.Error(), "does not match requested campaign") {
if err == nil || !strings.Contains(err.Error(), "does not match expected campaign") {
t.Fatalf("error = %v, want campaign mismatch failure", err)
}
}
@@ -208,7 +208,7 @@ func restoreDiscoveryConfig() *config.Config {
func restoreDiscoveryKeys(cfg *config.Config) (sessionPrefix, manifestKey, runIDKey string) {
sessionPrefix = artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
manifestKey, runIDKey = artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
manifestKey, runIDKey = artifacts.ResolveCurrentStateKeys(sessionPrefix)
return sessionPrefix, manifestKey, runIDKey
}

View File

@@ -427,7 +427,7 @@ func seedRestoreCommittedState(t *testing.T, fake *storage.FakeBackend, pipeline
}
sessionPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.SessionID)
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
manifestKey, runIDKey := artifacts.ResolveCurrentStateKeys(sessionPrefix)
seedRestoreObject(fake, runIDKey, []byte("20260519T010203Z-a1b2c3d4\n"))
seedRestoreObject(fake, manifestKey, restoreManifestJSON(t, cfg.Session.SessionID, cfg.Session.Campaign))
@@ -465,7 +465,7 @@ func seedRestorePreviousCurrent(t *testing.T, fake *storage.FakeBackend, cfg *co
func seedRestorePreviousCurrentManifestOnly(t *testing.T, fake *storage.FakeBackend, cfg *config.Config) {
t.Helper()
previousPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.PreviousSessionID)
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(previousPrefix)
manifestKey, runIDKey := artifacts.ResolveCurrentStateKeys(previousPrefix)
previousRunID := "20260426T010203Z-a1b2c3d4"
seedRestoreObject(fake, runIDKey, []byte(previousRunID+"\n"))

View File

@@ -338,7 +338,7 @@ func classifyRestoreAction(
if err != nil {
return RestoreAction{}, fmt.Errorf("checksum local file: %w", err)
}
remotePath, err := downloadObjectToTemp(ctx, store, action.RemoteKey, "narratio-restore-plan-remote-*.tmp")
remotePath, err := storage.DownloadObjectToTemp(ctx, store, action.RemoteKey, "narratio-restore-plan-remote-*.tmp")
if err != nil {
return RestoreAction{}, fmt.Errorf("download remote object: %w", err)
}

View File

@@ -325,7 +325,7 @@ func configureRestorePlanPreviousRequirement(cfg *config.Config, required bool)
func restorePlanCurrentState(t *testing.T, cfg *config.Config) *RemoteCurrentState {
t.Helper()
sessionPrefix := artifacts.S3SessionPrefix("dnd", cfg.Session.Campaign, cfg.Session.SessionID)
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
manifestKey, runIDKey := artifacts.ResolveCurrentStateKeys(sessionPrefix)
return &RemoteCurrentState{
Bucket: "test-bucket",
SessionPrefix: sessionPrefix,

View File

@@ -5,7 +5,6 @@ import (
"flag"
"fmt"
"io"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
"gitea.maximumdirect.net/eric/narratio/internal/config"
@@ -14,48 +13,23 @@ import (
// Resume continues execution from the first non-succeeded stage in the manifest.
func Resume(ctx context.Context, args []string, out io.Writer) error {
positionalSessionID, args := pullLeadingSessionID(args)
fs := flag.NewFlagSet("resume", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var pipelinePath string
var campaignPath string
var campaignFilePath string
var sessionPath string
var sessionID string
var previousSessionID string
var flags commonConfigFlags
var force bool
var selectedArtifacts artifactSelectionFlag
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
addCommonConfigFlags(fs, &flags)
fs.BoolVar(&force, "force", false, "force stage execution")
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute and publish (comma-separated or repeatable)")
if err := fs.Parse(args); err != nil {
return fmt.Errorf("resume: invalid flags: %w", err)
if err := parseSessionAwareFlags("resume", fs, args, &flags.sessionID); err != nil {
return err
}
if positionalSessionID == "" {
if err := applyParsedSessionIDArg("resume", fs, &sessionID); err != nil {
return err
}
} else {
if fs.NArg() != 0 {
return fmt.Errorf("resume: unexpected positional arguments")
}
if err := applyPositionalSessionID("resume", positionalSessionID, &sessionID); err != nil {
return err
}
}
if strings.TrimSpace(sessionID) == "" {
if flags.sessionID == "" {
return fmt.Errorf("resume: session_id is required")
}
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
SessionID: sessionID,
PreviousSessionID: previousSessionID,
})
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
if err != nil {
return fmt.Errorf("resume: %w", err)
}

View File

@@ -5,55 +5,29 @@ import (
"flag"
"fmt"
"io"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/config"
)
// Run executes the pipeline plan and persists manifest state.
func Run(ctx context.Context, args []string, out io.Writer) error {
positionalSessionID, args := pullLeadingSessionID(args)
fs := flag.NewFlagSet("run", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var pipelinePath string
var campaignPath string
var campaignFilePath string
var sessionPath string
var sessionID string
var previousSessionID string
var flags commonConfigFlags
var force bool
var selectedArtifacts artifactSelectionFlag
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
addCommonConfigFlags(fs, &flags)
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute and publish (comma-separated or repeatable)")
if err := fs.Parse(args); err != nil {
return fmt.Errorf("run: invalid flags: %w", err)
if err := parseSessionAwareFlags("run", fs, args, &flags.sessionID); err != nil {
return err
}
if positionalSessionID == "" {
if err := applyParsedSessionIDArg("run", fs, &sessionID); err != nil {
return err
}
} else {
if fs.NArg() != 0 {
return fmt.Errorf("run: unexpected positional arguments")
}
if err := applyPositionalSessionID("run", positionalSessionID, &sessionID); err != nil {
return err
}
}
if strings.TrimSpace(sessionID) == "" {
if flags.sessionID == "" {
return fmt.Errorf("run: session_id is required")
}
cfg, err := loadCommandConfig(ctx, pipelinePath, campaignPath, campaignFilePath, sessionPath, config.SessionLoadOptions{
SessionID: sessionID,
PreviousSessionID: previousSessionID,
})
cfg, err := loadCommandConfig(ctx, flags.pipelinePath, flags.campaignPath, flags.campaignFilePath, flags.sessionPath, flags.sessionOptions())
if err != nil {
return fmt.Errorf("run: %w", err)
}

View File

@@ -23,19 +23,10 @@ func RunStage(ctx context.Context, args []string, out io.Writer) error {
fs := flag.NewFlagSet("run-stage", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var pipelinePath string
var campaignPath string
var campaignFilePath string
var sessionPath string
var sessionID string
var previousSessionID string
var flags commonConfigFlags
var force bool
var selectedArtifacts artifactSelectionFlag
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
addCommonConfigFlags(fs, &flags)
fs.BoolVar(&force, "force", false, "force stage execution (reserved for future behavior)")
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute or publish (comma-separated or repeatable)")
@@ -47,16 +38,21 @@ func RunStage(ctx context.Context, args []string, out io.Writer) error {
case 2:
stageName = strings.TrimSpace(fs.Arg(0))
positionalSessionID = strings.TrimSpace(fs.Arg(1))
case 1:
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("run-stage: expected stage name and session_id")
}
stageName = strings.TrimSpace(fs.Arg(0))
default:
return fmt.Errorf("run-stage: expected stage name and session_id")
}
} else if fs.NArg() != 0 {
return fmt.Errorf("run-stage: unexpected positional arguments")
}
if err := applyPositionalSessionID("run-stage", positionalSessionID, &sessionID); err != nil {
if err := applyPositionalSessionID("run-stage", positionalSessionID, &flags.sessionID); err != nil {
return err
}
if strings.TrimSpace(sessionID) == "" {
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("run-stage: session_id is required")
}
normalizedArtifacts, err := selectedArtifacts.Normalize()
@@ -70,12 +66,12 @@ func RunStage(ctx context.Context, args []string, out io.Writer) error {
summary, err := runSingleStageCommand(ctx, singleStageCommand{
CommandName: "run-stage",
StageName: stageName,
PipelinePath: pipelinePath,
CampaignPath: campaignPath,
CampaignFilePath: campaignFilePath,
SessionPath: sessionPath,
SessionID: sessionID,
PreviousSessionID: previousSessionID,
PipelinePath: flags.pipelinePath,
CampaignPath: flags.campaignPath,
CampaignFilePath: flags.campaignFilePath,
SessionPath: flags.sessionPath,
SessionID: flags.sessionID,
PreviousSessionID: flags.previousSessionID,
Force: force,
SelectedArtifacts: normalizedArtifacts,
})
@@ -97,40 +93,18 @@ func RunStage(ctx context.Context, args []string, out io.Writer) error {
// Analyze force-runs the analyze stage.
func Analyze(ctx context.Context, args []string, out io.Writer) error {
positionalSessionID, args := pullLeadingSessionID(args)
fs := flag.NewFlagSet("analyze", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var pipelinePath string
var campaignPath string
var campaignFilePath string
var sessionPath string
var sessionID string
var previousSessionID string
var flags commonConfigFlags
var selectedArtifacts artifactSelectionFlag
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
addCommonConfigFlags(fs, &flags)
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to execute during analyze (comma-separated or repeatable)")
if err := fs.Parse(args); err != nil {
return fmt.Errorf("analyze: invalid flags: %w", err)
if err := parseSessionAwareFlags("analyze", fs, args, &flags.sessionID); err != nil {
return err
}
if positionalSessionID == "" {
if err := applyParsedSessionIDArg("analyze", fs, &sessionID); err != nil {
return err
}
} else {
if fs.NArg() != 0 {
return fmt.Errorf("analyze: unexpected positional arguments")
}
if err := applyPositionalSessionID("analyze", positionalSessionID, &sessionID); err != nil {
return err
}
}
if strings.TrimSpace(sessionID) == "" {
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("analyze: session_id is required")
}
normalizedArtifacts, err := selectedArtifacts.Normalize()
@@ -141,12 +115,12 @@ func Analyze(ctx context.Context, args []string, out io.Writer) error {
summary, err := runSingleStageCommand(ctx, singleStageCommand{
CommandName: "analyze",
StageName: "analyze",
PipelinePath: pipelinePath,
CampaignPath: campaignPath,
CampaignFilePath: campaignFilePath,
SessionPath: sessionPath,
SessionID: sessionID,
PreviousSessionID: previousSessionID,
PipelinePath: flags.pipelinePath,
CampaignPath: flags.campaignPath,
CampaignFilePath: flags.campaignFilePath,
SessionPath: flags.sessionPath,
SessionID: flags.sessionID,
PreviousSessionID: flags.previousSessionID,
Force: true,
SelectedArtifacts: normalizedArtifacts,
})
@@ -166,40 +140,18 @@ func Analyze(ctx context.Context, args []string, out io.Writer) error {
// Publish force-runs the publish stage.
func Publish(ctx context.Context, args []string, out io.Writer) error {
positionalSessionID, args := pullLeadingSessionID(args)
fs := flag.NewFlagSet("publish", flag.ContinueOnError)
fs.SetOutput(io.Discard)
var pipelinePath string
var campaignPath string
var campaignFilePath string
var sessionPath string
var sessionID string
var previousSessionID string
var flags commonConfigFlags
var selectedArtifacts artifactSelectionFlag
fs.StringVar(&pipelinePath, "config", "", "path to pipeline.yml (optional; defaults searched)")
fs.StringVar(&campaignPath, "campaign", "", "campaign ID")
fs.StringVar(&campaignFilePath, "campaign-file", "", "path to campaign.yml")
fs.StringVar(&sessionPath, "session", "", "path to session.yml")
fs.StringVar(&previousSessionID, "previous-session-id", "", "expected previous session identifier")
addCommonConfigFlags(fs, &flags)
fs.Var(&selectedArtifacts, "artifacts", "configured artifact names to publish (comma-separated or repeatable)")
if err := fs.Parse(args); err != nil {
return fmt.Errorf("publish: invalid flags: %w", err)
if err := parseSessionAwareFlags("publish", fs, args, &flags.sessionID); err != nil {
return err
}
if positionalSessionID == "" {
if err := applyParsedSessionIDArg("publish", fs, &sessionID); err != nil {
return err
}
} else {
if fs.NArg() != 0 {
return fmt.Errorf("publish: unexpected positional arguments")
}
if err := applyPositionalSessionID("publish", positionalSessionID, &sessionID); err != nil {
return err
}
}
if strings.TrimSpace(sessionID) == "" {
if strings.TrimSpace(flags.sessionID) == "" {
return fmt.Errorf("publish: session_id is required")
}
normalizedArtifacts, err := selectedArtifacts.Normalize()
@@ -210,12 +162,12 @@ func Publish(ctx context.Context, args []string, out io.Writer) error {
summary, err := runSingleStageCommand(ctx, singleStageCommand{
CommandName: "publish",
StageName: "publish",
PipelinePath: pipelinePath,
CampaignPath: campaignPath,
CampaignFilePath: campaignFilePath,
SessionPath: sessionPath,
SessionID: sessionID,
PreviousSessionID: previousSessionID,
PipelinePath: flags.pipelinePath,
CampaignPath: flags.campaignPath,
CampaignFilePath: flags.campaignFilePath,
SessionPath: flags.sessionPath,
SessionID: flags.sessionID,
PreviousSessionID: flags.previousSessionID,
Force: true,
SelectedArtifacts: normalizedArtifacts,
})

View File

@@ -96,7 +96,7 @@ func executeStages(ctx context.Context, cfg *config.Config, stages []stage.Stage
if needsRemoteLocksForRun(env.Config, stages) {
locks, err := loadEffectiveLocks(ctx, env.Config, env.ObjectStore)
if err != nil {
return nil, fmt.Errorf("load remote archive locks: %w", err)
return nil, fmt.Errorf("load remote publish locks: %w", err)
}
applyEffectiveLocks(env.Config, locks.All)
}
@@ -234,14 +234,14 @@ func executeStages(ctx context.Context, cfg *config.Config, stages []stage.Stage
env.Logger.Info("stage succeeded", "stage", s.Name())
}
if err := runPostArchiveCleanup(ctx, env, manifestPath, m, executed); err != nil {
if err := runPostPublishCleanup(ctx, env, manifestPath, m, executed); err != nil {
failedAt := nowUTC()
runManifest.MarkFailed(failedAt, err.Error())
syncRunManifestIdentityFromSession(m, runManifest)
if saveErr := runManifestStore.SaveRun(ctx, runManifestPath, runManifest); saveErr != nil {
return nil, fmt.Errorf("post-archive cleanup failed (%v) and run-manifest save failed (%v)", err, saveErr)
return nil, fmt.Errorf("post-publish cleanup failed (%v) and run-manifest save failed (%v)", err, saveErr)
}
return nil, fmt.Errorf("post-archive cleanup: %w", err)
return nil, fmt.Errorf("post-publish cleanup: %w", err)
}
completedAt := nowUTC()
runManifest.MarkSucceeded(completedAt)

View File

@@ -248,7 +248,7 @@ func TestNeedsObjectStoreForRunPrepareWithPreviousRequirements(t *testing.T) {
}
}
func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testing.T) {
func TestExecuteStagesPublishSkipsRequiredUnselectedConfiguredOutput(t *testing.T) {
cfg := testConfig(t)
cfg.Pipeline.Storage.S3 = &config.StorageS3Config{
Bucket: "my-dnd-archive",
@@ -283,7 +283,7 @@ func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testi
t.Fatalf("Save manifest error = %v", err)
}
archiveStageImpl, err := stage.Select("publish")
publishStageImpl, err := stage.Select("publish")
if err != nil {
t.Fatalf("Select(publish) error = %v", err)
}
@@ -293,7 +293,7 @@ func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testi
cfg,
[]stage.Stage{
selectedAnalyzeArtifactStage{expected: []string{"player_handout"}},
archiveStageImpl,
publishStageImpl,
},
RunOptions{
SelectedArtifacts: []string{"player_handout"},
@@ -304,7 +304,7 @@ func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testi
t.Fatalf("executeStages() error = %v", err)
}
if len(summary.Executed) != 2 || summary.Executed[0] != "analyze" || summary.Executed[1] != "publish" {
t.Fatalf("executed = %#v, want analyze and archive", summary.Executed)
t.Fatalf("executed = %#v, want analyze and publish", summary.Executed)
}
loadedManifest, err := store.Load(context.Background(), summary.ManifestPath)
@@ -321,7 +321,7 @@ func TestExecuteStagesArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testi
t.Fatalf("skipped item = %#v, want object", skipped[0])
}
if item["source"] != "narratio.artifact.session_recap" || item["dest"] != "artifacts/session_recap.md" || item["required"] != true {
t.Fatalf("skipped item = %#v, want required session_recap promotion", item)
t.Fatalf("skipped item = %#v, want required session_recap published output", item)
}
}
@@ -427,10 +427,10 @@ func TestExecuteStagesPlaceholderSuccessUpdatesManifest(t *testing.T) {
}
if name == "publish" {
if sr.Metadata == nil || sr.Metadata["stage"] != "publish" {
t.Fatalf("archive metadata missing stage=publish: %#v", sr.Metadata)
t.Fatalf("publish metadata missing stage=publish: %#v", sr.Metadata)
}
if sr.Metadata["skipped"] != true {
t.Fatalf("archive metadata missing skipped=true for test config without archive section: %#v", sr.Metadata)
t.Fatalf("publish metadata missing skipped=true for test config without publish section: %#v", sr.Metadata)
}
continue
}
@@ -729,7 +729,7 @@ func TestExecuteStagesRunManifestRecordsSkippedStage(t *testing.T) {
}
}
func TestExecuteStagesRunLocalArtifactsAndCanonicalPromotion(t *testing.T) {
func TestExecuteStagesRunLocalArtifactsAndCanonicalSync(t *testing.T) {
cfg := testConfig(t)
stages := []stage.Stage{
BuildFullPlan()[0], // prepare
@@ -778,7 +778,7 @@ func TestExecuteStagesRunLocalArtifactsAndCanonicalPromotion(t *testing.T) {
}
for _, p := range canonicalChecks {
if _, statErr := os.Stat(p); statErr != nil {
t.Fatalf("canonical promoted artifact missing at %q: %v", p, statErr)
t.Fatalf("canonical published artifact missing at %q: %v", p, statErr)
}
}
@@ -864,7 +864,7 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
{name: "merge", env: &Env{Seriatim: &seriatim.FakeRunner{Err: errors.New("merge fail")}}},
{name: "polish", env: &Env{Audita: &audita.FakeRunner{Err: errors.New("polish fail")}}},
{name: "analyze", env: &Env{Scriptorium: &scriptorium.FakeRunner{RunErr: errors.New("analyze fail")}}},
{name: "publish", env: &Env{ObjectStore: &storage.FakeBackend{UploadErr: errors.New("archive fail")}}},
{name: "publish", env: &Env{ObjectStore: &storage.FakeBackend{UploadErr: errors.New("publish fail")}}},
{name: "notify", env: &Env{Notifier: &notify.FakeSender{Err: errors.New("notify fail")}}},
}
@@ -961,13 +961,13 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
runID := "20260516T010203Z-0a1b2c3d"
runWorkDir := filepath.Join(cfg.Pipeline.Workspace.Root, "work", cfg.Session.Campaign, cfg.Session.SessionID, runID)
if err := os.MkdirAll(filepath.Join(runWorkDir, "inputs"), 0o755); err != nil {
t.Fatalf("mkdir archive inputs dir: %v", err)
t.Fatalf("mkdir publish inputs dir: %v", err)
}
if err := os.WriteFile(filepath.Join(runWorkDir, "inputs", "session.yml"), []byte("session_id: 2026-05-03\n"), 0o644); err != nil {
t.Fatalf("write archive fixture session.yml: %v", err)
t.Fatalf("write publish fixture session.yml: %v", err)
}
if err := os.WriteFile(filepath.Join(runWorkDir, "manifest.json"), []byte("{}\n"), 0o644); err != nil {
t.Fatalf("write archive fixture manifest.json: %v", err)
t.Fatalf("write publish fixture manifest.json: %v", err)
}
seed := manifest.New(cfg.Session.SessionID, time.Now().UTC())
@@ -981,7 +981,7 @@ func TestAdapterBackedStageFailureMarksManifestFailed(t *testing.T) {
seed.MarkStageSucceeded(name, time.Now().UTC(), nil)
}
if err := tc.env.ManifestStore.Save(context.Background(), manifestPathFor(cfg), seed); err != nil {
t.Fatalf("seed archive manifest: %v", err)
t.Fatalf("seed publish manifest: %v", err)
}
}

View File

@@ -41,3 +41,21 @@ func applyParsedSessionIDArg(command string, fs *flag.FlagSet, sessionID *string
return fmt.Errorf("%s: unexpected positional arguments", command)
}
}
func resolveParsedSessionID(command, positionalSessionID string, fs *flag.FlagSet, sessionID *string) error {
if strings.TrimSpace(positionalSessionID) == "" {
return applyParsedSessionIDArg(command, fs, sessionID)
}
if fs.NArg() != 0 {
return fmt.Errorf("%s: unexpected positional arguments", command)
}
return applyPositionalSessionID(command, positionalSessionID, sessionID)
}
func parseSessionAwareFlags(command string, fs *flag.FlagSet, args []string, sessionID *string) error {
positionalSessionID, args := pullLeadingSessionID(args)
if err := fs.Parse(args); err != nil {
return fmt.Errorf("%s: invalid flags: %w", command, err)
}
return resolveParsedSessionID(command, positionalSessionID, fs, sessionID)
}

View File

@@ -71,15 +71,36 @@ func TestExecutePositionalSessionIDMismatchFails(t *testing.T) {
}
}
func TestExecuteSessionIDFlagFails(t *testing.T) {
func TestExecuteSessionIDFlagMismatchFails(t *testing.T) {
var stdout bytes.Buffer
var stderr bytes.Buffer
code := Execute([]string{"session", "status", "2026-05-03", "--session-id", "2026-05-04"}, &stdout, &stderr)
if code == 0 {
t.Fatal("exit code = 0, want non-zero")
}
if !strings.Contains(stderr.String(), "flag provided but not defined: -session-id") {
t.Fatalf("stderr = %q, want invalid --session-id flag", stderr.String())
if !strings.Contains(stderr.String(), "does not match expected session id") {
t.Fatalf("stderr = %q, want positional/flag mismatch", stderr.String())
}
}
func TestExecuteSessionIDFlagAcceptedWithoutPositional(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
var stdout bytes.Buffer
var stderr bytes.Buffer
code := Execute([]string{
"session", "status",
"--session-id", "2026-05-03",
"--config", pipelinePath,
"--campaign-file", campaignPath,
"--session", sessionPath,
}, &stdout, &stderr)
if code != 0 {
t.Fatalf("exit code = %d, want 0; stderr=%q", code, stderr.String())
}
if !strings.Contains(stdout.String(), "Session: 2026-05-03") {
t.Fatalf("stdout = %q, want status output", stdout.String())
}
}
@@ -212,7 +233,7 @@ func TestExecuteSessionSubcommandsAcceptPositionalSessionID(t *testing.T) {
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspaceRoot)
fake := &storage.FakeBackend{}
sessionPrefix := artifacts.S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
manifestKey, runIDKey := artifacts.ResolveCurrentStateKeys(sessionPrefix)
fake.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
fake.SeedObject(storage.FakeObject{Key: manifestKey, Data: restoreManifestJSON(t, "2026-05-03", "sample-campaign")})
var storeInitCalls int

View File

@@ -0,0 +1,145 @@
package artifactpolicy
import (
"fmt"
"regexp"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/artifactmodel"
"gitea.maximumdirect.net/eric/narratio/internal/pathsafe"
)
const (
SourceBoundsSession = "narratio.bounds.session"
configuredSourcePrefix = "narratio.artifact."
previousConfiguredSrcPrefix = "narratio.previous_session.artifact."
)
var configuredSourceRE = regexp.MustCompile(`^narratio\.artifact\.([a-z][a-z0-9_]*)$`)
var previousSourceRE = regexp.MustCompile(`^narratio\.previous_session\.artifact\.([a-z][a-z0-9_]*)$`)
type SourceKind string
const (
SourceKindBuiltIn SourceKind = "built_in"
SourceKindConfiguredArtifact SourceKind = "configured_artifact"
SourceKindPreviousArtifact SourceKind = "previous_session_configured_artifact"
)
// Source describes one normalized artifact source identifier.
type Source struct {
ID string
Kind SourceKind
ConfiguredKey string
}
// ConfiguredSourceID converts a configured artifact key into source id form.
func ConfiguredSourceID(key string) string {
return configuredSourcePrefix + strings.TrimSpace(key)
}
// PreviousSessionSourceID converts a configured artifact key into previous-session source id form.
func PreviousSessionSourceID(key string) string {
return previousConfiguredSrcPrefix + strings.TrimSpace(key)
}
// ParseConfiguredSource extracts configured key from narratio.artifact.<key>.
func ParseConfiguredSource(source string) (string, bool) {
matches := configuredSourceRE.FindStringSubmatch(strings.TrimSpace(source))
if len(matches) != 2 {
return "", false
}
return matches[1], true
}
// ParsePreviousSessionSource extracts configured key from narratio.previous_session.artifact.<key>.
func ParsePreviousSessionSource(source string) (string, bool) {
matches := previousSourceRE.FindStringSubmatch(strings.TrimSpace(source))
if len(matches) != 2 {
return "", false
}
return matches[1], true
}
// ClassifySource classifies a source id as built-in, configured, or previous-session configured.
func ClassifySource(source string) (Source, error) {
trimmed := strings.TrimSpace(source)
if trimmed == "" {
return Source{}, fmt.Errorf("artifact source is required")
}
if _, ok := artifactmodel.LookupRuntimeTranscriptArtifact(trimmed); ok {
return Source{ID: trimmed, Kind: SourceKindBuiltIn}, nil
}
if trimmed == SourceBoundsSession {
return Source{ID: trimmed, Kind: SourceKindBuiltIn}, nil
}
if key, ok := ParseConfiguredSource(trimmed); ok {
return Source{ID: trimmed, Kind: SourceKindConfiguredArtifact, ConfiguredKey: key}, nil
}
if key, ok := ParsePreviousSessionSource(trimmed); ok {
return Source{ID: trimmed, Kind: SourceKindPreviousArtifact, ConfiguredKey: key}, nil
}
return Source{}, fmt.Errorf("unsupported artifact source %q", source)
}
// ValidatePublishSource validates that a source is publish-compatible and references a known configured artifact.
func ValidatePublishSource(source string, configured map[string]string) (Source, error) {
classified, err := ClassifySource(source)
if err != nil {
return Source{}, fmt.Errorf("must be a built-in source id or narratio.artifact.<name>")
}
if classified.Kind == SourceKindPreviousArtifact {
return Source{}, fmt.Errorf("must be a built-in source id or narratio.artifact.<name>")
}
if classified.Kind == SourceKindConfiguredArtifact {
if configured == nil {
return Source{}, fmt.Errorf("configured artifact %q is not defined in pipeline.scriptorium.artifacts", classified.ConfiguredKey)
}
if _, ok := configured[classified.ConfiguredKey]; !ok {
return Source{}, fmt.Errorf("configured artifact %q is not defined in pipeline.scriptorium.artifacts", classified.ConfiguredKey)
}
}
return classified, nil
}
// DeriveDefaultPublishedDestination returns the default publish destination for one source.
func DeriveDefaultPublishedDestination(source Source, configured map[string]string) (string, error) {
switch source.Kind {
case SourceKindBuiltIn:
if spec, ok := artifactmodel.LookupRuntimeTranscriptArtifact(source.ID); ok {
return pathsafe.NormalizeRelativeDestination(spec.CanonicalRelPath)
}
if source.ID == SourceBoundsSession {
return pathsafe.NormalizeRelativeDestination("artifacts/session_bounds.json")
}
return "", fmt.Errorf("unsupported built-in source %q", source.ID)
case SourceKindConfiguredArtifact:
if configured == nil {
return "", fmt.Errorf("configured artifact %q is not defined in pipeline.scriptorium.artifacts", source.ConfiguredKey)
}
outputPath, ok := configured[source.ConfiguredKey]
if !ok {
return "", fmt.Errorf("configured artifact %q is not defined in pipeline.scriptorium.artifacts", source.ConfiguredKey)
}
if strings.TrimSpace(outputPath) == "" {
return "", fmt.Errorf("pipeline.scriptorium.artifacts.%s.output_path is empty", source.ConfiguredKey)
}
return pathsafe.NormalizeRelativeDestination(outputPath)
default:
return "", fmt.Errorf("publish destination cannot be derived from source %q", source.ID)
}
}
// ResolvePublishedDestination validates and normalizes an explicit destination,
// or derives one when omitted.
func ResolvePublishedDestination(sourceID, explicitDest string, configured map[string]string) (string, error) {
source, err := ValidatePublishSource(sourceID, configured)
if err != nil {
return "", err
}
if strings.TrimSpace(explicitDest) != "" {
return pathsafe.NormalizeRelativeDestination(explicitDest)
}
return DeriveDefaultPublishedDestination(source, configured)
}

View File

@@ -0,0 +1,92 @@
package artifactpolicy
import (
"strings"
"testing"
)
func TestClassifySource(t *testing.T) {
tests := []struct {
name string
source string
wantKind SourceKind
wantKey string
wantErrLike string
}{
{name: "built in transcript", source: "narratio.transcript.final_trimmed", wantKind: SourceKindBuiltIn},
{name: "built in bounds", source: "narratio.bounds.session", wantKind: SourceKindBuiltIn},
{name: "configured artifact", source: "narratio.artifact.session_recap", wantKind: SourceKindConfiguredArtifact, wantKey: "session_recap"},
{name: "previous session configured", source: "narratio.previous_session.artifact.session_recap", wantKind: SourceKindPreviousArtifact, wantKey: "session_recap"},
{name: "unsupported", source: "narratio.unknown", wantErrLike: "unsupported artifact source"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ClassifySource(tt.source)
if tt.wantErrLike != "" {
if err == nil || !strings.Contains(err.Error(), tt.wantErrLike) {
t.Fatalf("ClassifySource() error = %v, want like %q", err, tt.wantErrLike)
}
return
}
if err != nil {
t.Fatalf("ClassifySource() error = %v", err)
}
if got.Kind != tt.wantKind {
t.Fatalf("ClassifySource().Kind = %q, want %q", got.Kind, tt.wantKind)
}
if got.ConfiguredKey != tt.wantKey {
t.Fatalf("ClassifySource().ConfiguredKey = %q, want %q", got.ConfiguredKey, tt.wantKey)
}
})
}
}
func TestValidatePublishSource(t *testing.T) {
configured := map[string]string{"session_recap": "artifacts/session_recap.md"}
if _, err := ValidatePublishSource("narratio.artifact.session_recap", configured); err != nil {
t.Fatalf("ValidatePublishSource(configured) error = %v", err)
}
if _, err := ValidatePublishSource("narratio.previous_session.artifact.session_recap", configured); err == nil {
t.Fatal("ValidatePublishSource(previous) error = nil, want error")
}
if _, err := ValidatePublishSource("narratio.artifact.missing", configured); err == nil {
t.Fatal("ValidatePublishSource(missing configured) error = nil, want error")
}
}
func TestResolvePublishedDestination(t *testing.T) {
configured := map[string]string{"session_recap": "artifacts/session_recap.md"}
got, err := ResolvePublishedDestination("narratio.transcript.final_trimmed", "", configured)
if err != nil {
t.Fatalf("ResolvePublishedDestination(built-in) error = %v", err)
}
if got != "transcripts/final.trimmed.json" {
t.Fatalf("built-in destination = %q, want transcripts/final.trimmed.json", got)
}
got, err = ResolvePublishedDestination("narratio.artifact.session_recap", "", configured)
if err != nil {
t.Fatalf("ResolvePublishedDestination(configured) error = %v", err)
}
if got != "artifacts/session_recap.md" {
t.Fatalf("configured destination = %q, want artifacts/session_recap.md", got)
}
got, err = ResolvePublishedDestination("narratio.transcript.final_trimmed", "published/../published/final.json", configured)
if err != nil {
t.Fatalf("ResolvePublishedDestination(explicit) error = %v", err)
}
if got != "published/final.json" {
t.Fatalf("explicit destination = %q, want published/final.json", got)
}
}
func TestResolvePublishedDestinationRejectsTraversal(t *testing.T) {
configured := map[string]string{"session_recap": "artifacts/session_recap.md"}
_, err := ResolvePublishedDestination("narratio.transcript.final_trimmed", "../escape.txt", configured)
if err == nil {
t.Fatal("ResolvePublishedDestination() error = nil, want traversal rejection")
}
}

View File

@@ -6,10 +6,10 @@ import (
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/artifactmodel"
"gitea.maximumdirect.net/eric/narratio/internal/artifactpolicy"
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
)
@@ -40,8 +40,6 @@ const (
// ErrSessionArtifactNotFound is returned when no readable artifact exists for a known ID.
var ErrSessionArtifactNotFound = errors.New("session artifact not found")
var configuredArtifactSourceRE = regexp.MustCompile(`^narratio\.artifact\.([a-z][a-z0-9_]*)$`)
var previousSessionArtifactSourceRE = regexp.MustCompile(`^narratio\.previous_session\.artifact\.([a-z][a-z0-9_]*)$`)
type artifactContentKind string
@@ -107,43 +105,39 @@ func (e *SessionArtifactNotFoundError) Unwrap() error {
// NormalizeSessionArtifactSource validates canonical artifact IDs.
func NormalizeSessionArtifactSource(source string) (string, error) {
normalized := strings.TrimSpace(source)
if normalized == "" {
return "", fmt.Errorf("artifact source is required")
}
if _, ok := artifactRegistry[normalized]; !ok {
classified, err := artifactpolicy.ClassifySource(source)
if err != nil {
return "", fmt.Errorf("unsupported artifact source %q", source)
}
return normalized, nil
if classified.Kind != artifactpolicy.SourceKindBuiltIn {
return "", fmt.Errorf("unsupported artifact source %q", source)
}
if _, ok := artifactRegistry[classified.ID]; !ok {
return "", fmt.Errorf("unsupported artifact source %q", source)
}
return classified.ID, nil
}
// IsConfiguredArtifactSource returns true when source is narratio.artifact.<name>.
func IsConfiguredArtifactSource(source string) bool {
return configuredArtifactSourceRE.MatchString(strings.TrimSpace(source))
_, ok := artifactpolicy.ParseConfiguredSource(source)
return ok
}
// ConfiguredArtifactName extracts <name> from narratio.artifact.<name>.
func ConfiguredArtifactName(source string) (string, bool) {
matches := configuredArtifactSourceRE.FindStringSubmatch(strings.TrimSpace(source))
if len(matches) != 2 {
return "", false
}
return matches[1], true
return artifactpolicy.ParseConfiguredSource(source)
}
// IsPreviousSessionArtifactSource returns true when source is narratio.previous_session.artifact.<name>.
func IsPreviousSessionArtifactSource(source string) bool {
_, ok := PreviousSessionArtifactName(source)
_, ok := artifactpolicy.ParsePreviousSessionSource(source)
return ok
}
// PreviousSessionArtifactName extracts <name> from narratio.previous_session.artifact.<name>.
func PreviousSessionArtifactName(source string) (string, bool) {
matches := previousSessionArtifactSourceRE.FindStringSubmatch(strings.TrimSpace(source))
if len(matches) != 2 {
return "", false
}
return matches[1], true
return artifactpolicy.ParsePreviousSessionSource(source)
}
// ResolveSessionArtifact resolves a symbolic source to a readable local session artifact path.

View File

@@ -4,6 +4,8 @@ import (
"fmt"
"sort"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/artifactpolicy"
)
const (
@@ -47,7 +49,7 @@ func NewArtifactCatalog() *ArtifactCatalog {
// ConfiguredArtifactSourceID converts a configured artifact key into canonical source ID.
func ConfiguredArtifactSourceID(key string) string {
return "narratio.artifact." + strings.TrimSpace(key)
return artifactpolicy.ConfiguredSourceID(key)
}
// RegisterBuiltIns registers built-in source definitions used by runtime artifact resolution.

View File

@@ -0,0 +1,203 @@
package artifacts
import (
"context"
"errors"
"fmt"
"os"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
)
var (
ErrCurrentRunPointerMissing = errors.New("current run pointer missing")
ErrCurrentManifestMissing = errors.New("current manifest missing")
)
type CurrentRunPointerMissingError struct {
Key string
}
func (e *CurrentRunPointerMissingError) Error() string {
return fmt.Sprintf("%s: %q", ErrCurrentRunPointerMissing, e.Key)
}
func (e *CurrentRunPointerMissingError) Unwrap() error {
return ErrCurrentRunPointerMissing
}
type CurrentManifestMissingError struct {
Key string
}
func (e *CurrentManifestMissingError) Error() string {
return fmt.Sprintf("%s: %q", ErrCurrentManifestMissing, e.Key)
}
func (e *CurrentManifestMissingError) Unwrap() error {
return ErrCurrentManifestMissing
}
type CurrentState struct {
SessionPrefix string
CurrentRunIDKey string
CurrentManifestKey string
RunID string
Manifest *manifest.Manifest
}
type CurrentStateValidation struct {
ExpectedCampaign string
ExpectedSessionID string
ExpectedRunID string
ValidateRunID bool
}
func LoadCurrentRunPointer(ctx context.Context, store storage.ObjectStore, currentRunIDKey string) (string, error) {
if store == nil {
return "", fmt.Errorf("object store is required")
}
key := strings.TrimSpace(currentRunIDKey)
if key == "" {
return "", fmt.Errorf("current run pointer key is required")
}
exists, err := store.Exists(ctx, key)
if err != nil {
return "", fmt.Errorf("check current run pointer %q: %w", key, err)
}
if !exists {
return "", &CurrentRunPointerMissingError{Key: key}
}
localPath, err := storage.DownloadObjectToTemp(ctx, store, key, "narratio-current-run-id-*.txt")
if err != nil {
return "", fmt.Errorf("download current run pointer %q: %w", key, err)
}
defer func() { _ = os.Remove(localPath) }()
data, err := os.ReadFile(localPath)
if err != nil {
return "", fmt.Errorf("read downloaded current run pointer %q: %w", key, err)
}
runID := strings.TrimSpace(string(data))
if runID == "" {
return "", fmt.Errorf("current run pointer %q is empty", key)
}
return runID, nil
}
func LoadCurrentManifest(ctx context.Context, store storage.ObjectStore, currentManifestKey string) (*manifest.Manifest, error) {
if store == nil {
return nil, fmt.Errorf("object store is required")
}
key := strings.TrimSpace(currentManifestKey)
if key == "" {
return nil, fmt.Errorf("current manifest key is required")
}
exists, err := store.Exists(ctx, key)
if err != nil {
return nil, fmt.Errorf("check current manifest %q: %w", key, err)
}
if !exists {
return nil, &CurrentManifestMissingError{Key: key}
}
localPath, err := storage.DownloadObjectToTemp(ctx, store, key, "narratio-current-manifest-*.json")
if err != nil {
return nil, fmt.Errorf("download current manifest %q: %w", key, err)
}
defer func() { _ = os.Remove(localPath) }()
manifestStore := &manifest.LocalStore{}
m, err := manifestStore.Load(ctx, localPath)
if err != nil {
return nil, fmt.Errorf("current manifest decode failed: %w", err)
}
return m, nil
}
func LoadCurrentState(
ctx context.Context,
store storage.ObjectStore,
sessionPrefix string,
validation CurrentStateValidation,
) (*CurrentState, error) {
prefix := strings.TrimSpace(sessionPrefix)
if prefix == "" {
return nil, fmt.Errorf("session prefix is required")
}
currentManifestKey, currentRunIDKey := ResolveCurrentStateKeys(prefix)
runID, err := LoadCurrentRunPointer(ctx, store, currentRunIDKey)
if err != nil {
return nil, err
}
m, err := LoadCurrentManifest(ctx, store, currentManifestKey)
if err != nil {
return nil, err
}
state := &CurrentState{
SessionPrefix: prefix,
CurrentRunIDKey: currentRunIDKey,
CurrentManifestKey: currentManifestKey,
RunID: runID,
Manifest: m,
}
if err := ValidateCurrentStateIdentity(state, validation); err != nil {
return nil, err
}
return state, nil
}
func ValidateCurrentStateIdentity(state *CurrentState, validation CurrentStateValidation) error {
if state == nil || state.Manifest == nil {
return fmt.Errorf("current state with manifest is required")
}
expectedSessionID := strings.TrimSpace(validation.ExpectedSessionID)
expectedCampaign := strings.TrimSpace(validation.ExpectedCampaign)
expectedRunID := strings.TrimSpace(validation.ExpectedRunID)
manifestSessionID := strings.TrimSpace(state.Manifest.SessionID)
manifestCampaign := strings.TrimSpace(state.Manifest.Campaign)
manifestRunID := strings.TrimSpace(state.Manifest.RunID)
if expectedSessionID != "" && manifestSessionID != expectedSessionID {
return fmt.Errorf(
"current manifest session_id %q does not match expected session_id %q",
manifestSessionID,
expectedSessionID,
)
}
if expectedCampaign != "" {
if manifestCampaign == "" {
return fmt.Errorf("current manifest campaign is required")
}
if manifestCampaign != expectedCampaign {
return fmt.Errorf(
"current manifest campaign %q does not match expected campaign %q",
manifestCampaign,
expectedCampaign,
)
}
}
if expectedRunID == "" && validation.ValidateRunID {
expectedRunID = strings.TrimSpace(state.RunID)
}
if expectedRunID != "" {
if manifestRunID == "" {
return fmt.Errorf("current manifest run_id is required")
}
if manifestRunID != expectedRunID {
return fmt.Errorf(
"current run pointer %q references run %q but current manifest run_id is %q",
state.CurrentRunIDKey,
expectedRunID,
manifestRunID,
)
}
}
return nil
}

View File

@@ -0,0 +1,140 @@
package artifacts
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"time"
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
)
func TestLoadCurrentStateMissingRunPointer(t *testing.T) {
store := &storage.FakeBackend{}
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{})
if err == nil {
t.Fatal("LoadCurrentState() error = nil, want missing run pointer error")
}
var missing *CurrentRunPointerMissingError
if !errors.As(err, &missing) {
t.Fatalf("errors.As(err, *CurrentRunPointerMissingError) = false; err=%v", err)
}
if !errors.Is(err, ErrCurrentRunPointerMissing) {
t.Fatalf("errors.Is(err, ErrCurrentRunPointerMissing) = false; err=%v", err)
}
}
func TestLoadCurrentStateMissingManifest(t *testing.T) {
store := &storage.FakeBackend{}
_, _, runIDKey := testCurrentStateKeys()
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{})
if err == nil {
t.Fatal("LoadCurrentState() error = nil, want missing manifest error")
}
var missing *CurrentManifestMissingError
if !errors.As(err, &missing) {
t.Fatalf("errors.As(err, *CurrentManifestMissingError) = false; err=%v", err)
}
if !errors.Is(err, ErrCurrentManifestMissing) {
t.Fatalf("errors.Is(err, ErrCurrentManifestMissing) = false; err=%v", err)
}
}
func TestLoadCurrentStateEmptyRunPointerFails(t *testing.T) {
store := &storage.FakeBackend{}
_, manifestKey, runIDKey := testCurrentStateKeys()
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte(" \n\t")})
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: testManifestJSON(t, "2026-05-03", "sample-campaign", "20260519T010203Z-a1b2c3d4")})
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{})
if err == nil || !strings.Contains(err.Error(), "is empty") {
t.Fatalf("error = %v, want empty run pointer failure", err)
}
}
func TestLoadCurrentStateMalformedManifestFails(t *testing.T) {
store := &storage.FakeBackend{}
_, manifestKey, runIDKey := testCurrentStateKeys()
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: []byte("{invalid json")})
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{})
if err == nil || !strings.Contains(err.Error(), "current manifest decode failed") {
t.Fatalf("error = %v, want manifest decode failure", err)
}
}
func TestLoadCurrentStateCampaignMismatchFails(t *testing.T) {
store := &storage.FakeBackend{}
seedCurrentState(t, store, "2026-05-03", "wrong-campaign", "20260519T010203Z-a1b2c3d4")
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{
ExpectedCampaign: "sample-campaign",
})
if err == nil || !strings.Contains(err.Error(), "does not match expected campaign") {
t.Fatalf("error = %v, want campaign mismatch failure", err)
}
}
func TestLoadCurrentStateSessionMismatchFails(t *testing.T) {
store := &storage.FakeBackend{}
seedCurrentState(t, store, "wrong-session", "sample-campaign", "20260519T010203Z-a1b2c3d4")
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{
ExpectedSessionID: "2026-05-03",
})
if err == nil || !strings.Contains(err.Error(), "does not match expected session_id") {
t.Fatalf("error = %v, want session mismatch failure", err)
}
}
func TestLoadCurrentStateRunIDMismatchFails(t *testing.T) {
store := &storage.FakeBackend{}
seedCurrentState(t, store, "2026-05-03", "sample-campaign", "different-run-id")
_, err := LoadCurrentState(context.Background(), store, testCurrentSessionPrefix(), CurrentStateValidation{
ValidateRunID: true,
})
if err == nil || !strings.Contains(err.Error(), "current manifest run_id") {
t.Fatalf("error = %v, want run mismatch failure", err)
}
}
func testCurrentSessionPrefix() string {
return S3SessionPrefix("dnd", "sample-campaign", "2026-05-03")
}
func testCurrentStateKeys() (sessionPrefix, manifestKey, runIDKey string) {
sessionPrefix = testCurrentSessionPrefix()
manifestKey, runIDKey = ResolveCurrentStateKeys(sessionPrefix)
return sessionPrefix, manifestKey, runIDKey
}
func seedCurrentState(t *testing.T, store *storage.FakeBackend, sessionID, campaign, manifestRunID string) {
t.Helper()
_, manifestKey, runIDKey := testCurrentStateKeys()
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("20260519T010203Z-a1b2c3d4\n")})
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: testManifestJSON(t, sessionID, campaign, manifestRunID)})
}
func testManifestJSON(t *testing.T, sessionID, campaign, runID string) []byte {
t.Helper()
now := time.Date(2026, 5, 19, 23, 0, 0, 0, time.UTC).Format(time.RFC3339Nano)
payload := map[string]any{
"session_id": sessionID,
"campaign": campaign,
"run_id": runID,
"created_at": now,
"updated_at": now,
"stages": map[string]any{},
}
data, err := json.Marshal(payload)
if err != nil {
t.Fatalf("marshal manifest payload: %v", err)
}
return append(data, '\n')
}

View File

@@ -75,9 +75,9 @@ func TestSessionPreviousPathsForCampaign(t *testing.T) {
t.Fatalf("SessionPreviousArtifactPathForCampaign() = %q, want %q", artifactPath, wantArtifactPath)
}
archiveRelativeArtifactPath := SessionPreviousArtifactPathForCampaign(root, "forsaken", "2026-04-19", "artifacts/session_recap.md")
if archiveRelativeArtifactPath != wantArtifactPath {
t.Fatalf("SessionPreviousArtifactPathForCampaign(archive-relative) = %q, want %q", archiveRelativeArtifactPath, wantArtifactPath)
previousRelativeArtifactPath := SessionPreviousArtifactPathForCampaign(root, "forsaken", "2026-04-19", "artifacts/session_recap.md")
if previousRelativeArtifactPath != wantArtifactPath {
t.Fatalf("SessionPreviousArtifactPathForCampaign(previous-relative) = %q, want %q", previousRelativeArtifactPath, wantArtifactPath)
}
}
@@ -102,7 +102,7 @@ func TestSessionPreviousPathsFromSessionPaths(t *testing.T) {
got = SessionPreviousArtifactPath(paths, "artifacts/session_recap.md")
want = filepath.Join(paths.PreviousArtifactsDir, "session_recap.md")
if got != want {
t.Fatalf("SessionPreviousArtifactPath(archive-relative) = %q, want %q", got, want)
t.Fatalf("SessionPreviousArtifactPath(previous-relative) = %q, want %q", got, want)
}
}

View File

@@ -8,8 +8,8 @@ import (
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
)
// ResolveArchiveBucket resolves archive bucket identity with manifest-first precedence.
func ResolveArchiveBucket(cfg *config.Config, m *manifest.Manifest) string {
// ResolvePublishBucket resolves publish bucket identity with manifest-first precedence.
func ResolvePublishBucket(cfg *config.Config, m *manifest.Manifest) string {
if m != nil && strings.TrimSpace(m.S3Bucket) != "" {
return strings.TrimSpace(m.S3Bucket)
}
@@ -19,8 +19,8 @@ func ResolveArchiveBucket(cfg *config.Config, m *manifest.Manifest) string {
return strings.TrimSpace(cfg.Pipeline.Storage.S3.Bucket)
}
// ResolveArchiveSessionPrefix resolves archive session prefix with manifest-first precedence.
func ResolveArchiveSessionPrefix(cfg *config.Config, m *manifest.Manifest) (string, error) {
// ResolvePublishSessionPrefix resolves publish session prefix with manifest-first precedence.
func ResolvePublishSessionPrefix(cfg *config.Config, m *manifest.Manifest) (string, error) {
if m != nil && strings.TrimSpace(m.S3SessionPrefix) != "" {
return strings.TrimSpace(m.S3SessionPrefix), nil
}
@@ -47,8 +47,8 @@ func ResolveArchiveSessionPrefix(cfg *config.Config, m *manifest.Manifest) (stri
return sessionPrefix, nil
}
// ResolveArchiveRunPrefix resolves archive run prefix with manifest-first precedence.
func ResolveArchiveRunPrefix(cfg *config.Config, m *manifest.Manifest) (string, error) {
// ResolvePublishRunPrefix resolves publish run prefix with manifest-first precedence.
func ResolvePublishRunPrefix(cfg *config.Config, m *manifest.Manifest) (string, error) {
if m != nil {
runPrefix := strings.TrimSpace(m.S3RunPrefix)
if runPrefix != "" {
@@ -56,7 +56,7 @@ func ResolveArchiveRunPrefix(cfg *config.Config, m *manifest.Manifest) (string,
}
}
sessionPrefix, err := ResolveArchiveSessionPrefix(cfg, m)
sessionPrefix, err := ResolvePublishSessionPrefix(cfg, m)
if err != nil {
return "", err
}
@@ -71,7 +71,7 @@ func ResolveArchiveRunPrefix(cfg *config.Config, m *manifest.Manifest) (string,
return S3RunPrefix(sessionPrefix, runID), nil
}
// ResolveArchiveCurrentStateKeys returns current pointer keys for a session prefix.
func ResolveArchiveCurrentStateKeys(sessionPrefix string) (manifestKey, runIDKey string) {
// ResolveCurrentStateKeys returns current pointer keys for a session prefix.
func ResolveCurrentStateKeys(sessionPrefix string) (manifestKey, runIDKey string) {
return S3CurrentManifestKey(sessionPrefix), S3CurrentRunPointerKey(sessionPrefix)
}

View File

@@ -8,7 +8,7 @@ import (
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
)
func TestResolveArchiveBucketPrefersManifestThenConfig(t *testing.T) {
func TestResolvePublishBucketPrefersManifestThenConfig(t *testing.T) {
cfg := &config.Config{
Pipeline: &config.PipelineConfig{
Storage: config.StorageConfig{
@@ -17,15 +17,15 @@ func TestResolveArchiveBucketPrefersManifestThenConfig(t *testing.T) {
},
}
if got := ResolveArchiveBucket(cfg, &manifest.Manifest{S3Bucket: "manifest-bucket"}); got != "manifest-bucket" {
if got := ResolvePublishBucket(cfg, &manifest.Manifest{S3Bucket: "manifest-bucket"}); got != "manifest-bucket" {
t.Fatalf("bucket = %q, want manifest-bucket", got)
}
if got := ResolveArchiveBucket(cfg, &manifest.Manifest{}); got != "cfg-bucket" {
if got := ResolvePublishBucket(cfg, &manifest.Manifest{}); got != "cfg-bucket" {
t.Fatalf("bucket = %q, want cfg-bucket", got)
}
}
func TestResolveArchiveSessionPrefixPrefersManifestThenConfig(t *testing.T) {
func TestResolvePublishSessionPrefixPrefersManifestThenConfig(t *testing.T) {
cfg := &config.Config{
Pipeline: &config.PipelineConfig{
Storage: config.StorageConfig{
@@ -39,17 +39,17 @@ func TestResolveArchiveSessionPrefixPrefersManifestThenConfig(t *testing.T) {
}
m := &manifest.Manifest{S3SessionPrefix: "manifest/session/prefix/"}
got, err := ResolveArchiveSessionPrefix(cfg, m)
got, err := ResolvePublishSessionPrefix(cfg, m)
if err != nil {
t.Fatalf("ResolveArchiveSessionPrefix() error = %v", err)
t.Fatalf("ResolvePublishSessionPrefix() error = %v", err)
}
if got != "manifest/session/prefix/" {
t.Fatalf("session prefix = %q, want manifest/session/prefix/", got)
}
got, err = ResolveArchiveSessionPrefix(cfg, &manifest.Manifest{})
got, err = ResolvePublishSessionPrefix(cfg, &manifest.Manifest{})
if err != nil {
t.Fatalf("ResolveArchiveSessionPrefix() error = %v", err)
t.Fatalf("ResolvePublishSessionPrefix() error = %v", err)
}
want := "dnd/campaigns/forsaken/sessions/2026-04-19/"
if got != want {
@@ -57,7 +57,7 @@ func TestResolveArchiveSessionPrefixPrefersManifestThenConfig(t *testing.T) {
}
}
func TestResolveArchiveRunPrefixPrefersManifestThenDerived(t *testing.T) {
func TestResolvePublishRunPrefixPrefersManifestThenDerived(t *testing.T) {
cfg := &config.Config{
Pipeline: &config.PipelineConfig{
Storage: config.StorageConfig{
@@ -74,9 +74,9 @@ func TestResolveArchiveRunPrefixPrefersManifestThenDerived(t *testing.T) {
RunID: "20260516T010203Z-1a2b3c4d",
S3RunPrefix: "manifest/run/prefix/",
}
got, err := ResolveArchiveRunPrefix(cfg, m)
got, err := ResolvePublishRunPrefix(cfg, m)
if err != nil {
t.Fatalf("ResolveArchiveRunPrefix() error = %v", err)
t.Fatalf("ResolvePublishRunPrefix() error = %v", err)
}
if got != "manifest/run/prefix/" {
t.Fatalf("run prefix = %q, want manifest/run/prefix/", got)
@@ -85,9 +85,9 @@ func TestResolveArchiveRunPrefixPrefersManifestThenDerived(t *testing.T) {
m = &manifest.Manifest{
RunID: "20260516T010203Z-1a2b3c4d",
}
got, err = ResolveArchiveRunPrefix(cfg, m)
got, err = ResolvePublishRunPrefix(cfg, m)
if err != nil {
t.Fatalf("ResolveArchiveRunPrefix() error = %v", err)
t.Fatalf("ResolvePublishRunPrefix() error = %v", err)
}
want := "dnd/campaigns/forsaken/sessions/2026-04-19/runs/20260516T010203Z-1a2b3c4d/"
if got != want {
@@ -95,12 +95,12 @@ func TestResolveArchiveRunPrefixPrefersManifestThenDerived(t *testing.T) {
}
}
func TestResolveArchiveIdentityErrorsAreDeterministic(t *testing.T) {
func TestResolvePublishIdentityErrorsAreDeterministic(t *testing.T) {
cfgNoS3 := &config.Config{
Pipeline: &config.PipelineConfig{},
Session: &config.SessionConfig{SessionID: "2026-04-19", Campaign: "forsaken"},
}
_, err := ResolveArchiveSessionPrefix(cfgNoS3, &manifest.Manifest{})
_, err := ResolvePublishSessionPrefix(cfgNoS3, &manifest.Manifest{})
if err == nil || !strings.Contains(err.Error(), "pipeline.storage.s3 configuration is required") {
t.Fatalf("error = %v, want missing storage.s3", err)
}
@@ -113,14 +113,14 @@ func TestResolveArchiveIdentityErrorsAreDeterministic(t *testing.T) {
},
Session: &config.SessionConfig{SessionID: "2026-04-19", Campaign: "forsaken"},
}
_, err = ResolveArchiveRunPrefix(cfg, &manifest.Manifest{})
_, err = ResolvePublishRunPrefix(cfg, &manifest.Manifest{})
if err == nil || !strings.Contains(err.Error(), "run id is required") {
t.Fatalf("error = %v, want missing run id", err)
}
}
func TestResolveArchiveCurrentStateKeys(t *testing.T) {
manifestKey, runIDKey := ResolveArchiveCurrentStateKeys("dnd/campaigns/forsaken/sessions/2026-04-19/")
func TestResolveCurrentStateKeys(t *testing.T) {
manifestKey, runIDKey := ResolveCurrentStateKeys("dnd/campaigns/forsaken/sessions/2026-04-19/")
if manifestKey != "dnd/campaigns/forsaken/sessions/2026-04-19/current/manifest.json" {
t.Fatalf("manifest key = %q", manifestKey)
}

View File

@@ -43,9 +43,9 @@ func TestS3KeyConstruction(t *testing.T) {
t.Fatalf("manifest key = %q", manifestKey)
}
promoted := S3PublishedOutputKey(sessionPrefix, "transcripts/final.trimmed.json")
if promoted != "dnd/campaigns/forsaken/sessions/2026-04-19/transcripts/final.trimmed.json" {
t.Fatalf("promoted key = %q", promoted)
publishedKey := S3PublishedOutputKey(sessionPrefix, "transcripts/final.trimmed.json")
if publishedKey != "dnd/campaigns/forsaken/sessions/2026-04-19/transcripts/final.trimmed.json" {
t.Fatalf("published key = %q", publishedKey)
}
runRelative := S3RunRelativeDestinationKey(runPrefix, `logs\whisperx.stdout.log`)

View File

@@ -153,7 +153,7 @@ storage:
}
}
func TestSpoolAndArchiveDefaults(t *testing.T) {
func TestSpoolAndPublishDefaults(t *testing.T) {
pipelinePath, sessionPath := writeConfigFiles(t, testPipelineBaseYAML, testSessionBaseYAML)
cfg, err := Load(pipelinePath, sessionPath)
@@ -171,13 +171,13 @@ func TestSpoolAndArchiveDefaults(t *testing.T) {
t.Fatalf("workspace.cleanup_after_publish = true, want false")
}
if cfg.Pipeline.Publish == nil {
t.Fatal("archive should be initialized by defaults")
t.Fatal("publish should be initialized by defaults")
}
if cfg.Pipeline.Publish.Enabled == nil || !*cfg.Pipeline.Publish.Enabled {
t.Fatalf("archive.enabled = %#v, want true", cfg.Pipeline.Publish.Enabled)
t.Fatalf("publish.enabled = %#v, want true", cfg.Pipeline.Publish.Enabled)
}
if cfg.Pipeline.Publish.UploadRun == nil || !*cfg.Pipeline.Publish.UploadRun {
t.Fatalf("archive.upload_run = %#v, want true", cfg.Pipeline.Publish.UploadRun)
t.Fatalf("publish.upload_run = %#v, want true", cfg.Pipeline.Publish.UploadRun)
}
if len(cfg.Pipeline.Publish.Outputs) != 1 {
t.Fatalf("publish.outputs len = %d, want 1 default", len(cfg.Pipeline.Publish.Outputs))
@@ -194,7 +194,7 @@ func TestSpoolAndArchiveDefaults(t *testing.T) {
}
}
func TestArchivePromotionValidation(t *testing.T) {
func TestPublishOutputValidation(t *testing.T) {
tests := []struct {
name string
ruleYML string
@@ -278,7 +278,7 @@ publish:
}
}
func TestArchivePromotionLegacyTranscriptSourcesRejected(t *testing.T) {
func TestPublishOutputLegacyTranscriptSourcesRejected(t *testing.T) {
legacyTranscriptSources := []string{
"narratio.transcript." + "merged",
"narratio.transcript." + "full",
@@ -307,7 +307,7 @@ publish:
}
}
func TestArchivePromotionDerivesDestinationWhenOmitted(t *testing.T) {
func TestPublishOutputDerivesDestinationWhenOmitted(t *testing.T) {
tests := []struct {
name string
pipelineYML string
@@ -359,7 +359,7 @@ publish:
}
}
func TestArchiveLockValidation(t *testing.T) {
func TestPublishLockValidation(t *testing.T) {
tests := []struct {
name string
pipelineYML string
@@ -439,7 +439,7 @@ publish:
}
}
func TestArchiveLockLegacyTranscriptSourcesRejected(t *testing.T) {
func TestPublishLockLegacyTranscriptSourcesRejected(t *testing.T) {
legacyTranscriptSources := []string{
"narratio.transcript." + "merged",
"narratio.transcript." + "full",
@@ -467,7 +467,7 @@ publish:
}
}
func TestArchiveLockUnknownFieldFailsStrictDecode(t *testing.T) {
func TestPublishLockUnknownFieldFailsStrictDecode(t *testing.T) {
pipelineYAML := testPipelineBaseYAML + `
publish:
locks:
@@ -481,7 +481,7 @@ publish:
}
}
func TestArchiveLegacyFromToFailsStrictDecode(t *testing.T) {
func TestPublishLegacyFromToFailsStrictDecode(t *testing.T) {
pipelineYAML := testPipelineBaseYAML + `
publish:
outputs:
@@ -495,7 +495,7 @@ publish:
}
}
func TestArchiveLockStoreBytesStrictDecodeAndValidation(t *testing.T) {
func TestPublishLockStoreBytesStrictDecodeAndValidation(t *testing.T) {
store, err := LoadPublishLockStoreBytes("locks.yml", []byte(`locks:
- source: narratio.transcript.final_trimmed
reason: reviewed
@@ -524,7 +524,7 @@ func TestArchiveLockStoreBytesStrictDecodeAndValidation(t *testing.T) {
}
}
func TestMergeArchiveLockRulesStaticWins(t *testing.T) {
func TestMergePublishLockRulesStaticWins(t *testing.T) {
merged := MergePublishLockRules(
[]PublishLockRule{{Source: "narratio.transcript.final_trimmed", Reason: "static"}},
[]PublishLockRule{

View File

@@ -1,6 +1,7 @@
package config
import (
"errors"
"fmt"
"net/url"
"path/filepath"
@@ -9,6 +10,8 @@ import (
"time"
"gitea.maximumdirect.net/eric/narratio/internal/artifactmodel"
"gitea.maximumdirect.net/eric/narratio/internal/artifactpolicy"
"gitea.maximumdirect.net/eric/narratio/internal/pathsafe"
)
// Validate checks resolved configuration for required fields and parseable durations.
@@ -133,6 +136,7 @@ func validatePublish(cfg *PublishConfig, scriptorium *ScriptoriumConfig) error {
if cfg == nil {
return nil
}
configuredOutputs := scriptoriumOutputPathMap(scriptorium)
seenDest := map[string]struct{}{}
for i, item := range cfg.Outputs {
prefix := fmt.Sprintf("pipeline.publish.outputs[%d]", i)
@@ -140,22 +144,29 @@ func validatePublish(cfg *PublishConfig, scriptorium *ScriptoriumConfig) error {
if source == "" {
return fmt.Errorf("%s.source is required", prefix)
}
if _, err := publishSourceKnown(source, scriptorium); err != nil {
if _, err := artifactpolicy.ValidatePublishSource(source, configuredOutputs); err != nil {
return fmt.Errorf("%s.source %q is unsupported: %w", prefix, item.Source, err)
}
dest := strings.TrimSpace(item.Dest)
if dest == "" {
derivedDest, err := derivePublishOutputDest(source, scriptorium)
derivedDest, err := artifactpolicy.ResolvePublishedDestination(source, "", configuredOutputs)
if err != nil {
return fmt.Errorf("%s.dest is required when destination cannot be derived from %q: %w", prefix, source, err)
}
dest = derivedDest
cfg.Outputs[i].Dest = derivedDest
}
if err := validateRelativeSafePath(prefix+".dest", dest); err != nil {
return err
normalizedDest, err := pathsafe.NormalizeRelativeDestination(dest)
if err != nil {
switch {
case errors.Is(err, pathsafe.ErrRelativePathAbsolute):
return fmt.Errorf("%s.dest must be a relative path", prefix)
case errors.Is(err, pathsafe.ErrRelativePathEscape):
return fmt.Errorf("%s.dest must not contain path traversal", prefix)
default:
return fmt.Errorf("%s.dest must be non-empty", prefix)
}
}
normalizedDest := filepath.ToSlash(filepath.Clean(dest))
if _, ok := seenDest[normalizedDest]; ok {
return fmt.Errorf("%s.dest %q duplicates another publish output destination", prefix, dest)
}
@@ -173,6 +184,7 @@ func validatePublish(cfg *PublishConfig, scriptorium *ScriptoriumConfig) error {
func ValidatePublishLockRules(locks []PublishLockRule, scriptorium *ScriptoriumConfig, label string) ([]PublishLockRule, error) {
seenLocks := map[string]struct{}{}
out := make([]PublishLockRule, 0, len(locks))
configuredOutputs := scriptoriumOutputPathMap(scriptorium)
if strings.TrimSpace(label) == "" {
label = "publish.locks"
}
@@ -182,7 +194,7 @@ func ValidatePublishLockRules(locks []PublishLockRule, scriptorium *ScriptoriumC
if source == "" {
return nil, fmt.Errorf("%s.source is required", prefix)
}
if _, err := publishSourceKnown(source, scriptorium); err != nil {
if _, err := artifactpolicy.ValidatePublishSource(source, configuredOutputs); err != nil {
return nil, fmt.Errorf("%s.source %q is unsupported: %w", prefix, item.Source, err)
}
if _, ok := seenLocks[source]; ok {
@@ -224,48 +236,15 @@ func MergePublishLockRules(staticLocks, remoteLocks []PublishLockRule) []Publish
return out
}
func publishSourceKnown(source string, scriptorium *ScriptoriumConfig) (string, error) {
trimmed := strings.TrimSpace(source)
if _, ok := artifactmodel.LookupRuntimeTranscriptArtifact(trimmed); ok {
return "", nil
}
switch trimmed {
case "narratio.bounds.session":
return "", nil
}
matches := narratioArtifactSourceRE.FindStringSubmatch(trimmed)
if len(matches) != 2 {
return "", fmt.Errorf("must be a built-in source id or narratio.artifact.<name>")
}
artifactKey := matches[1]
func scriptoriumOutputPathMap(scriptorium *ScriptoriumConfig) map[string]string {
out := map[string]string{}
if scriptorium == nil || len(scriptorium.Artifacts) == 0 {
return "", fmt.Errorf("configured artifact %q is not defined in pipeline.scriptorium.artifacts", artifactKey)
return out
}
if _, ok := scriptorium.Artifacts[artifactKey]; !ok {
return "", fmt.Errorf("configured artifact %q is not defined in pipeline.scriptorium.artifacts", artifactKey)
for key, artifactCfg := range scriptorium.Artifacts {
out[strings.TrimSpace(key)] = strings.TrimSpace(artifactCfg.OutputPath)
}
return artifactKey, nil
}
func derivePublishOutputDest(source string, scriptorium *ScriptoriumConfig) (string, error) {
trimmed := strings.TrimSpace(source)
if spec, ok := artifactmodel.LookupRuntimeTranscriptArtifact(trimmed); ok {
return spec.CanonicalRelPath, nil
}
switch trimmed {
case "narratio.bounds.session":
return filepath.ToSlash(filepath.Join(PathArtifactsDirSegment, "session_bounds.json")), nil
}
artifactKey, err := publishSourceKnown(trimmed, scriptorium)
if err != nil {
return "", err
}
artifactCfg := scriptorium.Artifacts[artifactKey]
outputPath := strings.TrimSpace(artifactCfg.OutputPath)
if outputPath == "" {
return "", fmt.Errorf("pipeline.scriptorium.artifacts.%s.output_path is empty", artifactKey)
}
return outputPath, nil
return out
}
func validateSecrets(cfg *SecretsConfig) error {
@@ -657,8 +636,6 @@ func publishUploadConfiguredForS3(pipeline *PipelineConfig) bool {
var windowsAbsPathRE = regexp.MustCompile(`^[A-Za-z]:[\\/].*`)
var envVarNameRE = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
var scriptoriumArtifactKeyRE = regexp.MustCompile(`^[a-z][a-z0-9_]*$`)
var narratioArtifactSourceRE = regexp.MustCompile(`^narratio\.artifact\.([a-z][a-z0-9_]*)$`)
var narratioPreviousSessionArtifactSourceRE = regexp.MustCompile(`^narratio\.previous_session\.artifact\.([a-z][a-z0-9_]*)$`)
func validateScriptoriumInputSource(artifactName, inputName, source string, configuredArtifacts map[string]struct{}) (string, error) {
trimmedSource := strings.TrimSpace(source)
@@ -667,8 +644,8 @@ func validateScriptoriumInputSource(artifactName, inputName, source string, conf
}
if strings.HasPrefix(trimmedSource, "narratio.previous_session.artifact") {
matches := narratioPreviousSessionArtifactSourceRE.FindStringSubmatch(trimmedSource)
if len(matches) != 2 {
referenced, ok := artifactpolicy.ParsePreviousSessionSource(trimmedSource)
if !ok {
return "", fmt.Errorf(
"pipeline.scriptorium.artifacts.%s.inputs.%s.source %q must reference configured artifact key matching ^[a-z][a-z0-9_]*$",
artifactName,
@@ -676,7 +653,6 @@ func validateScriptoriumInputSource(artifactName, inputName, source string, conf
source,
)
}
referenced := matches[1]
if _, ok := configuredArtifacts[referenced]; !ok {
return "", fmt.Errorf(
"pipeline.scriptorium.artifacts.%s.inputs.%s.source %q references unknown artifact %q",
@@ -689,8 +665,8 @@ func validateScriptoriumInputSource(artifactName, inputName, source string, conf
return "", nil
}
matches := narratioArtifactSourceRE.FindStringSubmatch(trimmedSource)
if len(matches) != 2 {
referenced, ok := artifactpolicy.ParseConfiguredSource(trimmedSource)
if !ok {
return "", fmt.Errorf(
"pipeline.scriptorium.artifacts.%s.inputs.%s.source %q is unsupported",
artifactName,
@@ -698,7 +674,6 @@ func validateScriptoriumInputSource(artifactName, inputName, source string, conf
source,
)
}
referenced := matches[1]
if _, ok := configuredArtifacts[referenced]; !ok {
return "", fmt.Errorf(
"pipeline.scriptorium.artifacts.%s.inputs.%s.source %q references unknown artifact %q",

View File

@@ -0,0 +1,38 @@
package pathsafe
import (
"errors"
"path"
"regexp"
"strings"
)
var (
ErrRelativePathRequired = errors.New("relative path is required")
ErrRelativePathAbsolute = errors.New("relative path must not be absolute")
ErrRelativePathEscape = errors.New("relative path escapes root")
)
var windowsAbsPathRE = regexp.MustCompile(`^[a-zA-Z]:[\\/]`)
// NormalizeRelativeDestination validates and normalizes a relative destination
// path to slash-separated form.
func NormalizeRelativeDestination(relative string) (string, error) {
trimmed := strings.TrimSpace(relative)
if trimmed == "" {
return "", ErrRelativePathRequired
}
if strings.HasPrefix(trimmed, "/") || strings.HasPrefix(trimmed, `\`) || windowsAbsPathRE.MatchString(trimmed) {
return "", ErrRelativePathAbsolute
}
normalized := strings.ReplaceAll(trimmed, `\`, "/")
cleaned := path.Clean(normalized)
if cleaned == "." || cleaned == "" {
return "", ErrRelativePathRequired
}
if cleaned == ".." || strings.HasPrefix(cleaned, "../") {
return "", ErrRelativePathEscape
}
return cleaned, nil
}

View File

@@ -0,0 +1,43 @@
package pathsafe
import (
"errors"
"testing"
)
func TestNormalizeRelativeDestination(t *testing.T) {
tests := []struct {
name string
input string
want string
wantErr error
}{
{name: "simple", input: "artifacts/session_recap.md", want: "artifacts/session_recap.md"},
{name: "backslashes become slashes", input: `artifacts\session_recap.md`, want: "artifacts/session_recap.md"},
{name: "cleans dot segments", input: "artifacts/./session_recap.md", want: "artifacts/session_recap.md"},
{name: "reject empty", input: "", wantErr: ErrRelativePathRequired},
{name: "reject dot", input: ".", wantErr: ErrRelativePathRequired},
{name: "reject unix absolute", input: "/artifacts/session_recap.md", wantErr: ErrRelativePathAbsolute},
{name: "reject windows absolute", input: `C:\artifacts\session_recap.md`, wantErr: ErrRelativePathAbsolute},
{name: "reject traversal", input: "../artifacts/session_recap.md", wantErr: ErrRelativePathEscape},
{name: "reject traversal after clean", input: "a/../../session_recap.md", wantErr: ErrRelativePathEscape},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := NormalizeRelativeDestination(tt.input)
if tt.wantErr != nil {
if !errors.Is(err, tt.wantErr) {
t.Fatalf("NormalizeRelativeDestination() error = %v, want %v", err, tt.wantErr)
}
return
}
if err != nil {
t.Fatalf("NormalizeRelativeDestination() error = %v", err)
}
if got != tt.want {
t.Fatalf("NormalizeRelativeDestination() = %q, want %q", got, tt.want)
}
})
}
}

View File

@@ -2,17 +2,19 @@ package previouscache
import (
"context"
"errors"
"fmt"
"os"
"path"
"path/filepath"
"sort"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
"gitea.maximumdirect.net/eric/narratio/internal/artifactpolicy"
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
"gitea.maximumdirect.net/eric/narratio/internal/config"
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
"gitea.maximumdirect.net/eric/narratio/internal/pathsafe"
)
const (
@@ -90,86 +92,28 @@ func BuildPlan(
campaign,
previousSessionID,
)
currentManifestKey, currentRunIDKey := artifacts.ResolveArchiveCurrentStateKeys(previousSessionPrefix)
result := &Plan{}
runPointerExists, err := store.Exists(ctx, currentRunIDKey)
current, err := artifacts.LoadCurrentState(ctx, store, previousSessionPrefix, artifacts.CurrentStateValidation{
ExpectedSessionID: previousSessionID,
ExpectedCampaign: campaign,
ValidateRunID: true,
})
if err != nil {
return nil, fmt.Errorf("check previous-session current run pointer %q: %w", currentRunIDKey, err)
}
if !runPointerExists {
if len(requiredNames) > 0 {
return nil, fmt.Errorf("required previous-session artifacts unavailable: remote current run pointer missing: %q", currentRunIDKey)
var runPointerMissing *artifacts.CurrentRunPointerMissingError
var manifestMissing *artifacts.CurrentManifestMissingError
if errors.As(err, &runPointerMissing) || errors.As(err, &manifestMissing) {
if len(requiredNames) > 0 {
return nil, fmt.Errorf("required previous-session artifacts unavailable: remote %w", err)
}
result.SkippedMissing = optionalNames
return result, nil
}
result.SkippedMissing = optionalNames
return result, nil
}
runIDTemp, err := downloadObjectToTemp(ctx, store, currentRunIDKey, "narratio-previous-run-id-*.txt")
if err != nil {
return nil, fmt.Errorf("download previous-session current run pointer %q: %w", currentRunIDKey, err)
}
defer func() { _ = os.Remove(runIDTemp) }()
runIDBytes, err := os.ReadFile(runIDTemp)
if err != nil {
return nil, fmt.Errorf("read previous-session current run pointer %q: %w", currentRunIDKey, err)
}
previousRunID := strings.TrimSpace(string(runIDBytes))
if previousRunID == "" {
return nil, fmt.Errorf("previous-session current run pointer %q is empty", currentRunIDKey)
}
result.PreviousRunID = previousRunID
manifestExists, err := store.Exists(ctx, currentManifestKey)
if err != nil {
return nil, fmt.Errorf("check previous-session current manifest %q: %w", currentManifestKey, err)
}
if !manifestExists {
if len(requiredNames) > 0 {
return nil, fmt.Errorf("required previous-session artifacts unavailable: remote current manifest missing: %q", currentManifestKey)
}
result.SkippedMissing = optionalNames
return result, nil
}
manifestTemp, err := downloadObjectToTemp(ctx, store, currentManifestKey, "narratio-previous-manifest-*.json")
if err != nil {
return nil, fmt.Errorf("download previous-session current manifest %q: %w", currentManifestKey, err)
}
defer func() { _ = os.Remove(manifestTemp) }()
manifestStore := &manifest.LocalStore{}
previousManifest, err := manifestStore.Load(ctx, manifestTemp)
if err != nil {
return nil, fmt.Errorf("decode downloaded previous-session manifest %q: %w", currentManifestKey, err)
}
if strings.TrimSpace(previousManifest.SessionID) != previousSessionID {
return nil, fmt.Errorf(
"previous-session manifest session_id %q does not match configured previous_session_id %q",
strings.TrimSpace(previousManifest.SessionID),
previousSessionID,
)
}
if strings.TrimSpace(previousManifest.Campaign) != campaign {
return nil, fmt.Errorf(
"previous-session manifest campaign %q does not match current campaign %q",
strings.TrimSpace(previousManifest.Campaign),
campaign,
)
}
if strings.TrimSpace(previousManifest.RunID) == "" {
return nil, fmt.Errorf("previous-session manifest run_id is required")
}
if strings.TrimSpace(previousManifest.RunID) != previousRunID {
return nil, fmt.Errorf(
"previous-session current run pointer %q references run %q but current manifest run_id is %q",
currentRunIDKey,
previousRunID,
strings.TrimSpace(previousManifest.RunID),
)
return nil, fmt.Errorf("load previous-session current state: %w", err)
}
result.PreviousRunID = strings.TrimSpace(current.RunID)
currentManifestKey := current.CurrentManifestKey
previousManifest := current.Manifest
manifestRel, err := relativeToSession(paths, paths.PreviousManifestPath)
if err != nil {
@@ -188,7 +132,7 @@ func BuildPlan(
if len(candidates) == 0 {
if requirement.Required {
return nil, fmt.Errorf(
"required previous-session artifact %q is unavailable in previous-session manifest/archive",
"required previous-session artifact %q is unavailable in previous-session manifest/published-state",
requirement.Name,
)
}
@@ -214,7 +158,7 @@ func BuildPlan(
if selectedRel == "" {
if requirement.Required {
return nil, fmt.Errorf(
"required previous-session artifact %q object missing from archive candidate keys",
"required previous-session artifact %q object missing from published candidate keys",
requirement.Name,
)
}
@@ -278,14 +222,14 @@ func artifactRelativePathCandidates(
) []string {
candidates := []string{}
appendCandidate := func(v string) {
normalized, err := normalizeArchiveRelativePath(v)
normalized, err := pathsafe.NormalizeRelativeDestination(v)
if err != nil {
return
}
candidates = append(candidates, normalized)
}
sourceID := artifacts.ConfiguredArtifactSourceID(artifactName)
sourceID := artifactpolicy.ConfiguredSourceID(artifactName)
if rel, ok := manifestArtifactRelativePathBySourceID(previousManifest, sourceID); ok {
appendCandidate(rel)
base := path.Base(rel)
@@ -354,7 +298,7 @@ func deriveManifestRelativePath(previousManifest *manifest.Manifest, localPath s
return "", false
}
if !filepath.IsAbs(trimmed) {
normalized, err := normalizeArchiveRelativePath(filepath.ToSlash(trimmed))
normalized, err := pathsafe.NormalizeRelativeDestination(filepath.ToSlash(trimmed))
if err != nil {
return "", false
}
@@ -369,7 +313,7 @@ func deriveManifestRelativePath(previousManifest *manifest.Manifest, localPath s
if err != nil {
return "", false
}
normalized, err := normalizeArchiveRelativePath(filepath.ToSlash(rel))
normalized, err := pathsafe.NormalizeRelativeDestination(filepath.ToSlash(rel))
if err != nil {
return "", false
}
@@ -417,7 +361,7 @@ func manifestPublishedPaths(previousManifest *manifest.Manifest) []string {
if !ok {
continue
}
normalized, err := normalizeArchiveRelativePath(asString)
normalized, err := pathsafe.NormalizeRelativeDestination(asString)
if err != nil {
continue
}
@@ -426,21 +370,6 @@ func manifestPublishedPaths(previousManifest *manifest.Manifest) []string {
return dedupeOrderedStrings(out)
}
func normalizeArchiveRelativePath(rel string) (string, error) {
trimmed := strings.TrimSpace(rel)
if trimmed == "" {
return "", fmt.Errorf("relative path is required")
}
cleaned := filepath.ToSlash(filepath.Clean(filepath.FromSlash(trimmed)))
if cleaned == "." || cleaned == "" {
return "", fmt.Errorf("relative path is required")
}
if filepath.IsAbs(trimmed) || strings.HasPrefix(cleaned, "/") || cleaned == ".." || strings.HasPrefix(cleaned, "../") {
return "", fmt.Errorf("path must be a clean relative path")
}
return cleaned, nil
}
func relativeToSession(paths artifacts.SessionPaths, localPath string) (string, error) {
root := filepath.Clean(paths.Root)
if strings.TrimSpace(root) == "" {
@@ -450,7 +379,7 @@ func relativeToSession(paths artifacts.SessionPaths, localPath string) (string,
if err != nil {
return "", fmt.Errorf("resolve previous-cache relative path: %w", err)
}
normalized, err := normalizeArchiveRelativePath(filepath.ToSlash(rel))
normalized, err := pathsafe.NormalizeRelativeDestination(filepath.ToSlash(rel))
if err != nil {
return "", fmt.Errorf("resolve previous-cache relative path: %w", err)
}
@@ -476,20 +405,3 @@ func dedupeOrderedStrings(values []string) []string {
}
return out
}
func downloadObjectToTemp(ctx context.Context, store storage.ObjectStore, key, pattern string) (string, error) {
tmp, err := os.CreateTemp("", pattern)
if err != nil {
return "", fmt.Errorf("create temp file: %w", err)
}
path := tmp.Name()
if err := tmp.Close(); err != nil {
_ = os.Remove(path)
return "", fmt.Errorf("close temp file: %w", err)
}
if err := store.Download(ctx, key, path); err != nil {
_ = os.Remove(path)
return "", err
}
return path, nil
}

View File

@@ -100,11 +100,71 @@ func TestBuildPlanValidatesPreviousManifestIdentity(t *testing.T) {
_, err := BuildPlan(context.Background(), cfg, paths, []artifacts.PreviousArtifactRequirement{
{Name: "session_recap", Required: true},
}, store)
if err == nil || !strings.Contains(err.Error(), "does not match configured previous_session_id") {
if err == nil || !strings.Contains(err.Error(), "does not match expected session_id") {
t.Fatalf("BuildPlan() error = %v, want identity validation error", err)
}
}
func TestBuildPlanMissingCurrentRunPointerSkipsOptionalRequirements(t *testing.T) {
cfg, paths := previousCacheTestConfig(t)
store := &storage.FakeBackend{}
plan, err := BuildPlan(context.Background(), cfg, paths, []artifacts.PreviousArtifactRequirement{
{Name: "session_recap", Required: false},
}, store)
if err != nil {
t.Fatalf("BuildPlan() error = %v", err)
}
if strings.Join(plan.SkippedMissing, ",") != "session_recap" {
t.Fatalf("SkippedMissing = %#v, want session_recap", plan.SkippedMissing)
}
}
func TestBuildPlanMissingCurrentRunPointerFailsRequiredRequirements(t *testing.T) {
cfg, paths := previousCacheTestConfig(t)
store := &storage.FakeBackend{}
_, err := BuildPlan(context.Background(), cfg, paths, []artifacts.PreviousArtifactRequirement{
{Name: "session_recap", Required: true},
}, store)
if err == nil || !strings.Contains(err.Error(), `required previous-session artifacts unavailable: remote current run pointer missing`) {
t.Fatalf("BuildPlan() error = %v, want required missing run pointer error", err)
}
}
func TestBuildPlanMissingCurrentManifestSkipsOptionalRequirements(t *testing.T) {
cfg, paths := previousCacheTestConfig(t)
store := &storage.FakeBackend{}
previousPrefix := artifacts.S3SessionPrefix("dnd", cfg.Session.Campaign, cfg.Session.PreviousSessionID)
_, runIDKey := artifacts.ResolveCurrentStateKeys(previousPrefix)
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("previous-run\n")})
plan, err := BuildPlan(context.Background(), cfg, paths, []artifacts.PreviousArtifactRequirement{
{Name: "session_recap", Required: false},
}, store)
if err != nil {
t.Fatalf("BuildPlan() error = %v", err)
}
if strings.Join(plan.SkippedMissing, ",") != "session_recap" {
t.Fatalf("SkippedMissing = %#v, want session_recap", plan.SkippedMissing)
}
}
func TestBuildPlanMissingCurrentManifestFailsRequiredRequirements(t *testing.T) {
cfg, paths := previousCacheTestConfig(t)
store := &storage.FakeBackend{}
previousPrefix := artifacts.S3SessionPrefix("dnd", cfg.Session.Campaign, cfg.Session.PreviousSessionID)
_, runIDKey := artifacts.ResolveCurrentStateKeys(previousPrefix)
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("previous-run\n")})
_, err := BuildPlan(context.Background(), cfg, paths, []artifacts.PreviousArtifactRequirement{
{Name: "session_recap", Required: true},
}, store)
if err == nil || !strings.Contains(err.Error(), `required previous-session artifacts unavailable: remote current manifest missing`) {
t.Fatalf("BuildPlan() error = %v, want required missing manifest error", err)
}
}
func previousCacheTestConfig(t *testing.T) (*config.Config, artifacts.SessionPaths) {
t.Helper()
workspaceRoot := t.TempDir()
@@ -134,7 +194,7 @@ func previousCacheTestConfig(t *testing.T) (*config.Config, artifacts.SessionPat
func seedPreviousCurrent(t *testing.T, store *storage.FakeBackend, cfg *config.Config, m *manifest.Manifest) {
t.Helper()
previousPrefix := artifacts.S3SessionPrefix(cfg.Pipeline.Storage.S3.RootPrefix, cfg.Session.Campaign, cfg.Session.PreviousSessionID)
manifestKey, runIDKey := artifacts.ResolveArchiveCurrentStateKeys(previousPrefix)
manifestKey, runIDKey := artifacts.ResolveCurrentStateKeys(previousPrefix)
store.SeedObject(storage.FakeObject{Key: runIDKey, Data: []byte("previous-run\n")})
data, err := marshalManifestForPreviousCacheTest(m)
if err != nil {
@@ -143,7 +203,7 @@ func seedPreviousCurrent(t *testing.T, store *storage.FakeBackend, cfg *config.C
store.SeedObject(storage.FakeObject{Key: manifestKey, Data: data})
}
func previousManifestWithOutput(t *testing.T, cfg *config.Config, rel string, promoted []string) *manifest.Manifest {
func previousManifestWithOutput(t *testing.T, cfg *config.Config, rel string, publishedPaths []string) *manifest.Manifest {
t.Helper()
m := manifest.New(cfg.Session.PreviousSessionID, time.Date(2026, 4, 26, 10, 0, 0, 0, time.UTC))
m.Campaign = cfg.Session.Campaign
@@ -154,11 +214,11 @@ func previousManifestWithOutput(t *testing.T, cfg *config.Config, rel string, pr
{SourceID: "narratio.artifact.session_recap", LocalPath: filepath.Join(filepath.Dir(filepath.Dir(m.LocalWorkDir)), filepath.FromSlash(rel))},
})
}
if promoted != nil {
if publishedPaths != nil {
if m.Stages["publish"] == nil {
m.MarkStageSucceeded("publish", time.Date(2026, 4, 26, 10, 2, 0, 0, time.UTC), nil)
}
m.Stages["publish"].Metadata = map[string]any{"published_paths": promoted}
m.Stages["publish"].Metadata = map[string]any{"published_paths": publishedPaths}
}
return m
}

View File

@@ -12,6 +12,7 @@ import (
"time"
"gitea.maximumdirect.net/eric/narratio/internal/adapters/scriptorium"
"gitea.maximumdirect.net/eric/narratio/internal/artifactpolicy"
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
"gitea.maximumdirect.net/eric/narratio/internal/config"
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
@@ -549,12 +550,8 @@ func extractPlanNames(plans []analyzeArtifactExecutionPlan) []string {
}
func configuredArtifactNameFromSourceID(sourceID string) string {
trimmed := strings.TrimSpace(sourceID)
const prefix = "narratio.artifact."
if !strings.HasPrefix(trimmed, prefix) {
return ""
}
return strings.TrimPrefix(trimmed, prefix)
name, _ := artifactpolicy.ParseConfiguredSource(sourceID)
return name
}
func discoverProcessedTranscript(m *manifest.Manifest, paths artifacts.SessionPaths) (string, string, error) {
@@ -631,7 +628,11 @@ func resolveScriptoriumInput(
runtimeCatalog *artifacts.ArtifactCatalog,
) (string, bool, *artifacts.ResolvedSessionArtifact, error) {
source := strings.TrimSpace(inputCfg.Source)
if artifacts.IsPreviousSessionArtifactSource(source) {
classified, classifyErr := artifactpolicy.ClassifySource(source)
if classifyErr != nil {
return "", false, nil, classifyErr
}
if classified.Kind == artifactpolicy.SourceKindPreviousArtifact {
resolved, err := artifacts.ResolvePreviousSessionArtifactWithCatalog(paths, m, source, runtimeCatalog)
if err == nil {
copy := resolved
@@ -656,17 +657,13 @@ func resolveScriptoriumInput(
return resolved.Path, true, &copy, nil
}
if errors.Is(err, artifacts.ErrSessionArtifactNotFound) {
if artifacts.IsConfiguredArtifactSource(source) {
if classified.Kind == artifactpolicy.SourceKindConfiguredArtifact {
if inputCfg.Required {
return "", false, nil, fmt.Errorf("configured artifact source %q is unavailable", source)
}
return "", false, nil, nil
}
normalized, normalizeErr := artifacts.NormalizeSessionArtifactSource(source)
if normalizeErr != nil {
return "", false, nil, normalizeErr
}
switch normalized {
switch classified.ID {
case artifacts.ArtifactTranscriptPolished:
return "", false, nil, nil
case artifacts.ArtifactTranscriptFinal:

View File

@@ -75,7 +75,7 @@ func All() []Stage {
normalizeStage{},
trimStage{},
analyzeStage{},
archiveStage{},
publishStage{},
placeholderStage{name: "notify"},
}
}

View File

@@ -197,10 +197,10 @@ func TestStagesReturnExpectedMetadata(t *testing.T) {
}
if s.Name() == "publish" {
if result.Metadata["stage"] != "publish" {
t.Fatalf("archive metadata = %#v, want stage=publish", result.Metadata)
t.Fatalf("publish metadata = %#v, want stage=publish", result.Metadata)
}
if result.Metadata["uploaded"] != true {
t.Fatalf("archive metadata = %#v, want uploaded=true", result.Metadata)
t.Fatalf("publish metadata = %#v, want uploaded=true", result.Metadata)
}
continue
}
@@ -222,10 +222,10 @@ func TestStagesReturnExpectedMetadata(t *testing.T) {
t.Fatalf("scriptorium run calls = %d, want 0 when scriptorium config is absent", len(sc.RunRequests))
}
if len(st.Requests) != 0 {
t.Fatalf("storage archive calls = %d, want 0", len(st.Requests))
t.Fatalf("storage publish calls = %d, want 0", len(st.Requests))
}
if _, ok := st.Objects["dnd/campaigns/sample-campaign/sessions/2026-05-03/runs/"+m.RunID+"/manifest.json"]; !ok {
t.Fatalf("archive upload missing manifest key in fake object store")
t.Fatalf("publish upload missing manifest key in fake object store")
}
if len(nf.Requests) != 1 {
t.Fatalf("notify calls = %d, want 1", len(nf.Requests))

View File

@@ -193,7 +193,7 @@ func TestHydratePreviousSessionArtifactsDoesNotUseLocalPreviousWorkspaceState(t
writeFile(t, filepath.Join(seed.PreviousSessionRoot, "artifacts", "session_recap.md"), "# local stale recap\n")
_, err := hydratePreviousSessionArtifacts(context.Background(), env, sessionPaths, requirements)
if err == nil || !strings.Contains(err.Error(), "object missing from archive") {
if err == nil || !strings.Contains(err.Error(), "object missing from published candidate keys") {
t.Fatalf("error = %v, want remote-object-missing failure", err)
}
}
@@ -273,7 +273,7 @@ func seedPreviousCurrentState(
campaign := strings.TrimSpace(env.Config.Session.Campaign)
rootPrefix := strings.TrimSpace(env.Config.Pipeline.Storage.S3.RootPrefix)
previousSessionPrefix := artifacts.S3SessionPrefix(rootPrefix, campaign, previousSessionID)
manifestKey, runPointerKey := artifacts.ResolveArchiveCurrentStateKeys(previousSessionPrefix)
manifestKey, runPointerKey := artifacts.ResolveCurrentStateKeys(previousSessionPrefix)
previousRunID := "20260510T010203Z-a1b2c3d4"
previousSessionRoot := filepath.Join(t.TempDir(), "work", campaign, previousSessionID)

View File

@@ -13,18 +13,19 @@ import (
"time"
"gitea.maximumdirect.net/eric/narratio/internal/adapters/storage"
"gitea.maximumdirect.net/eric/narratio/internal/artifactpolicy"
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
"gitea.maximumdirect.net/eric/narratio/internal/config"
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
)
type archiveStage struct{}
type archiveUploadFile struct {
type publishStage struct{}
type publishUploadFile struct {
RelativePath string
LocalPath string
}
var archivePrerequisiteStages = []string{
var publishPrerequisiteStages = []string{
"prepare",
"transcribe",
"merge",
@@ -34,9 +35,9 @@ var archivePrerequisiteStages = []string{
"analyze",
}
func (archiveStage) Name() string { return "publish" }
func (publishStage) Name() string { return "publish" }
func (archiveStage) Declares() IODecl {
func (publishStage) Declares() IODecl {
return IODecl{
Inputs: []artifacts.Ref{
{Kind: "manifest", Category: "input", RelativePath: "manifest.json"},
@@ -44,23 +45,23 @@ func (archiveStage) Declares() IODecl {
}
}
func (archiveStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*StageResult, error) {
func (publishStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*StageResult, error) {
if env == nil || env.Config == nil || env.Config.Pipeline == nil || env.Config.Session == nil {
return nil, fmt.Errorf("publish: resolved config must include pipeline and session")
}
if archiveDisabled(env) {
if publishDisabled(env) {
return &StageResult{
Metadata: map[string]any{
"stage": "publish",
"skipped": true,
"archive_enabled": false,
"publish_enabled": false,
"audio_upload_skipped": true,
"current_pointer_written": false,
},
}, nil
}
if archiveRunUploadDisabled(env) {
if publishRunUploadDisabled(env) {
return &StageResult{
Metadata: map[string]any{
"stage": "publish",
@@ -79,7 +80,7 @@ func (archiveStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*S
return nil, fmt.Errorf("publish: remote object store backend is required when publish run upload is enabled")
}
runRoot, err := resolveArchiveRunRoot(env, m)
runRoot, err := resolvePublishRunRoot(env, m)
if err != nil {
return nil, fmt.Errorf("publish: resolve run root: %w", err)
}
@@ -91,15 +92,15 @@ func (archiveStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*S
return nil, fmt.Errorf("publish: run root %q is not a directory", runRoot)
}
runPrefix, err := artifacts.ResolveArchiveRunPrefix(env.Config, m)
runPrefix, err := artifacts.ResolvePublishRunPrefix(env.Config, m)
if err != nil {
return nil, fmt.Errorf("publish: resolve s3 run prefix: %w", err)
}
sessionPrefix, err := artifacts.ResolveArchiveSessionPrefix(env.Config, m)
sessionPrefix, err := artifacts.ResolvePublishSessionPrefix(env.Config, m)
if err != nil {
return nil, fmt.Errorf("publish: resolve s3 session prefix: %w", err)
}
bucket := artifacts.ResolveArchiveBucket(env.Config, m)
bucket := artifacts.ResolvePublishBucket(env.Config, m)
if bucket == "" {
return nil, fmt.Errorf("publish: resolve s3 bucket: bucket is required")
}
@@ -108,21 +109,21 @@ func (archiveStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*S
return nil, fmt.Errorf("publish: run id is required")
}
manifestSource, err := resolveArchiveRunManifestSource(runRoot)
manifestSource, err := resolvePublishRunManifestSource(runRoot)
if err != nil {
return nil, fmt.Errorf("publish: resolve run manifest source: %w", err)
}
runFiles, err := collectArchiveRunFiles(runRoot, manifestSource)
runFiles, err := collectPublishRunFiles(runRoot, manifestSource)
if err != nil {
return nil, fmt.Errorf("publish: collect run files: %w", err)
}
sessionPaths := archiveSessionPaths(env, m)
previousFiles, err := collectArchivePreviousFiles(sessionPaths.PreviousDir)
sessionPaths := publishSessionPaths(env, m)
previousFiles, err := collectPublishPreviousFiles(sessionPaths.PreviousDir)
if err != nil {
return nil, fmt.Errorf("publish: collect previous files: %w", err)
}
runtimeCatalog, err := buildArchiveRuntimeArtifactCatalog(sessionPaths, env.Config.Pipeline.Scriptorium)
runtimeCatalog, err := buildPublishRuntimeArtifactCatalog(sessionPaths, env.Config.Pipeline.Scriptorium)
if err != nil {
return nil, fmt.Errorf("publish: build runtime artifact catalog: %w", err)
}
@@ -148,12 +149,12 @@ func (archiveStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*S
}
publishedUploaded := make([]string, 0, len(publishOutputs))
for _, promotion := range publishOutputs {
key := artifacts.S3PublishedOutputKey(sessionPrefix, promotion.Dest)
if _, err := env.ObjectStore.Upload(ctx, promotion.LocalPath, key, storage.UploadOptions{}); err != nil {
return nil, fmt.Errorf("publish: upload published output source %q to %q: %w", promotion.Source, key, err)
for _, publishedOutput := range publishOutputs {
key := artifacts.S3PublishedOutputKey(sessionPrefix, publishedOutput.Dest)
if _, err := env.ObjectStore.Upload(ctx, publishedOutput.LocalPath, key, storage.UploadOptions{}); err != nil {
return nil, fmt.Errorf("publish: upload published output source %q to %q: %w", publishedOutput.Source, key, err)
}
publishedUploaded = append(publishedUploaded, promotion.Dest)
publishedUploaded = append(publishedUploaded, publishedOutput.Dest)
}
previousUploaded := make([]string, 0, len(previousFiles))
@@ -165,8 +166,8 @@ func (archiveStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (*S
previousUploaded = append(previousUploaded, file.RelativePath)
}
currentManifestKey, currentRunPointerKey := artifacts.ResolveArchiveCurrentStateKeys(sessionPrefix)
manifestTempPath, err := writeCurrentManifestSnapshot(m, archiveMetadataPreview(
currentManifestKey, currentRunPointerKey := artifacts.ResolveCurrentStateKeys(sessionPrefix)
manifestTempPath, err := writeCurrentManifestSnapshot(m, publishMetadataPreview(
bucket,
runPrefix,
sessionPrefix,
@@ -249,7 +250,7 @@ type publishSkippedUnselectedOutput struct {
Required bool
}
func archiveDisabled(env *Env) bool {
func publishDisabled(env *Env) bool {
cfg := env.Config.Pipeline.Publish
if cfg == nil {
return true
@@ -257,7 +258,7 @@ func archiveDisabled(env *Env) bool {
return cfg.Enabled != nil && !*cfg.Enabled
}
func archiveRunUploadDisabled(env *Env) bool {
func publishRunUploadDisabled(env *Env) bool {
cfg := env.Config.Pipeline.Publish
if cfg == nil {
return true
@@ -269,7 +270,7 @@ func validatePublishPrerequisites(m *manifest.Manifest) error {
if m == nil {
return fmt.Errorf("manifest is required")
}
for _, stageName := range archivePrerequisiteStages {
for _, stageName := range publishPrerequisiteStages {
sr := m.Stages[stageName]
if sr == nil {
return fmt.Errorf("prerequisite stage %q has not succeeded", stageName)
@@ -281,7 +282,7 @@ func validatePublishPrerequisites(m *manifest.Manifest) error {
return nil
}
func resolveArchiveRunRoot(env *Env, m *manifest.Manifest) (string, error) {
func resolvePublishRunRoot(env *Env, m *manifest.Manifest) (string, error) {
sessionID := strings.TrimSpace(env.Config.Session.SessionID)
if sessionID == "" && m != nil {
sessionID = strings.TrimSpace(m.SessionID)
@@ -312,7 +313,7 @@ func resolveArchiveRunRoot(env *Env, m *manifest.Manifest) (string, error) {
return canonical, nil
}
func resolveArchiveSessionRoot(env *Env, m *manifest.Manifest) (string, error) {
func resolvePublishSessionRoot(env *Env, m *manifest.Manifest) (string, error) {
sessionID := strings.TrimSpace(env.Config.Session.SessionID)
if sessionID == "" && m != nil {
sessionID = strings.TrimSpace(m.SessionID)
@@ -330,7 +331,7 @@ func resolveArchiveSessionRoot(env *Env, m *manifest.Manifest) (string, error) {
return filepath.Clean(artifacts.SessionWorkDirForCampaign(env.Config.Pipeline.Workspace.Root, campaign, sessionID)), nil
}
func archiveSessionPaths(env *Env, m *manifest.Manifest) artifacts.SessionPaths {
func publishSessionPaths(env *Env, m *manifest.Manifest) artifacts.SessionPaths {
sessionID := strings.TrimSpace(env.Config.Session.SessionID)
if sessionID == "" && m != nil {
sessionID = strings.TrimSpace(m.SessionID)
@@ -356,17 +357,18 @@ func resolvePublishOutputs(
skippedOptionalOutputs := make([]string, 0)
skippedUnselectedOutputs := make([]publishSkippedUnselectedOutput, 0)
lockedOutputs := make([]publishLockedOutput, 0)
lockSet := archiveLockSet(locks)
selectedSet := archiveSelectedArtifactSet(selectedArtifactKeys)
lockSet := publishLockSet(locks)
selectedSet := publishSelectedArtifactSet(selectedArtifactKeys)
configuredOutputs := configuredOutputPathMapFromCatalog(catalog)
for _, rule := range rules {
source := strings.TrimSpace(rule.Source)
required := rule.Required == nil || *rule.Required
dest, err := resolvePublishOutputDest(rule, catalog)
dest, err := resolvePublishOutputDest(rule, configuredOutputs)
if err != nil {
return nil, nil, nil, nil, fmt.Errorf("source %q: %w", source, err)
}
if len(selectedSet) > 0 {
if key, ok := artifacts.ConfiguredArtifactName(source); ok {
if key, ok := artifactpolicy.ParseConfiguredSource(source); ok {
if _, selected := selectedSet[key]; !selected {
skippedUnselectedOutputs = append(skippedUnselectedOutputs, publishSkippedUnselectedOutput{
Source: source,
@@ -422,7 +424,7 @@ func resolvePublishOutputs(
return out, skippedOptionalOutputs, skippedUnselectedOutputs, lockedOutputs, nil
}
func archiveSelectedArtifactSet(selected []string) map[string]struct{} {
func publishSelectedArtifactSet(selected []string) map[string]struct{} {
if len(selected) == 0 {
return nil
}
@@ -437,7 +439,7 @@ func archiveSelectedArtifactSet(selected []string) map[string]struct{} {
return out
}
func archiveLockSet(locks []config.PublishLockRule) map[string]config.PublishLockRule {
func publishLockSet(locks []config.PublishLockRule) map[string]config.PublishLockRule {
out := make(map[string]config.PublishLockRule, len(locks))
for _, lock := range locks {
source := strings.TrimSpace(lock.Source)
@@ -451,37 +453,25 @@ func archiveLockSet(locks []config.PublishLockRule) map[string]config.PublishLoc
return out
}
func resolvePublishOutputDest(rule config.PublishOutputRule, catalog *artifacts.ArtifactCatalog) (string, error) {
dest := strings.TrimSpace(rule.Dest)
if dest == "" {
entry, ok := catalog.Lookup(strings.TrimSpace(rule.Source))
if !ok {
return "", fmt.Errorf("destination omitted and source is unknown")
}
dest = strings.TrimSpace(entry.CanonicalRelPath)
if dest == "" {
return "", fmt.Errorf("destination omitted and no canonical destination is available")
}
}
return normalizeArchiveRelativePath(dest)
func resolvePublishOutputDest(rule config.PublishOutputRule, configured map[string]string) (string, error) {
return artifactpolicy.ResolvePublishedDestination(rule.Source, rule.Dest, configured)
}
func normalizeArchiveRelativePath(rel string) (string, error) {
trimmed := strings.TrimSpace(rel)
if trimmed == "" {
return "", fmt.Errorf("relative path is required")
func configuredOutputPathMapFromCatalog(catalog *artifacts.ArtifactCatalog) map[string]string {
out := map[string]string{}
if catalog == nil {
return out
}
cleaned := filepath.ToSlash(filepath.Clean(filepath.FromSlash(trimmed)))
if cleaned == "." || cleaned == "" {
return "", fmt.Errorf("relative path is required")
for _, entry := range catalog.ListConfigured() {
if strings.TrimSpace(entry.ConfiguredKey) == "" {
continue
}
out[entry.ConfiguredKey] = strings.TrimSpace(entry.CanonicalRelPath)
}
if filepath.IsAbs(trimmed) || strings.HasPrefix(cleaned, "/") || cleaned == ".." || strings.HasPrefix(cleaned, "../") {
return "", fmt.Errorf("path must be a clean relative path")
}
return cleaned, nil
return out
}
func buildArchiveRuntimeArtifactCatalog(
func buildPublishRuntimeArtifactCatalog(
paths artifacts.SessionPaths,
scriptoriumCfg *config.ScriptoriumConfig,
) (*artifacts.ArtifactCatalog, error) {
@@ -548,8 +538,8 @@ func resolveConfiguredArtifactLocalPath(paths artifacts.SessionPaths, configured
return filepath.Join(paths.Root, rel), nil
}
func collectArchiveRunFiles(runRoot, manifestPath string) ([]archiveUploadFile, error) {
files := make([]archiveUploadFile, 0, 64)
func collectPublishRunFiles(runRoot, manifestPath string) ([]publishUploadFile, error) {
files := make([]publishUploadFile, 0, 64)
err := filepath.WalkDir(runRoot, func(path string, d fs.DirEntry, walkErr error) error {
if walkErr != nil {
return walkErr
@@ -563,7 +553,7 @@ func collectArchiveRunFiles(runRoot, manifestPath string) ([]archiveUploadFile,
return fmt.Errorf("relative dir from %q to %q: %w", runRoot, path, err)
}
relDir = filepath.ToSlash(relDir)
// Preserve existing behavior: audio is not uploaded in archive run record.
// Preserve existing behavior: audio is not uploaded in publish run record.
if relDir == "audio" || strings.HasPrefix(relDir, "audio/") {
return filepath.SkipDir
}
@@ -574,7 +564,7 @@ func collectArchiveRunFiles(runRoot, manifestPath string) ([]archiveUploadFile,
return fmt.Errorf("relative path from %q to %q: %w", runRoot, path, err)
}
rel = filepath.ToSlash(rel)
files = append(files, archiveUploadFile{
files = append(files, publishUploadFile{
RelativePath: rel,
LocalPath: path,
})
@@ -594,11 +584,11 @@ func collectArchiveRunFiles(runRoot, manifestPath string) ([]archiveUploadFile,
if manifestInfo.IsDir() {
return nil, fmt.Errorf("manifest path %q is a directory", manifestPath)
}
files = append(files, archiveUploadFile{
files = append(files, publishUploadFile{
RelativePath: "manifest.json",
LocalPath: manifestPath,
})
seen := map[string]archiveUploadFile{}
seen := map[string]publishUploadFile{}
for _, file := range files {
seen[file.RelativePath] = file
}
@@ -613,7 +603,7 @@ func collectArchiveRunFiles(runRoot, manifestPath string) ([]archiveUploadFile,
return files, nil
}
func collectArchivePreviousFiles(previousDir string) ([]archiveUploadFile, error) {
func collectPublishPreviousFiles(previousDir string) ([]publishUploadFile, error) {
previousDir = filepath.Clean(strings.TrimSpace(previousDir))
if previousDir == "" {
return nil, fmt.Errorf("previous directory is required")
@@ -629,7 +619,7 @@ func collectArchivePreviousFiles(previousDir string) ([]archiveUploadFile, error
return nil, fmt.Errorf("previous path %q is not a directory", previousDir)
}
files := make([]archiveUploadFile, 0, 16)
files := make([]publishUploadFile, 0, 16)
err = filepath.WalkDir(previousDir, func(path string, d fs.DirEntry, walkErr error) error {
if walkErr != nil {
return walkErr
@@ -642,7 +632,7 @@ func collectArchivePreviousFiles(previousDir string) ([]archiveUploadFile, error
return fmt.Errorf("relative path from %q to %q: %w", previousDir, path, err)
}
rel = filepath.ToSlash(rel)
files = append(files, archiveUploadFile{
files = append(files, publishUploadFile{
RelativePath: filepath.ToSlash(filepath.Join(config.PathPreviousDirSegment, rel)),
LocalPath: path,
})
@@ -658,7 +648,7 @@ func collectArchivePreviousFiles(previousDir string) ([]archiveUploadFile, error
return files, nil
}
func resolveArchiveRunManifestSource(runRoot string) (string, error) {
func resolvePublishRunManifestSource(runRoot string) (string, error) {
path := filepath.Join(filepath.Clean(runRoot), "manifest.json")
info, err := os.Stat(path)
if err != nil {
@@ -684,7 +674,7 @@ func directoryExists(path string) (bool, error) {
return false, err
}
func writeCurrentManifestSnapshot(m *manifest.Manifest, archiveMetadata map[string]any) (string, error) {
func writeCurrentManifestSnapshot(m *manifest.Manifest, publishMetadata map[string]any) (string, error) {
if m == nil {
return "", fmt.Errorf("manifest is required")
}
@@ -717,7 +707,7 @@ func writeCurrentManifestSnapshot(m *manifest.Manifest, archiveMetadata map[stri
now := time.Now().UTC()
clone.MarkStageSucceeded("publish", now, nil)
if sr := clone.Stages["publish"]; sr != nil {
sr.Metadata = archiveMetadata
sr.Metadata = publishMetadata
}
data, err := json.MarshalIndent(&clone, "", " ")
@@ -757,7 +747,7 @@ func writeCurrentRunIDPointer(runID string) (string, error) {
return path, nil
}
func archiveMetadataPreview(
func publishMetadataPreview(
bucket, runPrefix, sessionPrefix string,
runUploaded []string,
publishedUploaded []string,

View File

@@ -17,11 +17,11 @@ import (
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
)
func TestArchiveSkipsWhenDisabled(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishSkipsWhenDisabled(t *testing.T) {
env, m, _ := publishFixture(t)
env.Config.Pipeline.Publish.Enabled = boolPtr(false)
result, err := archiveStage{}.Run(context.Background(), env, m)
result, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
@@ -29,15 +29,15 @@ func TestArchiveSkipsWhenDisabled(t *testing.T) {
t.Fatalf("metadata = %#v, want skipped=true", result.Metadata)
}
if len(env.ObjectStore.(*storage.FakeBackend).Uploads) != 0 {
t.Fatalf("unexpected uploads when archive disabled")
t.Fatalf("unexpected uploads when publish disabled")
}
}
func TestArchiveSkipsRunUploadWhenDisabled(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishSkipsRunUploadWhenDisabled(t *testing.T) {
env, m, _ := publishFixture(t)
env.Config.Pipeline.Publish.UploadRun = boolPtr(false)
result, err := archiveStage{}.Run(context.Background(), env, m)
result, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
@@ -49,11 +49,11 @@ func TestArchiveSkipsRunUploadWhenDisabled(t *testing.T) {
}
}
func TestArchiveFailsWhenPrerequisiteNotSucceeded(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishFailsWhenPrerequisiteNotSucceeded(t *testing.T) {
env, m, _ := publishFixture(t)
m.Stages["trim"].Status = manifest.StatusFailed
_, err := archiveStage{}.Run(context.Background(), env, m)
_, err := publishStage{}.Run(context.Background(), env, m)
if err == nil || !strings.Contains(err.Error(), `prerequisite stage "trim"`) {
t.Fatalf("Run() error = %v, want prerequisite failure", err)
}
@@ -62,11 +62,11 @@ func TestArchiveFailsWhenPrerequisiteNotSucceeded(t *testing.T) {
}
}
func TestArchiveUploadsRunRecordPromotionsAndCurrentPointer(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishUploadsRunRecordPublishedOutputsAndCurrentPointer(t *testing.T) {
env, m, _ := publishFixture(t)
fake := env.ObjectStore.(*storage.FakeBackend)
result, err := archiveStage{}.Run(context.Background(), env, m)
result, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
@@ -94,10 +94,10 @@ func TestArchiveUploadsRunRecordPromotionsAndCurrentPointer(t *testing.T) {
trimmedKey := sessionPrefix + "transcripts/final.trimmed.json"
recapKey := sessionPrefix + "artifacts/session_recap.md"
if _, ok := fake.Objects[trimmedKey]; !ok {
t.Fatalf("missing promoted key %q", trimmedKey)
t.Fatalf("missing published output key %q", trimmedKey)
}
if _, ok := fake.Objects[recapKey]; !ok {
t.Fatalf("missing promoted key %q", recapKey)
t.Fatalf("missing published output key %q", recapKey)
}
currentManifestKey := sessionPrefix + "current/manifest.json"
@@ -136,8 +136,8 @@ func TestArchiveUploadsRunRecordPromotionsAndCurrentPointer(t *testing.T) {
}
}
func TestArchiveUploadsPreviousCacheWhenPresent(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishUploadsPreviousCacheWhenPresent(t *testing.T) {
env, m, _ := publishFixture(t)
fake := env.ObjectStore.(*storage.FakeBackend)
sessionRoot := artifacts.SessionWorkDirForCampaign(
env.Config.Pipeline.Workspace.Root,
@@ -147,7 +147,7 @@ func TestArchiveUploadsPreviousCacheWhenPresent(t *testing.T) {
writeStageTestFile(t, filepath.Join(sessionRoot, "previous", "manifest.json"), "{\"session_id\":\"2026-04-12\"}\n")
writeStageTestFile(t, filepath.Join(sessionRoot, "previous", "artifacts", "session_recap.md"), "# previous recap\n")
result, err := archiveStage{}.Run(context.Background(), env, m)
result, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
@@ -155,20 +155,20 @@ func TestArchiveUploadsPreviousCacheWhenPresent(t *testing.T) {
previousManifestKey := m.S3SessionPrefix + "previous/manifest.json"
previousRecapKey := m.S3SessionPrefix + "previous/artifacts/session_recap.md"
if _, ok := fake.Objects[previousManifestKey]; !ok {
t.Fatalf("missing archived previous manifest key %q", previousManifestKey)
t.Fatalf("missing published previous manifest key %q", previousManifestKey)
}
if _, ok := fake.Objects[previousRecapKey]; !ok {
t.Fatalf("missing archived previous artifact key %q", previousRecapKey)
t.Fatalf("missing published previous artifact key %q", previousRecapKey)
}
if result.Metadata["previous_files_uploaded"] != 2 {
t.Fatalf("metadata previous_files_uploaded = %#v, want 2", result.Metadata["previous_files_uploaded"])
}
}
func TestArchiveToleratesMissingPreviousCache(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishToleratesMissingPreviousCache(t *testing.T) {
env, m, _ := publishFixture(t)
result, err := archiveStage{}.Run(context.Background(), env, m)
result, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
@@ -177,35 +177,35 @@ func TestArchiveToleratesMissingPreviousCache(t *testing.T) {
}
}
func TestArchiveUsesCustomPromotionRules(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishUsesCustomOutputRules(t *testing.T) {
env, m, _ := publishFixture(t)
env.Config.Pipeline.Publish.Outputs = []config.PublishOutputRule{
{Source: "narratio.transcript.final_trimmed", Dest: "published/trimmed.json", Required: boolPtr(true)},
{Source: "narratio.artifact.session_recap", Dest: "published/recap.md", Required: boolPtr(true)},
}
_, err := archiveStage{}.Run(context.Background(), env, m)
_, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
fake := env.ObjectStore.(*storage.FakeBackend)
if _, ok := fake.Objects[m.S3SessionPrefix+"published/trimmed.json"]; !ok {
t.Fatalf("missing custom promoted trimmed key")
t.Fatalf("missing custom published trimmed key")
}
if _, ok := fake.Objects[m.S3SessionPrefix+"published/recap.md"]; !ok {
t.Fatalf("missing custom promoted recap key")
t.Fatalf("missing custom published recap key")
}
}
func TestArchiveSkipsOptionalMissingPromotion(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishSkipsOptionalMissingOutput(t *testing.T) {
env, m, _ := publishFixture(t)
env.Config.Pipeline.Publish.Outputs = []config.PublishOutputRule{
{Source: "narratio.transcript.final_trimmed", Dest: "transcripts/final.trimmed.json", Required: boolPtr(true)},
{Source: "narratio.transcript.base", Dest: "transcripts/base.json", Required: boolPtr(false)},
}
result, err := archiveStage{}.Run(context.Background(), env, m)
result, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
@@ -216,8 +216,8 @@ func TestArchiveSkipsOptionalMissingPromotion(t *testing.T) {
}
}
func TestArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishSkipsRequiredUnselectedConfiguredOutput(t *testing.T) {
env, m, _ := publishFixture(t)
env.SelectedArtifactKeys = []string{"player_handout"}
env.Config.Pipeline.Scriptorium.Artifacts["player_handout"] = config.ScriptoriumArtifactConfig{
Enabled: true,
@@ -226,15 +226,15 @@ func TestArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testing.T) {
}
fake := env.ObjectStore.(*storage.FakeBackend)
result, err := archiveStage{}.Run(context.Background(), env, m)
result, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
if _, ok := fake.Objects[m.S3SessionPrefix+"transcripts/final.trimmed.json"]; !ok {
t.Fatalf("missing built-in promoted trimmed key")
t.Fatalf("missing built-in published trimmed key")
}
if _, ok := fake.Objects[m.S3SessionPrefix+"artifacts/session_recap.md"]; ok {
t.Fatalf("unexpected unselected recap promotion upload")
t.Fatalf("unexpected unselected recap published output upload")
}
skipped := result.Metadata["skipped_unselected_outputs"].([]map[string]any)
if len(skipped) != 1 {
@@ -251,8 +251,8 @@ func TestArchiveSkipsRequiredUnselectedConfiguredPromotion(t *testing.T) {
}
}
func TestArchiveSelectedConfiguredPromotionStillFailsWhenMissing(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishSelectedConfiguredOutputStillFailsWhenMissing(t *testing.T) {
env, m, _ := publishFixture(t)
env.SelectedArtifactKeys = []string{"session_recap"}
sessionRoot := artifacts.SessionWorkDirForCampaign(
env.Config.Pipeline.Workspace.Root,
@@ -263,26 +263,26 @@ func TestArchiveSelectedConfiguredPromotionStillFailsWhenMissing(t *testing.T) {
t.Fatalf("remove recap: %v", err)
}
_, err := archiveStage{}.Run(context.Background(), env, m)
_, err := publishStage{}.Run(context.Background(), env, m)
if err == nil || !strings.Contains(err.Error(), `required output source unavailable: "narratio.artifact.session_recap"`) {
t.Fatalf("Run() error = %v, want required selected output failure", err)
}
}
func TestArchiveLockedSelectedPromotionSkipsAsLocked(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishLockedSelectedOutputSkipsAsLocked(t *testing.T) {
env, m, _ := publishFixture(t)
env.SelectedArtifactKeys = []string{"session_recap"}
env.Config.Pipeline.Publish.Locks = []config.PublishLockRule{
{Source: "narratio.artifact.session_recap", Reason: "reviewed"},
}
fake := env.ObjectStore.(*storage.FakeBackend)
result, err := archiveStage{}.Run(context.Background(), env, m)
result, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
if _, ok := fake.Objects[m.S3SessionPrefix+"artifacts/session_recap.md"]; ok {
t.Fatalf("unexpected locked recap promotion upload")
t.Fatalf("unexpected locked recap published output upload")
}
if result.Metadata["locked_output_count"] != 1 {
t.Fatalf("locked_output_count = %#v, want 1", result.Metadata["locked_output_count"])
@@ -293,8 +293,8 @@ func TestArchiveLockedSelectedPromotionSkipsAsLocked(t *testing.T) {
}
}
func TestArchiveLockedUnselectedConfiguredPromotionSkipsAsUnselected(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishLockedUnselectedConfiguredOutputSkipsAsUnselected(t *testing.T) {
env, m, _ := publishFixture(t)
env.SelectedArtifactKeys = []string{"player_handout"}
env.Config.Pipeline.Scriptorium.Artifacts["player_handout"] = config.ScriptoriumArtifactConfig{
Enabled: true,
@@ -305,7 +305,7 @@ func TestArchiveLockedUnselectedConfiguredPromotionSkipsAsUnselected(t *testing.
{Source: "narratio.artifact.session_recap", Reason: "reviewed"},
}
result, err := archiveStage{}.Run(context.Background(), env, m)
result, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
@@ -318,25 +318,25 @@ func TestArchiveLockedUnselectedConfiguredPromotionSkipsAsUnselected(t *testing.
}
}
func TestArchiveSkipsLockedRequiredPromotionAndCommits(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishSkipsLockedRequiredOutputAndCommits(t *testing.T) {
env, m, _ := publishFixture(t)
env.Config.Pipeline.Publish.Locks = []config.PublishLockRule{
{Source: "narratio.transcript.final_trimmed", Reason: "human reviewed"},
}
fake := env.ObjectStore.(*storage.FakeBackend)
result, err := archiveStage{}.Run(context.Background(), env, m)
result, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
trimmedKey := m.S3SessionPrefix + "transcripts/final.trimmed.json"
if _, ok := fake.Objects[trimmedKey]; ok {
t.Fatalf("locked promotion key %q should not be uploaded", trimmedKey)
t.Fatalf("locked published output key %q should not be uploaded", trimmedKey)
}
recapKey := m.S3SessionPrefix + "artifacts/session_recap.md"
if _, ok := fake.Objects[recapKey]; !ok {
t.Fatalf("unlocked promotion key %q should be uploaded", recapKey)
t.Fatalf("unlocked published output key %q should be uploaded", recapKey)
}
runTrimmedKey := m.S3RunPrefix + "trim/outputs/transcripts/final.trimmed.json"
if _, ok := fake.Objects[runTrimmedKey]; !ok {
@@ -365,7 +365,7 @@ func TestArchiveSkipsLockedRequiredPromotionAndCommits(t *testing.T) {
locked[0]["required"] != true ||
locked[0]["local_path"] == "" ||
locked[0]["provenance"] == "" {
t.Fatalf("locked promotion metadata = %#v", locked[0])
t.Fatalf("locked published output metadata = %#v", locked[0])
}
currentManifestKey := m.S3SessionPrefix + "current/manifest.json"
@@ -374,8 +374,8 @@ func TestArchiveSkipsLockedRequiredPromotionAndCommits(t *testing.T) {
t.Fatalf("unmarshal current manifest: %v", err)
}
stages := current["stages"].(map[string]any)
archive := stages["publish"].(map[string]any)
meta := archive["metadata"].(map[string]any)
publishRecord := stages["publish"].(map[string]any)
meta := publishRecord["metadata"].(map[string]any)
if meta["locked_output_count"] != float64(1) {
t.Fatalf("current manifest locked_output_count = %#v, want 1", meta["locked_output_count"])
}
@@ -385,8 +385,8 @@ func TestArchiveSkipsLockedRequiredPromotionAndCommits(t *testing.T) {
}
}
func TestArchiveLockedRequiredMissingPromotionSucceeds(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishLockedRequiredMissingOutputSucceeds(t *testing.T) {
env, m, _ := publishFixture(t)
env.Config.Pipeline.Publish.Outputs = []config.PublishOutputRule{
{Source: "narratio.transcript.base", Dest: "transcripts/base.json", Required: boolPtr(true)},
}
@@ -394,7 +394,7 @@ func TestArchiveLockedRequiredMissingPromotionSucceeds(t *testing.T) {
{Source: "narratio.transcript.base", Reason: "manual merge is locked"},
}
result, err := archiveStage{}.Run(context.Background(), env, m)
result, err := publishStage{}.Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
@@ -402,22 +402,22 @@ func TestArchiveLockedRequiredMissingPromotionSucceeds(t *testing.T) {
fake := env.ObjectStore.(*storage.FakeBackend)
mergedKey := m.S3SessionPrefix + "transcripts/base.json"
if _, ok := fake.Objects[mergedKey]; ok {
t.Fatalf("locked missing promotion key %q should not be uploaded", mergedKey)
t.Fatalf("locked missing published output key %q should not be uploaded", mergedKey)
}
if _, ok := fake.Objects[m.S3SessionPrefix+"current/run_id.txt"]; !ok {
t.Fatalf("current run pointer should be written for locked missing promotion")
t.Fatalf("current run pointer should be written for locked missing output")
}
locked := result.Metadata["locked_outputs"].([]map[string]any)
if len(locked) != 1 {
t.Fatalf("locked_outputs = %#v, want one item", locked)
}
if locked[0]["local_path"] != "" || locked[0]["provenance"] != "" {
t.Fatalf("locked missing promotion metadata = %#v, want empty local path/provenance", locked[0])
t.Fatalf("locked missing published output metadata = %#v, want empty local path/provenance", locked[0])
}
}
func TestArchiveLockDoesNotOverwriteExistingPromotion(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishLockDoesNotOverwriteExistingOutput(t *testing.T) {
env, m, _ := publishFixture(t)
env.Config.Pipeline.Publish.Locks = []config.PublishLockRule{
{Source: "narratio.transcript.final_trimmed", Reason: "already published"},
}
@@ -425,40 +425,40 @@ func TestArchiveLockDoesNotOverwriteExistingPromotion(t *testing.T) {
trimmedKey := m.S3SessionPrefix + "transcripts/final.trimmed.json"
fake.SeedObject(storage.FakeObject{Key: trimmedKey, Data: []byte("previously published\n")})
if _, err := (archiveStage{}).Run(context.Background(), env, m); err != nil {
if _, err := (publishStage{}).Run(context.Background(), env, m); err != nil {
t.Fatalf("Run() error = %v", err)
}
got := string(fake.Objects[trimmedKey].Data)
if got != "previously published\n" {
t.Fatalf("locked promotion object contents = %q, want existing object preserved", got)
t.Fatalf("locked published output object contents = %q, want existing object preserved", got)
}
for _, upload := range fake.Uploads {
if upload.Key == trimmedKey {
t.Fatalf("locked promotion key %q was uploaded", trimmedKey)
t.Fatalf("locked published output key %q was uploaded", trimmedKey)
}
}
}
func TestArchiveFailsWhenRequiredPromotionMissing(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishFailsWhenRequiredOutputMissing(t *testing.T) {
env, m, _ := publishFixture(t)
env.Config.Pipeline.Publish.Outputs = []config.PublishOutputRule{
{Source: "narratio.transcript.base", Dest: "transcripts/base.json", Required: boolPtr(true)},
}
_, err := archiveStage{}.Run(context.Background(), env, m)
_, err := publishStage{}.Run(context.Background(), env, m)
if err == nil || !strings.Contains(err.Error(), "required output source unavailable") {
t.Fatalf("Run() error = %v, want required output source unavailable failure", err)
}
}
func TestArchiveFailsWhenCanonicalRunRootMissing(t *testing.T) {
env, m, runRoot := archiveFixture(t)
func TestPublishFailsWhenCanonicalRunRootMissing(t *testing.T) {
env, m, runRoot := publishFixture(t)
if err := os.RemoveAll(runRoot); err != nil {
t.Fatalf("remove run root: %v", err)
}
_, err := archiveStage{}.Run(context.Background(), env, m)
_, err := publishStage{}.Run(context.Background(), env, m)
if err == nil {
t.Fatal("expected missing run-root error, got nil")
}
@@ -467,8 +467,8 @@ func TestArchiveFailsWhenCanonicalRunRootMissing(t *testing.T) {
}
}
func TestArchiveDoesNotWriteCurrentPointerWhenPromotionUploadFails(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishDoesNotWriteCurrentPointerWhenOutputUploadFails(t *testing.T) {
env, m, _ := publishFixture(t)
fake := env.ObjectStore.(*storage.FakeBackend)
trimmedKey := m.S3SessionPrefix + "transcripts/final.trimmed.json"
@@ -480,26 +480,26 @@ func TestArchiveDoesNotWriteCurrentPointerWhenPromotionUploadFails(t *testing.T)
originalUpload := fake.Upload
_ = originalUpload
// Use UploadErr toggle by checking call sequence in postcondition.
// First failure point is promotion upload; simulate by setting error immediately before promotion key write.
// First failure point is published output upload; simulate by setting error immediately before published output key write.
// We cannot hook FakeBackend per-key without changing public behavior; use dedicated backend wrapper instead.
env.ObjectStore = &promotionFailingStore{delegate: fake, failKey: failingKey}
env.ObjectStore = &publishedOutputFailingStore{delegate: fake, failKey: failingKey}
_, err := archiveStage{}.Run(context.Background(), env, m)
_, err := publishStage{}.Run(context.Background(), env, m)
if err == nil || !strings.Contains(err.Error(), "upload published output source") {
t.Fatalf("Run() error = %v, want published output upload failure", err)
}
if _, ok := fake.Objects[m.S3SessionPrefix+"current/run_id.txt"]; ok {
t.Fatalf("unexpected current pointer write on promotion failure")
t.Fatalf("unexpected current pointer write on published output failure")
}
fake.UploadErr = origUploadErr
}
func TestArchiveDoesNotWriteCurrentPointerWhenCurrentManifestUploadFails(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishDoesNotWriteCurrentPointerWhenCurrentManifestUploadFails(t *testing.T) {
env, m, _ := publishFixture(t)
fake := env.ObjectStore.(*storage.FakeBackend)
env.ObjectStore = &promotionFailingStore{delegate: fake, failKey: m.S3SessionPrefix + "current/manifest.json"}
env.ObjectStore = &publishedOutputFailingStore{delegate: fake, failKey: m.S3SessionPrefix + "current/manifest.json"}
_, err := archiveStage{}.Run(context.Background(), env, m)
_, err := publishStage{}.Run(context.Background(), env, m)
if err == nil || !strings.Contains(err.Error(), "current manifest") {
t.Fatalf("Run() error = %v, want current manifest upload failure", err)
}
@@ -508,17 +508,17 @@ func TestArchiveDoesNotWriteCurrentPointerWhenCurrentManifestUploadFails(t *test
}
}
func TestArchiveFailsWithoutObjectStore(t *testing.T) {
env, m, _ := archiveFixture(t)
func TestPublishFailsWithoutObjectStore(t *testing.T) {
env, m, _ := publishFixture(t)
env.ObjectStore = nil
_, err := archiveStage{}.Run(context.Background(), env, m)
_, err := publishStage{}.Run(context.Background(), env, m)
if err == nil || !strings.Contains(err.Error(), "object store") {
t.Fatalf("Run() error = %v, want object store backend failure", err)
}
}
func archiveFixture(t *testing.T) (*Env, *manifest.Manifest, string) {
func publishFixture(t *testing.T) (*Env, *manifest.Manifest, string) {
t.Helper()
root := t.TempDir()
@@ -549,7 +549,7 @@ func archiveFixture(t *testing.T) (*Env, *manifest.Manifest, string) {
m.S3Bucket = "my-dnd-archive"
m.S3SessionPrefix = artifacts.S3SessionPrefix("dnd", campaign, sessionID)
m.S3RunPrefix = artifacts.S3RunPrefix(m.S3SessionPrefix, runID)
for _, name := range archivePrerequisiteStages {
for _, name := range publishPrerequisiteStages {
m.MarkStageSucceeded(name, time.Date(2026, 5, 16, 1, 2, 3, 0, time.UTC), nil)
}
@@ -591,27 +591,27 @@ func archiveFixture(t *testing.T) (*Env, *manifest.Manifest, string) {
return env, m, runRoot
}
type promotionFailingStore struct {
type publishedOutputFailingStore struct {
delegate *storage.FakeBackend
failKey string
}
func (s *promotionFailingStore) List(ctx context.Context, prefix string) ([]storage.ObjectInfo, error) {
func (s *publishedOutputFailingStore) List(ctx context.Context, prefix string) ([]storage.ObjectInfo, error) {
return s.delegate.List(ctx, prefix)
}
func (s *promotionFailingStore) Download(ctx context.Context, key, localPath string) error {
func (s *publishedOutputFailingStore) Download(ctx context.Context, key, localPath string) error {
return s.delegate.Download(ctx, key, localPath)
}
func (s *promotionFailingStore) Upload(ctx context.Context, localPath, key string, opts storage.UploadOptions) (storage.ObjectInfo, error) {
func (s *publishedOutputFailingStore) Upload(ctx context.Context, localPath, key string, opts storage.UploadOptions) (storage.ObjectInfo, error) {
if strings.TrimSpace(key) == strings.TrimSpace(s.failKey) {
return storage.ObjectInfo{}, errors.New("forced upload failure")
}
return s.delegate.Upload(ctx, localPath, key, opts)
}
func (s *promotionFailingStore) Exists(ctx context.Context, key string) (bool, error) {
func (s *publishedOutputFailingStore) Exists(ctx context.Context, key string) (bool, error) {
return s.delegate.Exists(ctx, key)
}