eric/narratio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 18 Wiki Activity

Add configuration source reporting

Browse Source
This commit is contained in:
Eric Rakestraw
2026-08-30 14:56:22 +00:00
parent a102db36af
commit dde7f76ecb
11 changed files with 654 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/cli.md
View File

@@ -78,11 +78,12 @@ Commands with additional positionals keep their command-specific order:
## Command Reference
### `config validate` and `config show`
### `config validate`, `config show`, and `config sources`
```bash
narratio config validate [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--profile <name>]
narratio config show [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--profile <name>]
narratio config sources [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--profile <name>]
```
These commands resolve the selected profile, defaults, ordinary paths, and—if
@@ -102,6 +103,15 @@ writes one deterministic, secret-free YAML document containing defaulted and
expanded concrete configuration. It omits composition declarations, artifact
family declarations, and runtime provenance.
`config sources` reports the same fully validated resolution without printing
effective values. Its header identifies the root, ordered imports, selected
profile and overlay, selected campaign, party mode/source, and digest. The
remaining tab-separated records are sorted as `path`, `role`, and `source`.
Roles distinguish root, import, profile, centralized default, campaign, party,
legacy-player, and generated family ownership. A generated party member has
one family record and one party record at the same logical path. The output
never reads or prints secret values.
### `version`
```bash

14
docs/config.md
View File

@@ -44,10 +44,10 @@ the durable copied session input.
### Read-only effective pipeline inspection
`narratio config validate` and `narratio config show` use the same `--config`,
`--campaign`, `--campaign-file`, and `--profile` selection rules as pipeline
commands, but do not select, discover, or load a session. They do not read
credential values or create runtime state.
`narratio config validate`, `narratio config show`, and `narratio config
sources` use the same `--config`, `--campaign`, `--campaign-file`, and
`--profile` selection rules as pipeline commands, but do not select, discover,
or load a session. They do not read credential values or create runtime state.
Campaign selection is optional only when the resolved pipeline has no
`scriptorium.artifact_families`. When families are declared, Narratio selects a
@@ -56,7 +56,11 @@ then parses the campaign-owned party and expands concrete artifacts and any
family publish rules before validation. `config validate` prints the resulting
root, profile, and effective digest. `config show` emits the normalized
effective pipeline YAML, with defaults and concrete expansion included but
composition and family declarations omitted. The [CLI reference](cli.md#config-validate-and-config-show)
composition and family declarations omitted. `config sources` prints a stable
source projection instead of effective values: root/import/profile/default
ownership plus campaign/party and generated-family records. Canonical derived
players trace to the party; a legacy configured players file is explicitly
marked as a legacy player source. The [CLI reference](cli.md#config-validate-config-show-and-config-sources)
owns command syntax and output conventions.
### Identity segments

11
docs/internal/configuration.md
View File

@@ -68,6 +68,17 @@ family declarations. The result contains no composition envelope or private
provenance fields and has one trailing newline; commands do not marshal runtime
objects directly.
`EffectivePipelineSources` and `EffectiveCampaignSources` provide the separate
safe provenance projection for `config sources`. Pipeline ownership begins with
the complete logical field paths retained during composition and classifies
each contributor as root, import, profile, or centralized default. The
projection replaces generated concrete member paths with paired family and
canonical-party records, and does the same for generated publish rules.
Campaign records identify campaign-owned fields and party inputs; canonical
derived players point to the party source, while legacy players retain a
dedicated legacy-player role. The application command only joins these sorted
records with selection metadata and never reparses configuration files.
Campaign context construction also reads and classifies the campaign-owned
party source through `ParseParty`. A canonical party retains its raw bytes and
normalized roster in runtime-only `ResolvedParty` provenance, while a legacy

2
docs/roadmap/implementation.md
View File

@@ -1002,7 +1002,7 @@ without creating a session, workspace, run, or external adapter.
## Stage 18 — Read-Only `config sources`
**Status: Pending**
**Status: Completed**
### Goal

26
docs/troubleshooting.md
View File

@@ -71,6 +71,32 @@ Safe fix:
Relevant reference: [Configuration](./config.md).
## Unexpected imported or profile value
Symptom:
- an effective configuration value differs from the root file, or a duplicate
ownership/configuration error is hard to locate.
Diagnostics:
```bash
narratio config sources --config /path/pipeline.yml --profile testing
```
Add `--campaign` or `--campaign-file` when the pipeline has party-driven
artifact families. The output identifies each effective logical field's root,
import, profile, default, campaign, party, or family source without printing
the field value or credential contents.
Safe fix:
- move a duplicated base field so it has one owner;
- correct the selected profile or its overlay; or
- correct the campaign party/family declaration that owns generated values.
Relevant reference: [Configuration inspection](./config.md#read-only-effective-pipeline-inspection).
## Audio mode conflict
Symptom:

102
internal/app/config_commands.go
View File

@@ -6,6 +6,7 @@ import (
"flag"
"fmt"
"io"
"sort"
"strings"
"gitea.maximumdirect.net/eric/narratio/internal/config"
@@ -21,12 +22,15 @@ type inspectionFlags struct {
type inspectionConfig struct {
PipelinePath string
Pipeline *config.PipelineConfig
CampaignPath string
Campaign *config.CampaignConfig
Party config.ResolvedParty
}
// Config dispatches read-only pipeline configuration commands.
func Config(ctx context.Context, args []string, out io.Writer) error {
if len(args) == 0 {
return fmt.Errorf("config: expected subcommand: validate|show")
return fmt.Errorf("config: expected subcommand: validate|show|sources")
}
if args[0] == "--help" || args[0] == "-h" {
writeConfigCommandUsage(out)
@@ -37,6 +41,8 @@ func Config(ctx context.Context, args []string, out io.Writer) error {
return ConfigValidate(ctx, args[1:], out)
case "show":
return ConfigShow(ctx, args[1:], out)
case "sources":
return ConfigSources(ctx, args[1:], out)
default:
return fmt.Errorf("config: unknown subcommand %q", args[0])
}
@@ -83,6 +89,37 @@ func ConfigShow(_ context.Context, args []string, out io.Writer) error {
return err
}
// ConfigSources validates and reports deterministic effective configuration
// ownership without loading session state or runtime collaborators.
func ConfigSources(_ context.Context, args []string, out io.Writer) error {
flags, err := parseInspectionFlags("config sources", args, out)
if err != nil {
if errors.Is(err, flag.ErrHelp) {
return nil
}
return err
}
resolved, err := resolveInspectionConfig(flags)
if err != nil {
return fmt.Errorf("config sources: %w", err)
}
records, err := config.EffectivePipelineSources(resolved.Pipeline, resolved.Party)
if err != nil {
return fmt.Errorf("config sources: %w", err)
}
records = append(records, config.EffectiveCampaignSources(resolved.CampaignPath, resolved.Campaign, resolved.Party)...)
sortInspectionSourceRecords(records)
if err := writeInspectionSourceHeader(out, resolved); err != nil {
return err
}
for _, record := range records {
if _, err := fmt.Fprintf(out, "%s\t%s\t%s\n", record.Path, record.Role, record.Source); err != nil {
return err
}
}
return nil
}
func parseInspectionFlags(command string, args []string, out io.Writer) (inspectionFlags, error) {
fs := flag.NewFlagSet(command, flag.ContinueOnError)
fs.SetOutput(out)
@@ -155,6 +192,9 @@ func resolveInspectionConfig(flags inspectionFlags) (*inspectionConfig, error) {
return &inspectionConfig{
PipelinePath: loaded.PipelinePath,
Pipeline: loaded.Pipeline,
CampaignPath: loaded.CampaignPath,
Campaign: loaded.Campaign,
Party: loaded.Party,
}, nil
}
@@ -194,7 +234,63 @@ func inspectionProfile(pipeline *config.PipelineConfig) string {
}
func writeConfigCommandUsage(out io.Writer) {
fmt.Fprintln(out, "Usage: narratio config <validate|show>")
fmt.Fprintln(out, "Usage: narratio config <validate|show|sources>")
fmt.Fprintln(out)
fmt.Fprintln(out, "Use config validate to check an effective pipeline or config show to print normalized effective YAML.")
fmt.Fprintln(out, "Use config validate to check an effective pipeline, config show to print normalized YAML, or config sources to report ownership.")
}
func writeInspectionSourceHeader(out io.Writer, resolved *inspectionConfig) error {
if _, err := fmt.Fprintf(out, "root: %s\n", inspectionRootPath(resolved)); err != nil {
return err
}
imports := config.EffectivePipelineImports(resolved.Pipeline)
if len(imports) == 0 {
if _, err := fmt.Fprintln(out, "imports: none"); err != nil {
return err
}
}
for _, imported := range imports {
if _, err := fmt.Fprintf(out, "import: %s\n", config.NormalizedConfigurationPath(imported)); err != nil {
return err
}
}
if profile, ok := config.SelectedPipelineProfile(resolved.Pipeline); ok {
if _, err := fmt.Fprintf(out, "profile: name=%s selection=%s overlay=%s\n", profile.Name, profile.Source, config.NormalizedConfigurationPath(profile.OverlayPath)); err != nil {
return err
}
} else if _, err := fmt.Fprintln(out, "profile: none"); err != nil {
return err
}
if resolved.Campaign == nil {
if _, err := fmt.Fprintln(out, "campaign: none"); err != nil {
return err
}
if _, err := fmt.Fprintln(out, "party: none"); err != nil {
return err
}
} else {
if _, err := fmt.Fprintf(out, "campaign: id=%s source=%s\n", config.CampaignID(resolved.Campaign), config.NormalizedConfigurationPath(resolved.CampaignPath)); err != nil {
return err
}
if _, err := fmt.Fprintf(out, "party: mode=%s source=%s\n", resolved.Party.Mode, config.NormalizedConfigurationPath(resolved.Party.Source.Path)); err != nil {
return err
}
}
if _, err := fmt.Fprintf(out, "digest: %s\n", config.EffectivePipelineDigest(resolved.Pipeline)); err != nil {
return err
}
_, err := fmt.Fprintln(out, "path\trole\tsource")
return err
}
func sortInspectionSourceRecords(records []config.EffectivePipelineSourceRecord) {
sort.Slice(records, func(left, right int) bool {
if records[left].Path != records[right].Path {
return records[left].Path < records[right].Path
}
if records[left].Role != records[right].Role {
return records[left].Role < records[right].Role
}
return records[left].Source < records[right].Source
})
}

156
internal/app/config_commands_test.go
View File

@@ -21,11 +21,12 @@ func TestConfigCommandHelpAndDispatch(t *testing.T) {
want string
code int
}{
{name: "top level help", args: []string{"config", "--help"}, want: "Usage: narratio config <validate|show>"},
{name: "top level help", args: []string{"config", "--help"}, want: "Usage: narratio config <validate|show|sources>"},
{name: "validate help", args: []string{"config", "validate", "--help"}, want: "Usage: narratio config validate"},
{name: "show help", args: []string{"config", "show", "--help"}, want: "Usage: narratio config show"},
{name: "unknown", args: []string{"config", "sources"}, want: `config: unknown subcommand "sources"`, code: 1},
{name: "missing", args: []string{"config"}, want: "config: expected subcommand: validate|show", code: 1},
{name: "sources help", args: []string{"config", "sources", "--help"}, want: "Usage: narratio config sources"},
{name: "unknown", args: []string{"config", "diff"}, want: `config: unknown subcommand "diff"`, code: 1},
{name: "missing", args: []string{"config"}, want: "config: expected subcommand: validate|show|sources", code: 1},
{name: "session flag", args: []string{"config", "validate", "--session", "session.yml"}, want: "flag provided but not defined", code: 1},
{name: "stage flag", args: []string{"config", "show", "--from", "prepare"}, want: "flag provided but not defined", code: 1},
{name: "force flag", args: []string{"config", "show", "--force"}, want: "flag provided but not defined", code: 1},
@@ -86,6 +87,13 @@ whisperx:
if strings.Contains(show, secret) {
t.Fatalf("show output leaked a raw secret: %q", show)
}
var sourcesOut bytes.Buffer
if err := ConfigSources(context.Background(), []string{"--config", pipelinePath}, &sourcesOut); err != nil {
t.Fatalf("ConfigSources() error = %v", err)
}
if strings.Contains(sourcesOut.String(), secret) {
t.Fatalf("sources output leaked a raw secret: %q", sourcesOut.String())
}
if calls != 0 {
t.Fatalf("inspection invoked configured external endpoint %d time(s)", calls)
}
@@ -189,6 +197,133 @@ whisperx:
}
}
func TestConfigSourcesReportsStableOwnership(t *testing.T) {
workspaceRoot := t.TempDir()
pipelinePath, campaignPath, _ := writeValidConfigFiles(t, workspaceRoot)
enableCommandTestProfiles(t, pipelinePath)
args := []string{"--config", pipelinePath, "--campaign-file", campaignPath, "--profile", "testing"}
var first, second bytes.Buffer
if err := ConfigSources(context.Background(), args, &first); err != nil {
t.Fatalf("first ConfigSources() error = %v", err)
}
if err := ConfigSources(context.Background(), args, &second); err != nil {
t.Fatalf("second ConfigSources() error = %v", err)
}
if first.String() != second.String() {
t.Fatalf("sources output is not deterministic:\nfirst=%q\nsecond=%q", first.String(), second.String())
}
output := first.String()
for _, want := range []string{
"root: ",
"imports: none",
"profile: name=testing selection=cli overlay=",
"campaign: id=sample-campaign source=",
"party: mode=legacy source=",
"digest: ",
"path\trole\tsource\n",
"workspace.root\troot\t",
"whisperx.language\tprofile\t",
"campaign.inputs.players_file\tlegacy_player\t",
"trim.enabled\tdefault\tdefault",
} {
if !strings.Contains(output, want) {
t.Fatalf("sources output missing %q:\n%s", want, output)
}
}
}
func TestConfigSourcesReportsCanonicalFamilyAndPublishOrigins(t *testing.T) {
pipelinePath, campaignPath := writeInspectionFamilyConfig(t)
var out bytes.Buffer
if err := ConfigSources(context.Background(), []string{"--config", pipelinePath, "--campaign-file", campaignPath}, &out); err != nil {
t.Fatalf("ConfigSources() error = %v", err)
}
output := out.String()
partyPath := filepath.Join(filepath.Dir(campaignPath), "party.yml")
for _, want := range []string{
"party: mode=canonical source=" + partyPath,
"derived.players\tparty\t" + partyPath,
"scriptorium.artifacts.character_note_arannis.enabled\tfamily\t" + pipelinePath,
"scriptorium.artifacts.character_note_arannis.enabled\tparty\t" + partyPath,
"scriptorium.artifacts.character_note_arannis.depends_on\tfamily\t" + pipelinePath,
"scriptorium.artifacts.character_note_arannis.inputs.prior.source\tparty\t" + partyPath,
"publish.outputs[",
"\tfamily\t" + pipelinePath,
"\tparty\t" + partyPath,
} {
if !strings.Contains(output, want) {
t.Fatalf("sources output missing %q:\n%s", want, output)
}
}
}
func TestConfigSourcesPreservesProfileOwnershipThroughFamilyExpansion(t *testing.T) {
pipelinePath, campaignPath := writeInspectionFamilyConfig(t)
data, err := os.ReadFile(pipelinePath)
if err != nil {
t.Fatal(err)
}
profilePath := filepath.Join(filepath.Dir(pipelinePath), "testing.yml")
composition := "composition:\n default_profile: testing\n profiles:\n testing:\n overlay: testing.yml\n"
if err := os.WriteFile(pipelinePath, append([]byte(composition), data...), 0o644); err != nil {
t.Fatal(err)
}
mustWriteTestFile(t, profilePath, `scriptorium:
artifact_families:
character_note:
enabled: true
prompt_id: dnd.character_note
`)
var out bytes.Buffer
if err := ConfigSources(context.Background(), []string{"--config", pipelinePath, "--campaign-file", campaignPath}, &out); err != nil {
t.Fatalf("ConfigSources() error = %v", err)
}
want := "scriptorium.artifacts.character_note_arannis.enabled\tfamily\t" + profilePath
if !strings.Contains(out.String(), want) {
t.Fatalf("sources output missing profile-generated ownership %q:\n%s", want, out.String())
}
}
func TestConfigSourcesReportsRootImportProfileAndDefaultOwnership(t *testing.T) {
dir := t.TempDir()
rootPath := filepath.Join(dir, "pipeline.yml")
importPath := filepath.Join(dir, "base.yml")
profilePath := filepath.Join(dir, "testing.yml")
mustWriteTestFile(t, rootPath, `composition:
imports: [base.yml]
default_profile: testing
profiles:
testing:
overlay: testing.yml
workspace:
root: /srv/narratio
whisperx:
transcribe_url: https://transcription.example.com/transcribe
`)
mustWriteTestFile(t, importPath, "storage:\n backend: local\n")
mustWriteTestFile(t, profilePath, "whisperx:\n language: fr\n")
var out bytes.Buffer
if err := ConfigSources(context.Background(), []string{"--config", rootPath}, &out); err != nil {
t.Fatalf("ConfigSources() error = %v", err)
}
output := out.String()
for _, want := range []string{
"import: " + importPath,
"profile: name=testing selection=default overlay=" + profilePath,
"workspace.root\troot\t" + rootPath,
"storage.backend\timport\t" + importPath,
"whisperx.language\tprofile\t" + profilePath,
"trim.enabled\tdefault\tdefault",
} {
if !strings.Contains(output, want) {
t.Fatalf("sources output missing %q:\n%s", want, output)
}
}
}
func writeInspectionFamilyConfig(t *testing.T) (string, string) {
t.Helper()
dir := t.TempDir()
@@ -204,10 +339,25 @@ whisperx:
transcribe_url: https://transcription.example.com/transcribe
scriptorium:
artifact_families:
character_meta:
enabled: false
for_each: party.characters
output_path_pattern: artifacts/characters/{character_id}/meta.md
character_note:
enabled: false
for_each: party.characters
output_path_pattern: artifacts/characters/{character_id}/note.md
member_dependencies: [character_meta]
inputs:
prior: {source: narratio.member_artifact.character_meta, required: true}
member_vars:
character_name: character.name
publish:
enabled: true
required: true
publish:
enabled: true
upload_run: false
`)
mustWriteTestFile(t, campaignPath, `campaign_id: sample-campaign
inputs:

16
internal/config/artifact_families.go
View File

@@ -365,10 +365,22 @@ func artifactFamilySource(cfg *PipelineConfig, family string) string {
return ""
}
prefix := "scriptorium.artifact_families." + family + "."
var fallback string
for _, ownership := range cfg.resolution.ownership {
if strings.HasPrefix(ownership.path, prefix) && len(ownership.sources) > 0 {
return ownership.sources[0]
if !strings.HasPrefix(ownership.path, prefix) || len(ownership.sources) == 0 {
continue
}
for _, source := range ownership.sources {
if source != pipelineDefaultOwnershipSource {
return source
}
if fallback == "" {
fallback = source
}
}
}
if fallback != "" {
return fallback
}
return pipelineDefaultOwnershipSource
}

3
internal/config/artifact_families_test.go
View File

@@ -81,6 +81,9 @@ characters:
t.Fatalf("zeta class summary = %#v", got)
}
catalog := ArtifactFamilies(cfg.Pipeline)
if got, want := catalog.Families["character_meta"].Source, absolutePath(t, pipelinePath); got != want {
t.Fatalf("family source = %q, want %q", got, want)
}
if got := catalog.Families["character_meta"].Members; !reflect.DeepEqual(got, []string{"character_meta_alpha", "character_meta_zeta"}) {
t.Fatalf("meta members = %#v", got)
}

7
internal/config/effective_digest.go
View File

@@ -13,8 +13,9 @@ const pipelineDefaultOwnershipSource = "default"
// PipelineProfileProvenance identifies the profile selected while resolving a
// pipeline. It contains only non-secret selection metadata.
type PipelineProfileProvenance struct {
Name string
Source string
Name string
Source string
OverlayPath string
}
// SelectedPipelineProfile reports the profile used to resolve cfg, if any.
@@ -23,7 +24,7 @@ func SelectedPipelineProfile(cfg *PipelineConfig) (*PipelineProfileProvenance, b
return nil, false
}
selection := cfg.resolution.selectedProfile
return &PipelineProfileProvenance{Name: selection.name, Source: selection.source}, true
return &PipelineProfileProvenance{Name: selection.name, Source: selection.source, OverlayPath: selection.overlayPath}, true
}
// EffectivePipelineDigest reports the deterministic secret-free digest for a

323
internal/config/effective_pipeline.go
View File

@@ -2,6 +2,9 @@ package config
import (
"fmt"
"path/filepath"
"sort"
"strings"
"gopkg.in/yaml.v3"
)
@@ -16,6 +19,122 @@ func EffectivePipelineRootPath(cfg *PipelineConfig) string {
return cfg.resolution.rootPath
}
// EffectivePipelineImports reports the ordered, absolute import paths that
// contributed to cfg.
func EffectivePipelineImports(cfg *PipelineConfig) []string {
if cfg == nil || cfg.resolution == nil {
return nil
}
return append([]string(nil), cfg.resolution.imports...)
}
// NormalizedConfigurationPath returns a clean absolute path when possible for
// display-only configuration provenance.
func NormalizedConfigurationPath(path string) string {
return normalizedProvenancePath(path)
}
// EffectivePipelineSourceRecord identifies one effective logical field and a
// safe source that contributed to it. Generated values intentionally have two
// records: their family declaration and the canonical party that supplied the
// member-specific value.
type EffectivePipelineSourceRecord struct {
Path string
Role string
Source string
}
// EffectivePipelineSources projects pipeline ownership after defaults and
// optional family expansion. It never includes effective values or raw secret
// material, only logical paths and source identifiers.
func EffectivePipelineSources(cfg *PipelineConfig, party ResolvedParty) ([]EffectivePipelineSourceRecord, error) {
if cfg == nil {
return nil, fmt.Errorf("pipeline config is required")
}
data, err := MarshalEffectivePipeline(cfg)
if err != nil {
return nil, err
}
document, err := parseCompositionBytes("effective pipeline", data)
if err != nil {
return nil, err
}
paths := effectivePipelineFieldPaths(document.root)
owners := effectivePipelineOwners(cfg)
families := ArtifactFamilies(cfg)
publishCount := 0
if cfg.Publish != nil {
publishCount = len(cfg.Publish.Outputs)
}
records := make([]EffectivePipelineSourceRecord, 0, len(paths)+publishCount)
for _, path := range paths {
if path == "publish.outputs" {
continue
}
if familyKey, ok := generatedArtifactFamilyForPath(path, families); ok {
records = appendGeneratedArtifactSources(records, path, familyKey, families, owners, party)
continue
}
records = appendPipelineOwners(records, path, owners[path], cfg)
}
records = appendPublishOutputSources(records, cfg, families, party, owners)
sort.Slice(records, func(left, right int) bool {
if records[left].Path != records[right].Path {
return records[left].Path < records[right].Path
}
if records[left].Role != records[right].Role {
return records[left].Role < records[right].Role
}
return records[left].Source < records[right].Source
})
return records, nil
}
// EffectiveCampaignSources projects selected campaign and party ownership
// without retaining or rendering campaign values. Canonical players are a
// derived party value, while a legacy players file remains explicitly marked
// as legacy input provenance.
func EffectiveCampaignSources(campaignPath string, campaign *CampaignConfig, party ResolvedParty) []EffectivePipelineSourceRecord {
if campaign == nil {
return nil
}
campaignSource := normalizedProvenancePath(campaignPath)
records := []EffectivePipelineSourceRecord{
{Path: "campaign.campaign_id", Role: "campaign", Source: campaignSource},
{Path: "campaign.inputs.autocorrect_file", Role: "campaign", Source: campaignSource},
{Path: "campaign.inputs.glossary_file", Role: "campaign", Source: campaignSource},
{Path: "campaign.inputs.speakers_file", Role: "campaign", Source: campaignSource},
}
if strings.TrimSpace(campaign.Inputs.SpellCatalogFile) != "" {
records = append(records, EffectivePipelineSourceRecord{Path: "campaign.inputs.spell_catalog_file", Role: "campaign", Source: campaignSource})
}
if party.Mode == PartyModeCanonical {
partySource := normalizedProvenancePath(party.Source.Path)
records = append(records,
EffectivePipelineSourceRecord{Path: "campaign.inputs.party_file", Role: "party", Source: partySource},
EffectivePipelineSourceRecord{Path: "derived.players", Role: "party", Source: partySource},
)
} else {
partySource := normalizedProvenancePath(party.Source.Path)
records = append(records, EffectivePipelineSourceRecord{Path: "campaign.inputs.party_file", Role: "party", Source: partySource})
if strings.TrimSpace(campaign.Inputs.PlayersFile) != "" {
records = append(records, EffectivePipelineSourceRecord{
Path: "campaign.inputs.players_file", Role: "legacy_player", Source: campaignInputSourcePath(campaignPath, campaign.Inputs.PlayersFile),
})
}
}
sort.Slice(records, func(left, right int) bool {
if records[left].Path != records[right].Path {
return records[left].Path < records[right].Path
}
if records[left].Role != records[right].Role {
return records[left].Role < records[right].Role
}
return records[left].Source < records[right].Source
})
return records
}
// MarshalEffectivePipeline renders the validated, normalized pipeline as one
// deterministic YAML document. Composition declarations and runtime-only
// resolution data are excluded. Artifact family declarations are also omitted
@@ -58,3 +177,207 @@ func removeResolutionOnlyPipelineFields(document *compositionDocument) {
scriptorium.fields[index].order = index
}
}
func effectivePipelineFieldPaths(node *compositionNode) []string {
var paths []string
var visit func(*compositionNode)
visit = func(current *compositionNode) {
if current == nil {
return
}
if current.kind == yaml.MappingNode && len(current.fields) > 0 {
for _, field := range current.fields {
visit(field.value)
}
return
}
paths = append(paths, current.path)
}
visit(node)
return paths
}
func effectivePipelineOwners(cfg *PipelineConfig) map[string][]string {
owners := make(map[string][]string)
if cfg == nil || cfg.resolution == nil {
return owners
}
for _, ownership := range cfg.resolution.ownership {
owners[ownership.path] = append([]string(nil), ownership.sources...)
}
return owners
}
func generatedArtifactFamilyForPath(path string, families ArtifactFamilyCatalog) (string, bool) {
const prefix = "scriptorium.artifacts."
if !strings.HasPrefix(path, prefix) {
return "", false
}
memberPath := strings.TrimPrefix(path, prefix)
memberKey, _, _ := strings.Cut(memberPath, ".")
member, exists := families.Members[memberKey]
if !exists {
return "", false
}
_, exists = families.Families[member.Family]
return member.Family, exists
}
func appendGeneratedArtifactSources(records []EffectivePipelineSourceRecord, path, familyKey string, families ArtifactFamilyCatalog, owners map[string][]string, party ResolvedParty) []EffectivePipelineSourceRecord {
for _, source := range generatedArtifactSources(path, familyKey, families, owners) {
role := "family"
if source == pipelineDefaultOwnershipSource {
role = "default"
}
records = append(records, EffectivePipelineSourceRecord{Path: path, Role: role, Source: normalizedProvenanceSource(source)})
}
if source := normalizedProvenancePath(party.Source.Path); source != "" {
records = append(records, EffectivePipelineSourceRecord{Path: path, Role: "party", Source: source})
}
return records
}
func generatedArtifactSources(path, familyKey string, families ArtifactFamilyCatalog, owners map[string][]string) []string {
const artifactPrefix = "scriptorium.artifacts."
memberPath := strings.TrimPrefix(path, artifactPrefix)
_, suffix, _ := strings.Cut(memberPath, ".")
familyPrefix := "scriptorium.artifact_families." + familyKey + "."
candidates := []string{familyPrefix + suffix}
switch {
case suffix == "output_path":
candidates = []string{familyPrefix + "output_path_pattern"}
case suffix == "depends_on":
candidates = []string{familyPrefix + "depends_on", familyPrefix + "member_dependencies"}
case strings.HasPrefix(suffix, "vars."):
variable := strings.TrimPrefix(suffix, "vars.")
candidates = []string{familyPrefix + "vars." + variable, familyPrefix + "member_vars." + variable}
}
sources := sourcesForPipelinePaths(owners, candidates)
if len(sources) != 0 {
return sources
}
if family, ok := families.Families[familyKey]; ok && family.Source != "" {
return []string{family.Source}
}
return []string{pipelineDefaultOwnershipSource}
}
func sourcesForPipelinePaths(owners map[string][]string, paths []string) []string {
seen := make(map[string]struct{})
var sources []string
for _, path := range paths {
for _, source := range owners[path] {
if _, exists := seen[source]; exists {
continue
}
seen[source] = struct{}{}
sources = append(sources, source)
}
}
return sources
}
func appendPipelineOwners(records []EffectivePipelineSourceRecord, path string, sources []string, cfg *PipelineConfig) []EffectivePipelineSourceRecord {
if len(sources) == 0 {
sources = []string{pipelineDefaultOwnershipSource}
}
for _, source := range sources {
records = append(records, EffectivePipelineSourceRecord{
Path: path, Role: pipelineSourceRole(cfg, source), Source: normalizedProvenanceSource(source),
})
}
return records
}
func appendPublishOutputSources(records []EffectivePipelineSourceRecord, cfg *PipelineConfig, families ArtifactFamilyCatalog, party ResolvedParty, owners map[string][]string) []EffectivePipelineSourceRecord {
if cfg == nil || cfg.Publish == nil {
return records
}
generated := make(map[string]string)
for familyKey, family := range families.Families {
if family.Publish == nil || !family.Publish.Enabled {
continue
}
for _, key := range family.Members {
generated["narratio.artifact."+key] = familyKey
}
}
for index, output := range cfg.Publish.Outputs {
path := fmt.Sprintf("publish.outputs[%d]", index)
if familyKey, ok := generated[strings.TrimSpace(output.Source)]; ok {
records = appendGeneratedPublishSources(records, path, familyKey, families, owners, party)
continue
}
records = appendPipelineOwners(records, path, owners["publish.outputs"], cfg)
}
return records
}
func appendGeneratedPublishSources(records []EffectivePipelineSourceRecord, path, familyKey string, families ArtifactFamilyCatalog, owners map[string][]string, party ResolvedParty) []EffectivePipelineSourceRecord {
familyPrefix := "scriptorium.artifact_families." + familyKey + ".publish."
sources := sourcesForPipelinePaths(owners, []string{familyPrefix + "enabled", familyPrefix + "required", familyPrefix + "dest_pattern"})
if len(sources) == 0 {
if family, ok := families.Families[familyKey]; ok && family.Source != "" {
sources = []string{family.Source}
} else {
sources = []string{pipelineDefaultOwnershipSource}
}
}
for _, source := range sources {
role := "family"
if source == pipelineDefaultOwnershipSource {
role = "default"
}
records = append(records, EffectivePipelineSourceRecord{Path: path, Role: role, Source: normalizedProvenanceSource(source)})
}
if source := normalizedProvenancePath(party.Source.Path); source != "" {
records = append(records, EffectivePipelineSourceRecord{Path: path, Role: "party", Source: source})
}
return records
}
func pipelineSourceRole(cfg *PipelineConfig, source string) string {
if source == pipelineDefaultOwnershipSource {
return "default"
}
if cfg == nil || cfg.resolution == nil {
return "pipeline"
}
if source == cfg.resolution.rootPath {
return "root"
}
if cfg.resolution.selectedProfile != nil && source == cfg.resolution.selectedProfile.overlayPath {
return "profile"
}
for _, imported := range cfg.resolution.imports {
if source == imported {
return "import"
}
}
return "pipeline"
}
func normalizedProvenanceSource(source string) string {
if source == pipelineDefaultOwnershipSource {
return source
}
return normalizedProvenancePath(source)
}
func normalizedProvenancePath(path string) string {
if strings.TrimSpace(path) == "" {
return ""
}
abs, err := filepath.Abs(path)
if err != nil {
return filepath.Clean(path)
}
return filepath.Clean(abs)
}
func campaignInputSourcePath(campaignPath, configuredPath string) string {
if filepath.IsAbs(configuredPath) {
return normalizedProvenancePath(configuredPath)
}
return normalizedProvenancePath(filepath.Join(filepath.Dir(campaignPath), configuredPath))
}