# Pipeline-Scoped CLI References ## Purpose Make command-line reference binding match the pipeline-level mental model used by configuration and by subprocess callers. A caller should be able to supply each shared external reference once, while retaining explicit syntax for the less common case in which one lane or one stage binding needs a different source. ## Motivation And Current Problem Notarius already accepts repeatable `--reference selector=path` and `--without-reference selector` flags. The current unqualified form, `--reference slot=path`, succeeds only when exactly one selected chunk, extract, merge, or normalize target declares that slot. If several targets consume a shared reference such as `party`, `glossary`, or `spell_catalog`, the command is rejected as ambiguous and the caller must repeat stage-qualified bindings. That behavior is internally precise but does not match the public configuration model. A reference declared at pipeline scope is shared with every compatible target, while target-local configuration provides the exceptional override. It is therefore surprising for the least-qualified CLI syntax to mean "find one unique target" rather than "supply this reference to the pipeline." The mismatch is especially costly for subprocess use. The primary complete D&D workflow expects an orchestrator to provide one transcript, one output root, and a small collection of campaign reference files. Requiring the orchestrator to know and enumerate every internal consumer of those shared files couples it to lane composition, makes commands needlessly long, and creates maintenance work whenever another compatible D&D module is added. ## Target CLI Contract `--reference` remains repeatable, but selector qualification expresses scope: | Form | Target scope | | --- | --- | | `slot=path` | Every selected pipeline target that declares `slot`. | | `chunk.slot=path` | The selected chunk binding. | | `lane.slot=path` | Every extract, merge, or normalize binding in `lane` that declares `slot`. | | `lane.extract.slot=path` | The extract binding in `lane`. | | `lane.merge.slot=path` | The merge binding in `lane`. | | `lane.normalize.slot=path` | The normalize binding in `lane`. | The current stage-wide `merge.slot=path` shorthand is removed. A merge-specific override must name its lane as `lane.merge.slot=path`; this keeps the grammar hierarchical and avoids another uniqueness-dependent selector. `--without-reference` uses the same selector forms without `=path`. It removes matching external bindings only. It never removes a generated artifact handoff, and resolution continues to reject a missing required reference. Broad selectors are expected to match multiple compatible targets. They fail when they match no selected target, when the named lane is not selected, or when a stage-qualified target does not declare the slot. Errors should identify the selector and relevant scope without requiring callers to understand private resolver structures. ## Precedence And Conflict Policy Command-line references are operational overrides and take precedence over external file references supplied at pipeline, step, lane, or binding scope in configuration. Among CLI selectors that affect the same concrete target: 1. an exact `lane.stage.slot` or `chunk.slot` selector wins over a lane-scoped selector; 2. a lane-scoped selector wins over a pipeline-scoped selector; and 3. the last occurrence wins among selectors with equal scope and action. This ordering lets an orchestrator provide shared defaults once and express only genuine exceptions: ```sh --reference party=/refs/party.txt \ --reference npc-registry.party=/refs/npc-party-context.txt ``` Binding and unbinding the same concrete target at the same specificity is a configuration error rather than an argument-order-dependent result. A more-specific unbind may carve an exception out of a broader binding, and a more-specific binding may restore an exception to a broader unbind. Generated artifact references remain a distinct source form. A CLI file reference must not silently replace, remove, or coexist with a generated handoff for the same concrete target and slot. Resolution fails with a target-specific conflict and directs the caller to narrow or remove the CLI selector. Same-run generated registry and eligibility handoffs in the complete D&D pipeline therefore remain controlled by pipeline composition. CLI reference paths continue to resolve relative to the process working directory and retain CLI provenance. Subprocess guidance must recommend absolute paths. The reference source is auxiliary context and does not affect the generated prompt session identifier. ## Subprocess Target State Passing shared references is a first-class part of the documented subprocess workflow. The complete D&D invocation should have this shape: ```sh notarius run dnd-session \ --config /absolute/path/to/notarius.yml \ --input /absolute/path/to/transcripts/final.trimmed.json \ --output-dir /absolute/path/to/notarius-output \ --reference party=/absolute/path/to/references/party.txt \ --reference players=/absolute/path/to/references/players.txt \ --reference glossary=/absolute/path/to/references/glossary.txt \ --reference spell_catalog=/absolute/path/to/references/spells.json \ --json ``` Only references actually available to and desired by the deployment need to be passed. The selected modules continue to determine accepted slot names, media types, requiredness, and size limits. The orchestrator passes references as separate argument-vector elements rather than building a shell command, captures standard output and standard error separately, checks exit status, and uses the JSON receipt's `output_directory` to locate the generated run bundle. The documentation must clearly distinguish: - the input file supplied by `--input`; - the output *root* supplied by `--output-dir` and the run-specific output path returned in the receipt; - external file references supplied by `--reference`; and - generated references produced and consumed within the configured ordered pipeline. ## Required Implementation Changes ### CLI selector model and parsing - Replace the unique-target interpretation of unqualified and lane-qualified selectors with explicit pipeline, lane, chunk, and exact-binding scopes. - Remove parsing and help text for the stage-wide `merge.slot` form. - Preserve repeatable flag handling, non-empty selector/path checks, supported stage names, and syntax-error exit classification. - Represent selector scope explicitly enough that precedence and diagnostics do not depend on inferring intent from empty fields. ### Target expansion and precedence - Refactor CLI reference resolution to return every compatible selected target for a broad selector rather than requiring uniqueness. - Resolve overlapping bind and unbind requests into one deterministic action per concrete target and slot using the precedence policy above. - Produce exact `pipeline.ReferenceBinding` and `pipeline.ReferenceUnbind` values after CLI scoping is resolved. Keep broad-selector policy at the CLI boundary rather than adding CLI grammar or D&D knowledge to the generic pipeline framework. - Preserve CLI provenance and the existing rule that command-line paths are materialized relative to the working directory. - Detect generated-reference conflicts before file access or stage execution and retain required-slot validation after effective bindings are known. - Ensure lane selection is honored for compact pipelines. Explicit ordered pipelines continue to use their complete configured lane set because they do not support `--only`. ### Public and internal documentation - Rewrite the reference-selector section of `docs/cli.md`, which is the canonical owner of flag syntax, selector semantics, precedence, and command errors. - Update `docs/consumers/subprocess.md` so reference-bearing invocation is part of the primary subprocess workflow rather than an unillustrated aside. - Update `docs/consumers/dnd-pipeline.md` with the complete D&D subprocess invocation and explain which references are external versus generated. - Lightly update `docs/config.md` to link the configuration hierarchy to the CLI override contract without duplicating CLI syntax. - Update `docs/internal/cli.md` and, only where necessary, the reference resolution discussion in `docs/internal/pipeline.md` to describe the implemented expansion and precedence boundary. - Do not expand the README quickstart or duplicate complete configuration files in prose. The maintained example configuration remains the canonical copyable pipeline definition. No ADR is required. This is a deliberate public CLI usability correction that fits the existing architectural decisions: configuration remains centralized, operational overrides remain explicit, generated handoffs remain part of pipeline composition, and broad CLI syntax is translated into existing exact framework bindings at the application boundary. The CLI and configuration references are the durable owners of the resulting current behavior once it is implemented. ## Testing And Validation Add lean offline behavioral coverage at the CLI contract boundary for: - one pipeline-scoped reference reaching multiple compatible chunk and lane targets; - one lane-scoped reference reaching both extract and normalize bindings; - exact-binding and lane-scoped exceptions overriding broader values; - final-occurrence behavior at equal specificity; - broad selectors with no matches and selectors naming unselected lanes or undeclared slots; - bind/unbind specificity and same-specificity conflicts; - rejection of external/generated conflicts and protection of required generated handoffs; - working-directory resolution and CLI provenance for expanded bindings; and - removal of the `merge.slot` shorthand. Prefer package-level resolution and representative `RunWithOptions` tests over duplicating the full selector matrix through end-to-end fixtures. Existing tests that encode unique-target ambiguity should be rewritten or removed rather than retained as historical change detectors. Do not assert complete error strings when stable classification and a concise semantic fragment provide sufficient confidence. Validation for the completed change should include: ```sh go test ./internal/cli ./internal/core/config ./internal/framework/pipeline go test ./... go vet ./... go build ./cmd/notarius ``` Also validate both maintained example configurations against their selected pipelines, verify every changed documentation link, and manually review the D&D subprocess command against the actual flag parser and complete example configuration. ## Non-Goals - Passing reference contents directly in command-line arguments or environment variables. - Adding a reference-manifest file, glob syntax, wildcard selectors, or a second pipeline-reference flag. - Allowing the CLI to define undeclared reference slots or bypass module media type, size, UTF-8, requiredness, or materialization validation. - Replacing generated same-run artifacts with external files or changing ordered pipeline dependencies. - Changing configuration-file reference path resolution, prompt-session derivation, output directory allocation, or the run-result receipt. - Adding D&D-specific selector behavior to generic CLI or pipeline packages. - Preserving uniqueness-dependent `slot` or `lane.slot` behavior as an alias. ## Acceptance Criteria - A subprocess caller can supply `party`, `players`, `glossary`, and `spell_catalog` once each and have every compatible selected D&D target receive the corresponding external file. - `slot=path` has pipeline scope and never fails merely because several selected targets declare the slot. - Lane and exact-binding selectors provide deterministic, documented exceptions with specificity-based precedence. - A broad selector that matches nothing is rejected before source parsing or LLM work. - CLI external references override configured external paths but never silently replace or remove generated artifact references. - All effective CLI bindings retain correct provenance and absolute-path materialization behavior. - `docs/cli.md` owns the exact public syntax, and both subprocess guides present shared CLI references as the primary orchestration workflow. - Current behavior outside `docs/roadmap/` is not documented until the code lands, and no complete example is duplicated outside `examples/`. - Focused tests, repository-wide tests, vetting, build, maintained configuration validation, and documentation-link review pass.