diff --git a/docs/roadmap/feature-request-notarius-config-profile.yml b/docs/roadmap/feature-request-notarius-config-profile.yml new file mode 100644 index 0000000..b8f4f17 --- /dev/null +++ b/docs/roadmap/feature-request-notarius-config-profile.yml @@ -0,0 +1,102 @@ +--- +title: Select a Notarius configuration profile through Narratio +status: proposed +consumer: rakestraw.dnd Ansible collection and its pipeline role + +use_case: >- + The Ansible pipeline role deploys Narratio and Notarius as independently + configured applications. Operators select Narratio's testing profile to + develop prompts and other pipeline assets with inexpensive models while + retaining production settings in the base configuration. That selection + currently cannot select a Notarius configuration profile, leaving extraction + on its production settings. + +desired_outcome: >- + Narratio's testing overlay can explicitly select Notarius's testing overlay. + Notarius owns the resulting model choices and any other testing-specific + settings. The operator selects testing once when invoking Narratio, without + editing production files or separately invoking Notarius. + +rationale: >- + Both applications already use ConfigKit configuration overlays. Selecting + the downstream overlay preserves each application's configuration ownership + and supports future Notarius testing adjustments without adding a Narratio + configuration field for every downstream setting. This is explicit profile + composition, not a requirement for arbitrary recursive orchestration. + +requested_interface: + field: pipeline.notarius.config_profile + value: Optional Notarius ConfigKit configuration profile name + supplied: >- + Invoke Notarius with --config-profile and the selected name when extraction + executes. The setting is available in Narratio's base configuration and + its ordinary configuration overlays. + omitted_or_null: >- + Do not supply a profile-selection flag. Preserve existing Notarius + configuration selection behavior. + downstream_semantics: >- + Retain Notarius's existing selector semantics, including the default + selector for its base configuration. Notarius resolves profiles relative + to its own configuration root, not Narratio's root. + explicit_selection: >- + Do not automatically forward Narratio's profile name. The selected + Notarius profile may have the same name or a different name. + distinction: >- + This selects a ConfigKit configuration overlay through --config-profile, + not a PromptKit LLM profile through --llm-profile. The Notarius overlay + can itself select the desired LLM profiles using native Notarius settings. + +illustrative_consumer_configuration: + note: >- + These are separate files managed by the Ansible suite, not a proposed + combined configuration document. The Notarius pipeline and LLM profile + names are illustrative; the referenced LLM profile must be available to + Notarius. + narratio_profiles_testing_yml: | + notarius: + config_profile: testing + notarius_profiles_testing_yml: | + pipelines: + dnd-session: + llm_profile: gemini-flash-latest + operator_workflow: >- + Select --profile testing on the ordinary Narratio invocation. Narratio + selects Notarius's testing configuration overlay when it runs extraction; + other Narratio testing overrides continue to apply normally. + +observable_behavior: + - >- + Notarius owns overlay discovery, composition, validation, and the effective + downstream settings. A missing or invalid selected overlay must not silently + fall back to production or be reported as successful extraction. + - >- + Changing the selected Notarius configuration profile must not silently + reuse Narratio's completed extraction from the previous selection. + Existing downstream invalidation behavior applies when extraction changes. + - >- + Editing Notarius overlay contents behind an unchanged selector retains + the existing operator responsibility to force extraction when needed. + This request does not require inspecting or tracking private Notarius files. + - >- + Existing credential forwarding remains independently configurable through + notarius.llm_api_key, including in the Narratio testing overlay when the + selected Notarius settings require a different credential reference. + +consumer_follow_up: >- + After Narratio releases this interface, the standalone Narratio Ansible role + can expose an optional variable and omit the field when unset. The suite role + can deploy Notarius's testing overlay and select it from Narratio's testing + overlay, then distribute the changes through the collection. These consumer + updates are separate from this upstream feature request. + +scope_boundaries: + - No Notarius change is currently required; v0.7.0 supports --config-profile. + - No separate Narratio-side Notarius LLM-profile selector is requested. + - No automatic profile-name propagation or downstream configuration rewriting. + - No new credential discovery, copying, or provider-name remapping. + - This request specifies consumer behavior, not internal Narratio code changes. + +references: + narratio_configuration: ../config.md + narratio_notarius_integration: ../integrations/notarius.md + notarius_cli: https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/cli.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..f93cf30 --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,248 @@ +# Implementation Plan: Notarius Configuration Profile Selection + +Status: Ready for implementation; all stages pending. + +## Scope and Working Instructions + +Implement the [feature roadmap](notarius-config-profile.md), whose authority is +the [consumer request](feature-request-notarius-config-profile.yml). Complete +the numbered stages in order. Each stage is a bounded task for a gpt-5.6-terra +coding agent. Do not treat completion of an intermediate stage as completion of +the feature. Record completed stages and actual verification results here. + +Before each stage, read `docs/development.md`, all documents under +`docs/policy/`, the feature roadmap, and that stage's preparation list. Follow +AGENTS.md discovery guidance. Use existing collaborators and fixtures; do not +build a new configuration, cache, or profile-management subsystem. Tests must +remain offline and independent of installed Notarius binaries and credentials. +Do not change dependencies, implement Ansible changes, tag, or release as part +of this plan. + +## Implementation Decisions + +These choices resolve implementation details without adding restrictions to +the feature request: + +- Represent the authored field as `ConfigProfile *string` with YAML/JSON name + `config_profile` and `omitempty`, in both typed configuration and its safe + YAML projection. Nil means omitted/null. A non-nil pointer preserves an + explicitly supplied empty string as well as `default` and named selections. +- Carry the same optional value in the Notarius adapter request. Nil emits no + flag; non-nil emits `--config-profile` followed by the exact supplied string + as a separate argument, including an empty string. Do not trim, lowercase, + substitute Narratio's profile, impose a name regex, or inspect profile files. + Notarius owns validation and selector interpretation. +- For reuse evidence only, use one small stage-local helper that maps nil, + `""`, and exact `"default"` to the empty identity; other values remain + unchanged. Both extraction fingerprints use it. This does not mutate the + authored field or change CLI forwarding. Do not attempt to recreate the + downstream parser's complete normalization rules. +- Add `ConfigProfile string` with `json:"config_profile,omitempty"` to both + existing fingerprint payloads, populated only where extraction is enabled. + Preserve existing field ordering and the current semantic fingerprint version + (1). With no non-base selector the serialized payload remains unchanged; + a named selection adds evidence that old records cannot already possess. + No dual-reader, manifest migration, new durable metadata field, or broad + fingerprint-version bump is needed. This is a minimal additive encoding, + not a permanent promise to freeze fingerprint bytes across future releases. +- Profile changes trigger the existing non-resumable/invalidation paths, not + force mode. Preserve configured resume/cache controls, force precedence, + credentials, output contracts, and stage-range behavior. Do not run extraction + outside the operator's selected range or add a separate testing workspace. +- Preserve the existing division of ownership: private downstream file changes + behind an unchanged selector require operator-directed force when needed. + No downstream configuration validation subprocess or runtime version gate + is added. + +## Stage 1: Add Configuration and Adapter Support + +Status: Pending. + +Preparation: + +- `docs/config.md` (composition, profiles, Notarius, credentials). +- `docs/integrations/notarius.md` (invocation and failure behavior). +- `internal/config/config.go`, `credential_projection.go`, + `pipeline_configkit.go`, `pipeline_normalize.go`, and `notarius_test.go`. +- `internal/adapters/notarius/runner.go`, `subprocess.go`, `fake.go`, and + `subprocess_test.go`. + +Work: + +1. Add the optional field to `NotariusConfig` and the `safeNotarius` projection, + including its returned value. Let existing ConfigKit composition handle + overrides and null; do not add a default or resolve this name as a path. +2. Add the optional field to `notarius.RunRequest` and append the argument pair + when non-nil in `SubprocessRunner.Run`, before `--json`. Leave subprocess + environment construction and error propagation unchanged. +3. Audit any explicit Notarius configuration/request copies for the new field. + Update actual omissions without introducing a generic cloning framework. +4. Extend real configuration-load tests to cover a base selection, an overlay + replacing it with a differently named downstream selection, and an overlay + clearing it with null. Cover omitted, empty, and `default` values compactly. + Verify the safe effective-YAML output retains a supplied value, including + explicit empty, and round-trips without leaking credential values. Reuse + existing credential tests rather than duplicating their full matrix. +5. Extend adapter invocation tests for omitted and supplied selections, + including empty and `default`. Assert the argument/value boundary rather + than imposing new ordering requirements on unrelated arguments. Reuse the + existing nonzero-exit test with a selected profile to prove there is no + fallback or successful receipt processing after failure. + +Verification: + +```sh +go test ./internal/config ./internal/adapters/notarius +``` + +Completion: Configuration composition and safe inspection preserve the field, +and the adapter forwards it correctly. The executing stage is connected in +Stage 2 together with reuse tracking, so it cannot start using untracked profile +selections as an intermediate implementation. + +## Stage 2: Wire Extraction and Profile-Sensitive Reuse + +Status: Pending. + +Preparation: + +- `docs/internal/stage-extract.md`. +- `internal/stage/extract.go`, `extract_resume.go`, + `semantic_contracts_delivery.go`, and their focused tests. +- In particular, `TestExtractStageResolvesManifestInputAndBuildsExactRequest`, + `TestExtractSemanticConfigSensitivityAndOperationalExclusions`, and + `TestExtractStageResumeValidationRequiresFreshEvidence`. + +Work: + +1. Pass `cfg.ConfigProfile` to the adapter request when extraction executes. + Keep the field independent of credential forwarding and force overrides. +2. Implement the reuse-identity helper and additive fingerprint fields described + above. Update both `extractSemanticConfig` and `fingerprintDocument`, using + the same identity in semantic comparison and `extractionFingerprint`. + Keep disabled extraction insensitive to unused downstream settings. +3. Extend the existing stage request test to prove the authored selection + reaches the adapter. Cover ordinary and forced execution sufficiently to + show that profile forwarding does not alter existing cache-control precedence. +4. Extend semantic-sensitivity coverage: base to named, named to different named, + and named to base change evidence; unchanged selection does not. Verify + nil/empty/`default` share base evidence and disabled settings remain inert. +5. Exercise the focused resume validator with an actual valid recorded bundle: + it remains reusable for the same selection and becomes non-resumable for a + changed selection. This guards the second fingerprint path independently + of the application's semantic comparison. +6. Check the additive encoding against the prior no-profile payload shape once + in focused coverage or a recorded development check. Avoid whole-manifest + snapshots and a permanent historical compatibility harness; retain existing + input/output/reference sorting and all unrelated fingerprint fields. + +Verification: + +```sh +go test ./internal/stage ./internal/adapters/notarius ./internal/config +``` + +Completion: Selected profiles reach actual extraction, and both existing reuse +checks account for the selection without forcing downstream cache refresh or +changing the manifest schema. + +## Stage 3: Verify Application Lifecycle Behavior + +Status: Pending. + +Preparation: + +- `internal/app/extract_lifecycle_test.go`, `runner_test.go`, and + `run_control_test.go`. +- Existing fixtures and tests for changed extraction outcomes, failure/retry, + and independent render/extract invalidation. + +Work: + +1. Extend the existing lifecycle fixtures rather than building another harness. + Run a successful extraction with a configured profile, then verify ordinary + reuse for that same profile. Change only the downstream selection and verify + that extraction executes again with the new value, upstream transcripts and + render remain current, and succeeded dependents are invalidated/rerun through + the existing runner policy. Include returning from a named selection to base. +2. Cover the changed-profile failure boundary with the existing fake runner: + return an ordinary downstream error and verify extraction is failed, the + previous result is not advertised as current success, and dependents are not + silently reused. Reuse existing retry coverage where it already proves the + recovery behavior. Do not assert Notarius-specific diagnostic wording. +3. Ensure one configuration-to-execution path proves explicit composition: + Narratio's testing overlay can select a differently named Notarius overlay; + without the field, selecting a Narratio profile does not populate the + downstream request. Use real configuration loading and the existing injected + runner, not a production provider or sibling repository dependency. +4. Review existing operational-exclusion and force tests for continued validity. + Do not broaden invalidation to the full effective configuration digest or + Narratio's profile label, and do not automatically change cache options. + +Verification: + +```sh +go test ./internal/app ./internal/stage +``` + +Completion: Tests demonstrate the requested workflow, including selection +changes and failure, across the real Narratio configuration/runner boundaries. +Notarius's own overlay loader and cache internals remain outside these tests. + +## Stage 4: Document the Feature and Validate the Completed Change + +Status: Pending. + +Preparation: Canonical documentation owners listed in the feature roadmap, +`examples/README.md`, and the existing development-focused operations guidance. + +Work: + +1. Document the optional field in `docs/config.md`, including omitted/null, + explicit base selectors, and independence from Narratio's profile and + `notarius.llm_api_key`. Do not add profile-name restrictions. +2. Update the invocation in `docs/integrations/notarius.md` to include the + optional flag and identify Notarius v0.7.0 as a known supporting version, + not an unconditional version requirement. Preserve the distinction from + `--llm-profile` and ordinary downstream failure handling. +3. Update `docs/internal/stage-extract.md` to describe selection-sensitive reuse + and retain its distinction from private file content and operational settings. + Make clear this is an explicit downstream setting, not blanket invalidation + by Narratio's profile label or effective digest. +4. Add concise development workflow guidance in `docs/operations.md`: explicit + testing-overlay composition; different downstream names are allowed; + switching selection triggers extraction when that stage is in scope; edits + behind an unchanged selector may need force; testing is not a separate + workspace. Preserve the existing distinction between extract+analyze and + analyze-only regeneration. +5. Update the maintained annotated pipeline example to show the optional field + without requiring an externally installed testing overlay (use a commented + named-selection example). Do not make previously runnable examples depend + on new downstream files. Link to canonical owners instead of duplicating + full contracts or adding new top-level Narratio flags. +6. Review the final diff against every observable behavior and scope boundary + in the consumer request. Mark the implementation stages and roadmap status + accurately after verification; do not remove the original request or alter + historical release notes. + +Run the repository checks with the maintained Go 1.26 toolchain: + +```sh +go test ./... +go test -race ./... +go vet ./... +go build ./... +go test ./internal/doccheck +go test ./internal/config -run '^TestExamplesLoadAndValidate$' +git diff --check +``` + +Completion: The requested interface, failure and reuse behavior, examples, and +canonical documentation agree; all checks pass. No release is part of this +implementation sprint. + +## Open Questions + +None. The consumer request resolves the product behavior. The choices above +settle implementation details without adding new operator restrictions, +downstream configuration ownership, or cache guarantees. diff --git a/docs/roadmap/notarius-config-profile.md b/docs/roadmap/notarius-config-profile.md new file mode 100644 index 0000000..0676a17 --- /dev/null +++ b/docs/roadmap/notarius-config-profile.md @@ -0,0 +1,151 @@ +# Notarius Configuration Profile Selection + +Status: Proposed; not implemented. + +## Goal and Authority + +Implement the consumer behavior in the +[feature request](feature-request-notarius-config-profile.yml): let a Narratio +configuration overlay explicitly select a Notarius configuration overlay. +Operators can select testing once in Narratio while each application retains +ownership of its own model choices and other settings. + +This roadmap describes the target state and required change areas, not an +implementation sequence; the [implementation plan](implementation.md) owns +the ordered work and implementation decisions. The feature request is the scope +authority; the mechanisms suggested below do not establish additional +user-facing contracts. + +## Underlying Issue + +Narratio can select its own testing profile, but extraction currently invokes +Notarius without a configuration-profile selector. Consequently, a Narratio +testing invocation cannot explicitly select the independently deployed Notarius +testing overlay through pipeline configuration. + +Adding individual downstream model or tuning fields would duplicate Notarius's +configuration responsibilities. Passing its existing profile selector provides +the requested flexibility without that duplication. + +## Targeted End State + +### Explicit Configuration Composition + +`pipeline.notarius.config_profile` is an optional Notarius configuration-profile +name, usable in Narratio's base configuration and ordinary overlays. For +example, Narratio's `profiles/testing.yml` may contain: + +```yaml +notarius: + config_profile: testing +``` + +Selecting Narratio's `--profile testing` then selects that explicit downstream +profile whenever extraction executes. The two profile names need not match. +Narratio does not automatically forward its own profile name. + +When supplied, the selection is passed through Notarius's `--config-profile` +argument. Omitted or null values add no profile-selection flag and preserve +existing downstream selection behavior. Notarius's selector semantics remain +authoritative, including empty and `default` selections for the composed base +configuration. Narratio does not introduce a stricter profile-name policy. + +This field selects a ConfigKit configuration overlay, not a PromptKit profile +through `--llm-profile`. The Notarius overlay owns any resulting LLM-profile +selection. Notarius resolves overlays relative to its own configuration root; +Narratio neither resolves those files against its own root nor rewrites them. + +### Execution, Reuse, and Failure + +A changed downstream profile selection must not silently reuse Narratio's +completed extraction from the previous selection. Existing dependent-stage +invalidation applies when extraction changes; the feature does not change the +pipeline order or transcript-generation behavior. + +Notarius continues to own downstream overlay discovery, composition, validation, +and cache compatibility. A missing or invalid selected overlay must fail +extraction through the existing subprocess error path, without silently falling +back to production or reporting successful extraction. + +Existing resume and chunk-cache controls, including force precedence, remain +in place. Profile selection does not itself mean forced regeneration or cache +deletion. Narratio's decision to execute extraction and Notarius's decision to +reuse compatible work remain separate responsibilities. + +Changes to private Notarius configuration behind an unchanged selector retain +the existing operator responsibility to force extraction when needed. There is +no new downstream file inspection or content-tracking requirement. + +Credential forwarding remains independently configured through +`notarius.llm_api_key`. A Narratio testing overlay may select a different +credential reference alongside the downstream profile; no new credential +discovery or provider-name mapping is introduced. + +## Required Change Areas + +### Configuration and Inspection + +Extend the typed Notarius configuration and its existing safe effective-YAML +projection to carry the optional field. Preserve the ordinary configuration +composition behavior for base values, overlay overrides, and null values. +Configuration inspection must reflect the selected field without exposing +credential values. No replacement configuration loader or downstream preflight +loader is needed. + +### Stage and Adapter Boundary + +Carry the selection from resolved configuration through the extraction request +to the subprocess adapter. Construct the optional CLI flag using the existing +argument-vector mechanism, alongside the current configuration path, references, +credential forwarding, and cache controls. Retain normal Notarius error handling. + +### Extraction Reuse Evidence + +Account for the downstream selection in the existing extraction reuse evidence. +The current semantic-configuration and invocation fingerprints are the relevant +change points. Keep their treatment of selection consistent so that a profile +change cannot bypass the required rerun and downstream invalidation. + +This roadmap does not require a new manifest schema, separate profile-indexed +workspaces, historical per-profile result retention, or an upgrade-wide forced +rerun. + +### Verification and Documentation + +Use focused behavioral coverage for configuration composition and inspection, +explicit argument forwarding and omission, changed-selection reuse decisions, +and ordinary subprocess failure handling. Reuse existing test coverage where +it already protects unchanged credentials, force behavior, and invalidation; +do not duplicate Notarius's complete profile-validation or cache test suites. + +When implemented, update the canonical +[configuration reference](../config.md), +[Notarius integration contract](../integrations/notarius.md), +[extract internals](../internal/stage-extract.md), and relevant maintained +examples. The development workflow in [Operations](../operations.md) should +explain explicit profile composition and the existing need to force extraction +after private downstream edits. Clarify that profile selection does not create +an isolated testing workspace or separate current extraction state. + +## Compatibility and Scope Boundaries + +Notarius v0.7.0 already documents `--config-profile`; no upstream change is +required. Document the capability needed when the option is used, without adding +an unconditional runtime version gate for existing unconfigured deployments. + +This feature adds no Narratio CLI selector, automatic profile propagation, +Notarius LLM-profile selector, downstream configuration rewriting, new +credential mechanism, or generalized recursive orchestration. Existing output +and receipt contracts remain unchanged. + +The consumer's Ansible role and suite updates are follow-up work outside this +repository. They can deploy downstream overlays and populate this field after +the interface is released. + +## Scope Review + +No stronger requirements or new invariants are proposed beyond the feature +request and existing repository contracts. In particular, minimizing one-time +upgrade invalidation and avoiding reruns between equivalent base selectors are +implementation considerations, not additional acceptance guarantees imposed +by this roadmap. No user decisions are currently required before planning.