diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index c561df7..4009c9d 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -47,8 +47,10 @@ only; do not inspect or monitor unreleased work in anticipation of adoption. Independent per-side domain selectors and opt-in comparison assertions are now included in the [working roadmap](promptkit-v0.19.0.md#additional-configkit-operator-workflows). -Their binding and outcome contracts remain open under roadmap OD-9 and OD-10; -retaining those proposals does not change current comparison behavior. +Roadmap OD-9 selects explicit application-declared right-side bindings with +inheritance and collection replacement; empty-collection grammar remains open. +OD-10 selects an optional assertion outcome with a distinct difference status. +These accepted plans do not change the currently implemented behavior. [Roadmap OD-5](promptkit-v0.19.0.md#od-5-detailed-comparison-selection-and-shape) selects an optional detailed report while retaining the concise default for @@ -65,7 +67,8 @@ structured diagnostics belong to `config validate`; the suite retains six subcommands. The roadmap records the agreed explanation modes and help behavior; [OD-6](promptkit-v0.19.0.md#od-6-failure-evidence-and-structured-diagnostic-delivery) selects optional validation details, a concise default, and available evidence -on failure. Report shapes and filters remain open. +on failure. OD-8 selects focused display, source selection, and report filters; +report shapes and exact filter grammar remain open. It also includes focused effective display, source-ID selection, metadata/diagnostic filters, independent comparison selectors, and assertion mode. The original four shared proposals had ConfigKit @@ -73,11 +76,10 @@ adoption identifiers OD-3 through OD-6; the new roadmap has its own decision num belongs in `ckconfig`, with PromptKit exposing native evidence through its configuration scope. CLI syntax and report contracts remain subject to review. -Transient acquisition overrides are an explicit open scope decision in -[roadmap OD-11](promptkit-v0.19.0.md#od-11-transient-configuration-overrides), -with a recommendation for opt-in CLI support and alternatives using -application-specific flags or a structured override document. Persistent -configuration editing remains deferred. +Transient acquisition overrides are included as an opt-in loader capability in +[roadmap OD-11](promptkit-v0.19.0.md#od-11-transient-configuration-overrides). +Loader composition, numeric conversion, unusual path syntax, and comparison-side +clearing remain design tasks. Persistent configuration editing remains deferred. Discovery, import defaults, collection identities, and field-level resolution recording remain available through application loaders. Discovery tracing, @@ -179,7 +181,7 @@ operator or consumer need and an appropriate ownership/behavior decision: - JSON reports and output-file management or reloadable export. - Shell completion extensions, including alias discovery described above and dynamic discovery of configured or remote values. -- Persistent configuration mutation. Transient CLI override scope is owned by +- Persistent configuration mutation. Transient CLI overrides are selected under roadmap OD-11; field metadata and provenance inspection are scoped under [configuration inspection extensions](#configuration-inspection-extensions). - Credential diagnostics beyond PromptKit's native smoke checks and additional diff --git a/docs/roadmap/promptkit-v0.19.0.md b/docs/roadmap/promptkit-v0.19.0.md index 9f11a0d..fe96819 100644 --- a/docs/roadmap/promptkit-v0.19.0.md +++ b/docs/roadmap/promptkit-v0.19.0.md @@ -116,11 +116,11 @@ diagnostics while retaining concise validation by default. Report shapes and filter spellings remain proposals under OD-4, OD-6, and OD-8. No separate path inventory or diagnostics subcommand is planned. -Transient configuration overrides are an open scope decision under OD-11, with -a recommendation to include them as an optional loader capability. Illustrative -`config validate --set /workers=0` and `config show --set /server/port=8080` -would apply explicit overrides during acquisition without editing source files. -Their inclusion and exact CLI/API contract remain for review. +Transient configuration overrides are included under OD-11 as an opt-in loader +capability. `config validate --set /workers=0` and +`config show --set /server/port=8080` apply temporary overrides during acquisition +without editing source files. Numeric conversion, unusual pointer keys, loader +composition, and comparison-side grammar remain to be finalized. ### Compatibility and native semantics @@ -281,11 +281,11 @@ application; the comparison provider still receives two selections. **Comparison assertions.** Base a difference outcome on native record presence, including hidden changes whose redacted snapshots look equal. Equal effective values can have different histories or explicitness. Comparison also does not -detect changed credential bytes behind an unchanged secret reference. Define -the opt-in outcome and report-delivery behavior under OD-10; retain the current -ordinary diff outcome. A load or comparison failure is not evidence of equality. +detect changed credential bytes behind an unchanged secret reference. Use +the agreed opt-in outcome and report-delivery behavior under OD-10; retain the +current ordinary diff outcome. A load or comparison failure is not evidence of equality. -**Transient override candidate.** If selected in OD-11, parse explicitly supplied +**Transient overrides.** As agreed in OD-11, parse explicitly supplied CLI values into native `Override` records and pass them through a declared loader capability. ConfigKit owns override path validity, supported values, precedence, decoding, resolution, provenance, and validation. Its configuration overrides @@ -318,9 +318,9 @@ errors remain errors even if an override would replace the affected value. Retain the six-command suite and wire native PromptKit facets through `ConfigScope`. Add focused effective display, source selection, and bounded metadata/diagnostic filters using those evidence capabilities. Extend typed - comparison selection and add the selected assertion outcome. If OD-11 is - accepted, implement explicit override acquisition and comparison-side wiring - as a separate reviewable unit. Keep boundary coverage with each change. + comparison selection and add the agreed assertion outcome. Implement explicit + override acquisition and comparison-side wiring under OD-11 as a separate + reviewable unit. Keep boundary coverage with each change. 5. **Update operator and consumer documentation.** The CLI reference owns syntax, report schemas, availability, and outcomes; GoDoc owns callback and value contracts. Extend maintained simple/composed examples to demonstrate @@ -370,14 +370,16 @@ Default tests remain offline and independent of real credentials. - [ ] Text and file input names and aliases follow OD-2. Literal content, including empty text, reaches native inline inputs; file bindings remain file references. Examples, help, and documentation use the selected meanings. - Resolve and cover duplicate-name precedence across the two input forms. + Verify that the last command-line occurrence for a name wins across text/file + forms and all aliases, preserving actual occurrence order. - [ ] Output-contract override and appended-message flags remain outside this adoption, with their operator use cases retained in future work. - [ ] Opt-in namespace selectors combine declared suite defaults with supplied fields in one native view update. Omission preserves each declared default; explicit empty values clear only the selected default. Qualified selections and catalog filters retain their native meanings. Custom scoped loaders remain - usable without the option. + usable without the option. Canonical namespace names and scoped aliases match + OD-3, with an explicit applicability table. - [ ] The full configuration suite retains six subcommands. Explanation discovery and validation diagnostics are modes of existing commands, with capability-aware help and selected mode/flag combination checks. @@ -394,17 +396,21 @@ Default tests remain offline and independent of real credentials. report-delivery failures without extra configuration loads or comparisons. - [ ] Focused display preserves native snapshot content and distinguishes missing/historical-only paths from explicit null. Whole-document display - retains its current contract. + retains its current contract. Focused display uses one scope and native YAML + without a snapshot wrapper; absent effective paths and unknown source IDs + produce clear operational failures. - [ ] Source-ID selection works on successful and failed acquisitions. Path, explicitness, severity, and code filters preserve the selected evidence and source references without masking the acquisition outcome. - [ ] Independent comparison selectors preserve the chosen inheritance and repeatable-value rules, side-specific checks, and shared-state lifecycle. + Explicit empty right-side collection selection is designed and covered. - [ ] Assertion mode distinguishes equality, native differences (including - hidden changes), usage errors, and operational failures under the selected - outcome contract. Report-delivery failures remain observable. -- [ ] OD-11 has an explicit scope decision. If included, only supplied CLI - overrides reach opted-in loaders, with typed values, safe flag labels, + hidden changes), usage errors, and operational failures with statuses + `0`, `3`, `2`, and `1` respectively, without changing ordinary diff outcomes. + Report-delivery failures remain observable. +- [ ] Only supplied CLI overrides reach opted-in loaders under OD-11, with typed + values and safe flag labels, selected comparison-side behavior, native errors, and returned provenance preserved. No tests independently re-establish ConfigKit override semantics. - [ ] Default validation remains concise; `--details` selects the additional @@ -451,7 +457,7 @@ from configuration to deliver it in this roadmap. General execution, prepared handle lifecycle, sessions, and observer convenience APIs remain application responsibilities. Persistent configuration editing, new live probes, dynamic completion, JSON report formats, and unrelated deferred proposals are not added by this -roadmap. Transient acquisition overrides remain an explicit choice in OD-11. +roadmap. Transient acquisition overrides are included under OD-11. Output-contract override and appended-message flags are deferred under OD-2 and retained in [future work](future.md#promptkit-inspection-and-selection-extensions). @@ -578,9 +584,12 @@ choose whether these bindings fit their workflow and may opt into aliases. None of these aliases is application-wide by default. Do not add prompt-input flags to smoke tests, which use their own diagnostic content. -Duplicate-name precedence across text and file bindings remains a design point -for review. The current recommendation is last command-line occurrence wins -across both forms; this is not yet an agreed cross-binding ordering contract. +**Agreed input precedence:** the last command-line occurrence for a given input +name wins across text and file bindings, including every alias. Implementation +must preserve actual occurrence order rather than applying one flag collection +after the other in a fixed order. Select the necessary binding composition +during implementation; do not claim this behavior from independent flag binding +alone. **Deferred output-contract overrides:** no CLI override is included in this adoption. Retain the candidate in @@ -618,9 +627,17 @@ The standard option owns composition instead of requiring an application callback to merge every selection. Applications needing custom namespace selection can retain their own loading behavior without enabling the option. -**Remaining design:** select exact flag names, scoped convenience aliases, -and applicability to individual commands. These details are not settled by -agreement on default composition. +**Agreed names:** + +| Canonical selector | Scoped alias within applicable `promptkit` commands | +| --- | --- | +| `--llm-prompt-namespace` | `--prompt-namespace` | +| `--llm-profile-namespace` | `--profile-namespace` | +| `--llm-backend-namespace` | `--backend-namespace` | + +Keep catalog `--namespace` filtering separate. Applicability to individual +commands and exact public helper signatures remain design tasks; install only +selectors with a defined use for that operation. ### OD-4: Configuration path and metadata presentation @@ -715,81 +732,81 @@ validation across all scopes. ### OD-7: Module release and interim workspace coordination -**Recommendation:** prepare a new ckconfig minor release for the shared API/CLI -features, then update the still-unpublished ckprompt candidate to require it. -Confirm available versions under the existing release procedure. During joint -development use the ignored aligned workspace; record that workspace success -does not establish independent ckprompt compatibility with its old ckconfig -requirement. Before final independent validation, pin the released ckconfig -version actually supplying the APIs. Publication requires its own authorization. +**Agreed:** complete the planned integration work together rather than splitting +out a dependency-only PromptKit release. With no current consumers, an early +compatibility release has no demonstrated benefit. Prepare a new ckconfig minor +release for the shared API/CLI features, then update the still-unpublished +ckprompt candidate to require that released version. Confirm candidate status +and available versions under the release procedure before choosing versions. -**Alternative:** publish a compatibility-only ckprompt release first and deliver -the additional CLI features in subsequent releases. This makes v0.19.0 fixes -available sooner, but creates an extra consumer upgrade and does not complete -this roadmap's targeted end state. A disposable local module replacement can -help test intermediate changes, but cannot substitute for final validation of -the declared released dependency. +Use the ignored aligned workspace during joint development. Workspace success +does not establish independent ckprompt correctness with its old ckconfig +requirement. Before final independent validation, pin the released ckconfig +version supplying the APIs and validate both independent modules and the aligned +workspace. Release ckconfig before ckprompt. Publication remains a separate +action requiring authorization; this decision does not authorize publication. ### OD-8: Focused configuration reports and filtering -**Recommendation:** extend existing commands: `config show --path POINTER` and +**Agreed:** add `config show --path POINTER` and `config sources --source-id ID`; add subtree/explicitness filters to -explanation discovery and severity/code filters to validation diagnostics. Keep -unfiltered reports as the default within those modes. Use the existing -explanation capability for selected effective YAML, and recommend an operational -failure for an absent effective path or -unknown source ID, consistent with unknown-path explanation. Recommend starting -path-selected display with one scope; whole-document `--scope all` remains -available. Specify presence-aware explicitness filtering so historical-only -entries are not labeled implicitly defined. These choices reuse existing -operator vocabulary and native evidence with little additional acquisition work. +`config explain --list` and severity/code filters to `config validate --details`. +Keep reports unfiltered by default. The purpose is to reduce noise for a human +operator investigating a known part of the configuration. -**Alternative:** offer a structured mode of `config show --path POINTER` with -a snapshot envelope containing `present`, `redacted`, and `yaml`, and return -absence as report data. This is convenient for scripts and preserves -unknown/null/absence distinctions without relying on exit status, but adds a -report format choice alongside native effective YAML. Aggregate path-selected -reports are also viable if scope-labeled per-scope absence and delivery semantics -are designed. Filters can be deferred -in favor of downstream YAML tools, reducing CLI surface but making routine -operator inspection more cumbersome. +Focused show emits the selected native effective YAML directly, without a +snapshot wrapper containing presence/redaction/YAML fields. Use the existing +explanation capability to obtain the native snapshot. An absent effective path +or unknown source ID produces a clear operational failure. Explicitly present +null remains a valid value; historical-only paths are not effective values. +Path-selected display initially uses one scope, while whole-document +`--scope all` remains available. + +Filters change presentation, not validation outcomes. Keep source references +needed to interpret the selected findings. Apply subtree filtering to parsed +paths; distinguish unavailable current metadata on historical-only paths from +an effective value that was not explicitly supplied. + +**Remaining design:** exact filter names, values, combination rules, and empty +filtered-result presentation. No additional snapshot-wrapper report or aggregate +path-selected display is selected for this adoption. ### OD-9: Independent comparison selector bindings -**Recommendation:** let applications declare which domain selectors support -right-side comparison values, with explicit metadata for their `--against-*` -spellings. Omission inherits the left value and presence; an explicit right -value replaces it. For repeatable selectors, recommend replacing the inherited -collection with the supplied right collection. This follows config/overlay -selection conventions and supports comparisons such as two campaigns without -requiring different files. Review the selected refinement of ADR 0012 and flag -collision handling before implementation; exact helper signatures remain open. +**Agreed:** applications declare which domain selectors support right-side +comparison values, with explicit metadata for their `--against-*` spellings. +Do not generate comparison flags for every application option automatically. +Omission inherits the left value and presence; an explicit right value replaces +it. For repeatable selectors, the supplied right collection replaces the +inherited collection rather than appending to it. This supports comparisons +such as two campaigns without requiring different files or repeating every +unchanged selection. -**Alternative:** require symmetric explicit left/right selectors for opted-in -bindings. This makes each side independent and easier to reason about when -defaults are complex, but adds repetition and departs from the existing -right-inherits-left convention. Either design keeps application domain parsing -in its declared bindings and avoids inventing a generic selector language. +Domain parsing stays in the declared typed bindings; no generic selector +language is introduced. Review the refinement of ADR 0012 and flag collision +handling before implementation. Exact helper signatures and an explicit way to +select an empty right-side collection remain open. An empty string element is +not necessarily an empty collection; do not equate them implicitly. ### OD-10: Comparison assertion outcomes -**Recommendation:** add opt-in `--check`, retaining normal report delivery and -ordinary diff behavior when omitted. Recommend status `0` for equality and a -domain difference status `3`; retain current usage `2` and operational `1` -classification. This lets scripts distinguish differences from failed -comparison. The status is a proposal, not an existing contract. Apply existing -reporting outcome rules, including preservation of a nonzero domain outcome -when presentation also fails; callers can inspect the separate presentation -evidence. Document that interaction explicitly. +**Agreed:** add opt-in `--check`, retaining normal report delivery and ordinary +diff behavior when omitted. Under assertion mode, use status `0` for equality, +`3` for differences, `2` for usage errors, and `1` for operational failures. +This distinguishes a successful comparison that found differences from a +comparison that could not run. Ordinary diff remains successful when it finds +differences. -**Alternative:** use status `1` for both differences and operational failures -in the opt-in assertion mode. This fits simple shell checks and avoids a new -difference status, but requires additional evidence to distinguish detected -changes from failed comparison. Both approaches remain modes of `config diff`. +Base the difference outcome on native records, including concealed changes, +not equality of the displayed safe values. Apply existing reporting outcome +rules, including preservation of a nonzero domain outcome when presentation +also fails; callers can inspect the separate presentation evidence. Document +and test that interaction explicitly. These are selected future behaviors, +not claims about the currently implemented CLI. ### OD-11: Transient configuration overrides -**Recommendation:** include a declared, opt-in override capability for standard +**Agreed scope:** include a declared, opt-in override capability for standard configuration commands, with repeatable `--set POINTER=JSON` values passed to application acquisition as native ConfigKit overrides. This enables validation, inspection, and comparison of temporary settings through consistent CLI syntax. @@ -799,7 +816,10 @@ required. The loader contract must state how it combines these records with any existing application-owned overrides; the generic adapter cannot inject them into an arbitrary loader after the load has completed. -If selected, settle these details before implementation: +Prefer inline assignments for ordinary one-value experiments. This is the +largest remaining loader integration design item; revisit scope explicitly if +the required acquisition changes prove disproportionate rather than silently +dropping the agreed feature. Settle these details before implementation: - Split at the first `=`; JSON supplies typed null, boolean, string, numeric, array, and object values. Specify numeric conversion to ConfigKit-supported @@ -814,22 +834,16 @@ If selected, settle these details before implementation: language. Empty, false, zero, and null remain intentional supplied values. - For comparison, recommend `--set` as the shared baseline and `--against-set` as a right-side replacement list when supplied, following - OD-9's proposed repeatable-selector rule. Define an explicit way to select + OD-9's agreed repeatable-selector rule. Define an explicit way to select an empty right list before finalizing this grammar; appending or replacing individual paths would need a different selected rule. - Reflect native CLI origins and subsequent resolution in existing reports. Test CommandKit parsing, forwarding, and outcome/report adaptation, leaving ConfigKit's override algorithm to its own tests. -**Alternative A:** retain application-specific typed flags and loader translation -to `Options.Overrides`. This gives applications tailored vocabulary and avoids -generic JSON numeric/path syntax, but repeats CLI infrastructure and prevents -uniform temporary-setting workflows across applications. This is a viable -deferral if the generic acquisition contract is too disruptive for this sprint. - -**Alternative B:** accept a structured override document containing path/value -records instead of inline `--set` assignments. It handles complex values and -unusual pointer keys naturally and makes an empty comparison-side list explicit, -but adds file-input handling and is less convenient for one-value shell changes. -It still needs deliberate typed-value decoding and loader opt-in. Neither -alternative authorizes persistent configuration mutation. +A structured override file is not selected as the primary interface: it handles +complex values and unusual keys well but adds file setup to ordinary one-value +experiments. It remains a possible supplemental form if needed to resolve the +open grammar cases. Application-specific typed flags remain available, but are +not a substitute for the selected standard CLI capability. No persistent +configuration mutation is authorized by this decision.