From 18ca21fa4561bb4201288a04e6b2a0ac796ee550 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 9 Oct 2026 04:50:10 +0000 Subject: [PATCH] Retire completed roadmap documentation --- docs/roadmap/implementation.md | 271 -------------------- docs/roadmap/inspection.md | 220 ---------------- docs/roadmap/listing.md | 120 --------- docs/roadmap/performance.md | 193 -------------- docs/roadmap/releasekit.md | 124 --------- docs/roadmap/remediation.md | 243 ------------------ docs/roadmap/scriptorium-feature-request.md | 176 ------------- 7 files changed, 1347 deletions(-) delete mode 100644 docs/roadmap/implementation.md delete mode 100644 docs/roadmap/inspection.md delete mode 100644 docs/roadmap/listing.md delete mode 100644 docs/roadmap/performance.md delete mode 100644 docs/roadmap/releasekit.md delete mode 100644 docs/roadmap/remediation.md delete mode 100644 docs/roadmap/scriptorium-feature-request.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 36a3ff4..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,271 +0,0 @@ -# Profile And Prompt Listing Implementation Plan - -Implement the [listing roadmap](listing.md) in the numbered order below. -Each stage is scoped for one Luna coding-agent prompt and should leave a -compiling, tested unit. All four stages are complete. - -This replaces the completed performance plan, preserved in -[Git history](https://gitea.maximumdirect.net/eric/promptkit/src/commit/7325a34c070c77c4784ab76c916fc8a4d79b5c46/docs/roadmap/implementation.md). -For older inspection-roadmap references to this pathname, the completed -[inspection plan](https://gitea.maximumdirect.net/eric/promptkit/src/commit/481e2d494c5a4e8b7f3a2a6f1cceaeb3f7c71d05/docs/roadmap/implementation.md) -is also preserved. These historical plans are not work for this sprint. - -## Baseline And Working Rules - -Planning baseline: `7325a34c070c77c4784ab76c916fc8a4d79b5c46`. -There are no listing methods yet. Configuration already captures profiles in -`loaded.indexedProfiles` and prompts in `loaded.prompts`, after the final -ConfigKit handoff. `internal/engine.Engine` currently retains the runner and -the operator inspection view. - -Start with [development](../development.md), then read the applicable -[agent](../policy/agents.md), [architecture](../policy/architecture.md), -[testing](../policy/testing.md), and -[documentation](../policy/documentation.md) policies. Reuse unchanged documents -already read. Consult [internal sources](../internal/sources.md), the -[component overview](../internal/overview.md), the -[consumer guide](../consumers/pkg-promptkit.md), existing inspection GoDoc, and -the [configuration ADR](../adr/0001-configuration-boundary.md). - -For each stage: inspect the current worktree, preserve unrelated edits, implement -only its scope, format changed Go files, run focused verification, review the -diff, update this plan's stage status with brief evidence, and commit the -nonempty completed unit with a plain-English message. Report any blocked -validation honestly. Publication and release preparation are separate tasks. - -Use the existing module versions. No dependency upgrade, configuration-format -change, execution refactor, new public package, or release-tool change is needed. -The storage and wiring choices below are implementation decisions, not permanent -consumer invariants. Tests protect outcomes, not private helper names, clone -counts, allocation counts, or timing thresholds. - -## Selected Public Contract - -Use these root-package declarations: - -```go -type PromptIdentity struct { - ID string - Version string -} - -func (e *Engine) ListProfiles(ctx context.Context) ([]string, error) -func (e *Engine) ListPrompts(ctx context.Context) ([]PromptIdentity, error) -``` - -- List final effective captured identities, including maintained and consumer - profiles. Inherited profiles keep their own keys. Return each profile ID and - each prompt ID/version pair once; omit historical-only contributions. -- Preserve literal spelling. Sort profiles lexically by ID; sort prompts - lexically by ID, then version. Do not interpret versions as SemVer or select - a preferred version. Existing omitted-version inspection remains unchanged. -- Results are caller-owned. Concurrent listing and other supported engine - operations cannot mutate the captured inventory through these results. -- Empty inventories succeed. Nil and zero-value public engines return an error - matching `ErrInvalidConfig`, consistent with existing facade checks. -- For a valid engine, context cancellation/deadline errors retain their standard - identity. Return no partial inventory on error. Follow existing inspection - precedence: validate the public engine before delegating context handling. - Check context before and after the synchronous copy/conversion work; do not - promise immediate interruption or deterministic precedence in a cancel/return - race. No new nil-context behavior is selected. -- `PromptIdentity` follows other inspection values: no stable JSON - representation is promised. Do not add custom serialization or pin nil versus - nonnil empty slices in GoDoc or tests. -- Membership is not execution readiness. Listing performs no per-item full - inspection, target validation, credential resolution, resource reads, - rendering, admission, generation, or ConfigKit provenance traversal. - -The roadmap owns scope and intent; implemented GoDoc will own the exact API. -Do not add backend listing, filters, pagination, reloads, runtime mutation, -provider discovery, new caller limits, or stronger catalog availability promises. - -## Stage 1 — Capture And Read Identity Inventories - -**Status: Complete (2026-10-09).** Verified with -`go test ./internal/configuration ./internal/domain` and -`go vet ./internal/configuration ./internal/domain`. Adapter tests cover -effective sorted identities, literal keys, partial profiles, empty loads, -caller-owned results, and canceled contexts. - -**Scope:** `internal/domain`, `internal/configuration`, and focused adapter tests. - -1. Add an internal `domain.PromptIdentity` with only `ID` and `Version` strings. - Keep it separate from the eventual public declaration; no public type alias. -2. Add private profile-ID and prompt-identity slices to `loaded`. Populate them - once at the end of `load`, after both `materializePrompts` and - `buildExecutionProfiles` have succeeded. Enumerate the finished - `indexedProfiles` keys and the finished `prompts` ID/version keys, not - ConfigKit metadata or raw contributions. -3. Build the ordered slices with existing `sortedKeys` or standard-library - sorting. Flatten actual prompt versions; an ID with no versions contributes - no prompt identity. Store only strings, not targets, definitions, or histories. -4. Add `ListProfiles(context.Context) ([]string, error)` and - `ListPrompts(context.Context) ([]domain.PromptIdentity, error)` on `loaded`, - in a focused `internal/configuration/listing.go`. Check context, copy the - captured slice, check context again, and return the independent result. - Ordinary slice copying suffices because elements contain only strings. - Reads are linear in returned inventory size and require no repeated sorting. -5. Add adapter tests using the real `Load` and existing mapping helpers. Cover - a small unsorted profile/prompt fixture, multiple versions such as `"10"` - and `"2"`, literal whitespace/slash keys, a partial profile, and an empty - load. Mutate returned entries and verify subsequent lists are unchanged. - Cover an already-canceled context for each method without timing assertions. - -Do not change existing lookup methods or add inventory mutation. Keep -construction complexity proportionate; there is no benchmark gate. - -**Verify:** format changed files; run -`go test ./internal/configuration ./internal/domain` and -`go vet ./internal/configuration ./internal/domain`. - -**Done when:** successful loads retain sorted identity-only inventories and -return owned lists without rereading ConfigKit or requiring runnable targets. - -## Stage 2 — Wire The Public Facade And Its Basic Contract - -**Status: Complete (2026-10-09).** Verified with -`go test ./internal/engine .`, `go vet ./internal/engine .`, and -`git diff --check`. External contract tests cover both listing methods, -inspection of listed identities, invalid engines, cancellation, and expired -deadlines. Depends on Stage 1. - -**Scope:** `internal/engine`, root API/conversion, and focused facade tests. - -1. Add a private, consumer-owned inventory interface in `internal/engine` - containing just the two listing methods from Stage 1. Store the loaded - snapshot through this interface during `New`, alongside the existing - operator inspection view. Delegate new internal engine methods to it. - There is no need to extend execution repository interfaces or the use-case - runner, add optional capabilities, or provide a full-inspection fallback. -2. Add `listing.go` at the module root, containing public `PromptIdentity` and - the two methods. Apply the same `e == nil || e.internal == nil` check as - existing inspection methods, returning `ErrInvalidConfig` with useful - context. Map delegated errors through `mapPublicError`. -3. Return the already-owned profile slice. Convert internal prompt identities - into a separate public slice by copying their two strings. Check context - after conversion so a detected cancellation yields no partial result. - Do not export an internal type in any public signature. -4. Write concise GoDoc covering effective membership, exact lexical ordering, - all prompt versions, captured-state behavior, caller ownership, concurrent - use, empty success, nil-engine and context errors, and absence of readiness - checks or external work. Identifiers may be sensitive application data; - do not describe these lists as automatically safe for disclosure. -5. Add external-package basic tests in `listing_contract_test.go`: a usable - public engine lists a configured profile and two exact prompt versions; - returned identities can be inspected; nil/zero engines match - `ErrInvalidConfig`; canceled and expired contexts preserve context identity - with no partial inventory. Use real constructors and synthetic configuration. - -**Verify:** format changed files; run -`go test ./internal/engine .` and `go vet ./internal/engine .`. -Existing root boundary tests must continue to pass; do not weaken them. - -**Done when:** consumers can call both methods through the root facade and -receive independent public values with the selected error behavior. - -## Stage 3 — Verify Effective Membership, Ownership, And Concurrent Use - -**Status: Complete (2026-10-09).** Verified with -`go test -race ./internal/configuration ./internal/engine .` and -`git diff --check`. External tests cover layered effective membership, -maintained profiles, empty inventories, caller ownership, concurrent listing -and inspection, captured sources, and the absence of execution work. Depends on -Stage 2. - -**Scope:** focused additions to the external-package listing contract tests. -Production edits are limited to defects those tests reveal. - -1. Build one representative layered engine with a consumer base profile, an - inherited child, an overlay of the same profile ID, and multiple prompt - versions. Verify effective identities, no duplicate overlay entry, and - child/base identities remaining distinct. Inspect one listed identity to - confirm it resolves to the effective captured definition. -2. Verify maintained profiles are included without asserting a complete catalog - inventory or a specific upstream model. The two PromptKit-owned standard - profile names may serve as stable membership examples under the existing - release policy; other expectations should use fixture-controlled identities. -3. Exercise empty effective inventories through a public engine: a final main - mapping with `profiles: null` and `prompts: null` clears the sections above - the catalog underlays. Assert successful zero-length lists, not a particular - nil/empty representation. This also checks that lower-layer historical - profile paths are not returned. Use existing ConfigKit semantics; add no - source-validation pass or special listing-only removal rule. -4. Mutate elements in both public results and relist to verify independent - storage across calls. Add one bounded concurrent test that lists and mutates - independent results while other goroutines list/inspect the same captured - engine. Coordinate deterministically; avoid sleeps or shared test-owned - mutable result slices. -5. Cover snapshot/no-execution behavior with a compact fixture: construct from - an application filesystem, then remove/change its content after construction - and verify listing still reports the captured identities. Inject existing - model-client and artifact-reader stubs that fail if invoked. Include a - partial profile or a named but unavailable API-key environment variable; - listing succeeds without provider readiness. Do not add new extension points - or exact private-call-count assertions solely for these tests. - -Place effective-loading/ordering mechanics in Stage 1's adapter tests and -public assembled outcomes here. Avoid duplicating every loader, inheritance, -or existing inspection case. No execution/network call is required to prove -identity discovery. - -**Verify:** `go test -race ./internal/configuration ./internal/engine .`. - -**Done when:** public tests demonstrate effective—not historical—membership, -independent results, concurrent safety, and listing without execution work. - -## Stage 4 — Document And Verify The Completed Feature - -**Status: Complete (2026-10-09).** Verified with the full offline race suite, -repository-wide vet, maintained consumer examples, changed documentation-link -checks, and `git diff --check`. The listing roadmap and canonical consumer, -format, and internal documentation now describe the implemented API. Depends -on Stage 3. - -**Scope:** owning documentation, final review, and complete offline verification. - -1. Update `doc.go` to include listing in its operation and concurrent-use - summaries, and `PromptIdentity` among values without stable JSON contracts. - Keep exact behavior in the method/type GoDoc from Stage 2. -2. Add a short consumer-guide discovery example listing identities and then - inspecting a selected entry. Distinguish listing from configuration history - and from execution readiness; link to the exact GoDoc. A new runnable example - project or CLI is unnecessary. -3. Update `docs/internal/sources.md` and `docs/internal/overview.md` for captured - inventories and direct internal-engine delegation. The execution runner has - not acquired a listing responsibility. Add a brief format-reference link to - the listing guidance where it currently introduces inspection, without - changing configuration semantics. -4. Repair old `implementation.md` links in the completed `performance.md`, - `inspection.md`, and historical sections of `remediation.md` to their - respective history links at the top of this plan. Preserve finding statuses - and recorded evidence; do not retire unrelated roadmaps or imply their - work remains pending. This is navigation repair, not another feature. -5. Review the entire feature diff against the listing roadmap and selected - contract. Confirm only effective indexes are used; no runtime imports of - ConfigKit, provenance traversal, readiness checks, resource reads, new - limits, or stable serialization promises have been introduced. -6. Run the complete offline race suite and full vet, plus maintained examples - if changed or affected. Validate changed documentation links and - `git diff --check`. Reuse successful focused checks for unchanged content. -7. Mark the listing roadmap and this plan complete with concise verification - evidence. Commit the completed unit. Leave release notes, roadmap retirement, - tags, and publication for the later release-preparation task. - -Use the existing local Go installation and prepared module cache. Suggested -final commands (not new release gates): - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ - go test -race -vet=off -mod=readonly -count=1 ./... -GOTOOLCHAIN=local GOWORK=off GOPROXY=off go vet -mod=readonly ./... -git diff --check -``` - -**Done when:** all four stages are verified, listing is documented by its -canonical owners, and the feature is ready for ordinary release preparation. - -## Open Questions - -None. The selected signatures, result shape, ordering, ownership, error -behavior, implementation boundary, and verification scope are specified above. diff --git a/docs/roadmap/inspection.md b/docs/roadmap/inspection.md deleted file mode 100644 index 9b4aaa1..0000000 --- a/docs/roadmap/inspection.md +++ /dev/null @@ -1,220 +0,0 @@ -# Effective Prompt Requirement Inspection - -Status: Implemented (2026-10-08). The -[inspection implementation plan](https://gitea.maximumdirect.net/eric/promptkit/src/commit/481e2d494c5a4e8b7f3a2a6f1cceaeb3f7c71d05/docs/roadmap/implementation.md) records the completed work units and -final verification. Current public API details live in GoDoc; the consumer -guide, format reference, and internal documents link to those canonical -contracts. - -## Purpose - -Let consumers enumerate a selected prompt's inputs and caller variables before -its input artifacts exist, with each name classified as required, conditional, -or optional for the supplied context. Scriptorium can present that inventory -to operators and Narratio can compare planned bindings with exact input names -before undertaking expensive pipeline work. - -The originating request is [Scriptorium's feature request](scriptorium-feature-request.md). -Promptkit supplies application-neutral inspection; consumers own presentation, -binding checks, and their own workflows. - -## Original Review Baseline - -At review baseline `b595dca`: - -- [InspectPrompt](../consumers/pkg-promptkit.md#inspect-a-prompt-before-preparation) - selects a captured prompt and reports declared inputs and output metadata, - without analyzing templates or resolving an execution profile. -- [InspectProfile](../consumers/pkg-promptkit.md#inspect-a-profile-before-prompt-work) - reports a captured profile's effective target separately. -- [Prepare](../consumers/pkg-promptkit.md#prepare-without-model-execution) - selects an execution context, loads artifacts and applicable schemas, and - renders messages. It needs actual input data. -- Engine construction captures final definitions and compiled message/session - templates. Rendering uses Go templates, string-valued caller variables, and - the `input` helper. Required input declarations are checked independently of - branches; an executed helper also needs its named input. -- Declared content types and descriptions currently use string fields. The - new inspection needs declaration information and effective metadata presence, - including the difference between an absent field and an explicitly empty one. - -The [format reference](../formats.md#messages-and-templates), existing GoDoc, -and [internal source document](../internal/sources.md) own current behavior. - -## Target Public Operation - -Add `Engine.InspectRequirements` with dedicated public request/result values. -Preserve `InspectPrompt`, `InspectProfile`, preparation, and execution. - -The request supplies a prompt ID, optional version and profile selectors, -caller variables, and an optional direct session ID. It needs no artifact -references or contents, output path, or credentials. - -Use execution's selection semantics: exact identities, omitted-version -ambiguity, request profile or prompt default, and normalized direct session-ID -bypass. Report the selected prompt ID/version and effective profile ID. -Selecting a profile establishes identity, not generation readiness; partial -profiles remain inspectable. - -Use the configured engine's captured definitions. Source selection remains a -consumer choice. Results describe that captured context, not a later engine or -execution with different variables. - -## Result And Classification - -Return an inventory of inputs and a separate inventory of caller variables. -Each entry has one of three requirement classifications: - -- **Required:** A declaration or recognized definite access establishes that - the caller must supply it. -- **Conditional:** Its necessity remains unresolved, including references on - paths whose applicability the supported analysis cannot determine. -- **Optional:** It is known not to be required, including recognizable references - used only on known inactive paths and nonrequired declarations or lookups. - -A required declaration stays required even in an inactive branch. A definite -helper use promotes an optional input declaration to required. Aggregate repeated -occurrences by exact name: required takes precedence over conditional, which -takes precedence over optional. Conservative conditional classification is -acceptable where analysis cannot establish necessity; inspection need not prove -equivalence between alternative execution paths. - -Input entries distinguish declarations from discovered undeclared references -and preserve available descriptions/content types. They do not infer input -schemas or introduce format enforcement. Variables identify caller-map keys, -not template-local symbol names, and report whether each key was supplied. -An empty supplied string is present; absence is separate from classification. - -Include recognizable references on known inactive paths in both inventories -as optional entries with an inactive-use explanation. Keep explanations separate -from declared input descriptions. An applicable required or conditional use -elsewhere overrides the optional classification. Any applicable occurrence, -including an optional lookup, removes an inactive-only explanation. Known -inactivity introduces neither an execution requirement nor discovery uncertainty. - -Deduplicate names and provide predictable ordering. Results also identify the -selected context and provide discovery completeness with machine-readable -limitation categories and safe explanations. Exact public spellings belong in -the implementation plan and, once implemented, GoDoc. No Promptkit JSON wire -schema is part of this feature. - -## Simple Discovery Boundary - -Use parsed template syntax, not raw-text matching or rendering with placeholder -artifacts. The intended algorithm is straightforward: - -1. Seed the inventory from input declarations and their effective metadata. -2. Visit the selected message templates and applicable session template, - collecting recognizable input and caller-variable references. -3. Classify occurrences using their surrounding path and a small amount of - supplied-context evaluation. -4. Aggregate entries and report any unresolved discovery. - -The source inventory is finite and ordinarily small. Direct traversal and -name-indexed aggregation are sufficient; no size caps, analysis budgets, -branch-combination enumeration, or global analysis cache are needed. - -Recognize literal input names, direct caller-variable access, literal-key root -lookups, direct variable-derived input names, and ordinary `if`/`with` branches. -Use simple truthiness, negation, and equality/inequality where straightforward. -Empty strings and the nonempty string `"false"` follow actual Go truthiness. -The implementation plan defines this small initial coverage precisely. - -Retain enough lexical/dot scope awareness to distinguish caller variables from -locals. Complicated assignments, pipelines, computed names, ranges, or invoked -named templates can remain outside complete analysis. Preserve identifiable -references and report limitations rather than implementing a general template -interpreter, joining assignment environments, or propagating possible-name sets. -No cross-branch proof of a common required dependency is necessary. - -On a known inactive path, collect names visible in syntax without evaluating -its expressions. Literal input names and recognizable caller keys remain in -the inventory; skipped dynamic input names need not be resolved. Unsupported -skipped expressions do not themselves make discovery incomplete. - -A direct nonblank session ID excludes the bypassed session template from both -inventories. Never-invoked template definitions do not create requirements. -Declarations and references in selected messages remain relevant. - -Complex computed names may be missed by the inventory. That is an accepted -analysis limitation, not a reason to reject an otherwise valid prompt. -Inspection must expose potential incompleteness rather than imply that unknown -names are absent. Template authors benefit from keeping dependency expressions -simple, but this feature adds no template-authoring prohibition. - -## Completeness And Limitations - -Completeness describes discovery, not satisfaction or readiness. A missing -variable used only for direct text substitution can be a known required entry -with complete discovery. A missing value used to select a branch or name can -leave dependencies unresolved. - -When unavailable input content, unresolved names, missing decision values, or -unsupported analysis affect discovery, return known entries with successful -incomplete status and structured reasons. Conservative incompleteness is -acceptable; inspection need not prove an exact requirement set through complex -expressions. Unsupported pure text with no effect on dependencies need not -introduce uncertainty. - -Unresolved names can potentially refer to existing optional entries. Account -for that possibility in classification, while leaving established required -entries required. Do not label a potentially needed entry definitively optional. - -An empty complete input inventory, or a complete input inventory containing -only optional entries, establishes that no artifact bindings are needed. -An empty incomplete inventory does not. Consumers must use classifications, -not inventory length, to identify bindings that need preparation. - -## Effects, Errors, And Disclosure - -Inspection uses captured resources without future artifact reads, schema -preparation, credential lookup, model calls, capacity admission, observations, -run IDs, or execution artifact creation. Necessary configuration/resource -failures can still prevent engine construction or inspection. - -Preserve publicly classifiable selection errors and ordinary context -cancellation. Discovery uncertainty is a successful result, not an operational -error or a new definition-validity rule. A complete result does not promise -successful rendering, generation, future artifact validity, or provider -availability. - -Results and diagnostics exclude credential values, artifact payloads, -raw/rendered prompt content, and unrelated supplied values. Selected identities, -input/variable names, and declared descriptions are intentional operator -metadata. A directly resolved input identifier may equal a supplied variable -value; reporting that binding name is necessary, without echoing its originating -expression or variable map. - -## Implemented Outcome - -The root API selects a captured prompt and profile identity, then returns -caller-owned input and variable inventories with conservative classifications -and structured discovery limitations. The internal prompt walker uses captured -template syntax and effective metadata; inspection does not enter artifact, -schema, credential, rendering, provider, or execution preparation. - -The final verification and the commit sequence are recorded in the -[inspection implementation plan](https://gitea.maximumdirect.net/eric/promptkit/src/commit/481e2d494c5a4e8b7f3a2a6f1cceaeb3f7c71d05/docs/roadmap/implementation.md). The exact public contract is owned by -[GoDoc](../../requirements.go); consumer usage, framework-format cross-reference, -and internal selection/source descriptions are maintained in their respective -documents. - -## Acceptance Outcomes - -- Required `transcript` is reported exactly before its artifact exists. -- Literal undeclared helper names are discoverable without definition rejection. -- Input and variable references are classified as required, conditional, or - optional, including recognizable inactive references in both inventories. -- Required declarations stay required; definite uses promote optional entries; - other occurrences never erase an established required use. -- Simple supplied-context branches and direct dynamic names follow execution; - complicated or content-dependent cases preserve known entries and limitations. -- Caller keys remain distinct from template locals, and supplied empty values - remain distinct from absent keys. -- Selection and session bypass follow execution without readiness checks. -- Complete artifact-free inventories remain distinguishable from empty partial - discovery; completeness remains separate from supplied-variable presence. -- Results/errors preserve the disclosure and public classification boundaries. -- Existing inspection/execution contracts remain intact. Application APIs, - pipeline planning, binding validation, presentation, provider preflight, input - schemas, and a general-purpose template evaluator remain outside this scope. diff --git a/docs/roadmap/listing.md b/docs/roadmap/listing.md deleted file mode 100644 index 2dd8af7..0000000 --- a/docs/roadmap/listing.md +++ /dev/null @@ -1,120 +0,0 @@ -# Profile And Prompt Listing - -Status: Implemented (2026-10-09). All public listing behavior and verification -are complete; see the [implementation plan](implementation.md) for completion -and verification evidence. - -## Purpose - -Let consumers discover the profiles and prompt versions available in an engine -without knowing their identifiers in advance or interpreting configuration -history. This supports application selectors, CLI discovery, and configuration -editors while keeping PromptKit's public API small. - -Listing supplies identities; the existing inspection methods supply details. -The [implementation plan](implementation.md) records the completed work. This -roadmap captures feature scope; the public GoDoc owns the current API behavior. - -## Current Baseline - -Review baseline: `03c44923c8a14565dca30f65d93bde79e0962ea2`. - -- `Engine.InspectProfile` and `Engine.InspectPrompt` inspect a known identity. - Neither enumerates available entries. -- `Engine.InspectConfiguration` exposes ConfigKit's captured inspection view. - Consumers can enumerate historical paths and filter for surviving effective - entries, but that is an indirect configuration-inspection workflow. -- The configuration adapter already captures profiles by literal ID and prompts - by literal ID/version pair. Those execution-facing indexes are sufficient - for listing; no new configuration loading or registry is needed. - -The [consumer guide](../consumers/pkg-promptkit.md) describes existing inspection -workflows. The [format reference](../formats.md) owns keyed definitions and -literal identifier semantics. Preserve the final-configuration boundary in -[ADR 0001](../adr/0001-configuration-boundary.md) and the facade ownership in -the [architecture policy](../policy/architecture.md). - -## Target End State - -### Small Public Listing API - -Add two engine methods, following the context and error conventions of the -existing prompt/profile inspection methods: - -- `ListProfiles(ctx)` returns profile IDs. -- `ListPrompts(ctx)` returns small identity values containing `ID` and `Version`. - -Expose only the identity value needed for prompt listing. GoDoc will own the -exact signatures and behavior when implemented. Callers can pass the returned -identities to `InspectProfile`, `InspectPrompt`, or their execution requests. -Bulk inspection results, prompt text, execution targets, source provenance, -and credential information do not belong in these results. - -### Effective Captured Inventory - -List the identities present in the engine's final captured configuration, -including maintained catalog profiles and consumer-defined profiles and -prompts after ConfigKit layering and inheritance. A profile's configured alias -or inherited identity remains its own list entry, not its base profile's name. -Historical or superseded contributions do not create additional entries. - -Each effective profile ID appears once. Each effective prompt ID/version pair -appears once, including every configured version of a multi-version prompt. -Listing does not choose a default or latest version; the existing omitted-version -selection behavior remains unchanged. - -Preserve exact identifier spelling. Return profiles in ascending lexical ID -order and prompts in ascending lexical ID order, then lexical version order. -Version ordering is for deterministic presentation, not semantic-version -comparison or a recommendation about which version to execute. - -### Lightweight Snapshot Read - -Listing reads the engine's captured identity indexes. It does not reload sources -or files, analyze configuration provenance, resolve credentials, read artifacts -or schemas, render, acquire backend capacity, or contact providers. - -A profile need not be execution-ready to be listed. Presence in the inventory -does not promise valid credentials, provider availability, or successful -execution. Catalog entries remain subject to the existing -[catalog compatibility policy](../policy/release.md#catalog-profile-compatibility). - -Returned slices are caller-owned; modifying one result cannot change the -engine or later results. Listing supports concurrent engine use without adding -mutable registry state. An empty inventory is a successful result. A nil engine -uses `ErrInvalidConfig`, and cancellation preserves the context error under the -existing inspection conventions. - -## Work Needed From The Baseline - -- Add identity-only enumeration to the captured configuration adapter and route - it through the internal engine to the public facade. Keep internal - representations behind the existing package boundary. -- Add the two public methods and the small prompt identity value, with GoDoc. - Avoid obtaining each item's full inspection or copying profile parameters - and prompt content merely to list its identity. -- Keep traversal and allocation proportional to the inventory returned. - Prefer straightforward reuse of captured indexes; ordering can be prepared - during construction rather than repeatedly sorting unchanged identities. - Exact storage layouts and allocation counts are implementation choices. -- Add focused tests for effective inventory membership, multiple prompt - versions, literal identities and ordering, caller ownership, empty results, - and the selected error/cancellation behavior. Include a representative public - workflow that lists entries and inspects a returned identity. Reuse existing - construction coverage rather than retesting ConfigKit's resolution machinery. -- Add concise listing guidance to the consumer guide and update the relevant - internal documentation when implemented. Preserve existing inspection and - execution contracts. - -Backend enumeration, filtering, pagination, runtime registration, and live -catalog/provider discovery are outside this feature's scope. No new caller -limits, readiness checks, release gates, or performance guarantees are needed. - -## Completion - -Consumers can enumerate effective profiles and exact prompt versions through -the public facade, obtain independent deterministic results, and use those -identities with existing operations. Verification follows the -[testing policy](../policy/testing.md), including focused ownership and -concurrent-use checks proportionate to the implementation. Current documentation -describes the implemented API without duplicating ConfigKit history semantics. diff --git a/docs/roadmap/performance.md b/docs/roadmap/performance.md deleted file mode 100644 index 5f59164..0000000 --- a/docs/roadmap/performance.md +++ /dev/null @@ -1,193 +0,0 @@ -# Computational Efficiency Improvements - -Status: Complete (2026-10-09). The ConfigKit request (item 1) was submitted, -and local items 2–7 were implemented or completed with the evidence and -conditional ownership decision recorded in the -[performance implementation plan](https://gitea.maximumdirect.net/eric/promptkit/src/commit/7325a34c070c77c4784ab76c916fc8a4d79b5c46/docs/roadmap/implementation.md). No dependency upgrade or public -performance guarantee was part of this work. - -## Purpose - -Reduce avoidable CPU work and allocations in Promptkit's existing configuration, -preparation, rendering, and execution paths. Large payloads and concurrent -library use make redundant full-payload copies especially worth addressing. -Prefer small, idiomatic improvements over additional framework machinery. - -This roadmap captures the intended outcomes and recommended changes. The -[performance implementation plan](https://gitea.maximumdirect.net/eric/promptkit/src/commit/7325a34c070c77c4784ab76c916fc8a4d79b5c46/docs/roadmap/implementation.md) owns staged work and verification. -This roadmap introduces no new public performance guarantee. Existing contracts -remain owned by GoDoc, the -[format reference](../formats.md), and the -[integration contract](../integrations/openai-compatible-chat.md). Preserve the -boundaries in the [architecture policy](../policy/architecture.md) and -[configuration ADR](../adr/0001-configuration-boundary.md). - -## Review Evidence - -The review used existing package benchmarks and temporary offline probes at -baseline `de6443f`. Approximate measurements on the review machine included: - -- Configuration construction for 256 effective definitions: 69 ms, 50 MB of - cumulative allocations, and roughly 539,000 allocations per construction. - Allocation profiling pointed predominantly to ConfigKit's path and metadata - machinery. Indexed selection remained fast. -- Basic validation of a 1 MiB nonempty response: an additional 1 MiB allocation - and about 0.2 ms, caused by conversion to a string for the whitespace check. -- Isolated template execution with about 272 KiB of output: changing the - output writer from a buffer to a string builder reduced cumulative allocation - from about 816 KiB to 544 KiB and execution time by about 27%. -- Request encoding with a 1 MiB extra-parameter value: a single-serialization - diagnostic roughly halved encoding time and cumulative allocated bytes. -- Reading a 1 MiB regular file: approximately 4.5 MiB of cumulative allocations - after warm-up, including repeated buffer growth. - -These are local measurements of particular fixtures, not end-to-end provider -latency predictions, retained-heap measurements, or acceptance thresholds. -Snapshot-copy and inspection opportunities below were identified from code; -their benefits still need measurement. - -## Scope And Target Outcomes - -### 1. Submit A ConfigKit Performance Feature Request - -Completed on 2026-10-08. The request is recorded in ConfigKit's -`docs/roadmap/promptkit-performance-request.md`, committed as `d53021e`. -It includes the downstream fixture, dependency and toolchain versions, -repeated benchmark results, reproduction commands, and allocation-profile -observations, while preserving existing configuration and inspection semantics. - -Acceptance, design, and implementation remain with ConfigKit. Upstream -improvements will become available through a later ConfigKit dependency upgrade; -no further Promptkit-specific work remains for this item at present. Local -performance improvements can proceed independently. - -### 2. Avoid Response Copies In Basic Validation - -In the [validator](../../internal/validate/standard_validator.go), perform the -empty/whitespace-only check directly on the artifact's byte slice, rather than -converting the entire body to a string. `bytes.TrimSpace` is the recommended -standard-library operation. - -The target is to remove the response-sized temporary allocation while retaining -current Unicode whitespace behavior, validation results, and cancellation -handling. Empty, whitespace-only, and nonempty outputs keep their existing -meaning; this is not a validation-policy change. - -### 3. Avoid The Final Rendered-Message Copy - -Use a string-oriented output writer where the -[renderer](../../internal/prompt/go_renderer.go) currently accumulates template -output in a byte buffer and then obtains a copied string. A local -`strings.Builder` is the recommended approach for message output; apply the -same simplification to session rendering where appropriate. - -The target is fewer output-sized allocations with unchanged rendered bytes, -template behavior, and failure/cancellation handling. Preserve captured parses, -request-specific template bindings, and the existing per-render artifact-text -cache. No shared writer pool or global render cache is needed for this change. - -### 4. Eliminate Duplicate Serialization Of Provider Extras - -Refine [request encoding](../../internal/llm/openai_compatible_client.go) so -provider extra-parameter values are not serialized once for validation and -again as part of the complete request. Assemble and encode the request without -discarding an earlier serialization of the same values. - -Preserve the reserved-field checks, accepted input types, wire semantics, useful -error context, and existing public error categories. Consolidating encoding is -not permission to skip validation or turn invalid requests into transport -failures. Continue freezing one serialized request for transport retries. - -### 5. Reduce Artifact File-Read Buffer Growth - -Improve allocation in the -[ordinary artifact reader](../../internal/artifact/reader.go), which currently -grows an initially empty slice despite having the opened file's metadata. -Use the opened size as a sensible initial-capacity hint, with safe integer -handling, rather than repeatedly copying already-read content. - -The actual read still determines the returned content and size. Retain regular -file checks, chunked cancellation handling, error behavior, and reading through -EOF, including when the file changes after its metadata was obtained. This -optimization introduces no new file-size limit or filesystem policy. - -### 6. Consolidate Redundant Prepared-State Copies Where Justified - -Review ownership through -[prepared execution](../../internal/usecase/prepared_execution.go) and -[public conversion](../../convert.go). Creation currently builds two deep -snapshots, and returning details adds further internal and public copies. -Large schemas and nested provider maps can make this expensive. Copying message -structures does not itself duplicate their immutable string contents; measure -mutable payload copies separately from message length. - -Where internal ownership is demonstrably exclusive, consolidate overlapping -copies or transfer owned values rather than cloning again. Retain copies needed -to isolate engine state, the execution payload, returned details, and injected -collaborators. Preserve prepared-execution lifecycle and concurrency behavior. -Before removing a copy, identify which component owns each mutable value and -whether another component can still access it. In particular, preserve the -independence and availability of details after execution or discard described -by the [public handle contract](../../prepared_execution.go). - -Measure representative cases before selecting the refactor. There is no target -copy count and no requirement to remove every defensive copy; retain the simpler -design where savings do not justify additional ownership complexity. - -### 7. Keep Identity-Only Inspection Lightweight - -[Requirement inspection](../../internal/usecase/requirements_inspection.go) -selects a full execution profile, including a deep copy of provider extras, -although it uses only the selected profile identity. Review nearby prompt -selection for similarly unnecessary copying of data that inspection does not -consume. - -Use narrower internal selection or defer payload copying where that reduces -work without duplicating selection semantics. Preserve exact identity lookup, -omitted-selector behavior, partial-profile inspectability, and existing public -errors. The target is inspection cost driven by the information being inspected, -not unrelated provider payloads. This requires no new public lookup API and -does not move provenance analysis into ordinary prompt execution. - -## Benchmarking And Verification - -Use focused, offline before/after benchmarks to establish useful savings in -time, allocated bytes, and allocation count. Cover representative small inputs -as well as larger messages, responses, files, schemas, and provider maps where -they exercise the affected path. - -Preserve reproducible fixtures and comparison methods for the temporary probes -behind the review figures. Record the baseline and changed revisions, Go and -dependency versions, relevant machine details, benchmark commands, and repeated -results alongside the comparison. The existing construction benchmark already -provides a reproducible fixture; capture equivalent evidence for the other -affected paths before using their preliminary figures to judge improvements. - -Distinguish cumulative allocations from retained memory. Where relevant, -measure retained heap with representative lifetimes and concurrency, especially -for builder-backed strings and long-lived prepared handles. A reduction in -allocated bytes alone does not establish a reduction in retained memory. - -Add coverage for the production captured-template renderer: the existing -artifact-reference benchmark uses the reparsing fallback. Include representative -assembled preparation and execution with an injected model client so local -overhead can be measured independently of provider latency. Use repeatable -comparisons rather than machine-specific timing or allocation thresholds. - -Follow the [testing policy](../policy/testing.md): protect observable behavior -and consequential ownership, cancellation, concurrency, and error-boundary -risks, reusing existing protection where sufficient. Benchmarks support these -changes; they are not additional release gates or public speed guarantees. - -## Intended End State - -ConfigKit has received a reproducible downstream performance request for its -consideration. Promptkit avoids the identified unnecessary validation, -rendering, encoding, and file-buffer allocations. Prepared-state and inspection -copying are simplified where evidence supports a maintainable improvement. - -Consumers retain the same substantive API behavior and ownership protections. -Existing indexed selection, captured templates, per-render text reuse, -operation-local schema reuse, serialized retry reuse, and constant-time queue -removal remain useful foundations. New caches, execution limits, configuration -semantics, and release gates are not part of this work. diff --git a/docs/roadmap/releasekit.md b/docs/roadmap/releasekit.md deleted file mode 100644 index ed2865d..0000000 --- a/docs/roadmap/releasekit.md +++ /dev/null @@ -1,124 +0,0 @@ -# ReleaseKit v0.3.0 Upgrade Roadmap - -Status: Complete. ReleaseKit v0.3.0 is selected and the upgraded integration -is verified. Current workflow requirements belong in the -[release procedure](../release.md) and [release policy](../policy/release.md). - -## Purpose - -Upgrade PromptKit's existing ReleaseKit integration to benefit from upstream -validation efficiency and cancellation improvements. Keep reusable release -mechanics upstream and maintain only PromptKit's settings, thin invocation -wrappers, and product-specific checks locally. - -This is a focused tooling upgrade, not another release-process migration. -The [release policy](../policy/release.md) and -[release procedure](../release.md) continue to own PromptKit's release -requirements and workflow. The upgrade was implemented as one focused unit. - -## Current Baseline And Reviewed Upstream - -PromptKit baseline: `ea6c7d5f0f2c008d9c57c8ac25fbd6faa18b5fc6`. - -- `go.mod` declares the ReleaseKit CLI as a Go tool dependency, selecting - `gitea.maximumdirect.net/eric/ReleaseKit v0.2.0`. PromptKit already declares - `go 1.26.0`. -- `scripts/releasekit.sh` builds the module-selected native executable with the - active local Go toolchain; `scripts/release.sh` delegates publication to it. -- `releasekit.json` selects shared validation and the maintained offline - consumer-example check. PromptKit publishes source-only tags. -- `scripts/test-release.py` exercises local wiring in disposable repositories, - but its executable-metadata assertion hard-codes ReleaseKit `v0.2.0`. -- The runbook links to v0.2.0 upstream documentation. The completed original - [adoption roadmap](https://gitea.maximumdirect.net/eric/promptkit/src/commit/ea6c7d5f0f2c008d9c57c8ac25fbd6faa18b5fc6/docs/roadmap/releasekit.md) - remains available in Git history. - -The reviewed upstream release is `v0.3.0`, commit -`e43df4416e0ed987349b2ea04a2c68c25e552772`. Its local tag was checked against -the upstream remote. The -[release notes](https://gitea.maximumdirect.net/eric/ReleaseKit/src/tag/v0.3.0/docs/releases/v0.3.0.md) -describe reduced Git subprocess work, lower validation retention, fewer command -output copies, and improved cancellation during reader shutdown. Public -signatures, CLI commands, and profile schema are unchanged. The minimum Go -version rises to 1.26.0; PromptKit already satisfies it. - -Race-enabled tests passed for the reviewed upstream Git, validation, Go-tool, -process, and runtime-version packages. This evidence supports adoption; it does -not replace verification of PromptKit's upgraded integration. Upstream delivery -and runtime-version improvements require no new PromptKit integration. - -## Target End State - -### Module-Selected Tooling - -The existing tool declaration selects ReleaseKit v0.3.0 through PromptKit's -module graph. Module and checksum changes are committed, with no local -filesystem replacement or dependency on the sibling checkout. - -The current wrappers retain native command exit statuses and build the tool -using the same active local Go installation used for candidate validation. -There is no ReleaseKit runtime import in the PromptKit library. This upgrade -does not require raising PromptKit's existing Go minimum. - -### Existing Workflow And Validation - -Keep the current release profile and selected gates unchanged. Continue the -main-before-tag publication workflow and successful-push completion point, -without CI or module-availability waits. Existing release-note and recovery -requirements remain governed by the canonical policy and runbook. - -The upstream performance improvements apply through the dependency upgrade. -No local reimplementation, benchmark threshold, extra release gate, hosted -delivery integration, or additional profile capability is part of this scope. - -### Maintainable Wiring Verification - -The disposable release fixture verifies that its native executable was built -from the ReleaseKit version selected by the candidate module, rather than from -a second hard-coded version. Preserve its existing same-Go, publication, -validation-failure, usage-status, and unchanged-module-metadata coverage. - -Keep this verification focused on PromptKit's wiring. ReleaseKit owns tests of -its internal batching, parsing, cancellation, and publication implementation. - -## Changes Needed From The Baseline - -- **`go.mod` and `go.sum`:** upgrade the existing ReleaseKit tool dependency to - v0.3.0 and reconcile module metadata. Review resulting dependency selections; - upstream's own dependency versions are unchanged from v0.2.0. -- **`scripts/test-release.py`:** derive the expected ReleaseKit version from - the selected candidate module and use it in the build-metadata assertion and - diagnostics. Remove the duplicate v0.2.0 expectation. -- **`docs/release.md`:** advance current candidate-check and failure-recovery - links to the adopted v0.3.0 documentation. Preserve the existing procedure. -- **Notices, where applicable:** review selected upstream licensing and any - incorporated material under the [OSS policy](../policy/oss.md). Update only - notices warranted by actual use; tool use alone does not require a new - attribution entry. Preserve historical release records. - -The existing wrappers and `releasekit.json` need no intended changes. Update -other current documentation only where the upgrade makes it inaccurate; do not -rewrite historical references merely because they identify an earlier release. - -## Verification And Completion - -Prepare dependency caches before offline verification. Confirm that the -module-selected native tool builds with the active compatible local Go -installation. Run `python3 scripts/test-release.py` to exercise publication -and failure cases exclusively against disposable local repositories. Review -the module diff and updated documentation links, and use ordinary PromptKit -validation proportionate to the dependency change. - -Completion means v0.3.0 is selected, wiring verification passes without a -duplicated version pin, and current documentation agrees with the adopted -tool. No real tag publication is part of implementation verification. Future -PromptKit release preparation remains a separate task under the runbook. - -Implementation verification passed with Go 1.26.8: the module-selected wrapper -reported `releasekit v0.3.0`, the disposable publication and failure fixtures -passed, and the ordinary offline Go suite, full vet, and maintained consumer -examples passed. The fixtures also verified native usage status, same-Go -bootstrap, and unchanged candidate module metadata. Release profile and wrapper -contents are unchanged; no real PromptKit tag was published. Upstream notices -were reviewed; the tooling-only upgrade introduced no incorporated material -requiring a new attribution entry. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 1e436ac..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,243 +0,0 @@ -# Implementation Review Remediation - -Status: All recorded findings have been resolved in separate remediation units. - -## Computational Efficiency Review - -Review baseline: `8a5c0de`, compared with planning commit `3fde1f6`, the -[performance roadmap](performance.md), and [performance implementation plan](https://gitea.maximumdirect.net/eric/promptkit/src/commit/7325a34c070c77c4784ab76c916fc8a4d79b5c46/docs/roadmap/implementation.md). - -### Finding 1 — Release A Stale File Reservation For An Empty Body (P3) - -**Status: Resolved.** Completed empty bodies now release stale file-size -reservations using the existing compaction path. The stale-size table covers -empty content, metadata, and hashes; the retained-artifact benchmark includes -an empty-body fixture without enforcing capacity or allocation thresholds. -Three offline, one-iteration samples for four retained empty artifacts changed -from 4,194,928–4,194,976 retained bytes to 624–640 bytes. Initial reservation -allocations still occur; this fix reduces retained storage, not cumulative -allocation. The nonempty benchmark remains near 1 MiB for four 256 KiB bodies. -Artifact package race tests and vet passed, and the internal documentation -now accurately describes the compaction condition. - -Location: [file reader](../../internal/artifact/reader.go), lines 137–166. - -The new file-size reservation is compacted only when the completed body is -nonempty. If the opened file reports a positive size but yields no bytes -(for example, because it is truncated before the read), the empty artifact -retains the entire reservation. An offline diagnostic using the existing -opened-file test seam reported a 16 MiB size and returned immediate EOF: -the result had zero body bytes and size zero, but retained a 1 MiB backing -array. The previous reader did not reserve that storage for an empty result. - -This is a retained-memory edge case, not incorrect content, a permanent leak, -or a problem with ordinary empty files whose size hint is zero. It leaves -Stage 4's stale-hint cleanup incomplete. - -Remediation: release the reserved backing storage when the completed body is -empty, preserving existing empty-artifact semantics. Keep size and hash based -on the actual body, and retain the existing EOF and cancellation behavior. -No new input limit or public capacity guarantee is needed. - -Verification: extend the stale-size cases to include an empty actual file -with a positive reported size; check accepted content, metadata, and hash. -Use a focused retained-memory diagnostic or benchmark to verify that the -reservation is released, rather than making an exact capacity or allocation -count a durable test requirement. - -Also correct the accompanying wording in -[Ordinary Artifacts](../internal/sources.md#ordinary-artifacts): the current -predicate compacts when unused capacity exceeds the body length (total -capacity exceeds twice the body length), not when unused capacity exceeds -twice the body length. - -Review verification: the complete offline race suite, repository-wide Go vet, -and maintained consumer examples passed. No additional actionable architecture -gaps or strengthened public contracts were identified in this review. - -## Effective Requirement Inspection Review - -Review baseline: `a051edd82d9d20cdc82dfebf25adb689dc58b01f`. -Scope: implementation commits after `a6e0ffe`, compared with the simplified -[feature roadmap](inspection.md) and [inspection implementation plan](https://gitea.maximumdirect.net/eric/promptkit/src/commit/481e2d494c5a4e8b7f3a2a6f1cceaeb3f7c71d05/docs/roadmap/implementation.md). -The existing ReleaseKit documentation finding is retained below. - -### Finding 1 — Report Lost Caller-Map Scope As Incomplete (P2) - -**Status: Resolved.** Root-map `with` identity is retained; unknown local-map -accesses produce structured limitations and affect optional variable entries. -Focused regressions cover the reproduced forms, simple parentheses, and -inactive scopes. The package/race suite passed before completion of this unit. - -Locations: [requirement walker](../../internal/prompt/requirements.go), -`visit`'s `WithNode` handling (lines 481–503), `visitCommand` (lines 675–690), -and `visitVariable` (lines 694–708). - -The walker treats every `with` body as nonroot and suppresses an unresolved -condition issue unless a recognizable dependency was recorded in that body. -Unsupported local-map accesses are similarly discarded without an issue when -the dependency counter does not change. Losing the ability to recognize a -caller key can therefore be mistaken for proof that no key is referenced. - -With `Vars: {"audience": "visible"}`, all six of these valid templates render -`visible`, but inspection reports `Complete: true`, no variables, and no issues: - -```gotemplate -{{with .}}{{.audience}}{{end}} -{{with $}}{{.audience}}{{end}} -{{$alias := .}}{{with $alias}}{{.audience}}{{end}} -{{with .}}{{index . "audience"}}{{end}} -{{$alias := .}}{{index $alias "audience"}} -{{$alias := .}}{{$alias.audience}} -``` - -This is not merely accepted failure to enumerate a complex reference: the -result incorrectly claims complete discovery. Consumers can consequently -accept an incomplete variable inventory as exhaustive. - -Remediation: distinguish known nonroot values from unknown scope. Preserve -straightforward root identity where already determinable, or report an -unsupported-scope limitation when a reachable access may hide caller keys. -Unrecognized access must not depend on successfully recording a named -dependency before it can mark discovery incomplete. Apply the existing unknown -caller-key effect handling to affected optional entries as needed. Do not add -alias-value propagation or general assignment interpretation; conservative -partial results satisfy the approved scope. - -Verification: add focused package cases for root-map `with`, local-map field -access, and local-map `index`. A result may enumerate the known key correctly -or remain explicitly incomplete, but must not silently omit it from complete -discovery. Retain known-string dot isolation and inactive-path behavior. - -### Finding 2 — Avoid Recursive Rescanning Of Unsupported Names (P2) - -**Status: Resolved.** The duplicate fallback traversal was removed. A focused -source-relative work regression failed before the fix (4,096 accesses for a -277-byte expression) and passed afterward, preserving partial-result metadata. -No production limits or caches were added; prompt/use-case race tests passed. - -Location: [requirement walker](../../internal/prompt/requirements.go), -`evaluateInputName` (lines 231–244), through `evaluateValue`, -`evaluateCommand`, and `visitCommand`. - -Unsupported name evaluation already scans the expression for recognizable -references. `evaluateInputName` then calls `visit` on that same subtree again. -When a computed name contains another `input` with an unsupported computed -name, both traversals recursively repeat the same fallback. Work grows -exponentially even though the source grows linearly. - -An isolated diagnostic starts with `.selector` and repeatedly wraps it as -`input (printf "%s" (EXPRESSION))`. Observed visits to that single caller key -are 2, 4, 8, 64, and 1,024 at depths 1, 2, 3, 6, and 10 respectively. The -depth-10 source is only 233 bytes. Result deduplication masks the repeated work. - -Remediation: collect recognizable references once when simple name evaluation -falls back to syntax enumeration. Use one conservative traversal rather than -recursively reevaluating the same unsupported subtree. Preserve classifications, -known references, and safe issues; more expression support, memoization -machinery, or new size/depth/work restrictions are unnecessary. - -Verification: retain existing computed-name results and exercise a nested -unsupported expression. Use a small growth benchmark or equivalent focused -work check to guard against repeated recursive scans, without making exact -visit counts or a wall-clock threshold a public guarantee. - -### Finding 3 — Qualify The Public Variable-Disclosure Statement (P3) - -**Status: Resolved.** Request and method GoDoc now expressly distinguish caller -values from intentionally reported resolved input identifiers. Existing direct -name and privacy coverage passed in the assembled verification; no runtime -behavior or confidentiality mechanism changed. - -Location: [public request GoDoc](../../requirements.go), `Vars` (lines 22–24). - -The comment says that supplied values are not returned in inspection. -However, `{{input .binding}}` with `Vars: {"binding": "transcript"}` intentionally -returns `transcript` as an input identifier. The approved roadmap explicitly -permits that necessary identifier disclosure while excluding unrelated values. -The current wording can be read as a stronger privacy guarantee than was -requested or implemented. - -Remediation: state that caller values are not returned as values, while a -resolved input identifier can equal its supplying variable value. Put this -exception in the canonical GoDoc; preserve existing unrelated-value and -diagnostic disclosure protections. Do not redact legitimate binding names or -introduce a new confidentiality mechanism. - -Verification: review GoDoc against a direct dynamic-name result and the existing -privacy contract test. A documentation-only correction does not need another -full-suite run. - -### Review Evidence And Architecture - -- The root facade owns exported values, request copying, result copying, and - error conversion. The use-case runner reuses captured prompt/profile - selection and retrieves the exact snapshot without readiness checks. -- Configuration captures effective input metadata presence at the final - ConfigKit handoff. Parsed templates are shared read-only; request analysis - and inventories remain operation-local in `internal/prompt`. -- Inspection avoids artifact/schema loading, credentials, provider calls, - admission, observations, and run IDs. Required declarations, inactive input - and variable inventories, exact names, session bypass, and independent - returned metadata are implemented and covered by focused tests. -- The existing full offline race suite passed: - `go test -race -vet=off -mod=readonly -count=1 ./...`. - `go vet -mod=readonly ./...` and `python3 scripts/check-examples.py` also passed. - Commands used - `GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off`. -- Temporary Go overlay probes, outside the repository, reproduced Finding 1 - against real template execution and quantified the repeated work in Finding 2. - No implementation or maintained test files were changed during review. -- No new template rejection, caller limit, input-format constraint, readiness - requirement, or ConfigKit-boundary expansion was found in these commits. - Finding 3 identifies the one overstated documentation guarantee. - -## Earlier ReleaseKit Adoption Review - -Review baseline: `d5f3464`. The release implementation matches the selected -architecture; no runtime defect or additional release gate was identified. - -### Finding 1 — Complete Documentation Alignment (P3) - -**Status: Resolved.** The obsolete heading instruction was removed without -changing note title, summary, compatibility, or licensing-footer requirements. -The adoption roadmap records completion and past integration verification, -links to current canonical owners, and identifies its original baseline as -historical. No release checks or publication gates changed. - -Two descriptions were left behind during the cutover: - -- `docs/policy/release.md`, under Release Documentation, still says to use the - “Promptkit section structure below.” The former fixed-heading structure was - removed, and the adopted policy and runbook expressly require no fixed - section headings. This reference leaves the note-writing instruction unclear. -- `docs/roadmap/releasekit.md` still describes remaining adoption work as - pending and integration validation as future work. The implementation plan - records completion, and the review verified the assembled workflow. - -Remove the obsolete section-structure instruction while preserving the title, -summary, applicable compatibility guidance, and licensing-footer requirements. -Update the feature roadmap's implementation status, or retire it together with -the completed implementation plan during release preparation and repair links. -Do not reintroduce a heading validator or add a publication gate. - -Verification: review the affected descriptions for agreement with the actual -profile and runbook, and check changed local links. The disposable fixture -already demonstrates acceptance of a note without fixed section headings; -these prose corrections do not require repeating the complete Go suite. - -### Review Evidence - -- `python3 scripts/test-release.py` passed: successful annotated publication, - one race-enabled suite and additional check, same-Go tool build, unchanged - module metadata, failed additional validation preventing publication, and - native usage-error classification. -- A disposable clone of the review baseline, with only a synthetic release - note added, passed `scripts/releasekit.sh check v0.0.0` offline with the real - profile: race-enabled tests, full vet, Go formatting, Markdown navigation, - module-distribution checks, and the three real consumer examples. -- Shell syntax checks passed; the runtime root package's dependency list has - no ReleaseKit import. No runtime Go code or module metadata changed during - the implementation commits after `1889915`. -- No real Promptkit publication was performed. The adopted checks and retired - gates match the agreed scope; no substantive contract strengthening was found. diff --git a/docs/roadmap/scriptorium-feature-request.md b/docs/roadmap/scriptorium-feature-request.md deleted file mode 100644 index d06c790..0000000 --- a/docs/roadmap/scriptorium-feature-request.md +++ /dev/null @@ -1,176 +0,0 @@ -# Feature Request: Inspect Effective Prompt Requirements - -Status: proposed consumer request from Scriptorium, recorded 2026-10-08. -This document specifies requested public API behavior. API names and types are -open for agreement; implementation structure is left to Promptkit. - -## Consumer Need - -Scriptorium exposes Promptkit through CLI and HTTP interfaces for executing -predefined prompts. A subprocess consumer, Narratio, uses Scriptorium to produce -D&D session analysis from transcripts and other pipeline products. - -Narratio needs to check its input bindings before starting expensive upstream -work. At that point, the eventual input files often do not exist. A pipeline -source ID and a prompt input name are separate identities: a transcript product -may need to be bound under the exact prompt input name `transcript`. - -Scriptorium needs Promptkit to describe the selected execution context's input -and variable requirements through a supported public API. Scriptorium can then -present those requirements to operators and executable consumers without -interpreting prompt definitions or duplicating framework semantics. - -## Public API Gap - -The [consumer guide](../consumers/pkg-promptkit.md#inspect-a-prompt-before-preparation) -describes `InspectPrompt` as declared-input inspection without profile -resolution. That is useful, but declared inputs alone do not describe all -requirements introduced by templates. `InspectProfile` addresses profile -inspection separately, and `Prepare` loads inputs and renders messages. - -The requested operation serves a different purpose: discover effective caller -requirements before input artifacts are available. Existing inspection and -execution operations should retain their established contracts. - -## Requested Operation And Selection - -Provide a read-only public operation on a configured engine. Scriptorium needs -to supply: - -- A prompt ID and optional version selector. -- An optional execution profile selector. -- Supplied template variables. -- An optional direct session ID, because it can bypass a session-ID template. - -Selection and interpretation should follow the same public semantics as an -equivalent execution request, including omitted selectors, exact identities, -variable values, and session-ID override behavior. The result should identify -the selected prompt ID/version and effective profile ID, including selection -through the prompt's default profile. - -Inspection uses the engine's configured sources. Scriptorium continues to own -application configuration discovery, CLI flags, and source selection. The API -should require neither input references or contents nor an output path. - -## Required Result Information - -Expose public values with documented meanings that Scriptorium can project into -its own versioned JSON response and readable text output: - -- Selected prompt and profile identities. -- Discoverable caller-supplied input names, including declared inputs and - references in message and applicable session-ID templates. -- Whether each input is required, optional, or conditional for the supplied - context, with an explanation where available. Preserve exact execution names. -- Declared input descriptions and content-type metadata, distinguishing absent - metadata from a known declaration. -- Caller-supplied variable requirements separately from artifact inputs, - including whether supplied values satisfy the known requirements. -- Whether discovery is complete for the supplied context, together with - machine-readable reasons and safe explanations for unresolved requirements. - -Provide predictable ordering for repeatable presentation and enough public -classification information that callers do not need to parse diagnostic prose. -Exact type names, fields, and ordering conventions remain API design choices. -Scriptorium owns its subprocess JSON schema; a Promptkit-specific JSON wire -format is not required by this request. - -The [format reference](../formats.md#inputs) currently provides input -content-type metadata, rather than an input-schema declaration. Report the -meaning of existing metadata accurately. This request does not introduce input -format enforcement, input schemas, or variable defaults. Where a supported -format provides defaults or constraints, inspection should reflect their actual -execution semantics rather than infer them from names. - -## Context-Sensitive And Incomplete Discovery - -A declared required input remains required independently of whether a template -branch references it. An input whose only requirement comes from an inactive -branch should not be classified as required for that context. - -Use supplied variables when they determine active branches or dynamically -selected names. Variable interpretation must agree with execution, including -template truthiness and presence semantics. Include requirements from an active -session-ID template; account for its bypass by a direct session ID. - -Where unavailable input contents, missing variables, dynamic names, or other -analysis limitations prevent a complete determination, return the known -requirements with explicit uncertainty. Explain conditions through public -information where possible, without revealing template text or supplied values. -A valid prompt remains valid when its requirements cannot be fully discovered. -An undeclared template input reference is not, by itself, a new definition error. - -An empty complete result establishes that no caller-supplied inputs are needed -for the inspected context. An empty incomplete result does not. Completeness -describes discovery, not satisfaction: a known required variable may still be -missing, even when its requirement has been fully identified. If its absence -also leaves input dependencies unknown, report that uncertainty separately. - -Incomplete discovery should be a successful result with explicit status, rather -than a selection error. Document the discovery boundary so Scriptorium can -distinguish a fully determined requirement set from a useful partial result. - -## Effects, Errors, And Safety - -Inspection should work without future input artifacts, generation credentials, -model access, capacity reservation, or creation of execution artifacts. Reading -configuration and prompt resources needed for inspection is expected. -Profile selection establishes identity; this operation is not a generation -readiness or credential check. - -Invalid selectors, unknown or ambiguous selections, unreadable necessary -resources, and invalid definitions that prevent inspection should produce -publicly classifiable errors consistent with existing selection semantics. -Scriptorium needs stable categories or error identities and safe descriptions -to produce structured CLI failures. It should not need to match error-message -text or access implementation packages. - -Results and consumer-facing diagnostics should exclude credential values, -input payloads, supplied variable values, and rendered or raw prompt contents. -Identifiers and declared descriptions remain useful operator-facing metadata. - -For a complete result, the reported requirements should agree with execution -for the same effective configuration and supplied context. An incomplete -result should accurately describe what is known and what remains uncertain. -Neither result guarantees generation success or validates future input bytes. -Later configuration or prompt-resource changes may change a later execution's -requirements. - -## Consumer Acceptance Scenarios - -1. A required `transcript` input is reported with its exact name, allowing - Scriptorium's caller to detect a binding named `session_transcript` instead. -2. A literal template reference to an undeclared input is discoverable without - requiring that input's contents or rejecting the reference as undeclared. -3. A declared optional glossary remains optional when no active template use - makes it required. A declared required input remains required in an inactive - template branch. -4. Supplied variables determine conditional requirements and dynamic names - where discoverable. Content-dependent or otherwise unresolved cases return - known entries with explicit incompleteness. -5. Changing the prompt version or profile reports the corresponding selected - identities. Omitted selections follow execution's defaults and ambiguity - rules. -6. Session-template dependencies are reported when applicable; a direct session - ID removes requirements introduced solely by the bypassed template. -7. Missing caller variables are reported separately from missing artifact - bindings, without returning supplied values or prompt contents. -8. Inspection succeeds before artifacts exist and without credentials or - provider access, assuming necessary configuration and resources are valid - and readable. -9. Unknown prompts, ambiguous versions, and invalid profile selections yield - classifiable failures; analysis uncertainty yields an incomplete result. -10. A genuinely input-free context is distinguishable from an incomplete - discovery that happened to find no input names. - -## Scope And Ownership - -Promptkit supplies the public requirement-discovery semantics. Scriptorium owns -CLI exposure, text and JSON presentation, stream handling, and exit statuses. -Narratio owns its source catalog, bindings, pipeline ordering, dependency graph, -and session availability checks. - -No Narratio-specific identifiers or pipeline concepts are needed in Promptkit's -API. A graph renderer, HTTP endpoint, batch interface, changes to generation -behavior, and execution-resource preflight are outside this request. This is a -consumer capability request, not an implementation plan.