Files
promptkit/docs/roadmap/implementation.md

695 lines
28 KiB
Markdown

# Prompt-Definition Inspection Implementation Plan
**Status:** Complete.
## Purpose
This document is the decision-complete implementation plan for
[prompt-definition inspection](prompt-inspection.md). It is written for a
coding agent that will implement each stage in order.
The feature roadmap owns the motivation, consumer workflow, policy choices,
compatibility requirements, non-goals, and target end state. This document
owns the concrete design, file-level changes, implementation sequence, test
ownership, documentation work, validation commands, and completion gates.
## Implementation Rules
- Complete the stages in order. Keep the repository compiling and the focused
tests passing at every stage boundary.
- Preserve unrelated working-tree changes. In particular, retain the accepted
feature roadmap and the corresponding future-catalog and Weatherreporter
wishlist edits that may already be uncommitted.
- Follow every policy under `docs/policy/`, the task-specific reading guide in
`docs/development.md`, and the accepted behavior in
`prompt-inspection.md`.
- Keep the supported API in the root `promptkit` package. Prompt repositories
and internal prompt definitions remain below Go's `internal/` boundary.
- Reuse the exact prompt source, ID/version selection, definition
normalization, referenced-content loading, and prompt hashing used by
`Prepare`, `PrepareExecution`, and `Run`. Do not create a second parser,
source abstraction, or hash algorithm.
- Preserve the observable validation order, error identities, and results of
existing preparation and execution methods.
- The new inspection operation must not require a profile, backend, request
override, credential, artifact, schema source, renderer, validator,
capacity admission, output repairer, or model client.
- Treat structural validation as the existing prompt-repository contract.
Do not add template parsing, input media-type enforcement, schema loading,
schema compilation, profile validation, source enumeration, or provider
connectivity checks.
- Return only the metadata accepted by the feature roadmap. Do not expose raw
YAML, prompt descriptions, source paths, message or session templates,
cache-control declarations, rendered content, or schema bodies.
- Return caller-owned public values. The input-definition slice must not alias
repository, runner, engine, or another result's state.
- Keep tests lean and behavior-focused. Reuse existing prompt-repository and
preparation coverage rather than duplicating their complete format and
source matrices.
- Update exact contracts in GoDoc with the exported declarations. Update
current-state consumer, format, and internal documentation only after the
corresponding code exists.
- Do not add release notes, change a module version, create a release, commit,
or tag as part of this work.
## Fixed Design
### Public API
Add these root-package values in `types.go` immediately after
`ProfileInspection`:
```go
type PromptInputDefinition struct {
Name string
Required bool
ContentType string
Description string
}
type PromptInspection struct {
PromptID string
PromptVersion string
PromptHash string
DefaultProfileID string
Inputs []PromptInputDefinition
OutputContract OutputContract
}
```
Add this method to `Engine` in `engine.go`, immediately before
`InspectProfile`:
```go
func (e *Engine) InspectPrompt(
ctx context.Context,
promptID string,
promptVersion string,
) (*PromptInspection, error)
```
Do not accept a `RunRequest`, inputs, variables, profile ID, output override,
or options on this method. Exact inspection of one prompt ID and optional
version is the complete operation.
`PromptInputDefinition` and `PromptInspection` have no stable JSON contract.
Do not add JSON tags, `MarshalJSON`, `UnmarshalJSON`, or custom string
representations. Their nested `OutputContract` continues to use its existing
stable JSON representation when encoded independently, but that does not make
the enclosing inspection value stable.
The exact type and method GoDoc must establish:
- `promptID` is required and whitespace-only is invalid;
- nonblank `promptID` and `promptVersion` values are passed to ordinary prompt
selection unchanged rather than trimmed or otherwise canonicalized;
- lookup is case-sensitive and exact;
- an empty version succeeds only when exactly one definition has the selected
ID, while a nonempty version selects one exact ID/version pair;
- the engine's configured prompt source and existing source-option selection
are used without merging, fallback, or enumeration;
- a successful result proves that the selected definition and referenced
message content files were structurally loaded through the ordinary prompt
repository;
- input metadata is returned in definition order and includes name, required
status, content type, and description;
- `DefaultProfileID` is declared metadata only and is not resolved;
- `OutputContract` is the normalized declared contract, not a request-level
effective override;
- a JSON Schema path is returned when declared, but the schema and its
references are not loaded or compiled;
- `PromptHash` is the same opaque equality value used by
`PreparedRun.PromptHash` for the same selected definition and observed
source state;
- hash spelling, length, encoding, algorithm, and security properties are not
public contracts;
- prompt bodies, templates, source paths, schemas, rendered messages, and
execution settings are not returned;
- no profile, credential, artifact, rendering, schema, validation, capacity,
provider, or model-generation work is performed;
- the returned result and input slice are caller-owned;
- filesystem-backed inspection is a point-in-time lookup and does not freeze
a definition for later execution;
- a nil engine matches `ErrInvalidConfig`;
- a blank prompt ID matches `ErrInvalidRequest`;
- an absent exact ID or version matches `ErrPromptNotFound` and not
`ErrPromptLoad`;
- malformed, unreadable, duplicate, ambiguous, referenced-content, or hashing
failures match `ErrPromptLoad`;
- cancellation observed during lookup matches `ErrPromptLoad` while
preserving the context error through `errors.Is`; and
- the method returns no partial result on error.
Update the `Engine` type GoDoc to include `InspectPrompt` among operations safe
for concurrent calls. Do not imply that mutable injected filesystems or other
collaborators become safe when their own contracts do not provide that
guarantee.
Update `doc.go` in the same stage:
- include `Engine.InspectPrompt` in the package operation list;
- include prompt inspection in the concurrency and caller-ownership summary;
- list `PromptInputDefinition` and `PromptInspection` among construction and
inspection values without stable JSON representations; and
- retain the existing statement that exposed hashes are opaque.
Do not add either new value to the stable JSON list.
### Internal Result
Add this internal value to `internal/domain/domain.go` near
`PromptDefinition` and `PromptInput`:
```go
// PromptInspection is the resolved result of exact prompt inspection.
type PromptInspection struct {
PromptID string
PromptVersion string
PromptHash string
DefaultProfileID string
Inputs []PromptInput
OutputContract OutputContract
}
```
Do not add JSON or YAML tags. The internal value is a use-case result, not a
source format or persistence contract.
The use-case operation must copy the definition's `Inputs` slice before
putting it on this result. `PromptInput` currently contains only scalar fields,
so a new slice with value copies is sufficient.
### Shared Prompt Selection And Hashing
Add `internal/usecase/prompt_inspection.go`. Define one private selection
value:
```go
type resolvedPromptDefinition struct {
definition *domain.PromptDefinition
hash string
}
```
Add a private runner helper:
```go
func (r *Runner) resolvePromptDefinition(
ctx context.Context,
promptID string,
promptVersion string,
) (*resolvedPromptDefinition, error)
```
The helper must perform these operations in order:
1. reject a blank or whitespace-only `promptID` with `ErrInvalidRequest`;
2. pass every nonblank ID and the version to the repository unchanged;
3. reject a nil runner or nil prompt repository with `ErrPromptLoad` rather
than panicking;
4. call `promptdef.Repository.GetPromptDefinition` exactly once;
5. wrap every repository error as `ErrPromptLoad` with `%w` while preserving
the repository error, including `promptdef.ErrPromptDefinitionNotFound`;
6. reject a nil definition returned without an error as `ErrPromptLoad`;
7. calculate the equality value with the existing
`hashPromptDefinition` function exactly once;
8. classify a hash failure as `ErrPromptLoad` using the existing preparation
wording and behavior; and
9. return the repository-owned definition for internal per-call use and the
hash.
Do not trim a nonblank ID or version, clone the complete prompt definition,
parse its templates, inspect multiple definitions, load schemas, or resolve
the default profile.
Refactor the prompt-loading and hashing block in
`Runner.resolvePreparation` to call `resolvePromptDefinition`. Retain the
existing early blank-ID check before direct session normalization so a request
with both a blank prompt ID and an invalid direct session preserves its
current error priority. The shared helper may defensively repeat the blank-ID
check.
After the helper returns, ordinary preparation uses
`selection.definition` and `selection.hash` exactly where it currently uses
the repository result and prompt hash. Leave direct-session normalization,
profile selection, target resolution, credentials, output-contract
resolution, artifacts, schemas, rendering, admission, generation, and all
later ordering unchanged.
This refactor must preserve:
- current prompt ID and version selection behavior;
- existing public not-found versus prompt-load classification;
- prompt default-profile selection;
- prompt hashing before any request-specific definition copy or session
template clearing;
- `PreparedRun.PromptHash`, `RunResult.PromptHash`, and prepared-execution
behavior; and
- all current `Prepare`, `PrepareExecution`, `Run`, and `RunPrepared` results
and failure ordering.
Do not change `hashPromptDefinition` or introduce a second equality
calculation. Its current serialization and SHA-256 implementation remain an
internal mechanism behind the opaque public value.
### Internal Inspection Operation
Add this method in `internal/usecase/prompt_inspection.go`:
```go
func (r *Runner) InspectPrompt(
ctx context.Context,
promptID string,
promptVersion string,
) (*domain.PromptInspection, error)
```
It performs these operations:
1. reject a blank or whitespace-only ID as `ErrInvalidRequest`;
2. if the supplied context is already canceled, return an error wrapping both
`ErrPromptLoad` and `ctx.Err()` without consulting the repository;
3. call `resolvePromptDefinition` with the original ID and version;
4. allocate and copy the selected definition's input slice in its existing
order;
5. return the selected definition's normalized ID, version, default profile,
declared `Validation` contract, copied inputs, and shared prompt hash.
The returned `OutputContract` comes directly from the loaded definition's
normalized `Validation` value. Do not call `resolveOutputContract`, because no
request override participates in inspection.
Do not call profile or backend repositories, `resolveProfileSelection`,
`resolveExecutionTarget`, credential checks, schema loaders, artifact readers,
renderers, validators, capacity admission, output repair, or the model client.
Do not return the definition itself.
The operation returns no partial result. Repository cancellation that occurs
after lookup begins remains wrapped by `ErrPromptLoad` through the shared
helper while preserving the underlying context error where the repository
does so.
### Root Conversion
Add this conversion in `convert.go` near
`fromDomainProfileInspection`:
```go
func fromDomainPromptInspection(
inspection *domain.PromptInspection,
) *PromptInspection
```
The conversion must:
- return `nil` for a nil internal value;
- copy every scalar identity, hash, and default-profile field;
- allocate a new `[]PromptInputDefinition` in the same order and copy every
name, required flag, content type, and description;
- convert the contract through the existing
`fromDomainOutputContract`; and
- return a non-aliased root value.
Do not expose internal `PromptInput` or `PromptDefinition` types at the root
boundary. Do not use serialization as a copying mechanism.
### Root Facade And Error Mapping
`Engine.InspectPrompt` follows the existing inspection facade pattern:
1. reject a nil engine or nil runner with `ErrInvalidConfig`;
2. pass the context, ID, and version directly to
`Runner.InspectPrompt`;
3. map internal failures through the existing `mapPublicError`; and
4. convert a successful result with `fromDomainPromptInspection`.
No request conversion is needed. Do not add a public sentinel or typed error.
The existing error mapping already has the required ordering:
- an underlying `promptdef.ErrPromptDefinitionNotFound` maps to
`ErrPromptNotFound` before the enclosing use-case `ErrPromptLoad` is
considered;
- other `usecase.ErrPromptLoad` failures map to `ErrPromptLoad`; and
- `usecase.ErrInvalidRequest` maps to `ErrInvalidRequest`.
Do not reorder or otherwise change `errors.go` unless a focused public test
proves the current mapping fails a required identity. Preserve underlying
repository and context errors through `errors.Is`.
### Ownership, Consistency, And Concurrency
The internal result has a copied input slice, and the root conversion creates
another public slice. A consumer may mutate the result and its inputs without
affecting:
- the prompt repository or selected definition;
- the engine;
- a later `InspectPrompt` call;
- `Prepare`, prepared execution, or `Run`; or
- another result already returned to a caller.
`OutputContract` and each input element contain scalar values, so no deeper
mutable tree exists in the accepted result shape.
No new mutable engine state, source cache, global registry, lock, or goroutine
is needed. Concurrency safety follows from existing repository contracts and
per-call result allocation. Inspection does not freeze mutable source state;
prepared execution remains the exact snapshot-to-execution workflow.
### Test Ownership
Add focused internal tests in
`internal/usecase/prompt_inspection_test.go`. Reuse package fakes where doing
so remains clearer than adding a new fixture, or define one small counting
prompt repository local to this test file.
The internal tests own:
- blank-ID rejection without repository access;
- a pre-canceled context matching both `ErrPromptLoad` and the context error
without repository access;
- exactly one repository lookup with the original nonblank ID and version;
- a successful result containing normalized identity, hash, default profile,
complete ordered input metadata, and the declared output contract;
- successful operation with every non-prompt runner collaborator nil;
- copied inputs across repeated inspections;
- missing-prompt wrapping that retains
`promptdef.ErrPromptDefinitionNotFound`;
- nil repository and nil returned definition defenses; and
- representative preservation of ordinary preparation after it is switched
to the shared selection-and-hash helper.
Combine these into a small number of readable behavioral tests. Prompt
repository tests remain the owners of YAML discovery, strict decoding,
normalization, content-file containment and reading, duplicate selection, and
ID/version matrices. Existing preparation tests remain the owners of session,
profile, artifact, schema, rendering, credential, and execution ordering.
Add external-package public contract tests in `public_contract_test.go`. They
own:
- the exported method and result shape through normal Go use;
- successful exact-version inspection from an `fs.FS` prompt source with
multiple versions;
- normalized input and output metadata, including a declared JSON Schema path,
without configuring or loading a schema source;
- referenced `content_file` loading without returning its body;
- no default-profile resolution, demonstrated by a nonexistent declared
profile;
- no model invocation, using an existing deterministic fake client where
useful;
- nil-engine, blank-ID, missing exact version, ambiguous omitted version,
malformed definition or referenced-content, and pre-canceled public error
identities;
- that not-found does not also match `ErrPromptLoad`;
- caller ownership across repeated inspection and later preparation; and
- equality between `PromptInspection.PromptHash` and
`PreparedRun.PromptHash` for the same prompt and observed source state.
Use a simple executable prompt with an in-memory profile for the hash
equivalence test. Use a separate JSON Schema declaration for the
schema-independent inspection test so `Prepare` is not needed there.
Do not add JSON golden or round-trip tests because the new inspection values
deliberately have no stable JSON contract. Do not duplicate the full
prompt-repository parser, source-option precedence, schema-validation,
preparation, or public error suites at the root layer.
Existing tests that must continue passing without weakened assertions include:
- prompt-definition filesystem and `fs.FS` repository tests;
- prompt-source option replacement tests;
- `Prepare` and `Run` prompt selection and hashing tests;
- direct-session prompt-hash invariance tests;
- prepared-execution frozen-source and details tests;
- public error mapping tests; and
- profile inspection tests.
### Documentation Ownership
After the implementation and public tests pass, update current-state
documentation:
- `docs/consumers/pkg-promptkit.md`: add a concise task-oriented section after
engine construction showing `Engine.InspectPrompt`, explaining how a
consumer can validate declared inputs and output workflow before preparing,
and distinguishing it from `InspectProfile`, `Prepare`, and prepared
execution. Link to GoDoc for exact fields and errors.
- `docs/formats.md`: note that exact prompt inspection uses the same prompt
source, strict decoding, content-file resolution, and ID/version selection
described by the format reference. Do not redefine the exported method.
- `docs/internal/runner.md`: add a shared prompt-selection boundary for
inspection and preparation, including shared hashing, and explain that
inspection stops before every execution-dependent collaborator.
- `docs/internal/sources.md`: record that exact prompt inspection performs one
point-in-time prompt lookup, validates referenced message content through
the repository, and does not parse templates or read profile, input, or
schema sources.
- `docs/internal/overview.md`: add prompt inspection to the existing root
facade and `internal/usecase` responsibility descriptions. Do not add a new
component or package row.
- `docs/roadmap/future.md`: remove the statement that prompt inspection is in
active planning and leave structured capacity errors intact.
- `docs/roadmap/weatherreporter-promptkit-wishlist.md`: change the prompt
inspection disposition from accepted planning to implemented behavior,
link to durable consumer guidance or GoDoc, and remove or condense proposed
API detail that would compete with the implemented declarations.
- `docs/roadmap/prompt-inspection.md`: change its status to `Complete` only
after code, tests, current-state documentation, and full validation are
complete.
- `docs/roadmap/implementation.md`: change its status to `Complete` only after
every completion gate in this plan is satisfied.
The accepted feature roadmap already describes only purpose, scope, policy,
and target end state; it contains no stage sequence to remove during this
planning pass.
Do not update release guidance in this feature implementation. A later release
pass decides whether the additive API warrants a supplemental release
document.
## Stage 1: Share Prompt Selection And Implement Internal Inspection
### Objective
Add the internal inspection result and operation, share exact prompt loading
and hashing with ordinary preparation, and prove the internal behavior without
publishing the root API yet.
### Implementation Prompt
Implement only Stage 1 of
`docs/roadmap/implementation.md`. Read the complete feature roadmap,
implementation rules, and fixed design above before editing.
1. Add `domain.PromptInspection` to `internal/domain/domain.go` without source
or serialization tags.
2. Add `internal/usecase/prompt_inspection.go` with
`resolvedPromptDefinition`, `resolvePromptDefinition`, and
`Runner.InspectPrompt` exactly as specified.
3. Refactor `Runner.resolvePreparation` in `internal/usecase/runner.go` to use
the shared prompt selection and hash while preserving existing validation
and execution ordering.
4. Add lean behavioral tests in
`internal/usecase/prompt_inspection_test.go`.
5. Run the focused validation below and repair regressions before ending the
stage.
Do not add the root public type or method, update current-state documentation,
or alter prompt formats, template behavior, profile selection, schema
handling, capacity, validation, model-client, or provider behavior in this
stage.
### Focused Validation
Run:
```sh
gofmt -w internal/domain/domain.go \
internal/usecase/prompt_inspection.go \
internal/usecase/prompt_inspection_test.go \
internal/usecase/runner.go
go test ./internal/usecase ./internal/promptdef
go test ./internal/usecase -run \
'TestRunner(InspectPrompt|Prepare|Run|PrepareExecution|RunPrepared)'
go vet ./internal/usecase ./internal/promptdef
```
If actual existing test names do not match the focused expression, run the
smallest truthful package or expression that covers the listed behavior
rather than weakening or skipping assertions.
### Completion Gate
Stage 1 is complete only when:
- inspection loads and hashes one exact prompt through the configured
repository;
- prompt selection and hashing are shared with ordinary preparation;
- nonblank ID and version values reach the repository unchanged;
- successful inspection needs no non-prompt runner collaborator;
- inputs are copied and no definition or content body escapes;
- cancellation and load failures preserve the required internal identities;
- ordinary preparation and execution behavior remains unchanged; and
- no root public API or current-state documentation claims the feature yet.
## Stage 2: Publish The Root Facade And Public Contract
### Objective
Expose the minimal caller-owned inspection API through `Engine`, preserve
public error and JSON compatibility, and protect the consumer-visible
contract.
### Implementation Prompt
Implement only Stage 2 of
`docs/roadmap/implementation.md` after Stage 1 satisfies its completion gate.
Re-read the fixed public API, conversion, error, ownership, and test sections
before editing.
1. Add `PromptInputDefinition` and `PromptInspection` with exact GoDoc to
`types.go` immediately after `ProfileInspection`. Do not add JSON tags.
2. Add `fromDomainPromptInspection` to `convert.go` using a newly allocated
input slice and the existing output-contract converter.
3. Add `Engine.InspectPrompt` and exact GoDoc to `engine.go` immediately before
`InspectProfile`.
4. Update `Engine` GoDoc to include concurrent prompt inspection.
5. Update package GoDoc in `doc.go` for operation discovery, concurrency,
ownership, opaque hashes, and non-stable JSON classification.
6. Add compact external-package contract coverage in
`public_contract_test.go`, reusing existing fixtures and fakes where they
remain clear.
7. Confirm that existing `errors.go` mapping meets the plan; do not change it
unless a required public identity test fails for a genuine mapping reason.
8. Run the focused validation below and repair regressions before ending the
stage.
Do not add enumeration, source validation across a corpus, request overrides,
template or schema output, profile resolution, JSON stability, caching, new
errors, or current-state prose documentation in this stage.
### Focused Validation
Run:
```sh
gofmt -w doc.go types.go convert.go engine.go public_contract_test.go
go test .
go test ./internal/usecase ./internal/promptdef
go test . -run \
'Test(InspectPrompt|.*Prompt.*PublicError|.*Prompt.*Contract)'
go vet .
go build .
```
If the repository's actual focused test names differ, use the implemented test
names rather than weakening or skipping the intended assertions.
### Completion Gate
Stage 2 is complete only when:
- a consumer can call `Engine.InspectPrompt` through the root package;
- exact ID/version selection matches ordinary preparation;
- the result contains only accepted identity, input, default-profile,
output-contract, and opaque equality metadata;
- no prompt body, schema body, profile, credential, or execution setting is
exposed or resolved;
- nil, blank, missing, malformed, ambiguous, referenced-content, and
cancellation errors have the required public identities;
- not-found remains distinct from prompt-load failure;
- `PromptHash` matches ordinary preparation for the same source state;
- repeated calls and caller mutation cannot alter engine-owned state;
- no stable JSON or new error contract was introduced; and
- existing public preparation, execution, profile-inspection, and stable JSON
tests remain unchanged and passing.
## Stage 3: Update Documentation And Validate The Repository
### Objective
Make implemented prompt inspection discoverable in its canonical
documentation, reconcile temporary roadmap state, and complete full repository
validation.
### Implementation Prompt
Implement only Stage 3 of
`docs/roadmap/implementation.md` after Stages 1 and 2 satisfy their completion
gates.
1. Update `docs/consumers/pkg-promptkit.md`, `docs/formats.md`,
`docs/internal/runner.md`, `docs/internal/sources.md`, and
`docs/internal/overview.md` according to the documentation ownership
section above.
2. Update the future catalog and Weatherreporter wishlist disposition so they
no longer describe prompt inspection as merely accepted work.
3. Check every changed Markdown link and confirm its file and heading target.
4. Run the full validation sequence below.
5. Only after every check passes, set the feature roadmap and this
implementation plan to `**Status:** Complete.`
6. Re-run `git diff --check` after the status edits.
Do not add release notes, a new example program, a new public package, or a
duplicate API reference. Keep detailed contracts in GoDoc and task-oriented
usage in the consumer guide.
### Full Validation
Run from the repository root:
```sh
gofmt -w internal/domain/domain.go \
internal/usecase/prompt_inspection.go \
internal/usecase/prompt_inspection_test.go \
internal/usecase/runner.go \
doc.go types.go convert.go engine.go public_contract_test.go
gofmt -l $(git ls-files '*.go')
go test ./...
go test -race ./...
go vet ./...
go build ./...
go run ./examples/go-library/prepare
git diff --check
git status --short
```
The `gofmt -l` command must print no paths. The maintained example must remain
offline and must not require a real credential or provider.
Inspect the final diff and confirm:
- only files required by this feature and pre-existing user changes are
present;
- no credentials, private source content, workspace files, local
replacements, generated binaries, or unrelated formatting changes were
added;
- the public declarations and GoDoc own exact API behavior;
- current-state documentation describes only implemented behavior and links
to canonical owners;
- roadmap documents contain scope or completion status rather than a
duplicate current API reference; and
- no release, commit, or tag was created.
### Completion Gate
The implementation is complete only when:
- every Stage 1 and Stage 2 gate remains satisfied;
- the complete ordinary and race-enabled suites pass;
- vet, build, formatting, the maintained offline example, Markdown links, and
whitespace checks pass;
- consumer, format, internal, future, and wishlist documentation are
consistent with the implemented boundary;
- both roadmap statuses are `Complete`;
- the working tree contains no unintended files or changes; and
- the repository is ready for maintainer review without a commit or release
having been created by this plan.
## Open Questions
None. The accepted feature roadmap and the fixed decisions above fully specify
the implementation boundary.