446 lines
25 KiB
Markdown
446 lines
25 KiB
Markdown
# Effective Prompt Requirement Inspection Implementation Plan
|
|
|
|
Implement the [feature roadmap](inspection.md) in order. Each numbered stage is
|
|
scoped for one Luna coding-agent prompt. This plan replaces the earlier
|
|
interpreter-style analysis scope with simple enumeration and conservative
|
|
classification. The completed ReleaseKit plan remains available in Git history.
|
|
|
|
## Baseline And Working Rules
|
|
|
|
- Planning baseline: `e459c7d`. The inspection API is not implemented.
|
|
`InspectPrompt` reports declarations, `Prepare` needs artifacts, and the
|
|
configured engine already captures compiled templates.
|
|
- Start with [development](../development.md), then the
|
|
[agent](../policy/agents.md), [architecture](../policy/architecture.md),
|
|
[testing](../policy/testing.md), and [documentation](../policy/documentation.md)
|
|
policies. Reuse unchanged policies already read.
|
|
- Follow [ADR 0001](../adr/0001-configuration-boundary.md): interpret only final
|
|
effective configuration at the ConfigKit handoff. Operator provenance is
|
|
separate; inspection does not examine raw sources or intermediate values.
|
|
- Preserve unrelated work and commit completed, verified units under the agent
|
|
policy. Keep implemented internal documentation accurate as stages land.
|
|
- Preserve existing construction, inspection, preparation, rendering, and
|
|
execution contracts. An inspection limitation is not a definition error.
|
|
- Use captured parsed syntax, direct traversal, and maps keyed by exact name.
|
|
No general evaluator, assignment-value propagation, branch-environment joins,
|
|
possible-name sets, branch combinations, analysis budget, or global cache is
|
|
needed. Do not add caller limits, defaults, input schemas, or authoring bans.
|
|
- The root package owns public declarations/conversion. `internal/prompt`
|
|
owns discovery, `internal/usecase` coordinates selection, and
|
|
`internal/configuration` supplies captured syntax/metadata. Runtime/domain
|
|
values remain independent of ConfigKit.
|
|
- No new dependency, upstream ConfigKit change, provider access, CLI/HTTP
|
|
feature, application wire format, or publication is part of these stages.
|
|
|
|
## Concrete Public Values
|
|
|
|
Add root `requirements.go` in Stage 5:
|
|
|
|
```go
|
|
func (e *Engine) InspectRequirements(
|
|
ctx context.Context, req RequirementInspectionRequest,
|
|
) (*RequirementInspection, error)
|
|
|
|
type RequirementInspectionRequest struct {
|
|
PromptID string
|
|
PromptVersion string
|
|
ProfileID string
|
|
Vars map[string]string
|
|
SessionID string
|
|
}
|
|
|
|
type RequirementKind string // values: required, conditional, optional
|
|
|
|
type InputRequirement struct {
|
|
Name string
|
|
Kind RequirementKind
|
|
Declared bool
|
|
ContentType *string
|
|
Description *string
|
|
Explanation string
|
|
}
|
|
|
|
type VariableRequirement struct {
|
|
Name string
|
|
Kind RequirementKind
|
|
Supplied bool
|
|
Explanation string
|
|
}
|
|
|
|
type RequirementIssueCode string
|
|
|
|
type RequirementIssue struct {
|
|
Code RequirementIssueCode
|
|
InputName string
|
|
VariableName string
|
|
Description string
|
|
}
|
|
|
|
type RequirementInspection struct {
|
|
PromptID string
|
|
PromptVersion string
|
|
ProfileID string
|
|
Inputs []InputRequirement
|
|
Variables []VariableRequirement
|
|
Complete bool
|
|
Issues []RequirementIssue
|
|
}
|
|
```
|
|
|
|
Use root-owned values and internal counterparts in
|
|
`internal/domain/requirements.go`; do not expose internal aliases.
|
|
Kind constants are `RequirementRequired`, `RequirementConditional`, and
|
|
`RequirementOptional`. Retain four useful issue categories:
|
|
|
|
| Constant | Value | Discovery limitation |
|
|
| --- | --- | --- |
|
|
| `RequirementIssueMissingVariable` | `missing_variable` | An absent value prevents determining a dependency decision/name. |
|
|
| `RequirementIssueInputContent` | `input_content` | Unavailable artifact content affects dependencies. |
|
|
| `RequirementIssueDynamicInputName` | `dynamic_input_name` | A referenced input name cannot be determined. |
|
|
| `RequirementIssueUnsupportedConstruct` | `unsupported_construct` | Analysis does not cover a dependency-relevant expression or scope. |
|
|
|
|
An empty associated name means that no particular entry was identified.
|
|
Descriptions and explanations are safe human-readable summaries, not strings
|
|
callers must parse. There is no expression tree, source offset, JSON tag/schema,
|
|
or stable diagnostic wording contract.
|
|
|
|
### Classification And Ownership
|
|
|
|
- Seed declared inputs from final effective definitions. Required declarations
|
|
stay required independently of syntax. Others start optional.
|
|
- Collect occurrences with definite, possible, or inactive reachability.
|
|
Definite helper/direct required-variable accesses are required; possible ones
|
|
are conditional; recognizable inactive references are optional. A lookup
|
|
that execution permits to be absent remains optional unless another access
|
|
establishes a stronger requirement.
|
|
- Aggregate exact names with required > conditional > optional. No proof that
|
|
a name in both unresolved branches is required is necessary: conditional is
|
|
acceptable until an independent definite access/declaration establishes it.
|
|
- Include recognizable inactive inputs and caller variables in both inventories.
|
|
Give inactive-only entries an `Explanation`, independent of declared input
|
|
`Description`. Any applicable occurrence, including an optional lookup,
|
|
removes the inactive-only explanation; required/conditional uses also
|
|
override optional classification. Inactivity is not an uncertainty issue.
|
|
- Input metadata pointers distinguish absence/null from an explicit string,
|
|
including an empty string. `Declared` distinguishes metadata ownership from
|
|
a helper-only reference. Do not infer new input formats.
|
|
- `Supplied` means presence of the exact caller key, including empty strings.
|
|
It is independent of classification and completeness; do not echo values.
|
|
- Sort input and variable entries lexically by exact name. Deduplicate issues
|
|
by code/input-name/variable-name and sort by that tuple. Do not promise a
|
|
nil-versus-empty slice distinction.
|
|
- Return independent slices and metadata pointers, copy request variables at
|
|
the facade as for `RunRequest`, and keep captured syntax immutable. Retain
|
|
callers' existing responsibility for concurrently mutating requests.
|
|
|
|
### Deliberately Small Recognition Scope
|
|
|
|
Use the standard template parse tree and `template.IsTrue` for known values.
|
|
Only determine a value when needed for a recognized name or simple condition.
|
|
|
|
| Syntax | Initial handling |
|
|
| --- | --- |
|
|
| Literal `input "name"` | Collect the exact name without reading its body. Parenthesized literal wrappers and a literal piped directly to `input` are equivalent. |
|
|
| Direct root `.name` or `$.name` | Collect a required caller key when active; supplied strings are known, absence is unresolved rather than an inspection error. |
|
|
| `index` of the root map with one literal string key | Collect an optional lookup; its value is supplied text or the string zero value if absent, following execution. |
|
|
| `input .name`, `input $.name`, or `input (index . "name")` | Resolve a directly available string as an input identifier on applicable paths. |
|
|
| One simple value piped directly to `input`, or parenthesized simple expressions | Recognize the same bounded forms without evaluating arbitrary pipelines. |
|
|
| String/Boolean/nil literals and simple direct values | Use known values for conditions; no coercion of caller strings. |
|
|
| `not` of a simple predicate; two-string `eq`/`ne` | Recognize negation and ordinary equality/inequality without adding the full comparison family. |
|
|
| `if`/`else`, `with` | Classify branch occurrences using a recognized condition; unresolved branches are possible, not guesses. |
|
|
| Locals, changed dot, and root binding | Retain scope identity sufficient to distinguish root-map accesses from local/string/unknown values; no local-value propagation is required. |
|
|
| Other pipelines/functions, `and`/`or`, assignments, ranges, invoked named templates, or complex names | Collect safely identifiable references conservatively and record dependency-relevant limitations; do not interpret their execution. |
|
|
|
|
An unsupported expression does not automatically make every argument a definite
|
|
access: short-circuiting or unknown scopes can make references merely possible.
|
|
Keep complex forms outside complete analysis instead of emulating their rules.
|
|
Do not interpret `$local` as a caller key or `.field` on a string/unknown dot as
|
|
a root lookup. Explicit root accesses remain recognizable only while the root
|
|
binding's identity is known. Unsupported changes to a binding make its identity
|
|
unknown where affected; no assignment-value or cross-branch reconstruction is
|
|
needed.
|
|
|
|
Walk known inactive subtrees for syntactically identifiable literal input names
|
|
and caller keys. Do not evaluate skipped expressions, resolve skipped dynamic
|
|
input names, or produce uncertainty solely from skipped constructs. Names
|
|
hidden behind unsupported scopes need not be invented. Exclude bypassed session
|
|
templates and never-invoked definitions entirely.
|
|
|
|
### Simple Algorithm And Completeness
|
|
|
|
1. Seed declaration entries and visit each selected template.
|
|
2. Maintain only reachability and the scope identity needed to recognize caller
|
|
keys; collect names and minimal classification evidence in indexed summaries.
|
|
3. For simple conditions, visit both branches with definite/possible/inactive
|
|
reachability as appropriate. A possible parent never becomes definite just
|
|
because an inner predicate is known true; a known inactive parent stays
|
|
inactive. No branch environment or assignment-value joins are needed.
|
|
4. Aggregate/finalize names, metadata, presence, explanations, and limitations.
|
|
|
|
Use shared traversal helpers for active/possible/inactive syntax rather than
|
|
separate interpreters. Do not enumerate execution combinations. Prefer a direct
|
|
pass through syntax and final aggregation; no special machinery is warranted
|
|
by ordinary prompt sizes.
|
|
|
|
`Complete` means that the supported analysis established the requirement
|
|
inventory/classifications. Conservative false is acceptable for difficult cases.
|
|
Missing direct text-substitution values alone do not make discovery incomplete.
|
|
Unresolved branches with dependency-bearing references return conditional
|
|
entries and incomplete status; no proof of identical branches is required.
|
|
Unknown pure text without a dependency effect need not change completeness.
|
|
|
|
An unsupported call that may hide dependencies or an unresolved active input
|
|
name makes discovery incomplete. Preserve recognized names; do not create a
|
|
wildcard entry. Maintain simple flags for possible unknown input/caller-key
|
|
effects, promoting affected optional entries to conditional at finalization,
|
|
rather than repeatedly rescanning the inventory. Known required entries remain
|
|
required. Clear inactive-only explanations where those unknown effects undermine
|
|
the conclusion.
|
|
|
|
A complete empty or all-optional input inventory needs no artifact bindings.
|
|
An incomplete inventory does not prove that no others are needed. A complete
|
|
result is not a rendering/readiness/provider guarantee.
|
|
|
|
## Stage 1 — Capture Effective Metadata And Template Snapshots
|
|
|
|
**Status: Complete (2026-10-08).** Effective input metadata presence and
|
|
construction-time templates are available through an internal snapshot without
|
|
changing prompt hashes or the existing inspection shape.
|
|
|
|
**Outcome:** The analyzer can obtain existing compiled templates and declaration
|
|
metadata without artifact reads or changing old inspection/execution behavior.
|
|
|
|
**Read:** `internal/configuration/load.go`, `prompts.go`,
|
|
`internal/prompt/template.go`, `internal/domain/domain.go`,
|
|
`internal/promptdef/hash.go`, and the internal source document.
|
|
|
|
**Work:**
|
|
|
|
1. Retain content-type/description presence alongside configuration
|
|
`promptEntry`, rather than changing the hashed prompt definition or old
|
|
public inspection shape.
|
|
2. Capture effective per-input presence from `Resolution.InputInfo` during the
|
|
existing Resolve callback before prompt normalization. Use final
|
|
prompt-ID/version/input-index paths, following the existing presence-capture
|
|
pattern. Materialize values from the normalized final declaration.
|
|
3. Add `prompt.RequirementSnapshot` with captured templates and metadata, and
|
|
narrow internal `GetRequirementSnapshot(id, version string)
|
|
(RequirementSnapshot, bool)` access on the captured configuration adapter.
|
|
Use selected exact keys, without reparsing or reopening resources.
|
|
4. Preserve renderer/constructor paths and update internal source documentation.
|
|
|
|
**Verify:** Focused final-value metadata tests for absent/null, empty, and
|
|
nonempty values, plus equivalent-definition hashes and old inspection behavior.
|
|
Use actual configuration capture rather than another source-loading matrix.
|
|
Run `go test ./internal/configuration ./internal/promptdef ./internal/prompt`;
|
|
commit.
|
|
|
|
## Stage 2 — Enumerate Literal Inputs And Caller Keys
|
|
|
|
**Outcome:** An internal syntax walker produces deduplicated inventories and
|
|
does not confuse caller variables with template locals.
|
|
|
|
**Work:**
|
|
|
|
1. Add internal result counterparts in `internal/domain/requirements.go` and
|
|
an analyzer entry in `internal/prompt/requirements.go`, taking context,
|
|
selected definition/snapshot, supplied variables, and session bypass.
|
|
2. Seed declarations, visit executable message roots and the applicable session
|
|
root, and collect literal input references and direct/literal-index caller
|
|
keys. Exclude uncalled definitions and a bypassed session template.
|
|
3. Implement aggregation, exact names, supplied-key presence, independent
|
|
metadata, sorting, and safe issues/explanations. Add ordinary traversal
|
|
cancellation checks without stronger interruption promises or workers.
|
|
4. Track scope identity, not local values: root-map versus nonroot/unknown dot
|
|
and local bindings. For unsupported scope changes, keep safely recognizable
|
|
references and report incomplete analysis rather than inventing caller keys.
|
|
5. Until Stage 3, classify branch references conservatively as possible and
|
|
unsupported dependency-bearing expressions as limitations.
|
|
6. Update the implemented internal component/source descriptions.
|
|
|
|
**Verify:** Literal declared/undeclared input names, repeated references,
|
|
required unused declarations, optional declarations/lookups, missing versus
|
|
empty direct variables, root versus local references, and empty input-free
|
|
templates. Test unsupported-scope uncertainty, not assignment interpretation.
|
|
Run `go test ./internal/prompt ./internal/domain`; commit.
|
|
|
|
## Stage 3 — Classify Simple Branches And Inactive References
|
|
|
|
**Outcome:** Ordinary supplied-context branches give useful classifications;
|
|
inactive inputs and variables remain optional inventory.
|
|
|
|
**Work:**
|
|
|
|
1. Add only the simple condition recognition in the scope table: direct values,
|
|
literal-key lookups, literals, negation, and two-string equality/inequality.
|
|
Use Go truthiness; a supplied `"false"` string is truthy.
|
|
2. For `if`/`with`, collect the predicate's references under the parent's
|
|
reachability. Visit selected and skipped bodies with the appropriate
|
|
reachability and dot identity. In unresolved branches, mark accesses possible.
|
|
3. Collect syntactically recognizable inactive input/caller names without
|
|
evaluating skipped expressions. Add inactive-only explanations; required
|
|
declarations or other definite/possible uses take precedence.
|
|
4. Unknown/content-dependent conditions preserve known entries and limitations.
|
|
An unresolved text-only branch need not be incomplete; dependency-bearing
|
|
unresolved branches can remain incomplete without cross-branch proofs.
|
|
5. Do not add assignment environments, alias-value propagation, branch joins,
|
|
more comparison families, or short-circuit evaluation for complex conditions.
|
|
|
|
**Verify:** Known true/false branches, nested reachability, string truthiness,
|
|
simple comparisons, `with` dot versus explicit root, missing decision values,
|
|
and content-dependent conditions. Include inactive literal input and caller
|
|
references, promotion by active reuse, and declared required inputs.
|
|
Confirm skipped unsupported/dynamic expressions do not introduce uncertainty
|
|
and an all-optional complete inventory needs no artifacts.
|
|
Compare a few supported cases with the real renderer.
|
|
Run `go test ./internal/prompt`; commit.
|
|
|
|
## Stage 4 — Resolve Direct Names And Finish Limitation Reporting
|
|
|
|
**Outcome:** Simple supplied names resolve exactly; complicated names and
|
|
constructs return useful partial inventories without speculative analysis.
|
|
|
|
**Work:**
|
|
|
|
1. Recognize the direct variable/lookup-derived helper names, simple
|
|
parentheses, and direct-to-input pipelines in the scope table.
|
|
Use a known string as an identifier without trimming or coercion.
|
|
2. Missing decision/name values, input-body dependence, unresolved helper
|
|
names, and dependency-relevant unsupported syntax produce the four issue
|
|
categories. Collect recognized references conservatively from unsupported
|
|
subtrees without executing functions, loops, or named calls.
|
|
3. Keep skipped computed input names unresolved without inventing an entry or
|
|
marking incompleteness from inactivity alone. The syntactically visible
|
|
caller key can still appear as an optional variable.
|
|
4. Finalize unknown-name/scope effects on affected optional entries using the
|
|
simple flags described above. Required declarations/definite accesses stay
|
|
required; uncertainty overrides misleading inactive-only explanations.
|
|
5. Finish independent result values and safe diagnostics. No possible-name
|
|
sets, underlying expression/value traces, placeholder names, or payload reads.
|
|
|
|
**Verify:** Exact direct names, optional literal-key lookups, empty versus
|
|
absent supplied names, and equivalent simple parenthesized/piped forms.
|
|
Use a few complex assignment/pipeline/range/named-call examples to prove
|
|
incomplete-success behavior, not broad interpretation coverage.
|
|
Check unresolved-name effects on optional entries and absent unrelated values
|
|
in results/explanations. Run `go test ./internal/prompt`; commit.
|
|
|
|
## Stage 5 — Wire Selection And The Public Facade
|
|
|
|
**Outcome:** Consumers can inspect a selected captured prompt before artifacts
|
|
exist, without execution collaborators or readiness checks.
|
|
|
|
**Work:**
|
|
|
|
1. Add `internal/usecase/requirements_inspection.go` and an internal request.
|
|
Check the nonblank prompt ID, use existing session normalization, honor
|
|
context cancellation, and select via `resolvePromptDefinition`.
|
|
2. Reuse request-profile/default-profile selection and
|
|
`resolveProfileSelection` for identity only. Share a small private selector
|
|
helper with preparation if needed, preserving its error identities/order.
|
|
3. Obtain the captured snapshot through the narrow repository capability and
|
|
invoke the analyzer. Missing required internal capability/snapshot is
|
|
operational `ErrPromptLoad`, not a successful partial result or source fallback.
|
|
4. Do not call preparation, target-completeness/credential checks, environment
|
|
lookups, artifact/schema readers, renderer, validator, admission, generation,
|
|
observations, or run-ID machinery.
|
|
5. Forward through `internal/engine` and add root `requirements.go` with the
|
|
planned values/method, conversion/copy helpers, GoDoc, and `mapPublicError`.
|
|
Nil engine follows `ErrInvalidConfig`. Operational errors return no result;
|
|
discovery limitations return an incomplete result and nil error.
|
|
6. Preserve public selection/session/cancellation error categories. Do not add
|
|
an error sentinel for incomplete discovery. Update internal runner/component
|
|
documents for the implemented method.
|
|
|
|
**Verify:** Focused selection tests cover explicit/default/partial profiles,
|
|
exact versions/ambiguity, direct-session bypass, cancellation, nil engine,
|
|
operational errors, and incomplete success. Use failing/counting execution
|
|
collaborators only to prove absence of the relevant effects.
|
|
Run affected use-case/engine tests and `go test ./...` for assembled facade
|
|
wiring; commit.
|
|
|
|
## Stage 6 — Protect Public Inventory And Ownership
|
|
|
|
**Outcome:** Public contract coverage protects the consumer need without
|
|
duplicating the syntax walker's detailed cases.
|
|
|
|
**Work:**
|
|
|
|
1. Add external-package `requirements_inspection_contract_test.go` with real
|
|
configured engines and minimal synthetic definitions.
|
|
2. Cover exact required `transcript`, undeclared literals, declaration metadata,
|
|
simple context-selected names, and inactive input/variable entries with safe
|
|
explanations. Missing required variables and complete discovery are separate.
|
|
3. Cover conditional/incomplete results and empty/all-optional complete
|
|
inventories. A direct session ID excludes its template-only inventory
|
|
entries; declarations/message references remain.
|
|
4. Prove inspection before artifacts exist and without credentials/schema
|
|
preparation/provider access. Use a valid partial profile and, where useful,
|
|
an explicit direct-key-required profile without its key.
|
|
5. Compare a small set of supported definite requirements with later preparation
|
|
using synthetic artifacts. Optional inactive inventory is not an executed
|
|
access; incomplete cases need not satisfy complete-analysis parity.
|
|
6. Verify captured file-backed content survives later file change/removal.
|
|
Mutate returned slices/metadata and repeat inspection; exercise simultaneous
|
|
inspections alongside preparation to protect captured-state ownership.
|
|
7. Check disclosure with marker values: unrelated variable values, direct
|
|
session values, template text, and credentials stay absent. Binding names,
|
|
caller keys, declared descriptions, and safe inactive explanations are allowed.
|
|
8. Root tests own public values/errors/assembled behavior; analyzer tests own
|
|
recognition/classification. Avoid duplicated interpretation matrices.
|
|
|
|
**Verify:** Run focused contract/affected package tests and `-race` for
|
|
ownership cases. Reuse unchanged prior checks rather than rerunning the full
|
|
suite solely for this unit; commit.
|
|
|
|
## Stage 7 — Document And Verify The Assembled Feature
|
|
|
|
**Outcome:** Implemented semantics and analysis limits are documented in their
|
|
canonical owners, with final verification ready for user review.
|
|
|
|
**Work:**
|
|
|
|
1. Finish GoDoc for classifications, presence, metadata, explanations, issues,
|
|
completeness, errors/ownership, and the deliberately small supported scope.
|
|
Complex valid templates may yield partial results; no authoring ban is added.
|
|
2. Update the consumer guide to compare the inspection APIs and preparation.
|
|
Show required input discovery, optional inactive input/variable inventory,
|
|
missing required variables, and explicit incomplete discovery.
|
|
3. Update relevant format cross-references and internal source/runner/component
|
|
documents. Keep exact implemented API contracts in GoDoc rather than copying
|
|
them into every guide.
|
|
4. Add a small inspection use to the maintained preparation example, preserving
|
|
its existing assertions. Verify it locally without adding another release gate.
|
|
5. Review roadmap acceptance against implementation/tests. Confirm no general
|
|
interpreter, stronger caller restriction, raw/intermediate configuration
|
|
validation, execution-readiness check, or speculative completeness promise
|
|
was introduced.
|
|
6. On final changed code, run one assembled offline verification using the local
|
|
Go installation and prepared module caches:
|
|
`go test -race -vet=off -mod=readonly -count=1 ./...`,
|
|
`go vet -mod=readonly ./...`, and
|
|
`python3 scripts/check-examples.py`. Set `GOTOOLCHAIN=local`, `GOWORK=off`,
|
|
`GOPROXY=off`, `GONOPROXY=none`, and `GOSUMDB=off`.
|
|
Check changed Go formatting, local documentation links, and `git diff --check`.
|
|
Release-tool fixtures need not rerun when tooling is unchanged.
|
|
7. Record implemented status and verification in these temporary review
|
|
documents, commit, and report remaining limitations honestly. Release
|
|
preparation later owns version selection, notes, retirement, tags, and pushes.
|
|
|
|
## Completion Criteria
|
|
|
|
- The separate public operation returns input and caller-variable inventories
|
|
classified required, conditional, or optional for the selected context.
|
|
- Recognizable inactive references remain optional entries in both inventories,
|
|
with safe explanations and aggregation that preserves stronger requirements.
|
|
- Simple branches/direct names agree with execution; complex cases preserve
|
|
recognized names and explicit limitations without assignment/path solving.
|
|
- Effective metadata/captured templates come from final configuration and remain
|
|
immutable; existing inspection, hashes, and execution behavior remain intact.
|
|
- Selection, session bypass, missing versus empty caller keys, metadata presence,
|
|
classification, and completeness have distinct documented meanings.
|
|
- No artifacts, credentials, provider access, schemas, or execution state are
|
|
needed for inspection. Results preserve disclosure and public error boundaries.
|
|
- Focused tests, the maintained example, canonical docs, and final assembled
|
|
checks demonstrate the feature. No significant design questions remain open.
|