eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Settle requirement inspection scope

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-08 17:54:20 +00:00
parent 71f6683fda
commit e459c7d54b
2 changed files with 129 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files

148
docs/roadmap/implementation.md
View File

@@ -9,10 +9,11 @@ ReleaseKit adoption plan; that plan remains available in Git history.
- Baseline: `7827ca043b5de84a18199ac8f4b3056691aacbb5`. There is no effective
requirement-inspection API yet. `InspectPrompt` reports declarations;
`Prepare` needs artifacts. The configured engine captures compiled templates.
- The stages use the roadmap's recommended practical analysis core and
active-context inventory. The previously open scope choices and alternatives
are recorded under Open Questions; do not silently substitute broader analysis
or an all-reference inventory.
- The agreed scope is the practical analysis core, with statically identifiable
inactive undeclared input references included as optional inventory entries
and explained as inactive. Distinguish this inventory from applicable
dependencies; it does not require evaluating skipped expressions or adopting
broader Go-template interpretation.
- Start with [development](../development.md), then read the
[agent](../policy/agents.md), [architecture](../policy/architecture.md),
[testing](../policy/testing.md), and [documentation](../policy/documentation.md)
@@ -65,6 +66,7 @@ type InputRequirement struct {
Declared bool
ContentType *string
Description *string
Explanation string
}
type VariableRequirement struct {
@@ -114,11 +116,17 @@ establish that an input/variable is absent.
- Always report declarations. A declared required input stays required.
Active template accesses can promote an optional declaration to required.
Undeclared references on known inactive paths are omitted.
Include statically identifiable undeclared references on known inactive paths
as optional entries with an inactive-use explanation.
- An entry is required when its use is established for the context, conditional
when a possible use depends on unresolved analysis, and optional when no
applicable use needs it and unresolved effects do not undermine that conclusion.
An established required use dominates conditional/optional uses.
- `Explanation` is a safe human-readable summary of inactive-only use, separate
from declared `Description`. Its wording is not a machine-readable contract.
Do not add an uncertainty issue for known inactivity. Required or conditional
uses elsewhere override inactive-only classification and explanation.
Caller variables used solely on known inactive paths are not requirements.
- Aggregate inputs and variables by exact execution name. Do not trim request
variable keys or discovered helper names. Configuration normalization remains
with the existing configuration adapter.
@@ -144,6 +152,10 @@ establish that an input/variable is absent.
whether they are satisfied. `{{.audience}} {{input "note"}}` can yield complete
discovery with a missing required `audience` and known required `note`.
Reading the body of `note` is unnecessary to identify that dependency.
An empty complete input set, or a complete inventory containing only optional
entries, establishes that no artifact bindings are needed. Nonempty inventory
alone does not establish that artifacts are needed; incomplete discovery does
not establish their absence.
Unknown values carry their cause internally. Emit issues and mark incomplete
when uncertainty affects a dependency decision/name, or an opaque construct
@@ -153,13 +165,16 @@ An unknown branch containing only text need not make discovery incomplete.
For an unresolved branch, analyze both paths once and combine access summaries:
a name definitely needed on both paths can stay required; names possible on
only some paths are conditional. Merge branch-local unresolved issues.
Compare requirement summaries, not rendered content, and do not enumerate
combinations of independent conditions. Conservative incompleteness is valid
Compare applicable requirement summaries, not inactive inventory or rendered
content, and do not enumerate combinations of independent conditions.
Conservative incompleteness is valid
where the core cannot prove an exact set; do not claim completeness there.
Unresolved dynamic names/opaque input effects must not leave otherwise optional
declarations falsely definitive. Mark affected declared optional inputs
conditional; an unrestricted unknown input name can affect all such declarations.
input entries falsely definitive. Mark affected optional entries conditional,
including previously inactive-only undeclared entries; an unrestricted unknown
input name can affect all such entries. Clear inactive-only explanations where
possible active uses undermine them.
An unknown-name issue also describes potential undiscovered undeclared names.
Known required declarations remain required.
@@ -187,6 +202,18 @@ A numeric literal or unsupported pure output expression is not itself a caller
requirement. Discover its evaluated arguments where that is safe; its value can
remain unknown until a later dependency decision needs it. Known short-circuit
paths must not collect dependencies from skipped operands or branches.
They can contribute optional inactive inventory as described below, without
collecting skipped caller-variable requirements or uncertainty.
For a known inactive path, use a non-evaluating syntax walk to collect literal
`input` names, including equivalent parenthesized/piped literal forms. Walk
skipped branch bodies and short-circuit operands without reading caller values,
mutating local environments, or invoking templates/functions. Unresolved
dynamic names and unsupported expressions in these skipped paths do not make
discovery incomplete. Never-invoked definitions and a directly bypassed session
template remain outside this inventory. Track inactive-only occurrences
separately from definite/possible uses so aggregation can classify a shared
name correctly.
Use a small private known/unknown value model. Branch joins retain identical
known values and can retain finite possible string names from direct
@@ -249,7 +276,9 @@ discoverable through an internal analyzer.
`input "name"` calls, deduplicate names, and aggregate required accesses.
Analyze the session first when applicable and messages in their existing order.
4. Add aggregation/finalization helpers for classification, independent metadata,
ordering, safe issue construction, and `Complete`. Do not produce prompt text.
ordering, safe explanations/issue construction, and `Complete`. Keep
inactive-only occurrence information separate from applicable accesses for
Stage 5. Do not produce prompt text.
5. Until later stages implement control flow, return conservative incomplete
results for constructs that can hide dependencies. Temporary partial coverage
is internal implementation status, not a new public promise.
@@ -304,6 +333,8 @@ with Go templates.
3. Once a preceding operand leaves access uncertain, treat later potentially
evaluated operands conditionally rather than definitely accessed or skipped.
Preserve unknown causes for later dependency decisions.
Stage 5 adds optional inventory for definitely skipped helper expressions;
these are not applicable input or variable dependencies.
4. Keep unsupported/incompatible evaluation as an internal unknown value with
a safe category. Do not leak values through errors or convert it into a new
construction/request rejection.
@@ -316,27 +347,36 @@ comparisons, multi-operand `eq`, and `and`/`or` skips containing helper calls
or missing variables. Compare representative supported expressions with the
existing Go renderer. Run `go test ./internal/prompt`; commit.
## Stage 5 — Analyze Known Branches And Lexical Scope
## Stage 5 — Analyze Known Branches, Scope, And Inactive Inventory
**Outcome:** Supplied context resolves ordinary `if`/`with` dependencies;
known inactive uses are omitted.
statically identifiable inactive input references remain optional inventory.
**Work:**
1. Add `if`/`else` and `with` traversal for known predicate values.
Analyze only the selected path.
Evaluate only the selected path for applicable dependencies.
2. Preserve root `$`, `with` dot changes, pipeline-declared symbols, block
shadowing, and assignments to outer symbols. Follow Go's lexical lifetimes.
3. Never reinterpret `.field` against a known string dot as a root-map lookup.
Unsupported field operations have unknown values, not invented variables.
4. Retain declarations regardless of branch activity; omit undeclared helper
and variable accesses solely on a known inactive path.
5. Keep uncertain predicates using the temporary conservative handling until
4. Add the non-evaluating inactive syntax walk specified above. Include literal
helper names from skipped branches and short-circuit operands as optional
entries with a safe inactive-use explanation. Preserve declarations and
aggregate other uses of the same name; required/conditional uses dominate.
5. Do not evaluate skipped predicates, resolve skipped dynamic names, mutate
branch environments, or collect skipped caller-variable requirements/issues.
Do not inspect uncalled definitions or a bypassed session template.
6. Keep uncertain predicates using the temporary conservative handling until
Stage 6; do not guess their truth from placeholders.
**Verify:** Both known branches, nested `if`/`with`, root access from changed dot,
declarations versus assignments/shadowing, and inactive helpers/missing variables.
Include declared-required and declared-optional inputs in these cases.
Test inactive undeclared literal and equivalent parenthesized/piped names,
inactive explanations, active reuse overriding inactivity, and skipped dynamic
or unsupported expressions causing no uncertainty. Verify that a complete
all-optional inventory requires no artifacts.
Run `go test ./internal/prompt`; commit.
## Stage 6 — Join Unresolved Paths And Finalize Completeness
@@ -354,7 +394,8 @@ results without inventing definitive classifications.
or a declared required input still dominates.
3. Emit uncertainty only when it affects discovered requirements. An unresolved
text-only branch can still have a complete requirement set; no output rendering
is needed to establish that.
is needed to establish that. Inactive inventory entries alone do not change
applicable dependency summaries or introduce incompleteness.
4. Support input-body-caused unknown conditions without reading the body:
the predicate's known helper name is required; dependent branches are partial.
5. Handle out-of-core controls/calls conservatively. Inspect identifiable
@@ -372,7 +413,7 @@ returning incomplete rather than errors. Run `go test ./internal/prompt`; commit
## Stage 7 — Finish Dynamic Names And Conservative Inventory Aggregation
**Outcome:** Direct variable/alias names resolve exactly; unresolved names
preserve known candidates and correctly affect declaration classifications.
preserve known candidates and correctly affect optional inventory classifications.
**Work:**
@@ -385,17 +426,20 @@ preserve known candidates and correctly affect declaration classifications.
3. Emit dynamic-name issues together with useful missing-variable, input-content,
unsupported, or expression-unresolved categories from the underlying value.
Do not echo values beyond a resolved/known possible binding identifier.
4. Make potentially targeted declared optional inputs conditional: candidate-only
uncertainty affects matching declarations; unrestricted/opaque input effects
can affect all optional declarations. Leave known required inputs required.
4. Make potentially targeted optional input entries conditional: candidate-only
uncertainty affects matching entries; unrestricted/opaque input effects
can affect all optional entries, including inactive undeclared inventory.
Remove inactive-only explanations when possible active uses undermine them.
Leave known required inputs required.
5. Finalize deduplication, lexical ordering, metadata propagation, variable
`Supplied` flags, and completeness. No placeholder `"*"` input name,
invented variable default, payload read, or binding-validation API.
`Supplied` flags, inactive-use explanations, and completeness. No placeholder
`"*"` input name, invented variable default, payload read, or binding-validation API.
**Verify:** Exact dynamic names, empty supplied values versus missing values,
aliases selected by known branches, finite names from unknown assignments,
unrestricted names, repeated uses across messages/session, and optional
declaration uncertainty. Verify results contain no unrelated variable values.
declaration/inactive inventory uncertainty. Verify results contain no unrelated
variable values or skipped-variable requirements.
Run `go test ./internal/prompt`; commit.
## Stage 8 — Wire Selection And The Public Engine Facade
@@ -425,6 +469,8 @@ credentials, schema preparation, or execution collaborators.
selected identities and forward through `internal/engine`.
6. Add the public values/method in root `requirements.go`, conversion helpers,
GoDoc, and normal `mapPublicError` translation. Copy mutable values as above.
Document inactive inventory and `Explanation` separately from declaration
metadata and structured uncertainty; do not promise diagnostic wording.
Nil engine matches `ErrInvalidConfig`; operational errors return no partial
result. Successful incomplete discovery returns a result and nil error.
7. Preserve existing public error categories for absent prompt/profile,
@@ -452,17 +498,22 @@ duplicating the analyzer's detailed case matrix.
2. Prove exact `transcript` discovery, undeclared literal inputs, required and
optional declarations in inactive branches, context-selected names, and
session-only requirements removed by a direct session ID.
Include optional inactive undeclared entries with explanations, active reuse
overriding inactivity, and bypassed session names absent from the inventory.
3. Cover complete-but-unsatisfied variable requirements, conditional/unknown
dependencies, and empty complete versus empty incomplete input sets.
Cover complete all-optional inventories and no caller requirements from
solely inactive variable accesses.
4. Preserve classifiable errors for selection/cancellation. Test inspection
against a valid partial profile and an explicit direct-key-required profile
without an API key; missing runtime schemas and future artifact files must
not prevent inspection. Inject fail-if-called artifact/model/validator
collaborators where supported to prove the relevant absence of execution.
5. Use a small parity table: known-context inspector requirements must agree
with `Prepare`/the renderer's input and variable semantics when actual
synthetic artifacts are later supplied. Do not use execution failure order
as the definition of all discovered requirements.
5. Use a small parity table: known-context applicable inspector requirements
must agree with `Prepare`/the renderer's input and variable semantics when
actual synthetic artifacts are later supplied. Do not use execution failure
order as the definition of all discovered requirements.
Do not compare optional inventory entries to actual renderer helper accesses.
6. Test source capture using a file-backed message changed/deleted after engine
construction. New inspection uses the captured content, like existing runs.
7. Mutate returned metadata/slices and repeat inspection; run representative
@@ -471,6 +522,7 @@ duplicating the analyzer's detailed case matrix.
8. Verify safe diagnostics/result formatting with marker values: unrelated
variable values, direct session value, template text, and credential values
are absent; input identifiers and declared descriptions are allowed.
Include inactive explanations in the disclosure checks.
9. Keep analyzer tests as the owner of detailed expression/branch semantics;
root tests own public values, errors, ownership, and assembled behavior.
@@ -491,6 +543,8 @@ and the complete feature is ready for review, not automatically published.
2. Add task-oriented consumer guidance comparing the three inspection methods
and preparation. Demonstrate required `transcript`, missing variables, and
explicit incomplete discovery without a Promptkit JSON format.
Explain optional inactive inventory and reading requirement classifications
rather than treating every input entry as a binding to prepare.
3. Update relevant format cross-references, internal source/runner/component
documents, and development reading guidance. Do not duplicate the exact
public contract outside GoDoc or describe unsupported constructs as invalid.
@@ -526,42 +580,10 @@ and the complete feature is ready for review, not automatically published.
remains a structured successful partial result, not guessed absence or error.
- Missing values, conditional requirements, metadata absence, and session bypass
are distinguishable without leaking unrelated content.
- Inactive undeclared references appear as optional entries with explanations;
they do not introduce skipped-variable requirements or uncertainty. Complete
all-optional inventories remain distinguishable from incomplete discovery.
- Public classification and ownership tests pass; no provider/resource preflight,
arbitrary limit, application wire schema, or new template-validity rule was added.
- Canonical documentation and maintained examples reflect the implemented
discovery boundary. Successful checks for the final snapshot are reported.
## Open Questions
### Confirm The Practical-Core Scope
The roadmap still records this as open. This plan uses the recommended core,
made precise in the table above, and treats other dependency-relevant constructs
as incomplete without affecting their execution validity.
**Recommendation:** Use that practical core for this feature. It addresses the
consumer scenarios and gives maintainable, testable completeness while avoiding
a full interpreter. Extend coverage later when concrete prompts justify it.
**Alternative:** Include ranges, invoked named templates, numeric comparison,
and more built-in value pipelines in this implementation. That improves complete
discovery for more prompts, but needs additional independent stages and more
semantic parity coverage. It still cannot resolve artifact-content-dependent
decisions before artifacts exist.
### Confirm The Inactive-Reference Inventory
This plan uses the roadmap recommendation: declarations are always reported,
while undeclared references solely on known inactive paths are omitted.
**Recommendation:** Keep that effective-context inventory. It avoids confusing
potential future prompt uses with bindings the current context needs.
**Alternative:** Include statically identifiable inactive undeclared references
as optional entries with an inactive-use explanation. That is useful for broader
operator exploration, but expands result/issue semantics and requires tests and
documentation distinguishing inventory from applicable dependencies.
These questions concern the two previously open scope choices, not a new
publication or human sign-off gate. The stages above consistently implement the
recommended baseline; revise that baseline if a different option is selected.