Files
promptkit/docs/roadmap/implementation.md

841 lines
42 KiB
Markdown

# Audit Remediation Implementation Plan
## Purpose
This document is the decision-complete implementation plan for the accepted
findings in the [codebase audit](audit.md). It is written for a
`gpt-5.6-terra` coding agent that will implement one numbered stage per prompt,
in order.
The audit remains the evidence and rationale for each finding. This plan owns
implementation order, selected policy decisions, required code and test work,
and stage gates. It does not activate the future public output-repair feature
described in the [future feature catalog](future.md); it only corrects and
protects the retained internal repair machinery on which that later feature
may build.
## Implementation Policies
Every stage must follow the [development guide](../development.md),
[architecture policy](../policy/architecture.md),
[testing policy](../policy/testing.md), and
[documentation policy](../policy/documentation.md). Before changing a
subsystem, read the focused current-state documents identified by the
development guide and inspect the exact implementation and tests named by the
stage.
Apply these rules throughout:
- Implement exactly one stage per agent prompt. Do not combine stages or begin
a later stage early.
- Inspect the working tree before editing and preserve unrelated changes.
- Use the code knowledge graph to locate symbols, callers, and dependency
paths; confirm important conclusions against source.
- Keep the root package as the public facade and implementation under
`internal/`. Do not expose internal representations or add a public package.
- Put source-neutral invariants in their assigned internal owner while
preserving source-specific normalization, error classification, and public
translation at existing boundaries.
- Add regression protection at the narrowest stable owner in the same stage
as a behavioral fix. Retain only representative integration coverage at
higher layers.
- Do not add tests to raise coverage percentages. Do not preserve tests that
assert an incidental algorithm, private constant, dormant serialization
shape, or duplicated lower-layer truth table.
- Keep all tests deterministic, offline, race-safe, credential-free, and free
of fixed-port or mutable-service assumptions.
- Do not add engine-wide caches, generic facade abstractions, scheduler
changes, provider retry policy, or new consumer configuration unless a stage
explicitly requires it.
- Update canonical GoDoc and current-state documents in the same stage as the
behavior they describe. Do not describe a later stage as already
implemented.
- Format changed Go files. Run the stage's focused commands, then at least
`go test ./...` and `go vet ./...`. Run focused race tests wherever the
stage changes ownership, cancellation, shared state, or lifecycle behavior.
- Do not commit, push, tag, or publish unless separately instructed.
## Decisions Fixed By This Plan
The implementing agent must not reopen these choices:
1. **JSON-compatible numbers:** accept every value Go can faithfully encode as
a JSON number: every signed and unsigned integer width, finite `float32` and
`float64` values, and a `json.Number` whose text is valid JSON-number syntax.
Do not impose the current IEEE-754 safe-integer restriction. Reject NaN,
infinities, and malformed `json.Number` text. Preserve supported concrete
numeric types when copying.
2. **JSON-shaped traversal bounds:** allow at most 100 JSON container levels
and 100,000 produced JSON value nodes per `Copy` or `CopyMap` operation.
Count the root, each map/slice/array container, and every produced child
value; map keys are not separate nodes. Pointer and interface indirection
do not add JSON depth or an extra node. Repeated appearances of an acyclic
shared value count each produced occurrence. Continue rejecting active-path
cycles and return deterministic, path-aware validation errors on either
bound.
3. **Execution timeout bound:** a positive `TimeoutSeconds` must fit in
`time.Duration` after multiplication by `time.Second`. Derive the maximum
from `math.MaxInt64` and `time.Second`; do not duplicate its numeric literal
in tests or documentation.
4. **Output contracts:** the only valid formats are `text`, `markdown`, and
`json`; the only valid validation modes are `none`, `basic`, `json`, and
`json_schema`; repair attempts are non-negative; and `json_schema` requires
a nonblank schema path. A non-nil request replacement defaults an empty
format to `text` before shared validation. It does not default an empty
validation mode.
5. **Prompt content paths:** every `content_file` is an exact, relative path
resolved from its prompt file and contained by the configured prompt source
root. Directory, `fs.FS`, and single-file sources all reject absolute and
escaping paths. A single-file source's root is the containing directory of
that selected prompt file. Trimming determines only whether a value is
blank; it must not change the path opened. OS containment must account for
symlinks; containment inside an injected `fs.FS` remains expressed in that
filesystem's namespace.
6. **Profile IDs:** normalize file-backed IDs with `strings.TrimSpace` once,
just as in-memory IDs are normalized. Use the normalized value for
selection, duplicate detection, results, and diagnostics. A whitespace-only
ID is invalid, and IDs that become equal after normalization are
duplicates.
7. **Ordinary artifact files:** the built-in `File` reader supports regular
files, including symlinks whose targets are regular files. It rejects
directories, FIFOs, devices, sockets, and other non-regular targets before
consuming them. It remains unrestricted by an application root and does
not introduce an application-specific byte limit.
8. **Validation cancellation:** do not return early by abandoning goroutines
around `fs.FS` or the JSON Schema dependency. Promptkit must check
cancellation before, between, and after work it controls; read opened files
in context-checked chunks; and let a canceled context win before publishing
a result after synchronous decode, compile, or validation calls. Go's
`fs.FS` and the current JSON Schema library expose no general mechanism to
preempt a blocked `Open`, `Read`, compile, or validation method, so canonical
documentation must describe this synchronous limitation rather than claim
impossible asynchronous interruption.
9. **Successful provider-response limit:** the built-in OpenAI-compatible
client accepts at most 16 MiB (`16 << 20` bytes) for the complete successful
HTTP response body, including surrounding whitespace. The limit is fixed,
internal, and application-neutral. Exactly the limit is allowed; the first
byte beyond it fails as `internal/llm.ErrMalformedResponse`. Do not add a
public setting. Non-success response parsing remains outside this audit
remediation and belongs to the separate structured-generation-error
roadmap.
10. **Repair machinery:** retain and fix the internal repairer, cumulative
usage, and bounded repair state machine. The public engine must continue to
install no repairer and remain single-pass. Do not activate public repair
in this plan.
## Stage 1: Centralize Execution-Setting And Session Invariants
**Findings:** S05-F01, S17-F01, S14-F02. This stage also resolves the
source-specific evidence in S02-F02, S08-F01, and S11-F01.
Add a source-neutral execution-setting validator to `internal/domain`. It must
validate temperature, maximum tokens, top-p, and timeout on a domain execution
target: temperature and top-p must be finite and within their closed ranges,
maximum tokens must be non-negative, and timeout must be non-negative and no
greater than the derived duration-safe maximum. Keep optional-pointer presence,
profile required fields, normalization, and error wrapping outside this
validator.
Use that owner from:
- in-memory profile validation in the root package;
- OS and `fs.FS` profile validation;
- resolved request/target validation in `internal/usecase`; and
- the built-in model client as a defensive final boundary.
Remove the duplicated scalar comparisons from those callers. Preserve
`ErrInvalidConfig` for in-memory construction, profile-load identities for file
profiles, `ErrInvalidRequest` for runtime overrides, and the LLM package's
defensive invalid-request identity. Explicit numeric zero must retain its
presence semantics.
Update `internal/domain.NormalizeSessionID` to reject invalid UTF-8 before
trimming or rune counting. Preserve the existing blank and 256-code-point
rules. Direct requests must still map failures to `ErrInvalidRequest`, while
session-template failures remain renderer failures.
Add one domain-owned table for every exact setting boundary, finite neighbors,
NaN, both infinities, negative values, and the timeout representability edge.
Retain small boundary-integration cases for in-memory profiles, both file
source forms, request overrides through `Prepare` and `PrepareExecution`, and
the model-client defense. Add malformed UTF-8 session cases before, within,
and after otherwise valid content.
Update the architecture policy and internal component overview so
`internal/domain` explicitly owns source-neutral invariants for its shared
execution values, without claiming ownership of source-specific policy.
Run focused domain, profile, use-case, root, and LLM tests, including the
affected race-enabled request and profile cases, followed by the repository
test and vet gates.
## Stage 2: Centralize Output-Contract Legality
**Finding:** S17-F02, including the request-boundary symptom S11-F02.
Add one pure `internal/domain` validator for `OutputContract`. It must enforce
the format, validation-mode, non-negative repair-attempt, and JSON-Schema path
rules fixed above. It must not load schemas or apply source/request defaults.
Make prompt-definition normalization call the shared validator after its file-
specific normalization. Keep prompt-required fields and contextual
`ErrInvalidPromptDefinition` ownership in `internal/promptdef`. Make request
resolution default an empty replacement format to `text`, then call the same
validator and translate failure to `ErrInvalidRequest` before artifact,
rendering, validation, admission, or generation work. Keep schema loading and
compilation in `internal/validate`.
Add a domain table covering every supported and unsupported enum, empty values,
negative and non-negative repair counts, and schema-path relationships. Retain
small prompt-source and use-case integration tables that prove correct error
categories and parity between `Prepare` and `PrepareExecution`; do not repeat
the entire domain table at those layers.
Update the architecture and internal overview language added in Stage 1 to
include source-neutral output-contract invariants. Run focused domain,
prompt-definition, use-case, and root tests, then repository test and vet
gates.
## Stage 3: Make JSON-Compatible Value Handling Coherent And Bounded
**Findings:** S05-F02, S05-F03, S05-F04.
Refactor `internal/jsonvalue` around the numeric and traversal decisions fixed
by this plan. Remove the safe-integer restriction and apply one numeric rule to
all supported representations. Preserve concrete named and unnamed scalar,
map, slice, and array types where the existing contract promises preservation;
keep nil versus empty container distinctions and `Copy` versus `CopyMap` empty-
key behavior.
Extend the traversal state to track JSON container depth and produced-node
work. Enforce the 100-level and 100,000-node limits before allocation or
descent would cross them. Continue using active-path identity for cycle
detection; do not use alias memoization that would make distinct JSON paths
share mutable output. Errors must identify the structural path and whether the
depth or work budget was exceeded.
Expand the focused package tables by behavior branch: signed and unsigned
integer widths, ordinary and named finite floats, `json.Number`, pointers and
interfaces, named maps/slices/arrays, nil and empty values, mixed nested trees,
arrays, mutation isolation, active cycles, alternating just-below/at/over
depth, and shared acyclic subgraphs just below and over the work budget. Tests
must derive their edges from package constants or relationships instead of
copying unexplained literals.
Retain only representative public/backend/profile/prepared integration cases
that prove error translation and ownership. Update public GoDoc only if it
currently states the narrower safe-integer behavior; otherwise the existing
finite JSON-compatible-number contract remains canonical. Update the relevant
public value GoDoc and format/internal documentation to state that excessively
deep or large JSON-shaped values are rejected for safety; keep the exact
numeric limits owned by the internal constants rather than duplicating them
throughout consumer documentation. Run focused package and caller tests,
focused race tests, repository tests, and vet.
## Stage 4: Consolidate Stable Public JSON And Remove Dormant Internal JSON
**Findings:** S02-F01, S02-F05, S05-F05.
Refactor `json.go` so each public value has one ordinary field mapping. Use
private aliases or embedded wire representations for ordinary fields and keep
only timestamp, millisecond-duration, and intentional omission exceptions
explicit. Preserve every existing JSON name and omission rule.
Before converting `duration_ms`, reject values outside the millisecond range
that can be multiplied by `time.Millisecond` without overflow. Derive both
edges from `time.Duration` bounds. Return a contextual decode error and do not
partially update the receiver on failure.
Add fully populated `PreparedRun` and `RunResult` contract cases. Verify all
ordinary fields, intentional omissions, zero and nonzero timing, complete
round trips, the largest safe positive and negative millisecond values, and
their first unsafe neighbors.
Remove unused JSON tags and serialization tests from
`internal/domain.PreparedRun` after confirming production never marshals that
type. Keep credential absence protected at preparation/clone producers and
move any useful cache-control JSON assertion to the public `PreparedRun`
contract. Do not retain a parallel internal wire format.
Run focused domain and root JSON tests, repository tests, and vet.
## Stage 5: Harden Public Ownership And Diagnostic Contracts
**Findings:** S01-F01, S02-F03, S02-F04, S13-F01.
Extend the existing run-request formatting test with distinct input URI,
input-body, variable, and API-key sentinels. Require their absence from
`String`, `GoString`, `%v`, `%+v`, and `%#v` while retaining positive structural
summary assertions.
Add one focused public-LLM-adapter ownership test. Have the injected client
mutate and retain prompt messages, cache-control pointers, nested target extra
parameters, and structured-output schema values; prove the domain/prepared
source remains unchanged and later details or execution cannot race with those
mutations.
Add one direct all-field mapping test for `OpenAICompatibleProfile`. Populate
every field distinctly and compare the complete returned `Profile`. Keep only
the existing higher-level cases that prove normal validation and nested-value
ownership.
Make copied `PreparedExecution` values format opaquely by using value-receiver
formatting behavior shared by non-nil pointers and values. A nil pointer may
use Go's normal `<nil>` formatting, but formatting must never panic or expose
internal types, field names, addresses, credentials, or content. Cover original
pointers, copied values, zero values, and nil pointers under string, Go-string,
and ordinary fmt verbs, and prove formatting does not claim or discard a
handle.
Run focused root tests and the affected prepared/adapter race tests, followed
by repository tests and vet.
## Stage 6: Correct Engine Construction Edges And Immutable Defaults
**Findings:** S03-F01, S03-F02, S06-F01.
Change the shared single-file option helper so trimming is used only for the
blank-input check. Perform `Stat`, path decomposition, storage, diagnostics,
and later access with the exact caller path for prompt, profile, and schema
files. Add one compact table covering existing leading- and trailing-whitespace
names through all three options.
Strengthen engine construction tests with three discriminating cases:
- reverse the argument order of in-memory, ordinary, fallback, and built-in
profile categories while retaining fixed category precedence;
- collide `Config.ProfileDir` with an ordinary profile option and prove the
option replaces the configuration source; and
- place a valid same-category replacement after an invalid option and prove
construction still fails at the earlier invalid option.
Convert `internal/defaults.LLMRequestTimeoutDefault` from a variable to a
constant without changing its value or adding a setter. Do not add a test that
mutates or pins a noncontractual default; existing client deadline behavior is
the verification owner.
Run focused engine construction, default-client construction, and race tests,
then repository tests and vet.
## Stage 7: Contain And Preserve Prompt Content Paths
**Finding:** S07-F01.
Refactor prompt content resolution so both repository forms receive an
explicit source-root abstraction. Enforce the path decision fixed by this plan
before any content read. Use exact parsed path text after a separate blank
check. For OS sources, canonicalize the root and resolved target sufficiently
to reject symlink escape; for injected `fs.FS`, use its clean relative path
namespace. A parent component that remains inside the root is valid. Absolute,
escaping, and symlink-escaping targets are invalid.
Apply the same behavioral table to an OS directory, `WithPromptFS`, and a
single-file source: ordinary sibling, nested parent still within root, parent
escape, absolute path, symlink escape where supported, and existing names with
leading or trailing whitespace. Prove rejected targets cause no outside read
and public operations preserve `ErrPromptLoad`.
Update the framework format reference and internal source document to make the
single-file root and absolute-path rule explicit. Run focused prompt-definition
and public source tests, including race tests, then repository tests and vet.
## Stage 8: Correct Prompt Selection, Strictness, Coverage, And Lookup Cost
**Findings:** S07-F02, S07-F03, S07-F04, S07-F06.
Correct both existing prompt repository paths before consolidating them in the
next stage:
- Recover selector metadata from YAML `id` and `version`; never use a filename
stem as an identity.
- Apply normalized ID and requested-version selection before semantic
normalization or `content_file` reads.
- Associate strict YAML, semantic, and content errors only with a reliably
matching selected definition. An unidentifiable malformed file is unrelated
to point lookup; a reliably selected malformed file remains authoritative.
- Require exactly one YAML document. Comments and trailing whitespace are
allowed; a second empty or populated document and malformed trailing YAML
are `ErrInvalidYAML`.
- Continue scanning the YAML metadata required for duplicate detection, but
open content only for selected candidates. A selected content file is opened
once; unrelated and different-version bodies are never opened.
Add paired OS and `fs.FS` regressions for same-stem/different-ID malformed
files, same-ID/different-version invalid files, selected malformed definitions,
additional YAML documents, duplicates, and counting filesystem behavior.
Add a compact normalization table for the previously uncovered missing
version, blank input name, blank message role, invalid output format, negative
repair attempts, and explicit blank default profile. Output-contract rows
should exercise the shared Stage 2 owner rather than recreate its full table.
Run focused prompt-definition, use-case inspection, and root source tests,
focused race tests, repository tests, and vet.
## Stage 9: Unify Prompt Repository Semantics
**Finding:** S07-F05.
After Stage 8 establishes correct behavior in both paths, replace their
duplicated discovery-to-selection algorithms with one source-neutral prompt
selection and normalization flow. Introduce only the small internal source
adapter needed for YAML discovery, bytes, exact content opening, display paths,
and root containment. Keep genuine OS and `fs.FS` mechanics at the adapter
edge.
Move exact selection, version filtering, strict one-document decoding,
selected-error classification, normalization, duplicate handling, and
not-found behavior into the shared flow. Preserve point-in-time source access;
do not cache catalogs or definitions across operations.
Turn the Stage 8 behavior matrix into a shared suite over both adapters and
retain source-specific tests only for distinct path and I/O failures. Delete
superseded duplicate helpers and tests only after the shared suite protects
their meaningful behavior. Use a counting filesystem and a before/after
benchmark over small and large prompt catalogs to confirm unrelated content is
not read and the refactor adds no second scan; do not enforce wall-clock
thresholds.
Update the internal source document to describe the unified semantic owner.
Run focused package, integration, race, repository test, and vet gates.
## Stage 10: Correct Profile Source Validation And Identity
**Findings:** S08-F02, S08-F03, S08-F04, S08-F05. The non-finite scalar
symptom S08-F01 is already resolved by Stage 1.
Make file profile normalization pass `extra_params` through
`internal/jsonvalue.CopyMap` before publishing a domain profile. Preserve
`ErrInvalidProfile` and source path context for empty keys, non-finite values,
nested invalid data, or traversal-budget failures. Do not move reserved
OpenAI-compatible field policy into the profile package.
Use YAML metadata ID as the only selector; never infer authority from a
filename. Normalize the decoded ID once according to this plan. Require exactly
one YAML document in both metadata and strict selected decoding, so trailing
raw credentials, unknown fields, empty documents, and malformed YAML cannot be
ignored. Reliably selected malformed definitions must stop overlay fallback;
unrelated malformed files must not.
Add shared OS and `fs.FS` tables for invalid/valid extra parameters,
same-stem/different-ID malformed files beside a valid profile, fallback
behavior, additional documents, leading/trailing/blank IDs, normalized
duplicates, and exact inspection/preparation of the normalized ID. Retain only
representative public error-translation cases.
Update the framework format and internal source documents if needed to state
ID normalization and one-document behavior. Run focused profile, use-case,
root, and race tests, followed by repository tests and vet.
## Stage 11: Eliminate Duplicate Profile Decoding
**Finding:** S08-F06.
Refactor point lookup so each file receives one metadata pass and only
canonical ID matches receive strict full decoding and normalization. Reuse
bytes already read for metadata; do not decode every unrelated full profile or
turn the repository into a cache. Preserve deterministic duplicate detection,
strict selected errors, overlay fallthrough only on not-found, and fresh
point-in-time reads on every operation.
Add counting/parser-observation tests where stable behavior can be observed,
plus benchmarks for small and large catalogs reporting time and allocations.
Exercise valid selection, unrelated malformed files, selected malformed files,
duplicates, overlay fallthrough, and repeated lookup. Do not add brittle exact
allocation thresholds to ordinary tests.
Run focused profile and root integration tests, benchmarks for diagnostic
comparison, race tests, repository tests, and vet.
## Stage 12: Correct Artifact Semantics, Cancellation, And Hash Tests
**Findings:** S09-F01, S09-F02, S09-F05.
Treat an explicitly typed empty inline reference as a valid zero-byte artifact,
including `InlineWithURI`. Keep absence at the input map/reference boundary and
compute the same metadata and opaque equality value used for other bodies.
For ordinary file references, inspect the target before opening and again
after opening; reject anything that is not a regular file under the decision
above. Replace unbounded `io.ReadAll` with a normal synchronous chunked read
that checks `ctx.Err()` before open, before and after each read, and before
publishing the artifact. Do not return partial artifacts, add a hidden size
limit, or launch an abandoned reader goroutine.
Add source-parity cases for empty and nonempty inline, inline-with-URI, and file
content. Add a platform-appropriate FIFO regression proving the known FIFO is
rejected without requiring an external writer, and cancellation cases for a
pre-canceled file and a progressing regular-file read. Run them repeatedly and
under the race detector.
Replace exact SHA-256 literals with relational assertions: nonempty and stable
for repeat reads, equal for equal inline/file bodies, unequal for changed
bodies, and propagated opaquely through preparation. Do not document or test a
specific algorithm.
Update public GoDoc and the internal source document to describe regular-file
support and cancellation checkpoints. Run focused package, use-case, root,
race, repository test, and vet gates.
## Stage 13: Make Rendering Cancellation-Aware And Reuse Artifact Text
**Findings:** S09-F03, S09-F04.
Check context before session work, before and after each template parse and
execution, before and after every message, and before returning the completed
prompt. Make the `input` helper return an error when cancellation is observed.
Do not run template execution in a detached goroutine.
Within one `Render` call, lazily convert each named artifact body to text once
and memoize that string for the session and all messages. Build the cached
string in 64 KiB chunks with one pre-grown `strings.Builder`, checking the
context between chunks. Preserve bytes exactly, including invalid UTF-8; do not
cache across render calls or mutate artifacts. Unknown and nil inputs retain
their current errors, and a canceled conversion must not publish or cache a
partial string.
Add deterministic tests for pre-cancellation, cancellation during the chunked
input conversion, and cancellation observed after final-message execution.
Require the context identity and no partial prompt while preserving active-
context template errors. Add benchmarks for one and repeated references across
session and messages; report allocations without hard-coded timing limits.
Run focused renderer/use-case/root tests, benchmarks, repeated race tests,
repository tests, and vet.
## Stage 14: Preserve Exact JSON Validation Semantics
**Findings:** S10-F01, S10-F05.
Create one helper for decoding exactly one JSON value with
`json.Decoder.UseNumber` and required EOF after trailing whitespace. Use it for
schema documents and JSON Schema instance values so large integers, precise
decimals, and exponents retain exact `json.Number` semantics through
compilation, prepared metadata, copying, and validation.
For plain `ValidationJSON`, use a non-materializing complete-document syntax
check such as `json.Valid`; do not build a generic tree. Preserve the current
result distinction: malformed generated JSON is a completed failed validation,
not an operational error, and original output bytes remain unchanged.
Add focused OS and `fs.FS` cases around `2^53`, `1e400`, precise decimals,
ordinary numbers, malformed syntax, and trailing values. Exercise schema
`const`, minimum/maximum, and `multipleOf`, and verify the public structured
schema retains exact numeric values. Add benchmarks for scalar, object, and
large-array JSON validation with allocation reporting but no wall-clock
contract.
Update format/internal validation documentation only where it currently
implies float64-limited semantics. Run focused validator/use-case/root tests,
benchmarks, race tests, repository tests, and vet.
## Stage 15: Escape Schema Resources And Compile Once Per Operation
**Findings:** S10-F02, S10-F03.
Represent schema compiler resources with `url.URL` rather than string
concatenation. Use canonical escaped file URLs for OS paths and a private
scheme URL whose path segments are escaped for `fs.FS`. Preserve separators,
decode resource paths exactly once at the loader boundary, and continue
rejecting remote and escaping references. Legal filenames containing percent,
space, `#`, `?`, or Unicode must compile, including contained relative
references.
Unify JSON Schema preparation around `validate.PreparedValidation`:
- the shared preparation pipeline must create one operation-local compiled
plan and derive provider-facing root schema metadata from that plan;
- `Prepare` may discard the plan after returning metadata;
- `Run` must retain and use the plan for its one operation so the schema graph
is not loaded or compiled again during validation; and
- `PrepareExecution` must retain the same plan in its frozen payload.
Remove the document-only `SchemaDocumentLoader` capability if it has no
remaining production caller. Do not add an engine-wide or cross-operation
schema cache. Keep a clear private preparation carrier in `internal/usecase`
if needed so public `domain.PreparedRun` remains free of validator interfaces.
Replace the existing legal-filename expected failure with valid behavior and
retain a genuine compiler-registration failure only if reachable through a
valid source. Add public parity tests for invalid keywords, malformed and
missing direct/second-level references, unsupported dialects, escapes, remote
references, and valid multi-document graphs. A counting source must show each
document read once per operation and fresh reads across separate operations.
Update internal source, validator, and runner documentation for the unified
plan lifetime. Run focused validator/use-case/root tests, race tests,
repository tests, and vet.
## Stage 16: Make Validation Cancellation Authoritative
**Finding:** S10-F04.
Apply the cancellation decision fixed above. Thread context through schema
resource loaders and all Promptkit-controlled read/decode helpers. Read opened
schema files in context-checked chunks. Check the context immediately before
and after JSON decoding, schema compilation, and schema execution; if
cancellation occurred during a synchronous dependency call, return the context
error instead of a schema or successful validation result. Do not publish a
partial plan or validation result.
Do not place arbitrary `fs.FS` calls or JSON Schema work in goroutines merely
to race them against `ctx.Done()`. Tests must therefore distinguish:
- prompt cancellation before work;
- cancellation between controlled read chunks;
- cancellation that becomes authoritative immediately after a synchronous
compile or validation call returns; and
- the documented limitation that Promptkit cannot preempt a dependency method
that never returns.
Use deterministic controlled readers/contexts rather than sleeps. Assert no
goroutine growth or leaked work and preserve operational validation and public
context identities. Update validator GoDoc and internal source/runner documents
to state the synchronous cancellation boundary accurately.
Run focused cancellation tests normally, repeatedly, and under the race
detector, followed by repository tests and vet.
## Stage 17: Repair And Protect The Retained Internal Repair Path
**Findings:** S12-F01, S12-F02, S12-F03.
Retain the internal repair architecture. Extend `RepairRequest` with
`ExecutionTargetPresence` and carry the resolved presence bits unchanged into
the default repairer's `GenerateRequest`. Factor one use-case-local constructor
for common initial/repair generation fields—effective target, presence,
credential, backend identity, session, and structured output—while keeping the
initial and repair prompts intentionally separate.
Accumulate every completed generation response's five token-usage fields into
run-level usage. The final content/raw output/artifact continues to come from
the last candidate, while usage includes initial generation and every completed
repair exactly once. A repair call that returns an error still returns no
partial public result under current error semantics.
Replace the one-attempt-only repair coverage with a compact state-machine
table for:
- initial success with no repair;
- ineligible basic validation despite a positive budget;
- explicit zero and inherited-zero presence across initial and repair calls;
- success before a larger budget is exhausted;
- exact exhaustion of a larger budget; and
- advancement of attempt number, maximum, prior output, diagnostics, final
status, cumulative usage, and collaborator call count.
Retain the distinct capacity integration proving initial and repair generation
use the same backend pool and one whole-run admission lease, but simplify it if
the new focused table makes repair-state assertions redundant.
Update the internal runner and capacity documents for presence fidelity and
cumulative usage. Do not alter `NewRunner` to install a repairer, public GoDoc
that says the engine is single-pass, or the future public repair roadmap.
Run focused use-case, prepared, capacity, and race tests, followed by repository
tests and vet.
## Stage 18: Preserve Transport Error Identities And Test Deterministically
**Findings:** S14-F01, S04-F01, S16-F02. Timeout overflow S14-F02 is already
resolved through Stage 1's shared bound.
Preserve the underlying `http.Client.Do` error in the chain while retaining
`internal/llm.ErrRequestFailed` and the public generation category. Do not add
headers, request content, endpoints, or provider bodies to error text.
Cancellation and deadline identities must survive caller cancellation, caller
deadline, generation deadline, and whole-request client timeout.
Replace the port-9999 test with a controlled round tripper or `httptest`
endpoint that records the selected URL and returns a deliberate result. It must
make no host-dependent connection and must separately prove empty configured
base acceptance and request-endpoint precedence.
Extend the existing external ordinary-run cancellation test to require both
`ErrLLMGenerate` and `context.Canceled`; retain lower-layer tests only for their
distinct error owners.
Update the OpenAI-compatible integration and internal LLM documents for error
identity behavior. Run focused LLM and root tests with repetition and the race
detector, followed by repository tests and vet.
## Stage 19: Validate And Compose Effective Provider Endpoints
**Finding:** S14-F03.
Add one source-neutral OpenAI-compatible base-endpoint validator in
`internal/domain`, alongside the effective execution target invariant. It must
trim surrounding configuration whitespace, require absolute HTTP or HTTPS with
a host, and reject user information, query, and fragment. Use it from backend
registration, in-memory and file profiles, resolved request overrides, and the
built-in client defense while preserving each boundary's existing config,
profile-load, invalid-request, or LLM error category.
An empty configured base URL remains valid for a built-in client because a
resolved request endpoint may supply it later. Validate only a nonempty
configured base at construction, and always validate the final selected
endpoint before transport. Backends and endpoint-only profiles retain their
existing nonempty endpoint requirements.
Compose the completion URL through parsed URL operations (prefer
`url.JoinPath`) so nested paths and trailing slashes reach exactly one
`/chat/completions` suffix. Never append to a raw string.
Add endpoint tables for HTTP and HTTPS, hosts, nested paths, repeated trailing
slashes, queries, fragments, user information, relative paths, missing hosts,
unsupported schemes, and request/profile/config error mapping. Require every
invalid selected endpoint to fail before transport.
Update the architecture/internal overview for domain endpoint invariants and
the OpenAI-compatible integration and internal LLM documents for URL behavior.
Run focused domain/backend/profile/use-case/LLM/root tests, repetition and race
tests where ownership crosses packages, followed by repository tests and vet.
## Stage 20: Bound And Strictly Frame Successful Provider Responses
**Findings:** S14-F04, S14-F05.
Enforce the 16 MiB successful-response decision without first copying the
entire body. Reject an over-limit `Content-Length` immediately, but also wrap
the body in a counting/limited reader that reads at most one byte beyond the
limit so chunked or dishonest responses cannot bypass it. Exactly-limit bodies
remain valid. Always close the body; do not drain an unbounded oversized
stream.
Decode exactly one response object. After the first decode, require only
trailing JSON whitespace and EOF. A second value, non-whitespace suffix,
truncated body, malformed JSON, or size overflow returns
`ErrMalformedResponse` with no partial response and no provider content in the
error.
Add streaming tests just below, at, and one byte over the limit with and
without `Content-Length`, plus a continuing oversized stream. Assert bounded
bytes read, timely return, no partial result, and closure. Add trailing
whitespace success and trailing garbage/second-value failures. Retain ordinary
response mapping and redaction cases.
Document the fixed successful-response boundary and strict one-document rule
in the integration and internal LLM documents. Explicitly leave bounded
non-success error-envelope parsing to the structured-generation-error roadmap.
Run focused transport tests normally, repeatedly, and under race, followed by
repository tests and vet.
## Stage 21: Consolidate Transport Test Scaffolding
**Finding:** S14-F06.
After transport behavior is stable, introduce one small recording-provider
fixture for common request capture and successful/error response setup.
Organize focused tables around request mapping, authentication, endpoint
composition, timeout/error identity, and response framing. Keep specialized
round trippers/readers for cancellation, deadlines, byte counts, continuing
streams, and body closure.
Retain every existing durable assertion for method, path, headers,
authentication, omission and explicit presence, reserved fields, cache
control, structured output, response mapping, usage, error redaction, and
timeout precedence. Retain all Stage 18 through 20 regressions. Delete repeated
servers, generic-map decoding, and response literals only where the fixture
makes the owning behavior clearer; do not replace wire assertions with a broad
snapshot.
Run the LLM suite normally, with shuffle/repetition, and under the race
detector. Deliberately inspect the resulting test inventory against the audit's
transport matrix before running repository tests and vet.
## Stage 22: Restore One Canonical Maintainer Validation Workflow
**Finding:** S16-F01.
Make `docs/development.md` the canonical owner of the complete local maintainer
workflow, as assigned by the documentation policy. Its validation section must
include, from the repository root:
```sh
go test ./...
go test -race ./...
go vet ./...
go build ./...
go run ./examples/go-library/prepare
go run ./examples/go-library/run
```
It must also own the Go formatting, local Markdown link, `git diff --check`,
workspace/vendor/replacement, generated-output, credential, and working-tree
hygiene checks used before accepting changes.
Change the testing policy to state the semantic requirements and link to that
canonical workflow instead of maintaining a partial competing command list.
Change the release procedure to invoke the development-guide validation as a
release prerequisite rather than presenting a separately maintained copy;
retain release-specific metadata, candidate, tag, and publication commands in
the release document.
Run both examples offline and confirm that a missing or invalid Run example
fixture makes its command fail. Validate all changed Markdown links and ensure
current-state documentation describes only the implemented workflow.
## Stage 23: Complete Traceability And Final Validation
This final stage introduces no new behavior. Review the final tree against the
finding-to-stage table below and the evidence in `audit.md`. Confirm every
canonical group is implemented and every source-specific symptom retains its
required regression and error boundary. Do not mark a finding resolved merely
because a nearby refactor landed.
Run the complete development-guide workflow, including both examples, all
formatting and link checks, and repository hygiene. Also run shuffled ordinary
tests and repeated race-enabled tests for the changed concurrency,
cancellation, prepared, validation, repair, and transport packages. Run the
accepted performance benchmarks for prompt lookup, profile lookup, rendering,
and JSON validation and record only qualitative before/after conclusions; do
not establish release timing promises.
Inspect canonical GoDoc, formats, integration, architecture, and internal
documents against the final implementation. Confirm the public engine still
performs no output repair and the future repair entry remains future work.
Confirm the structured-generation-error feature was not implemented as part of
transport remediation.
Leave `audit-sequence.md`, `audit.md`, and this plan in place for maintainer
review. Retire them only in a separately authorized roadmap-cleanup pass after
the remediation has been reviewed and accepted.
## Finding-To-Stage Traceability
| Stage | Canonical findings | Historical or source-specific records handled with the canonical owner |
| ---: | --- | --- |
| 1 | S05-F01, S17-F01, S14-F02 | S02-F02, S08-F01, S11-F01 |
| 2 | S17-F02 | S11-F02 |
| 3 | S05-F02, S05-F03, S05-F04 | None |
| 4 | S02-F01, S02-F05, S05-F05 | None |
| 5 | S01-F01, S02-F03, S02-F04, S13-F01 | None |
| 6 | S03-F01, S03-F02, S06-F01 | None |
| 7 | S07-F01 | None |
| 8 | S07-F02, S07-F03, S07-F04, S07-F06 | None |
| 9 | S07-F05 | None |
| 10 | S08-F02, S08-F03, S08-F04, S08-F05 | S08-F01 was handled in Stage 1 |
| 11 | S08-F06 | None |
| 12 | S09-F01, S09-F02, S09-F05 | None |
| 13 | S09-F03, S09-F04 | None |
| 14 | S10-F01, S10-F05 | None |
| 15 | S10-F02, S10-F03 | None |
| 16 | S10-F04 | None |
| 17 | S12-F01, S12-F02, S12-F03 | None |
| 18 | S14-F01, S04-F01, S16-F02 | S14-F02 was handled in Stage 1 |
| 19 | S14-F03 | None |
| 20 | S14-F04, S14-F05 | None |
| 21 | S14-F06 | None |
| 22 | S16-F01 | None |
The table maps all 49 canonical remediation groups exactly once. S02-F02 is
the one superseded historical finding retained as evidence under S17-F01;
S08-F01, S11-F01, and S11-F02 retain their source-specific regression
responsibilities without being double-counted as canonical groups.
## Open Questions
None. The numeric contract, resource bounds, path and identity rules,
validation-cancellation limitation, repair retention, transport response
limit, and documentation ownership required to implement these stages are
fixed above.