eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Consolidate and close the codebase audit

Browse Source
This commit is contained in:
Eric Rakestraw
2026-08-11 17:21:29 +00:00
parent 7c562a9374
commit 14e03f19d0
1 changed files with 250 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

251
audit.md
View File

@@ -182,7 +182,7 @@ behavior, and transport mechanics were not audited.
- **Category:** duplication
- **Severity:** medium
- **Confidence:** high
- **Status:** accepted
- **Status:** superseded by S17-F01; evidence retained
- **Affected code:** `profiles.go` (`validatePublicProfile`,
`toDomainProfile`, and the memory profile repository) and
`internal/profile/filesystem_repository.go` (`validateProfile` and
@@ -4355,3 +4355,252 @@ go vet ./...
final accepted set without new measurement or boundary evidence.
- The Stage 0 baseline remains absent and was not backfilled during this
synthesis-driven review.
## Stage 18: Final Audit Synthesis
### Final-Tree Recheck
All 53 historical finding records were rechecked against the final audited
tree. The Stage 1 review recorded
`ebf1602635e108e2a7ac1abd3a3ca24a620104ce` as its clean code snapshot, and
the complete commit range from that snapshot through Stage 17 changes only
`audit.md`. No production code, test, fixture, example, canonical contract, or
policy changed during the audit sequence. Consequently, none of the confirmed
failure probes or high-confidence test and ownership analyses has been fixed
or invalidated in the final tree.
The current code graph was rechecked at 1,753 nodes and 7,036 edges, including
489 functions, 100 methods, 15 interfaces, 106 imports, 1,469 calls, 714 test
relationships, and 53 structural-similarity edges. Its public entry points,
package boundaries, dependency direction, and hotspots remain consistent with
the Stage 17 architecture review. Exact source checks reconfirmed the two
cross-cutting merge groups below. Every retained finding has either confirmed
runtime evidence or high-confidence source, contract, and test evidence; no
medium- or low-confidence observation enters the final accepted set.
### Merge And Supersession Registry
The audit history is preserved in place. One exact policy-duplication record is
superseded, while source-specific behavioral symptoms remain visible under
their shared root causes:
| Canonical group | Historical records | Final disposition |
| --- | --- | --- |
| S17-F01, execution-setting acceptance | S02-F02, S08-F01, S11-F01, S17-F01 | S02-F02 is superseded by S17-F01 because both identify the same missing shared profile/execution-setting owner. S08-F01 and S11-F01 remain accepted downstream correctness evidence for profile and request ingress, respectively. The group is counted once as high-severity correctness, with duplicated policy as its root cause. |
| S17-F02, output-contract legality | S11-F02, S17-F02 | S17-F02 owns the split source-neutral policy; S11-F02 remains the confirmed request-boundary symptom and its error-ordering regression requirement. The group is counted once as medium-severity correctness. |
The other 47 historical IDs remain standalone canonical findings. Similar
effects in separate owners, such as additional YAML documents in prompt and
profile parsers or cancellation in artifact, validation, and transport
boundaries, were not merged: each requires a distinct package fix, preserves a
different error boundary, and has its own regression owner. Keeping those
records separate avoids speculative centralization while still allowing a
later implementation plan to schedule related work together.
After supersession there are 52 accepted evidence records and one superseded
record. After the two root-cause groups are counted once rather than counting
their downstream evidence again, the final planning input contains 49
canonical remediation groups.
### Final Finding Counts
| Category | Canonical findings |
| --- | ---: |
| Correctness | 27 |
| Testing | 13 |
| Efficiency | 4 |
| Duplication | 2 |
| Contract-documentation consistency | 2 |
| Clarity | 1 |
| **Total** | **49** |
| Severity | Canonical findings |
| --- | ---: |
| High | 7 |
| Medium | 36 |
| Low | 6 |
| **Total** | **49** |
The canonical groups comprise 33 confirmed and 16 high-confidence findings.
High severity is reserved for unbounded work or input, source containment,
unresponsive cancellation, data-semantic corruption, and public transport
failure. Medium severity covers bounded but material correctness,
compatibility, policy-drift, test-protection, and measured cost risks. Low
severity is limited to clarity, deterministic workflow protection, and test
maintenance. These ratings describe consumer or maintainer impact, not the
estimated size of a fix.
### Canonical Finding Registry And Recommended Order
This ordering is a risk and dependency sequence for later planning, not an
implementation roadmap. The separate implementation pass must decide work
units, exact files, acceptance criteria, and validation.
| Order | Work class | Canonical IDs | Dependency rationale |
| ---: | --- | --- | --- |
| 1 | High-impact behavioral, data-integrity, resource, and cancellation fixes | S05-F03, S07-F01, S09-F02, S10-F01, S17-F01 (including S08-F01 and S11-F01), S14-F01, S14-F04 | Establish bounds, containment, exact data semantics, shared numeric invariants, and cancellation/resource identities before refactoring or optimizing their paths. |
| 2 | Medium behavioral fixes in public values, normalization, and sources | S02-F01, S03-F01, S05-F01, S07-F02, S07-F03, S08-F02, S08-F03, S08-F04, S08-F05, S09-F01, S09-F03 | Stabilize source selection, parsing, value shape, and error behavior before source consolidation and traversal optimization. S08-F02 should consume bounded shared JSON-value validation after S05-F03. |
| 3 | Medium behavioral fixes in validation, orchestration, and transport | S10-F02, S10-F03, S10-F04, S17-F02 (including S11-F02), S12-F01, S12-F02, S14-F02, S14-F03, S14-F05 | Normalize contracts before schema work, establish one valid compiled schema path, preserve initial/repair semantics, and fix transport framing before reorganizing tests or measuring residual cost. |
| 4 | Contract and documentation consistency decisions | S05-F02, S13-F01 | Decide the public numeric acceptance rule before changing JSON-value boundaries; align copied-handle formatting with the already documented opaque contract. These are confirmed mismatches, not speculative documentation cleanup. |
| 5 | Safe production refactors with clear owners | S02-F05, S07-F05 | Consolidate stable JSON field ownership and prompt source-neutral selection only after their current behavior and defects are fixed and protected. |
| 6 | Consequential medium test gaps | S01-F01, S02-F03, S02-F04, S03-F02, S04-F01, S05-F04, S07-F06, S12-F03 | Add protection at each named owner alongside or immediately after its behavioral work; do not reproduce lower-layer truth tables at the public facade. |
| 7 | Clarity, deterministic workflow, and test consolidation | S06-F01, S05-F05, S09-F05, S14-F06, S16-F01, S16-F02 | Remove writable non-policy state, delete or rewrite low-value tests while retaining stronger contracts, make transport tests environment-independent, and synchronize the maintained offline workflow. |
| 8 | Demonstrated performance work | S07-F04, S08-F06, S09-F04, S10-F05 | Optimize only after correctness and ownership changes settle each path. Preserve the counting, allocation, and behavior invariants specified by the findings and compare representative catalog, artifact, schema, and output sizes. |
The registry accounts for every canonical group exactly once: 27 behavioral
correctness groups in orders 1 through 3, two contract-consistency decisions,
two safe production refactors, 13 testing findings split by impact and
maintenance type, one clarity finding, and four measured efficiency findings.
### Dependency And Root-Cause Notes
- S05-F03 precedes S05-F04 and S08-F02 so expanded JSON-shape protection and
file-profile consumption do not institutionalize an unbounded shared copier.
- S17-F01 is the source-neutral owner for the S08-F01 and S11-F01 regressions;
their profile/config and invalid-request error mappings remain separate test
responsibilities.
- S07-F01, S07-F02, and S07-F03 should establish correct prompt-source
behavior before S07-F05 unifies its OS and `fs.FS` algorithms. S07-F04 then
removes unnecessary content reads from the unified selection path, while
S07-F06 protects the resulting semantic rule set.
- S08-F01 through S08-F05 establish profile validity and selection before
S08-F06 changes decoding work. Point-in-time lookup and overlay fallthrough
remain invariants; no engine-wide cache is implied.
- S10-F01 through S10-F04 establish exact JSON semantics, URL-safe schema
loading, compiled preparation, and cancellation before S10-F05 replaces
JSON-mode materialization. S10-F03 owns one read/compile plan per operation,
not cross-operation schema caching.
- S17-F02 normalizes request contracts before schema loading. S11-F02's public
preparation parity and invalid-request ordering remain the regression
boundary for that shared policy.
- S12-F01 and S12-F02 correct repair request and result semantics before
S12-F03 expands multi-attempt progression coverage.
- S14-F01 through S14-F05 and deterministic replacement of S16-F02 should
settle transport behavior before S14-F06 consolidates its fixture matrix.
The refactor must retain every durable wire assertion and specialized
cancellation, deadline, limit, and body-lifecycle transport.
- S02-F01 should settle duration decoding before S02-F05 consolidates the
stable JSON mapping around it. S05-F02 separately requires an explicit
public numeric policy choice rather than an inferred interoperability limit.
- S16-F01 requires one canonical maintainer-command owner. Update the
development guide and synchronize documents that link to its workflow; do
not create another independent command list.
### Work-Type Separation
- **Behavioral fixes:** The 27 correctness groups change accepted inputs,
source containment or selection, cancellation, validation, execution
accounting, response framing, or public error behavior. Each requires a
regression at its narrowest stable owner plus only the public integration
needed to protect mapping or orchestration.
- **Safe refactors:** S02-F05 and S07-F05 have clear semantic owners and drift
evidence. S06-F01 is a small clarity change that removes writable state.
None should alter supported behavior.
- **Performance work:** S07-F04, S08-F06, S09-F04, and S10-F05 have a stated
frequency, scale, measurement or complexity model, and benchmark or
counting invariant. No other performance idea is accepted.
- **Test gaps:** Eight medium findings protect consequential redaction,
ownership, mapping, precedence, cancellation, JSON shape, prompt semantics,
and repair progression. Add them at the owners already named in their
findings rather than chasing coverage percentages.
- **Test consolidation and deterministic maintenance:** S05-F05 can remove an
unused internal serialization boundary, S09-F05 can replace algorithm
literals with hash relationships, and S14-F06 can consolidate HTTP
mechanics. S16-F02 replaces a fixed-port assumption, while S16-F01 adds the
already maintained Run example to canonical validation. Each record states
the protection that must remain.
- **Documentation synchronization:** S05-F02 requires GoDoc and format
documentation only if the chosen numeric contract differs from today's
finite-number promise. S16-F01 synchronizes validation documentation through
its canonical development-guide owner. Implementing either S17 owner in
`internal/domain` requires the architecture policy and internal component
overview to name the new invariant responsibility in the same change.
Other behavior changes should update only their existing canonical GoDoc,
format, integration, or internal owner when observable claims change.
### Rejected And Residual Work
The final set excludes every lower-confidence observation and every Stage 17
candidate rejected for lacking a shared semantic owner, drift risk, runtime
measurement, or durable boundary. In particular, it excludes coverage-driven
tests, generic facade and source-option abstractions, interface collapsing,
constructor parameter objects, cross-operation caches, lock or fairness
changes without contention evidence, complexity-only function splitting, and
an unmeasured extra-parameter marshaling optimization. Stage 15 found no
capacity, race, deadlock, fairness, or resource-release defect, so no
concurrency remediation is invented at closeout.
Residual uncertainty is limited and explicit:
- Stage 0 was never executed, so there is no independently recorded initial
package inventory, validation baseline, or initial coverage snapshot. The
unchanged Stage 1 code commit and final full validation establish a
reproducible final reference but cannot recreate that missing historical
baseline.
- Performance figures in S09-F04 and S10-F05 came from temporary diagnostic
benchmarks on the audit host. They confirm allocation and asymptotic waste
but are not release performance promises; implementation must retain
representative benchmarks rather than target the recorded wall-clock
numbers.
- Provider transport review used deterministic local servers and transports,
as required by testing policy. No live or paid provider was contacted, so
upstream behavior outside the documented OpenAI-compatible contract remains
intentionally untested.
These uncertainties do not lower confidence in an accepted finding. There are
no unresolved observations awaiting promotion or rejection.
### Audit Coverage Summary
Stages 1 through 17 reviewed every root public value, formatter, error and JSON
boundary; configuration, extension adapter, engine option and operation;
internal domain, JSON-value, backend, defaults, built-in profile, file catalog,
prompt definition, profile, artifact, renderer, validator, use-case, prepared
execution, model client, and capacity component; all 25 test files and 281
top-level tests; both maintained offline Go-library examples and their
fixtures; and the cross-package dependency, similarity, interface, complexity,
and responsibility map. Public, package, failure, cancellation, resource,
serialization, source, provider-wire, lifecycle, and concurrency paths all
have a coverage ledger or accepted finding.
The audit produced 49 canonical remediation groups. It found no production or
test change during the findings-only sequence, no lower-confidence accepted
item, no unsupported consumer dependency, and no code or test modification
mixed into the audit artifact.
### Final Validation
After the Stage 18 documentation edit, ordinary and race-enabled package
tests, static analysis, build, and both maintained offline examples passed:
```sh
go test ./...
go test -race ./...
go vet ./...
go build ./...
go run ./examples/go-library/prepare
go run ./examples/go-library/run
```
Every tracked Go file passed `gofmt`. All 180 maintained Markdown links had an
existing repository target or supported external form, all 18 local anchors
resolved, and the one published Markdown target returned HTTP 200. Module
metadata remained:
```text
gitea.maximumdirect.net/eric/promptkit 1.25.5
promptkit gitea.maximumdirect.net/eric/promptkit
```
No tracked Go workspace, vendor tree, or module replacement exists.
`git diff --check` passed, the diff from the Stage 1 code snapshot through the
final audit contains only `audit.md`, and the working tree immediately before
staging contained only that authorized documentation change.
### Closeout
`audit.md` is now the complete findings-only input to a separate remediation-
planning prompt. The later `implementation.md` pass must turn the ordering and
dependencies above into decision-complete work units. This audit does not
authorize or schedule code, test, fixture, contract, or durable-documentation
changes.