diff --git a/audit.md b/audit.md index fe0b718..1bd4ed1 100644 --- a/audit.md +++ b/audit.md @@ -182,7 +182,7 @@ behavior, and transport mechanics were not audited. - **Category:** duplication - **Severity:** medium - **Confidence:** high -- **Status:** accepted +- **Status:** superseded by S17-F01; evidence retained - **Affected code:** `profiles.go` (`validatePublicProfile`, `toDomainProfile`, and the memory profile repository) and `internal/profile/filesystem_repository.go` (`validateProfile` and @@ -4355,3 +4355,252 @@ go vet ./... final accepted set without new measurement or boundary evidence. - The Stage 0 baseline remains absent and was not backfilled during this synthesis-driven review. + +## Stage 18: Final Audit Synthesis + +### Final-Tree Recheck + +All 53 historical finding records were rechecked against the final audited +tree. The Stage 1 review recorded +`ebf1602635e108e2a7ac1abd3a3ca24a620104ce` as its clean code snapshot, and +the complete commit range from that snapshot through Stage 17 changes only +`audit.md`. No production code, test, fixture, example, canonical contract, or +policy changed during the audit sequence. Consequently, none of the confirmed +failure probes or high-confidence test and ownership analyses has been fixed +or invalidated in the final tree. + +The current code graph was rechecked at 1,753 nodes and 7,036 edges, including +489 functions, 100 methods, 15 interfaces, 106 imports, 1,469 calls, 714 test +relationships, and 53 structural-similarity edges. Its public entry points, +package boundaries, dependency direction, and hotspots remain consistent with +the Stage 17 architecture review. Exact source checks reconfirmed the two +cross-cutting merge groups below. Every retained finding has either confirmed +runtime evidence or high-confidence source, contract, and test evidence; no +medium- or low-confidence observation enters the final accepted set. + +### Merge And Supersession Registry + +The audit history is preserved in place. One exact policy-duplication record is +superseded, while source-specific behavioral symptoms remain visible under +their shared root causes: + +| Canonical group | Historical records | Final disposition | +| --- | --- | --- | +| S17-F01, execution-setting acceptance | S02-F02, S08-F01, S11-F01, S17-F01 | S02-F02 is superseded by S17-F01 because both identify the same missing shared profile/execution-setting owner. S08-F01 and S11-F01 remain accepted downstream correctness evidence for profile and request ingress, respectively. The group is counted once as high-severity correctness, with duplicated policy as its root cause. | +| S17-F02, output-contract legality | S11-F02, S17-F02 | S17-F02 owns the split source-neutral policy; S11-F02 remains the confirmed request-boundary symptom and its error-ordering regression requirement. The group is counted once as medium-severity correctness. | + +The other 47 historical IDs remain standalone canonical findings. Similar +effects in separate owners, such as additional YAML documents in prompt and +profile parsers or cancellation in artifact, validation, and transport +boundaries, were not merged: each requires a distinct package fix, preserves a +different error boundary, and has its own regression owner. Keeping those +records separate avoids speculative centralization while still allowing a +later implementation plan to schedule related work together. + +After supersession there are 52 accepted evidence records and one superseded +record. After the two root-cause groups are counted once rather than counting +their downstream evidence again, the final planning input contains 49 +canonical remediation groups. + +### Final Finding Counts + +| Category | Canonical findings | +| --- | ---: | +| Correctness | 27 | +| Testing | 13 | +| Efficiency | 4 | +| Duplication | 2 | +| Contract-documentation consistency | 2 | +| Clarity | 1 | +| **Total** | **49** | + +| Severity | Canonical findings | +| --- | ---: | +| High | 7 | +| Medium | 36 | +| Low | 6 | +| **Total** | **49** | + +The canonical groups comprise 33 confirmed and 16 high-confidence findings. +High severity is reserved for unbounded work or input, source containment, +unresponsive cancellation, data-semantic corruption, and public transport +failure. Medium severity covers bounded but material correctness, +compatibility, policy-drift, test-protection, and measured cost risks. Low +severity is limited to clarity, deterministic workflow protection, and test +maintenance. These ratings describe consumer or maintainer impact, not the +estimated size of a fix. + +### Canonical Finding Registry And Recommended Order + +This ordering is a risk and dependency sequence for later planning, not an +implementation roadmap. The separate implementation pass must decide work +units, exact files, acceptance criteria, and validation. + +| Order | Work class | Canonical IDs | Dependency rationale | +| ---: | --- | --- | --- | +| 1 | High-impact behavioral, data-integrity, resource, and cancellation fixes | S05-F03, S07-F01, S09-F02, S10-F01, S17-F01 (including S08-F01 and S11-F01), S14-F01, S14-F04 | Establish bounds, containment, exact data semantics, shared numeric invariants, and cancellation/resource identities before refactoring or optimizing their paths. | +| 2 | Medium behavioral fixes in public values, normalization, and sources | S02-F01, S03-F01, S05-F01, S07-F02, S07-F03, S08-F02, S08-F03, S08-F04, S08-F05, S09-F01, S09-F03 | Stabilize source selection, parsing, value shape, and error behavior before source consolidation and traversal optimization. S08-F02 should consume bounded shared JSON-value validation after S05-F03. | +| 3 | Medium behavioral fixes in validation, orchestration, and transport | S10-F02, S10-F03, S10-F04, S17-F02 (including S11-F02), S12-F01, S12-F02, S14-F02, S14-F03, S14-F05 | Normalize contracts before schema work, establish one valid compiled schema path, preserve initial/repair semantics, and fix transport framing before reorganizing tests or measuring residual cost. | +| 4 | Contract and documentation consistency decisions | S05-F02, S13-F01 | Decide the public numeric acceptance rule before changing JSON-value boundaries; align copied-handle formatting with the already documented opaque contract. These are confirmed mismatches, not speculative documentation cleanup. | +| 5 | Safe production refactors with clear owners | S02-F05, S07-F05 | Consolidate stable JSON field ownership and prompt source-neutral selection only after their current behavior and defects are fixed and protected. | +| 6 | Consequential medium test gaps | S01-F01, S02-F03, S02-F04, S03-F02, S04-F01, S05-F04, S07-F06, S12-F03 | Add protection at each named owner alongside or immediately after its behavioral work; do not reproduce lower-layer truth tables at the public facade. | +| 7 | Clarity, deterministic workflow, and test consolidation | S06-F01, S05-F05, S09-F05, S14-F06, S16-F01, S16-F02 | Remove writable non-policy state, delete or rewrite low-value tests while retaining stronger contracts, make transport tests environment-independent, and synchronize the maintained offline workflow. | +| 8 | Demonstrated performance work | S07-F04, S08-F06, S09-F04, S10-F05 | Optimize only after correctness and ownership changes settle each path. Preserve the counting, allocation, and behavior invariants specified by the findings and compare representative catalog, artifact, schema, and output sizes. | + +The registry accounts for every canonical group exactly once: 27 behavioral +correctness groups in orders 1 through 3, two contract-consistency decisions, +two safe production refactors, 13 testing findings split by impact and +maintenance type, one clarity finding, and four measured efficiency findings. + +### Dependency And Root-Cause Notes + +- S05-F03 precedes S05-F04 and S08-F02 so expanded JSON-shape protection and + file-profile consumption do not institutionalize an unbounded shared copier. +- S17-F01 is the source-neutral owner for the S08-F01 and S11-F01 regressions; + their profile/config and invalid-request error mappings remain separate test + responsibilities. +- S07-F01, S07-F02, and S07-F03 should establish correct prompt-source + behavior before S07-F05 unifies its OS and `fs.FS` algorithms. S07-F04 then + removes unnecessary content reads from the unified selection path, while + S07-F06 protects the resulting semantic rule set. +- S08-F01 through S08-F05 establish profile validity and selection before + S08-F06 changes decoding work. Point-in-time lookup and overlay fallthrough + remain invariants; no engine-wide cache is implied. +- S10-F01 through S10-F04 establish exact JSON semantics, URL-safe schema + loading, compiled preparation, and cancellation before S10-F05 replaces + JSON-mode materialization. S10-F03 owns one read/compile plan per operation, + not cross-operation schema caching. +- S17-F02 normalizes request contracts before schema loading. S11-F02's public + preparation parity and invalid-request ordering remain the regression + boundary for that shared policy. +- S12-F01 and S12-F02 correct repair request and result semantics before + S12-F03 expands multi-attempt progression coverage. +- S14-F01 through S14-F05 and deterministic replacement of S16-F02 should + settle transport behavior before S14-F06 consolidates its fixture matrix. + The refactor must retain every durable wire assertion and specialized + cancellation, deadline, limit, and body-lifecycle transport. +- S02-F01 should settle duration decoding before S02-F05 consolidates the + stable JSON mapping around it. S05-F02 separately requires an explicit + public numeric policy choice rather than an inferred interoperability limit. +- S16-F01 requires one canonical maintainer-command owner. Update the + development guide and synchronize documents that link to its workflow; do + not create another independent command list. + +### Work-Type Separation + +- **Behavioral fixes:** The 27 correctness groups change accepted inputs, + source containment or selection, cancellation, validation, execution + accounting, response framing, or public error behavior. Each requires a + regression at its narrowest stable owner plus only the public integration + needed to protect mapping or orchestration. +- **Safe refactors:** S02-F05 and S07-F05 have clear semantic owners and drift + evidence. S06-F01 is a small clarity change that removes writable state. + None should alter supported behavior. +- **Performance work:** S07-F04, S08-F06, S09-F04, and S10-F05 have a stated + frequency, scale, measurement or complexity model, and benchmark or + counting invariant. No other performance idea is accepted. +- **Test gaps:** Eight medium findings protect consequential redaction, + ownership, mapping, precedence, cancellation, JSON shape, prompt semantics, + and repair progression. Add them at the owners already named in their + findings rather than chasing coverage percentages. +- **Test consolidation and deterministic maintenance:** S05-F05 can remove an + unused internal serialization boundary, S09-F05 can replace algorithm + literals with hash relationships, and S14-F06 can consolidate HTTP + mechanics. S16-F02 replaces a fixed-port assumption, while S16-F01 adds the + already maintained Run example to canonical validation. Each record states + the protection that must remain. +- **Documentation synchronization:** S05-F02 requires GoDoc and format + documentation only if the chosen numeric contract differs from today's + finite-number promise. S16-F01 synchronizes validation documentation through + its canonical development-guide owner. Implementing either S17 owner in + `internal/domain` requires the architecture policy and internal component + overview to name the new invariant responsibility in the same change. + Other behavior changes should update only their existing canonical GoDoc, + format, integration, or internal owner when observable claims change. + +### Rejected And Residual Work + +The final set excludes every lower-confidence observation and every Stage 17 +candidate rejected for lacking a shared semantic owner, drift risk, runtime +measurement, or durable boundary. In particular, it excludes coverage-driven +tests, generic facade and source-option abstractions, interface collapsing, +constructor parameter objects, cross-operation caches, lock or fairness +changes without contention evidence, complexity-only function splitting, and +an unmeasured extra-parameter marshaling optimization. Stage 15 found no +capacity, race, deadlock, fairness, or resource-release defect, so no +concurrency remediation is invented at closeout. + +Residual uncertainty is limited and explicit: + +- Stage 0 was never executed, so there is no independently recorded initial + package inventory, validation baseline, or initial coverage snapshot. The + unchanged Stage 1 code commit and final full validation establish a + reproducible final reference but cannot recreate that missing historical + baseline. +- Performance figures in S09-F04 and S10-F05 came from temporary diagnostic + benchmarks on the audit host. They confirm allocation and asymptotic waste + but are not release performance promises; implementation must retain + representative benchmarks rather than target the recorded wall-clock + numbers. +- Provider transport review used deterministic local servers and transports, + as required by testing policy. No live or paid provider was contacted, so + upstream behavior outside the documented OpenAI-compatible contract remains + intentionally untested. + +These uncertainties do not lower confidence in an accepted finding. There are +no unresolved observations awaiting promotion or rejection. + +### Audit Coverage Summary + +Stages 1 through 17 reviewed every root public value, formatter, error and JSON +boundary; configuration, extension adapter, engine option and operation; +internal domain, JSON-value, backend, defaults, built-in profile, file catalog, +prompt definition, profile, artifact, renderer, validator, use-case, prepared +execution, model client, and capacity component; all 25 test files and 281 +top-level tests; both maintained offline Go-library examples and their +fixtures; and the cross-package dependency, similarity, interface, complexity, +and responsibility map. Public, package, failure, cancellation, resource, +serialization, source, provider-wire, lifecycle, and concurrency paths all +have a coverage ledger or accepted finding. + +The audit produced 49 canonical remediation groups. It found no production or +test change during the findings-only sequence, no lower-confidence accepted +item, no unsupported consumer dependency, and no code or test modification +mixed into the audit artifact. + +### Final Validation + +After the Stage 18 documentation edit, ordinary and race-enabled package +tests, static analysis, build, and both maintained offline examples passed: + +```sh +go test ./... +go test -race ./... +go vet ./... +go build ./... +go run ./examples/go-library/prepare +go run ./examples/go-library/run +``` + +Every tracked Go file passed `gofmt`. All 180 maintained Markdown links had an +existing repository target or supported external form, all 18 local anchors +resolved, and the one published Markdown target returned HTTP 200. Module +metadata remained: + +```text +gitea.maximumdirect.net/eric/promptkit 1.25.5 +promptkit gitea.maximumdirect.net/eric/promptkit +``` + +No tracked Go workspace, vendor tree, or module replacement exists. +`git diff --check` passed, the diff from the Stage 1 code snapshot through the +final audit contains only `audit.md`, and the working tree immediately before +staging contained only that authorized documentation change. + +### Closeout + +`audit.md` is now the complete findings-only input to a separate remediation- +planning prompt. The later `implementation.md` pass must turn the ordering and +dependencies above into decision-complete work units. This audit does not +authorize or schedule code, test, fixture, contract, or durable-documentation +changes.