diff --git a/docs/roadmap/audit-plan.md b/docs/roadmap/audit-plan.md new file mode 100644 index 0000000..870704d --- /dev/null +++ b/docs/roadmap/audit-plan.md @@ -0,0 +1,992 @@ +# Repository Audit Plan + +Status: Ready for execution. + +## Purpose + +This document defines a staged audit of Weatherreporter for correctness, +efficiency, architectural coherence, refactoring and deduplication +opportunities, and compliance with the canonical testing policy. It is a +temporary coordination document under `docs/roadmap/`; it does not describe +implemented product behavior. + +The audit is intentionally divided into prompts that are narrow enough for one +LLM agent to complete with evidence and record before moving to the next +stage. Each stage reviews production behavior, its focused tests, and the +canonical documentation that owns the relevant contract. Later cross-cutting +stages then assess the suite and codebase as portfolios rather than repeating +the subsystem reviews. + +The expected production baseline at the time this plan was prepared is commit +`151c536` (`Add comparison diagnostics to the future roadmap`). Stage 1 must +record the actual baseline used for the audit and disclose any difference. + +## Audit Deliverables + +The audit produces two temporary roadmap documents: + +- `docs/roadmap/audit.md` is the evidence ledger and authoritative record of + scope, stage coverage, validation results, findings, retained design choices, + and final disposition. +- `docs/roadmap/audit-remediation.md` is created only after finding + verification and contains accepted remediation work divided into + independently implementable stages. + +Do not place future work or audit status in current-state architecture, +internal, integration, user, or operator documentation. When remediation is +complete, move durable facts to their canonical owners and remove the +temporary audit documents according to the documentation policy. + +## Audit Rules + +These rules apply to every stage: + +1. Read `docs/development.md`, the stage-specific documents it identifies, + `docs/policy/architecture.md`, and `docs/policy/testing.md` before reviewing + the affected code. Read `docs/policy/documentation.md` when assessing or + changing audit documents. +2. Prefer codebase-memory graph tools for code discovery, call tracing, + dependency analysis, similarity analysis, and hotspot discovery. Confirm + that the graph index represents the recorded baseline and refresh it when + necessary. Use text search for literals, errors, configuration, templates, + fixtures, and other non-code content. +3. Treat graph metrics, coverage percentages, static-analysis warnings, long + functions, and repeated text as investigation leads, not findings. A + finding requires evidence of a defect, material risk, policy violation, or + avoidable maintenance cost. +4. Review focused production code and focused tests together. Record both + missing protection and tests whose cost, duplication, or implementation + coupling exceeds their value. +5. Keep the discovery audit read-only except for the audit documents. Do not + fix production code, rewrite tests, update canonical documentation, change + dependencies, or reformat unrelated files during an audit stage. +6. Preserve all pre-existing user changes. If the worktree contains changes + outside the audit documents, record them as baseline exclusions and do not + modify them. +7. Use deterministic, offline validation. Do not use live Weather API, + Promptkit provider, or Distributor credentials or services. Do not install + new analysis tools merely to complete the audit. +8. Never expose credentials, endpoints, rendered prompts, schemas, provider + payloads, model output, private environment values, or other sensitive + diagnostics in the audit log. +9. Record important areas reviewed with no recommended action. In particular, + distinguish intentional explicitness from duplication that should be + abstracted. +10. Complete and record one stage before starting the next. A stage should not + commit, tag, push, or create a release unless its execution prompt + separately authorizes that action. + +If a candidate requires investigation too large for its assigned stage, +record the evidence and unanswered question without guessing. Stage 26 must +either reject it, resolve it from existing evidence, or insert one narrowly +scoped supplemental investigation stage before final synthesis. + +## Finding Model + +Use stable sequential IDs such as `AUD-001`. Never recycle an ID after a +finding is rejected or consolidated. + +Every finding in `docs/roadmap/audit.md` must use this structure: + +```markdown +### AUD-001: Concise title + +- Stage: 5 +- Status: candidate | confirmed | accepted | deferred | rejected | resolved +- Severity: critical | high | medium | low +- Confidence: high | medium | low +- Category: correctness | efficiency | architecture | duplication | testing | documentation | security +- Area: package, file, symbol, workflow, or contract +- Evidence: exact code paths, traces, tests, commands, or reproduction +- Contract at risk: documented behavior, policy, invariant, or caller expectation +- Impact: realistic failure mode or maintenance cost +- Recommendation: smallest credible corrective direction +- Test implications: missing, misplaced, redundant, brittle, or sufficient protection +- Validation: evidence that would prove remediation complete +- Related findings: none or stable finding IDs +- Remediation reference: pending or roadmap stage/commit +``` + +Severity and confidence are independent: + +- Critical means likely credential disclosure, destructive data loss, broad + corruption, or inability to trust a published artifact. +- High means an important public contract, safety invariant, concurrency rule, + or common workflow can fail materially. +- Medium means a bounded correctness problem or substantial recurring + maintenance and drift risk. +- Low means a localized problem with modest impact or a worthwhile cleanup + that is safe to defer. + +Use high confidence only for a reproduction, direct invariant violation, +unambiguous path analysis, or equally strong evidence. Refactoring preference +alone is not a correctness finding. + +Retained decisions should be recorded separately from findings with IDs such +as `RET-001`. Each retained decision should state what was inspected, why the +current explicit or duplicated-looking design is appropriate, and what future +condition would justify reconsideration. + +## Standard Stage Procedure + +Unless a stage says otherwise, its single prompt should: + +1. Review the relevant canonical documentation and existing audit entries. +2. Use the graph to inventory definitions, callers, callees, dependencies, + high-complexity paths, and tests in scope. +3. Inspect the implementation and tests at the narrowest stable boundaries. +4. Run focused deterministic tests or diagnostic commands where they add + evidence. +5. Append findings, retained decisions, open questions, commands, and a concise + no-findings statement where appropriate to `docs/roadmap/audit.md`. +6. Update the audit coverage matrix and mark only the current stage complete. +7. Run `git diff --check` on the audit documents and stop. + +A stage is complete only when its scoped areas are accounted for in the log; +passing tests alone is not completion. + +## Stage 1: Establish The Baseline And Audit Ledger + +### Goal + +Create a reproducible audit baseline and the structured log used by every +later stage. + +### Scope + +- Current commit, branch, worktree, Go version, module/workspace context, and + codebase-memory index identity. +- Repository package, source-file, test-file, fixture, embedded-asset, and + canonical-document inventory. +- Baseline contributor validation. + +### Work + +1. Verify the expected production baseline or record why the audit uses a + different commit. Record and exclude any unrelated worktree changes. +2. Create `docs/roadmap/audit.md` with metadata, scope and exclusions, + validation results, an executive-summary placeholder, a stage coverage + matrix, a risk-to-test coverage matrix, findings and retained-decisions + sections, and open questions. +3. Confirm or refresh the repository graph index and record its project name. +4. Run the canonical baseline commands from `docs/development.md` and + `go vet ./...`. Record exact commands, pass/fail status, and bounded failure + evidence; do not fix failures. + +### Exit Evidence + +- The audit is tied to an exact production baseline. +- Every later stage appears in the coverage matrix as pending. +- Baseline failures, environmental limitations, and exclusions are explicit. + +## Stage 2: Audit Architecture And Dependency Direction + +### Goal + +Determine whether the implemented package graph and principal workflows match +the architecture policy before auditing local details. + +### Scope + +- `cmd/weatherreporter` and all `internal` package-level import and call + boundaries. +- `docs/policy/architecture.md`, `docs/development.md`, and applicable ADRs. +- Entry points, external adapters, dependency-neutral contracts, publication + boundaries, and stateless-execution invariants. + +### Work + +Use architecture, import, call-path, and data-flow views to identify dependency +cycles, policy flowing into the wrong package, external types escaping adapter +boundaries, duplicated orchestration ownership, implicit durable state, and +unusually coupled packages. Trace representative generate, batch, compare, +collection, publication, and notification paths without attempting detailed +subsystem review. + +### Exit Evidence + +- Every normative architecture boundary has an implementation owner or a + recorded finding. +- Suspected local problems are routed to their later subsystem stage rather + than over-investigated here. + +## Stage 3: Audit Report Identity And Time Foundations + +### Goal + +Review the small, high-fan-in foundations that define report identity, valid +periods, dates, clocks, and timezones. + +### Scope + +- `internal/report` +- `internal/timeutil` +- Related portions of `docs/cli.md`, `docs/config.md`, + `docs/internal/report-registry.md`, and focused tests. + +### Work + +Check alias resolution, registry consistency, batch membership, prompt and +template identities, period policies, daylight-saving behavior, date parsing, +clock control, timezone failures, boundary dates, and repeated mapping logic. +Assess whether tests own durable contracts rather than private registry shape. + +### Exit Evidence + +- Report identity and temporal rules are traced to their callers. +- Boundary and DST risks have sufficient evidence or findings. + +## Stage 4: Audit Configuration, Secrets, And Validation + +### Goal + +Verify configuration precedence, normalization, validation, secrets handling, +and rendering of configured paths and notification values. + +### Scope + +- `internal/config` +- `examples/` +- `docs/config.md` and configuration-related architecture and operations rules. +- Focused configuration and Promptkit configuration tests. + +### Work + +Review defaults, file parsing, CLI override application, unknown fields, +report/module overrides, Promptkit settings, secret-directory behavior, +environment interaction, URL and filesystem validation, notification +templates, error identity, redaction, and consistency between loaded and +programmatically constructed configurations. Check examples against the +executable schema without contacting external systems. + +### Exit Evidence + +- Precedence and validation paths are explicitly mapped. +- Secret-bearing values and failure diagnostics are accounted for. +- Repeated config policy has one owner or a finding. + +## Stage 5: Audit CLI Parsing, Wiring, And Output Contracts + +### Goal + +Verify that command parsing, executor construction, output routing, summaries, +quiet mode, and exit behavior implement the documented CLI contract. + +### Scope + +- `cmd/weatherreporter` +- `internal/cli`, excluding the detailed comparison execution behavior saved + for Stage 18. +- `docs/cli.md`, `docs/internal/cli.md`, and representative CLI tests. + +### Work + +Review command/flag ownership, validation ordering, common option reuse, +stdout/stderr separation, safe structured errors, quiet behavior, executor +construction frequency, cancellation propagation, help output, and result to +exit-status mapping. Look for parser-policy duplication and tests coupled to +incidental wording or wiring. + +### Exit Evidence + +- Every non-comparison command and major failure class has a documented result + and exit path. +- CLI tests are assessed at the parser, summary, and assembled-workflow + boundaries without duplicating lower-layer cases. + +## Stage 6: Audit Weather Data Acquisition And Collection + +### Goal + +Verify the external Weather API boundary and conversion into normalized, +app-facing collected data. + +### Scope + +- `internal/adapters/weatherapi` +- `internal/weatherdata` +- `internal/collect` +- `docs/integrations/weatherapi.md`, `docs/internal/weather-data.md`, and + `docs/internal/collect.md`. +- Adapter fixtures and focused tests. + +### Work + +Review endpoint construction, timeouts, HTTP status handling, decoding, +required versus optional sources, `null` and malformed payload behavior, +source provenance and timestamps, warnings, units, cancellation, response body +lifecycle, input limits where applicable, and normalization boundaries. Check +that fixtures are minimal, synthetic, and credential-free. Treat similar +source-fetch code as intentional unless semantics are truly identical. + +### Exit Evidence + +- Each source has an explicit required/optional and failure policy. +- Transport, decoding, normalization, and orchestration tests have distinct + ownership. + +## Stage 7: Audit Forecast And Fact Derivation + +### Goal + +Review deterministic meteorological derivation for correctness, boundary +behavior, and unnecessary repeated computation. + +### Scope + +- `internal/forecast` +- `internal/facts` +- `docs/internal/forecast-derivation.md` and `docs/internal/facts.md`. +- Focused derivation tests. + +### Work + +Trace normalized inputs through periods, daily summaries, dayparts, alert +overlap, precipitation, collected facts, and derived facts. Review empty and +partial inputs, units, time boundaries, ordering, determinism, aliasing and +mutation, numerical comparisons, and hidden nested work. Judge test cases by +meteorological risk and invariant coverage rather than line coverage. + +### Exit Evidence + +- Important derivation rules and edge cases have named test owners. +- Any performance concern identifies a realistic input size and repeated work. + +## Stage 8: Audit Module Contracts, Registry, And Source-Facing Briefing Modules + +### Goal + +Review module composition and the first half of briefing construction without +combining it with the more complex planning and SPC rules. + +### Scope + +- `internal/module` +- `internal/briefing/modules.go`, `package.go`, `metadata_module.go`, + `current_conditions_module.go`, `narrative_forecast_module.go`, + `hourly_forecast_module.go`, `area_forecast_discussion_module.go`, + `weather_story_module.go`, `alert_digest_module.go`, and their directly + related tests. +- `docs/internal/module.md` and relevant portions of + `docs/internal/briefing.md` and `docs/internal/prompt-input.md`. + +### Work + +Review registry initialization, module IDs, supported reports, dependency and +ordering rules, missing-data policy, option schemas, snapshots, exports, +source-to-module transformations, error context, and mutable/shared state. +Inspect `BuildModule` branches in this limited scope and record complexity +drivers for Stage 23. + +### Exit Evidence + +- Registry metadata, builders, and config validation agree for the modules in + scope. +- Module tests protect public composition and value contracts rather than the + switch structure itself. + +## Stage 9: Audit Derived, Planning, Formatting, And SPC Briefing Modules + +### Goal + +Complete the briefing audit with the modules containing denser domain and +presentation rules. + +### Scope + +- Remaining `internal/briefing` implementation and tests, including daily and + daypart summaries, Daily/Today/Tomorrow planning, outdoor windows, + precipitation timing, SPC outlooks and discussions, summary helpers, format + helpers, and embedded SPC definitions. +- Remaining relevant portions of `docs/internal/briefing.md`. + +### Work + +Review domain thresholds, ordering, date and overlap rules, option handling, +missing data, definition lookup, user-facing value construction, stable prompt +exports, repeated day-style logic, and formatting policy embedded in helpers. +Check fixtures and table tests for meaningful categories and boundary values. + +### Exit Evidence + +- Every remaining default module and embedded definition source is covered by + the audit ledger. +- Similar planning modules are classified as appropriate explicit variation or + a concrete drift risk. + +## Stage 10: Audit Prompt Inputs, Assets, And Neutral Execution Contracts + +### Goal + +Verify the curated prompt-data boundary and dependency-neutral execution model. + +### Scope + +- `internal/promptinput` +- `internal/promptassets` +- `internal/promptexec` +- `docs/internal/prompt-input.md`, applicable generated-text contract docs, and + focused tests. + +### Work + +Review package construction, allowed exports, deterministic YAML, ordering, +normalization, serialization failure, embedded prompt/schema identity, +inspection and execution request/result contracts, error categories, +cancellation semantics, and safe error projection. Inspect high-depth marshal +logic for realistic complexity, not metric value alone. + +### Exit Evidence + +- Raw weather and dependency-specific Promptkit types do not cross the curated + boundary. +- Serialization and error contracts have durable, non-duplicated tests. + +## Stage 11: Audit Promptkit Adaptation And Secure Prompt Debugging + +### Goal + +Review the external Promptkit integration and explicit sensitive-debug path as +one security-sensitive boundary. + +### Scope + +- `internal/adapters/promptkit` +- `internal/promptdebug` +- `docs/integrations/promptkit.md`, `docs/internal/promptkit-adapter.md`, and + relevant operations guidance. +- Focused adapter and debug tests. + +### Work + +Check type containment, configuration translation, exact prompt/profile +inspection, credential reporting, execution and cancellation, concurrency +safety, error classification and redaction, explicit debug-root authorization, +path containment, symlink rejection, file modes, collision handling, partial +writes, and callback failure behavior. Do not record sensitive fixture values. + +### Exit Evidence + +- Normal results and logs cannot expose content-rich or credential-bearing + Promptkit details. +- Debug filesystem and concurrency invariants have deterministic local tests. + +## Stage 12: Audit Generated-Text Validation And Catalog Contracts + +### Goal + +Verify generated prose schemas, strict decoding, normalization, validation, +and report-to-handler catalog compatibility separately from rendering. + +### Scope + +- `internal/generatedtext` except `render_context.go` and its focused tests. +- Generated-text schemas and their ownership in `internal/reporttemplate`. +- `docs/internal/generatedtext.md` and relevant integration contracts. + +### Work + +Review Daily, Today, Tomorrow, and Hourly types; strict JSON behavior; required +and optional fields; whitespace normalization; unknown fields; array and size +behavior; schema pairing; catalog lookup; error safety; and deliberate shared +day-style helpers. Check that similar public contracts remain independently +protected without semantically duplicating all implementation cases. + +### Exit Evidence + +- Each report's raw-output-to-validated-value path is explicit. +- Go validation, embedded schema behavior, and tests agree or have findings. + +## Stage 13: Audit Render Contexts, Templates, And Markdown Rendering + +### Goal + +Verify deterministic context assembly and repository-owned Markdown output. + +### Scope + +- `internal/generatedtext/render_context.go` and focused tests. +- `internal/reporttemplate`, embedded schemas/templates/partials, and tests. +- `docs/internal/reporttemplate.md`, relevant generated-text documentation, and + `docs/templates.md`. + +### Work + +Review module-to-context mapping, report identity checks, deterministic facts, +generated prose slots, missing values, escaping and formatting, template +lookup, schema-template pairing, partial behavior, output determinism, and +error context. Assess golden or complete-output assertions for semantic review +value and incidental whitespace coupling. + +### Exit Evidence + +- Every template field has one authoritative producer. +- Deterministic facts cannot be silently replaced by generated prose. +- Shared template/context mechanics do not erase intentional report variation. + +## Stage 14: Audit Application Preparation And Prompt Preflight + +### Goal + +Review reusable application mechanics that resolve and prepare a report before +publication or batch policy is applied. + +### Scope + +- `internal/app/prepared_report.go` +- `internal/app/prompt_inspection.go` +- `internal/app/prompt_generate.go` +- `internal/app/profile_execution.go` +- Their focused tests and relevant portions of + `docs/internal/prepared-report.md` and app orchestration documentation. + +### Work + +Trace validation ordering, collection boundaries, immutable preparation, +single-build guarantees, prompt/profile inspection, executor requests, +generated-text handling, rendering, cancellation, debug callbacks, error +classification, and partial result construction. Check aliasing and shared +state assumptions used by both ordinary and comparison workflows. + +### Exit Evidence + +- Preconditions that must fail before collection or model execution are + enumerated and tested at one owning layer. +- Prepared inputs are demonstrably immutable or a finding explains the risk. + +## Stage 15: Audit Single-Report Generation And Atomic Output + +### Goal + +Verify the complete ordinary generation workflow and operator-owned Markdown +publication. + +### Scope + +- `internal/app/app.go` single-report paths +- `internal/app/output.go` +- `internal/fileutil` +- Generation and output tests. +- Relevant portions of `docs/internal/app-orchestration.md` and + `docs/operations.md`. + +### Work + +Review stage ordering, result truthfulness, destination validation, atomic +writes, cancellation immediately before publication, preservation of existing +outputs, temporary cleanup, path handling, file modes, post-publication error +semantics, and error wrapping. Trace success and each consequential failure +path through the CLI result without re-auditing CLI parsing. + +### Exit Evidence + +- The publication linearization point and cancellation behavior are explicit. +- Pre-publication failures preserve prior output and tests prove that invariant + through a stable filesystem boundary. + +## Stage 16: Audit Batch Orchestration And Distributor Notification + +### Goal + +Verify batch planning, sequential execution, partial failures, publication +accounting, and notification coordination. + +### Scope + +- `internal/app/batch_plan.go`, `batch_notification.go`, and batch paths in + `app.go` +- Batch tests +- `internal/adapters/distributor` +- `docs/internal/app-orchestration.md`, + `docs/internal/distributor-adapter.md`, `docs/integrations/distributor/`, and + relevant operations rules. + +### Work + +Review validate-before-collection and destination-planning guarantees, +collect-once behavior, report ordering, continued independent execution, +counters, partial results, all-success notification gate, upload source +selection, exactly-once interaction requirements, cancellation, response body +handling, safe failures, and separation of report versus notification failure. + +### Exit Evidence + +- Every batch outcome maps consistently to published files, counters, + notification state, safe summaries, and exit status. +- Distributor tests use realistic local boundaries without duplicating app + policy. + +## Stage 17: Audit Comparison Contracts And Transactional Publication + +### Goal + +Review comparison identity, durable bundle compatibility, destination +recognition, and destructive replacement safety independently of concurrent +execution. + +### Scope + +- `internal/comparison` +- `docs/integrations/comparison-bundle.md` and + `docs/internal/comparison-publication.md`. +- Focused comparison model and publication tests. + +### Work + +Review schema validation, ordered profile filenames, path normalization, +hashing, manifest encoding, safe errors, destination classification, staging, +authorization at commit time, symlink and broad-target rejection, replacement, +rollback, retained backups, cleanup failure truthfulness, cancellation +linearization, and recovery diagnostics. + +### Exit Evidence + +- Every filesystem mutation has an explicit authorization and recovery rule. +- Durable compatibility and transaction state are protected at the package + boundary with deterministic filesystem seams. + +## Stage 18: Audit Comparison Execution And CLI Integration + +### Goal + +Verify prepare-once concurrent profile execution and its end-to-end result and +failure behavior. + +### Scope + +- `internal/app/comparison.go` +- `internal/app/comparison_execution.go` +- Comparison-related app tests +- Comparison-specific paths and tests in `internal/cli` +- `docs/internal/comparison-execution.md`, relevant app orchestration and CLI + contracts, and the comparison bundle contract. + +### Work + +Review ordered explicit profiles, sequential preflight, one collection and +preparation, immutable identical YAML, one executor, goroutine lifecycle, +backend-owned capacity, cancellation, isolated profile failures, deterministic +result ordering, debug identities, coherent partial bundle publication, +committed-cleanup failures, never-notify policy, safe summaries, and exit +status. + +### Exit Evidence + +- Completion order cannot change durable or CLI ordering. +- All goroutines are joined and shared values are concurrency-safe. +- Preflight, profile-local, publication, and post-commit failure classes remain + distinct and truthfully represented. + +## Stage 19: Audit Test Hermeticity And Execution Hygiene + +### Goal + +Assess suite-wide compliance with deterministic, offline, repeatable execution +requirements without yet judging coverage sufficiency. + +### Scope + +- All Go tests, test fixtures, helper scripts, embedded test assets, and golden + update mechanisms. +- Execution Requirements, Test Types And Assets, Use Test Doubles Deliberately, + and Go-Specific Guidance in `docs/policy/testing.md`. + +### Work + +Inventory external networking, subprocesses, environment reads/writes, +process-global mutation, clocks, randomness, sleeps, deadlines, filesystem +roots, permissions assumptions, credentials, `t.Parallel`, goroutine cleanup, +goldens, and test ordering dependencies. Use targeted searches to locate +candidates, then inspect each candidate in context. Do not label all sleeps, +subprocesses, or serial tests as violations automatically. + +### Exit Evidence + +- Every potentially nondeterministic or machine-specific test mechanism is + classified as controlled, justified, or a finding. +- Live infrastructure and secret independence are explicitly confirmed or + disproved. + +## Stage 20: Audit Test Risk Coverage And Ownership + +### Goal + +Determine whether consequential contracts and failure modes are protected at +the narrowest stable layer, independent of coverage percentage. + +### Scope + +- The full test suite viewed as a risk portfolio. +- The architecture policy's invariants and the `What Deserves Tests`, `Choose + The Right Boundary`, `Avoid Semantic Duplication`, and `Definition Of + Sufficient` sections of the testing policy. + +### Work + +Complete the risk-to-test matrix created in Stage 1 for CLI, configuration, +domain rules, adapters, prompt contracts, schemas/templates, atomic output, +comparison replacement, concurrency, cancellation, partial success, +notification, compatibility, security, idempotency, and recovery. For each +risk, identify the owning test boundary, meaningful negative paths, and any +unnecessary higher-level repetition. Use coverage only to locate surprising +gaps in critical branches. + +### Exit Evidence + +- Every important invariant has a credible test owner, an accepted rationale + for omission, or a finding. +- Missing tests are described by the defect they would catch, not by uncovered + lines. + +## Stage 21: Audit Test Durability, Duplication, And Maintenance Cost + +### Goal + +Identify tests whose lifetime cost exceeds their protection and opportunities +to consolidate tests without weakening confidence. + +### Scope + +- The full suite, with emphasis on large fixtures, dense test files, + cross-layer duplication, and tests associated with high-change code. +- `Test Behavior, Not Implementation`, `Separate Mechanism From Policy`, + `Regression Tests`, and `Deleting Or Rewriting Tests` in the testing policy. + +### Work + +Review exact error-string assertions, duplicated defaults and thresholds, +private-helper coupling, mock choreography, repeated workflow fixtures, +complete-output snapshots, redundant tables, oversize helper frameworks, and +regression tests subsumed by stronger invariants. Use graph test relationships +and change history as discovery signals. For every proposed deletion or +consolidation, name the protected behavior and where equivalent or stronger +protection would remain. + +### Exit Evidence + +- Test cleanup findings state both confidence retained and maintenance cost + removed. +- Tests are not criticized merely for being numerous, explicit, or + integration-style. + +## Stage 22: Audit Cross-Cutting Efficiency And Complexity + +### Goal + +Find credible efficiency problems across subsystem boundaries after local +behavior is understood. + +### Scope + +- Graph complexity and loop-depth candidates. +- Generate, batch, comparison, collection, module construction, + serialization, schema validation, and publication hot paths. +- Allocation, repeated parsing or registry construction, avoidable I/O, + redundant work, algorithmic scaling, and concurrency coordination. + +### Work + +Trace the highest-complexity and highest-fan-in functions and follow repeated +work through representative workflows. Establish realistic input cardinality +and frequency before recording a finding. Run a focused benchmark or temporary +diagnostic only when it can distinguish a meaningful issue; do not retain +temporary code. Separate latency dominated by external boundaries from local +algorithmic cost. + +### Exit Evidence + +- Each efficiency finding includes a workload model and measurable or + defensible impact. +- Metric-only candidates are rejected or retained as watch items rather than + promoted to findings. + +## Stage 23: Audit Cross-Cutting Refactoring And Deduplication Opportunities + +### Goal + +Identify behavior-preserving structural improvements that local stages may not +reveal, while avoiding abstractions that erase domain distinctions. + +### Scope + +- Similar code and semantic graph clusters across packages. +- Repeated mappings, validation, path construction, result translation, error + projection, report variants, module variants, workflow stages, and test + fixtures. +- Package APIs, responsibility boundaries, and change-coupled files. + +### Work + +Compare structurally or semantically similar implementations and determine +whether they share policy, merely resemble one another, or are intentionally +explicit. Require a concrete drift history, repeated change burden, ownership +ambiguity, or correctness risk for a refactoring finding. Prefer the smallest +shared helper or clearer owner; do not propose broad frameworks, package +proliferation, or public API consolidation without commensurate evidence. + +### Exit Evidence + +- Every refactoring proposal names behavior that must remain stable and a + proportionate test boundary. +- Important false-positive duplication candidates are recorded as retained + decisions. + +## Stage 24: Audit Documentation Coherence And Executable Contracts + +### Goal + +Verify that documentation used as audit authority still matches executable +behavior and follows canonical ownership rules. + +### Scope + +- `README.md`, `docs/`, and `examples/`. +- Commands, flags, config fields/defaults, report/module/prompt/template/schema + IDs, logical artifact paths, status and exit behavior, integration shapes, + internal flows, links, and roadmap lifecycle. +- `docs/policy/documentation.md`. + +### Work + +Use findings from subsystem stages rather than rereading implementation from +scratch. Check volatile claims against executable authorities, test links and +maintained examples, identify parallel definitions and stale current-state +claims, confirm future work remains in roadmaps, and inspect examples and docs +for secrets or private infrastructure. Record the canonical owner that should +change during remediation; do not update it during the audit. + +### Exit Evidence + +- Each material contract disagreement identifies both executable authority and + canonical documentation owner. +- Documentation duplication is distinguished from useful linked summaries. + +## Stage 25: Run Dynamic Robustness And Final Diagnostic Validation + +### Goal + +Use repository-wide dynamic checks to expose concurrency, order dependence, +flakiness, and unexpectedly weak critical packages after static review is +complete. + +### Scope + +- Entire default test suite and CLI help path. +- Race-enabled execution, randomized test order, repeated execution, vet, and + diagnostic coverage. + +### Work + +Run and record, subject to environmental support: + +```sh +go test ./... +go test -race ./... +go test -shuffle=on -count=3 ./... +go vet ./... +go test -coverprofile=coverage.out ./... +go run ./cmd/weatherreporter --help +git diff --check +``` + +Remove the diagnostic coverage artifact after inspecting package and critical +branch signals. Investigate failures only far enough to classify and record +them; do not change production or test code. If a full command is infeasible, +record the reason and run the narrowest meaningful substitute. + +### Exit Evidence + +- Every command has a result in the audit log. +- Failures are reproducible or explicitly classified as environmental or + intermittent candidates. +- No generated diagnostic artifacts remain in the worktree. + +## Stage 26: Verify, Consolidate, And Triage Findings + +### Goal + +Turn accumulated candidates into a coherent, evidence-backed final finding +set before prescribing implementation work. + +### Scope + +- Every finding, retained decision, open question, validation result, and + coverage-matrix entry in `docs/roadmap/audit.md`. + +### Work + +Reproduce or re-check high-impact candidates, merge duplicates, link related +root causes, resolve conflicting recommendations, calibrate severity and +confidence, and change each candidate to confirmed, deferred, or rejected. +Verify that recommendations name the smallest corrective direction without +prematurely specifying an implementation. Insert a supplemental investigation +stage if a potentially high-impact question still lacks enough evidence. + +### Exit Evidence + +- No candidate status remains. +- Every rejected finding explains why the evidence or cost-benefit case was + insufficient. +- Every confirmed finding has actionable validation and test implications. + +## Stage 27: Produce The Remediation Roadmap And Close The Audit + +### Goal + +Publish the final audit summary and translate accepted findings into safe, +reviewable implementation stages. + +### Scope + +- Confirmed findings and retained decisions. +- Final coverage and risk matrices. +- `docs/roadmap/audit-remediation.md`. + +### Work + +1. Write the audit executive summary, overall assessment, reviewed scope, + exclusions, validation summary, and prioritized finding table. +2. Decide which confirmed findings are accepted now, explicitly deferred, or + require a separate product/architecture decision. +3. Create the remediation roadmap with correctness, security, data integrity, + and destructive-operation fixes before efficiency or cleanup work. Divide + it into dependency-ordered stages, each sized for one LLM prompt with + affected areas, locked behavior, tests, documentation owners, and exit + commands. +4. Keep behavior-preserving refactors separate from intentional contract + changes. Do not bundle unrelated findings merely because they touch the same + package. +5. Record how the audit documents will be retired after remediation and where + durable decisions or contracts must move. +6. Run `git diff --check` and verify that only intended audit documents have + changed. + +### Exit Evidence + +- The audit ledger is complete and contains no unresolved scope gaps. +- Every accepted finding maps to exactly one remediation stage or an explicit + prerequisite decision. +- Every remediation stage is independently executable and verifiable in one + prompt. +- No production code, tests, dependencies, or canonical current-state + documentation changed during the discovery audit. + +## Final Audit Acceptance Criteria + +The audit is complete when: + +- all 27 planned stages, plus any explicitly inserted supplemental + investigation stages, are complete in the coverage matrix; +- the repository baseline and exclusions are reproducible; +- correctness and safety findings are supported by direct evidence; +- efficiency findings include realistic workload impact; +- refactoring findings identify concrete drift or maintenance cost; +- important intentional duplication and explicitness are recorded; +- the test suite has been assessed for hermeticity, risk sufficiency, ownership, + durability, and semantic duplication under the canonical testing policy; +- executable contracts and canonical documentation have been reconciled in the + findings; +- repository-wide validation results are recorded without leaving diagnostic + artifacts; and +- accepted work is represented by a dependency-ordered, one-prompt-per-stage + remediation roadmap.