eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Add audit workflow plan

Browse Source
This commit is contained in:
Eric Rakestraw
2026-08-12 14:52:26 +00:00
parent 151c536cb9
commit e7c7262404
1 changed files with 992 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

992
docs/roadmap/audit-plan.md Normal file
View File

@@ -0,0 +1,992 @@
# Repository Audit Plan
Status: Ready for execution.
## Purpose
This document defines a staged audit of Weatherreporter for correctness,
efficiency, architectural coherence, refactoring and deduplication
opportunities, and compliance with the canonical testing policy. It is a
temporary coordination document under `docs/roadmap/`; it does not describe
implemented product behavior.
The audit is intentionally divided into prompts that are narrow enough for one
LLM agent to complete with evidence and record before moving to the next
stage. Each stage reviews production behavior, its focused tests, and the
canonical documentation that owns the relevant contract. Later cross-cutting
stages then assess the suite and codebase as portfolios rather than repeating
the subsystem reviews.
The expected production baseline at the time this plan was prepared is commit
`151c536` (`Add comparison diagnostics to the future roadmap`). Stage 1 must
record the actual baseline used for the audit and disclose any difference.
## Audit Deliverables
The audit produces two temporary roadmap documents:
- `docs/roadmap/audit.md` is the evidence ledger and authoritative record of
scope, stage coverage, validation results, findings, retained design choices,
and final disposition.
- `docs/roadmap/audit-remediation.md` is created only after finding
verification and contains accepted remediation work divided into
independently implementable stages.
Do not place future work or audit status in current-state architecture,
internal, integration, user, or operator documentation. When remediation is
complete, move durable facts to their canonical owners and remove the
temporary audit documents according to the documentation policy.
## Audit Rules
These rules apply to every stage:
1. Read `docs/development.md`, the stage-specific documents it identifies,
`docs/policy/architecture.md`, and `docs/policy/testing.md` before reviewing
the affected code. Read `docs/policy/documentation.md` when assessing or
changing audit documents.
2. Prefer codebase-memory graph tools for code discovery, call tracing,
dependency analysis, similarity analysis, and hotspot discovery. Confirm
that the graph index represents the recorded baseline and refresh it when
necessary. Use text search for literals, errors, configuration, templates,
fixtures, and other non-code content.
3. Treat graph metrics, coverage percentages, static-analysis warnings, long
functions, and repeated text as investigation leads, not findings. A
finding requires evidence of a defect, material risk, policy violation, or
avoidable maintenance cost.
4. Review focused production code and focused tests together. Record both
missing protection and tests whose cost, duplication, or implementation
coupling exceeds their value.
5. Keep the discovery audit read-only except for the audit documents. Do not
fix production code, rewrite tests, update canonical documentation, change
dependencies, or reformat unrelated files during an audit stage.
6. Preserve all pre-existing user changes. If the worktree contains changes
outside the audit documents, record them as baseline exclusions and do not
modify them.
7. Use deterministic, offline validation. Do not use live Weather API,
Promptkit provider, or Distributor credentials or services. Do not install
new analysis tools merely to complete the audit.
8. Never expose credentials, endpoints, rendered prompts, schemas, provider
payloads, model output, private environment values, or other sensitive
diagnostics in the audit log.
9. Record important areas reviewed with no recommended action. In particular,
distinguish intentional explicitness from duplication that should be
abstracted.
10. Complete and record one stage before starting the next. A stage should not
commit, tag, push, or create a release unless its execution prompt
separately authorizes that action.
If a candidate requires investigation too large for its assigned stage,
record the evidence and unanswered question without guessing. Stage 26 must
either reject it, resolve it from existing evidence, or insert one narrowly
scoped supplemental investigation stage before final synthesis.
## Finding Model
Use stable sequential IDs such as `AUD-001`. Never recycle an ID after a
finding is rejected or consolidated.
Every finding in `docs/roadmap/audit.md` must use this structure:
```markdown
### AUD-001: Concise title
- Stage: 5
- Status: candidate | confirmed | accepted | deferred | rejected | resolved
- Severity: critical | high | medium | low
- Confidence: high | medium | low
- Category: correctness | efficiency | architecture | duplication | testing | documentation | security
- Area: package, file, symbol, workflow, or contract
- Evidence: exact code paths, traces, tests, commands, or reproduction
- Contract at risk: documented behavior, policy, invariant, or caller expectation
- Impact: realistic failure mode or maintenance cost
- Recommendation: smallest credible corrective direction
- Test implications: missing, misplaced, redundant, brittle, or sufficient protection
- Validation: evidence that would prove remediation complete
- Related findings: none or stable finding IDs
- Remediation reference: pending or roadmap stage/commit
```
Severity and confidence are independent:
- Critical means likely credential disclosure, destructive data loss, broad
corruption, or inability to trust a published artifact.
- High means an important public contract, safety invariant, concurrency rule,
or common workflow can fail materially.
- Medium means a bounded correctness problem or substantial recurring
maintenance and drift risk.
- Low means a localized problem with modest impact or a worthwhile cleanup
that is safe to defer.
Use high confidence only for a reproduction, direct invariant violation,
unambiguous path analysis, or equally strong evidence. Refactoring preference
alone is not a correctness finding.
Retained decisions should be recorded separately from findings with IDs such
as `RET-001`. Each retained decision should state what was inspected, why the
current explicit or duplicated-looking design is appropriate, and what future
condition would justify reconsideration.
## Standard Stage Procedure
Unless a stage says otherwise, its single prompt should:
1. Review the relevant canonical documentation and existing audit entries.
2. Use the graph to inventory definitions, callers, callees, dependencies,
high-complexity paths, and tests in scope.
3. Inspect the implementation and tests at the narrowest stable boundaries.
4. Run focused deterministic tests or diagnostic commands where they add
evidence.
5. Append findings, retained decisions, open questions, commands, and a concise
no-findings statement where appropriate to `docs/roadmap/audit.md`.
6. Update the audit coverage matrix and mark only the current stage complete.
7. Run `git diff --check` on the audit documents and stop.
A stage is complete only when its scoped areas are accounted for in the log;
passing tests alone is not completion.
## Stage 1: Establish The Baseline And Audit Ledger
### Goal
Create a reproducible audit baseline and the structured log used by every
later stage.
### Scope
- Current commit, branch, worktree, Go version, module/workspace context, and
codebase-memory index identity.
- Repository package, source-file, test-file, fixture, embedded-asset, and
canonical-document inventory.
- Baseline contributor validation.
### Work
1. Verify the expected production baseline or record why the audit uses a
different commit. Record and exclude any unrelated worktree changes.
2. Create `docs/roadmap/audit.md` with metadata, scope and exclusions,
validation results, an executive-summary placeholder, a stage coverage
matrix, a risk-to-test coverage matrix, findings and retained-decisions
sections, and open questions.
3. Confirm or refresh the repository graph index and record its project name.
4. Run the canonical baseline commands from `docs/development.md` and
`go vet ./...`. Record exact commands, pass/fail status, and bounded failure
evidence; do not fix failures.
### Exit Evidence
- The audit is tied to an exact production baseline.
- Every later stage appears in the coverage matrix as pending.
- Baseline failures, environmental limitations, and exclusions are explicit.
## Stage 2: Audit Architecture And Dependency Direction
### Goal
Determine whether the implemented package graph and principal workflows match
the architecture policy before auditing local details.
### Scope
- `cmd/weatherreporter` and all `internal` package-level import and call
boundaries.
- `docs/policy/architecture.md`, `docs/development.md`, and applicable ADRs.
- Entry points, external adapters, dependency-neutral contracts, publication
boundaries, and stateless-execution invariants.
### Work
Use architecture, import, call-path, and data-flow views to identify dependency
cycles, policy flowing into the wrong package, external types escaping adapter
boundaries, duplicated orchestration ownership, implicit durable state, and
unusually coupled packages. Trace representative generate, batch, compare,
collection, publication, and notification paths without attempting detailed
subsystem review.
### Exit Evidence
- Every normative architecture boundary has an implementation owner or a
recorded finding.
- Suspected local problems are routed to their later subsystem stage rather
than over-investigated here.
## Stage 3: Audit Report Identity And Time Foundations
### Goal
Review the small, high-fan-in foundations that define report identity, valid
periods, dates, clocks, and timezones.
### Scope
- `internal/report`
- `internal/timeutil`
- Related portions of `docs/cli.md`, `docs/config.md`,
`docs/internal/report-registry.md`, and focused tests.
### Work
Check alias resolution, registry consistency, batch membership, prompt and
template identities, period policies, daylight-saving behavior, date parsing,
clock control, timezone failures, boundary dates, and repeated mapping logic.
Assess whether tests own durable contracts rather than private registry shape.
### Exit Evidence
- Report identity and temporal rules are traced to their callers.
- Boundary and DST risks have sufficient evidence or findings.
## Stage 4: Audit Configuration, Secrets, And Validation
### Goal
Verify configuration precedence, normalization, validation, secrets handling,
and rendering of configured paths and notification values.
### Scope
- `internal/config`
- `examples/`
- `docs/config.md` and configuration-related architecture and operations rules.
- Focused configuration and Promptkit configuration tests.
### Work
Review defaults, file parsing, CLI override application, unknown fields,
report/module overrides, Promptkit settings, secret-directory behavior,
environment interaction, URL and filesystem validation, notification
templates, error identity, redaction, and consistency between loaded and
programmatically constructed configurations. Check examples against the
executable schema without contacting external systems.
### Exit Evidence
- Precedence and validation paths are explicitly mapped.
- Secret-bearing values and failure diagnostics are accounted for.
- Repeated config policy has one owner or a finding.
## Stage 5: Audit CLI Parsing, Wiring, And Output Contracts
### Goal
Verify that command parsing, executor construction, output routing, summaries,
quiet mode, and exit behavior implement the documented CLI contract.
### Scope
- `cmd/weatherreporter`
- `internal/cli`, excluding the detailed comparison execution behavior saved
for Stage 18.
- `docs/cli.md`, `docs/internal/cli.md`, and representative CLI tests.
### Work
Review command/flag ownership, validation ordering, common option reuse,
stdout/stderr separation, safe structured errors, quiet behavior, executor
construction frequency, cancellation propagation, help output, and result to
exit-status mapping. Look for parser-policy duplication and tests coupled to
incidental wording or wiring.
### Exit Evidence
- Every non-comparison command and major failure class has a documented result
and exit path.
- CLI tests are assessed at the parser, summary, and assembled-workflow
boundaries without duplicating lower-layer cases.
## Stage 6: Audit Weather Data Acquisition And Collection
### Goal
Verify the external Weather API boundary and conversion into normalized,
app-facing collected data.
### Scope
- `internal/adapters/weatherapi`
- `internal/weatherdata`
- `internal/collect`
- `docs/integrations/weatherapi.md`, `docs/internal/weather-data.md`, and
`docs/internal/collect.md`.
- Adapter fixtures and focused tests.
### Work
Review endpoint construction, timeouts, HTTP status handling, decoding,
required versus optional sources, `null` and malformed payload behavior,
source provenance and timestamps, warnings, units, cancellation, response body
lifecycle, input limits where applicable, and normalization boundaries. Check
that fixtures are minimal, synthetic, and credential-free. Treat similar
source-fetch code as intentional unless semantics are truly identical.
### Exit Evidence
- Each source has an explicit required/optional and failure policy.
- Transport, decoding, normalization, and orchestration tests have distinct
ownership.
## Stage 7: Audit Forecast And Fact Derivation
### Goal
Review deterministic meteorological derivation for correctness, boundary
behavior, and unnecessary repeated computation.
### Scope
- `internal/forecast`
- `internal/facts`
- `docs/internal/forecast-derivation.md` and `docs/internal/facts.md`.
- Focused derivation tests.
### Work
Trace normalized inputs through periods, daily summaries, dayparts, alert
overlap, precipitation, collected facts, and derived facts. Review empty and
partial inputs, units, time boundaries, ordering, determinism, aliasing and
mutation, numerical comparisons, and hidden nested work. Judge test cases by
meteorological risk and invariant coverage rather than line coverage.
### Exit Evidence
- Important derivation rules and edge cases have named test owners.
- Any performance concern identifies a realistic input size and repeated work.
## Stage 8: Audit Module Contracts, Registry, And Source-Facing Briefing Modules
### Goal
Review module composition and the first half of briefing construction without
combining it with the more complex planning and SPC rules.
### Scope
- `internal/module`
- `internal/briefing/modules.go`, `package.go`, `metadata_module.go`,
`current_conditions_module.go`, `narrative_forecast_module.go`,
`hourly_forecast_module.go`, `area_forecast_discussion_module.go`,
`weather_story_module.go`, `alert_digest_module.go`, and their directly
related tests.
- `docs/internal/module.md` and relevant portions of
`docs/internal/briefing.md` and `docs/internal/prompt-input.md`.
### Work
Review registry initialization, module IDs, supported reports, dependency and
ordering rules, missing-data policy, option schemas, snapshots, exports,
source-to-module transformations, error context, and mutable/shared state.
Inspect `BuildModule` branches in this limited scope and record complexity
drivers for Stage 23.
### Exit Evidence
- Registry metadata, builders, and config validation agree for the modules in
scope.
- Module tests protect public composition and value contracts rather than the
switch structure itself.
## Stage 9: Audit Derived, Planning, Formatting, And SPC Briefing Modules
### Goal
Complete the briefing audit with the modules containing denser domain and
presentation rules.
### Scope
- Remaining `internal/briefing` implementation and tests, including daily and
daypart summaries, Daily/Today/Tomorrow planning, outdoor windows,
precipitation timing, SPC outlooks and discussions, summary helpers, format
helpers, and embedded SPC definitions.
- Remaining relevant portions of `docs/internal/briefing.md`.
### Work
Review domain thresholds, ordering, date and overlap rules, option handling,
missing data, definition lookup, user-facing value construction, stable prompt
exports, repeated day-style logic, and formatting policy embedded in helpers.
Check fixtures and table tests for meaningful categories and boundary values.
### Exit Evidence
- Every remaining default module and embedded definition source is covered by
the audit ledger.
- Similar planning modules are classified as appropriate explicit variation or
a concrete drift risk.
## Stage 10: Audit Prompt Inputs, Assets, And Neutral Execution Contracts
### Goal
Verify the curated prompt-data boundary and dependency-neutral execution model.
### Scope
- `internal/promptinput`
- `internal/promptassets`
- `internal/promptexec`
- `docs/internal/prompt-input.md`, applicable generated-text contract docs, and
focused tests.
### Work
Review package construction, allowed exports, deterministic YAML, ordering,
normalization, serialization failure, embedded prompt/schema identity,
inspection and execution request/result contracts, error categories,
cancellation semantics, and safe error projection. Inspect high-depth marshal
logic for realistic complexity, not metric value alone.
### Exit Evidence
- Raw weather and dependency-specific Promptkit types do not cross the curated
boundary.
- Serialization and error contracts have durable, non-duplicated tests.
## Stage 11: Audit Promptkit Adaptation And Secure Prompt Debugging
### Goal
Review the external Promptkit integration and explicit sensitive-debug path as
one security-sensitive boundary.
### Scope
- `internal/adapters/promptkit`
- `internal/promptdebug`
- `docs/integrations/promptkit.md`, `docs/internal/promptkit-adapter.md`, and
relevant operations guidance.
- Focused adapter and debug tests.
### Work
Check type containment, configuration translation, exact prompt/profile
inspection, credential reporting, execution and cancellation, concurrency
safety, error classification and redaction, explicit debug-root authorization,
path containment, symlink rejection, file modes, collision handling, partial
writes, and callback failure behavior. Do not record sensitive fixture values.
### Exit Evidence
- Normal results and logs cannot expose content-rich or credential-bearing
Promptkit details.
- Debug filesystem and concurrency invariants have deterministic local tests.
## Stage 12: Audit Generated-Text Validation And Catalog Contracts
### Goal
Verify generated prose schemas, strict decoding, normalization, validation,
and report-to-handler catalog compatibility separately from rendering.
### Scope
- `internal/generatedtext` except `render_context.go` and its focused tests.
- Generated-text schemas and their ownership in `internal/reporttemplate`.
- `docs/internal/generatedtext.md` and relevant integration contracts.
### Work
Review Daily, Today, Tomorrow, and Hourly types; strict JSON behavior; required
and optional fields; whitespace normalization; unknown fields; array and size
behavior; schema pairing; catalog lookup; error safety; and deliberate shared
day-style helpers. Check that similar public contracts remain independently
protected without semantically duplicating all implementation cases.
### Exit Evidence
- Each report's raw-output-to-validated-value path is explicit.
- Go validation, embedded schema behavior, and tests agree or have findings.
## Stage 13: Audit Render Contexts, Templates, And Markdown Rendering
### Goal
Verify deterministic context assembly and repository-owned Markdown output.
### Scope
- `internal/generatedtext/render_context.go` and focused tests.
- `internal/reporttemplate`, embedded schemas/templates/partials, and tests.
- `docs/internal/reporttemplate.md`, relevant generated-text documentation, and
`docs/templates.md`.
### Work
Review module-to-context mapping, report identity checks, deterministic facts,
generated prose slots, missing values, escaping and formatting, template
lookup, schema-template pairing, partial behavior, output determinism, and
error context. Assess golden or complete-output assertions for semantic review
value and incidental whitespace coupling.
### Exit Evidence
- Every template field has one authoritative producer.
- Deterministic facts cannot be silently replaced by generated prose.
- Shared template/context mechanics do not erase intentional report variation.
## Stage 14: Audit Application Preparation And Prompt Preflight
### Goal
Review reusable application mechanics that resolve and prepare a report before
publication or batch policy is applied.
### Scope
- `internal/app/prepared_report.go`
- `internal/app/prompt_inspection.go`
- `internal/app/prompt_generate.go`
- `internal/app/profile_execution.go`
- Their focused tests and relevant portions of
`docs/internal/prepared-report.md` and app orchestration documentation.
### Work
Trace validation ordering, collection boundaries, immutable preparation,
single-build guarantees, prompt/profile inspection, executor requests,
generated-text handling, rendering, cancellation, debug callbacks, error
classification, and partial result construction. Check aliasing and shared
state assumptions used by both ordinary and comparison workflows.
### Exit Evidence
- Preconditions that must fail before collection or model execution are
enumerated and tested at one owning layer.
- Prepared inputs are demonstrably immutable or a finding explains the risk.
## Stage 15: Audit Single-Report Generation And Atomic Output
### Goal
Verify the complete ordinary generation workflow and operator-owned Markdown
publication.
### Scope
- `internal/app/app.go` single-report paths
- `internal/app/output.go`
- `internal/fileutil`
- Generation and output tests.
- Relevant portions of `docs/internal/app-orchestration.md` and
`docs/operations.md`.
### Work
Review stage ordering, result truthfulness, destination validation, atomic
writes, cancellation immediately before publication, preservation of existing
outputs, temporary cleanup, path handling, file modes, post-publication error
semantics, and error wrapping. Trace success and each consequential failure
path through the CLI result without re-auditing CLI parsing.
### Exit Evidence
- The publication linearization point and cancellation behavior are explicit.
- Pre-publication failures preserve prior output and tests prove that invariant
through a stable filesystem boundary.
## Stage 16: Audit Batch Orchestration And Distributor Notification
### Goal
Verify batch planning, sequential execution, partial failures, publication
accounting, and notification coordination.
### Scope
- `internal/app/batch_plan.go`, `batch_notification.go`, and batch paths in
`app.go`
- Batch tests
- `internal/adapters/distributor`
- `docs/internal/app-orchestration.md`,
`docs/internal/distributor-adapter.md`, `docs/integrations/distributor/`, and
relevant operations rules.
### Work
Review validate-before-collection and destination-planning guarantees,
collect-once behavior, report ordering, continued independent execution,
counters, partial results, all-success notification gate, upload source
selection, exactly-once interaction requirements, cancellation, response body
handling, safe failures, and separation of report versus notification failure.
### Exit Evidence
- Every batch outcome maps consistently to published files, counters,
notification state, safe summaries, and exit status.
- Distributor tests use realistic local boundaries without duplicating app
policy.
## Stage 17: Audit Comparison Contracts And Transactional Publication
### Goal
Review comparison identity, durable bundle compatibility, destination
recognition, and destructive replacement safety independently of concurrent
execution.
### Scope
- `internal/comparison`
- `docs/integrations/comparison-bundle.md` and
`docs/internal/comparison-publication.md`.
- Focused comparison model and publication tests.
### Work
Review schema validation, ordered profile filenames, path normalization,
hashing, manifest encoding, safe errors, destination classification, staging,
authorization at commit time, symlink and broad-target rejection, replacement,
rollback, retained backups, cleanup failure truthfulness, cancellation
linearization, and recovery diagnostics.
### Exit Evidence
- Every filesystem mutation has an explicit authorization and recovery rule.
- Durable compatibility and transaction state are protected at the package
boundary with deterministic filesystem seams.
## Stage 18: Audit Comparison Execution And CLI Integration
### Goal
Verify prepare-once concurrent profile execution and its end-to-end result and
failure behavior.
### Scope
- `internal/app/comparison.go`
- `internal/app/comparison_execution.go`
- Comparison-related app tests
- Comparison-specific paths and tests in `internal/cli`
- `docs/internal/comparison-execution.md`, relevant app orchestration and CLI
contracts, and the comparison bundle contract.
### Work
Review ordered explicit profiles, sequential preflight, one collection and
preparation, immutable identical YAML, one executor, goroutine lifecycle,
backend-owned capacity, cancellation, isolated profile failures, deterministic
result ordering, debug identities, coherent partial bundle publication,
committed-cleanup failures, never-notify policy, safe summaries, and exit
status.
### Exit Evidence
- Completion order cannot change durable or CLI ordering.
- All goroutines are joined and shared values are concurrency-safe.
- Preflight, profile-local, publication, and post-commit failure classes remain
distinct and truthfully represented.
## Stage 19: Audit Test Hermeticity And Execution Hygiene
### Goal
Assess suite-wide compliance with deterministic, offline, repeatable execution
requirements without yet judging coverage sufficiency.
### Scope
- All Go tests, test fixtures, helper scripts, embedded test assets, and golden
update mechanisms.
- Execution Requirements, Test Types And Assets, Use Test Doubles Deliberately,
and Go-Specific Guidance in `docs/policy/testing.md`.
### Work
Inventory external networking, subprocesses, environment reads/writes,
process-global mutation, clocks, randomness, sleeps, deadlines, filesystem
roots, permissions assumptions, credentials, `t.Parallel`, goroutine cleanup,
goldens, and test ordering dependencies. Use targeted searches to locate
candidates, then inspect each candidate in context. Do not label all sleeps,
subprocesses, or serial tests as violations automatically.
### Exit Evidence
- Every potentially nondeterministic or machine-specific test mechanism is
classified as controlled, justified, or a finding.
- Live infrastructure and secret independence are explicitly confirmed or
disproved.
## Stage 20: Audit Test Risk Coverage And Ownership
### Goal
Determine whether consequential contracts and failure modes are protected at
the narrowest stable layer, independent of coverage percentage.
### Scope
- The full test suite viewed as a risk portfolio.
- The architecture policy's invariants and the `What Deserves Tests`, `Choose
The Right Boundary`, `Avoid Semantic Duplication`, and `Definition Of
Sufficient` sections of the testing policy.
### Work
Complete the risk-to-test matrix created in Stage 1 for CLI, configuration,
domain rules, adapters, prompt contracts, schemas/templates, atomic output,
comparison replacement, concurrency, cancellation, partial success,
notification, compatibility, security, idempotency, and recovery. For each
risk, identify the owning test boundary, meaningful negative paths, and any
unnecessary higher-level repetition. Use coverage only to locate surprising
gaps in critical branches.
### Exit Evidence
- Every important invariant has a credible test owner, an accepted rationale
for omission, or a finding.
- Missing tests are described by the defect they would catch, not by uncovered
lines.
## Stage 21: Audit Test Durability, Duplication, And Maintenance Cost
### Goal
Identify tests whose lifetime cost exceeds their protection and opportunities
to consolidate tests without weakening confidence.
### Scope
- The full suite, with emphasis on large fixtures, dense test files,
cross-layer duplication, and tests associated with high-change code.
- `Test Behavior, Not Implementation`, `Separate Mechanism From Policy`,
`Regression Tests`, and `Deleting Or Rewriting Tests` in the testing policy.
### Work
Review exact error-string assertions, duplicated defaults and thresholds,
private-helper coupling, mock choreography, repeated workflow fixtures,
complete-output snapshots, redundant tables, oversize helper frameworks, and
regression tests subsumed by stronger invariants. Use graph test relationships
and change history as discovery signals. For every proposed deletion or
consolidation, name the protected behavior and where equivalent or stronger
protection would remain.
### Exit Evidence
- Test cleanup findings state both confidence retained and maintenance cost
removed.
- Tests are not criticized merely for being numerous, explicit, or
integration-style.
## Stage 22: Audit Cross-Cutting Efficiency And Complexity
### Goal
Find credible efficiency problems across subsystem boundaries after local
behavior is understood.
### Scope
- Graph complexity and loop-depth candidates.
- Generate, batch, comparison, collection, module construction,
serialization, schema validation, and publication hot paths.
- Allocation, repeated parsing or registry construction, avoidable I/O,
redundant work, algorithmic scaling, and concurrency coordination.
### Work
Trace the highest-complexity and highest-fan-in functions and follow repeated
work through representative workflows. Establish realistic input cardinality
and frequency before recording a finding. Run a focused benchmark or temporary
diagnostic only when it can distinguish a meaningful issue; do not retain
temporary code. Separate latency dominated by external boundaries from local
algorithmic cost.
### Exit Evidence
- Each efficiency finding includes a workload model and measurable or
defensible impact.
- Metric-only candidates are rejected or retained as watch items rather than
promoted to findings.
## Stage 23: Audit Cross-Cutting Refactoring And Deduplication Opportunities
### Goal
Identify behavior-preserving structural improvements that local stages may not
reveal, while avoiding abstractions that erase domain distinctions.
### Scope
- Similar code and semantic graph clusters across packages.
- Repeated mappings, validation, path construction, result translation, error
projection, report variants, module variants, workflow stages, and test
fixtures.
- Package APIs, responsibility boundaries, and change-coupled files.
### Work
Compare structurally or semantically similar implementations and determine
whether they share policy, merely resemble one another, or are intentionally
explicit. Require a concrete drift history, repeated change burden, ownership
ambiguity, or correctness risk for a refactoring finding. Prefer the smallest
shared helper or clearer owner; do not propose broad frameworks, package
proliferation, or public API consolidation without commensurate evidence.
### Exit Evidence
- Every refactoring proposal names behavior that must remain stable and a
proportionate test boundary.
- Important false-positive duplication candidates are recorded as retained
decisions.
## Stage 24: Audit Documentation Coherence And Executable Contracts
### Goal
Verify that documentation used as audit authority still matches executable
behavior and follows canonical ownership rules.
### Scope
- `README.md`, `docs/`, and `examples/`.
- Commands, flags, config fields/defaults, report/module/prompt/template/schema
IDs, logical artifact paths, status and exit behavior, integration shapes,
internal flows, links, and roadmap lifecycle.
- `docs/policy/documentation.md`.
### Work
Use findings from subsystem stages rather than rereading implementation from
scratch. Check volatile claims against executable authorities, test links and
maintained examples, identify parallel definitions and stale current-state
claims, confirm future work remains in roadmaps, and inspect examples and docs
for secrets or private infrastructure. Record the canonical owner that should
change during remediation; do not update it during the audit.
### Exit Evidence
- Each material contract disagreement identifies both executable authority and
canonical documentation owner.
- Documentation duplication is distinguished from useful linked summaries.
## Stage 25: Run Dynamic Robustness And Final Diagnostic Validation
### Goal
Use repository-wide dynamic checks to expose concurrency, order dependence,
flakiness, and unexpectedly weak critical packages after static review is
complete.
### Scope
- Entire default test suite and CLI help path.
- Race-enabled execution, randomized test order, repeated execution, vet, and
diagnostic coverage.
### Work
Run and record, subject to environmental support:
```sh
go test ./...
go test -race ./...
go test -shuffle=on -count=3 ./...
go vet ./...
go test -coverprofile=coverage.out ./...
go run ./cmd/weatherreporter --help
git diff --check
```
Remove the diagnostic coverage artifact after inspecting package and critical
branch signals. Investigate failures only far enough to classify and record
them; do not change production or test code. If a full command is infeasible,
record the reason and run the narrowest meaningful substitute.
### Exit Evidence
- Every command has a result in the audit log.
- Failures are reproducible or explicitly classified as environmental or
intermittent candidates.
- No generated diagnostic artifacts remain in the worktree.
## Stage 26: Verify, Consolidate, And Triage Findings
### Goal
Turn accumulated candidates into a coherent, evidence-backed final finding
set before prescribing implementation work.
### Scope
- Every finding, retained decision, open question, validation result, and
coverage-matrix entry in `docs/roadmap/audit.md`.
### Work
Reproduce or re-check high-impact candidates, merge duplicates, link related
root causes, resolve conflicting recommendations, calibrate severity and
confidence, and change each candidate to confirmed, deferred, or rejected.
Verify that recommendations name the smallest corrective direction without
prematurely specifying an implementation. Insert a supplemental investigation
stage if a potentially high-impact question still lacks enough evidence.
### Exit Evidence
- No candidate status remains.
- Every rejected finding explains why the evidence or cost-benefit case was
insufficient.
- Every confirmed finding has actionable validation and test implications.
## Stage 27: Produce The Remediation Roadmap And Close The Audit
### Goal
Publish the final audit summary and translate accepted findings into safe,
reviewable implementation stages.
### Scope
- Confirmed findings and retained decisions.
- Final coverage and risk matrices.
- `docs/roadmap/audit-remediation.md`.
### Work
1. Write the audit executive summary, overall assessment, reviewed scope,
exclusions, validation summary, and prioritized finding table.
2. Decide which confirmed findings are accepted now, explicitly deferred, or
require a separate product/architecture decision.
3. Create the remediation roadmap with correctness, security, data integrity,
and destructive-operation fixes before efficiency or cleanup work. Divide
it into dependency-ordered stages, each sized for one LLM prompt with
affected areas, locked behavior, tests, documentation owners, and exit
commands.
4. Keep behavior-preserving refactors separate from intentional contract
changes. Do not bundle unrelated findings merely because they touch the same
package.
5. Record how the audit documents will be retired after remediation and where
durable decisions or contracts must move.
6. Run `git diff --check` and verify that only intended audit documents have
changed.
### Exit Evidence
- The audit ledger is complete and contains no unresolved scope gaps.
- Every accepted finding maps to exactly one remediation stage or an explicit
prerequisite decision.
- Every remediation stage is independently executable and verifiable in one
prompt.
- No production code, tests, dependencies, or canonical current-state
documentation changed during the discovery audit.
## Final Audit Acceptance Criteria
The audit is complete when:
- all 27 planned stages, plus any explicitly inserted supplemental
investigation stages, are complete in the coverage matrix;
- the repository baseline and exclusions are reproducible;
- correctness and safety findings are supported by direct evidence;
- efficiency findings include realistic workload impact;
- refactoring findings identify concrete drift or maintenance cost;
- important intentional duplication and explicitness are recorded;
- the test suite has been assessed for hermeticity, risk sufficiency, ownership,
durability, and semantic duplication under the canonical testing policy;
- executable contracts and canonical documentation have been reconciled in the
findings;
- repository-wide validation results are recorded without leaving diagnostic
artifacts; and
- accepted work is represented by a dependency-ordered, one-prompt-per-stage
remediation roadmap.