Complete the Stage 24 documentation audit

This commit is contained in:
2026-08-12 17:52:46 +00:00
parent fa19452dec
commit 4f52555389

View File

@@ -1,6 +1,6 @@
# Repository Audit Ledger
Status: In progress; Stages 1-23 complete.
Status: In progress; Stages 1-24 complete.
This temporary roadmap document is the evidence ledger for the staged audit
defined by [the audit plan](audit-plan.md). It records audit evidence and
@@ -196,6 +196,21 @@ labels. Report-specific generated types/builders, explicit prompt allowlist
projections, package-local safety copies and path diagnostics, config-to-
briefing validation, top-level workflow orchestration, and the current narrow
resolved-value clone remain intentionally explicit.
Stage 24 found five low-severity documentation-coherence issues: public and
internal CLI descriptions have drifted from executable help and comparison
dispatch; focused internal guides name a nonexistent forecast API, an absent
collection workflow, and report tests that do not exist; two completed
comparison roadmaps retain parallel current command and artifact definitions;
and the shared executor's concurrent-call requirement remains implicit rather
than part of its substitutability contract. The configuration reference also
omits accepted report-key normalization and timezone-offset bounds. Other
configuration fields/defaults, integration shapes, schema, template,
report/module/prompt identities, maintained examples, local and external
links, fence languages, security examples, and
the scoped atomic-visibility wording otherwise agree with their executable or
historical owners. Existing implementation defects that violate correctly
stated intended contracts remain assigned to their earlier findings rather
than being duplicated as documentation defects.
Subsystem conclusions and final disposition remain pending the later stages.
## Baseline Metadata
@@ -380,7 +395,7 @@ with evidence about meaningful risks, test ownership, gaps, and duplication.
| Test durability, duplication, and maintenance cost | Repository-wide suite | 21 | Partly insufficient. Most tests assert stable behavior with direct fixtures and appropriately narrow fakes. `AUD-039`, `AUD-061`, and `AUD-063` identify duplicated or implementation-coupled assertions; `AUD-062` identifies avoidable production waits; `AUD-064` identifies incomplete failure cleanup. Each consolidation retains a named behavioral owner. |
| Cross-cutting efficiency | Graph metrics, workflow traces, and focused latency/allocation diagnostics | 22 | Partly insufficient. Fixed-size registries, service-sized derivation, template parsing, batch composition, and ordinary profile fan-out are proportionate. `AUD-065` records one redundant Weather API request plus serial independent source fetches; `AUD-066` records repeated full comparison-bundle reads during replacement. |
| Cross-cutting maintainability | Graph similarity, change history, caller traces, and focused structural review | 23 | Partly insufficient. `AUD-067` through `AUD-073` identify duplicated canonicalization/template/date/requirement policy, parallel metadata authority, dormant internal surfaces, and unsafe duplicate capitalization. Important report, prompt-curation, safety-copy, validation, and orchestration similarities remain explicit under `RET-053` through `RET-058`. |
| Documentation and executable-contract coherence | Canonical documents, code, schemas, templates, examples | 24 | Pending |
| Documentation and executable-contract coherence | Canonical documents, code, schemas, templates, examples | 24 | Partly insufficient. `AUD-074` through `AUD-078` record CLI/help drift, stale internal flow/API/test claims, completed-roadmap duplication, an implicit concurrent-executor requirement, and incomplete configuration normalization/bounds. Other volatile identities and maintained examples match their executable owners; linked summaries, historical release notes, and the accepted ADR remain appropriately distinct. |
| Dynamic robustness and diagnostic checks | Repository-wide deterministic checks | 25 | Pending |
## Findings
@@ -3208,6 +3223,231 @@ with evidence about meaningful risks, test ownership, gaps, and duplication.
- Related findings: `AUD-024`, `AUD-025`, `AUD-067`
- Remediation reference: pending
### AUD-074: CLI help and internal guidance disagree with action behavior
- Stage: 24
- Status: candidate
- Severity: low
- Confidence: high
- Category: documentation
- Area: `internal/cli/root.go`, `internal/cli.writeActionResult`,
`docs/cli.md`, and `docs/internal/cli.md`
- Evidence: Executable root help includes `--date` in Daily, Today, and
comparison usage but omits it from the Options list. Its `--quiet` text says
that only successful action output is suppressed, while
`writeActionResult` returns before writing every result-bearing summary or
routine batch status, including failed summaries; the focused comparison
test confirms failed quiet execution. `docs/cli.md` first describes that
actual all-summary behavior, but its flag table repeats the successful-only
wording. `docs/internal/cli.md` additionally says dispatch is limited to
generation and batch even though `Runner.Run` has a complete comparison
branch and focused comparison command tests.
- Executable authority and canonical documentation owner: CLI parsing, root
help, dispatch, and result output in `internal/cli` are executable authority.
`docs/cli.md` owns the public flag/output contract and
`docs/internal/cli.md` owns dispatch mechanics.
- Contract at risk: Help and canonical CLI documentation must give one usable
accepted-flag matrix and one truthful account of which result channels quiet
mode suppresses.
- Impact: An operator can miss the date option in the normal Options reference
or expect a failed action summary under `--quiet`; a maintainer can also
follow an internal flow description that omits an implemented top-level
action.
- Recommendation: Add `--date` with its command eligibility to executable
help, describe `--quiet` as suppressing all action summaries and routine
batch status in help and the one public flag row, and update CLI internals to
include comparison dispatch and its result shape. Keep the complete
acceptance matrix only in `docs/cli.md` and link to it from internals.
- Test implications: Extend help coverage to require the date option and the
accurate quiet wording. Retain the existing behavioral quiet tests rather
than duplicating every failed action in help tests.
- Validation: Root and comparison help agree with `docs/cli.md`; generate,
run, and comparison failed-result tests remain silent under `--quiet`; and
CLI internals name all three action families.
- Related findings: `AUD-010`, `AUD-071`
- Remediation reference: pending
### AUD-075: Internal guides describe nonexistent APIs, flows, and tests
- Stage: 24
- Status: candidate
- Severity: low
- Confidence: high
- Category: documentation
- Area: `docs/internal/forecast-derivation.md`,
`docs/internal/collect.md`, `docs/internal/report-registry.md`, and
`docs/internal/prompt-input.md`
- Evidence: Forecast internals name `BuildPeriodDailySummaries` and
period-summary failure behavior, but graph and source search find only
`BuildDailySummary`; `facts.BuildDerived` intentionally builds one civil-day
summary for Daily, Today, and Tomorrow. Collection internals say generation,
batch, and "explicit bundle fetching" use `collect.Run`, while its sole
production caller is the default app collector used by the report workflows
and no explicit fetch command or application workflow exists. The report
guide says focused report tests cover run IDs, resolved output names,
composition defaults, and override validation, but `go test
./internal/report -list .` lists only five tests and does not cover Daily
run-ID uniqueness (`AUD-003`) or the claimed override surface. Prompt-input
internals likewise claim focused atomic-save tests even though
`promptinput.Save` has no caller or focused test (`AUD-001`).
- Executable authority and canonical documentation owner: The graph and
package symbols/tests in `internal/forecast`, `internal/facts`,
`internal/collect`, `internal/report`, and `internal/promptinput` are the
executable authorities. Their matching focused documents under
`docs/internal/` are the canonical documentation owners.
- Contract at risk: Focused internal guides should describe the implemented
package boundary and real verification inventory, not imply unavailable
entry points or protection that the suite does not provide.
- Impact: Maintainers can design against a nonexistent multi-day constructor
or fetch workflow and can overestimate regression protection for run
identity, overrides, or atomic prompt-package persistence.
- Recommendation: Replace the forecast description with the one-day
`BuildDailySummary`/facts flow, remove the absent explicit-fetch claim, and
narrow each verification paragraph to tests that exist. Coordinate the
prompt save/load wording with the dormant-surface decision in `AUD-072`
rather than documenting a speculative workflow.
- Test implications: Documentation correction does not itself require new
production tests. Add the Daily identity regression under `AUD-003`; remove
save/load-only claims or tests if `AUD-072` retires those APIs.
- Validation: Every named function and workflow has a production symbol and
caller, every claimed focused behavior is discoverable in the listed test
package, and internal-document links still resolve.
- Related findings: `AUD-001`, `AUD-003`, `AUD-072`
- Remediation reference: pending
### AUD-076: Completed comparison roadmaps duplicate current contracts
- Stage: 24
- Status: candidate
- Severity: low
- Confidence: high
- Category: documentation
- Area: `docs/roadmap/profile-comparison.md` and
`docs/roadmap/implementation.md`
- Evidence: Both roadmaps are marked implemented or complete, yet together
retain detailed current command syntax, accepted reports and flags, profile
count/order, destination selection, bundle layout and manifest fields,
failure/publication policy, concurrency, debug behavior, test requirements,
and documentation end state. Those volatile contracts are already owned by
`docs/cli.md`, `docs/operations.md`,
`docs/integrations/comparison-bundle.md`,
`docs/integrations/promptkit.md`, and focused comparison internals. The
implementation plan itself says the two files should not be archived as a
second current-state reference but defers cleanup, while the documentation
policy requires completed temporary roadmaps to be removed or archived
according to repository practice.
- Executable authority and canonical documentation owner: Comparison parsing,
execution, model validation, and publication code plus the embedded assets
are executable authority. The CLI, operations, integration, architecture,
and focused-internal documents assigned by
`docs/policy/documentation.md` are the canonical reader-facing owners;
completed roadmaps own neither current behavior nor durable history.
- Contract at risk: One volatile fact must have one current documentation
owner, and completed roadmaps must not remain parallel reference manuals.
- Impact: Later command, schema, or publication changes can update canonical
documents while leaving two credible, highly detailed stale definitions for
maintainers and coding agents.
- Recommendation: Remove the completed implementation plan and feature roadmap
after preserving any durable rationale that is not already in architecture,
an ADR, or current contracts. If repository practice requires an archive,
reduce it to non-authoritative historical status without the duplicated
reference definitions and remove incoming task-guide links.
- Test implications: No runtime test is required. Validate all incoming local
links after retirement and keep current comparison contract tests with their
executable owners.
- Validation: Searches find one canonical definition for each comparison
command, artifact, and lifecycle contract; active roadmaps contain only
future or audit coordination work; all repository links resolve.
- Related findings: none
- Remediation reference: pending
### AUD-077: Concurrent executor substitutability is only implicit
- Stage: 24
- Status: candidate
- Severity: low
- Confidence: high
- Category: documentation
- Area: `internal/promptexec.Executor`,
`docs/internal/comparison-execution.md`, and
`docs/integrations/promptkit.md`
- Evidence: Comparison starts one goroutine per selected profile and invokes
the same `promptexec.Executor` concurrently. The production Promptkit
adapter and race-tested fake support that use, and the two documentation
owners describe concurrent profile execution. The `Executor` interface
comment, however, specifies callback and result behavior without requiring
implementations to permit concurrent `Execute` calls. A conforming-looking
replacement can therefore be unsafe under an implemented caller pattern
that is not part of its written contract.
- Executable authority and canonical documentation owner: The comparison
execution call graph is executable authority for concurrent use;
`internal/promptexec.Executor` owns the project interface contract,
`docs/internal/comparison-execution.md` owns the internal scheduling flow,
and `docs/integrations/promptkit.md` owns the consumer-visible integration
boundary.
- Contract at risk: Every implementation of a shared project-owned interface
must know the concurrency behavior required for safe substitution.
- Impact: A future executor fake or adapter can serialize mutable state without
synchronization, pass ordinary single-report tests, and race or corrupt
comparison execution.
- Recommendation: State on `Executor` that comparison may call `Execute`
concurrently and implementations must be safe for that use. Add a concise
linked statement to comparison internals; keep backend capacity policy in
the Promptkit integration owner rather than duplicating it in the interface.
- Test implications: Retain the race-enabled comparison test with one shared
executor. New adapters should run that contract or an equivalent focused
concurrent-call test; no timing-only assertion is needed.
- Validation: The interface and internal guide explicitly state the concurrent
call requirement, the Promptkit adapter remains race-clean, and single-report
executors retain unchanged behavior.
- Related findings: `AUD-058`
- Remediation reference: pending
### AUD-078: Configuration normalization and timezone bounds are underdocumented
- Stage: 24
- Status: candidate
- Severity: low
- Confidence: high
- Category: documentation
- Area: `internal/report.IDForConfigKey`,
`internal/timeutil.parseUTCOffset`, and `docs/config.md`
- Evidence: The configuration reference says supported report keys treat
hyphens and underscores as equivalent. The executable lookup additionally
trims leading/trailing whitespace and folds case before normalization, and
focused lookup tests preserve that compatibility. The timezone reference
gives numeric-offset examples but no accepted bound; executable parsing
accepts signed hour forms or `HH:MM` through `23:59` and rejects larger hour
or minute components. These are accepted-value and validation rules owned by
configuration documentation, not incidental internal mechanics.
- Executable authority and canonical documentation owner: Report config-key
lookup in `internal/report` and timezone parsing in `internal/timeutil` are
executable authority. `docs/config.md` owns user-selectable configuration
values, normalization, and validation bounds.
- Contract at risk: Operators and compatibility reviewers need the canonical
configuration reference to describe the complete accepted spelling and
numeric range for user-controlled values.
- Impact: A configuration can rely on accepted case/whitespace normalization
without that compatibility being visible, and an offset can fail near the
parser boundary without the reference explaining the valid range. A later
cleanup could also narrow these behaviors as apparently undocumented
implementation details.
- Recommendation: Document trimming, case folding, and hyphen/underscore
equivalence for report keys, and state the signed numeric offset grammar and
current `23:59` component bound. If the unusually broad offset range is not
intended product policy, narrow code and docs together in a separately
reviewed behavior change rather than silently changing it during
documentation remediation.
- Test implications: Existing report lookup and timezone parsing tests remain
executable owners. Add only a documentation-validation assertion if the
repository later automates reference extraction; do not duplicate parser
tables in a prose test.
- Validation: Every documented report-key spelling and offset boundary is
accepted, representative out-of-range offsets are rejected, and the
configuration reference agrees with focused parser tests.
- Related findings: none
- Remediation reference: pending
## Retained Decisions
### RET-001: Keep the application package as the explicit composition owner
@@ -3897,6 +4137,45 @@ reference-bearing option is introduced, give module options an owner-defined
clone contract and make the registry/construction test prove every registered
shape participates.
### RET-059: Keep concise linked summaries at audience boundaries
The README's one-command orientation, development task router, operational
workflow snippets, configuration cross-references, template identity table,
and integration links repeat only enough stable context for their audience and
then point to the assigned canonical owner. Some identifiers necessarily
appear in both executable assets and their owning explanation, while module,
prompt, template, and schema names also appear where maintainers must keep a
cross-contract pairing aligned. This is useful navigation, not parallel
reference maintenance. Retain these short summaries and links; do not copy
complete flag, field, schema, status, or path tables into non-owning documents.
### RET-060: Keep intended safety contracts authoritative while fixing code
Several earlier findings are executable violations of correctly stated
architecture, operations, integration, or artifact contracts: Weather API and
Distributor endpoint/diagnostic safety, prompt/schema/catalog alignment,
ordinary output preservation and cancellation, comparison recognition and
transaction outcomes, and completed-profile failure identity. Rewriting those
documents to advertise the unsafe or internally inconsistent behavior would
turn defects into contracts. Retain the intended canonical statement, repair
the executable authority under `AUD-011`, `AUD-012`, `AUD-029` through
`AUD-031`, `AUD-036` through `AUD-040`, `AUD-043` through `AUD-050`, and
`AUD-053` through `AUD-058`, and review the owner in the same remediation for
any clarified limits or compatibility effect.
### RET-061: Keep guarantee language bounded to implemented scopes
Current documents use atomic and transactional language for completed-file or
directory namespace visibility and rollback; none promises file or parent
directory synchronization against power loss. Promptkit documentation assigns
backend capacity to Promptkit and describes concurrent human-selected profile
sets without promising arbitrary fan-out performance. The release procedure
names the six produced OS/architecture binaries without converting that build
matrix into a broader host-capability promise for every filesystem test.
Retain these bounded statements. Introduce crash durability, a profile-count
service level, or a supported-platform policy only with executable evidence
and an explicitly assigned owner.
## Open Questions
No Stage 1 open questions or unexplained baseline failures remain.
@@ -4310,6 +4589,24 @@ Stage 23 routed these investigation leads to their assigned later stages:
rejection and commit-time authorization in their current safety owners.
Stage 26 must sequence those behavior fixes before extracting helpers.
Stage 24 routed these investigation leads to their assigned later stages:
- Stage 25 should validate the documented accepted/rejected configuration and
CLI examples through the executable boundary while exercising its broader
adversarial corpus; Stage 24 found no reason to add live-provider or
credential-dependent documentation checks.
- Stage 26 should remediate `AUD-074`, `AUD-075`, `AUD-077`, and `AUD-078`
beside their executable owners so help, interface comments, tests, and
canonical docs change together. It should retire the two completed
comparison roadmaps under `AUD-076` only after checking every incoming link.
- Remediation of earlier implementation findings must preserve the intended
canonical safety contract under `RET-060`; do not rewrite documentation to
legitimize current leakage, cancellation, recognition, identity, or
publication defects.
- Dormant APIs and broad render surfaces remain current executable facts until
Stage 26 decides `AUD-069`/`AUD-072`. Their focused docs should narrow or
disappear with that code change, not ahead of it.
## Stage Log
### Stage 1: Establish The Baseline And Audit Ledger
@@ -5599,3 +5896,61 @@ Stage 23 routed these investigation leads to their assigned later stages:
- Retained decisions: `RET-053` through `RET-058`.
- Open questions: the four leads recorded above are routed to their assigned
later stages.
### Stage 24: Audit Documentation Coherence And Executable Contracts
- Status: Complete.
- Scope reviewed: `README.md`, all current-state, policy, integration,
internal, ADR, release, release-note, and roadmap documents under `docs/`,
all three maintained examples, executable CLI help, configuration/report/time
parsing, embedded prompt/profile/schema/template assets, focused test
inventories, local and external links, fenced examples, sensitive-value
patterns, and every documentation lead routed from Stages 3-23.
- Exclusions: no canonical document, executable help, interface comment,
production code, test, example, or completed roadmap was remediated. Dynamic
adversarial validation remains Stage 25; consolidation and remediation
sequencing remain Stages 26-27.
#### Documentation And Executable-Authority Accounting
| Topic | Executable authority, canonical documentation owner, and evidence | Disposition |
| --- | --- | --- |
| Product orientation and contributor routing | `cmd/weatherreporter` and the assembled workflow establish the product; `README.md` owns the concise orientation and `docs/development.md` owns task routing/package inventory. Their summaries link onward and do not reproduce complete volatile references. | Coherent linked summaries under `RET-059`. |
| CLI commands, flags, output, and dispatch | `internal/cli/root.go` and `writeActionResult` are executable authority; `docs/cli.md` owns the public contract and `docs/internal/cli.md` owns mechanics. Date/quiet help and comparison-dispatch statements disagree under `AUD-074`; accepted actions, report/date matrix, paths, summary fields, and failure exits otherwise match. | Partly insufficient; exact owners recorded in `AUD-074`. |
| Configuration fields, defaults, and examples | Config structs/defaults/loading/validation, `report.IDForConfigKey`, `timeutil.LoadLocation`, and the Promptkit adapter are executable authority; `docs/config.md` owns field behavior and `examples/` owns complete files. Defaults, precedence, fields, active/inactive validation, report/module IDs, profile precedence, and all three loaded/inspected examples match. Report-key normalization and numeric-offset bounds are incomplete under `AUD-078`. | Partly insufficient; exact owners recorded in `AUD-078`. |
| Report, module, prompt, profile, schema, and template identities | Report/briefing/generated-text registries and embedded prompt/profile/schema/template assets are executable authority; focused internal docs, `docs/templates.md`, and `docs/integrations/promptkit.md` own their reader-facing explanations. The four report/prompt/schema/template pairs, prompt version `2.0.0`, profile defaults/models, module catalog/compositions, partials, and schema fields match. Dormant/broad surfaces remain truthful current code until `AUD-069`/`AUD-072` remediation. | Coherent; no duplicate retirement finding. |
| Weather API and Distributor integrations | Config and both adapters plus the pinned Distributor module are executable authority; `docs/integrations/weatherapi.md`, `docs/integrations/distributor/`, and `docs/config.md` own wire and selection contracts. Endpoint paths/queries, envelopes, availability, retry statuses, bundle mappings, pinned version, upload/status flow, and logical paths match. Unsupported schemes and unsafe diagnostics violate intended contracts under `AUD-011`, `AUD-012`, and `AUD-048` through `AUD-050`; those executable owners should change, with canonical docs reviewed together under `RET-060`. | Earlier implementation findings retained; no duplicate documentation finding. |
| Prompt and rendering safety contracts | Embedded prompt/schema/template assets, prompt-input/validator/catalog code, and app preparation are executable authority; Promptkit integration, prompt-input/generated-text internals, template docs, and architecture own the written contracts. `AUD-029` through `AUD-031`, `AUD-036` through `AUD-040`, and `AUD-043`/`AUD-044` identify executable violations of correctly stated curation, identity, schema, and preflight expectations. | Repair executable owners and review canonical owners together under `RET-060`; do not document defects as supported behavior. |
| Output and comparison publication contracts | `internal/fileutil`, app publication, and `internal/comparison` are executable authority; `docs/cli.md`, `docs/operations.md`, `docs/integrations/comparison-bundle.md`, comparison internals, and architecture are canonical owners. `AUD-045`, `AUD-047`, and `AUD-053` through `AUD-056` identify executable violations of stated target-type, cancellation, recognition, filename, and recovery guarantees. Current atomic language promises namespace visibility, not power-loss durability. | Repair executable owners under `RET-060`; bounded terminology retained under `RET-061`. |
| Comparison execution and result identity | App scheduling/result projection is executable authority; CLI, Promptkit/comparison integration, and comparison internals own the written behavior. `AUD-058` owns the existing mixed failure/cancellation violation; `AUD-077` records the missing concurrent-call substitutability requirement. Current docs correctly avoid an arbitrary profile-count performance promise. | Partly insufficient under `AUD-077`; prior defect remains `AUD-058`; scaling scope retained under `RET-061`. |
| Focused internal flows and verification claims | Package symbols, callers, and `go test -list` inventories are executable authority; matching files under `docs/internal/` own implementation descriptions. Forecast, collection, report, and prompt-input claims name absent APIs/flows/tests under `AUD-075`; remaining internal flows agree with caller traces and focused suites. | Partly insufficient; exact owners recorded in `AUD-075`. |
| Roadmap, ADR, release-note, and release lifecycle | Documentation policy is the lifecycle authority. The accepted stateless ADR and versioned release notes are clearly historical, the release procedure owns current publication steps, and `docs/roadmap/future.md` labels proposals unimplemented. The implemented comparison roadmap and completed plan instead duplicate current volatile references under `AUD-076`. The active audit plan/ledger remain temporary coordination records. | Partly insufficient under `AUD-076`; historical distinctions otherwise coherent. |
| Links, examples, fences, and sensitive content | Every repository-relative Markdown link resolves; both external reference links returned HTTP 200; every fence is balanced and language-tagged. Maintained examples use synthetic `.example.com` endpoints or loopback, contain no credential values, and passed their owning suites. Security-pattern review found mechanisms/placeholders and the intentionally public module host, not private credentials, keys, environment dumps, or undisclosed infrastructure. | Coherent. |
#### Commands And Evidence
- Inventoried 46 documentation/example files and read every current-state
canonical owner, internal guide, maintained example, policy, ADR, release
procedure/note, and roadmap status relevant to Stage 24. Used prior-stage
findings for code behavior and graph search/snippets/traces only to resolve
disputed symbols, callers, and concurrency requirements.
- Compared root and comparison help with the public/internal CLI owners; ran
focused CLI and report test inventories; confirmed the missing forecast
symbol and absent explicit collection flow; and checked embedded prompt,
profile, schema, and template identities against their documentation.
- Ran focused configuration, Promptkit-adapter, report, and CLI suites; all
passed, including the maintained-example checks. A local-link checker found
no missing targets, a fence checker found no untagged or unclosed fences,
and both external NOAA/SPC references returned HTTP 200.
- Searched docs/examples for credentials, keys, tokens, authorization values,
endpoints, and private infrastructure. Only documented placeholders,
environment-variable names, synthetic endpoints, loopback, public upstream
references, and the repository's intentionally public module host appeared.
- Ran `go test ./...`, `go vet ./...`,
`go run ./cmd/weatherreporter --help`,
`go run ./cmd/weatherreporter compare --help`, and `git diff --check`; all
passed.
- Findings: `AUD-074` through `AUD-078`.
- Retained decisions: `RET-059` through `RET-061`.
- Open questions: none; remediation and dynamic-validation leads are routed
above without changing later-stage scope.