6244 lines
457 KiB
Markdown
6244 lines
457 KiB
Markdown
# Repository Audit Ledger
|
|
|
|
Status: Complete; all 27 planned stages complete.
|
|
|
|
This temporary roadmap document is the evidence ledger for the staged audit
|
|
defined by [the audit plan](audit-plan.md). It records audit evidence and
|
|
status, not implemented product behavior. Current contracts remain with the
|
|
canonical owners identified by the
|
|
[documentation policy](../policy/documentation.md).
|
|
|
|
## Executive Summary
|
|
|
|
Stage 1 established a clean, reproducible baseline. The repository-wide test
|
|
suite, CLI help check, formatting check, and vet check all pass. Stage 2 found
|
|
that the implemented package graph and principal workflows follow the intended
|
|
dependency direction and assigned ownership. It recorded one low-severity
|
|
candidate finding for unused internal persistence helpers left outside the
|
|
documented stateless workflows. Stage 3 found one medium-severity correctness
|
|
defect in wall-clock window construction on daylight-saving transition days
|
|
and one medium-severity test gap around Daily run-ID uniqueness. Report
|
|
registry identities, command and configuration lookup policies, batch
|
|
membership, civil-day periods, and rolling-hour periods otherwise match their
|
|
documented contracts. Stage 4 found three medium-severity validation defects:
|
|
unsupported missing-source keys are silently accepted, two single-report
|
|
notification identities may render blank, and programmatically constructed
|
|
report overrides are silently ignored. It also found one low-severity secret
|
|
loading atomicity defect. Defaults, file and CLI precedence, strict YAML field
|
|
handling, active Promptkit settings, output-path preservation, and maintained
|
|
examples otherwise match their contracts. Stage 5 found two medium-severity
|
|
CLI correctness defects: operating-system
|
|
interrupts never become workflow cancellation, and generate validates report
|
|
dates only after constructing its executor. It also found a medium-severity
|
|
test gap around assembled generate and ordinary batch result/exit behavior.
|
|
Command ownership, flag acceptance, output routing, quiet mode, comparison
|
|
error bounding, and the implemented result-to-exit mapping otherwise follow
|
|
the documented contract. Stage 6 found four medium-severity defects at the
|
|
Weather API boundary: unsupported URL schemes pass validation until transport,
|
|
non-2xx response bodies flow verbatim into normal diagnostics, the nominal
|
|
response limit silently truncates instead of rejecting oversized bodies, and
|
|
required hourly products accept periods without usable time bounds. It also
|
|
found one low-severity retry-policy defect because warmup retries permanent
|
|
HTTP failures. Source availability policy, checked-empty products, query
|
|
construction, cancellation, provenance, normalization ownership, and focused
|
|
offline fixtures otherwise match their contracts. Stage 7 found three
|
|
medium-severity derivation defects: metric fallback values lose their unit and
|
|
are compared or presented as US customary values, overnight dayparts omit
|
|
alerts that begin after midnight, and precipitation probabilities outside the
|
|
percentage domain flow into summaries. It also found one low-severity text
|
|
classification defect because ordinary “icy” conditions do not set the ice
|
|
indicator. Half-open period selection, deterministic ordering, alert parsing,
|
|
precipitation-window grouping, checked-empty SPC handling, report-family fact
|
|
selection, and realistic derivation cost otherwise match their contracts.
|
|
Stage 8 found two medium-severity module-boundary defects: pointer-shaped AFD
|
|
options pass registry/config validation but fail during construction, and an
|
|
empty weather-story object is published as available. It also found one
|
|
low-severity snapshot compatibility defect because validation accepts any
|
|
nonempty schema version. Default composition, supported-report rules,
|
|
source-module omission, checked-empty alerts, ordered snapshots, and curated
|
|
current/hourly prompt exports otherwise match their contracts. Stage 9 found
|
|
six medium-severity briefing defects: apparent temperature is
|
|
published as heat index, normalized daypart keys can overwrite one another,
|
|
planning applies configurable daypart names inconsistently, outdoor-window
|
|
ranking ignores snow, ice, and fog, subzero temperatures are described as
|
|
positive zero-degree bands, and embedded descriptions labeled as official SPC
|
|
definitions have drifted from the current SPC category table. Derived
|
|
precipitation timing, deterministic display ordering, missing-data behavior,
|
|
SPC overlap and threshold selection, and the intended Today versus
|
|
Daily/Tomorrow planning variation otherwise match their contracts. Stage 10
|
|
found two high-severity prompt-contract defects: the shared prompt names
|
|
nonexistent paths for primary derived facts and alert metadata, and it
|
|
incorrectly characterizes every retained SPC product as location-matched. It
|
|
also found three low-severity boundary and contract gaps: raw source-warning
|
|
types and transport paths cross into provider input, the exported YAML loader
|
|
accepts several malformed package shapes, and stanza serialization failures
|
|
lack a focused regression test. Prompt/profile/schema identities,
|
|
deterministic category ordering, neutral execution lifecycle and error
|
|
categories, safe bounded errors, debug opt-in, and output-copy isolation
|
|
otherwise match their contracts. Stage 11 found two medium-severity security
|
|
defects in the explicit debug path: common credential aliases and URL forms
|
|
survive heuristic redaction, and pathname-based symlink checks can be raced
|
|
before sensitive files are created or replaced. Promptkit type containment,
|
|
source precedence, exact inline execution, credential preflight, cancellation,
|
|
safe error classification, callback ordering, restrictive modes, atomic file
|
|
replacement, and ordinary concurrent execution otherwise match their
|
|
contracts. Stage 12 found one medium-severity catalog defect because known
|
|
schema/template pairs are accepted for the wrong report, one low-severity
|
|
schema/decoder mismatch because Go accepts case-variant property names, one
|
|
medium-severity resource and diagnostic-boundary defect because generated
|
|
prose and unknown-field errors are unbounded, and one low-severity test
|
|
maintenance defect from duplicated day-style and misplaced schema assertions.
|
|
Typed normalization, required-field semantics, immutable schema copies, and
|
|
the current registry's four identities otherwise match their contracts. Stage
|
|
13 found two medium-severity rendering-boundary defects: context builders do
|
|
not reject conflicting report identities, and unescaped generated prose can
|
|
create Markdown structure that impersonates deterministic sections. It also
|
|
found one low-severity missing-value defect because Today can emit an empty
|
|
daypart heading when every row lacks displayable conditions. Typed module
|
|
projection, stable daypart ordering, conditional deterministic sections,
|
|
template/partial lookup, actionable execution errors, and repeated-render
|
|
determinism otherwise match their contracts. Stage 14 found two medium-severity
|
|
application-boundary defects: generated-text catalog compatibility is checked
|
|
only after weather collection and report construction, and execution proceeds
|
|
without reconciling inspected, prepared, callback, completed-execution, and
|
|
schema-validation provenance. Side-effect-free prompt/profile inspection,
|
|
credential preflight, immutable prepared values, one-build comparison reuse,
|
|
per-execution copies, typed generated-text validation, debug failure handling,
|
|
and partial execution results otherwise match their contracts. Stage 15
|
|
found one medium-severity destination-safety defect because ordinary
|
|
publication replaces non-regular filesystem objects, one low-severity path
|
|
handling defect because a valid long output basename makes the temporary name
|
|
overflow only after expensive work and leaves its newly created directory, and
|
|
one medium-severity cancellation defect because context is checked before the
|
|
temporary write rather than at the rename linearization point. Ordinary
|
|
regular-file replacement, prior-output preservation before rename, restrictive
|
|
new-file modes, temporary cleanup, committed-path result truthfulness, and
|
|
post-publication notification semantics otherwise match their contracts.
|
|
Stage 16 found four medium-severity defects: unsupported Distributor endpoint
|
|
forms pass active configuration validation and fail only after report
|
|
publication, remote response text flows into normal notification diagnostics,
|
|
Distributor response bodies have no size boundary, and mid-batch cancellation
|
|
is flattened into ordinary report failures while later items continue. It also
|
|
found one low-severity adapter-test gap because every local Distributor test
|
|
replaces the production HTTP client boundary. Collect-once planning, complete
|
|
destination preflight, deterministic sequential execution, partial publication
|
|
accounting, published-file upload selection, and the all-success notification
|
|
gate otherwise match their contracts.
|
|
Stage 17 found four medium-severity comparison defects: manifest recognition
|
|
accepts duplicate and case-variant JSON fields, successful report paths need
|
|
not match the documented ordinal/profile filename, cancellation after moving
|
|
the prior bundle does not prevent replacement commit, and backup-cleanup
|
|
failures can leave only a partially deleted recovery artifact while describing
|
|
it as retained. It also found one low-severity path-handling defect because a
|
|
filesystem-valid long destination basename fails during staging and can leave
|
|
new parent directories. Flat versioned bundles, deterministic encoding and
|
|
hashing, exact file-set recognition, broad-target and final-symlink rejection,
|
|
commit-time destination reauthorization, rollback after install failure, and
|
|
committed-new-bundle truth otherwise match their contracts.
|
|
Stage 18 found one medium-severity comparison-execution defect: cancellation
|
|
observed after profile goroutines join rewrites every nonsuccessful outcome as
|
|
canceled, including independent profile failures that completed before the
|
|
cancellation. Sequential preflight, collect/prepare once, immutable per-profile
|
|
inputs, concurrent execution through one backend-governed executor, joined
|
|
goroutines, deterministic ordering and debug identities, coherent partial
|
|
publication, committed-path projection, never-notify policy, content-safe CLI
|
|
summaries, and result-bearing failure exit behavior otherwise match their
|
|
contracts.
|
|
Stage 19 found two low-severity test-hygiene defects: one Promptkit credential
|
|
test changes behavior when a test-named environment variable already exists,
|
|
and symlink tests in three packages fail on hosts that cannot create symlinks
|
|
instead of treating capability as a prerequisite. The default suite otherwise
|
|
uses only local HTTP servers and in-memory external-boundary fakes, controls
|
|
environment mutation and writable roots, keeps time and concurrency waits
|
|
bounded, joins normal-path goroutines, contains synthetic credential-free
|
|
fixtures, has no subprocess or golden-update mechanism, passes shuffled
|
|
repetition and the race detector, and succeeds with a scrubbed environment and
|
|
failing external proxies.
|
|
Stage 20 found that the important runtime and compatibility risks have credible
|
|
test owners at the narrowest stable package, adapter, application, filesystem,
|
|
or CLI boundary. It did not create a coverage-percentage target or duplicate
|
|
the missing regression cases already attached to `AUD-003`, `AUD-010`,
|
|
`AUD-033`, `AUD-045` through `AUD-047`, `AUD-052`, and `AUD-053` through
|
|
`AUD-060`. Coverage diagnostics reached 79.4% of statements and highlighted
|
|
thin wrappers, dormant compatibility helpers, and defensive error accessors,
|
|
but no additional consequential unowned failure mode. Representative app and
|
|
CLI tests appropriately prove cross-package ordering and result projection
|
|
without repeating every lower-level malformed-input case. Detailed test
|
|
durability, fixture cost, and consolidation remain Stage 21.
|
|
Stage 21 found four low-severity test-maintenance issues: a direct assertion of
|
|
private SPC policy constants is already subsumed by stronger observable risk-
|
|
digest cases; collection and Distributor polling tests unnecessarily pay 1.00
|
|
second and 0.25 second production wait intervals; generated-text render-context
|
|
tests repeat reporttemplate's report-specific Markdown assertions; and a shared
|
|
comparison concurrency helper releases blocked workers on one diagnostic path
|
|
but not on its timeout, early-completion, or duplicate-start exits. Exact error
|
|
checks are otherwise limited and generally protect user-visible or safety
|
|
semantics; dense configuration/domain tables, realistic filesystem tests,
|
|
small external-boundary fakes, and representative app/CLI workflows earn their
|
|
cost. No complete-output golden snapshots or oversized fixture framework was
|
|
found.
|
|
Stage 22 found one medium-severity efficiency defect because every Weather API
|
|
collection discards a current-conditions warmup response, fetches that source
|
|
again, and serializes all eight otherwise independent source requests. It also
|
|
found one low-severity replacement cost because an existing comparison bundle
|
|
is fully recognized four times, causing `4N+16` complete reads for `N`
|
|
successful reports. Service-sized derivation, fixed registries, template
|
|
parsing, fixed batch sequencing, and ordinary profile fan-out are otherwise
|
|
proportionate to their current workloads.
|
|
Stage 23 found seven low-severity structural issues: daypart-key normalization and
|
|
Distributor template parsing each have two policy implementations; report
|
|
metadata and render inputs retain parallel authorities and unused template
|
|
surfaces; fact requirements span string constants, registry declarations, and
|
|
availability switches; generate and comparison repeat report-date policy at
|
|
four sites; several internal compatibility surfaces have no production
|
|
consumer; and one duplicate capitalization path corrupts non-ASCII daypart
|
|
labels. Report-specific generated types/builders, explicit prompt allowlist
|
|
projections, package-local safety copies and path diagnostics, config-to-
|
|
briefing validation, top-level workflow orchestration, and the current narrow
|
|
resolved-value clone remain intentionally explicit.
|
|
Stage 24 found five low-severity documentation-coherence issues: public and
|
|
internal CLI descriptions have drifted from executable help and comparison
|
|
dispatch; focused internal guides name a nonexistent forecast API, an absent
|
|
collection workflow, and report tests that do not exist; two completed
|
|
comparison roadmaps retain parallel current command and artifact definitions;
|
|
and the shared executor's concurrent-call requirement remains implicit rather
|
|
than part of its substitutability contract. The configuration reference also
|
|
omits accepted report-key normalization and timezone-offset bounds. Other
|
|
configuration fields/defaults, integration shapes, schema, template,
|
|
report/module/prompt identities, maintained examples, local and external
|
|
links, fence languages, security examples, and
|
|
the scoped atomic-visibility wording otherwise agree with their executable or
|
|
historical owners. Existing implementation defects that violate correctly
|
|
stated intended contracts remain assigned to their earlier findings rather
|
|
than being duplicated as documentation defects.
|
|
Stage 26 rechecked the two high-severity prompt-contract findings and the
|
|
security, diagnostic-boundary, cancellation, and destructive-publication
|
|
findings against current assets, source, and focused tests. It confirmed 74
|
|
distinct findings and rejected four standalone entries after consolidating
|
|
them into broader root-cause findings. All 61 retained decisions remain
|
|
supported, all routed investigation questions now resolve to a finding or a
|
|
retained decision, and no high-impact uncertainty requires a supplemental
|
|
investigation stage. Remediation acceptance, priority, and implementation
|
|
sequencing remain exclusively Stage 27 work.
|
|
|
|
Stage 27 accepted all 74 confirmed findings for implementation and allocated
|
|
each exactly once across the dependency-ordered audit remediation roadmap. No
|
|
confirmed finding is
|
|
priority-deferred or blocked on a separate product/architecture decision. The
|
|
roadmap places generated-forecast integrity, security, correctness, data
|
|
integrity, destructive operations, and cancellation before efficiency,
|
|
behavior-preserving consolidation, test cleanup, and roadmap retirement. The
|
|
discovery audit changed only its temporary roadmap documents.
|
|
|
|
## Final Audit Summary
|
|
|
|
### Overall Assessment
|
|
|
|
Weatherreporter's architecture is coherent: dependency direction is acyclic,
|
|
external dependencies remain adapter-local, the normal workflow is stateless,
|
|
deterministic facts and repository templates remain authoritative, and the
|
|
default test suite is offline and repeatable. The audit nevertheless found two
|
|
high-severity shared-prompt defects that can systematically misdirect every
|
|
generated report, 43 medium-severity correctness/security/reliability/
|
|
compatibility/efficiency gaps, and 29 low-severity confirmed maintenance,
|
|
documentation, testing, cleanup, or edge-case gaps. Four low-severity entries
|
|
were rejected only as standalone duplicates and consolidated into broader
|
|
confirmed owners.
|
|
|
|
The highest remediation risks are prompt/data-package disagreement and
|
|
overstated hazard locality, sensitive or unbounded upstream diagnostics,
|
|
prompt-debug pathname races, Markdown structural injection, destructive output
|
|
target handling, comparison replacement/recovery truthfulness, and lost
|
|
cancellation or failure identity. The suite passes because many findings are
|
|
missing adversarial boundary cases or incorrect-but-current contracts rather
|
|
than ordinary happy-path breakage.
|
|
|
|
### Reviewed Scope And Exclusions
|
|
|
|
The audit covered all production and test packages, command paths,
|
|
configuration and maintained examples, embedded prompt/profile/schema/template
|
|
assets, normalized weather acquisition and derivation, briefing modules,
|
|
Promptkit and Distributor adapters, generated-text validation/rendering,
|
|
single/batch/comparison orchestration, filesystem publication, test quality,
|
|
dependency/complexity history, and all current, historical, policy, internal,
|
|
integration, release, and roadmap documentation inventoried by Stage 1.
|
|
|
|
It did not call live Weather API, Promptkit-provider, or Distributor services;
|
|
use real credentials or private infrastructure; prove power-loss durability;
|
|
define an unsupported-platform promise; perform production remediation; change
|
|
dependencies; rewrite canonical current-state contracts to legitimize defects;
|
|
or prepare a release. Those exclusions preserve deterministic offline audit
|
|
evidence and the architecture's stated boundaries.
|
|
|
|
### Prioritized Finding Summary
|
|
|
|
| Priority | Finding set | Assessment and disposition |
|
|
| --- | --- | --- |
|
|
| P0 | `AUD-030`, `AUD-031` | Two high-severity prompt-contract defects. Accepted first because they affect shared instructions for all generated reports. |
|
|
| P1 | Medium security, correctness, reliability, compatibility, cancellation, and destructive-publication findings | Forty-three medium findings accepted before efficiency or cleanup. Their package-level tests must protect malformed input, safe diagnostics, cancellation linearization, recovery state, and visible result truth. |
|
|
| P2 | Low-severity correctness/security edges plus deterministic test and operational quality | Accepted after the P0/P1 contract fixes, except where a low finding is a prerequisite mechanism for a related correctness fix. |
|
|
| P3 | `AUD-039`, `AUD-067` through `AUD-072` | Behavior-preserving consolidation and dormant-surface retirement accepted after the behavior they touch is corrected and locked by tests. |
|
|
| P4 | `AUD-075`, `AUD-076`, and audit-record retirement | Documentation lifecycle work accepted last, after implemented contracts have durable canonical owners. |
|
|
|
|
The remediation roadmap's allocation table and 56 stages are the authoritative
|
|
finding-to-stage mapping. All 74 confirmed findings are accepted now; none is
|
|
deferred and none requires a prerequisite decision. Rejected `AUD-001`,
|
|
`AUD-022`, and `AUD-032` flow only through `AUD-072`; rejected `AUD-063` flows
|
|
only through `AUD-039`.
|
|
|
|
### Final Validation Summary
|
|
|
|
The clean final checkout passed uncached default tests, repository-wide race
|
|
tests, three shuffled repetitions, vet, root CLI help, and whitespace checks.
|
|
Diagnostic coverage was 79.4% of statements; it was used to inspect weak
|
|
critical branches rather than as a target, and `coverage.out` was removed.
|
|
Focused adversarial evidence and narrow test seams support the confirmed
|
|
correctness and safety findings. No generated artifact, unexplained failure,
|
|
candidate status, deferred finding, or unresolved scope question remains.
|
|
|
|
## Baseline Metadata
|
|
|
|
| Item | Recorded baseline |
|
|
| --- | --- |
|
|
| Audit date | 2026-08-12 (UTC) |
|
|
| Commit | `e7c7262404ba0e8e74009ed840d38f7a0142b347` (`Add audit workflow plan`) |
|
|
| Expected commit from audit plan | `151c536cb91ebedb6039270b752bae219dfe33d0` (`Add comparison diagnostics to the future roadmap`) |
|
|
| Baseline difference | The audit uses the current `main`, one commit ahead of the expected commit. The intervening commit adds the audit workflow plan used to conduct this audit. |
|
|
| Branch | `main` |
|
|
| Initial worktree | Clean; `git status --porcelain=v1` produced no entries before the audit ledger was created. |
|
|
| Go toolchain | `go version go1.26.5 linux/amd64`; `go env GOVERSION` reported `go1.26.5`. |
|
|
| Module | `gitea.maximumdirect.net/eric/weatherreporter`; module file `/home/eric/Workspace/weatherreporter/go.mod`; `go 1.26`. |
|
|
| Workspace context | Module mode with no `go.work`; `go env GOWORK` was empty and the repository contains only `./go.mod`. |
|
|
| Graph project | `home-eric-Workspace-weatherreporter` |
|
|
| Graph refresh | Refreshed from the baseline checkout in `moderate` mode without a persisted artifact: 2,674 nodes and 11,572 edges. The indexer excluded `docs`, `examples`, embedded-asset directories, and test-data directories, so their inventories below come from tracked files rather than the graph. |
|
|
| Baseline exclusions | None. There were no pre-existing worktree changes. |
|
|
| Environmental limitations | None encountered. Validation was deterministic and offline. |
|
|
|
|
## Scope And Inventory
|
|
|
|
Stage 1 inventories the checked-out repository without judging subsystem
|
|
correctness or test sufficiency. The repository has 219 tracked files. Its Go
|
|
inventory contains 24 packages, 83 non-test source files, and 50 test files.
|
|
|
|
### Go Packages And Files
|
|
|
|
| Package directory | Production `.go` files | `_test.go` files |
|
|
| --- | ---: | ---: |
|
|
| `cmd/weatherreporter` | 1 | 0 |
|
|
| `internal/adapters/distributor` | 1 | 1 |
|
|
| `internal/adapters/promptkit` | 1 | 1 |
|
|
| `internal/adapters/weatherapi` | 1 | 1 |
|
|
| `internal/app` | 10 | 11 |
|
|
| `internal/briefing` | 21 | 6 |
|
|
| `internal/buildinfo` | 1 | 0 |
|
|
| `internal/cli` | 4 | 5 |
|
|
| `internal/collect` | 1 | 1 |
|
|
| `internal/comparison` | 2 | 2 |
|
|
| `internal/config` | 7 | 2 |
|
|
| `internal/facts` | 1 | 1 |
|
|
| `internal/fileutil` | 1 | 1 |
|
|
| `internal/forecast` | 3 | 1 |
|
|
| `internal/generatedtext` | 8 | 7 |
|
|
| `internal/module` | 1 | 1 |
|
|
| `internal/promptassets` | 1 | 1 |
|
|
| `internal/promptdebug` | 1 | 1 |
|
|
| `internal/promptexec` | 2 | 1 |
|
|
| `internal/promptinput` | 1 | 1 |
|
|
| `internal/report` | 8 | 1 |
|
|
| `internal/reporttemplate` | 2 | 1 |
|
|
| `internal/timeutil` | 3 | 2 |
|
|
| `internal/weatherdata` | 1 | 1 |
|
|
| **Total** | **83** | **50** |
|
|
|
|
### Fixtures
|
|
|
|
Nine tracked fixtures are present:
|
|
|
|
- `internal/adapters/weatherapi/testdata/alerts.json`
|
|
- `internal/adapters/weatherapi/testdata/convective_outlooks.json`
|
|
- `internal/adapters/weatherapi/testdata/current.json`
|
|
- `internal/adapters/weatherapi/testdata/discussion.json`
|
|
- `internal/adapters/weatherapi/testdata/hourly.json`
|
|
- `internal/adapters/weatherapi/testdata/narrative.json`
|
|
- `internal/adapters/weatherapi/testdata/observations.json`
|
|
- `internal/adapters/weatherapi/testdata/weather_story.json`
|
|
- `internal/forecast/testdata/daily_bundle.json`
|
|
|
|
### Embedded Assets
|
|
|
|
The three `//go:embed` declarations cover 26 tracked assets:
|
|
|
|
- one SPC definition file under `internal/briefing/assets/`;
|
|
- three Promptkit profiles, ten prompt files, and four generated-text schemas
|
|
under `internal/promptassets/assets/`; and
|
|
- four report templates and four template partials under
|
|
`internal/reporttemplate/templates/`.
|
|
|
|
The exact embedding owners are
|
|
`internal/briefing/spc_convective_outlook_definitions.go`,
|
|
`internal/promptassets/promptassets.go`, and
|
|
`internal/reporttemplate/reporttemplate.go`.
|
|
|
|
### Canonical Documents And Maintained Examples
|
|
|
|
There are 38 tracked, non-roadmap canonical documents and three maintained
|
|
example files. The canonical inventory is:
|
|
|
|
- product and maintainer references: `README.md`, `docs/cli.md`,
|
|
`docs/config.md`, `docs/development.md`, `docs/operations.md`,
|
|
`docs/release.md`, and `docs/templates.md`;
|
|
- policies: `docs/policy/architecture.md`,
|
|
`docs/policy/documentation.md`, and `docs/policy/testing.md`;
|
|
- architecture decisions: `docs/adr/0001-stateless-execution.md`;
|
|
- integration contracts: `docs/integrations/comparison-bundle.md`,
|
|
`docs/integrations/promptkit.md`, `docs/integrations/weatherapi.md`, and the
|
|
three documents under `docs/integrations/distributor/`;
|
|
- internal documents: all 17 tracked documents under `docs/internal/`; and
|
|
- release notes: the four tracked documents under `docs/releases/`.
|
|
|
|
The maintained examples are `examples/config.yml`,
|
|
`examples/minimal-config.yml`, and
|
|
`examples/weather-light-local-profile.yml`. Roadmaps are coordination records,
|
|
not canonical current-state documents; the four pre-existing roadmap files are
|
|
therefore outside the canonical count.
|
|
|
|
## Baseline Validation
|
|
|
|
| Command | Result | Evidence |
|
|
| --- | --- | --- |
|
|
| `go test ./...` | Pass | Exit 0. All 24 packages were evaluated; 22 package test suites passed and `cmd/weatherreporter` plus `internal/buildinfo` reported no test files. |
|
|
| `go run ./cmd/weatherreporter --help` | Pass | Exit 0. Help printed usage for help, version, four report generators, two batch commands, and comparison. |
|
|
| `git diff --check` | Pass | Exit 0 with no output on the clean pre-ledger baseline. |
|
|
| `go vet ./...` | Pass | Exit 0 with no diagnostics. |
|
|
|
|
Supplementary baseline commands were `git rev-parse HEAD`,
|
|
`git rev-list --left-right --count 151c536...HEAD`,
|
|
`git status --porcelain=v1`, `git branch --show-current`, `go version`,
|
|
`go env GOVERSION GOMOD GOWORK GO111MODULE`, `go list ./...`, tracked-file
|
|
inventory commands, graph index refresh, and graph architecture inspection.
|
|
|
|
## Stage Coverage
|
|
|
|
| Stage | Scope | Status |
|
|
| ---: | --- | --- |
|
|
| 1 | Establish the baseline and audit ledger | Complete |
|
|
| 2 | Audit architecture and dependency direction | Complete |
|
|
| 3 | Audit report identity and time foundations | Complete |
|
|
| 4 | Audit configuration, secrets, and validation | Complete |
|
|
| 5 | Audit CLI parsing, wiring, and output contracts | Complete |
|
|
| 6 | Audit weather data acquisition and collection | Complete |
|
|
| 7 | Audit forecast and fact derivation | Complete |
|
|
| 8 | Audit module contracts, registry, and source-facing briefing modules | Complete |
|
|
| 9 | Audit derived, planning, formatting, and SPC briefing modules | Complete |
|
|
| 10 | Audit prompt inputs, assets, and neutral execution contracts | Complete |
|
|
| 11 | Audit Promptkit adaptation and secure prompt debugging | Complete |
|
|
| 12 | Audit generated-text validation and catalog contracts | Complete |
|
|
| 13 | Audit render contexts, templates, and Markdown rendering | Complete |
|
|
| 14 | Audit application preparation and prompt preflight | Complete |
|
|
| 15 | Audit single-report generation and atomic output | Complete |
|
|
| 16 | Audit batch orchestration and Distributor notification | Complete |
|
|
| 17 | Audit comparison contracts and transactional publication | Complete |
|
|
| 18 | Audit comparison execution and CLI integration | Complete |
|
|
| 19 | Audit test hermeticity and execution hygiene | Complete |
|
|
| 20 | Audit test risk coverage and ownership | Complete |
|
|
| 21 | Audit test durability, duplication, and maintenance cost | Complete |
|
|
| 22 | Audit cross-cutting efficiency and complexity | Complete |
|
|
| 23 | Audit cross-cutting refactoring and deduplication opportunities | Complete |
|
|
| 24 | Audit documentation coherence and executable contracts | Complete |
|
|
| 25 | Run dynamic robustness and final diagnostic validation | Complete |
|
|
| 26 | Verify, consolidate, and triage findings | Complete |
|
|
| 27 | Produce the remediation roadmap and close the audit | Complete |
|
|
|
|
## Risk-To-Test Coverage
|
|
|
|
Stage 1 records where tests exist but does not infer sufficiency from package
|
|
counts or a passing suite. Assigned stages will replace these pending entries
|
|
with evidence about meaningful risks, test ownership, gaps, and duplication.
|
|
|
|
| Risk area | Current test owner or evidence source | Audit stage | Coverage assessment |
|
|
| --- | --- | ---: | --- |
|
|
| Architecture and dependency direction | Assembled app and CLI tests; graph traces | 2 | Sufficient at the architectural boundary: direct imports are acyclic, external dependency types remain adapter-local, and representative workflow ordering and publication boundaries have focused tests. The unused persistence APIs first recorded by rejected standalone `AUD-001` are consolidated into confirmed `AUD-072`. |
|
|
| Report identity, periods, dates, and timezones | `internal/report`, `internal/timeutil`, plus focused `internal/app` and `internal/forecast` callers | 3 | Partly insufficient. Registry identities, alias policies, batch membership, civil-day boundaries, and elapsed six-hour periods are coherent and exercised. `AUD-002` records incorrect wall-clock windows on daylight-saving transition days, and `AUD-003` records the missing regression test for Daily run-ID uniqueness across dates. |
|
|
| Configuration, validation, and secrets | `internal/config`, focused `internal/cli` and Promptkit adapter mapping, maintained examples | 4 | Partly insufficient. Defaults, precedence, known-field parsing, active URL/duration/template checks, output-path preservation, profile-source rules, secret redaction, and examples are exercised. `AUD-004` through `AUD-007` record unsupported missing-source keys, blank notification identities, constructed-report override inconsistency, and non-atomic secret environment mutation. |
|
|
| CLI parsing, output, and exit behavior | `cmd/weatherreporter`, `internal/cli`, representative app-facing CLI tests | 5 | Partly insufficient. Command/flag ownership, paths, one-executor mapping, stream separation, quiet mode, comparison safe errors, and visible result/exit rules are coherent. `AUD-008` records missing signal cancellation, `AUD-009` records late generate date validation, and `AUD-010` records missing assembled non-comparison CLI protection. |
|
|
| Weather transport and normalized collection | `internal/adapters/weatherapi`, `internal/collect`, `internal/weatherdata` | 6 | Partly insufficient. All eight source requests, required/optional policy, checked-empty products, retryable source statuses, cancellation, normalized provenance, and collection error ownership have focused offline coverage. `AUD-011` through `AUD-015` record unsupported schemes, unsafe response-body diagnostics, an unenforced body-size limit, structurally invalid required hourly periods, and overbroad warmup retries. |
|
|
| Forecast and fact derivation | `internal/forecast`, `internal/facts` | 7 | Partly insufficient. Half-open selection, stable ordering, alert parsing and clipping, precipitation-window grouping, report-family selection, and SPC checked-empty behavior have focused owners. `AUD-016` through `AUD-019` record unit collapse, overnight alert omission, invalid percentage acceptance, and incomplete ice-text classification. |
|
|
| Module and briefing contracts | `internal/module`, `internal/briefing` | 8-9 | Partly insufficient. Registry/default composition, report support, missing optional sources, checked-empty alerts, output identity, ordering, current/hourly prompt curation, precipitation timing, and SPC selection have focused owners. Confirmed `AUD-020`, `AUD-021`, and `AUD-023` through `AUD-028` record the distinct defects; rejected standalone `AUD-022` is consolidated into the dormant compatibility decision in `AUD-072`. |
|
|
| Prompt inputs, embedded assets, and execution contracts | `internal/promptinput`, `internal/promptassets`, `internal/promptexec` | 10 | Partly insufficient. Prompt/profile/schema identities, deterministic grouping, validation/execution result categories, safe bounded errors, explicit debug, and copy isolation have focused owners. Confirmed `AUD-029` through `AUD-031` and `AUD-033` record the distinct boundary and contract gaps; rejected standalone loader-hardening entry `AUD-032` is consolidated into `AUD-072`. |
|
|
| Promptkit boundary and sensitive debug output | `internal/adapters/promptkit`, `internal/promptdebug` | 11 | Partly insufficient. Exact prompt/profile inspection, source precedence, inline execution, credential checks, cancellation, safe classified errors, callback ordering, debug opt-in, static symlink rejection, restrictive modes, atomic replacement, and distinct concurrent captures have focused owners. `AUD-034` and `AUD-035` record incomplete credential redaction and a check/use symlink race. |
|
|
| Generated-text validation | `internal/generatedtext` | 12 | Partly insufficient. The four current registry definitions dispatch to typed validators with coherent required-field and normalization behavior, and schema bytes are independently copied. `AUD-036` through `AUD-039` record report-independent catalog matching, case-insensitive Go field aliases, unbounded prose/diagnostics, and duplicated or misplaced contract tests. |
|
|
| Render contexts and templates | `internal/generatedtext`, `internal/reporttemplate` | 13 | Partly insufficient. Typed module extraction, deterministic daypart ordering, report-specific sections, partial conditions, template lookup, and semantic output assertions are coherent. `AUD-040` through `AUD-042` record conflicting report identities, structural Markdown injection, and Today's empty daypart section. |
|
|
| Prompt preflight and prepared inputs | `internal/app` | 14 | Partly insufficient. Exact prompt input/output shape, profile selection, credentials, immutable preparation, per-execution copies, typed output validation, rendering, callback errors, and partial execution outcomes have focused owners. `AUD-043` and `AUD-044` record late generated-text catalog validation and missing end-to-end execution-provenance reconciliation. |
|
|
| Single-report publication and preservation | `internal/app`, `internal/fileutil` | 15 | Partly insufficient. Destination precedence, regular-file publication, same-directory temporary replacement, prior-output preservation, temporary cleanup, restrictive modes, truthful committed paths, and notification-after-publication have focused owners. `AUD-045` through `AUD-047` record destructive non-regular target handling, late failure for valid long basenames, and cancellation not checked at the rename boundary. |
|
|
| Batch partial success and notification | `internal/app`, `internal/adapters/distributor` | 16 | Partly insufficient. Collect-once planning, complete output preflight, deterministic sequential execution, independent partial success, report-only counters, published-source selection, and one all-success batch notification are coherent. `AUD-048` through `AUD-052` record late Distributor endpoint rejection, unsafe and unbounded response diagnostics, cancellation flattened into report failure, and the missing production HTTP adapter test boundary. |
|
|
| Comparison identity and transactional publication | `internal/comparison` | 17 | Partly insufficient. Versioned flat manifests, ordered identity helpers, canonical encoding, data-package hashing, exact file sets/types, read-only destination classification, commit-time reauthorization, sibling staging, replacement rollback, restrictive modes, and committed cleanup state have focused package owners. `AUD-053` through `AUD-057` record ambiguous manifest fields, unenforced report filenames, a replacement cancellation gap, partially deleted retained backups, and late long-destination staging failure. |
|
|
| Comparison concurrency and CLI behavior | `internal/app`, `internal/cli` | 18 | Partly insufficient. Explicit profiles are preflighted sequentially, one immutable prepared package feeds concurrent profile-local executions, goroutines join, slice positions lock durable/CLI order, partial bundles publish coherently, and safe summaries preserve committed paths and fail the action on any error. `AUD-058` records a missing mixed failure/cancellation case and the resulting overwrite of an already completed profile failure. |
|
|
| Test hermeticity and execution hygiene | All Go tests, fixtures, test assets, and helper mechanisms | 19 | Partly insufficient. The suite is offline, uses controlled clocks/roots/environment for almost every stateful case, has no subprocesses or automatic golden updates, passes race and shuffled-repeat checks, and succeeds with a scrubbed environment and broken external proxies. `AUD-059` records one ambient credential assumption; `AUD-060` records unguarded symlink-capability assumptions. |
|
|
| Test risk coverage and ownership | Repository-wide suite | 20 | Partly insufficient. Consequential contracts have narrow owners and representative cross-boundary workflows; every identified missing regression is already attached to the production or test-gap finding for the defect it would catch. Coverage diagnostics found no additional unowned critical branch. Accepted omissions are thin entry-point/delegation wrappers, defensive accessors, and dormant helpers already routed under `AUD-001`; broad duplication and durability remain Stage 21. |
|
|
| Test durability, duplication, and maintenance cost | Repository-wide suite | 21 | Partly insufficient. Most tests assert stable behavior with direct fixtures and appropriately narrow fakes. Confirmed `AUD-039` now owns both duplicated generated-text semantics and cross-owner render assertions formerly split into rejected `AUD-063`; `AUD-061` identifies one implementation-coupled assertion, `AUD-062` avoidable production waits, and `AUD-064` incomplete failure cleanup. Each consolidation retains a named behavioral owner. |
|
|
| Cross-cutting efficiency | Graph metrics, workflow traces, and focused latency/allocation diagnostics | 22 | Partly insufficient. Fixed-size registries, service-sized derivation, template parsing, batch composition, and ordinary profile fan-out are proportionate. `AUD-065` records one redundant Weather API request plus serial independent source fetches; `AUD-066` records repeated full comparison-bundle reads during replacement. |
|
|
| Cross-cutting maintainability | Graph similarity, change history, caller traces, and focused structural review | 23 | Partly insufficient. `AUD-067` through `AUD-073` identify duplicated canonicalization/template/date/requirement policy, parallel metadata authority, dormant internal surfaces, and unsafe duplicate capitalization. Important report, prompt-curation, safety-copy, validation, and orchestration similarities remain explicit under `RET-053` through `RET-058`. |
|
|
| Documentation and executable-contract coherence | Canonical documents, code, schemas, templates, examples | 24 | Partly insufficient. `AUD-074` through `AUD-078` record CLI/help drift, stale internal flow/API/test claims, completed-roadmap duplication, an implicit concurrent-executor requirement, and incomplete configuration normalization/bounds. Other volatile identities and maintained examples match their executable owners; linked summaries, historical release notes, and the accepted ADR remain appropriately distinct. |
|
|
| Dynamic robustness and diagnostic checks | Repository-wide deterministic checks | 25 | Sufficient for audit closure. Uncached, race-enabled, and three shuffled test runs passed; vet and CLI help passed; coverage remained 79.4% and exposed no additional consequential unowned branch. The diagnostic artifact was removed. |
|
|
| Finding verification and triage | All findings, retained decisions, routed questions, validation records, and coverage entries | 26 | Complete. Seventy-four findings are confirmed and four duplicate standalone entries are rejected into their named root-cause owners. No candidate or deferred status remains; all confirmed entries retain actionable test implications and validation criteria. |
|
|
|
|
## Findings
|
|
|
|
### AUD-001: Unused persistence helpers remain after the stateless redesign
|
|
|
|
- Stage: 2
|
|
- Status: rejected
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: architecture
|
|
- Area: `internal/promptinput.Save`,
|
|
`internal/adapters/weatherapi.SaveBundle`, and
|
|
`internal/fileutil.WriteJSONAtomic`
|
|
- Evidence: Graph inbound traces show `promptinput.Save` has no callers,
|
|
`weatherapi.SaveBundle` is called only by `TestSaveBundle`, and
|
|
`fileutil.WriteJSONAtomic` is called only by `SaveBundle` and its focused
|
|
test. Repository text search found no documentation or other call sites.
|
|
Representative generate, batch, comparison, and collection traces do not
|
|
reach any of these helpers. `promptinput.Save` and `SaveBundle` write
|
|
intermediate prompt input or normalized weather data to arbitrary paths,
|
|
while normal publication uses `fileutil.WriteFileAtomic` for selected
|
|
Markdown and `comparison.Publish` for selected bundles.
|
|
- Contract at risk: The architecture policy and ADR 0001 define normal
|
|
execution as an in-memory stateless transformation whose durable files are
|
|
operator-selected report outputs, comparison bundles, or explicitly
|
|
requested secure prompt debugging.
|
|
- Impact: These unreachable exported functions do not create state during
|
|
current workflows, but they retain an unsupported persistence surface and
|
|
low-value tests that can invite accidental reintroduction of intermediate
|
|
artifacts or require maintenance despite having no product caller.
|
|
- Recommendation: Remove the two unused domain/adapter save functions, remove
|
|
`WriteJSONAtomic` if it then has no production use, and delete or consolidate
|
|
tests that protect only those retired APIs.
|
|
- Test implications: `TestSaveBundle` and `TestWriteJSONAtomic` protect unused
|
|
persistence mechanisms; `promptinput.Save` has no focused test. Preserve
|
|
tests for `WriteFileAtomic`, normal report publication, comparison
|
|
publication, and explicit prompt-debug writes.
|
|
- Validation: Graph and text searches show no remaining production references
|
|
to the retired helpers; `go test ./...` passes; normal output, comparison,
|
|
and debug-publication tests remain unchanged and pass.
|
|
- Disposition: Rejected as a standalone finding. Its evidence is sound, but
|
|
the unused writers are one part of the dormant compatibility surface already
|
|
consolidated under `AUD-072`; a separate recommendation would split one
|
|
ownership decision without changing its priority or validation.
|
|
- Related findings: `AUD-072`
|
|
- Remediation reference: Not applicable; consolidated into `AUD-072` and
|
|
Remediation Stage 31.
|
|
|
|
### AUD-002: Clock windows shift on daylight-saving transition days
|
|
|
|
- Stage: 3
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/timeutil.ClockWindow` and
|
|
`internal/forecast.ResolveDayparts`
|
|
- Evidence: `ClockWindow` obtains the local civil-day start and creates its
|
|
boundaries with elapsed-duration additions such as
|
|
`day.Start.Add(17 * time.Hour)`. In `America/Chicago`, adding 17 elapsed
|
|
hours to local midnight produces 18:00 on the 2026-03-08 spring transition
|
|
and 16:00 on the 2026-11-01 fall transition; adding six hours produces
|
|
07:00 and 05:00 respectively. Its only production caller is
|
|
`forecast.ResolveDayparts`, which uses those boundaries for daily summaries
|
|
and facts. The focused time test covers only an overnight window in a fixed
|
|
offset zone, so both transition cases pass unnoticed.
|
|
- Contract at risk: Configured daypart clocks are local wall-clock boundaries.
|
|
Civil-day and timezone policy requires those clocks to retain their stated
|
|
local hour across 23-hour and 25-hour days.
|
|
- Impact: On daylight-saving transition dates, observations can enter or leave
|
|
a configured daypart an hour early or late, shifting derived daily summaries
|
|
and any downstream facts or report content that consume them.
|
|
- Recommendation: Construct each boundary as a civil date plus local clock in
|
|
the target location, including explicit handling for `24:00` and overnight
|
|
windows, rather than adding elapsed hours to midnight.
|
|
- Test implications: Add table-driven `America/Chicago` coverage for both
|
|
spring-forward and fall-back dates, asserting the requested wall-clock start
|
|
and end and the intended overnight date rollover. Retain the existing
|
|
half-open period assertions.
|
|
- Validation: The new regression test fails against the elapsed-addition
|
|
implementation and passes when transition-day boundaries retain their
|
|
configured local clocks; focused `timeutil` and `forecast` suites pass.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 6.
|
|
|
|
### AUD-003: Daily run-ID uniqueness lacks a focused contract test
|
|
|
|
- Stage: 3
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: testing
|
|
- Area: `internal/report.Resolved.runID`,
|
|
`validStartDateRunIDDisambiguator`, and focused report tests
|
|
- Evidence: Daily run IDs append the resolved valid-start date to the shared
|
|
generation timestamp and report ID. This distinguishes multiple Daily
|
|
reports planned for different dates in one batch, where `GeneratedAt` is the
|
|
same. Focused report tests assert that registry identities are nonempty and
|
|
app tests use nonempty or fixture run IDs, but no test resolves two Daily
|
|
dates at one clock instant and asserts distinct IDs. Downstream
|
|
`plannedReportsByRunID` rejects duplicate IDs before building a batch
|
|
notification, and Distributor idempotency also consumes the run ID.
|
|
- Contract at risk: A run ID must identify one resolved report product, while
|
|
one batch may legitimately contain several Daily products for different
|
|
valid dates.
|
|
- Impact: An accidental removal or weakening of the date disambiguator can
|
|
give distinct Daily products the same identity, causing batch notification
|
|
construction to fail and creating Distributor idempotency collisions even
|
|
though the existing focused suite remains green.
|
|
- Recommendation: Add a focused report test that resolves two Daily dates with
|
|
the same injected `Now`, asserts distinct nonempty run IDs, and verifies that
|
|
each ID carries the corresponding valid-start date. Prefer this relational
|
|
contract over freezing unrelated timestamp formatting.
|
|
- Test implications: The regression must fail if the Daily disambiguator is
|
|
removed while allowing harmless changes to the common run-ID timestamp
|
|
representation. An app-level assertion may additionally protect the
|
|
multi-Daily batch consumer, but should not duplicate the report invariant.
|
|
- Validation: Removing the date disambiguator makes the new focused test fail;
|
|
with the invariant intact, report and multi-Daily batch tests pass and
|
|
`plannedReportsByRunID` receives unique keys.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 6.
|
|
|
|
### AUD-004: Unsupported missing-source keys are silently accepted
|
|
|
|
- Stage: 4
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/config.Validate`, `MissingSourceConfig.Sources`, and
|
|
`internal/adapters/weatherapi.policyFor`
|
|
- Evidence: The configuration contract lists seven optional source keys and
|
|
states that hourly forecast data is always required. `Validate` checks only
|
|
that each map key is nonblank and its policy is one of `error`, `warn`, or
|
|
`none`; it never checks the key against the supported set. The Weather API
|
|
adapter looks up policies by its own literal source names and otherwise uses
|
|
the default. Its required-hourly path returns an error before consulting the
|
|
policy. Consequently a typo such as `alert: error`, an arbitrary key, or an
|
|
ineffective `hourly: none` loads successfully. Focused config tests cover an
|
|
invalid policy value but no unsupported key.
|
|
- Contract at risk: `missing_source.sources` accepts only the documented
|
|
optional source identities, and invalid configuration must fail loading
|
|
rather than silently changing the effective failure policy.
|
|
- Impact: An operator can believe a source-specific `error`, `warn`, or `none`
|
|
rule is active while runtime collection ignores it and applies the default;
|
|
the mistake is visible only when that source is missing or malformed.
|
|
- Recommendation: Give `internal/config` one authoritative supported-source
|
|
set and reject every other key, including `hourly`. Keep adapter source names
|
|
aligned with that owner without introducing a second independently maintained
|
|
list.
|
|
- Test implications: Add loaded and constructed configuration cases for every
|
|
supported key plus representative typo, unknown, blank, and required-hourly
|
|
keys. Assert behavior through `Validate` or `LoadFile`, not private map
|
|
iteration.
|
|
- Validation: The new unsupported-key cases fail against the current validator
|
|
and pass when validation rejects keys that cannot affect optional-source
|
|
policy; focused config and Weather API policy tests remain green.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 2.
|
|
|
|
### AUD-005: Single-report notification identities may render blank
|
|
|
|
- Stage: 4
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `validateDistributorNotify`, `RenderDistributorBundleID`, and
|
|
`RenderDistributorIdempotencyKey`
|
|
- Evidence: Enabled single-report notification validation rejects an empty
|
|
template and unknown variables, and `RenderDistributorPipelineID` rejects a
|
|
whitespace-only result. In contrast, the bundle-ID and idempotency-key
|
|
renderers return whitespace unchanged and validation does not perform a
|
|
nonblank result check. Batch renderers consistently reject whitespace-only
|
|
pipeline, bundle, and idempotency identities. The Distributor adapter checks
|
|
only equality with the empty string, so whitespace values also pass its
|
|
local request checks. Focused tests cover the single pipeline case and all
|
|
three batch cases, but omit the two single-report cases.
|
|
- Contract at risk: Enabled notification requires usable pipeline, bundle, and
|
|
idempotency identities; configuration validation must reject invalid active
|
|
templates before report execution.
|
|
- Impact: A whitespace bundle ID or idempotency key can reach Distributor after
|
|
report publication, risking rejected uploads or collisions between otherwise
|
|
distinct report deliveries while configuration preflight reports success.
|
|
- Recommendation: Apply the same trimmed-nonempty invariant to all three
|
|
single-report identity renderers and exercise those renderers during active
|
|
configuration validation.
|
|
- Test implications: Extend the existing single-report validation table with
|
|
whitespace-rendered bundle and idempotency identities, and retain the batch
|
|
table as the parallel contract owner.
|
|
- Validation: Both new cases fail against the current single-report renderers
|
|
and pass when all active rendered identities are nonblank; focused config,
|
|
app notification, and Distributor adapter tests pass.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 3.
|
|
|
|
### AUD-006: Constructed report overrides depend on inaccessible presence flags
|
|
|
|
- Stage: 4
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `ReportConfig`, `ReportDistributorConfig`,
|
|
`Config.ReportModuleOverrides`, and
|
|
`Config.ReportDistributorPathOverrides`
|
|
- Evidence: YAML unmarshalling sets private `deterministicModulesSet` and
|
|
`pathTemplatesSet` flags to distinguish omission from an explicit list.
|
|
Both public override accessors skip exported `DeterministicModules` and
|
|
`PathTemplates` values unless the corresponding private flag is true. Code
|
|
outside package `config` therefore cannot construct an equivalent override:
|
|
setting a nonempty exported slice in `Config.Reports` is silently ignored,
|
|
and `Validate` skips its contents as well. Tests named for loaded/constructed
|
|
consistency reside inside package `config` and manually set the private
|
|
flags, so they do not exercise the configuration shape available to app
|
|
callers. Graph traces show the accessors feed app report resolution and
|
|
single/batch Distributor path rendering.
|
|
- Contract at risk: Loaded and programmatically constructed configurations
|
|
with the same exported values should validate and produce the same report
|
|
module and path overrides.
|
|
- Impact: Repository callers and tests that construct a `config.Config` can
|
|
silently run default modules or Distributor paths instead of their requested
|
|
values, while invalid constructed overrides can pass validation unused.
|
|
- Recommendation: Represent presence through an API available to downstream
|
|
callers, such as constructors/setters or an optional-list representation
|
|
that preserves omitted versus explicitly empty YAML without private state
|
|
that exported values cannot establish.
|
|
- Test implications: Move or add behavior tests from an external config test
|
|
package or an app boundary so they can use only exported construction APIs;
|
|
assert both nonempty overrides and explicit-empty rejection.
|
|
- Validation: The external-boundary test fails with direct exported-field
|
|
construction today and passes when loaded and constructed forms yield the
|
|
same overrides and validation result.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 2.
|
|
|
|
### AUD-007: Failed configuration loads can leave secrets applied
|
|
|
|
- Stage: 4
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: security
|
|
- Area: `internal/config.Load` and `loadSecrets`
|
|
- Evidence: `Load` calls `loadSecrets` before `Validate`. The secret loader
|
|
validates and applies directory entries one at a time with `os.Setenv`, with
|
|
no preflight of all entries and no restoration of prior environment values.
|
|
A valid secret is therefore installed even when later configuration
|
|
validation fails, and an earlier secret remains installed when a later
|
|
invalid entry, unreadable value, or failed environment assignment aborts the
|
|
directory load. Existing tests cover successful replacement, line-ending
|
|
trimming, rejection diagnostics, and omission of secret values from errors,
|
|
but do not assert failure atomicity.
|
|
- Contract at risk: A rejected configuration should not partially apply its
|
|
secret-bearing environment interaction, and repeatable in-process loads
|
|
should not inherit state from an unsuccessful attempt.
|
|
- Impact: The normal CLI process exits after a load failure, limiting the
|
|
operational effect, but embedded runners, repeated actions, and tests can
|
|
observe or consume credentials from a configuration that was never accepted.
|
|
- Recommendation: Validate non-secret configuration before secret application,
|
|
preflight and read the complete secret directory before mutation, and either
|
|
apply environment changes atomically with rollback or return a scoped lookup
|
|
rather than mutating process-global state incrementally.
|
|
- Test implications: Add cases for a valid secret followed by an invalid entry
|
|
and for valid secrets paired with a later-invalid configuration, asserting
|
|
that prior set/unset environment state is restored.
|
|
- Validation: Both failure cases leave the environment exactly as it was before
|
|
loading; successful overwrite and single-line-ending behavior remain intact.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 2.
|
|
|
|
### AUD-008: Process interrupts do not cancel active workflows
|
|
|
|
- Stage: 5
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `cmd/weatherreporter.main` and CLI cancellation propagation
|
|
- Evidence: `main` calls `cli.Run` with `context.Background()`. Repository code
|
|
search found no `signal.Notify`, `signal.NotifyContext`, `os.Interrupt`, or
|
|
termination-signal handling. `Runner.Run` correctly forwards the supplied
|
|
context to generate, batch, and comparison workflows, and focused app tests
|
|
prove that context cancellation prevents pending publication and joins
|
|
comparison execution. The production executable never supplies a context
|
|
that an operator interrupt can cancel, so those paths are reachable only by
|
|
programmatic callers and tests.
|
|
- Contract at risk: Documented cancellation failures must reach workflow
|
|
cancellation checks so pre-publication destinations are preserved and a
|
|
bounded failed result can be finalized when available.
|
|
- Impact: `SIGINT` or `SIGTERM` terminates the process abruptly instead of
|
|
canceling in-flight collection, Promptkit work, publication preflight, or
|
|
notification. The CLI cannot emit its documented canceled result path or
|
|
allow workflow cleanup/coordination to finish, increasing interruption risk
|
|
around output and comparison transactions.
|
|
- Recommendation: Derive the root context from `signal.NotifyContext` for
|
|
operator interrupt and termination signals, stop signal delivery on return,
|
|
and pass that context through the existing CLI boundary. Preserve the
|
|
current nonzero failure exit and bounded error behavior.
|
|
- Test implications: Add a narrow executable-boundary test or a testable signal
|
|
context seam proving that an interrupt cancels the context passed to CLI.
|
|
Keep detailed destination-preservation and goroutine-join assertions with
|
|
their existing app/publication owners.
|
|
- Validation: An interrupt observed during a controlled blocking action
|
|
cancels its workflow context, returns the expected failed/canceled result
|
|
when available, and leaves the existing destination intact; ordinary help,
|
|
version, and successful exits are unchanged.
|
|
- Related findings: `AUD-010`
|
|
- Remediation reference: Remediation Stage 4.
|
|
|
|
### AUD-009: Generate validates report dates after constructing its executor
|
|
|
|
- Stage: 5
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/cli.Runner.resolveGenerateAction`
|
|
- Evidence: Generate validates the command name and syntactic flags, then loads
|
|
configuration and constructs the Promptkit executor before checking that
|
|
Daily has `--date` or parsing a supplied Daily/Today date. Executor
|
|
construction can open and parse configured profile files/directories and can
|
|
fail first. The equivalent comparison resolver validates profile selection
|
|
before configuration and performs report-date checks before constructing its
|
|
one executor. Focused comparison tests assert that input failures construct
|
|
no executor, but generate has no corresponding test. The production
|
|
`generate daily` diagnostic confirms the missing-date failure is pre-result,
|
|
but source tracing shows the executor has already been constructed.
|
|
- Contract at risk: Basic command input should be rejected before constructing
|
|
an external-dependency adapter or performing work unrelated to the invalid
|
|
request; pre-run errors should identify the user's actionable input failure.
|
|
- Impact: A missing or malformed report date can instead surface a profile
|
|
source/configuration error, needlessly reads configured Promptkit sources,
|
|
and follows secret loading from `AUD-007` even though the report request can
|
|
never execute.
|
|
- Recommendation: After flag parsing, reject a missing Daily date immediately;
|
|
then load configuration for the effective timezone, parse/default the date,
|
|
resolve paths, and construct the executor only after all request-local
|
|
validation succeeds.
|
|
- Test implications: Add generate input-failure cases with an injected executor
|
|
factory and assert zero factory/application calls and no stdout/stderr for a
|
|
missing Daily date, malformed Daily/Today date, unsupported flag, and
|
|
unexpected argument. Avoid duplicating `timeutil` parser cases.
|
|
- Validation: Every generate input failure returns its input diagnostic without
|
|
constructing an executor or emitting a partial summary; valid requests still
|
|
construct exactly one executor.
|
|
- Related findings: `AUD-007`, `AUD-010`
|
|
- Remediation reference: Remediation Stage 4.
|
|
|
|
### AUD-010: Assembled non-comparison CLI result paths lack durable tests
|
|
|
|
- Stage: 5
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: testing
|
|
- Area: `internal/cli.Runner.Run`, generate/run command tests, and
|
|
`cmd/weatherreporter`
|
|
- Evidence: The CLI suite directly tests generate summary field copying and
|
|
shared output helpers, but has no assembled `Runner.Run` generate success,
|
|
result-bearing failure, quiet failure, or pre-result failure test. Generate
|
|
calls `app.GenerateDetailed` directly, unlike the injectable batch and
|
|
comparison functions, which makes its user-visible wiring difficult to test.
|
|
Batch has one assembled notification-failure case but no representative
|
|
ordinary success, report partial failure, quiet failure, or returned-error
|
|
mapping case. Comparison has comprehensive parser, construction, success,
|
|
partial failure, cleanup failure, quiet, and pre-execution tests. The command
|
|
package has no test for executable exit or signal behavior.
|
|
- Contract at risk: Generate and run must translate result presence, action
|
|
status, quiet mode, stdout JSON, batch stderr lines, returned errors, and the
|
|
executable's success/failure exit consistently with the CLI reference.
|
|
- Impact: A change can suppress a required failed summary, emit routine output
|
|
in quiet mode, return success for a failed result, write to the wrong stream,
|
|
or construct dependencies for invalid input without a focused CLI test
|
|
failing, even while summary-helper and lower app tests remain green.
|
|
- Recommendation: Add the smallest generate application injection seam
|
|
symmetrical with batch/comparison, then cover one success, one result-bearing
|
|
failure, quiet failure, and pre-result failure. Add only representative batch
|
|
success/report-failure cases and one executable exit/cancellation check;
|
|
leave workflow details with app tests.
|
|
- Test implications: Prefer structured JSON/status/error assertions and typed
|
|
errors over complete output snapshots or private call choreography. These
|
|
tests should own CLI translation, not repeat publication, collection, prompt,
|
|
or notification mechanics.
|
|
- Validation: Mutations to result/error branching, quiet gating, stream
|
|
selection, or failure return status break a focused CLI test, while internal
|
|
app refactors that preserve request/result contracts require no CLI edits.
|
|
- Related findings: `AUD-008`, `AUD-009`
|
|
- Remediation reference: Remediation Stage 4.
|
|
|
|
### AUD-011: Unsupported Weather API URL schemes fail only at transport
|
|
|
|
- Stage: 6
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/config.Validate` and
|
|
`internal/adapters/weatherapi.New`
|
|
- Evidence: Both boundaries accept any parsed absolute URL with a scheme and
|
|
host. The adapter always constructs an `http.Request` and executes it with
|
|
`http.Client`, so a configured URL such as `ftp://weather.example.test`
|
|
passes configuration and adapter construction but fails during warmup with
|
|
an unsupported-protocol transport error. The integration contract defines
|
|
an HTTP request boundary, and focused configuration and adapter tests cover
|
|
malformed or missing URLs but no unsupported absolute scheme.
|
|
- Contract at risk: A syntactically valid configuration must select a transport
|
|
the Weather API adapter can execute, and intrinsic endpoint incompatibility
|
|
should fail validation rather than an attempted collection.
|
|
- Impact: An operator can pass configuration validation and Promptkit
|
|
preflight only to receive a runtime weather-fetch failure for a base URL that
|
|
can never work. This obscures an actionable configuration defect as external
|
|
availability and performs avoidable setup before failure.
|
|
- Recommendation: Define `http` and `https` as the supported Weather API
|
|
schemes, document that bound in the configuration and integration owners,
|
|
and reject every other scheme during config validation and defensively in
|
|
adapter construction.
|
|
- Test implications: Add relational config and adapter cases that accept local
|
|
HTTP and HTTPS shapes and reject a representative unsupported absolute URL
|
|
without making a request. Do not duplicate generic URL-parser cases.
|
|
- Validation: Unsupported schemes fail before Weather API or Promptkit work;
|
|
ordinary HTTP test servers and configured HTTPS URLs remain accepted.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 5.
|
|
|
|
### AUD-012: Weather API failures expose upstream response bodies
|
|
|
|
- Stage: 6
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: security
|
|
- Area: `internal/adapters/weatherapi.Client.warmupOnce` and
|
|
`fetchHTTPOnce`
|
|
- Evidence: Both non-2xx branches interpolate the complete body read by their
|
|
10 MiB reader into the returned error. That error is wrapped without
|
|
redaction by `fetchHTTP`, `FetchBundle`, and `collect.Run`, then reaches the
|
|
application/CLI error path. Focused tests require endpoint and status context
|
|
but neither require response text nor prove that marker secrets or large
|
|
provider diagnostics are absent. The integration contract promises endpoint
|
|
context; it does not make arbitrary upstream bodies part of the diagnostic
|
|
contract.
|
|
- Contract at risk: Normal errors and summaries must remain bounded and must
|
|
not expose provider response detail, credentials echoed by an intermediary,
|
|
internal infrastructure diagnostics, or large HTML/error payloads.
|
|
- Impact: A failing or compromised service can place up to 10 MiB of arbitrary
|
|
text into normal stderr and wrapped action errors. Besides disclosure, this
|
|
can overwhelm machine-readable output consumers and make the actionable
|
|
endpoint/status diagnostic difficult to use.
|
|
- Recommendation: Return the relative endpoint, status code, and a stable
|
|
classification only. If response text is operationally necessary, admit a
|
|
small sanitized allowlisted excerpt or capture it only through an explicit
|
|
secure diagnostic mechanism.
|
|
- Test implications: Serve non-2xx bodies containing a unique secret marker and
|
|
a large payload; assert that neither appears in the adapter/collection error
|
|
while endpoint, status, retry identity, and cancellation remain inspectable.
|
|
- Validation: Normal failure output is bounded and marker-free for warmup and
|
|
source requests, with existing actionable and retry tests still passing.
|
|
- Related findings: `AUD-013`
|
|
- Remediation reference: Remediation Stage 5.
|
|
|
|
### AUD-013: The Weather API response limit silently truncates bodies
|
|
|
|
- Stage: 6
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/adapters/weatherapi.Client.warmupOnce` and
|
|
`fetchHTTPOnce`
|
|
- Evidence: Both readers call `io.ReadAll(io.LimitReader(resp.Body, 10<<20))`
|
|
and never determine whether more bytes remain. Most oversized JSON is
|
|
incidentally rejected as truncated syntax, but a complete JSON envelope
|
|
followed by enough whitespace or later invalid bytes can be accepted from
|
|
the first 10 MiB while the remainder is ignored. Warmup accepts every 2xx
|
|
prefix regardless. No focused test defines behavior at, below, or above the
|
|
nominal limit.
|
|
- Contract at risk: The documented per-response input bound must distinguish a
|
|
complete response within the limit from an oversized response; provenance
|
|
hashes and decoding must describe the complete accepted source payload.
|
|
- Impact: A buggy or hostile service can make Weatherreporter accept a partial
|
|
response, hash only its prefix, or treat an oversized warmup as healthy.
|
|
Ordinary oversized products usually fail with a misleading JSON decode
|
|
error rather than an actionable limit error.
|
|
- Recommendation: Give the transport one shared bounded-body reader that reads
|
|
at most the maximum plus one byte, rejects excess input explicitly, and is
|
|
used by both warmup and source attempts. Keep the configured timeout and body
|
|
closure ownership unchanged.
|
|
- Test implications: Exercise one response at the limit and one over it,
|
|
including a valid JSON prefix with ignored trailing content; assert a stable
|
|
size failure and no source decoding or retry for a non-transient size error.
|
|
- Validation: No over-limit warmup or source response is accepted, while a
|
|
complete at-limit response remains readable and response bodies always
|
|
close.
|
|
- Related findings: `AUD-012`
|
|
- Remediation reference: Remediation Stage 5.
|
|
|
|
### AUD-014: Required hourly periods can lack usable time bounds
|
|
|
|
- Stage: 6
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/adapters/weatherapi.bundleBuilder.fetchHourly` and
|
|
`internal/weatherdata.ForecastRun`
|
|
- Evidence: Required hourly validation rejects a missing/null payload, a JSON
|
|
type error, and a zero-length `periods` slice. Go decoding leaves absent
|
|
`issuedAt`, `startTime`, and `endTime` fields as zero `time.Time` values, so
|
|
`{"data":{"periods":[{}]}}` passes collection with one hourly period.
|
|
The adapter also does not reject a period whose end is not after its start.
|
|
Downstream period selection depends on those bounds, and the focused required
|
|
hourly test covers only explicit `null` rather than structurally unusable
|
|
periods.
|
|
- Contract at risk: Hourly is the required normalized forecast product; a
|
|
present array must contain usable forecast periods rather than merely one
|
|
decodable object.
|
|
- Impact: Collection can report success and provenance for an hourly product
|
|
that cannot contribute to any report period. Later derivation may present an
|
|
empty or incomplete forecast as a successful collection instead of the
|
|
required-source failure.
|
|
- Recommendation: Validate the smallest required hourly invariant at the
|
|
normalization boundary: a nonzero issue time if contractually required and,
|
|
for every period, nonzero bounds with `endTime` after `startTime`. Keep
|
|
meteorological selection and overlap policy in Stage 7 owners.
|
|
- Test implications: Add table-driven adapter cases for missing bounds, zero or
|
|
reversed duration, and one valid period. Assert direct required-source
|
|
failure without duplicating downstream forecast selection tests.
|
|
- Validation: Every accepted required hourly period has a usable half-open time
|
|
range, and existing valid fixtures still normalize unchanged.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 5.
|
|
|
|
### AUD-015: Weather API warmup retries permanent HTTP failures
|
|
|
|
- Stage: 6
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/adapters/weatherapi.Client.warmup` and `warmupOnce`
|
|
- Evidence: Source fetching retries only transport/read failures and the
|
|
documented `408`, `429`, `500`, `502`, `503`, and `504` statuses. Warmup uses
|
|
a separate attempt implementation and retries every error until its budget
|
|
is exhausted, including permanent `400`, `401`, `403`, and `404` responses.
|
|
Focused warmup tests cover successful retry of `502` and exhausted `502`,
|
|
while the non-retryable status test covers only a source request.
|
|
- Contract at risk: The Weather API retry policy distinguishes transient
|
|
failures from permanent request/status failures consistently across HTTP
|
|
attempts.
|
|
- Impact: Permanent warmup failures cause redundant requests and delay an
|
|
actionable error by the full retry schedule. The small default budget bounds
|
|
the effect, so this is primarily avoidable latency and upstream load.
|
|
- Recommendation: Reuse the shared transport attempt classification for
|
|
warmup while retaining its distinct success rule that a readable 2xx body
|
|
need not decode as a source envelope.
|
|
- Test implications: Add a warmup `404` case that makes one request, retain the
|
|
`502` retry case, and assert cancellation interrupts its retry delay. Keep
|
|
source decoding and optional-source policy tests separate.
|
|
- Validation: Warmup retries only the documented transient classes and fails
|
|
permanent statuses immediately with endpoint/status context.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 5.
|
|
|
|
### AUD-016: Metric fallback values lose their unit during daypart derivation
|
|
|
|
- Stage: 7
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/forecast.SummarizeDaypart`, `numericIndicators`, and
|
|
unitless `Range`/`TimedValue` summary fields
|
|
- Evidence: Normalized forecast periods keep temperatures in explicit
|
|
Fahrenheit/Celsius fields and winds in explicit mph/km/h fields.
|
|
`SummarizeDaypart` selects the first populated temperature or wind field and
|
|
stores the raw number in a summary field with no unit. `numericIndicators`
|
|
likewise selects mph before km/h but compares either result to the mph
|
|
thresholds of 35 for gusts and 25 for sustained wind. Metric mode is an
|
|
accepted runtime setting, and focused derivation fixtures populate only
|
|
Fahrenheit and mph. Consequently, a metric-only gust of 35 km/h is marked
|
|
hazardous even though it is about 22 mph, while metric temperature and wind
|
|
summary values are indistinguishable from US customary values to immediate
|
|
facts consumers.
|
|
- Contract at risk: Derivation must preserve or normalize measurement units so
|
|
equivalent meteorological inputs produce equivalent summaries and hazard
|
|
classifications under every supported units setting.
|
|
- Impact: Metric reports can exaggerate wind hazards and carry Celsius or km/h
|
|
numbers into downstream fields that consumers interpret as Fahrenheit or
|
|
mph. The defect can materially change planning facts and generated report
|
|
content.
|
|
- Recommendation: Give derived ranges and timed values an explicit unit, or
|
|
convert metric fallbacks into one documented canonical unit before storing
|
|
or comparing them. Apply thresholds only in their declared unit and avoid
|
|
silently preferring one of two populated representations unless their
|
|
equivalence is validated.
|
|
- Test implications: Add paired US/metric cases for temperature, apparent
|
|
temperature, wind speed, and gust. Assert equivalent canonical summaries and
|
|
indicators at values immediately below, at, and above the thresholds,
|
|
including periods with both unit fields populated.
|
|
- Validation: Equivalent US and metric inputs produce the same heat, cold, and
|
|
wind decisions and unambiguous summary values; all immediate derived-fact
|
|
consumers retain correct units.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 6.
|
|
|
|
### AUD-017: Overnight dayparts omit alerts that begin after midnight
|
|
|
|
- Stage: 7
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/forecast.BuildDailySummary` and `overlapsWithin`
|
|
- Evidence: `BuildDailySummary` first limits alerts to the civil-day period and
|
|
then derives each daypart's alerts only from that reduced set. Configured
|
|
daypart windows may cross midnight, such as 22:00-06:00, and hourly period
|
|
selection correctly includes the next morning. An alert beginning at 01:00
|
|
the next day does not overlap the prior civil day, so it is discarded before
|
|
the overnight window is evaluated even though it overlaps that window.
|
|
Existing tests separately protect direct alert overlap and overnight hourly
|
|
grouping, but do not combine the two through daily-summary construction.
|
|
- Contract at risk: Each daypart summary must be derived from its complete
|
|
configured half-open period, including the next-day portion of a wrapping
|
|
window.
|
|
- Impact: A report can include next-morning forecast conditions in its
|
|
overnight daypart while omitting an active alert for exactly those hours,
|
|
understating overnight risk.
|
|
- Recommendation: Compute daypart alert overlaps directly from the source
|
|
alert run and each daypart period. Keep the top-level daily alert list scoped
|
|
to the civil day so its separate contract does not widen.
|
|
- Test implications: Add an integration case with an overnight daypart and an
|
|
alert wholly after midnight. Assert that the alert appears clipped in the
|
|
daypart, does not appear in the prior day's top-level list, and an alert
|
|
beginning exactly at the daypart end remains excluded.
|
|
- Validation: Every wrapping daypart includes all and only alerts overlapping
|
|
its full period, while daily alert summaries retain civil-day boundaries.
|
|
- Related findings: `AUD-002`
|
|
- Remediation reference: Remediation Stage 6.
|
|
|
|
### AUD-018: Invalid precipitation percentages flow into derived summaries
|
|
|
|
- Stage: 7
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/forecast.BuildPrecipTiming`, `SummarizeDaypart`, and
|
|
`weatherdata.ForecastPeriod.ProbabilityOfPrecipitationPercent`
|
|
- Evidence: Both derivation paths accept any non-nil probability without
|
|
enforcing the percentage domain. Values above 100 can become daypart or
|
|
timing maxima and open precipitation windows, while negative values can
|
|
become maxima in otherwise sparse input. The Weather API normalization
|
|
boundary decodes the field but does not range-check it. Focused tests cover
|
|
zero, ordinary values, and values immediately around the default 40-percent
|
|
threshold, but not values below zero or above 100.
|
|
- Contract at risk: A field named and consumed as a percentage must remain
|
|
within 0 through 100 before meteorological comparisons or presentation.
|
|
- Impact: Malformed upstream data can produce impossible precipitation facts
|
|
and misleading timing windows while collection and derivation both report
|
|
success.
|
|
- Recommendation: Enforce the numeric domain at the normalization boundary so
|
|
every consumer receives valid percentages. Define whether a malformed value
|
|
makes required hourly input fail or is omitted with an explicit warning;
|
|
derivation should not silently clamp it because that hides source corruption.
|
|
- Test implications: Add normalization and derivation cases for negative,
|
|
100, and above-100 values, plus non-finite values if the wire decoder can
|
|
produce them. Protect the exact threshold and zero semantics already tested.
|
|
- Validation: No accepted derived fact contains a probability outside 0-100,
|
|
and malformed source behavior is deterministic and visible under the chosen
|
|
source-failure policy.
|
|
- Related findings: `AUD-014`
|
|
- Remediation reference: Remediation Stage 6.
|
|
|
|
### AUD-019: Ordinary “icy” forecast text does not set the ice indicator
|
|
|
|
- Stage: 7
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/forecast.indicatorsForText`
|
|
- Evidence: Ice detection checks for the literal substrings `ice`, `freezing`,
|
|
and `sleet`. The common condition adjective `icy` contains none of those
|
|
byte sequences, so a description such as “Icy roads” does not set the ice
|
|
indicator. Focused tests exercise broader fixture summaries but provide no
|
|
table of representative positive and negative condition phrases.
|
|
- Contract at risk: The documented text indicators identify snow, ice, fog,
|
|
and wind mentions in normalized forecast descriptions.
|
|
- Impact: Reports can omit an ice hazard indicator for plainly icy conditions.
|
|
Other text or numerical signals may still reveal the condition, which keeps
|
|
severity low.
|
|
- Recommendation: Define a small explicit vocabulary or token-aware matcher
|
|
for supported condition forms, including `icy`, rather than relying only on
|
|
incidental substrings. Keep the vocabulary bounded to avoid false positives.
|
|
- Test implications: Add table-driven phrases for ice/icy/freezing/sleet,
|
|
mixed case, punctuation, and nearby negative words; retain separate tests
|
|
for snow, fog, wind, heat, and cold.
|
|
- Validation: Representative icy descriptions set the ice indicator without
|
|
broadening matches to unrelated words.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 6.
|
|
|
|
### AUD-020: Pointer-shaped AFD options pass validation but fail construction
|
|
|
|
- Stage: 8
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/briefing.ModuleDefinition.ValidateOptions`,
|
|
`buildAreaForecastDiscussionModule`, and
|
|
`internal/config.normalizeModuleOptions`
|
|
- Evidence: Option validation explicitly accepts either the declared value
|
|
type or a pointer to it. Configuration normalization returns an already-valid
|
|
typed value unchanged. The area-forecast-discussion builder, the only
|
|
in-scope builder that consumes its options, asserts only the non-pointer
|
|
`module.AreaForecastDiscussionOptions` type. Therefore
|
|
`&module.AreaForecastDiscussionOptions{...}` passes composition and config
|
|
validation but later fails module construction with an option-type error.
|
|
The focused pointer acceptance test uses `CurrentConditionsOptions`, whose
|
|
builder ignores options, so it does not protect the end-to-end option
|
|
contract.
|
|
- Contract at risk: An option shape accepted by registry and configuration
|
|
validation must be consumable by the corresponding builder without a later
|
|
type failure.
|
|
- Impact: A programmatically constructed report configuration can pass
|
|
preflight and then fail report preparation solely because it uses the
|
|
registry's documented accepted pointer form. Loaded YAML normally decodes to
|
|
the value form, limiting the affected path but making the failure surprising.
|
|
- Recommendation: Choose one canonical builder input shape. Prefer normalizing
|
|
accepted pointers to the declared value type before storing/building, or
|
|
reject pointers consistently if they are not part of the intended contract.
|
|
- Test implications: Exercise the AFD option consumer through composition,
|
|
constructed-config normalization, and `BuildModule` with both accepted
|
|
shapes. Assert identical selected sections, not merely validation success.
|
|
- Validation: Every option value accepted by `ValidateOptions` reaches the AFD
|
|
builder in a usable canonical form, and invalid shapes still fail during
|
|
preflight.
|
|
- Related findings: `AUD-006`
|
|
- Remediation reference: Remediation Stage 7.
|
|
|
|
### AUD-021: Empty weather-story data is published as available
|
|
|
|
- Stage: 8
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/briefing.buildWeatherStoryModule` and the normalized
|
|
`weatherdata.WeatherStory` source boundary
|
|
- Evidence: Weather-story collection accepts a decoded non-null object without
|
|
a minimum content invariant. Unlike current conditions, narrative, hourly,
|
|
and AFD builders, `buildWeatherStoryModule` performs no empty-value check; any
|
|
non-nil pointer, including `&weatherdata.WeatherStory{}`, emits a module with
|
|
`Available: true`, blank period labels, and otherwise empty fields. Focused
|
|
tests cover a populated story and a missing nil source, but not a checked
|
|
structurally empty object.
|
|
- Contract at risk: Source-facing module availability must mean that usable
|
|
source content exists; missing, empty, and available states must not be
|
|
conflated.
|
|
- Impact: Prompt and render consumers can be told a weather story is available
|
|
when there is no title, description, alternate text, image, or valid period,
|
|
producing misleading or empty report sections.
|
|
- Recommendation: Define the smallest usable weather-story invariant at the
|
|
normalization boundary and apply optional-source policy when it is not met.
|
|
Retain a defensive module-level empty check so programmatic fact inputs
|
|
cannot assert availability without content.
|
|
- Test implications: Add a local adapter case for an empty object and a
|
|
briefing case for an empty normalized story. Protect populated zero-valued
|
|
booleans/order independently from the content fields used to determine
|
|
availability.
|
|
- Validation: Empty or unusable stories follow the declared missing-source
|
|
policy or are omitted, while a usable story emits `Available: true` with its
|
|
structured fields intact.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 7.
|
|
|
|
### AUD-022: Module snapshots accept unsupported schema versions
|
|
|
|
- Stage: 8
|
|
- Status: rejected
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: compatibility
|
|
- Area: `internal/module.Snapshot.Validate`
|
|
- Evidence: A snapshot declares the exact version
|
|
`weatherreporter.modules.v1`, and `NewSnapshot` always assigns it, but
|
|
`Snapshot.Validate` rejects only an empty version. A manually decoded,
|
|
cloned, or programmatically constructed snapshot labeled with an old or
|
|
future nonempty version passes this validation and can continue to prompt or
|
|
render consumers. The prompt-input package's analogous validation requires
|
|
its exact current version. Module tests assert constructor output and missing
|
|
structural fields but do not exercise an unsupported version.
|
|
- Contract at risk: A versioned module envelope must reject versions whose
|
|
field and value semantics the current consumer does not implement.
|
|
- Impact: Current production construction uses the right constant, so ordinary
|
|
reports are unaffected. The exported validation boundary nevertheless gives
|
|
false compatibility assurance to alternate/test callers and to any future
|
|
decoded snapshot path.
|
|
- Recommendation: Require `SchemaVersion == SnapshotSchemaVersion` during
|
|
validation and return an error naming the supported version.
|
|
- Test implications: Add one focused unsupported-version case alongside the
|
|
current constructor/version and duplicate-output tests.
|
|
- Validation: Only the current module snapshot schema passes validation; all
|
|
normal snapshot construction and downstream consumers remain unchanged.
|
|
- Disposition: Rejected as a standalone compatibility fix. The only producer
|
|
is `NewSnapshot`, which always writes the current constant, and no decoded or
|
|
external snapshot workflow exists. The narrow validation inconsistency is
|
|
retained as part of the broader no-owner decision in `AUD-072`; hardening it
|
|
independently would add a contract for a surface the repository may retire.
|
|
- Related findings: `AUD-072`
|
|
- Remediation reference: Not applicable; consolidated into `AUD-072` and
|
|
Remediation Stage 31.
|
|
|
|
### AUD-023: Apparent temperature is published as heat index
|
|
|
|
- Stage: 9
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/briefing.derivedDailySummaryValue` and
|
|
`DerivedDailySummaryModule.HeatIndexMaxF`
|
|
- Evidence: Daypart derivation aggregates the source's generic
|
|
`ApparentTemperatureF` values into `DaypartSummary.ApparentTemperature`.
|
|
The daily briefing builder takes the maximum of that range unconditionally
|
|
and publishes it as `heat_index_max_f`. There is no heat-indicator,
|
|
temperature, humidity, or source-field check that establishes the value as a
|
|
heat index. The ordinary focused test uses a hot apparent value and therefore
|
|
confirms only the coincidentally valid case; no cold, mild, or wind-chill
|
|
case exists.
|
|
- Contract at risk: User-facing derived fields must retain the meteorological
|
|
meaning of their source values rather than relabel a generic feels-like
|
|
temperature as a specific heat hazard measurement.
|
|
- Impact: Cold and ordinary days can carry a purported heat-index maximum that
|
|
is actually wind chill or another apparent-temperature estimate. Prompt and
|
|
render consumers can then make a false heat claim from a correctly collected
|
|
source value.
|
|
- Recommendation: Preserve the field as apparent temperature with unit-aware
|
|
naming, or populate a heat-index field only from an explicit heat-index
|
|
source or a documented, validated derivation under heat-appropriate
|
|
conditions.
|
|
- Test implications: Add hot, mild, and below-freezing apparent-temperature
|
|
cases. Assert that only a semantically established heat index receives that
|
|
label, while generic apparent temperature remains available under its own
|
|
identity.
|
|
- Validation: Cold and mild inputs never emit a heat-index claim; hot inputs
|
|
retain an accurately named value; focused briefing and downstream schema or
|
|
render tests pass.
|
|
- Related findings: `AUD-016`
|
|
- Remediation reference: Remediation Stage 8.
|
|
|
|
### AUD-024: Normalized daypart keys can silently overwrite summaries
|
|
|
|
- Stage: 9
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `buildDerivedDaypartSummariesModule`, `daypartKey`, `normalizedKey`,
|
|
and daypart configuration validation
|
|
- Evidence: The builder stores every derived daypart in a map keyed by a
|
|
lowercase name whose non-alphanumeric runs become underscores. It assigns
|
|
without checking whether the key already exists. Configuration requires only
|
|
a nonblank name and parseable clocks; it does not require unique names or
|
|
unique normalized names. Consequently duplicate names, or distinct names
|
|
such as `early morning` and `early-morning`, produce the same key and the
|
|
later summary silently replaces the earlier one. Existing tests exercise
|
|
ordinary unique default names only.
|
|
- Contract at risk: Every configured local-time window selected into derived
|
|
facts must remain represented in the briefing module and prompt package.
|
|
- Impact: A valid configuration can silently lose a complete daypart's
|
|
temperature, precipitation, hazard, and alert facts, causing generated
|
|
reports to reason from an incomplete day.
|
|
- Recommendation: Define one canonical daypart identity policy and reject
|
|
duplicate canonical identities during configuration validation. Retain a
|
|
defensive collision error in the module builder so programmatic facts cannot
|
|
overwrite data silently.
|
|
- Test implications: Add exact-duplicate, case-only, punctuation-normalized,
|
|
and non-colliding Unicode/name cases at configuration and builder boundaries.
|
|
Assert an explicit error rather than map length alone.
|
|
- Validation: Every accepted daypart maps to one distinct output entry, and
|
|
every collision is rejected before prompt construction.
|
|
- Related findings: `AUD-025`
|
|
- Remediation reference: Remediation Stage 10.
|
|
|
|
### AUD-025: Planning applies configurable daypart identities inconsistently
|
|
|
|
- Stage: 9
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `buildTodayPlanning`, `buildMorningCommuteOvernightPlanning`, and
|
|
`daypartNamed`
|
|
- Evidence: Morning, overnight, afternoon, and evening role lookup uses
|
|
case-insensitive `daypartNamed`, while the commute/workday loop excludes only
|
|
names exactly equal to lowercase `overnight` or `evening`. Daypart names are
|
|
configurable nonblank strings, and neither configuration nor derivation
|
|
canonicalizes them into semantic roles. A configured `Overnight` window is
|
|
therefore used for overnight-watch guidance and also treated as a daytime
|
|
commute/workday window; renamed windows silently lose the special planning
|
|
role altogether. Focused planning tests use only lowercase default names.
|
|
- Contract at risk: Equivalent accepted configuration identities must produce
|
|
equivalent planning behavior, and display labels must not double as
|
|
inconsistently compared semantic keys.
|
|
- Impact: Harmless casing changes can duplicate overnight/evening hazards into
|
|
workday concerns, while customized labels can replace real readiness or
|
|
late-day analysis with generic fallback text.
|
|
- Recommendation: Separate stable semantic daypart roles from display names,
|
|
or explicitly constrain and canonicalize the names accepted by
|
|
configuration. Apply that policy once across every planning lookup and
|
|
exclusion.
|
|
- Test implications: Exercise default, mixed-case, and renamed configurations
|
|
through Today, Tomorrow, and Daily planning, asserting which windows feed
|
|
morning, workday, late-day, and overnight sections.
|
|
- Validation: Equivalent daypart configurations yield identical planning
|
|
classification, and unsupported role configurations fail explicitly rather
|
|
than degrading to fallback prose.
|
|
- Related findings: `AUD-024`
|
|
- Remediation reference: Remediation Stage 10.
|
|
|
|
### AUD-026: Outdoor-window ranking ignores snow, ice, and fog
|
|
|
|
- Stage: 9
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `scoreOutdoorWindow` and Today outdoor planning
|
|
- Evidence: Outdoor scoring considers precipitation probability, gust speed,
|
|
alert count, heat, and cold. It does not inspect the already-derived snow,
|
|
ice, or fog indicators. When one of those indicators occurs without a high
|
|
precipitation probability, gust, alert, heat, or cold signal, the daypart
|
|
receives score zero and the reason `quiet weather`; it can be selected as the
|
|
best outdoor window. The same planning helper file correctly treats snow and
|
|
ice as travel concerns and fog as a morning visibility concern, proving the
|
|
facts are available. Existing outdoor tests cover precipitation, wind,
|
|
alerts, heat, and cold but no indicator-only winter or fog case.
|
|
- Contract at risk: Best/worst outdoor guidance and its reasons must account
|
|
for material weather constraints already present in the selected daypart.
|
|
- Impact: A snowy, icy, or foggy period can be recommended as the day's best
|
|
outdoor window and explicitly called quiet, creating misleading planning
|
|
advice even when upstream classification is correct.
|
|
- Recommendation: Give each supported outdoor hazard an explicit scoring and
|
|
reason policy, sharing the indicator vocabulary with planning notes where
|
|
appropriate. Document tie behavior and avoid implying safety merely from a
|
|
zero score.
|
|
- Test implications: Add isolated snow, ice, and fog cases plus mixed hazards
|
|
and deterministic ties. Assert both ranking and user-facing reasons.
|
|
- Validation: None of the supported hazard-only periods is described as quiet,
|
|
and best/worst selection reflects the documented relative policy.
|
|
- Related findings: `AUD-019`
|
|
- Remediation reference: Remediation Stage 8.
|
|
|
|
### AUD-027: Subzero temperatures are rendered as positive zero-degree bands
|
|
|
|
- Stage: 9
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `temperatureBandPhrase`, `temperatureBandIndex`, and derived daypart
|
|
temperature trend prose
|
|
- Evidence: Both helpers compute a decade with Go integer division toward zero.
|
|
Values from `-9` through `-1` therefore receive decade `0`; the absolute
|
|
remainder then produces phrases such as `mid 0s` or `upper 0s`, which denote
|
|
positive temperatures. The same band index can also distort trend ordering
|
|
around zero. Focused phrase tests cover only positive values in the upper
|
|
60s through lower 80s, so realistic cold-weather values are unprotected.
|
|
- Contract at risk: Temperature phrases and trend summaries must preserve the
|
|
sign and ordering of the underlying Fahrenheit values.
|
|
- Impact: A dangerous below-zero forecast can be understated by roughly ten
|
|
degrees and described on the wrong side of zero in prompt-facing daypart
|
|
summaries.
|
|
- Recommendation: Define explicit signed temperature-band semantics, including
|
|
conventional `below zero` wording and monotonic band ordering across zero,
|
|
rather than applying positive-decade arithmetic to negative values.
|
|
- Test implications: Add table-driven values on both sides of every zero-area
|
|
boundary, including `-11`, `-10`, `-9`, `-5`, `-1`, `0`, `1`, and `9`, plus
|
|
rising/falling trends that cross zero.
|
|
- Validation: Every negative input produces unambiguously below-zero wording,
|
|
and band indices remain monotonic from colder to warmer values.
|
|
- Related findings: `AUD-016`
|
|
- Remediation reference: Remediation Stage 8.
|
|
|
|
### AUD-028: Embedded official SPC category descriptions have drifted
|
|
|
|
- Stage: 9
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `assets/spc_convective_outlook_definitions.json` and
|
|
`spcOutlookBackgroundDefinition`
|
|
- Evidence: The embedded asset labels one field `official_description` but
|
|
carries no source URL, source date, or revision metadata. The
|
|
[current NOAA Storm Prediction Center outlook reference](https://www.spc.noaa.gov/about/outlooks/),
|
|
updated March 3, 2026, defines
|
|
Thunderstorms as an area with a 10% or higher thunderstorm probability and
|
|
describes Marginal through High using current organization, coverage, and
|
|
intensity language. Multiple embedded strings differ materially, including
|
|
Thunderstorms as only `No severe thunderstorms expected` and Slight as
|
|
isolated intense storms that are short-lived or not widespread. Tests lock
|
|
the key set and nonblank grammar but do not compare externally owned meaning
|
|
or provenance. The CIG vocabulary and conditional-versus-probability warning
|
|
otherwise align with the current NOAA explanation.
|
|
- Contract at risk: Content presented as an official external definition must
|
|
identify and track its authoritative source closely enough to avoid
|
|
misrepresenting current risk terminology.
|
|
- Impact: Every matching SPC outlook can carry stale or inaccurately attributed
|
|
background text into prompts and reports, where users may treat it as NOAA's
|
|
current official description.
|
|
- Recommendation: Refresh the categorical descriptions from the current SPC
|
|
reference, record source URL and revision date beside the asset, and define a
|
|
periodic review/update owner. If repository prose is intentionally
|
|
interpretive, rename the field so it is not represented as verbatim official
|
|
guidance.
|
|
- Test implications: Retain schema/key/lookup tests, add required provenance,
|
|
and protect stable domain identifiers rather than freezing unsourced prose.
|
|
A maintainer review should compare externally owned wording at each update.
|
|
- Validation: Every `official_description` is traceable to the recorded
|
|
authoritative revision, and repository-authored summaries are labeled as
|
|
such; asset and briefing tests pass.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 11.
|
|
|
|
### AUD-029: Raw source warnings cross the curated prompt boundary
|
|
|
|
- Stage: 10
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: security
|
|
- Area: `internal/promptinput.Metadata`, `promptinput.Package.SourceWarnings`,
|
|
and `weatherdata.SourceWarning.Endpoint`
|
|
- Evidence: `promptinput.Build` copies the complete normalized
|
|
`[]weatherdata.SourceWarning` into the top-level prompt package, and YAML
|
|
serialization includes each warning's `endpoint`. The Weather API adapter
|
|
populates that field with the source's fixed relative transport path whenever
|
|
an optional source is missing or malformed. In contrast, the briefing
|
|
metadata module intentionally projects warnings into a prompt-safe type
|
|
without endpoint or provenance fields, and its test asserts those omissions.
|
|
No prompt-input test asserts that raw weather-domain fields or transport
|
|
paths are absent.
|
|
- Contract at risk: Only curated prompt-facing values may cross the model
|
|
boundary; raw weather-domain and transport/provenance fields must not be
|
|
serialized merely because they are present on an upstream type.
|
|
- Impact: An optional-source failure sends an internal relative transport path
|
|
to Promptkit and the selected model provider and duplicates a safer warning
|
|
summary already in the briefing. Current adapter paths contain no host,
|
|
query, or credentials, limiting present disclosure, but future fields added
|
|
to the raw shared type would serialize without a prompt-boundary review.
|
|
- Recommendation: Replace the top-level raw warning field with a prompt-owned
|
|
safe projection, or remove it in favor of the metadata-module summary. Admit
|
|
only reviewed source, code, severity, bounded safe message, and completeness
|
|
fields; never endpoint, query, hash, or response detail.
|
|
- Test implications: Build and marshal a package from a warning with a unique
|
|
endpoint marker, then assert the marker is absent while intended safe warning
|
|
facts remain. Keep the assertion at the serialized provider-boundary owner.
|
|
- Validation: The exact bytes submitted for prompt execution contain no raw
|
|
transport/provenance fields; ordinary warning meaning and existing
|
|
deterministic YAML tests remain intact.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 1.
|
|
|
|
### AUD-030: The shared prompt instructs nonexistent briefing paths
|
|
|
|
- Stage: 10
|
|
- Status: confirmed
|
|
- Severity: high
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/promptassets/assets/prompts/common/data_package.user.md`
|
|
and `internal/promptinput.BriefingStanzas`
|
|
- Evidence: The shared user prompt directs every report prompt to read
|
|
`briefing.derived_daily_summary`, `briefing.derived_daypart_summaries`,
|
|
`briefing.precip_timing`, and `briefing.outdoor_windows`. The deterministic
|
|
serializer places all four under `briefing.derived_summaries.<stanza>`.
|
|
It also directs alert relevance checks to `briefing.metadata.alerts`, but the
|
|
prompt-facing metadata module has no `alerts` field; relevant counts live in
|
|
`briefing.applicable_risk_products.alert_digest`. Asset tests verify prompt,
|
|
profile, input-schema, and output-schema identities and successfully inspect
|
|
the real Promptkit definitions, but do not relate prose field references to
|
|
a representative serialized package.
|
|
- Contract at risk: Embedded instructions and the exact curated input schema
|
|
must evolve as one provider-facing contract.
|
|
- Impact: All four normal report prompts can overlook their primary derived
|
|
summaries or search the wrong alert location, pushing the model toward raw
|
|
data, unsupported inference, or false statements about alert relevance.
|
|
- Recommendation: Update the common prompt to name the actual categorized
|
|
paths and define one maintainable contract manifest or representative
|
|
serialized fixture from which semantic path assertions can be made. Avoid a
|
|
brittle whole-prose snapshot.
|
|
- Test implications: Inspect all embedded report prompts and assert the shared
|
|
instruction references paths present in a representative real
|
|
`promptinput.MarshalYAML` output, including derived and alert examples.
|
|
- Validation: Every named input path in the shared instruction resolves in the
|
|
current serialized package and a category move or prompt-path regression
|
|
fails one focused compatibility test.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 1.
|
|
|
|
### AUD-031: The shared prompt overstates SPC product locality
|
|
|
|
- Stage: 10
|
|
- Status: confirmed
|
|
- Severity: high
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/promptassets/assets/prompts/common/data_package.user.md`,
|
|
`briefing.buildSPCConvectiveOutlooks`, and
|
|
`briefing.buildSPCConvectiveDiscussion`
|
|
- Evidence: The shared instruction says all active hazard and risk products
|
|
have already been filtered or matched to the forecast location and should be
|
|
treated as locally relevant. The SPC outlook module deliberately preserves
|
|
all selected overlapping records; only its categorical `risk_digest`
|
|
applies `ContainsLocation`, rank, type, and overlap filters. SPC discussions
|
|
are regional text selected by qualifying outlook day/type/rank/overlap, not
|
|
by point containment. Stage 9 retained this distinction as `RET-020` and
|
|
focused builder tests protect it, but the prompt-assets suite does not
|
|
protect the corresponding instruction semantics.
|
|
- Contract at risk: Safety-sensitive prompt guidance must distinguish
|
|
location-matched conclusions from regional context retained for grounded
|
|
interpretation.
|
|
- Impact: A model can present a non-containing SPC outlook record or regional
|
|
discussion as a hazard at the configured point, overstating severe-weather
|
|
risk despite the curated digest carrying the correct local conclusion.
|
|
- Recommendation: State explicitly that the alert relevant count and SPC
|
|
categorical risk digest are location-matched, while individual outlook
|
|
records and discussions require their containment/geographic limitations to
|
|
be preserved. Remove the blanket locality claim.
|
|
- Test implications: Add a focused semantic assertion for the shared
|
|
instruction and a representative package containing a non-local outlook
|
|
alongside a location-matched digest. Do not duplicate SPC selection tests.
|
|
- Validation: The embedded prompt cannot reasonably instruct a consumer to
|
|
treat regional or non-containing SPC material as point-local, while still
|
|
requiring locally relevant digest facts to lead the report.
|
|
- Related findings: `AUD-030`
|
|
- Remediation reference: Remediation Stage 1.
|
|
|
|
### AUD-032: The exported YAML loader accepts malformed package shapes
|
|
|
|
- Stage: 10
|
|
- Status: rejected
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/promptinput.LoadYAML` and `Validate`
|
|
- Evidence: Focused temporary probes confirmed that `LoadYAML` accepts a
|
|
package with no metadata stanza, ignores a value-map entry absent from
|
|
briefing order, ignores unknown top-level or report fields, and silently
|
|
accepts a trailing YAML document. `Validate` checks the current schema,
|
|
report identity, nonempty order, ordered-value presence, duplicates, and
|
|
known category placement, but not metadata presence or an exact
|
|
order/value bijection. The decoder neither enables `KnownFields(true)` nor
|
|
performs a second decode expecting EOF. Inbound graph traces find only test
|
|
callers for this loader, limiting current runtime impact.
|
|
- Contract at risk: Loading and validating an exported data package should
|
|
reject ambiguous, incomplete, or forward-unknown representations rather
|
|
than normalize them silently.
|
|
- Impact: Tooling or a future production consumer can accept a package that
|
|
loses stanzas on remarshal, lacks required provenance, carries unnoticed
|
|
unsupported fields, or hides a second document, giving a false compatibility
|
|
signal. The live Build-to-Marshal execution path remains valid.
|
|
- Recommendation: Require metadata exactly once, enforce an exact order/value
|
|
bijection, enable strict known-field decoding for structured fields, and
|
|
reject all content after the first YAML document.
|
|
- Test implications: Add a compact table for missing metadata, orphan value,
|
|
unknown root/nested fields, and a second document, retaining the current
|
|
round-trip, duplicate, misplaced, and old-schema cases.
|
|
- Validation: Each malformed probe fails with bounded context, one canonical
|
|
document round-trips deterministically, and no production execution behavior
|
|
changes.
|
|
- Disposition: Rejected as a standalone hardening finding. The malformed-input
|
|
probes are reproducible, but graph and text searches show that tests are the
|
|
loader's only callers. `AUD-072` owns the prior question—retire the loader or
|
|
identify a real consumer—so prescribing strict decoding first has
|
|
insufficient cost-benefit evidence.
|
|
- Related findings: `AUD-072`
|
|
- Remediation reference: Not applicable; consolidated into `AUD-072` and
|
|
Remediation Stage 31.
|
|
|
|
### AUD-033: Stanza serialization failures lack a focused contract test
|
|
|
|
- Stage: 10
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: testing
|
|
- Area: `internal/promptinput.BriefingStanzas.MarshalYAML` and `yamlNode`
|
|
- Evidence: Stanza values are normalized through JSON before YAML, and the
|
|
implementation wraps conversion failures with the stanza name. Existing
|
|
tests cover deterministic output, category placement, round trips,
|
|
uncategorized stanzas, and schema rejection, but none supplies an
|
|
unserializable value such as a non-finite float to execute this failure
|
|
branch. Normal registry outputs currently use fixed serializable shapes, so
|
|
no present default module triggers it.
|
|
- Contract at risk: A serialization failure must remain contextual and must
|
|
stop execution before any provider call; this behavior should not depend on
|
|
incidental library diagnostics.
|
|
- Impact: A future or changed prompt export can fail without a durable test
|
|
protecting stanza attribution, making provider-preflight errors harder to
|
|
diagnose or tempting unsafe fallback serialization. Current default reports
|
|
are unaffected.
|
|
- Recommendation: Add one focused prompt-input test using a deliberately
|
|
unsupported stanza value and assert a non-nil, stanza-attributed error. The
|
|
application-level no-provider-work consequence belongs to Stage 14.
|
|
- Test implications: Protect the repository context and error category, not
|
|
the complete encoding-library message; do not duplicate every unsupported
|
|
Go value shape. Stage 14 confirmed that application tests also do not inject
|
|
this failure and prove zero executor calls, so add that single consequence
|
|
assertion at the preparation owner rather than repeating serializer cases.
|
|
- Validation: The focused test fails if conversion errors lose stanza context
|
|
or are swallowed, and deterministic successful serialization remains green.
|
|
- Related findings: `AUD-043`
|
|
- Remediation reference: Remediation Stage 1.
|
|
|
|
### AUD-034: Prompt-debug credential redaction misses common aliases and URL forms
|
|
|
|
- Stage: 11
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: security
|
|
- Area: `internal/promptdebug.safePromptDebugEndpoint`,
|
|
`redactPromptDebugParameters`, and `isPromptDebugSecretKey`
|
|
- Evidence: Debug preparation deliberately persists the resolved provider
|
|
endpoint and effective extra parameters after key-based redaction. The key
|
|
normalizer recognizes strings containing credential, secret, password,
|
|
token, API key, or authorization. A deterministic temporary probe confirmed
|
|
that common synthetic aliases such as access-key, signature, auth, and cookie
|
|
retain their marker values in endpoint queries or nested parameters. URL
|
|
user information and the existing API-key fixture are redacted, but URL path
|
|
credentials and unrecognized query names are also retained. The probe was
|
|
removed after reproduction and no sensitive fixture value was recorded.
|
|
- Contract at risk: Prompt-debug artifacts may contain content-rich execution
|
|
detail, but their explicit safe mapping and focused tests promise that
|
|
provider credentials are not persisted.
|
|
- Impact: An operator profile or backend using an unrecognized credential
|
|
alias or signed URL can write a live credential into `preparation.json`.
|
|
Files are mode `0600` beneath a mode `0700` root, limiting exposure, but the
|
|
credential becomes durable and subject to backup, retention, or later
|
|
permission mistakes despite the artifact claiming redaction.
|
|
- Recommendation: Stop trying to prove arbitrary provider maps and complete
|
|
URLs safe through a denylist. Persist an allowlisted endpoint projection and
|
|
allowlisted non-secret execution parameters, or replace all provider-specific
|
|
extras with names/type summaries. Define how signed paths and queries are
|
|
omitted rather than expanding an indefinitely incomplete alias list.
|
|
- Test implications: Add table-driven synthetic markers across URL user info,
|
|
paths, common query aliases, case/separator variants, and nested maps/lists;
|
|
assert that no marker reaches serialized debug bytes while reviewed ordinary
|
|
parameters remain useful. Do not use real credential-like fixture values.
|
|
- Validation: Every supported debug projection is allowlisted, adversarial
|
|
marker tests find no value leak, and normal summaries remain content-free.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 12.
|
|
|
|
### AUD-035: Prompt-debug symlink checks can be raced before writes
|
|
|
|
- Stage: 11
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: security
|
|
- Area: `internal/promptdebug.ensureSecureDirectory`, `runDirectory`, and
|
|
`writeSecureJSON`
|
|
- Evidence: Directory validation walks path strings with `os.Lstat` and rejects
|
|
every symlink visible at that moment. `runDirectory` then joins and creates
|
|
paths by name, and `writeSecureJSON` later creates a temporary file and
|
|
renames it through the same pathname. No directory handle pins the validated
|
|
root or descendants, so a process able to replace a component in a writable
|
|
ancestor can swap it for a symlink after validation and before `Mkdir`,
|
|
`CreateTemp`, or `Rename`. Static root/component symlink tests and concurrent
|
|
missing-ancestor tests pass, but they cannot protect this check/use window.
|
|
- Contract at risk: Sensitive debug artifacts must remain beneath the
|
|
explicitly authorized root, and symlink rejection must hold at the file
|
|
operation rather than only during an earlier observation.
|
|
- Impact: Under an explicitly enabled debug run and a replaceable ancestor, a
|
|
local filesystem attacker can race sensitive rendered prompts or model
|
|
output into a directory outside the authorized root. Restrictive final modes
|
|
do not restore path containment after redirection.
|
|
- Recommendation: Anchor traversal and creation to opened directory handles
|
|
and use no-follow, directory-only operations for every component and final
|
|
replacement, with platform-specific handling where necessary. Alternatively
|
|
require and document a pre-existing trusted root while still pinning it for
|
|
each write; string containment and repeated `Lstat` are not sufficient.
|
|
- Test implications: Add a deterministic filesystem seam or supported-platform
|
|
race harness that replaces a component between validation and creation and
|
|
proves no artifact appears outside the root. Retain the cheaper static
|
|
symlink, mode, collision, and concurrent-distinct-run cases.
|
|
- Validation: Debug creation and atomic replacement remain relative to the
|
|
originally validated root handle across component swaps; attempted
|
|
redirection fails closed without an external file or partial target.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 12.
|
|
|
|
### AUD-036: The generated-text catalog accepts a known pair for the wrong report
|
|
|
|
- Stage: 12
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/generatedtext.LookupDefinition` and the generated-text
|
|
catalog
|
|
- Evidence: Catalog entries contain only schema ID, template ID, validator,
|
|
and render-context builder. `LookupDefinition` accepts the first matching
|
|
schema/template pair and copies the caller's report ID into the handler
|
|
without checking that the pair belongs to that report. A temporary probe
|
|
changed a copied Daily definition to the known `hourly`/`hourly` pair; lookup
|
|
returned an Hourly handler with no error. The current registry loop test
|
|
compares handler IDs with the same definition it supplied, and the report
|
|
registry test requires only nonempty schema/template IDs, so neither test
|
|
detects a coherent but wrong known pair.
|
|
- Contract at risk: Each generated-text report definition must resolve to its
|
|
one report-specific schema, validator, template, and context builder before
|
|
provider work begins; the separate Daily, Today, Tomorrow, and Hourly value
|
|
types preserve that identity even where fields currently match.
|
|
- Impact: A registry or constructed-definition regression can route a report
|
|
through another report's output contract. Depending on the selected pair it
|
|
can fail only after provider execution during context assembly, or render a
|
|
semantically wrong report if the downstream inputs remain structurally
|
|
compatible.
|
|
- Recommendation: Include `report.ID` in every catalog entry and require the
|
|
exact report/schema/template triple. Keep unknown schema, unknown template,
|
|
known-but-mismatched pair, and wrong-report diagnostics distinct and bounded.
|
|
- Test implications: Assert the exact four report-to-pair mappings from an
|
|
independent expectation table, and reject every known pair when attached to
|
|
a different report. Avoid deriving the expected pair from the definition
|
|
under test.
|
|
- Validation: Every current definition resolves once, and swapping any two
|
|
known pairs fails during lookup before prompt inspection or provider work.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 13.
|
|
|
|
### AUD-037: Typed decoding accepts property names rejected by the schema
|
|
|
|
- Stage: 12
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/generatedtext.decodeGeneratedText` and the four generated-
|
|
text schemas
|
|
- Evidence: The schemas require the exact lowercase names `summary`,
|
|
`forecast_discussion`, and `precipitation_timing` while rejecting additional
|
|
properties. Go's `encoding/json` matches tagged struct fields without regard
|
|
to ASCII case. A temporary probe showed that Hourly validation accepts
|
|
`Summary` and `Forecast_Discussion`, then normalizes them to the canonical
|
|
lowercase names. A real offline Promptkit execution against the embedded
|
|
schema classified the same response as failed. The exact-name raw-map check
|
|
happens only for `precipitation_timing`, leaving the other two fields
|
|
inconsistent.
|
|
- Contract at risk: The schema validator and repository-owned typed validator
|
|
should accept the same property vocabulary, and typed decoding is documented
|
|
as rejecting unknown fields without a second JSON Schema engine.
|
|
- Impact: The normal Promptkit path currently rejects these aliases first, but
|
|
direct handler callers or another conforming executor that reports completed
|
|
validation can accept output the canonical schema forbids. This weakens the
|
|
typed validator as an independent boundary and makes behavior depend on the
|
|
executor implementation.
|
|
- Recommendation: Inspect top-level raw object keys with exact matching before
|
|
struct decoding, rejecting case variants, duplicates, missing names, and
|
|
additions under one explicit policy. Preserve typed decoding for field
|
|
shapes and semantic normalization.
|
|
- Test implications: Add one shared exact-key contract across all four public
|
|
validators and an embedded-schema integration case proving the two layers
|
|
agree. Do not repeat every casing variant in each report-specific suite.
|
|
- Validation: A corpus of canonical, missing, added, case-varied, duplicate,
|
|
null, and wrong-type fields produces compatible schema and Go decisions.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 13.
|
|
|
|
### AUD-038: Generated prose and decoder diagnostics have no size boundary
|
|
|
|
- Stage: 12
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: reliability
|
|
- Area: the generated-text schemas, `decodeGeneratedText`, day-style arrays,
|
|
and application propagation of validation errors
|
|
- Evidence: None of the four schemas declares string-length, array-item, or
|
|
total-output bounds, and the Go validators impose none before decoding,
|
|
trimming, filtering, and normalizing. Embedded profiles intentionally omit
|
|
output-token limits. Temporary offline probes showed both Promptkit schema
|
|
validation and `ValidateHourly` accepting and copying a synthetic response
|
|
with a two-MiB summary. Another probe used a 64-KiB synthetic unknown-field
|
|
name; `encoding/json` copied the complete name into a 65,603-byte error.
|
|
`profileExecutionError` and `generatedReportError` wrap validator text
|
|
verbatim, so an executor that reaches this independent layer can carry that
|
|
content into ordinary errors. All probes were removed.
|
|
- Contract at risk: Provider output crosses a bounded, content-safe validation
|
|
boundary before normalization and rendering; ordinary errors must exclude
|
|
response-body content and remain safe to summarize or log.
|
|
- Impact: A provider, fake, or future executor can force repeated large
|
|
allocations in validation/normalization and produce an unexpectedly large
|
|
Markdown artifact from schema-valid prose. Malformed field names can also
|
|
turn provider-controlled response text into normal diagnostics. Provider
|
|
context limits and current Promptkit rejection of unknown fields reduce but
|
|
do not define the repository's accepted size or independent error contract.
|
|
- Recommendation: Define reviewed total raw-output, per-string, and day-style
|
|
paragraph-count/aggregate bounds; express compatible bounds in JSON Schema
|
|
and enforce the total before decoding. Translate decoder failures into
|
|
bounded categories that identify canonical fields without echoing arbitrary
|
|
keys or raw content.
|
|
- Test implications: Exercise exact boundary and one-over cases for total
|
|
bytes, string lengths, paragraph count/aggregate size, and oversized unknown
|
|
names with synthetic markers. Assert normalized bytes and every ordinary
|
|
error remain bounded without duplicating all cases per day-style report.
|
|
- Validation: Schema and Go limits agree, excessive output fails before
|
|
normalization/rendering, and no malformed provider value can make normal
|
|
error text exceed the documented diagnostic bound.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 13.
|
|
|
|
### AUD-039: Generated-text tests duplicate semantic and rendered-output ownership
|
|
|
|
- Stage: 12
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: testing
|
|
- Area: `internal/generatedtext/{daily,today,tomorrow,day_style,render_context}_test.go`
|
|
and schema/render assertions in
|
|
`internal/reporttemplate/reporttemplate_test.go`
|
|
- Evidence: Daily, Today, and Tomorrow each carry nearly line-for-line copies
|
|
of normalization and malformed-input tables, while the shared day-style
|
|
suite repeats summary, discussion, normalization, empty timing, missing
|
|
timing, unknown-field, and retired-field behavior across the same three
|
|
validators. Separately, `reporttemplate_test.go` imports `promptassets` and
|
|
repeats four schema-shape assertions, including three duplicated day-style
|
|
blocks, even though `reporttemplate` owns Markdown assets and
|
|
`promptassets_test.go` already owns the schema inventory, strict-object
|
|
shape, required fields, identity, and copy isolation. The four full
|
|
render-context tests additionally assert substantial report-specific
|
|
Markdown fragments and ordering already owned by reporttemplate; the two
|
|
large suites changed together 11 times with a 0.73 graph coupling score.
|
|
- Contract at risk: Similar public validators should remain independently
|
|
protected without copying shared semantics, and schema tests should reside
|
|
with the asset owner rather than an unrelated template package.
|
|
- Impact: One intentional day-style or schema change requires synchronized
|
|
edits across several files and packages, creates noisy failures outside the
|
|
responsible owner, and can still miss cross-layer discrepancies such as
|
|
`AUD-037` because duplicated examples are not an agreement test.
|
|
- Recommendation: Keep one table-driven shared day-style behavior suite with a
|
|
thin per-exported-validator identity/type case. Move complete schema-shape
|
|
ownership to `promptassets`; keep only a generated-text integration contract
|
|
that compares schema acceptance with typed validation where their semantics
|
|
intentionally overlap. Remove schema assertions from `reporttemplate`. Keep
|
|
report-specific typed projections in generatedtext and Markdown semantics in
|
|
reporttemplate, with one compact real-context render smoke test proving that
|
|
the boundaries compose.
|
|
- Test implications: Preserve distinct Hourly behavior and each public report
|
|
identity while deleting repeated semantic cases. A mechanical refactor of a
|
|
private day-style helper should not require unrelated test edits.
|
|
- Validation: The leaner suite still fails for a wrong report dispatch,
|
|
schema/type disagreement, required-field regression, or normalization
|
|
change, and report-template tests fail only for template/render behavior.
|
|
- Related findings: `AUD-037`, `AUD-063`
|
|
- Remediation reference: Remediation Stage 32.
|
|
|
|
### AUD-040: Render contexts accept conflicting report identities
|
|
|
|
- Stage: 13
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/generatedtext.BuildHourlyRenderContext`, the three day-style
|
|
context builders, and their module metadata projection
|
|
- Evidence: Each exported builder selects its report-specific title, context
|
|
type, and module surface from the function called, but none checks
|
|
`briefing.Metadata.ReportID` or reconciles the separately decoded
|
|
`Modules.Metadata.ReportID`. A temporary probe passed outer metadata labeled
|
|
Hourly and a snapshot metadata stanza labeled Daily to
|
|
`BuildDailyRenderContext`; it returned a Daily context without error, titled
|
|
it as Daily, and retained the conflicting Daily module identity. Focused
|
|
context tests supply consistent identities but have no mismatch case.
|
|
Combined with `AUD-036`, a Daily definition selecting the known Hourly pair
|
|
can reach the Hourly context builder with Daily metadata and render rather
|
|
than fail at this boundary.
|
|
- Contract at risk: A report-specific render context must be assembled only
|
|
from metadata and module values for that report, with one authoritative
|
|
identity before repository-owned Markdown rendering.
|
|
- Impact: A registry, preparation, or direct package regression can render the
|
|
wrong report shape while preserving conflicting metadata inside the same
|
|
context. Current top-level templates do not display `Modules.Metadata`, but
|
|
it is a documented template field and future edits can choose a different
|
|
identity or timing source without an error exposing the inconsistency.
|
|
- Recommendation: Make the expected report identity explicit in the common
|
|
builder path, reject mismatched outer metadata, and when a metadata stanza is
|
|
present verify its report ID, run ID, prompt ID, period, timezone, and other
|
|
duplicated canonical metadata against the outer value. Prefer one documented
|
|
template-facing owner for overlapping metadata rather than two unchecked
|
|
copies.
|
|
- Test implications: Add one shared mismatch table covering all four builders
|
|
and one outer-versus-module metadata disagreement case. Keep successful
|
|
report-specific context assertions without duplicating every metadata field
|
|
per report.
|
|
- Validation: Every builder rejects another report's metadata and any present
|
|
metadata stanza that disagrees with the canonical context input; all four
|
|
normal prepared contexts still render.
|
|
- Related findings: `AUD-036`
|
|
- Remediation reference: Remediation Stage 15.
|
|
|
|
### AUD-041: Dynamic prose can impersonate deterministic Markdown sections
|
|
|
|
- Stage: 13
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: security
|
|
- Area: all top-level report templates and their interpolation of generated
|
|
text and source-derived module strings through `text/template`
|
|
- Evidence: `reporttemplate.Render` uses `text/template`, which performs no
|
|
Markdown or HTML escaping, and the templates insert generated summary,
|
|
forecast-discussion, and precipitation-timing strings directly. The schemas
|
|
permit newlines and Markdown characters, while typed validation only trims
|
|
outer whitespace. A temporary probe placed a second-level `Alert Digest`
|
|
heading and fabricated warning bullet in an otherwise valid Daily summary;
|
|
rendering succeeded and preserved both as active Markdown immediately before
|
|
the repository-owned Alert Digest position. The same raw interpolation is
|
|
used for externally sourced alert labels, condition text, daypart display
|
|
strings, timing phrases, and other module values.
|
|
- Contract at risk: Generated text fills bounded prose slots only;
|
|
deterministic facts remain authoritative, and repository-owned templates
|
|
alone define Markdown structure.
|
|
- Impact: Malformed model output, indirect prompt injection through source
|
|
material, or unexpected upstream Markdown can create headings, bullets,
|
|
links, raw HTML, or formatting that visually masquerades as deterministic
|
|
alerts and forecast facts. Schema validation does not distinguish prose from
|
|
Markdown structure, so a validation-passed provider response can corrupt the
|
|
trust boundary of the published report.
|
|
- Recommendation: Define field-specific Markdown policies at the context or
|
|
renderer boundary. Preserve intended paragraph breaks while escaping or
|
|
rejecting block structure, raw HTML, unsafe links, control characters, and
|
|
inline delimiters where they are not part of the contract; render module
|
|
labels and values through reviewed inline-safe helpers. Do not rely solely on
|
|
prompt instructions or downstream Markdown sanitization.
|
|
- Test implications: Use synthetic markers across each generated prose slot
|
|
and representative module label/value positions; assert none can introduce
|
|
a heading, list item, code block, raw HTML block, or unsafe link while normal
|
|
punctuation and intended paragraphs remain readable. Include the complete
|
|
validation-to-render path for at least one report.
|
|
- Validation: Parsing rendered output as Markdown shows that only template-
|
|
owned nodes create report headings and deterministic list structure, and
|
|
adversarial dynamic strings remain text within their authorized slots.
|
|
- Related findings: `AUD-038`
|
|
- Remediation reference: Remediation Stage 15.
|
|
|
|
### AUD-042: Today can render a daypart heading with no rows or fallback
|
|
|
|
- Stage: 13
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `templates/partials/today_daypart_forecast.md.tmpl`
|
|
- Evidence: The Today partial decides its fallback from whether
|
|
`.Modules.Dayparts` is nonempty, then independently suppresses every row
|
|
whose summary lacks both `DominantConditionDisplay` and
|
|
`DominantCondition`. A temporary probe supplied one named daypart with no
|
|
displayable condition; output contained `## Daypart Forecast` but neither a
|
|
row nor `No daypart forecast details are available.` Existing tests protect
|
|
the zero-length fallback and suppression of one empty Evening row beside
|
|
valid rows, but not the all-filtered case.
|
|
- Contract at risk: Missing deterministic values should produce an explicit,
|
|
coherent fallback or omit the section, not leave an empty Markdown heading.
|
|
- Impact: When selected Today dayparts exist structurally but all lack usable
|
|
condition summaries, the published report presents a blank section that
|
|
implies lost content and gives the operator no explanation.
|
|
- Recommendation: Compute whether at least one Today row is displayable before
|
|
emitting the heading. Render valid rows when present; otherwise render the
|
|
existing no-details fallback, or consistently omit the complete section if
|
|
that becomes the documented policy.
|
|
- Test implications: Cover zero rows, all-filtered rows, mixed valid/filtered
|
|
rows, and all-valid rows through the shared partial's observable output.
|
|
- Validation: The Daypart Forecast heading is always followed by at least one
|
|
row or the explicit fallback, and mixed rows retain their existing order.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 15.
|
|
|
|
### AUD-043: Generated-text compatibility is validated after collection
|
|
|
|
- Stage: 14
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/app.InspectPromptExecution`,
|
|
`InspectPromptExecutions`, `InspectComparisonExecution`, and `prepareReport`
|
|
- Evidence: Prompt preflight validates the exact prompt ID/version, its one
|
|
YAML input, the configured output schema path, the effective profile, and
|
|
credentials before weather collection. It never validates the report's
|
|
generated-text template or complete catalog binding. `prepareReport` calls
|
|
`generatedtext.LookupDefinition` only after collection and after building
|
|
report facts, the module snapshot, metadata, the prompt package, and its
|
|
serialized YAML. A temporary offline probe changed a copied Daily definition
|
|
to an unknown template while retaining its valid prompt and schema;
|
|
`InspectPromptExecution` succeeded. The same definition fails catalog lookup
|
|
only during preparation. Current default registry entries are valid.
|
|
- Contract at risk: Every configuration-independent report, schema, template,
|
|
validator, and renderer precondition must fail before external collection or
|
|
model execution, leaving collected-data validation and serialization for the
|
|
post-collection preparation boundary.
|
|
- Impact: A registry regression or constructed definition with an unknown or
|
|
mismatched template can pass advertised preflight, perform all remote weather
|
|
requests, and build the complete prompt package before failing without a
|
|
model call. This wastes externally visible work and weakens the promise that
|
|
report-contract defects are configuration preflight failures.
|
|
- Recommendation: Resolve and validate the exact report/schema/template
|
|
catalog entry as part of pre-collection inspection or report resolution,
|
|
after strengthening lookup to enforce the report-specific triple in
|
|
`AUD-036`. Carry the validated immutable handler into preparation so the
|
|
post-collection path does not perform a weaker or duplicate lookup.
|
|
- Test implications: At the application owner, inject an invalid template and
|
|
known-but-wrong report pair and assert no collector or executor call. Keep
|
|
catalog pairing cases in `internal/generatedtext`; the app test should
|
|
protect only ordering and consequences.
|
|
- Validation: All invalid report/catalog bindings fail before collection in
|
|
ordinary, batch, and comparison preflight, while each valid definition is
|
|
resolved once and no model execution occurs on later preparation failures.
|
|
- Related findings: `AUD-036`
|
|
- Remediation reference: Remediation Stage 16.
|
|
|
|
### AUD-044: Execution provenance is accepted without reconciliation
|
|
|
|
- Stage: 14
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/app.inspectPromptContract`, `inspectPromptProfile`, and
|
|
`executePreparedProfile`
|
|
- Evidence: Inspection verifies returned prompt and profile IDs, input shape,
|
|
output shape, and credentials, but accepts an empty prompt hash, backend ID,
|
|
and model name. Execution sends the inspected prompt/profile IDs and an
|
|
independent data-package copy, then trusts the preparation callback to
|
|
replace result profile/backend/model identity. It does not compare the
|
|
prepared report definition with the inspection, require the callback, or
|
|
reconcile callback and completed `Execution` prompt ID/version/hash,
|
|
profile/backend/model, output contract, validation mode, or schema path. A
|
|
temporary executor probe returned mutually different inspected, callback,
|
|
and execution identities plus `ValidationPassed` for an unrelated mode and
|
|
schema path; `executePreparedProfile` accepted the raw Daily-shaped JSON and
|
|
rendered Markdown successfully. Existing fakes and the current Promptkit
|
|
adapter return coherent provenance, so focused tests never exercise a
|
|
mismatch.
|
|
- Contract at risk: A validation-passed output must be demonstrably produced
|
|
by the exact inspected prompt version and selected profile, against the
|
|
prepared report's exact schema, before repository-owned typed validation and
|
|
rendering can attribute it to that report.
|
|
- Impact: An adapter regression, stale inspection passed by an internal caller,
|
|
or alternate executor can render and later publish output from a different
|
|
prompt, profile, backend, model, or schema while the active result records
|
|
callback-supplied identity. Similar output shapes can make the error silent
|
|
rather than causing typed validation to fail.
|
|
- Recommendation: Define one application-owned provenance validator. Require
|
|
complete safe inspection identity, match it to the prepared definition and
|
|
request, count exactly one preparation callback, reconcile callback and
|
|
completed execution identity/hash fields, and require the expected JSON
|
|
Schema validation mode and path before accepting `ValidationPassed`.
|
|
- Test implications: Add one table-driven execution-contract suite that
|
|
changes one provenance field or omits the callback, plus one coherent happy
|
|
path. Assert mismatch errors are classified and content-safe, produce no
|
|
rendered bytes, and retain only verified partial-result fields.
|
|
- Validation: Every provenance mismatch fails before generated-text decoding
|
|
or rendering; coherent Promptkit and fake executions retain the exact
|
|
inspected prompt hash and selected profile/backend/model in results and
|
|
debug artifacts.
|
|
- Related findings: `AUD-036`, `AUD-037`, `AUD-040`
|
|
- Remediation reference: Remediation Stage 16.
|
|
|
|
### AUD-045: Single-report publication replaces non-regular destinations
|
|
|
|
- Stage: 15
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/app.validateOutputPath` and
|
|
`internal/fileutil.WriteFileAtomic`
|
|
- Evidence: Destination validation follows the final path with `os.Stat` and
|
|
rejects it only when the result is a directory. FIFOs, sockets, devices, and
|
|
other non-regular objects therefore pass; a symlink to a regular file also
|
|
passes even though rename replaces the symlink itself rather than its target.
|
|
`WriteFileAtomic` creates a regular temporary file and calls `os.Rename`
|
|
without reclassifying the target. A temporary Unix probe created a FIFO at an
|
|
explicit Daily output path, ran the complete offline generation workflow,
|
|
and confirmed successful publication had replaced the FIFO with a regular
|
|
Markdown file. Existing app tests protect only a directory collision, and
|
|
fileutil's rename-error test also uses a directory.
|
|
- Contract at risk: An operator-selected report destination may replace an
|
|
absent path or an existing regular report file, but publication must reject
|
|
filesystem objects whose semantics or ownership are not ordinary report
|
|
content.
|
|
- Impact: A typo or name collision can silently destroy a named pipe, Unix
|
|
socket entry, device node where permissions permit, or other special object
|
|
and report success. Accepted symlink behavior is also surprising: the link
|
|
is removed instead of atomically updating the file it names.
|
|
- Recommendation: Use `Lstat` to define and enforce an explicit final-component
|
|
policy before work and immediately before rename. Accept only absent or
|
|
regular-file destinations; either reject symlinks or document and implement
|
|
one deliberate link policy without following at validation and replacing at
|
|
publication.
|
|
- Test implications: Add a portable table for absent, regular, directory, and
|
|
symlink targets plus platform-gated FIFO/socket cases. Assert invalid targets
|
|
fail before prompt inspection or collection and remain unchanged.
|
|
- Validation: Every accepted existing target is a regular report file under
|
|
the documented symlink policy; non-regular objects retain their type and
|
|
identity with zero collector, executor, or notifier calls.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 17.
|
|
|
|
### AUD-046: Valid long output names fail through the temporary-name pattern
|
|
|
|
- Stage: 15
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/fileutil.WriteFileAtomic` and explicit single-report output
|
|
paths
|
|
- Evidence: The atomic writer gives `os.CreateTemp` the pattern
|
|
`.` plus the complete destination basename plus a random suffix and `.tmp`.
|
|
A destination component can therefore fit the filesystem's name limit while
|
|
its derived temporary component does not. A temporary offline probe used a
|
|
valid 248-byte `.md` basename beneath a missing directory on the current
|
|
filesystem. Resolution, prompt inspection, collection, model execution, and
|
|
rendering all completed; publication created the parent, then failed with
|
|
`ENAMETOOLONG` while creating the temporary file. A direct write to the exact
|
|
target succeeded afterward, proving the requested component itself was
|
|
valid. The failed run also left the newly created empty output directory,
|
|
showing that failure cleanup covers the temporary file but not parent
|
|
directories created by the same publication attempt.
|
|
- Contract at risk: A filesystem-valid operator-selected output name should
|
|
not become invalid solely because of an internal temporary naming scheme;
|
|
publication side effects and cleanup should also match the documented output
|
|
lifecycle.
|
|
- Impact: An explicit long output path fails only after all remote and model
|
|
work, returns no published path, and can leave a directory requiring manual
|
|
cleanup. Default report names are short, limiting current exposure to custom
|
|
output paths.
|
|
- Recommendation: Use a short fixed temporary prefix independent of the final
|
|
basename while retaining same-directory rename. Validate any unavoidable
|
|
platform path constraint before external work, and reconcile directory
|
|
creation/cleanup semantics without deleting concurrently created content.
|
|
- Test implications: Add a filesystem-boundary case whose final component is
|
|
valid but would overflow the old temp pattern; assert successful replacement,
|
|
mode, content, and no temporary residue. Add a publication-failure case for
|
|
the documented missing-directory lifecycle through an injectable stable
|
|
filesystem boundary rather than relying on a platform error string.
|
|
- Validation: Every accepted final basename can create its temporary sibling,
|
|
and a failed publication leaves neither a partial file nor an app-created
|
|
empty directory under the documented lifecycle.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 17.
|
|
|
|
### AUD-047: Cancellation is not checked at the publication linearization point
|
|
|
|
- Stage: 15
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: reliability
|
|
- Area: `internal/app.publishPromptReport`, `publicationContextError`, and
|
|
`internal/fileutil.WriteFileAtomic`
|
|
- Evidence: The application checks `ctx.Err()` once before calling the atomic
|
|
writer. The writer then creates missing directories, creates and writes the
|
|
temporary file, closes it, and finally publishes with `os.Rename`; it accepts
|
|
no context and performs no check immediately before rename. Thus cancellation
|
|
that becomes observable during directory creation or temporary-file I/O is
|
|
ignored and the destination is still replaced. The linearization point is
|
|
the successful same-directory rename. Current cancellation and deadline
|
|
tests arrange cancellation before `publishPromptReport` starts, so they prove
|
|
only the earlier check. Unbounded generated Markdown from `AUD-038` can widen
|
|
the unchecked write interval.
|
|
- Contract at risk: Cancellation observed immediately before publication must
|
|
preserve the prior operator-owned report; completed rendering does not itself
|
|
authorize replacement after the run is canceled.
|
|
- Impact: A canceled command can replace an existing report and, with
|
|
notification disabled, return success if cancellation occurs after the one
|
|
check. With notification enabled it can instead report a later notification
|
|
failure while retaining a publication the operator expected cancellation to
|
|
prevent.
|
|
- Recommendation: Make the atomic writer context-aware, cleanly write and close
|
|
its temporary file, then classify and return cancellation/deadline immediately
|
|
before rename. Keep the application check for fast failure, but make the
|
|
filesystem boundary own the final pre-commit observation.
|
|
- Test implications: Add a deterministic writer seam or controlled atomic-write
|
|
test that cancels after temporary-file close but before rename; assert the
|
|
old destination and empty `OutputPath` remain, no notifier runs, and no temp
|
|
survives. Retain the current cancellation-before-publication cases.
|
|
- Validation: Cancellation or deadline made observable at the final pre-rename
|
|
gate prevents replacement and returns the matching stable category; once
|
|
rename succeeds, results truthfully report publication regardless of later
|
|
notification outcome.
|
|
- Related findings: `AUD-038`
|
|
- Remediation reference: Remediation Stage 17.
|
|
|
|
### AUD-048: Unsupported Distributor endpoint forms fail only after publication
|
|
|
|
- Stage: 16
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/config.validateDistributorNotify`,
|
|
`internal/adapters/distributor.newDistributorUploadClient`, and
|
|
`internal/app.notifyBatch`
|
|
- Evidence: Active configuration accepts any URL with a scheme and host. The
|
|
pinned Distributor client accepts only `http` or `https` and additionally
|
|
rejects userinfo, query strings, and fragments. Consequently values such as
|
|
`ftp://distributor.example.test`, or an HTTP URL with userinfo or a query,
|
|
pass repository configuration validation. Client construction is deferred
|
|
until notification, which occurs after a single report or every successful
|
|
batch report has already been published. Existing config coverage rejects a
|
|
relative endpoint but has no cases for the forms rejected by the production
|
|
upload client.
|
|
- Contract at risk: The documented Distributor endpoint is an absolute HTTP(S)
|
|
base URL, and active configuration should reject unsupported transport forms
|
|
before expensive collection, model execution, and local publication.
|
|
- Impact: A configuration that can never contact Distributor can generate and
|
|
replace all selected Markdown files before reporting a notification failure.
|
|
Operators receive a late partial outcome instead of an actionable startup
|
|
error, and credentials embedded as URL userinfo can also be repeated in
|
|
repository-owned error context before the dependency rejects them.
|
|
- Recommendation: Validate the complete production endpoint grammar while
|
|
loading active Distributor configuration: require HTTP(S), a host, and no
|
|
userinfo, query, or fragment. Keep the dependency check as defense in depth
|
|
and avoid echoing endpoint userinfo in errors.
|
|
- Test implications: Extend enabled Distributor config tests with unsupported
|
|
schemes, userinfo, query, and fragment cases plus accepted HTTP(S) base paths.
|
|
Add an application assertion that invalid active endpoints fail before
|
|
collection or publication.
|
|
- Validation: Every endpoint accepted by configuration can construct the
|
|
production upload client; rejected endpoint forms perform no collection,
|
|
report generation, publication, or notification.
|
|
- Related findings: `AUD-005`, `AUD-011`
|
|
- Remediation reference: Remediation Stage 18.
|
|
|
|
### AUD-049: Distributor response text crosses into normal batch diagnostics
|
|
|
|
- Stage: 16
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: security
|
|
- Area: the pinned `distributor/pkg/upload` response errors,
|
|
`internal/adapters/distributor.wrapUploadError`,
|
|
`internal/app.notificationResultFromUpload`, and batch CLI output
|
|
- Evidence: For a non-success upload or status response, the pinned client uses
|
|
a decoded `error` string or otherwise the complete trimmed response body as
|
|
`HTTPError.Message`. The local adapter redacts only exact occurrences of the
|
|
configured bearer token, then preserves the remaining remote text in its
|
|
returned error or `StatusError`. Batch notification copies that text into
|
|
`BatchNotificationResult.Error`; JSON output retains it and normal stderr
|
|
prints it with `%q`. A terminal run's remote `error` field follows the same
|
|
path. Focused tests prove exact-token replacement but intentionally inject
|
|
benign strings and do not establish a safe remote-diagnostic vocabulary.
|
|
- Contract at risk: Normal command output must be safe for routine retention;
|
|
remote response bodies and provider diagnostics are not repository-owned
|
|
safe fields merely because one known token value was replaced.
|
|
- Impact: A misconfigured, compromised, or overly descriptive Distributor can
|
|
place credentials, infrastructure details, HTML, or other sensitive content
|
|
into durable command JSON, service logs, and terminal output. Variants of a
|
|
credential that do not exactly equal the loaded token survive redaction.
|
|
- Recommendation: Preserve typed status, retryability, conflict identity, and
|
|
local request context, but project remote messages to a bounded
|
|
repository-owned diagnostic category for normal results. Make raw upstream
|
|
response capture an explicit secure debug behavior if it is needed at all.
|
|
- Test implications: Use a local HTTP server that returns structured and plain
|
|
sensitive bodies for upload and status failures; assert normal result and
|
|
stderr/JSON projections contain stable safe context but none of the supplied
|
|
body. Retain exact bearer-token redaction as defense in depth.
|
|
- Validation: Arbitrary upstream body and status-error text cannot appear in
|
|
normal notification results or CLI output, while conflict, status, and local
|
|
identity remain actionable.
|
|
- Related findings: `AUD-012`, `AUD-034`
|
|
- Remediation reference: Remediation Stage 18.
|
|
|
|
### AUD-050: Distributor response bodies have no size boundary
|
|
|
|
- Stage: 16
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: reliability
|
|
- Area: the pinned `distributor/pkg/upload.Client.responseError`, accepted-upload
|
|
decoding, status decoding, and `internal/adapters/distributor.Upload`
|
|
- Evidence: The pinned client calls `io.ReadAll(response.Body)` for every HTTP
|
|
error with no limiting reader. Successful `202` upload responses and `200`
|
|
status responses are decoded directly from the body without a size limit;
|
|
status includes an arbitrary `json.RawMessage` report that the adapter and
|
|
application copy into results. The configured HTTP timeout bounds elapsed
|
|
time but not bytes or memory, and the local adapter adds no bounded transport
|
|
or decoder. Current fake-client tests never create an HTTP response body.
|
|
- Contract at risk: External response processing must enforce explicit resource
|
|
boundaries rather than trusting a remote service or transport timeout to
|
|
bound memory use.
|
|
- Impact: A Distributor endpoint can make one notification allocate memory
|
|
proportional to an arbitrarily large error, accepted response, or status
|
|
report, potentially terminating the process after all report files have
|
|
already been published. Large retained reports can also inflate CLI JSON.
|
|
- Recommendation: Add explicit, documented byte limits for accepted responses,
|
|
status responses/reports, and error diagnostics in the upstream client or a
|
|
repository-owned bounded boundary. Reject overflow distinctly; do not
|
|
silently truncate JSON or treat a prefix as a complete response.
|
|
- Test implications: Exercise each HTTP response class at the exact limit and
|
|
one byte over through a local server, including a valid oversized status
|
|
report and plain error body. Assert bounded failure, closed bodies, preserved
|
|
publication accounting, and no retry caused solely by local overflow.
|
|
- Validation: Memory use is bounded independently of remote body length and
|
|
oversized responses fail with a stable safe diagnostic.
|
|
- Related findings: `AUD-013`, `AUD-038`, `AUD-049`
|
|
- Remediation reference: Remediation Stage 18.
|
|
|
|
### AUD-051: Mid-batch cancellation is flattened into ordinary report failures
|
|
|
|
- Stage: 16
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: reliability
|
|
- Area: `internal/app.RunBatchDetailed`, `BatchError`, and batch CLI result
|
|
projection
|
|
- Evidence: The sequential report loop has no context check or cancellation
|
|
branch. Any cancellation returned during one report is stored as that item's
|
|
error, after which every later planned report is still passed to the same
|
|
generation pipeline with the canceled context and ordinarily fails too.
|
|
`RunBatchDetailed` then returns the aggregate with a nil error; `RunBatch` and
|
|
the CLI replace the cause with `BatchError`, which has no `Unwrap` method and
|
|
describes only a count of failed reports. Existing batch tests cover one
|
|
independent model failure and continued execution, but no cancellation or
|
|
deadline during the report sequence.
|
|
- Contract at risk: Continuing independent item failures must not turn shared
|
|
workflow cancellation into new work, and cancellation identity must survive
|
|
through partial results and the process-facing error path.
|
|
- Impact: An interrupted batch can invoke later model/debug/publication paths
|
|
unnecessarily, report several apparent generation failures for one shared
|
|
cancellation, and prevent callers from detecting `context.Canceled` or
|
|
`context.DeadlineExceeded`. Already published reports remain, but the summary
|
|
obscures why the batch stopped and which reports were never independently
|
|
attempted.
|
|
- Recommendation: Check context before starting each item and immediately after
|
|
an item returns. Stop scheduling later reports when cancellation or deadline
|
|
is observed, retain completed item/publication truth, and return a result plus
|
|
an error that preserves the context cause. Define an explicit unattempted or
|
|
canceled accounting state so totals and CLI summaries remain truthful, and
|
|
ensure the CLI does not overwrite a result-bearing context error.
|
|
- Test implications: Deterministically cancel during an early prompt and after
|
|
an early publication. Assert no later executor or notifier call, retained
|
|
successful files and counters, an explicit state for remaining planned
|
|
items, skipped batch notification, nonzero exit, and `errors.Is` for both
|
|
cancellation and deadline.
|
|
- Validation: Independent generation failures still allow later reports, while
|
|
shared context termination stops new work and remains identifiable at every
|
|
application and CLI boundary.
|
|
- Related findings: `AUD-008`, `AUD-047`
|
|
- Remediation reference: Remediation Stage 19.
|
|
|
|
### AUD-052: Distributor adapter tests bypass the production HTTP boundary
|
|
|
|
- Stage: 16
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: testing
|
|
- Area: `internal/adapters/distributor/client_test.go`
|
|
- Evidence: Every adapter test constructs `Client` with `newClient` and a fake
|
|
`uploadClientFactory`. No test invokes `New`, `newDistributorUploadClient`,
|
|
`distributorUploadClient.UploadFiles`, or its `Status` translation against a
|
|
local HTTP server. The suite therefore proves repository orchestration over a
|
|
substitute interface but not the production mapping to bundle entries,
|
|
endpoint paths, authorization, headers, response decoding, or context-aware
|
|
HTTP calls. The pinned dependency has its own unit tests, but those cannot
|
|
detect weatherreporter wiring regressions.
|
|
- Contract at risk: Adapter tests own realistic local external boundaries;
|
|
application tests should own publication and notification policy without
|
|
duplicating the dependency's internal retry implementation.
|
|
- Impact: Changes that drop a source mapping, creation time, idempotency key,
|
|
token, timeout, status field, or production constructor can pass all local
|
|
tests. The unsafe and unbounded response paths in `AUD-049` and `AUD-050` are
|
|
also invisible to the current suite.
|
|
- Recommendation: Add a small `httptest.Server` contract suite through `New`
|
|
that observes one upload and its status lookup, plus consequential error,
|
|
cancellation, and size-boundary cases. Keep fake-client tests only for fast
|
|
polling-clock or difficult injected failures, and do not repeat batch gating
|
|
or per-report publication policy in adapter tests.
|
|
- Test implications: The local server should inspect method, URL, bearer
|
|
authorization, idempotency header, archive mappings, and status projection;
|
|
call counts should demonstrate one application upload attempt while leaving
|
|
retry specifics to the pinned dependency's owner.
|
|
- Validation: At least one deterministic offline test traverses the actual
|
|
repository-to-dependency HTTP path, and each remaining fake has a boundary
|
|
behavior that the local server cannot express efficiently.
|
|
- Related findings: `AUD-049`, `AUD-050`
|
|
- Remediation reference: Remediation Stage 18.
|
|
|
|
### AUD-053: Comparison recognition accepts ambiguous manifest fields
|
|
|
|
- Stage: 17
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: compatibility
|
|
- Area: `internal/comparison.decodeManifest` and `RecognizeBundle`
|
|
- Evidence: The manifest decoder enables `DisallowUnknownFields`, but Go's
|
|
`encoding/json` still matches struct fields case-insensitively and accepts a
|
|
property more than once, using the last decoded value. A temporary package
|
|
probe confirmed that one manifest containing both `SchemaVersion` and
|
|
`schemaVersion` is accepted, and conflicting duplicate canonical properties
|
|
can likewise be resolved by their final occurrence. Recognition then treats
|
|
the directory as a current bundle eligible for destructive replacement.
|
|
Focused tests reject an unknown property and a trailing JSON value but do not
|
|
cover duplicate or case-variant names.
|
|
- Contract at risk: A replaceable bundle must exactly satisfy the current
|
|
manifest schema. Property names are case-sensitive canonical identities, and
|
|
duplicate JSON properties are ambiguous rather than an alternate encoding of
|
|
one logical manifest.
|
|
- Impact: A malformed or hand-edited directory can be authorized as a current
|
|
Weatherreporter bundle even though strict consumers may reject it or
|
|
interpret duplicate values differently. Because recognition gates
|
|
`--replace`, permissive decoding broadens the set of directories the program
|
|
is willing to move and recursively delete.
|
|
- Recommendation: Decode through a strict JSON object boundary that rejects
|
|
duplicate property names at every manifest object level and requires exact
|
|
canonical field spelling before typed unmarshalling and semantic validation.
|
|
Preserve the existing unknown-field and single-value checks.
|
|
- Test implications: Add package contract cases for duplicate top-level,
|
|
nested, result, and safe-error fields, conflicting duplicates, and
|
|
case-variant spellings. Assert `ErrUnrecognizedBundle` and preservation under
|
|
replacement planning for each malformed directory.
|
|
- Validation: Only exact, unique schema property names reach `Manifest.Validate`
|
|
and no ambiguous manifest can authorize replacement.
|
|
- Related findings: `AUD-037`
|
|
- Remediation reference: Remediation Stage 20.
|
|
|
|
### AUD-054: Manifest validation does not enforce canonical profile report names
|
|
|
|
- Stage: 17
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: compatibility
|
|
- Area: `internal/comparison.Manifest.Validate`, `LogicalBundle.Validate`, and
|
|
`RecognizeBundle`
|
|
- Evidence: `ReportFilename` implements the documented position width and
|
|
profile-slug algorithm, but manifest validation checks only that a successful
|
|
`reportPath` is a unique Markdown basename. Logical-bundle validation merely
|
|
requires its report path to equal that permissive manifest value. A temporary
|
|
package probe changed the first successful path from
|
|
`01-weather-light.md` to `arbitrary.md`; the complete logical bundle still
|
|
validated and could be published and recognized. Existing naming tests
|
|
exercise `ReportFilename` separately, so they do not connect that contract
|
|
to durable validation or replacement recognition.
|
|
- Contract at risk: Each successful report filename is
|
|
`NN-profile-slug.md`, derived from its one-based selected position, total
|
|
profile count, and authoritative logical profile ID. A recognized current
|
|
bundle must satisfy that durable layout rather than merely use any safe
|
|
basename.
|
|
- Impact: Weatherreporter can publish and later destructively recognize a
|
|
noncanonical bundle that consumers cannot reliably associate with the
|
|
documented selected-profile order. The naming helper and durable validator
|
|
can drift independently without a package-boundary failure.
|
|
- Recommendation: During manifest validation, derive the expected filename
|
|
with `ReportFilename(result.Position, manifest.Total, result.ProfileID)` for
|
|
every successful result and require exact equality. Keep failed results free
|
|
of report paths and retain the logical-bundle byte/path agreement check.
|
|
- Test implications: Add manifest, logical-bundle, encoded-round-trip, and
|
|
recognition cases for arbitrary safe basenames, wrong positions, wrong
|
|
ordinal widths, and incorrect or truncated slugs. One package-boundary
|
|
invariant can replace redundant helper-level examples if later test
|
|
consolidation warrants it.
|
|
- Validation: Every published and recognized successful report path is exactly
|
|
reproducible from the manifest's ordered profile identity.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 20.
|
|
|
|
### AUD-055: Replacement ignores cancellation after moving the prior bundle
|
|
|
|
- Stage: 17
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: reliability
|
|
- Area: `internal/comparison.publish` between `authorizeMovedDestination` and
|
|
replacement installation
|
|
- Evidence: Publication checks `ctx.Err()` immediately before the replacement
|
|
branch, then renames the prior target to a sibling backup and re-runs full
|
|
recognition against the moved entry. It performs no context check after that
|
|
potentially substantial reauthorization and before renaming staging into the
|
|
target. A temporary deterministic rename probe canceled the context after
|
|
the first rename; publication ignored the observable cancellation, installed
|
|
the new bundle, removed the backup, and returned `Committed: true` with no
|
|
error. The existing cancellation test cancels through `beforeCommit`, before
|
|
either rename, and therefore does not cover this transaction state.
|
|
- Contract at risk: Cancellation observed before the new bundle's install
|
|
linearization point must restore and preserve the prior recognized bundle.
|
|
Moving it to a recoverable sibling is preparation, not commit authorization.
|
|
- Impact: An operator can cancel while the old bundle is being reauthorized yet
|
|
still have it replaced and its backup deleted. The result reports success
|
|
even though cancellation was visible before the new target existed.
|
|
- Recommendation: Check context after moved-destination authorization and
|
|
immediately before installing staging. On cancellation or deadline, invoke
|
|
the existing guarded restoration path and preserve the context cause along
|
|
with any recovery failure details.
|
|
- Test implications: Cancel deterministically after the backup rename and
|
|
during moved-bundle recognition for both empty and recognized destinations.
|
|
Assert the old snapshot is restored, staging is removed, no backup remains
|
|
after successful restoration, `Committed` is false, and `errors.Is` retains
|
|
cancellation or deadline; also test retained-backup diagnostics when restore
|
|
fails.
|
|
- Validation: Every cancellation observable before the staging-to-target
|
|
rename either leaves or restores the exact prior destination and cannot
|
|
return a committed replacement.
|
|
- Related findings: `AUD-008`, `AUD-047`, `AUD-051`
|
|
- Remediation reference: Remediation Stage 21.
|
|
|
|
### AUD-056: Cleanup errors can leave an invalid retained backup
|
|
|
|
- Stage: 17
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: reliability
|
|
- Area: `internal/comparison.publish`, `PublicationResult`, and
|
|
`PublicationCleanupError`
|
|
- Evidence: After committing the new target, publication calls
|
|
`os.RemoveAll(backupDirectory)`. `RemoveAll` may delete some children before
|
|
returning its first error, but every error is reported as a committed result
|
|
with `RetainedBackupPath` and documented as the retained prior bundle. A
|
|
temporary deterministic cleanup probe removed the backup manifest and then
|
|
returned an error; publication advertised the absolute retained path even
|
|
though `RecognizeBundle` rejected the remaining directory. The focused test
|
|
fake returns an error without modifying the backup and asserts that the old
|
|
snapshot remains exact, which does not model the standard operation's partial
|
|
failure semantics.
|
|
- Contract at risk: Post-commit cleanup truth must distinguish an intact prior
|
|
bundle from partial cleanup remnants. Recovery guidance must not imply that a
|
|
path still contains a complete rollback artifact when deletion already
|
|
damaged it.
|
|
- Impact: An operator may preserve or archive a reported “backup” expecting the
|
|
prior bundle, only to discover later that required manifest, package, or
|
|
report files were already deleted. The new bundle remains truthfully
|
|
committed, but the recovery state and diagnostic overstate what survives.
|
|
- Recommendation: Model the post-cleanup state explicitly. On removal failure,
|
|
inspect the sibling without masking the original cause and report whether it
|
|
is an intact recognized prior bundle, partial remnants, or absent. Name result
|
|
fields and operational guidance accordingly; do not promise rollback
|
|
viability after recursive deletion has begun.
|
|
- Test implications: Through the deterministic cleanup seam, cover failure
|
|
before deletion, after deleting one file, and after removing the whole path.
|
|
Assert the committed new bundle remains recognized and the returned typed
|
|
state, safe summary, recovery path, and underlying error are truthful for
|
|
each case.
|
|
- Validation: Every cleanup failure reports the actual surviving state; only a
|
|
complete recognized old bundle is called a retained backup.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 21.
|
|
|
|
### AUD-057: Long comparison destinations fail through derived sibling names
|
|
|
|
- Stage: 17
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/comparison.PlanDestination`, staging creation in `publish`,
|
|
and `uniqueSiblingPath`
|
|
- Evidence: Destination planning accepts any filesystem-valid target basename,
|
|
but staging and backup patterns prepend a dot and append `.staging-` or
|
|
`.backup-` plus random text to that complete basename. On a filesystem with a
|
|
255-byte component limit, a 240-byte target component is valid and passes
|
|
read-only planning but `os.MkdirTemp` fails with `ENAMETOOLONG`. A temporary
|
|
probe also showed that publication creates previously missing parent
|
|
directories before this failure and leaves them behind. No focused test
|
|
covers boundary-length target components or the derived sibling names.
|
|
- Contract at risk: Every destination accepted at preflight must be capable of
|
|
forming its private transaction siblings, and failed staging must have an
|
|
explicit rule for parent-directory residue.
|
|
- Impact: An explicitly selected valid destination can fail only after
|
|
comparison generation has completed. If nested parents were missing, the
|
|
failed command also leaves empty operator-visible directories despite never
|
|
creating a bundle.
|
|
- Recommendation: Use bounded fixed transaction prefixes with random suffixes
|
|
that fit independently of the final basename, or reject targets whose
|
|
derived components cannot fit during read-only preflight. Decide and document
|
|
whether newly created empty parents are retained or safely rolled back.
|
|
- Test implications: Use filesystem-reported name limits where available and
|
|
cover the longest accepted final component, successful initial publication
|
|
and replacement, staging/backup cleanup, and the chosen missing-parent
|
|
failure lifecycle.
|
|
- Validation: Every preflighted basename can create both sibling forms, and a
|
|
pre-commit failure leaves only the documented parent-directory state.
|
|
- Related findings: `AUD-046`
|
|
- Remediation reference: Remediation Stage 21.
|
|
|
|
### AUD-058: Late cancellation overwrites completed profile failures
|
|
|
|
- Stage: 18
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: correctness
|
|
- Area: `internal/app.executeComparisonProfiles` and comparison cancellation
|
|
result construction
|
|
- Evidence: Every profile goroutine writes its own ordered outcome and joins
|
|
before aggregation. After the join, however, `executeComparisonProfiles`
|
|
checks `ctx.Err()` and calls `markCanceledComparisonOutcome` for every
|
|
outcome whose status is not `succeeded`. That sweep cannot distinguish an
|
|
unfinished cancellation from an independent failure that completed before a
|
|
peer caused cancellation; it replaces the earlier safe category, message,
|
|
and validation status with canceled/skipped. A temporary deterministic
|
|
two-profile probe released one profile into an ordinary generation failure,
|
|
waited until only its peer remained in flight, then canceled the shared
|
|
context. The completed outcome changed from the expected `generation`
|
|
category to `canceled`. Existing tests cover all-canceled execution and
|
|
independent failure without cancellation, but not their overlap.
|
|
- Contract at risk: Profile-local failures and workflow cancellation must
|
|
remain distinct and truthfully represented after all goroutines join; shared
|
|
cancellation may classify only work that did not independently complete.
|
|
- Impact: Operators and machine-readable CLI consumers lose the actual failure
|
|
class for an already completed provider or validation failure whenever a
|
|
concurrent peer later observes cancellation. This can hide the cause that
|
|
needs remediation and makes the ordered partial result historically false,
|
|
although cancellation correctly prevents bundle publication.
|
|
- Recommendation: Track whether each execution completed and preserve its
|
|
resulting outcome. Mark only unstarted or genuinely cancellation-terminated
|
|
work as canceled; use an explicit per-position execution state rather than
|
|
inferring completion from success alone.
|
|
- Test implications: Add a deterministic mixed-outcome test with one completed
|
|
generation or validation failure, one still-running profile, and subsequent
|
|
cancellation. Assert the first safe error and validation state survive, the
|
|
second is canceled, every goroutine is joined, and the application publishes
|
|
no bundle.
|
|
- Validation: For every completion ordering, independent failures retain their
|
|
original safe categories while only unfinished/canceled outcomes use the
|
|
cancellation category; result order and no-publication behavior remain
|
|
unchanged under the race detector.
|
|
- Related findings: `AUD-044`, `AUD-051`
|
|
- Remediation reference: Remediation Stage 22.
|
|
|
|
### AUD-059: Missing-credential test depends on the ambient environment
|
|
|
|
- Stage: 19
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: test reliability
|
|
- Area: `internal/adapters/promptkit.TestLocalBackendAndMissingCredentialBehavior`
|
|
- Evidence: The test creates a profile whose `api_key_env` is
|
|
`WEATHERREPORTER_TEST_MISSING_KEY` and expects inspection plus execution to
|
|
report a missing credential, but it never clears that process variable with
|
|
`t.Setenv`. A targeted run with the variable set to a synthetic value caused
|
|
the test to execute its fake provider successfully and fail its missing-key
|
|
assertion. The package's other credential-bearing execution test explicitly
|
|
sets `OPENROUTER_API_KEY`, and all Distributor/config environment tests use
|
|
`t.Setenv`, so this is the sole discovered uncontrolled credential read. No
|
|
live provider was contacted by the probe.
|
|
- Contract at risk: Default tests must be independent of real credentials and
|
|
ambient process state; a missing-credential case must establish absence
|
|
rather than assume a test-specific name is unused on every host.
|
|
- Impact: A developer, CI worker, or wrapper that defines this variable gets a
|
|
false suite failure and unexpectedly exercises the fake success path. The
|
|
unusual variable name limits frequency, and the injected client prevents a
|
|
live provider call.
|
|
- Recommendation: Call `t.Setenv("WEATHERREPORTER_TEST_MISSING_KEY", "")`
|
|
before inspection, keeping the test serial as required by Go's environment
|
|
mutation rules.
|
|
- Test implications: Retain the existing missing-credential assertions; verify
|
|
the focused test passes both when the parent process omits the variable and
|
|
when it supplies any value, because the test must own its effective state.
|
|
- Validation: A contaminated parent environment cannot change profile
|
|
inspection, provider-call count, error category, or test outcome, and the
|
|
full scrubbed/default suites continue to pass.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 23.
|
|
|
|
### AUD-060: Symlink tests assume host creation capability
|
|
|
|
- Stage: 19
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: test portability
|
|
- Area: symlink cases in `internal/app/output_test.go`,
|
|
`internal/comparison/publish_test.go`, and `internal/config/config_test.go`
|
|
- Evidence: Nine test sites call `os.Symlink`. Prompt-debug cases and one
|
|
comparison replacement table explicitly avoid unsupported Windows
|
|
semantics, but destination, recognition, output, and secret-entry cases in
|
|
the three named packages call `t.Fatal` for every creation error. Symlink
|
|
creation can be denied by host policy, filesystem capability, or Windows
|
|
privilege/developer-mode state even when the code under test is otherwise
|
|
runnable. The suite therefore fails during fixture setup rather than
|
|
classifying the unavailable prerequisite. Current Linux execution supports
|
|
symlinks and all cases pass, so the behavioral assertions themselves are
|
|
sound.
|
|
- Contract at risk: Machine-specific filesystem features used by tests must be
|
|
capability-controlled; the default suite should fail for product behavior,
|
|
not because an optional host operation is unavailable.
|
|
- Impact: Developers or CI runners on a restricted filesystem or unprivileged
|
|
Windows configuration can receive several unrelated package failures before
|
|
the intended symlink safety contracts execute. This does not affect runtime
|
|
behavior and supported Unix hosts retain coverage.
|
|
- Recommendation: Use a small package-local capability helper that attempts
|
|
symlink creation inside `t.TempDir`, skips only recognized unsupported or
|
|
permission-denied creation states, and treats all other setup errors as real
|
|
failures. Prefer capability detection to blanket operating-system skips so
|
|
enabled Windows hosts retain coverage.
|
|
- Test implications: Apply the same prerequisite handling to every symlink
|
|
case while preserving all assertions when creation succeeds. Exercise the
|
|
suite on one symlink-capable and one denied-capability host or CI job.
|
|
- Validation: Capable hosts still execute every symlink contract; incapable
|
|
hosts report narrowly explained skips rather than setup failures; other
|
|
filesystem and race tests remain unchanged.
|
|
- Related findings: `AUD-035`
|
|
- Remediation reference: Remediation Stage 23.
|
|
|
|
### AUD-061: SPC policy constants have a redundant implementation-coupled test
|
|
|
|
- Stage: 21
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: test maintenance
|
|
- Area: `internal/briefing.TestSPCRiskDigestDefaultPolicyConstants`
|
|
- Evidence: The test directly asserts that two unexported constants equal
|
|
`"categorical"` and `3`. The adjacent
|
|
`TestSPCConvectiveOutlooksRiskDigestFilters` already exercises the stronger
|
|
observable policy: categorical rank 3 is included, rank 2 is excluded, and a
|
|
non-categorical rank 30 product is excluded. Any consequential change to
|
|
either constant therefore fails the behavioral table without the direct
|
|
constant assertion.
|
|
- Contract at risk: Tests should protect the risk-digest selection policy, not
|
|
the private representation used to supply it.
|
|
- Impact: A semantics-preserving change that derives or configures the same
|
|
policy can require editing a test that provides no additional confidence.
|
|
The current cost is small but wholly redundant.
|
|
- Recommendation: Delete the constant test and retain the observable filter
|
|
table as the sole default-policy owner. Add relational boundary rows there
|
|
if the policy gains more dimensions rather than asserting its private fields.
|
|
- Test implications: Confidence remains in the module-level included/excluded
|
|
outputs, including both threshold and product-type boundaries; only eight
|
|
lines coupled to private names and storage disappear.
|
|
- Validation: Mutating either effective default changes the filter-table
|
|
outcome, while replacing the constants with an equivalent implementation
|
|
requires no test edit.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 24.
|
|
|
|
### AUD-062: Two integration tests pay production retry and polling intervals
|
|
|
|
- Stage: 21
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: test maintenance
|
|
- Area: `internal/collect.TestRunWrapsFetchError` and
|
|
`internal/adapters/distributor.TestUploadPollsUntilTerminalStatus`
|
|
- Evidence: An uncached JSON-timed suite reports these tests at 1.00 second and
|
|
0.25 second, respectively; every other top-level test completes in at most
|
|
0.04 second on the audit host. The collection test returns a retryable 502
|
|
from `/observations`, so production's one-second fetch retry delay dominates
|
|
a test whose assertions concern collection error wrapping. The Distributor
|
|
test deliberately returns accepted then succeeded and therefore pays the
|
|
fixed 250-millisecond production poll interval to assert two status calls.
|
|
Focused adapter tests separately own retry, cancellation, and timeout policy.
|
|
- Contract at risk: Tests should wait on events, not operational delays, unless
|
|
elapsed time is the contract under test.
|
|
- Impact: Every uncached suite spends avoidable time waiting; repeated and
|
|
shuffled runs multiply it. The waits also couple feedback cost to future
|
|
production timing changes without adding timing confidence.
|
|
- Recommendation: Make the collection wrapper case use a non-retryable source
|
|
failure, preserving its production-adapter traversal and error context. Give
|
|
the Distributor client a narrow test-controlled wait mechanism or interval
|
|
so accepted-to-terminal polling and call count remain real while elapsed
|
|
wall time approaches zero.
|
|
- Test implications: Collection still proves adapter construction, source
|
|
failure propagation, and collection context; Distributor still proves
|
|
accepted-to-terminal polling, report retention, and exactly two status
|
|
calls. Retry timing/cancellation remains in its focused adapter cases.
|
|
- Validation: Both tests preserve their current behavioral assertions, take no
|
|
production-sized sleep, and the suite's cancellation/timeout tests still
|
|
detect a wait mechanism that ignores context.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 24.
|
|
|
|
### AUD-063: Render-context tests duplicate template output ownership
|
|
|
|
- Stage: 21
|
|
- Status: rejected
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: test maintenance
|
|
- Area: `internal/generatedtext/render_context_test.go` and
|
|
`internal/reporttemplate/reporttemplate_test.go`
|
|
- Evidence: Each of the four full generated-text context tests first asserts
|
|
typed report/module projection, then calls `reporttemplate.Render` and checks
|
|
substantial report-specific Markdown fragments and ordering. The
|
|
reporttemplate suite independently owns those same four reports, partials,
|
|
absent-section behavior, ordering, and semantic output. The files are 1,397
|
|
and 1,183 lines, have changed 15 and 19 times in graph history, and changed
|
|
together 11 times with a 0.73 coupling score. The context suite also contains
|
|
a 185-line shared day-style projection test and four 126-147-line snapshot
|
|
fixtures, amplifying cross-owner edits when template wording changes.
|
|
- Contract at risk: Generated-text tests should own typed context construction;
|
|
reporttemplate tests should own Markdown semantics; only a representative
|
|
integration needs to prove the two contracts compose.
|
|
- Impact: A legitimate template-only change can fail two large suites and
|
|
require updating expectations outside the template owner. The duplicate
|
|
output checks add noisy diagnostics while offering little protection beyond
|
|
the focused template suite.
|
|
- Recommendation: Keep all report-specific typed context, optional-module, and
|
|
extraction assertions in generatedtext, and all report-specific Markdown
|
|
semantics in reporttemplate. Retain one small real-typed-context render smoke
|
|
case to protect integration. Consolidate only fixture fields common to the
|
|
day-style reports; keep report-specific planning and ordering values explicit.
|
|
- Test implications: Confidence remains in every context mapping and every
|
|
template output through their current focused owners, plus one cross-package
|
|
composition proof. Removing the other rendered-fragment blocks reduces
|
|
duplicated expectations and fixture-driven change amplification.
|
|
- Validation: Breaking any typed projection fails generatedtext; breaking any
|
|
report template or partial fails reporttemplate; making the two boundaries
|
|
incompatible fails the retained integration case.
|
|
- Disposition: Rejected as a separate finding because it is the cross-package
|
|
instance of the same duplicated contract ownership recorded by `AUD-039`.
|
|
Keeping two remediation entries would create overlapping edits and two
|
|
validation plans for one consolidation boundary.
|
|
- Related findings: `AUD-039`
|
|
- Remediation reference: Not applicable; consolidated into `AUD-039` and
|
|
Remediation Stage 32.
|
|
|
|
### AUD-064: Comparison test failure exits can leave workers blocked
|
|
|
|
- Stage: 21
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: test diagnostics
|
|
- Area: `internal/app.waitForProfileStarts` and its four comparison-execution
|
|
test callers
|
|
- Evidence: The helper waits up to five seconds for every profile to reach a
|
|
barrier. It calls `executor.releaseAll()` only when a debug callback reports
|
|
failure. An early aggregate result, timeout, or duplicate start calls
|
|
`t.Fatal` without releasing profiles already blocked in the executor, even
|
|
though the helper is shared by four tests. Normal paths release or cancel all
|
|
work and the race suite passes, so the defect appears only while diagnosing a
|
|
production regression or broken fixture.
|
|
- Contract at risk: Concurrency tests should clean up test-owned workers on
|
|
both success and failure so their primary diagnostic remains trustworthy.
|
|
- Impact: A regression that starts too few profiles incurs the full timeout and
|
|
leaves goroutines blocked until the package test process exits. Those leaks
|
|
can add secondary noise or interfere with later tests, obscuring the original
|
|
ordering/concurrency failure.
|
|
- Recommendation: Register idempotent `executor.releaseAll` cleanup before the
|
|
execution goroutine starts, and ensure the result goroutine is drainable or
|
|
joined on every helper exit. Keep the five-second timer only as a deadlock
|
|
diagnostic rather than weakening the concurrency requirement.
|
|
- Test implications: All four current ordering, failure isolation,
|
|
cancellation, and debug-reference assertions remain unchanged; only
|
|
failure-path cleanup is added.
|
|
- Validation: Deliberately suppressing one start produces the intended timeout
|
|
diagnosis and leaves no blocked comparison workers under the race detector
|
|
or a goroutine-leak probe.
|
|
- Related findings: `AUD-058`
|
|
- Remediation reference: Remediation Stage 25.
|
|
|
|
### AUD-065: Weather collection duplicates one source and serializes independent requests
|
|
|
|
- Stage: 22
|
|
- Status: confirmed
|
|
- Severity: medium
|
|
- Confidence: high
|
|
- Category: efficiency
|
|
- Area: `internal/adapters/weatherapi.Client.FetchBundle`, `warmup`, and the
|
|
eight `bundleBuilder.fetch*` calls
|
|
- Evidence: Every collection first performs a warmup GET against the default
|
|
`/conditions/current` endpoint, discards its body, and immediately fetches
|
|
`/conditions/current` again as one of eight source products. The remaining
|
|
seven products are independent HTTP endpoints, but `FetchBundle` waits for
|
|
all eight in a fixed serial chain. A temporary controlled diagnostic added
|
|
25 milliseconds of server latency to every healthy endpoint: five runs took
|
|
236-240 milliseconds apiece, made nine requests, and requested current
|
|
conditions twice. The diagnostic was removed after measurement.
|
|
- Contract at risk: Collection should preserve source policy, deterministic
|
|
normalized output, and prompt cancellation without making healthy latency
|
|
the sum of independent network round trips or issuing a request whose data
|
|
is always discarded.
|
|
- Impact: For approximately equal endpoint latency `L`, the current healthy
|
|
transport floor is about `9L`; one full `L` and one upstream response are
|
|
unconditionally redundant, while serialization adds roughly another `7L`
|
|
over concurrent independent acquisition. Generate, each batch, and each
|
|
comparison collect once, so every user-visible workflow pays this cost even
|
|
though later model latency is separate. Slow optional products also delay
|
|
all otherwise usable source results and their eventual policy decision.
|
|
- Recommendation: Remove the duplicate current-conditions warmup, or make the
|
|
successful current fetch satisfy readiness as well as collection. Acquire
|
|
independent products with a bounded, cancellation-aware group into
|
|
source-local results, then merge them in canonical source order so warning,
|
|
provenance, required-source, and error-precedence behavior stays explicit.
|
|
Do not share-mutate the current builder from request goroutines.
|
|
- Test implications: Add a delayed local-server case proving independent
|
|
requests overlap and current conditions are requested once. Retain focused
|
|
cases for warmup/retry policy if a distinct readiness operation remains, and
|
|
assert deterministic warning/provenance order, required-source failure,
|
|
optional-source policy, cancellation, and worker joining.
|
|
- Validation: A healthy collection performs only necessary requests and its
|
|
post-readiness duration approaches the slowest independent source rather
|
|
than their sum; existing normalized fixture output and failure policy remain
|
|
byte-for-byte or semantically unchanged as appropriate.
|
|
- Related findings: `AUD-013`, `AUD-015`
|
|
- Remediation reference: Remediation Stage 26.
|
|
|
|
### AUD-066: Comparison replacement repeatedly rereads complete bundles
|
|
|
|
- Stage: 22
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: efficiency
|
|
- Area: `internal/app.compareDetailed`,
|
|
`internal/comparison.PlanDestination`, `RecognizeBundle`, and `publish`
|
|
- Evidence: Replacing an existing bundle invokes `PlanDestination` before
|
|
prompt work, again after execution, at entry to `publish`, and once more at
|
|
the commit boundary. Every nonempty replacement plan calls
|
|
`RecognizeBundle`. One recognition reads the manifest once to decode it,
|
|
reads every declared directory entry in full (including that manifest and
|
|
the data package), then reads the data package a second time for its digest.
|
|
For `N` successful reports, one ordinary replacement therefore performs
|
|
four recognitions and `4N+16` complete file reads: even a two-profile bundle
|
|
causes 24 reads, while a ten-profile bundle causes 56. Each report is read
|
|
four times and the manifest and data package eight times.
|
|
- Contract at risk: Early destination rejection and commit-time
|
|
reauthorization are necessary, but each authorization pass should perform
|
|
only the content work needed to establish the recognized-bundle invariant.
|
|
- Impact: Small local bundles make the overhead modest, but the cost scales
|
|
linearly with profile count and full artifact size, including generated
|
|
Markdown already unbounded under `AUD-038`. Replacement on slower or remote
|
|
filesystems multiplies both I/O and pre-publication latency; the adjacent
|
|
post-execution and publisher-entry checks repeat the same scan with almost
|
|
no intervening stateful work.
|
|
- Recommendation: Preserve one early preflight and a final commit-time
|
|
reauthorization, but remove or combine adjacent ownership checks after
|
|
defining which layer owns each. Within recognition, decode the manifest and
|
|
hash the data package from their first reads, and validate report entries by
|
|
metadata plus a minimal open/readability check unless the bundle contract
|
|
adds report-content digests. Keep the final namespace/type checks needed by
|
|
the existing replacement threat model.
|
|
- Test implications: Instrument file-open/read operations or use a focused
|
|
filesystem seam to assert bounded passes for two and many reports. Retain
|
|
all current symlink, unexpected-entry, digest, concurrent replacement,
|
|
rollback, and commit-time reauthorization cases.
|
|
- Validation: Replacement performs one deliberate early scan and one
|
|
deliberate commit authorization without rereading any artifact inside a
|
|
scan; recognized and adversarial bundles retain their current outcomes.
|
|
- Related findings: `AUD-038`, `AUD-053`, `AUD-055`
|
|
- Remediation reference: Remediation Stage 27.
|
|
|
|
### AUD-067: Daypart key canonicalization has two independent owners
|
|
|
|
- Stage: 23
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: duplication
|
|
- Area: `internal/briefing.normalizedKey` and
|
|
`internal/generatedtext.normalizedDaypartKey`
|
|
- Evidence: Graph fingerprint comparison found two exact 17-line production
|
|
implementations: trim and lowercase the name, preserve Unicode letters and
|
|
digits, collapse every other run to one underscore, and trim edge
|
|
underscores. Briefing introduced its copy in `7b760a0`; generated-text added
|
|
a separately named copy later in `ddda424`. Briefing uses the key to store
|
|
daypart module values, while generated-text recomputes it from derived facts
|
|
to recover configured order. The same identity is also the collision domain
|
|
whose missing validation causes `AUD-024`.
|
|
- Contract at risk: One accepted daypart name must have one canonical identity
|
|
from configuration through module storage and render ordering.
|
|
- Impact: A future Unicode, punctuation, casing, or collision-policy change
|
|
must update two packages identically. Drift can leave a module entry present
|
|
under one key but unrecognized by ordered rendering under the other, causing
|
|
misplaced or omitted daypart output even after collision validation is
|
|
added.
|
|
- Recommendation: Give canonical daypart identity to the existing daypart
|
|
domain owner, preferably beside `forecast.DaypartDefinition`, and make
|
|
configuration collision checks, briefing keys, and generated-text ordering
|
|
call that one helper. Keep display labels separate from canonical roles as
|
|
required by `AUD-025`; do not introduce a generic string-normalization
|
|
package.
|
|
- Test implications: One focused domain table should own blank edges,
|
|
punctuation runs, casing, Unicode, and collision equivalence. Config should
|
|
test rejection consequences, while briefing/generated-text retain one
|
|
composition case each rather than repeating the normalization table.
|
|
- Validation: Every accepted name produces the same key at validation,
|
|
snapshot, prompt, and render boundaries; current default keys and rendered
|
|
ordering remain unchanged.
|
|
- Related findings: `AUD-024`, `AUD-025`
|
|
- Remediation reference: Remediation Stage 9.
|
|
|
|
### AUD-068: Distributor single and batch templates duplicate one parser
|
|
|
|
- Stage: 23
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: duplication
|
|
- Area: `internal/config.renderDistributorTemplate` and
|
|
`renderDistributorBatchTemplate`
|
|
- Evidence: The two 27-line functions are fingerprint-identical except for
|
|
their value type and variable-to-value callback. Both independently own the
|
|
brace scanner and the exact unclosed, empty, unknown, and unopened-variable
|
|
diagnostics. Batch support copied the second grammar in `32060bd`; the file
|
|
has seven historical changes, and config tests repeat malformed/unknown
|
|
template cases for the two families. Typed public wrappers and allowed
|
|
variable sets are legitimately distinct, but syntax is not.
|
|
- Contract at risk: Single-report and batch notification templates use one
|
|
interpolation grammar and must reject malformed syntax consistently.
|
|
- Impact: Any escaping, validation, diagnostic, or parser correction requires
|
|
synchronized edits and duplicated tests. Drift would make configuration
|
|
accept a template for one notification mode and reject the same syntax for
|
|
the other without a domain reason.
|
|
- Recommendation: Extract one private parser that accepts the diagnostic name
|
|
and a typed-family resolver callback returning a value and whether the
|
|
variable is allowed. Retain distinct value structs, allowed-variable sets,
|
|
nonempty-result rules, and public rendering functions; no general template
|
|
framework or new package is warranted.
|
|
- Test implications: Give the shared parser one malformed-syntax table. Keep
|
|
small single/batch tests for their variable vocabularies, value mappings,
|
|
nonempty requirements, report-path validation, and public error context.
|
|
- Validation: Every current valid template renders identically, current
|
|
malformed inputs retain their actionable field names, and a parser mutation
|
|
is detected once rather than by copied syntax suites.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 28.
|
|
|
|
### AUD-069: Preparation retains parallel metadata authorities and unused render surfaces
|
|
|
|
- Stage: 23
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: architecture
|
|
- Area: `internal/briefing.BuildMetadata`, `buildMetadataModule`,
|
|
`internal/app.preparedReport`, and
|
|
`internal/generatedtext/*RenderContext`/`*TemplateModules`
|
|
- Evidence: `BuildMetadata` and `buildMetadataModule` independently project the
|
|
same run ID, report ID, variant, prompt ID, generation time, units, timezone,
|
|
valid period, location, and source-warning facts. Preparation retains the
|
|
broader `briefing.Metadata` beside a snapshot containing `MetadataModule`,
|
|
then projects the former again into `promptinput.Metadata`. Render contexts
|
|
receive the outer metadata for display identity while also exposing the
|
|
decoded module metadata. All four contexts additionally expose complete
|
|
`Collected` and `Derived` values. Repository template search found no use of
|
|
`.Collected`, `.Derived`, or `.Modules.Metadata`; the duplicated identity
|
|
paths already permit the conflicts in `AUD-040`. The broad metadata path
|
|
predates the later module and render-context paths, rather than resulting
|
|
from one shared projection.
|
|
- Contract at risk: Prompt metadata, metadata stanza content, and displayed
|
|
report timing must derive from one prepared identity, while templates should
|
|
receive only fields they are authorized to render.
|
|
- Impact: A metadata addition or correction can require parallel builders,
|
|
application projection, snapshot fixtures, and context fixtures. More
|
|
importantly, callers can supply conflicting outer and stanza identities and
|
|
templates gain accidental access to raw deterministic facts that current
|
|
assets deliberately do not use.
|
|
- Recommendation: After resolving `AUD-040`, establish one smallest prepared
|
|
metadata value and derive prompt-package metadata, the metadata stanza, and
|
|
report display context from it through named projections. Remove unused
|
|
`Collected`/`Derived` fields and the unused template-module metadata field;
|
|
pass derived facts privately only while constructing ordered daypart views.
|
|
Keep prompt-package ownership in `promptinput` and module-value ownership in
|
|
`briefing` rather than replacing them with one cross-layer mega-struct.
|
|
- Test implications: Preserve one preparation test proving all three outputs
|
|
share identity and source-warning values, module tests for prompt-safe stanza
|
|
content, generated-text tests for report display fields and daypart order,
|
|
and template tests for unchanged Markdown. Delete fixture data that protects
|
|
only unreachable template fields.
|
|
- Validation: Current prompt YAML and rendered Markdown remain semantically or
|
|
byte equivalent as contractually required, conflicting identity inputs are
|
|
impossible or rejected, and template contexts contain no unused complete
|
|
fact bundle.
|
|
- Related findings: `AUD-029`, `AUD-040`, `AUD-063`
|
|
- Remediation reference: Remediation Stage 14.
|
|
|
|
### AUD-070: Fact requirement policy spans constants, declarations, and switches
|
|
|
|
- Stage: 23
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: architecture
|
|
- Area: `internal/module.FactRequirement`,
|
|
`internal/briefing.defaultModuleDefinitions`, `collectedFactAvailable`, and
|
|
`derivedFactAvailable`
|
|
- Evidence: Adding a module fact prerequisite requires a string constant in
|
|
`module`, use in one or more registry definitions, and a matching branch in
|
|
one of two availability switches in `briefing`. `FactRequirement` is a
|
|
string, and `NewModuleRegistry` validates neither known requirement names nor
|
|
collected-versus-derived placement; an unknown value is accepted and later
|
|
silently treated as unavailable. `modules.go` has changed 21 times as source,
|
|
derived, planning, SPC, and prompt-export modules were added, demonstrating
|
|
continuing evolution of this three-part policy. Focused default-composition
|
|
tests currently keep built-ins aligned but do not make one representation
|
|
authoritative.
|
|
- Contract at risk: Every declared prerequisite must have one known
|
|
availability predicate, and registry construction should reject rather than
|
|
defer unknown policy vocabulary.
|
|
- Impact: A new or renamed requirement can compile and validate into a registry
|
|
yet make its module always missing at runtime. The separate collected and
|
|
derived slices also allow a valid constant in the wrong category, producing
|
|
the same misleading absence without identifying the declaration error.
|
|
- Recommendation: Move requirement descriptors to `briefing`, where facts are
|
|
visible, and define each name, fact category, and availability predicate
|
|
once. Let definitions refer to those descriptors or validate identifiers
|
|
through one registry-owned map during `NewModuleRegistry`. Keep each
|
|
module's actual requirement list and missing-data behavior explicit in the
|
|
definition table.
|
|
- Test implications: One descriptor table should prove every identifier has a
|
|
predicate and correct category; constructor tests should reject unknown or
|
|
misplaced requirements. Retain behavior tests for materially distinct
|
|
missing-source policies, not one availability test per constant.
|
|
- Validation: Every default definition constructs, unknown/miscategorized
|
|
requirements fail at registry creation, and existing omit/empty/error results
|
|
remain unchanged for all reports.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 29.
|
|
|
|
### AUD-071: Generate and comparison repeat report-date policy at four sites
|
|
|
|
- Stage: 23
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: duplication
|
|
- Area: `internal/cli.parseGenerateFlags`, `parseComparisonFlags`,
|
|
`Runner.resolveGenerateAction`, and `Runner.resolveComparisonAction`
|
|
- Evidence: Both flag parsers independently decide that Daily and Today accept
|
|
`--date`. Both action resolvers then independently require it for Daily,
|
|
default it to the local current date for Today, parse it in the configured
|
|
timezone, and leave Tomorrow/Hourly without it. Generate acquired Today
|
|
policy in `4a02389`; comparison copied the action policy later in `b489c56`.
|
|
`root.go` has 36 historical changes, and
|
|
`TestResolveComparisonActionMatchesReportDatePolicies` exists specifically to
|
|
detect disagreement after the fact. Action-specific request construction and
|
|
diagnostics differ, but report/date capability does not.
|
|
- Contract at risk: The same report accepts, requires, defaults, and parses the
|
|
same date semantics regardless of whether one profile or several profiles
|
|
execute it.
|
|
- Impact: Adding a report variant or changing date semantics requires four
|
|
coordinated branches plus agreement tests. A missed site can expose a flag
|
|
that later fails, reject a valid comparison only, or choose different civil
|
|
dates for otherwise equivalent actions.
|
|
- Recommendation: Add one private CLI report-date policy descriptor and one
|
|
resolver that returns the effective local date from report kind, supplied
|
|
text, timezone, and clock. Let each parser consult `AcceptsDate` and each
|
|
action wrap missing-date diagnostics with its command name. Keep flag parsing
|
|
and request construction in CLI rather than moving them into the report
|
|
registry.
|
|
- Test implications: Give the shared policy a four-report table for acceptance,
|
|
required/default/forbidden behavior, invalid dates, and timezone-relative
|
|
Today defaults. Retain one thin generate and comparison wiring case each.
|
|
- Validation: Generate and compare preserve current accepted flags, errors, and
|
|
resolved dates for all four reports, and no duplicated report-kind switch
|
|
remains in their date path.
|
|
- Related findings: `AUD-009`
|
|
- Remediation reference: Remediation Stage 30.
|
|
|
|
### AUD-072: Dormant internal compatibility and persistence surfaces have no workflow owner
|
|
|
|
- Stage: 23
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: architecture
|
|
- Area: `internal/promptinput.{LoadYAML,Save}`,
|
|
`internal/adapters/weatherapi.SaveBundle`,
|
|
`internal/fileutil.WriteJSONAtomic`, `internal/module.Snapshot.Validate`,
|
|
`internal/forecast.{DifferenceAtLeast,CrossesAtOrAbove,CrossesBelow}`,
|
|
`internal/weatherdata.AlertRun.Raw`, and the normalized-JSON return from
|
|
`internal/generatedtext` validators
|
|
- Evidence: Production caller traces find no caller for `LoadYAML`; its
|
|
round-trip tests are the only consumer, while normal execution builds a
|
|
package and marshals YAML directly. The three exported forecast threshold
|
|
helpers are called only by `TestThresholdHelpers`. The Weather API adapter
|
|
copies every full alerts response into `AlertRun.Raw`, but no production or
|
|
test code reads that field; individual raw alert items and source hashes own
|
|
the needed payload/provenance. All four generated-text validators marshal and
|
|
return canonical normalized JSON, but the sole production handler caller
|
|
assigns that result to `_`; normalization behavior is otherwise consumed
|
|
through the typed value used for rendering. The persistence half of the same
|
|
obsolete prompt-input surface is already `AUD-001`, and hardening the unused
|
|
loader alone is `AUD-032`.
|
|
- Contract at risk: Internal package APIs and stored values should describe the
|
|
active stateless pipeline, not imply compatibility or persistence contracts
|
|
with no workflow owner.
|
|
- Impact: These surfaces retain tests, error branches, allocations, JSON shape,
|
|
and future design constraints without protecting current output. They can
|
|
invite a new caller to rely on accidental semantics such as YAML round-trip
|
|
loading, threshold helpers with no domain owner, a duplicate whole alert
|
|
envelope, or canonical JSON that the renderer never observes.
|
|
- Recommendation: Retire `LoadYAML` with `promptinput.Save`; remove the unused
|
|
threshold helpers and their representation-only test; remove `AlertRun.Raw`
|
|
and adapter assignments; and simplify the internal validator/handler result
|
|
to the typed value plus error. If a concrete tooling consumer is identified
|
|
before remediation, document and test its narrow contract instead of
|
|
retaining all surfaces speculatively. Make the same owner decision for
|
|
module snapshot decoding/version validation: retire the exported
|
|
compatibility implication if snapshots remain construction-only, or enforce
|
|
the exact supported version when a real decoding boundary is introduced.
|
|
- Test implications: Preserve Build/Validate/MarshalYAML execution tests,
|
|
forecast behavior that uses real threshold decisions, alert item parsing and
|
|
provenance hashes, typed normalization semantics, schema agreement, and all
|
|
normal generate/batch/comparison workflows. Remove only tests whose sole
|
|
subject is the retired surface; `AUD-039` still owns duplicated validator
|
|
semantics.
|
|
- Validation: Graph and text searches show no remaining caller or field
|
|
reference, prompt YAML and rendered reports remain unchanged, alert source
|
|
hashes still cover accepted payloads, and the repository suite passes.
|
|
- Related findings: `AUD-001`, `AUD-022`, `AUD-032`, `AUD-038`, `AUD-039`
|
|
- Remediation reference: Remediation Stage 31.
|
|
|
|
### AUD-073: Duplicate capitalization helpers disagree on Unicode safety
|
|
|
|
- Stage: 23
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: duplication
|
|
- Area: `internal/briefing.titleWord` and `sentenceCase`
|
|
- Evidence: `sentenceCase` trims, converts the string to runes, and uppercases
|
|
the first rune. `titleWord` instead uppercases `value[:1]` and appends
|
|
`value[1:]`, splitting any multibyte first rune. Its four production callers
|
|
place configured daypart names into derived summaries, workday concerns,
|
|
outdoor planning notes, and late-day watch prose. Configuration accepts
|
|
Unicode nonblank daypart names, so a name beginning with `é`, `ø`, or another
|
|
multibyte letter reaches this path and produces replacement/invalid text.
|
|
Existing tests use ASCII default names only.
|
|
- Contract at risk: Accepted Unicode display labels must remain valid and
|
|
readable when interpolated into deterministic briefing prose.
|
|
- Impact: A valid localized or operator-chosen daypart name can be corrupted in
|
|
several prompt-facing planning fields. Two helpers that express the same
|
|
first-character presentation policy also invite future call sites to choose
|
|
the unsafe implementation accidentally.
|
|
- Recommendation: Replace both with one private rune-safe first-letter helper,
|
|
choosing trimming explicitly at each call boundary if their whitespace
|
|
contracts differ. Keep canonical identity normalization separate under
|
|
`AUD-067`; this helper is display-only and must not assign semantic roles.
|
|
- Test implications: Add one small table for empty, ASCII, multibyte lowercase,
|
|
already-uppercase, and leading-space behavior. Retain one assembled planning
|
|
assertion proving a Unicode daypart label survives in prompt-facing prose.
|
|
- Validation: All current ASCII output is unchanged, every accepted Unicode
|
|
label remains valid UTF-8 with its first rune capitalized, and only one
|
|
capitalization implementation remains.
|
|
- Related findings: `AUD-024`, `AUD-025`, `AUD-067`
|
|
- Remediation reference: Remediation Stage 10.
|
|
|
|
### AUD-074: CLI help and internal guidance disagree with action behavior
|
|
|
|
- Stage: 24
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: documentation
|
|
- Area: `internal/cli/root.go`, `internal/cli.writeActionResult`,
|
|
`docs/cli.md`, and `docs/internal/cli.md`
|
|
- Evidence: Executable root help includes `--date` in Daily, Today, and
|
|
comparison usage but omits it from the Options list. Its `--quiet` text says
|
|
that only successful action output is suppressed, while
|
|
`writeActionResult` returns before writing every result-bearing summary or
|
|
routine batch status, including failed summaries; the focused comparison
|
|
test confirms failed quiet execution. `docs/cli.md` first describes that
|
|
actual all-summary behavior, but its flag table repeats the successful-only
|
|
wording. `docs/internal/cli.md` additionally says dispatch is limited to
|
|
generation and batch even though `Runner.Run` has a complete comparison
|
|
branch and focused comparison command tests.
|
|
- Executable authority and canonical documentation owner: CLI parsing, root
|
|
help, dispatch, and result output in `internal/cli` are executable authority.
|
|
`docs/cli.md` owns the public flag/output contract and
|
|
`docs/internal/cli.md` owns dispatch mechanics.
|
|
- Contract at risk: Help and canonical CLI documentation must give one usable
|
|
accepted-flag matrix and one truthful account of which result channels quiet
|
|
mode suppresses.
|
|
- Impact: An operator can miss the date option in the normal Options reference
|
|
or expect a failed action summary under `--quiet`; a maintainer can also
|
|
follow an internal flow description that omits an implemented top-level
|
|
action.
|
|
- Recommendation: Add `--date` with its command eligibility to executable
|
|
help, describe `--quiet` as suppressing all action summaries and routine
|
|
batch status in help and the one public flag row, and update CLI internals to
|
|
include comparison dispatch and its result shape. Keep the complete
|
|
acceptance matrix only in `docs/cli.md` and link to it from internals.
|
|
- Test implications: Extend help coverage to require the date option and the
|
|
accurate quiet wording. Retain the existing behavioral quiet tests rather
|
|
than duplicating every failed action in help tests.
|
|
- Validation: Root and comparison help agree with `docs/cli.md`; generate,
|
|
run, and comparison failed-result tests remain silent under `--quiet`; and
|
|
CLI internals name all three action families.
|
|
- Related findings: `AUD-010`, `AUD-071`
|
|
- Remediation reference: Remediation Stage 4.
|
|
|
|
### AUD-075: Internal guides describe nonexistent APIs, flows, and tests
|
|
|
|
- Stage: 24
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: documentation
|
|
- Area: `docs/internal/forecast-derivation.md`,
|
|
`docs/internal/collect.md`, `docs/internal/report-registry.md`, and
|
|
`docs/internal/prompt-input.md`
|
|
- Evidence: Forecast internals name `BuildPeriodDailySummaries` and
|
|
period-summary failure behavior, but graph and source search find only
|
|
`BuildDailySummary`; `facts.BuildDerived` intentionally builds one civil-day
|
|
summary for Daily, Today, and Tomorrow. Collection internals say generation,
|
|
batch, and "explicit bundle fetching" use `collect.Run`, while its sole
|
|
production caller is the default app collector used by the report workflows
|
|
and no explicit fetch command or application workflow exists. The report
|
|
guide says focused report tests cover run IDs, resolved output names,
|
|
composition defaults, and override validation, but `go test
|
|
./internal/report -list .` lists only five tests and does not cover Daily
|
|
run-ID uniqueness (`AUD-003`) or the claimed override surface. Prompt-input
|
|
internals likewise claim focused atomic-save tests even though
|
|
`promptinput.Save` has no caller or focused test (`AUD-001`).
|
|
- Executable authority and canonical documentation owner: The graph and
|
|
package symbols/tests in `internal/forecast`, `internal/facts`,
|
|
`internal/collect`, `internal/report`, and `internal/promptinput` are the
|
|
executable authorities. Their matching focused documents under
|
|
`docs/internal/` are the canonical documentation owners.
|
|
- Contract at risk: Focused internal guides should describe the implemented
|
|
package boundary and real verification inventory, not imply unavailable
|
|
entry points or protection that the suite does not provide.
|
|
- Impact: Maintainers can design against a nonexistent multi-day constructor
|
|
or fetch workflow and can overestimate regression protection for run
|
|
identity, overrides, or atomic prompt-package persistence.
|
|
- Recommendation: Replace the forecast description with the one-day
|
|
`BuildDailySummary`/facts flow, remove the absent explicit-fetch claim, and
|
|
narrow each verification paragraph to tests that exist. Coordinate the
|
|
prompt save/load wording with the dormant-surface decision in `AUD-072`
|
|
rather than documenting a speculative workflow.
|
|
- Test implications: Documentation correction does not itself require new
|
|
production tests. Add the Daily identity regression under `AUD-003`; remove
|
|
save/load-only claims or tests if `AUD-072` retires those APIs.
|
|
- Validation: Every named function and workflow has a production symbol and
|
|
caller, every claimed focused behavior is discoverable in the listed test
|
|
package, and internal-document links still resolve.
|
|
- Related findings: `AUD-001`, `AUD-003`, `AUD-072`
|
|
- Remediation reference: Remediation Stage 33.
|
|
|
|
### AUD-076: Completed comparison roadmaps duplicate current contracts
|
|
|
|
- Stage: 24
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: documentation
|
|
- Area: `docs/roadmap/profile-comparison.md` and
|
|
`docs/roadmap/implementation.md`
|
|
- Evidence: Both roadmaps are marked implemented or complete, yet together
|
|
retain detailed current command syntax, accepted reports and flags, profile
|
|
count/order, destination selection, bundle layout and manifest fields,
|
|
failure/publication policy, concurrency, debug behavior, test requirements,
|
|
and documentation end state. Those volatile contracts are already owned by
|
|
`docs/cli.md`, `docs/operations.md`,
|
|
`docs/integrations/comparison-bundle.md`,
|
|
`docs/integrations/promptkit.md`, and focused comparison internals. The
|
|
implementation plan itself says the two files should not be archived as a
|
|
second current-state reference but defers cleanup, while the documentation
|
|
policy requires completed temporary roadmaps to be removed or archived
|
|
according to repository practice.
|
|
- Executable authority and canonical documentation owner: Comparison parsing,
|
|
execution, model validation, and publication code plus the embedded assets
|
|
are executable authority. The CLI, operations, integration, architecture,
|
|
and focused-internal documents assigned by
|
|
`docs/policy/documentation.md` are the canonical reader-facing owners;
|
|
completed roadmaps own neither current behavior nor durable history.
|
|
- Contract at risk: One volatile fact must have one current documentation
|
|
owner, and completed roadmaps must not remain parallel reference manuals.
|
|
- Impact: Later command, schema, or publication changes can update canonical
|
|
documents while leaving two credible, highly detailed stale definitions for
|
|
maintainers and coding agents.
|
|
- Recommendation: Remove the completed implementation plan and feature roadmap
|
|
after preserving any durable rationale that is not already in architecture,
|
|
an ADR, or current contracts. If repository practice requires an archive,
|
|
reduce it to non-authoritative historical status without the duplicated
|
|
reference definitions and remove incoming task-guide links.
|
|
- Test implications: No runtime test is required. Validate all incoming local
|
|
links after retirement and keep current comparison contract tests with their
|
|
executable owners.
|
|
- Validation: Searches find one canonical definition for each comparison
|
|
command, artifact, and lifecycle contract; active roadmaps contain only
|
|
future or audit coordination work; all repository links resolve.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 34.
|
|
|
|
### AUD-077: Concurrent executor substitutability is only implicit
|
|
|
|
- Stage: 24
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: documentation
|
|
- Area: `internal/promptexec.Executor`,
|
|
`docs/internal/comparison-execution.md`, and
|
|
`docs/integrations/promptkit.md`
|
|
- Evidence: Comparison starts one goroutine per selected profile and invokes
|
|
the same `promptexec.Executor` concurrently. The production Promptkit
|
|
adapter and race-tested fake support that use, and the two documentation
|
|
owners describe concurrent profile execution. The `Executor` interface
|
|
comment, however, specifies callback and result behavior without requiring
|
|
implementations to permit concurrent `Execute` calls. A conforming-looking
|
|
replacement can therefore be unsafe under an implemented caller pattern
|
|
that is not part of its written contract.
|
|
- Executable authority and canonical documentation owner: The comparison
|
|
execution call graph is executable authority for concurrent use;
|
|
`internal/promptexec.Executor` owns the project interface contract,
|
|
`docs/internal/comparison-execution.md` owns the internal scheduling flow,
|
|
and `docs/integrations/promptkit.md` owns the consumer-visible integration
|
|
boundary.
|
|
- Contract at risk: Every implementation of a shared project-owned interface
|
|
must know the concurrency behavior required for safe substitution.
|
|
- Impact: A future executor fake or adapter can serialize mutable state without
|
|
synchronization, pass ordinary single-report tests, and race or corrupt
|
|
comparison execution.
|
|
- Recommendation: State on `Executor` that comparison may call `Execute`
|
|
concurrently and implementations must be safe for that use. Add a concise
|
|
linked statement to comparison internals; keep backend capacity policy in
|
|
the Promptkit integration owner rather than duplicating it in the interface.
|
|
- Test implications: Retain the race-enabled comparison test with one shared
|
|
executor. New adapters should run that contract or an equivalent focused
|
|
concurrent-call test; no timing-only assertion is needed.
|
|
- Validation: The interface and internal guide explicitly state the concurrent
|
|
call requirement, the Promptkit adapter remains race-clean, and single-report
|
|
executors retain unchanged behavior.
|
|
- Related findings: `AUD-058`
|
|
- Remediation reference: Remediation Stage 22.
|
|
|
|
### AUD-078: Configuration normalization and timezone bounds are underdocumented
|
|
|
|
- Stage: 24
|
|
- Status: confirmed
|
|
- Severity: low
|
|
- Confidence: high
|
|
- Category: documentation
|
|
- Area: `internal/report.IDForConfigKey`,
|
|
`internal/timeutil.parseUTCOffset`, and `docs/config.md`
|
|
- Evidence: The configuration reference says supported report keys treat
|
|
hyphens and underscores as equivalent. The executable lookup additionally
|
|
trims leading/trailing whitespace and folds case before normalization, and
|
|
focused lookup tests preserve that compatibility. The timezone reference
|
|
gives numeric-offset examples but no accepted bound; executable parsing
|
|
accepts signed hour forms or `HH:MM` through `23:59` and rejects larger hour
|
|
or minute components. These are accepted-value and validation rules owned by
|
|
configuration documentation, not incidental internal mechanics.
|
|
- Executable authority and canonical documentation owner: Report config-key
|
|
lookup in `internal/report` and timezone parsing in `internal/timeutil` are
|
|
executable authority. `docs/config.md` owns user-selectable configuration
|
|
values, normalization, and validation bounds.
|
|
- Contract at risk: Operators and compatibility reviewers need the canonical
|
|
configuration reference to describe the complete accepted spelling and
|
|
numeric range for user-controlled values.
|
|
- Impact: A configuration can rely on accepted case/whitespace normalization
|
|
without that compatibility being visible, and an offset can fail near the
|
|
parser boundary without the reference explaining the valid range. A later
|
|
cleanup could also narrow these behaviors as apparently undocumented
|
|
implementation details.
|
|
- Recommendation: Document trimming, case folding, and hyphen/underscore
|
|
equivalence for report keys, and state the signed numeric offset grammar and
|
|
current `23:59` component bound. If the unusually broad offset range is not
|
|
intended product policy, narrow code and docs together in a separately
|
|
reviewed behavior change rather than silently changing it during
|
|
documentation remediation.
|
|
- Test implications: Existing report lookup and timezone parsing tests remain
|
|
executable owners. Add only a documentation-validation assertion if the
|
|
repository later automates reference extraction; do not duplicate parser
|
|
tables in a prose test.
|
|
- Validation: Every documented report-key spelling and offset boundary is
|
|
accepted, representative out-of-range offsets are rejected, and the
|
|
configuration reference agrees with focused parser tests.
|
|
- Related findings: none
|
|
- Remediation reference: Remediation Stage 2.
|
|
|
|
## Retained Decisions
|
|
|
|
### RET-001: Keep the application package as the explicit composition owner
|
|
|
|
The package import graph, outbound traces from `GenerateDetailed`,
|
|
`RunBatchDetailed`, and `compareDetailed`, and their focused tests were
|
|
inspected. `internal/app` has deliberately broad outbound dependencies because
|
|
it sequences project-owned domain, collection, execution, publication, and
|
|
notification contracts; those dependencies do not point back into app. Splitting
|
|
that fan-out merely to reduce a graph metric would obscure workflow ownership.
|
|
Reconsider only if a second composition owner emerges or a coherent workflow
|
|
can move behind a narrower contract without duplicating ordering policy.
|
|
|
|
### RET-002: Keep external dependencies behind repository-owned contracts
|
|
|
|
Production import and data-flow inspection found Promptkit imports only in
|
|
`internal/adapters/promptkit` and Distributor imports only in
|
|
`internal/adapters/distributor`. The Promptkit adapter implements
|
|
`promptexec.Executor`; Weather API collection returns `weatherdata.Bundle`;
|
|
and Distributor results are translated before application and CLI summaries.
|
|
This explicit translation prevents dependency types and sensitive diagnostics
|
|
from becoming application contracts. Reconsider only when an upstream type is
|
|
intentionally adopted as a public repository contract with corresponding
|
|
architecture and compatibility changes.
|
|
|
|
### RET-003: Keep publication mechanisms separate by artifact contract
|
|
|
|
Filesystem-write discovery and call traces were inspected for single reports,
|
|
comparison bundles, and secure prompt debugging. `fileutil.WriteFileAtomic`
|
|
owns one-file replacement, `comparison.Publish` owns guarded transactional
|
|
directory replacement and recovery, and `promptdebug` owns secure explicit
|
|
diagnostic files. Their shared use of temporary paths and rename operations is
|
|
mechanical similarity, while their authorization, permissions, commit, and
|
|
recovery semantics differ materially. Reconsider common abstraction only if
|
|
multiple artifact kinds acquire the same complete transaction contract.
|
|
|
|
### RET-004: Keep collection as a narrow application-facing seam
|
|
|
|
The `app.Collector` contract, `collect.Run`, the Weather API adapter boundary,
|
|
and focused collection tests were inspected. Although `collect.Run` is small,
|
|
it keeps adapter creation and error context out of orchestration and gives app
|
|
tests a dependency-neutral deterministic seam. Reconsider if collection gains
|
|
no additional policy and an equally narrow project-owned adapter contract can
|
|
replace it without leaking transport construction into app.
|
|
|
|
### RET-005: Keep report definitions explicit and independently reviewable
|
|
|
|
The four registry definitions were compared across stable ID, prompt and
|
|
version, template and schema, period resolver, modules, output name,
|
|
Distributor paths, and run-ID policy. Their similar structure is contract
|
|
data, while meaningful differences include Daily's required date and dynamic
|
|
output identity, Today's current local day and Morning membership, Tomorrow's
|
|
next local day and both batch memberships, and Hourly's rolling elapsed
|
|
period. Explicit definitions make those differences visible and focused tests
|
|
assert the exact retained registry surface. Reconsider a shared declarative
|
|
builder only if the registry grows enough that it can reduce maintenance while
|
|
preserving independent review of every identity field.
|
|
|
|
### RET-006: Keep command and configuration report lookup policies separate
|
|
|
|
`IDForCommandName` accepts only the four exact CLI command names, while
|
|
`IDForConfigKey` deliberately trims space, folds case, and normalizes hyphens
|
|
for configuration keys. Tests also reject retired aliases. Combining the two
|
|
lookups because their successful mappings are similar would risk broadening
|
|
the CLI contract or narrowing configuration compatibility. Reconsider only if
|
|
the accepted-input policies intentionally converge or registry metadata can
|
|
generate both maps without changing either boundary.
|
|
|
|
### RET-007: Keep civil-day and rolling elapsed-time period policies distinct
|
|
|
|
Daily, Today, and Tomorrow use `CivilDay`, whose `AddDate` boundary correctly
|
|
produces 23-hour and 25-hour local days. Hourly instead adds six elapsed hours
|
|
to the injected clock, matching its documented rolling-horizon contract.
|
|
Batch coverage likewise walks the actual hourly instants within each civil
|
|
day, and focused app tests exercise daylight-saving dates. Unifying these
|
|
period calculations would conflate calendar and elapsed-time semantics.
|
|
Reconsider only if a report's product contract changes which kind of time it
|
|
represents.
|
|
|
|
### RET-008: Keep configured output paths declarative during loading
|
|
|
|
Configuration preserves a nonblank `output.directory` exactly as supplied and
|
|
does not clean, absolutize, inspect, create, expand, or rebase it. App and
|
|
publication owners later resolve it against the invocation working directory
|
|
and perform destination-specific preflight. This keeps configuration loading
|
|
free of working-directory and filesystem lifecycle policy while still rejecting
|
|
a meaningless whitespace-only setting. Reconsider only if a new configuration
|
|
contract requires an intrinsic path property that can be checked without
|
|
assuming an execution destination.
|
|
|
|
### RET-009: Keep inactive optional integrations free of active-field validation
|
|
|
|
Distributor templates and delivery settings are validated only when
|
|
notification is enabled, and batch templates only when both Distributor and
|
|
batch notification are enabled. Likewise, a blank Promptkit local endpoint
|
|
means the backend is unregistered and its configured default concurrency is
|
|
dormant; CLI mapping forwards the limit only with an endpoint, while the
|
|
adapter rejects an active limit without one. This lets complete defaults and
|
|
examples describe optional integrations without making unused credentials or
|
|
templates mandatory. Reconsider only if dormant configuration must become an
|
|
installation-time lint contract distinct from runtime configuration loading.
|
|
|
|
### RET-010: Keep Promptkit profile source precedence in its adapter boundary
|
|
|
|
Configuration owns mutual exclusion of `profile_file` and `profile_dir`, URL
|
|
and concurrency shape, and the selected profile ID. The Promptkit adapter owns
|
|
opening configured sources, complete-definition precedence, embedded and
|
|
built-in fallbacks, and classification of malformed profile errors before
|
|
collection. Maintained profile tests exercise that boundary offline. Moving
|
|
profile parsing into config would duplicate dependency-specific behavior and
|
|
leak Promptkit mechanics. Reconsider only if Weatherreporter adopts a
|
|
repository-owned profile format independent of Promptkit.
|
|
|
|
### RET-011: Keep command-specific parsers around one shared flag core
|
|
|
|
The three action parsers use separate `flag.FlagSet` values so unsupported
|
|
flags fail at the command that owns them: generate alone accepts `--out`, run
|
|
and compare accept `--out-dir`, compare alone accepts repeated `--profile` and
|
|
`--replace`, and only Daily/Today generate or compare accept `--date`.
|
|
`addCommonFlags` shares only config, units, timezone, and prompt-debug options.
|
|
This small explicit split keeps the acceptance matrix reviewable without a
|
|
general command framework. Reconsider if additional actions make the common
|
|
registration or post-parse checks materially larger or divergent.
|
|
|
|
### RET-012: Keep action summaries distinct and bounded by CLI ownership
|
|
|
|
Generate, batch, and comparison have different public result contracts, so
|
|
separate summary types and constructors are preferable to a generic result
|
|
envelope. Generate copies selected fields and a compact notification result;
|
|
batch preserves ordered report outcomes and independent notification status;
|
|
comparison locks its field order and converts top-level failures to bounded
|
|
safe categories. Pre-result failures emit no JSON, while result-bearing
|
|
failures emit a failed summary before returning an error, and one shared output
|
|
gate implements quiet mode. Reconsider common abstraction only if multiple
|
|
actions acquire the same complete summary and failure semantics.
|
|
|
|
### RET-013: Keep source-specific normalization explicit
|
|
|
|
The eight source methods share transport and missing-policy helpers but retain
|
|
visible source semantics: query parameters differ, hourly alone is required,
|
|
alerts alone treats explicit `null` as checked empty data, SPC empty arrays are
|
|
also checked data, and issue/update timestamps come from different payload
|
|
fields. Collapsing those methods into a generic descriptor would hide the
|
|
normalization decisions that need independent review. Reconsider a declarative
|
|
table only if it can express every source's availability, empty-value,
|
|
timestamp, and validation policy without callbacks that recreate the current
|
|
methods indirectly.
|
|
|
|
### RET-014: Keep selected forecast periods structurally intact
|
|
|
|
Hourly, narrative, and daily selection uses half-open overlap and returns the
|
|
complete normalized source periods in stable chronological order. It does not
|
|
fabricate clipped forecast periods at the requested boundary; alert overlap is
|
|
separately clipped because it explicitly represents an intersection. Focused
|
|
tests protect inclusion of an already-active hourly period at a rolling
|
|
horizon and exclusion at exact end boundaries. This preserves source identity,
|
|
values, and timestamps for downstream reasoning. Reconsider only if a consumer
|
|
requires synthetic boundary-aligned values and that interpolation or clipping
|
|
policy is made explicit rather than implied by selection.
|
|
|
|
### RET-015: Keep fact projection shallow under immutable workflow ownership
|
|
|
|
`BuildCollected` and `CollectedFacts.Bundle` allocate independent top-level
|
|
provenance and warning slices but retain pointers to normalized weather
|
|
products and nested values. Derivation treats those products as immutable, and
|
|
prepared-report construction later deep-copies complete inputs before handing
|
|
them to consumers. Deep-copying the full weather bundle at every fact boundary
|
|
would add repeated allocation without protecting a currently mutable seam.
|
|
Reconsider if facts begin mutating normalized products, an external consumer
|
|
can retain and mutate them concurrently, or the fact API promises complete
|
|
snapshot isolation rather than immutable workflow ownership.
|
|
|
|
### RET-016: Keep module envelopes separate from briefing policy
|
|
|
|
`internal/module` owns stable IDs, option and fact vocabulary, outputs, and
|
|
ordered snapshots without importing report definitions. `internal/briefing`
|
|
owns the registry that connects those neutral contracts to report support,
|
|
fact availability, missing-data policy, builders, and prompt exporters.
|
|
Configuration validates through that registry and application composition
|
|
builds through the same one, so policy is not duplicated at the neutral
|
|
envelope layer. Reconsider only if another independent builder implementation
|
|
needs the same policy metadata or the neutral package acquires report-specific
|
|
behavior.
|
|
|
|
### RET-017: Keep rich module values and curated prompt exports distinct
|
|
|
|
Module snapshots retain rich deterministic values for repository-owned render
|
|
contexts, while `DataPackageValue` selects a separately constructed prompt
|
|
value when the registry declares an exporter. Current conditions and hourly
|
|
forecast use this to omit lowercase, display-label, and precipitation-mention
|
|
helpers without weakening template facts; pass-through remains explicit for
|
|
modules whose complete structured value is intended for the prompt. Focused
|
|
tests compare both representations and prove prompt values are excluded from
|
|
snapshot JSON. Reconsider if the rich and prompt schemas converge completely
|
|
or a generated projection can preserve equally explicit field-level review.
|
|
|
|
### RET-018: Keep checked-empty alert modules distinct from missing sources
|
|
|
|
The alert digest uses normalized source provenance and alert presence to emit
|
|
separate `checked`, `missing`, active-count, and relevant-count facts. Its
|
|
registry behavior permits an explicit empty module, so a successful check with
|
|
no alerts does not disappear like an unavailable optional source. This
|
|
distinction is useful prompt and template information and has a focused
|
|
behavioral test. Reconsider only if the normalized source availability model
|
|
changes or all downstream consumers gain an equivalent status channel.
|
|
|
|
### RET-019: Keep Today planning richer than Daily and Tomorrow planning
|
|
|
|
Daily and Tomorrow intentionally share one morning, commute/workday, and
|
|
overnight planning shape, while Today adds outdoor-window guidance and a
|
|
late-day change watch. The distinction follows the report horizon: Today can
|
|
act on the remainder of the current day, whereas Daily and Tomorrow retain a
|
|
compact next-day readiness structure. Builders reuse the common planning
|
|
helper and focused tests protect both shapes. This is meaningful product
|
|
variation rather than drift. Reconsider only if report schemas intentionally
|
|
converge or a new horizon requires the same additional planning sections.
|
|
|
|
### RET-020: Keep SPC outlook records separate from selected briefing digests
|
|
|
|
The SPC outlook module preserves all selected overlapping outlook records and
|
|
adds background definitions only for known type/label pairs. Its risk digest
|
|
then applies a location-specific categorical severity threshold, while the
|
|
discussion module separately selects discussion days from qualifying
|
|
categorical outlooks. Focused tests cover type, rank, location, day, overlap,
|
|
unknown definitions, and checked-empty behavior. Keeping the complete selected
|
|
record set distinct from narrower editorial digests preserves evidence without
|
|
forcing every consumer through one filter. Reconsider only if SPC product
|
|
semantics or the report contract adopts a single shared selection policy.
|
|
|
|
### RET-021: Keep prompt category grouping explicit and serialization normalized
|
|
|
|
The prompt package owns a small fixed category vocabulary and emits metadata,
|
|
applicable risk products, derived summaries, narrative products, and raw data
|
|
in one explicit order while preserving module order inside each category.
|
|
Stanza values pass through JSON normalization before YAML, preventing Go- and
|
|
YAML-specific value shapes from becoming an accidental provider contract.
|
|
The highest scoped marshal complexity operates over those fixed categories
|
|
and service-sized module lists, so a generic reflective serializer would add
|
|
policy opacity without a realistic cost benefit. Reconsider if the module
|
|
catalog becomes large or category metadata can be generated from one reviewed
|
|
manifest without weakening unknown/misplaced-stanza rejection.
|
|
|
|
### RET-022: Keep neutral prompt execution lifecycle values dependency-owned
|
|
|
|
`promptexec` defines only repository-owned inspection and execution requests,
|
|
completed results, safe categories, validation rejections, and explicit debug
|
|
values. The interface carries context to both operations; callbacks run before
|
|
provider execution; validation rejection is a completed result, while
|
|
operational failure returns no result; errors preserve causes without exposing
|
|
their text; and mutable returned bytes/maps are copied. Promptkit production
|
|
types remain adapter-local. This is a coherent narrow seam rather than an
|
|
anemic duplicate of the dependency API. Reconsider only if a second executor
|
|
implementation demonstrates a missing neutral semantic; Promptkit-specific
|
|
cancellation and diagnostic translation remain Stage 11.
|
|
|
|
### RET-023: Keep Promptkit source resolution and prepared handles in the adapter
|
|
|
|
The adapter supplies embedded prompt/schema/profile filesystems and configured
|
|
profile sources directly to one Promptkit engine, then maps exact inspections
|
|
and one-use prepared executions into repository-owned values. It does not
|
|
reparse Promptkit YAML, merge profile fields, duplicate backend capacity, or
|
|
expose dependency handles. Focused tests cover complete-source precedence,
|
|
malformed matching definitions, built-in fallback, exact inline input,
|
|
callback-before-provider ordering, credential checks, cancellation, and all
|
|
stable error categories. Promptkit v0.5.0 explicitly documents its engine as
|
|
concurrency-safe and has its own concurrent prepare/run contract test, while
|
|
Weatherreporter's race-enabled adapter/app suites pass. Reconsider only if the
|
|
dependency contract changes or Weatherreporter adopts an independent profile
|
|
format requiring repository-owned resolution.
|
|
|
|
### RET-024: Keep explicit prompt-debug storage separate from normal publication
|
|
|
|
The debug writer is disabled without an absolute operator root and otherwise
|
|
owns versioned per-report/run JSON records, restrictive directory/file modes,
|
|
validated path segments, atomic same-file replacement, and distinct comparison
|
|
run directories. Application callbacks make a requested preparation-write
|
|
failure stop before provider generation, and execution-write failure prevents
|
|
report publication. These authorization, sensitivity, and failure semantics
|
|
differ materially from ordinary Markdown and comparison publication, so a
|
|
generic filesystem writer would obscure the contract. Retain the separate
|
|
owner while remediating `AUD-034` and `AUD-035`; reconsider shared mechanics
|
|
only after a common primitive can preserve every artifact's security and
|
|
transaction boundary.
|
|
|
|
### RET-025: Keep report-specific generated values over shared day-style mechanics
|
|
|
|
Daily, Today, and Tomorrow currently share the same three fields and one
|
|
private validation path, but they remain distinct exported value types and
|
|
catalog handlers. This lets context builders reject a value validated for a
|
|
different report and leaves each report free to evolve without converting a
|
|
temporary structural similarity into a public interchangeability promise.
|
|
The private `dayStyleFields` projection, trimming, nonblank filtering, and
|
|
normalization remove mechanical duplication without erasing identity. Retain
|
|
that split while fixing catalog identity in `AUD-036` and consolidating tests
|
|
under `AUD-039`; reconsider only if the reports intentionally adopt one shared
|
|
generated-text contract and migration policy.
|
|
|
|
### RET-026: Keep semantic typed validation after schema validation
|
|
|
|
Promptkit's embedded JSON Schema validates provider shape, while
|
|
`internal/generatedtext` produces repository-owned typed values, trims prose,
|
|
removes blank day-style paragraphs, requires usable summaries/discussions, and
|
|
normalizes the result. Those are distinct boundary and domain responsibilities:
|
|
schema success alone cannot construct the typed render input, and decoding
|
|
alone should not make execution correctness depend on a particular executor.
|
|
Retain both layers and make their overlapping vocabulary and limits agree when
|
|
remediating `AUD-037` and `AUD-038`; do not replace semantic validation with a
|
|
second general-purpose schema engine.
|
|
|
|
### RET-027: Keep report-specific contexts over one generic template map
|
|
|
|
Hourly, Daily, Today, and Tomorrow contexts expose concrete report metadata,
|
|
generated-text types, and module surfaces. A private day-style projection
|
|
shares metadata and common module extraction, while separate builders retain
|
|
Daily outdoor/planning values, Today current-conditions and planning behavior,
|
|
Tomorrow planning, Hourly's rolling-period labels, and distinct ordered
|
|
daypart row types. Missing optional stanzas remain typed nil pointers, and
|
|
module values cross the context boundary through a JSON-normalized typed copy.
|
|
This is useful compile-time review friction: replacing it with a generic map
|
|
would make misspelled fields, cross-report leakage, and incidental template
|
|
compatibility harder to detect. Retain the typed split while adding the
|
|
identity checks in `AUD-040`.
|
|
|
|
### RET-028: Keep templates as embedded assets with semantic output assertions
|
|
|
|
Each report's Markdown remains a separately reviewable embedded asset, and the
|
|
four shared partials centralize only identical alert, precipitation, or
|
|
daypart presentation policy. `missingkey=error`, template/partial-attributed
|
|
parse failures, and template-attributed execution failures make drift
|
|
actionable. Focused tests assert meaningful sections, conditions, exact
|
|
weather phrases where semantics matter, and relative ordering without a large
|
|
golden snapshot of every newline. Generated-text integration tests then render
|
|
real typed contexts across the package boundary. This combination protects
|
|
report variation and Markdown meaning with less incidental whitespace
|
|
coupling than full-output goldens; retain it while adding narrow regression
|
|
cases for `AUD-041` and `AUD-042`.
|
|
|
|
### RET-029: Keep one immutable prepared report as the execution boundary
|
|
|
|
`preparedReport` is the right shared boundary between collected weather and
|
|
profile execution. Preparation constructs facts, the ordered module snapshot,
|
|
briefing metadata, curated YAML, and the generated-text handler once. It deep-
|
|
copies mutable facts, snapshots, metadata, warnings, and YAML bytes; every
|
|
executor receives another byte copy, and every renderer receives newly cloned
|
|
typed inputs. Focused mutation tests protect caller and consumer isolation,
|
|
while concurrent comparison tests prove that one preparation can feed ordered
|
|
profile executions with identical bytes. Retain this explicit value boundary
|
|
rather than rebuilding report facts per profile or sharing mutable render
|
|
structures; complete catalog validation earlier as required by `AUD-043`.
|
|
|
|
### RET-030: Keep side-effect-free prompt and profile inspection before collection
|
|
|
|
Application preflight asks the neutral executor for one exact prompt contract
|
|
and the effective safe profile identity before collecting weather. Batch
|
|
inspection caches a shared effective profile, comparison preserves requested
|
|
profile order and a successfully inspected prefix, credential checks expose no
|
|
secret value, and inspection errors are classified without copying dependency
|
|
diagnostics into normal text. This is the appropriate owner and stage for
|
|
remote-work prerequisites. Retain the inspection seam while expanding it to
|
|
cover catalog compatibility and complete, later-reconciled provenance under
|
|
`AUD-043` and `AUD-044`.
|
|
|
|
### RET-031: Keep same-directory temporary replacement for one report file
|
|
|
|
The narrow `fileutil.WriteFileAtomic` mechanism gives ordinary report
|
|
publication the right visibility boundary: it creates a mode-`0600` temporary
|
|
file beside the selected destination, writes and closes complete Markdown, and
|
|
uses one same-directory rename as the linearization point. Failures before
|
|
rename leave an existing regular report unchanged, and deferred removal cleans
|
|
the temporary name after a rename error. This is simpler and safer than
|
|
truncating the destination in place or building a multi-file transaction for a
|
|
single artifact. Retain the mechanism while using a bounded temporary name,
|
|
enforcing valid target types, and adding the final context gate required by
|
|
`AUD-045` through `AUD-047`; do not describe atomic visibility as crash
|
|
durability without explicit synchronization.
|
|
|
|
### RET-032: Keep publication truth separate from notification success
|
|
|
|
`publishPromptReport` records `ReportResult.OutputPath` only after the atomic
|
|
rename succeeds, then constructs and sends the optional notification from that
|
|
exact operator-owned Markdown file. Every earlier failure returns an empty
|
|
published path; any request-construction or delivery failure afterward retains
|
|
the committed path and notification detail in the active result. The CLI
|
|
summary copies those fields and marks the action failed when the returned error
|
|
is non-nil, so it can distinguish an unpublished generation failure from a
|
|
published report whose delivery failed. Retain this ordering and result model;
|
|
batch aggregation policy remains Stage 16.
|
|
|
|
### RET-033: Keep collection before data-dependent planning and execution
|
|
|
|
Batch prompt/profile candidates are inspected before collection, weather is
|
|
collected exactly once, and only then does the application extend the fixed
|
|
Morning or Evening membership with Daily dates having complete local civil-day
|
|
hourly coverage. The resulting list is deterministic—Today then Tomorrow for
|
|
Morning, Tomorrow for Evening, followed by ascending Daily dates—and every
|
|
resolved destination is validated and stored before the first report executes.
|
|
This two-phase shape correctly reconciles early external-execution preflight
|
|
with a plan that cannot be known until forecast coverage exists. Retain the
|
|
single immutable collection and complete destination gate rather than
|
|
re-collecting per item or discovering later destinations during publication.
|
|
|
|
### RET-034: Keep sequential batch publication with independent item results
|
|
|
|
Each planned report runs through the shared prepared-generation and atomic
|
|
publication core in plan order with per-report notification suppressed. An
|
|
independent failure records that item's safe result and does not discard an
|
|
earlier committed file or prevent a later independent item from succeeding;
|
|
report counters are derived solely from appended report outcomes. This gives
|
|
operators useful partial output without implying a multi-file transaction.
|
|
Retain this behavior while distinguishing shared cancellation from an
|
|
independent failure as required by `AUD-051` and while carrying forward the
|
|
single-file publication corrections in `AUD-045` through `AUD-047`.
|
|
|
|
### RET-035: Keep batch delivery as one separate all-success notification
|
|
|
|
After the report loop, the application skips notification if any report failed.
|
|
When all reports succeeded, it renders identity and bundle paths once, rejects
|
|
duplicate bundle destinations, maps only the exact committed Markdown paths,
|
|
and invokes one batch notifier after every source exists. Notification failure
|
|
is represented separately from report counters, so all files and report
|
|
successes remain truthful while the overall action and exit status fail.
|
|
Retain this one-bundle coordination boundary and the adapter's delegation of
|
|
retry/idempotency mechanics to the pinned Distributor client; address endpoint,
|
|
response, and production-boundary test gaps without moving application gating
|
|
policy into the adapter.
|
|
|
|
### RET-036: Keep comparison bundles flat, versioned, and self-validating
|
|
|
|
One current schema indexes the exact shared data-package bytes, ordered profile
|
|
outcomes, bounded safe failures, and only the Markdown files produced by
|
|
successful profiles. Canonical JSON encoding is deterministic, the data package
|
|
has an explicit SHA-256 digest, artifact paths are flat basenames, and logical
|
|
bundle validation reconciles manifest outcomes with in-memory report payloads
|
|
before filesystem work. This is a compact durable consumer contract with no
|
|
application workspace semantics. Retain it while making strict decoding and
|
|
canonical report-name enforcement complete under `AUD-053` and `AUD-054`.
|
|
|
|
### RET-037: Keep destination planning read-only and reauthorize at commit time
|
|
|
|
`PlanDestination` cleans exact absolute paths without mutation, rejects the
|
|
filesystem root, working directory, final symlinks, non-directories, and
|
|
unrecognized nonempty content, and requires explicit replacement authority for
|
|
a current bundle. `Publish` does not trust the caller's earlier snapshot: it
|
|
replans after staging, then reauthorizes the exact entry again after moving it
|
|
to a private backup. This layered state classification protects ordinary
|
|
operator directories from stale preflight decisions. Retain both gates and the
|
|
moved-entry check; Stage 25 remains responsible for adversarial component swaps
|
|
inside the remaining syscall intervals.
|
|
|
|
### RET-038: Keep sibling staging and explicit post-commit cleanup state
|
|
|
|
Publication writes restrictive regular files into a private sibling directory,
|
|
places the manifest last, and makes the staging-to-target rename the point at
|
|
which new artifact paths become authoritative. Replacement first moves the
|
|
recognized old target aside, restores it if installing staging fails, and does
|
|
not roll back a successfully installed new bundle merely because old-backup
|
|
cleanup fails. `PublicationResult.Committed` and a typed cleanup error let the
|
|
application distinguish pre-commit failure from a valid committed bundle with
|
|
remaining cleanup work. Retain this transaction shape while adding the final
|
|
cancellation gate in `AUD-055` and truthful partial-cleanup state in `AUD-056`.
|
|
|
|
### RET-039: Keep ordered prepare-once comparison execution
|
|
|
|
Comparison validates the explicit profile list, inspects the prompt and each
|
|
profile sequentially in caller order, collects weather once, and prepares one
|
|
immutable logical report before starting model work. Each profile execution
|
|
receives a private copy of the same serialized YAML and writes only its own
|
|
preallocated result position. The application constructs and shares one
|
|
executor, leaves provider-specific capacity enforcement to its backend, and
|
|
waits for every started goroutine before returning. Retain this boundary: it
|
|
keeps compared inputs identical and makes completion timing irrelevant to
|
|
durable and CLI order while avoiding duplicate collection and derivation.
|
|
|
|
### RET-040: Keep comparison failures profile-local until one bundle decision
|
|
|
|
An individual generation, validation, render, or debug failure produces a
|
|
bounded failure entry without stopping peers. After every profile completes,
|
|
the application builds and validates one logical bundle containing the shared
|
|
data package and exactly the successful Markdown reports, publishes it once,
|
|
and then returns an aggregate action failure when any profile failed. Shared
|
|
cancellation instead prevents publication, and comparison has no notifier
|
|
path. Retain this single-publication and never-notify policy while correcting
|
|
the mixed cancellation accounting in `AUD-058`.
|
|
|
|
### RET-041: Keep comparison CLI summaries as safe result projections
|
|
|
|
The CLI preserves application result order, exposes report and bundle paths
|
|
only after they are authoritative, retains committed paths when old-backup
|
|
cleanup fails, and replaces internal errors with bounded stable categories and
|
|
messages. Quiet mode suppresses output without suppressing the returned error;
|
|
normal mode emits one JSON summary for result-bearing outcomes, and any
|
|
profile, publication, cleanup, cancellation, or aggregate error yields failed
|
|
status and a nonzero command result. Retain this projection rather than
|
|
reconstructing comparison state or provider diagnostics in the CLI.
|
|
|
|
### RET-042: Keep the default suite offline at external boundaries
|
|
|
|
Weather API behavior is exercised through loopback `httptest` servers with
|
|
synthetic responses, Promptkit uses injected fake provider clients, Distributor
|
|
uses an injected upload factory/client, and application/CLI workflows use
|
|
project-owned fakes. Literal external URLs identify data or configuration but
|
|
are not contacted. Retain these realistic local boundaries: they cover request,
|
|
response, cancellation, and orchestration behavior without live Weather API,
|
|
provider, Distributor, credential, DNS, or mutable infrastructure dependence.
|
|
|
|
### RET-043: Keep real filesystem tests rooted in test-owned directories
|
|
|
|
Writable filesystem behavior uses `t.TempDir`; tracked `testdata` and maintained
|
|
examples are read-only inputs. Tests exercise atomic replacement, recognition,
|
|
permissions, and path handling through the real filesystem, with Unix mode
|
|
assertions guarded where semantics differ and unreadable cases capability-
|
|
checked. Retain this stronger boundary while making symlink capability handling
|
|
consistent under `AUD-060`; fake path literals passed to upload doubles need no
|
|
host file.
|
|
|
|
### RET-044: Keep process-global and concurrent tests explicitly controlled
|
|
|
|
Environment-mutating tests use `t.Setenv`, remain serial, and receive automatic
|
|
restoration. The only `t.Parallel` calls cover pure comparison model helpers.
|
|
Concurrent debug and comparison tests coordinate through channels and wait
|
|
groups, use long timeouts only as failure diagnostics, and join all normal-path
|
|
workers; the repository-wide race suite passes. Retain this selective
|
|
parallelism and event-driven coordination while correcting the one ambient
|
|
environment omission in `AUD-059`.
|
|
|
|
### RET-045: Keep consequential risks at the narrowest stable test owner
|
|
|
|
Pure report, time, forecast, fact, module, schema, and naming rules belong to
|
|
their domain packages; HTTP/provider/upload translation belongs to adapters;
|
|
atomic file mechanics belong to `fileutil` and comparison publication;
|
|
workflow ordering and partial outcomes belong to `app`; and flags, structured
|
|
summaries, streams, and action status belong to `cli`. This distribution makes
|
|
the defect caught by each test legible and keeps malformed-input matrices near
|
|
the code that defines them. Retain representative higher-level proofs only
|
|
where correctness emerges from collaboration, such as collect-once preparation,
|
|
publish-before-notify, comparison ordering, and result-to-exit projection.
|
|
|
|
### RET-046: Keep coverage diagnostic rather than percentage-driven
|
|
|
|
The Stage 20 coverage run reported 79.4% statement coverage, with intentional
|
|
zeros among the process entry point, delegation wrappers, error `Unwrap`
|
|
methods, optional constructors, and dormant compatibility/persistence helpers.
|
|
Those lines do not independently justify tests. Retain coverage as a way to
|
|
locate surprising weak spots, while requiring each added regression to name a
|
|
realistic consequential defect and its stable owner. The currently important
|
|
weak spots are already recorded by defect rather than by uncovered line.
|
|
|
|
### RET-047: Keep dense contract tables explicit
|
|
|
|
The largest current files concentrate configuration fields, generated/rendered
|
|
report shapes, briefing rules, HTTP products, and filesystem transaction
|
|
states. Their size alone is not a defect. Configuration defaults and retired-
|
|
field rejections are user/compatibility contracts; meteorological threshold
|
|
tables make boundary cases reviewable; comparison tables enumerate destructive
|
|
destination and recovery states. Retain these explicit cases while removing
|
|
only the named semantic duplication in `AUD-039`, `AUD-061`, and `AUD-063`.
|
|
|
|
### RET-048: Keep small interaction fakes at external and orchestration seams
|
|
|
|
Promptkit, Distributor, collection, app, and CLI fakes record calls, results,
|
|
and ordering only where those interactions are requirements: one executor,
|
|
publish before notify, exact upload mappings, profile concurrency, cancellation
|
|
propagation, and no work after preflight failure. They do not reproduce full
|
|
provider implementations or general mock choreography. Retain these local
|
|
doubles; replacing them with a shared framework would increase indirection and
|
|
maintenance cost without improving confidence.
|
|
|
|
### RET-049: Keep semantic assertions instead of complete-output snapshots
|
|
|
|
The suite has no golden or full Markdown snapshot files. Exact whole-byte
|
|
assertions are confined to canonical compatibility encodings, normalized JSON,
|
|
atomic replacement payloads, and copy-isolation sentinels. Markdown tests use
|
|
ordered semantic fragments and explicit omissions, which make intentional
|
|
template changes reviewable without memorializing incidental whitespace.
|
|
Retain that style while consolidating the cross-owner render checks in
|
|
`AUD-063`.
|
|
|
|
### RET-050: Keep service-sized derivation and registries straightforward
|
|
|
|
Graph loop propagation led through daily context construction into fact and
|
|
forecast selection, but the concrete workload is bounded by four default
|
|
dayparts and Weather API runs of a few hundred periods. The work is linear per
|
|
selection, and Stage 7 already counted only low-thousands of simple overlap
|
|
checks for an ordinary derived build. Report and generated-text registries
|
|
contain four definitions, while the module registry is a fixed small catalog;
|
|
rebuilding those values is not a credible hot path at current cardinalities.
|
|
Retain the direct loops and explicit registries. Reconsider indexing or caching
|
|
only if forecast horizons, configured dayparts, or registries become dynamic
|
|
and materially larger.
|
|
|
|
### RET-051: Keep embedded template parsing until render volume changes
|
|
|
|
`reporttemplate.Render` reads and parses one top-level template plus four
|
|
embedded partials on each call. A temporary benchmark that deliberately
|
|
reached execution failure only after parsing measured 0.283-0.291 milliseconds
|
|
and about 91 KiB across 1,898 allocations per render on the audit host. The
|
|
fixed three-report batches therefore spend under one millisecond parsing, and
|
|
ordinary comparisons select only a handful of profiles; provider execution is
|
|
orders of magnitude slower. Keep the simple per-call ownership and error
|
|
context for now. A pre-parsed immutable set becomes justified if rendering
|
|
moves into a high-volume service loop or profiles routinely number in the
|
|
hundreds. The benchmark file was removed after measurement.
|
|
|
|
### RET-052: Keep prepare-once profile execution and fixed batch sequencing
|
|
|
|
Comparison builds facts, modules, and prompt input once, then starts one
|
|
profile-local execution per explicitly selected profile; the durable order is
|
|
held in a preallocated result slice and Promptkit owns provider capacity.
|
|
Per-profile JSON cloning and rendering provide mutation isolation, while the
|
|
documented and example workload is a human-selected comparison of at least two
|
|
profiles rather than an unbounded service queue. Morning and evening batches
|
|
likewise contain a fixed small report set and intentionally preserve
|
|
independent sequential publication before one notification. Retain both
|
|
designs at current workload. Treat routine selections in the tens or hundreds,
|
|
observable clone contention, or a larger dynamic batch as triggers for a
|
|
bounded worker design rather than inferring a defect from goroutine or graph
|
|
counts alone.
|
|
|
|
### RET-053: Keep report-specific generated types and thin builders explicit
|
|
|
|
Fingerprint analysis found exact or near-exact Daily/Tomorrow context builders,
|
|
four catalog type-dispatch adapters, three planning-module wrappers, and
|
|
parallel report definitions. Their similarity follows four intentionally
|
|
distinct report contracts: each public generated type, schema/template pair,
|
|
planning stanza, report ID, and typed context must remain independently
|
|
selectable and type-checked. Existing private day-style helpers already share
|
|
the mechanics that have one policy owner. Retain the thin variants rather than
|
|
introducing reflection, a generic report mega-type, or callbacks that obscure
|
|
which planning fields each report supports. `AUD-039` and `AUD-063` own test
|
|
duplication without erasing these production distinctions.
|
|
|
|
### RET-054: Keep rich-to-prompt projections as explicit allowlists
|
|
|
|
Current conditions, hourly periods, and derived dayparts copy many fields from
|
|
rich module values into narrower prompt structs. That field-by-field code is
|
|
deliberate curation: omission of template-only helpers, source detail, and
|
|
unneeded values is part of the prompt security and authority boundary under
|
|
`RET-017`. Code generation, reflection, embedding, or serialization through
|
|
the rich struct would make accidental prompt expansion harder to review.
|
|
Retain named projection functions and focused omission tests. Extract only
|
|
small semantic subprojections when multiple prompt types genuinely share the
|
|
same allowlist and change history.
|
|
|
|
### RET-055: Keep tiny safety copies and path checks package-local
|
|
|
|
Exact fingerprints found three ten-line string-map copy helpers in Promptkit,
|
|
prompt-debug, and neutral execution packages, plus similar absolute-path
|
|
checks in app and comparison. The copies sit at separate mutation/security
|
|
boundaries and their names state which owner is taking a copy. The path checks
|
|
attach different diagnostics and feed substantially different publication
|
|
policies. A generic utility package would save little code while coupling
|
|
unrelated contracts. Retain these local helpers; share only if a common type or
|
|
policy owner emerges, not because the mechanics happen to match.
|
|
|
|
### RET-056: Keep configuration validation against the briefing registry
|
|
|
|
`internal/config` imports `internal/briefing` to normalize typed module options
|
|
and validate report compositions against the same definitions used at runtime.
|
|
This edge prevents configuration from copying module IDs, option shapes,
|
|
supported-report rules, and stanza policy. Moving a second descriptor table
|
|
into config would create the duplication Stage 23 is meant to remove, while a
|
|
new abstraction package would split the current owner without another
|
|
consumer. Retain the dependency; narrow the requirement vocabulary within
|
|
briefing under `AUD-070` without moving runtime facts into config.
|
|
|
|
### RET-057: Keep workflow orchestration visibly distinct
|
|
|
|
Generate, fixed batch, and comparison share phases such as resolution,
|
|
inspection, collection, preparation, execution, and publication, but the
|
|
ordering and failure contracts differ materially. Batch validates all outputs,
|
|
continues independent report failures, publishes sequentially, and gates one
|
|
notification. Comparison validates explicit profiles, prepares once, executes
|
|
concurrently, and publishes one partial-capable transaction without
|
|
notification. Extracting a common workflow framework would hide those state
|
|
machines. Retain top-level guarded pipelines and consider only phase-local
|
|
helpers after `AUD-051`, `AUD-058`, `AUD-065`, and `AUD-066` settle their
|
|
behavior.
|
|
|
|
### RET-058: Keep the resolved-value clone explicit for current option shapes
|
|
|
|
`cloneResolved` copies report slices and the only reference-bearing module
|
|
option, AFD sections. Every other accepted option is presently value-only, and
|
|
mutation-isolation tests cover the complete prepared boundary. Reflection or
|
|
JSON cloning would add runtime failure modes and obscure ownership to solve a
|
|
hypothetical future shape. Retain the explicit switch now. If a second
|
|
reference-bearing option is introduced, give module options an owner-defined
|
|
clone contract and make the registry/construction test prove every registered
|
|
shape participates.
|
|
|
|
### RET-059: Keep concise linked summaries at audience boundaries
|
|
|
|
The README's one-command orientation, development task router, operational
|
|
workflow snippets, configuration cross-references, template identity table,
|
|
and integration links repeat only enough stable context for their audience and
|
|
then point to the assigned canonical owner. Some identifiers necessarily
|
|
appear in both executable assets and their owning explanation, while module,
|
|
prompt, template, and schema names also appear where maintainers must keep a
|
|
cross-contract pairing aligned. This is useful navigation, not parallel
|
|
reference maintenance. Retain these short summaries and links; do not copy
|
|
complete flag, field, schema, status, or path tables into non-owning documents.
|
|
|
|
### RET-060: Keep intended safety contracts authoritative while fixing code
|
|
|
|
Several earlier findings are executable violations of correctly stated
|
|
architecture, operations, integration, or artifact contracts: Weather API and
|
|
Distributor endpoint/diagnostic safety, prompt/schema/catalog alignment,
|
|
ordinary output preservation and cancellation, comparison recognition and
|
|
transaction outcomes, and completed-profile failure identity. Rewriting those
|
|
documents to advertise the unsafe or internally inconsistent behavior would
|
|
turn defects into contracts. Retain the intended canonical statement, repair
|
|
the executable authority under `AUD-011`, `AUD-012`, `AUD-029` through
|
|
`AUD-031`, `AUD-036` through `AUD-040`, `AUD-043` through `AUD-050`, and
|
|
`AUD-053` through `AUD-058`, and review the owner in the same remediation for
|
|
any clarified limits or compatibility effect.
|
|
|
|
### RET-061: Keep guarantee language bounded to implemented scopes
|
|
|
|
Current documents use atomic and transactional language for completed-file or
|
|
directory namespace visibility and rollback; none promises file or parent
|
|
directory synchronization against power loss. Promptkit documentation assigns
|
|
backend capacity to Promptkit and describes concurrent human-selected profile
|
|
sets without promising arbitrary fan-out performance. The release procedure
|
|
names the six produced OS/architecture binaries without converting that build
|
|
matrix into a broader host-capability promise for every filesystem test.
|
|
Retain these bounded statements. Introduce crash durability, a profile-count
|
|
service level, or a supported-platform policy only with executable evidence
|
|
and an explicitly assigned owner.
|
|
|
|
## Open Questions
|
|
|
|
No Stage 1 open questions or unexplained baseline failures remain.
|
|
|
|
Stage 2 routed these investigation leads without treating graph metrics or an
|
|
unusual edge as findings:
|
|
|
|
- `internal/config` imports `internal/briefing` to use module definitions while
|
|
validating report-module options. Stages 4, 8, and 23 should determine
|
|
whether this remains the smallest single-owner validation path or creates
|
|
avoidable registry coupling.
|
|
- The graph identifies `internal/cli.Run`, `internal/app.RunBatchDetailed`, and
|
|
`internal/app.compareDetailed` as relatively complex orchestration paths.
|
|
Their ownership and top-level ordering are coherent; Stages 5, 16, 18, 22,
|
|
and 23 should assess their local behavior and maintainability rather than
|
|
inferring a finding from metrics.
|
|
|
|
Stage 3 routed these investigation leads to their assigned later stages:
|
|
|
|
- `LoadLocation` accepts numeric offsets through `23:59`, while the config
|
|
reference does not state an offset range. Stage 4 should assess the effective
|
|
configuration contract and validation ownership before judging that bound.
|
|
- The report-registry document says focused tests cover run IDs and output
|
|
names, but Stage 3 found no behavioral run-ID assertion and only indirect
|
|
dynamic output-name coverage. `AUD-003` captures the consequential run-ID
|
|
gap; Stage 24 should reconcile the broader documentation claim with the
|
|
eventual executable contract.
|
|
- Prompt, template, schema, and module IDs are internally consistent at the
|
|
report registry. Stages 8 and 10-13 remain responsible for validating the
|
|
referenced registries, embedded assets, generated-text schemas, and render
|
|
compatibility rather than duplicating those audits here.
|
|
|
|
Stage 4 routed these investigation leads to their assigned later stages:
|
|
|
|
- Configuration accepts any absolute URL scheme with a host, matching the
|
|
documented term “absolute URL.” Stages 6, 11, and 16 should assess transport
|
|
scheme support at the Weather API, Promptkit, and Distributor boundaries
|
|
before narrowing the configuration contract.
|
|
- `weather_api.base_url` may be empty in a valid configuration because only
|
|
collection workflows require it. Stages 5-6 should verify action-specific
|
|
failure timing and diagnostics rather than making every non-collection
|
|
configuration require an endpoint.
|
|
- Report config-key normalization accepts trimmed and case-folded spellings in
|
|
addition to the documented hyphen/underscore equivalence. Stage 24 should
|
|
decide whether the canonical reference should describe that compatibility
|
|
or the executable contract should be narrowed.
|
|
- Distributor delivery redaction, runtime template consumption, and notification
|
|
idempotency remain assigned to Stage 16. Stage 4 accounts only for active
|
|
configuration and safe identity/path rendering before execution.
|
|
|
|
Stage 5 routed these investigation leads to their assigned later stages:
|
|
|
|
- Generate and batch summaries intentionally carry project-owned app error and
|
|
warning fields, while comparison applies an additional bounded safe-error
|
|
translation. Stages 6, 11, and 16 should verify that Weather API, Promptkit,
|
|
and Distributor producers never place sensitive endpoint, credential, or
|
|
provider detail in the non-comparison fields before Stage 24 judges the full
|
|
documented output claim.
|
|
- Generate and comparison repeat the Daily/Today flag-acceptance and date
|
|
defaulting policy. They currently agree, and the report registry remains the
|
|
period owner. Stage 23 should assess whether a shared CLI policy descriptor
|
|
would reduce drift without moving parsing into the domain registry.
|
|
- The static help text includes `--date` in usage but omits it from the Options
|
|
list and describes `--quiet` as suppressing successful output even though it
|
|
suppresses failed action summaries too. `docs/internal/cli.md` also still
|
|
says the CLI dispatches only generation and batch actions despite implemented
|
|
comparison dispatch. Stage 24 should reconcile these documentation/help
|
|
statements without expanding Stage 5 into the repository-wide documentation
|
|
audit.
|
|
- Batch status writes deliberately ignore stderr write failures while JSON
|
|
encoding errors are returned. Stage 25 should exercise broken-pipe and output
|
|
writer failures dynamically before deciding whether routine-status I/O must
|
|
affect the process exit.
|
|
|
|
Stage 6 routed these investigation leads to their assigned later stages:
|
|
|
|
- `facts.BuildCollected` and `facts.Bundle` copy source and warning slices but
|
|
not nested query maps or warning slices. Prepared-report construction later
|
|
deep-copies its complete inputs. Stage 7 should judge whether the earlier
|
|
derivation boundary promises mutation isolation or whether immutable
|
|
workflow ownership makes the shallow copy intentional.
|
|
- `AlertRun.Raw` preserves the complete alert payload in addition to the raw
|
|
alert items, but graph-augmented use search found no production consumer of
|
|
the full duplicate. Stage 23 should assess removal with other unused and
|
|
duplicate internal surfaces; alert-item schema and parsing remain assigned
|
|
to Stages 7-9.
|
|
- Source provenance retains the adapter-controlled query parameters in the
|
|
normalized bundle, while prompt-facing metadata deliberately omits the query
|
|
map. Stage 10 should confirm that no prompt input or debug artifact widens
|
|
that boundary, and Stage 24 should judge whether the internal documentation
|
|
needs an explicit non-secret-query invariant.
|
|
- The adapter accepts URL user information, base query parameters, and
|
|
fragments because the current absolute-URL check does not constrain them;
|
|
endpoint construction drops the fragment and carries base query values into
|
|
requests/provenance. The current configuration and examples use none of
|
|
these. Stage 24 should clarify the intended base-URL shape after remediation
|
|
of `AUD-011`, rather than Stage 6 inventing undocumented authentication or
|
|
query behavior.
|
|
|
|
Stage 7 routed these investigation leads to their assigned later stages:
|
|
|
|
- `DifferenceAtLeast`, `CrossesAtOrAbove`, and `CrossesBelow` are exported
|
|
forecast helpers with focused tests but no production callers in the graph.
|
|
Stage 23 should assess removing this unused comparison surface rather than
|
|
treating locally correct arithmetic as a derivation defect.
|
|
- The forecast derivation document describes
|
|
`BuildPeriodDailySummaries`, but no such symbol or equivalent multi-day
|
|
constructor exists in the implementation. Current facts intentionally build
|
|
one civil-day summary for Daily, Today, and Tomorrow. Stage 24 should
|
|
reconcile the named API and actual one-day report contract.
|
|
- Planning and derived briefing consumers assign Fahrenheit/mph meaning to
|
|
the unitless daypart values identified by `AUD-016`, and hazard wording may
|
|
amplify the missed text classification in `AUD-019`. Stage 9 should account
|
|
for those downstream effects while avoiding duplicate findings for the same
|
|
derivation causes.
|
|
|
|
Stage 8 routed these investigation leads to their assigned later stages:
|
|
|
|
- `BuildMetadata` and its broader `Metadata` type duplicate part of the
|
|
metadata module path, but the application retains only a subset for the
|
|
prompt package and uses the module stanza for briefing content. Stage 23
|
|
should assess consolidating this parallel construction without moving prompt
|
|
package ownership into briefing.
|
|
- Fact requirement constants, registry declarations, and the two availability
|
|
switches must evolve together. They currently agree and focused tests cover
|
|
default compositions, but Stage 23 should assess a definition shape that
|
|
reduces switch/table drift without hiding missing-data policy.
|
|
- Current/hourly rich and prompt structs deliberately differ under `RET-017`,
|
|
yet their field-by-field copy code is sizeable. Stage 23 should evaluate
|
|
whether small projection helpers or generated mappings reduce maintenance
|
|
while preserving reviewable prompt curation.
|
|
- AFD text, alert descriptions/instructions, weather-story content and URLs,
|
|
and narrative fields currently use pass-through prompt exports. Stage 10
|
|
should verify the complete serialized prompt boundary and intended curation;
|
|
Stage 8 accounts only for their source-to-module transformations.
|
|
|
|
Stage 9 routed these investigation leads to their assigned later stages:
|
|
|
|
- The derived and planning modules amplify the unit collapse in `AUD-016`, the
|
|
overnight-alert omission in `AUD-017`, invalid percentages in `AUD-018`, and
|
|
incomplete ice classification in `AUD-019`. Stage 9 did not duplicate those
|
|
upstream causes; remediation and later end-to-end tests should protect the
|
|
downstream wording and thresholds as well.
|
|
- `titleWord` uppercases the first byte rather than the first rune, and several
|
|
daypart-oriented helpers repeat display and semantic-name handling. The
|
|
consequential identity behavior is captured by `AUD-024` and `AUD-025`;
|
|
Stage 23 should assess Unicode-safe shared formatting and removal of repeated
|
|
day-style policy without turning display labels into domain identities.
|
|
- Prompt serialization and rendering of the newly audited fields remain with
|
|
Stages 10 and 13. Stage 24 should decide where the externally maintained SPC
|
|
definition source and review date belong in canonical documentation after
|
|
`AUD-028` is remediated.
|
|
|
|
Stage 10 routed these investigation leads to their assigned later stages:
|
|
|
|
- `promptinput.Save` and `LoadYAML` have no production callers, while the
|
|
prompt-input document describes ordinary atomic saves and focused save
|
|
coverage that do not exist. `AUD-001` and `AUD-032` capture the code risks;
|
|
Stages 23-24 should decide whether to remove the unused API and reconcile the
|
|
canonical document rather than expanding a dormant persistence workflow.
|
|
- JSON unmarshalling reconstructs stanza order from map iteration and therefore
|
|
does not promise deterministic order. There is no exported JSON loader or
|
|
production consumer, and YAML is the execution format. Stage 23 should
|
|
assess retiring or explicitly limiting this incidental surface instead of
|
|
Stage 10 inventing a second ordering contract.
|
|
- The neutral interface passes context and makes debug detail explicit, but
|
|
provider cancellation, dependency error translation, and secure prompt-debug
|
|
publication belong to Stage 11. Application proof that serialization failure
|
|
prevents provider work belongs to Stage 14 with `AUD-033` as its focused
|
|
package-level prerequisite.
|
|
|
|
Stage 11 routed these investigation leads to their assigned later stages:
|
|
|
|
- Debug writer construction creates or tightens the explicitly requested root
|
|
before prompt inspection and collection. This is authorized diagnostic state,
|
|
not an ordinary-workflow violation, but Stage 24 should make the possibility
|
|
of an empty root after later preflight failure explicit if operators need that
|
|
lifecycle detail.
|
|
- `marshalDebugParameters` and structured-schema capture discard impossible-
|
|
under-current-Promptkit JSON marshal errors. Promptkit v0.5.0 constrains those
|
|
values to JSON-compatible copies, so Stage 11 recorded no defect. Stage 23
|
|
should avoid generalizing the adapter around that assumption unless the
|
|
dependency contract changes.
|
|
- Application-level inspection ordering, exact prompt/profile consistency, and
|
|
prepared-input reuse remain Stage 14; comparison scheduling and cancellation
|
|
aggregation remain Stage 18. Stage 11 accounted for adapter context
|
|
propagation and Promptkit's concurrent-engine contract without duplicating
|
|
those orchestration findings.
|
|
|
|
Stage 12 routed these investigation leads to their assigned later stages:
|
|
|
|
- The application discards the canonical normalized JSON returned beside the
|
|
typed generated value. This is harmless in the current render path and the
|
|
package documents the return value, but Stage 23 should decide whether a
|
|
production consumer justifies retaining that API surface rather than
|
|
treating non-use alone as a defect.
|
|
- `encoding/json` also accepts duplicate object names with last-value wins.
|
|
The embedded schema validator follows parsed-JSON semantics and no canonical
|
|
contract currently promises duplicate rejection, so Stage 12 did not invent
|
|
a uniqueness rule. Stage 25 may include duplicates in its broader
|
|
adversarial corpus when validating the remediation for `AUD-037`.
|
|
- Context construction, template fields, partial execution, and Markdown
|
|
semantics remain Stage 13. Application proof that catalog and generated-text
|
|
failures occur before publication remains with Stages 14-18; Stage 12 traced
|
|
only enough of those callers to establish the raw-output validation boundary.
|
|
|
|
Stage 13 routed these investigation leads to their assigned later stages:
|
|
|
|
- Render contexts expose complete `.Collected` and `.Derived` values plus a
|
|
decoded `.Modules.Metadata` even though current templates use neither and
|
|
`.Report` already owns display metadata. This is not a current output defect,
|
|
but Stage 23 should assess narrowing the maintainer surface after `AUD-040`
|
|
establishes one canonical identity and timing source.
|
|
- `Render` reparses every top-level template and all four partials for every
|
|
report. The embedded set is small and measured behavior is deterministic, so
|
|
Stage 13 did not infer an efficiency defect. Stage 22 should compare a
|
|
pre-parsed immutable template set only if repository-wide profiling shows
|
|
rendering cost material.
|
|
- Current template tests use concise package-local context structs while
|
|
generated-text tests exercise the real typed contexts. That division gives
|
|
useful focused and integration coverage, but Stages 20-21 should assess the
|
|
large fixture surface and repeated report assertions across the whole test
|
|
portfolio rather than Stage 13 rewriting unrelated cases.
|
|
|
|
Stage 14 routed these investigation leads to their assigned later stages:
|
|
|
|
- `executePreparedProfile` deliberately performs no durable write and ordinary
|
|
generation checks context only after rendering, immediately before atomic
|
|
publication. Stage 15 owns the final cancellation, destination-preservation,
|
|
and partial-result truthfulness consequences; Stage 18 owns comparison-wide
|
|
cancellation aggregation after concurrently started executions.
|
|
- `cloneResolved` copies the current slice-bearing AFD option explicitly while
|
|
all other accepted module options are value-only. This is safe for the
|
|
present registry, but the manual type switch can miss a future reference-
|
|
bearing option. Stage 23 should assess an ownership-preserving clone contract
|
|
if module option shapes expand rather than introducing reflection during this
|
|
audit stage.
|
|
- Preparation clones typed values through JSON, so impossible-under-current-
|
|
shapes clone failures remain wrapped but are not independently injectable.
|
|
Stages 20-21 should decide whether existing successful mutation isolation and
|
|
the actionable wrapper are sufficient when consolidating application
|
|
fixtures; `AUD-033` separately owns the reachable stanza serialization gap.
|
|
|
|
Stage 15 routed these investigation leads to their assigned later stages:
|
|
|
|
- Existing directory symlinks are deliberately accepted for normal output
|
|
directories, but pathname components can change between preflight and rename.
|
|
`AUD-045` owns the final-component policy; Stage 25 should include a
|
|
deterministic component-swap case when validating remediation rather than
|
|
Stage 15 claiming containment the current operator-owned-path contract does
|
|
not define.
|
|
- The writer provides atomic visibility but does not call `Sync` on the file or
|
|
parent directory. No canonical document promises power-loss durability, so
|
|
Stage 15 did not invent that guarantee. Stage 24 should keep “atomic” wording
|
|
clear enough that operators do not read it as a persistence acknowledgment.
|
|
- `WriteJSONAtomic` remains outside the ordinary report path and is reached in
|
|
production only through unused persistence helpers. `AUD-001` already owns
|
|
that surface; Stage 23 should remove or justify it without mixing legacy JSON
|
|
persistence into the single-Markdown publication contract.
|
|
|
|
Stage 16 routed these investigation leads to their assigned later stages:
|
|
|
|
- A published report path can be replaced by another process between its
|
|
atomic rename and the later batch upload read, especially while subsequent
|
|
reports are generated. The current operator-owned-path contract does not
|
|
promise adversarial namespace containment. Stage 25 should test this race
|
|
alongside the component-swap cases already routed from Stage 15 rather than
|
|
changing batch source selection in isolation.
|
|
- The body-size and safe-diagnostic corrections in `AUD-049` and `AUD-050` may
|
|
be best implemented in a newer pinned Distributor client so all consumers
|
|
share one HTTP boundary. Stage 26 should confirm ownership and dependency
|
|
availability before the remediation roadmap chooses an adapter-local
|
|
transport wrapper or an upstream revision.
|
|
- `RunBatchDetailed` has coherent phase ownership but combines setup,
|
|
generation aggregation, and notification in one high-cognitive-complexity
|
|
function. Stages 22-23 should consider extracting phase helpers only after
|
|
the cancellation/accounting contract in `AUD-051` is settled; Stage 16 did
|
|
not treat the graph metric alone as a defect.
|
|
|
|
Stage 17 routed these investigation leads to their assigned later stages:
|
|
|
|
- `RecognizeBundle` reads every report file fully even though the manifest has
|
|
no report digest and recognition uses only file type and membership for those
|
|
entries. With generated Markdown already unbounded under `AUD-038`, Stage 22
|
|
should measure and assess this avoidable allocation alongside comparison
|
|
bundle construction rather than Stage 17 creating an unrelated size policy.
|
|
- Existing non-final symlink components are permitted when they resolve to
|
|
directories, while final target and bundle-entry symlinks are rejected.
|
|
Concurrent swaps remain possible between `Lstat`, reads, renames, and backup
|
|
cleanup. Stage 25 should exercise those adversarial component and cleanup
|
|
races and decide whether descriptor-relative operations are warranted;
|
|
Stage 17 verified the ordinary static states only.
|
|
- Sibling directory renames provide atomic visibility but publication does not
|
|
synchronize files or parent directories for power-loss durability. No
|
|
canonical contract promises crash persistence, matching the Stage 15
|
|
conclusion for single reports. Stage 24 should keep “transactional” and
|
|
“atomic” wording scoped to visible namespace state rather than durable-media
|
|
acknowledgment.
|
|
|
|
Stage 18 routed these investigation leads to their assigned later stages:
|
|
|
|
- Comparison starts one goroutine per selected profile and relies on the
|
|
Promptkit backend's configured capacity rather than adding an application
|
|
semaphore. This is the documented ownership split and current profile sets
|
|
are configuration-sized; Stage 22 should assess extreme profile-count
|
|
goroutine and retained-result cost with evidence rather than Stage 18
|
|
inventing a second concurrency limit.
|
|
- The project-owned executor interface is used concurrently by comparison, but
|
|
its type comment does not state that implementations must support concurrent
|
|
calls. The production adapter and race-tested application fake do so. Stage
|
|
24 should make that substitutability requirement explicit when reconciling
|
|
executable and written contracts.
|
|
- Comparison publication inherits the post-backup cancellation and cleanup
|
|
state defects in `AUD-055` and `AUD-056`. Stage 18 verified their committed
|
|
paths and safe CLI projection without duplicating package-owned findings;
|
|
Stages 25-26 should validate the repaired end-to-end states.
|
|
|
|
Stage 19 routed these investigation leads to their assigned later stages:
|
|
|
|
- The assembled collection success test pays the production one-second warmup
|
|
delay, and the Weather API timeout case uses one controlled 50-millisecond
|
|
handler sleep. Both are bounded and deterministic, so Stage 19 did not label
|
|
them hermeticity defects. Stages 21-22 should assess their marginal runtime
|
|
and whether a narrower delay seam would improve durability or feedback cost.
|
|
- Comparison-start and concurrent-debug helpers use five-second wall-clock
|
|
deadlines only as deadlock diagnostics; normal paths coordinate and join by
|
|
channels/wait groups. A timeout caused by a product regression can still
|
|
leave already-broken workers alive until the test process exits. Stage 21
|
|
should assess failure-path cleanup when reviewing test diagnostic quality,
|
|
rather than weakening the liveness guards here.
|
|
- The repository defines no explicit supported host/CI platform matrix. Stage
|
|
19 classified capability-sensitive tests against the policy's machine-state
|
|
rule, not an inferred product portability promise. Stage 24 should reconcile
|
|
supported-platform expectations before later validation treats platform-
|
|
specific skips as product coverage evidence.
|
|
|
|
Stage 20 routed these investigation leads to their assigned later stages:
|
|
|
|
- Generated-text policy assertions already cross package boundaries under
|
|
`AUD-039`, and report/template/application fixtures repeat some report-family
|
|
shapes. Stage 21 should assess marginal protection, change amplification, and
|
|
consolidation without moving schema, rendering, or workflow ownership.
|
|
- Several application and CLI suites deliberately use complete prepared
|
|
reports to prove collaboration, while package suites own the exhaustive
|
|
negative cases. Stage 21 should verify that these representatives remain
|
|
behavior-focused and diagnose failures locally; Stage 20 found no reason to
|
|
expand them into duplicated lower-level matrices.
|
|
- Broad parser, normalization, and path input spaces have table-driven boundary
|
|
tests but no fuzz targets. No specific uncovered defect justified adding a
|
|
percentage-driven target in Stage 20; Stage 25 should use its adversarial
|
|
corpus to decide whether a durable fuzz regression is warranted.
|
|
- The one-second assembled collection warmup and failure-only goroutine cleanup
|
|
concerns remain execution/durability questions for Stages 21-22. They do not
|
|
leave a consequential production invariant without a test owner.
|
|
|
|
Stage 21 routed these investigation leads to their assigned later stages:
|
|
|
|
- The collection and Distributor wait controls in `AUD-062` are test-cost
|
|
seams, not evidence that production retry or polling intervals are globally
|
|
inefficient. Stage 22 should assess runtime waits only in its broader
|
|
operational context; Stage 23 may choose the smallest ownership-preserving
|
|
injection shape during remediation planning.
|
|
- `config_test.go` and the generated-text/template suites are physically large,
|
|
but their explicit contract tables remain valuable under `RET-047`.
|
|
Stage 23 may consider file organization or narrow fixture builders only where
|
|
it reduces navigation/change cost without hiding report-specific values.
|
|
- Historical rejection tests for removed workspace, report, Distributor, and
|
|
execution settings remain plausible regression guards for statelessness and
|
|
strict compatibility. Stage 26 should consolidate them only if a stronger
|
|
executable forbidden-surface invariant demonstrably retains those named
|
|
protections.
|
|
- The absence of fuzz targets remains the Stage 25 adversarial-validation lead
|
|
recorded by Stage 20; Stage 21 did not treat it as a durability or test-count
|
|
issue.
|
|
|
|
Stage 22 routed these investigation leads to their assigned later stages:
|
|
|
|
- Fixed registries are cheap but are reconstructed through several public
|
|
helpers. Stage 23 should decide ownership and API shape from duplication and
|
|
coherence evidence, not introduce caches for a performance problem Stage 22
|
|
did not find.
|
|
- `AUD-065` needs deterministic error precedence and ordered provenance if
|
|
acquisition becomes concurrent. Stage 23 may identify a small result-merge
|
|
abstraction; Stage 26 should preserve Stage 6 source-policy findings instead
|
|
of treating concurrency alone as the remediation.
|
|
- `AUD-066` must retain early rejection and commit-time namespace
|
|
reauthorization. Stage 23 should clarify app-versus-publisher ownership, and
|
|
Stage 25 should exercise concurrent destination changes before Stage 26
|
|
consolidates reads.
|
|
- Profile fan-out remains proportionate for documented human-selected sets
|
|
under `RET-052`. Stage 24 should not imply an arbitrary scalability promise;
|
|
Stage 26 may consider a limit or worker bound only if operational evidence
|
|
establishes a larger supported workload.
|
|
|
|
Stage 23 routed these investigation leads to their assigned later stages:
|
|
|
|
- Stage 24 should reconcile docs that present `promptinput.Save`/`LoadYAML`,
|
|
normalized generated JSON, complete render-context facts, or template module
|
|
metadata as maintained surfaces if Stage 26 selects the retirement/narrowing
|
|
direction in `AUD-069` and `AUD-072`. Current-state documentation should not
|
|
be changed before remediation.
|
|
- Stage 25 should include a Unicode-leading daypart label in its broader
|
|
parser/configuration corpus and preserve the existing destination-race cases;
|
|
it should validate the findings rather than implement shared helpers.
|
|
- Stage 26 should consolidate `AUD-001`, `AUD-032`, and `AUD-072` into one
|
|
coherent dormant-surface decision; `AUD-024`, `AUD-025`, `AUD-067`, and
|
|
`AUD-073` into one daypart identity/display sequence; and `AUD-040` with
|
|
`AUD-069` so canonical metadata ownership precedes context narrowing.
|
|
- Refactoring for `AUD-065` should keep source-local collection results and one
|
|
deterministic merge point; refactoring for `AUD-066` should keep initial
|
|
rejection and commit-time authorization in their current safety owners.
|
|
Stage 26 must sequence those behavior fixes before extracting helpers.
|
|
|
|
Stage 24 routed these investigation leads to their assigned later stages:
|
|
|
|
- Stage 25 should validate the documented accepted/rejected configuration and
|
|
CLI examples through the executable boundary while exercising its broader
|
|
adversarial corpus; Stage 24 found no reason to add live-provider or
|
|
credential-dependent documentation checks.
|
|
- Stage 26 should remediate `AUD-074`, `AUD-075`, `AUD-077`, and `AUD-078`
|
|
beside their executable owners so help, interface comments, tests, and
|
|
canonical docs change together. It should retire the two completed
|
|
comparison roadmaps under `AUD-076` only after checking every incoming link.
|
|
- Remediation of earlier implementation findings must preserve the intended
|
|
canonical safety contract under `RET-060`; do not rewrite documentation to
|
|
legitimize current leakage, cancellation, recognition, identity, or
|
|
publication defects.
|
|
- Dormant APIs and broad render surfaces remain current executable facts until
|
|
Stage 26 decides `AUD-069`/`AUD-072`. Their focused docs should narrow or
|
|
disappear with that code change, not ahead of it.
|
|
|
|
Stage 26 resolved the accumulated leads. Each now maps to a confirmed finding,
|
|
a rejected duplicate with a named consolidated owner, or a retained decision.
|
|
No unexplained failure, unresolved high-impact question, or supplemental
|
|
investigation stage remains. The choice to accept, defer, or sequence
|
|
remediation is intentionally left to Stage 27.
|
|
|
|
## Stage Log
|
|
|
|
### Stage 1: Establish The Baseline And Audit Ledger
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: repository identity, worktree state, Go module and workspace
|
|
context, graph identity, tracked package/source/test/fixture/asset/document
|
|
inventory, and required baseline validation.
|
|
- Exclusions: none beyond the stage boundary; no production code, tests,
|
|
dependencies, examples, or canonical documents were changed.
|
|
- Result: the audit is tied to exact commit `e7c7262`; the expected-baseline
|
|
difference is explained; every required baseline command passes; and all
|
|
later stages remain pending.
|
|
- Findings: none.
|
|
- Retained decisions: none.
|
|
- Open questions: none.
|
|
|
|
### Stage 2: Audit Architecture And Dependency Direction
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: `cmd/weatherreporter`, production import boundaries for all
|
|
24 Go packages, principal call and data-flow paths, external adapter
|
|
containment, project-owned interfaces, publication boundaries, filesystem
|
|
writes, ADR 0001, focused orchestration tests, and the architecture policy.
|
|
- Exclusions: Detailed report/time, configuration, CLI, weather, domain,
|
|
module, prompt, rendering, comparison, and notification correctness remains
|
|
assigned to Stages 3-18. Cross-cutting complexity, deduplication, and full
|
|
documentation coherence remain assigned to Stages 22-24.
|
|
|
|
#### Boundary Accounting
|
|
|
|
| Normative boundary or invariant | Implementation owner and evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Binary entry and CLI ownership | `cmd/weatherreporter.main` calls `cli.Run`; `internal/cli` owns action parsing, config loading, executor construction, and bounded result output. Graph traces place app calls below action resolution. | Matches policy. |
|
|
| Configuration ownership | `internal/config` owns defaults, YAML loading, normalization, secrets, and validation. CLI loads config and passes effective values down; app does not parse config files. | Matches policy; the config-to-briefing validation edge is routed to Stages 4, 8, and 23. |
|
|
| Application orchestration | `GenerateDetailed`, `RunBatchDetailed`, and `compareDetailed` sequence preflight, collection, preparation, execution, publication, and notification through repository-owned values and interfaces. No lower package imports app. | Matches policy; retained as `RET-001`. |
|
|
| Report and domain ownership | `report`, `timeutil`, `weatherdata`, `forecast`, `facts`, `module`, `briefing`, `promptinput`, `generatedtext`, and `reporttemplate` form one-way deterministic dependencies below app. | Matches policy at package level; local rules remain for Stages 3 and 7-13. |
|
|
| Prompt execution boundary | `promptexec.Executor` is dependency-neutral. CLI constructs the Promptkit adapter once per action, app consumes only `promptexec` requests/results, and the prepared-report data flow supplies curated serialized module packages. | Matches policy; retained as `RET-002`. |
|
|
| Weather API boundary | App calls its `Collector` interface; the production implementation delegates through `collect.Run` to `adapters/weatherapi.FetchBundle`, which returns normalized `weatherdata.Bundle`. Production HTTP imports occur only in external adapters. | Matches policy; retained as `RET-004`. |
|
|
| Distributor boundary | App owns notification timing and repository request/result types; `distributorNotifier` translates them to adapter-local types and the adapter alone imports the Distributor dependency. Comparison exposes no notifier path. | Matches policy. |
|
|
| Prompt preflight and curated inputs | Generate inspects the exact prompt/profile before `collectWeather`; batch inspects all candidates before its one collection; comparison inspects prompt and all profiles before collection, then calls `prepareReport` once. Data-flow traces reach `promptinput.Build` and `MarshalYAML`, not raw bundle serialization into Promptkit. | Matches policy at workflow level; detailed checks remain for Stages 10, 11, 14, and 18. |
|
|
| Single output and notification order | `publishPromptReport` checks context, calls `fileutil.WriteFileAtomic`, records the path, and only then calls `notifyReport`. Focused tests cover pre-publication preservation and notification ordering. | Matches policy; publication split retained as `RET-003`. |
|
|
| Batch output and notification order | `RunBatchDetailed` validates every planned path before sequential execution, suppresses item notification, continues across item failures, and calls `notifyBatch` only after the loop. Focused tests cover preflight, partial failure, and notification-after-publication. | Matches policy at workflow level; detailed review remains for Stage 16. |
|
|
| Comparison publication | App preflights before external work and again before publication. `internal/comparison` owns bundle validation, guarded replacement, staging, commit, restoration, and cleanup results. The comparison request has no notifier and the trace does not reach Distributor. | Matches policy at workflow level; detailed review remains for Stages 17-18. |
|
|
| Stateless execution | Normal traces retain preparation and execution values in memory. Production filesystem writes are limited to selected Markdown publication, selected comparison bundles, explicit prompt-debug capture, and the unreachable helpers in `AUD-001`; no cache, workspace, history, receipt, or resume owner appears in the package graph. | Matches normal-runtime policy; `AUD-001` records the unused persistence surface. |
|
|
| Dependency cycles and direction | The compiler-derived direct-import inventory is acyclic. Entry packages point inward, adapters do not import app/CLI, domain packages do not import orchestration, and no external dependency type appears outside its production adapter. | Matches policy. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Refreshed graph project `home-eric-Workspace-weatherreporter` in moderate
|
|
mode. The graph's branch identity remains production baseline `e7c7262`;
|
|
Stage 1's intervening commit changes only this excluded audit document.
|
|
- Used graph architecture views for structure, dependencies, entry points,
|
|
hotspots, boundaries, layers, and clusters; queried the graph schema and
|
|
direct import edges.
|
|
- Used graph search, snippets, inbound/outbound call traces, and data-flow
|
|
traces for `main`, `cli.Run`, `GenerateDetailed`, `RunBatchDetailed`,
|
|
`compareDetailed`, `prepareReport`, `executePreparedProfile`, `collect.Run`,
|
|
all three external adapters, `publishPromptReport`,
|
|
`comparison.PlanDestination`, `comparison.Publish`, and Distributor
|
|
notification.
|
|
- Used compiler-derived `go list` direct imports to separate production edges
|
|
from test-only graph edges and confirm the build has no import cycle.
|
|
- Used graph-augmented code search to inventory external dependency imports,
|
|
HTTP ownership, and production filesystem writes. A bounded text search
|
|
confirmed the unused persistence helpers have no documentation or hidden
|
|
non-code callers.
|
|
- Ran
|
|
`go test ./internal/app ./internal/cli ./internal/collect ./internal/comparison ./internal/adapters/... ./internal/fileutil ./internal/promptdebug ./internal/promptexec`;
|
|
all focused packages passed.
|
|
- Findings: `AUD-001`.
|
|
- Retained decisions: `RET-001`, `RET-002`, `RET-003`, and `RET-004`.
|
|
- Open questions: the two leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 3: Audit Report Identity And Time Foundations
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all production and focused-test code in `internal/report`
|
|
and `internal/timeutil`; report and timezone contracts in `docs/cli.md`,
|
|
`docs/config.md`, and `docs/internal/report-registry.md`; and the focused
|
|
`internal/app` and `internal/forecast` callers that consume batch membership,
|
|
run IDs, civil periods, clock windows, and hourly coverage.
|
|
- Exclusions: Effective configuration validation and CLI date parsing remain
|
|
assigned to Stages 4-5. Forecast derivation beyond the time boundary remains
|
|
assigned to Stage 7. Module, prompt, schema, and template compatibility
|
|
remains assigned to Stages 8 and 10-13. Batch notification and Distributor
|
|
idempotency behavior remains assigned to Stage 16, and documentation-wide
|
|
coherence remains assigned to Stage 24.
|
|
|
|
#### Contract Accounting
|
|
|
|
| Contract or risk | Implementation and caller evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Stable report identities | The default registry contains exactly Daily, Today, Tomorrow, and Hourly. Each definition carries the documented prompt `2.0.0`, matching template/schema identity, modules, output name, and Distributor paths; focused registry tests assert the retained surface. | Matches the registry contract; explicit definitions retained as `RET-005`. Asset compatibility remains for later owner stages. |
|
|
| Alias resolution | Command lookup accepts exact current command names. Config lookup separately normalizes whitespace, case, and hyphens. Both reject retired aliases in focused tests. | Matches the distinct boundary contracts; separation retained as `RET-006`. |
|
|
| Batch membership and ordering | Morning explicitly plans Today then Tomorrow before eligible Daily dates; Evening explicitly plans Tomorrow before eligible Daily dates. Focused app tests cover ordering, flags, multiple Daily dates, and no eligible dates. | Matches CLI and registry contracts. Detailed partial-success and notification semantics remain for Stage 16. |
|
|
| Report period policy | Daily requires an explicit date; Today uses an explicit date or current local date; Tomorrow selects the next local civil day; Hourly spans six elapsed hours from the injected clock. `Resolve` defaults a missing location to UTC and a zero clock to the system clock. | Matches documented selection policy; calendar and elapsed policies retained separately as `RET-007`. |
|
|
| Civil-day and hourly DST behavior | `CivilDay` uses local midnight plus one calendar day, and batch hourly coverage walks actual instants, correctly accommodating 23-hour and 25-hour dates. Focused batch tests exercise DST civil days. | Sufficient for the reviewed policy. |
|
|
| Configured clock windows | `ClockWindow` adds elapsed clock durations to local midnight. The only production path reaches `forecast.ResolveDayparts`, daily summaries, and facts. Fixed-zone tests do not exercise either DST transition. | Incorrect on DST dates; recorded as `AUD-002`. |
|
|
| Date and timezone parsing | `ParseLocalDate` uses strict `YYYY-MM-DD` parsing in the selected location. `LoadLocation` covers IANA names, `America/` fallback, documented aliases, fixed US abbreviations, and signed offsets, returning contextual errors for invalid input. | Implementation is coherent for Stage 3. Effective config validation and CLI error presentation remain for Stages 4-5. |
|
|
| Clock control | All report resolvers accept an injected `Now`; only zero-valued input falls back to `time.Now`. Focused report and batch tests use fixed clocks. | Sufficient for deterministic period selection. |
|
|
| Daily output and run identity | Daily output names derive from the resolved valid-start date, and Daily run IDs append that date so same-clock multi-date products remain distinct. The downstream batch-notification map rejects duplicate IDs. | Implementation is coherent, but the identity invariant lacks a focused behavior test; recorded as `AUD-003`. |
|
|
| Half-open period operations | `Period.Valid`, `Contains`, `Overlaps`, and `Intersection` consistently use `[start,end)` semantics, and focused tests protect overlap boundaries. | Matches the internal period contract. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph search, source snippets, and inbound/outbound traces for the four
|
|
report definitions, registry construction and lookup, command/config mapping,
|
|
batch mapping and planning, `Resolve`, every period resolver, `CivilDay`,
|
|
`ClockWindow`, `ParseLocalDate`, `LoadLocation`, period overlap operations,
|
|
Daily output naming, run-ID construction, `ResolveDayparts`, batch hourly
|
|
coverage, and `plannedReportsByRunID`.
|
|
- Compared the implementation with `docs/cli.md`, `docs/config.md`, and
|
|
`docs/internal/report-registry.md`, then inspected the focused report, time,
|
|
forecast, and app tests. Bounded text searches were used for string and test
|
|
assertion discovery not represented by the code graph.
|
|
- Reproduced the clock-window defect with `America/Chicago` transition dates:
|
|
17 elapsed hours after midnight yields 18:00 on 2026-03-08 and 16:00 on
|
|
2026-11-01; six elapsed hours yields 07:00 and 05:00 respectively.
|
|
- Ran
|
|
`go test ./internal/report ./internal/timeutil ./internal/forecast ./internal/app`;
|
|
all focused packages passed.
|
|
- Findings: `AUD-002` and `AUD-003`.
|
|
- Retained decisions: `RET-005`, `RET-006`, and `RET-007`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 4: Audit Configuration, Secrets, And Validation
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all production and focused-test code in `internal/config`;
|
|
the configuration-to-executor mapping in `internal/cli`; Promptkit adapter
|
|
configuration and focused tests; all three maintained examples;
|
|
`docs/config.md`; and configuration-related architecture, operations,
|
|
documentation, and testing rules.
|
|
- Exclusions: CLI parsing and action-specific diagnostics remain assigned to
|
|
Stage 5. Weather API transport behavior remains assigned to Stage 6.
|
|
Promptkit source loading and sensitive execution diagnostics remain assigned
|
|
to Stage 11. Output publication preflight remains assigned to Stages 15 and
|
|
17, Distributor delivery remains assigned to Stage 16, and repository-wide
|
|
documentation coherence remains assigned to Stage 24.
|
|
|
|
#### Precedence And Validation Map
|
|
|
|
| Contract or risk | Implementation and consumer evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Discovery and precedence | `Load` starts with `Defaults`, merges the explicit or default YAML path, tolerates only an absent implicit default file, applies nonempty units/timezone overrides, normalizes report modules, loads configured secrets, and validates. Representative CLI coverage proves explicit units/timezone beat file values while unrelated file values remain. | Matches the documented defaults → file → CLI order. Environment variables do not override config fields. |
|
|
| YAML schema and migrations | `mergeFile` uses `KnownFields(true)`; custom notification, report, Distributor-path, and module decoders reject their own unknown fields. Retired `scriptorium`, recent-change, and workspace keys have focused migration/rejection tests. | Sufficient for known struct fields and retired top-level contracts. Unsupported keys inside the semantic missing-source map are `AUD-004`. |
|
|
| Defaults and partial merges | Exact documented defaults are built once in `Defaults`; YAML decoding merges partial nested structs over them, and focused tests cover default literals, omitted optional sections, batch partial configuration, and both maintained config examples. | Matches the canonical reference. Exact-default assertions protect user-visible contracts rather than private constants. |
|
|
| Output directory | Config accepts empty, relative, and absolute values, preserves them byte-for-byte, and rejects only nonempty whitespace. Graph traces place path resolution, inspection, creation, and publication below app/filesystem owners. | Matches policy; retained as `RET-008`. |
|
|
| Weather and daypart validation | Config checks absolute base URLs when present, positive timeout, nonnegative precision, nonempty units/timezone, resolvable timezone, JSON format, nonempty dayparts, names, and parseable clocks. Collection separately requires a base URL for collection actions. | Matches the documented split. The Stage 3 DST defect remains `AUD-002`. |
|
|
| Missing-source policy | Policy values and nonblank keys are validated, and the adapter consumes source-specific values by source name with a default fallback. Required hourly failures bypass optional-source policy. | Unsupported and ineffective keys pass silently; `AUD-004`. |
|
|
| Report/module overrides | Loaded aliases, duplicate normalized reports, module identities/order, compatibility, duplicate modules/stanzas, typed options, explicit empty lists, and Distributor path templates are validated through report/module registries. | Loaded YAML behavior is well covered. Exported constructed values cannot establish private presence state; `AUD-006`. Detailed module semantics remain for Stages 8-9. |
|
|
| Promptkit settings | Config validates mutually exclusive profile sources, positive timeout, optional absolute local endpoint, and nonnegative concurrency. CLI maps a dormant endpoint-less limit to the adapter zero value; adapter construction owns file/directory loading, backend registration, fallbacks, and safe error classification. | Matches the documented inactive/active split; retained as `RET-009` and `RET-010`. |
|
|
| Secret directory | Empty disables loading. A configured directory accepts only directly contained regular files with environment-variable basenames, rejects symlinks/directories/nonregular/unreadable entries, removes one LF or CRLF, overwrites the named environment value, and reports only directory/file names and operation context. | Values are absent from tested diagnostics, but failure application is not atomic; `AUD-007`. |
|
|
| Distributor configuration | Disabled notification skips active-field checks. Enabled notification validates endpoint, token variable name, timeout, failure policy, allowed template variables, batch activation, and report path safety/uniqueness. Templates render from repository-owned metadata rather than environment values. | Mostly sufficient. Two single-report rendered identities omit the nonblank invariant; `AUD-005`. |
|
|
| Filesystem and URL timing | Config validates intrinsic string/URL shape. Output destinations are inspected by app/publication owners, Promptkit sources by its adapter, and secret entries by the config loader. No audit command contacted an external service. | Matches ownership boundaries for Stage 4; transport-specific scheme support is routed to later stages. |
|
|
| Constructed configuration | `Validate`, report override accessors, and focused tests attempt to provide loaded/constructed consistency without mutating caller-owned option values. | General scalar validation is consistent, but report override presence is not externally constructible; `AUD-006`. |
|
|
| Maintained examples | Both YAML configuration examples load through the strict config suite. The endpoint-only `weather-light` profile is inspected through the real Promptkit adapter with embedded assets and no network or credential. All examples contain synthetic endpoints and no secrets. | Matches documentation and testing policy. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph search, source snippets, call traces, and graph-augmented test
|
|
searches for defaults, `Load`, YAML merging and migration rejection,
|
|
`Validate`, secret loading, report/module/path traversal, notification
|
|
renderers, CLI executor mapping, Promptkit adapter construction, Weather API
|
|
missing-source policy consumption, and app notification consumers.
|
|
- Compared executable fields, defaults, precedence, templates, paths, profile
|
|
settings, source keys, and secret behavior with `docs/config.md`, maintained
|
|
examples, and the configuration-related operations and architecture rules.
|
|
- Ran the production CLI with `examples/minimal-config.yml`. It reached offline
|
|
profile credential preflight and returned `missing_credential` before weather
|
|
collection, confirming that the endpoint-less local concurrency default is
|
|
intentionally dormant rather than rejected by adapter construction.
|
|
- Ran
|
|
`go test ./internal/config ./internal/adapters/promptkit ./internal/cli`;
|
|
all focused packages passed.
|
|
- Findings: `AUD-004`, `AUD-005`, `AUD-006`, and `AUD-007`.
|
|
- Retained decisions: `RET-008`, `RET-009`, and `RET-010`.
|
|
- Open questions: the four leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 5: Audit CLI Parsing, Wiring, And Output Contracts
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: `cmd/weatherreporter`, all production and representative
|
|
tests in `internal/cli`, `docs/cli.md`, `docs/internal/cli.md`, immediate
|
|
config/report/time/executor mapping, and app request/result/error boundaries.
|
|
Comparison parsing and CLI translation were reviewed, while detailed
|
|
comparison execution remains excluded.
|
|
- Exclusions: Weather collection, report generation, publication, notification,
|
|
and comparison execution correctness remain assigned to Stages 6-18.
|
|
Cross-cutting parser deduplication, test portfolio durability, documentation
|
|
coherence, and dynamic robustness remain assigned to Stages 21 and 23-25.
|
|
|
|
#### Command And Exit Accounting
|
|
|
|
| Command or risk | Parsing, result, and exit evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Root help and version | No arguments, `-h`, and `--help` print static help without config/provider setup. Exact `--version` prints the injected or build version; extra version arguments fail. Main prints any returned error to stderr and exits `1`; nil returns normally. | Matches the documented root contract. Help wording drift is routed to Stage 24. |
|
|
| Generate dispatch | Requires one of four exact report names. Command-specific flag sets accept only documented flags; Daily requires a date, Today defaults or parses one, and Tomorrow/Hourly reject `--date`. Config and explicit output/working-directory values map into one app request and one executor. | Functional mapping matches, but date validation occurs after executor construction; `AUD-009`. Assembled result/exit coverage is `AUD-010`. |
|
|
| Run dispatch | Requires exact Morning or Evening batch identity, accepts `--out-dir` rather than `--out`, resolves only explicit relative overrides against the absolute invocation directory, preserves configured fallback, and constructs one executor/request. | Matches the parser/wiring contract. Representative assembled result coverage is incomplete in `AUD-010`; detailed batch behavior remains Stage 16. |
|
|
| Compare dispatch | Requires a valid report and two distinct nonblank ordered profiles before config/executor work; applies the same report date eligibility; clears the configured default profile; maps exact bundle destination/replace/debug options; constructs one executor. | Matches the CLI contract. Focused tests cover parser, construction frequency, pre-execution failure, success/partial/cleanup summaries, quiet behavior, and safe errors. Detailed execution remains Stage 18. |
|
|
| Common flags and ownership | `addCommonFlags` registers config, units, timezone, and prompt debug only. Each action separately registers its output, quiet, date, profile, and replacement flags. Report and batch name lookup delegates to `internal/report`; config precedence delegates to `internal/config`. | Intentional explicitness retained as `RET-011`; repeated date policy is routed to Stage 23. |
|
|
| Validation ordering | Unknown commands/reports, unsupported flags, extra args, and comparison profile shape fail before configuration or executor creation. Generate performs Daily presence and Daily/Today parsing after config and executor creation. | Partly incorrect; `AUD-009`. |
|
|
| Output paths | CLI captures/validates one absolute working directory, preserves configured fallbacks, and resolves only nonempty relative explicit output overrides against that directory. App owns default naming and publication. | Matches CLI/config documentation and architecture ownership. |
|
|
| Stdout, stderr, and quiet | Result-bearing actions emit indented JSON to stdout. Run emits compact ordered item/notification/batch lines to stderr first. Quiet returns before both routine channels but the returned error still reaches main stderr. Pre-result failures emit no structured output. | Matches the documented separation; summary design retained as `RET-012`. Broken status-writer behavior is routed to Stage 25. |
|
|
| Generate summaries | Selected report, prompt, time, profile/model, warnings, validation, output, debug, notification, and error fields are copied; status is failed exactly when a result accompanies an error. | Shape matches documentation, but only the constructor is tested; assembled behavior is `AUD-010`. Producer redaction remains for Stages 6, 11, and 16. |
|
|
| Batch summaries and exit | Report counters remain independent of notification failure. Any failed report or failed notification makes summary status/error failed, emits status lines unless quiet, and returns typed `BatchError` so main exits nonzero. | Mapping is coherent and notification failure is assembled-tested. Broader representative translation coverage is `AUD-010`. |
|
|
| Comparison summaries | Field order is fixed by the explicit DTO; profile order is retained; unpublished artifact fields are omitted; errors are mapped to bounded aggregate, cancellation, deadline, Promptkit, destination, cleanup, or application categories. | Matches the documented safe CLI contract with strong focused coverage. |
|
|
| Cancellation | `Runner.Run` forwards its context to every app action, and app tests protect cancellation behavior. Production main supplies an uncancelable background context and has no signal bridge. | Programmatic propagation is correct; executable cancellation is `AUD-008`. |
|
|
| Test ownership | Parser/resolver tests own flag acceptance, paths, configuration mapping, and one-executor construction; summary tests own selected fields; comparison command tests own assembled translation. | Comparison is sufficient. Non-comparison assembled coverage is insufficient in `AUD-010`; lower workflow details correctly remain in app tests. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph inventory, source snippets, inbound/outbound traces, and
|
|
graph-augmented searches for root dispatch, every action parser/resolver,
|
|
common flag registration, working/output path resolution, executor mapping,
|
|
summary/status construction, output routing, error classification, app
|
|
cancellation consumers, and process signal handling.
|
|
- Compared executable help, accepted flags, summary fields, error/status rules,
|
|
and action wiring with `docs/cli.md`, `docs/internal/cli.md`, and the
|
|
configuration, architecture, documentation, and testing contracts.
|
|
- Ran `go test -cover ./internal/cli`; the suite passed at 75.8% statement
|
|
coverage. Coverage was used only to guide test inspection, not as a finding.
|
|
- Ran `go run ./cmd/weatherreporter --help`,
|
|
`go run ./cmd/weatherreporter --version`, and
|
|
`go run ./cmd/weatherreporter generate daily`; help and development-version
|
|
paths succeeded, while missing Daily date returned the documented pre-result
|
|
error and nonzero status without contacting an external service.
|
|
- Findings: `AUD-008`, `AUD-009`, and `AUD-010`.
|
|
- Retained decisions: `RET-011` and `RET-012`.
|
|
- Open questions: the four leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 6: Audit Weather Data Acquisition And Collection
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all production code and focused tests in
|
|
`internal/adapters/weatherapi`, `internal/weatherdata`, and
|
|
`internal/collect`; all eight adapter fixtures; the Weather API integration,
|
|
normalized weather-data, and collection internal documents; and immediate
|
|
app/prompt-facing consumers needed to account for provenance, warnings, and
|
|
error exposure.
|
|
- Exclusions: Meteorological selection, alert interpretation, forecast/fact
|
|
derivation, and aliasing policy remain assigned to Stage 7. Prompt-facing
|
|
curation remains assigned to Stages 8-10, app orchestration to Stages 14-16,
|
|
cross-cutting cleanup to Stage 23, documentation reconciliation to Stage 24,
|
|
and adversarial dynamic checks to Stage 25.
|
|
|
|
#### Source And Failure Accounting
|
|
|
|
| Source or transport risk | Required, empty, malformed, and provenance behavior | Disposition |
|
|
| --- | --- | --- |
|
|
| Warmup | Calls `/conditions/current` first with format, units, and precision; requires a readable 2xx body; closes it; and stops before source requests on failure. Context cancels requests and retry waits. | Endpoint and cancellation behavior match. Permanent statuses are retried contrary to the shared policy in `AUD-015`; body safety/limits are `AUD-012` and `AUD-013`. |
|
|
| Observations | Optional. Missing/null and malformed data use configured policy. A successful value records observation timestamp as issue time plus endpoint, query, fetch time, and compact-data hash. | Matches the source contract. |
|
|
| Current conditions | Optional. Missing/null and malformed data use configured policy; successful normalized fields and provenance are retained. | Matches the source contract. |
|
|
| Hourly forecast | Required regardless of optional-source policy. Missing/null, decode failure, and an empty period list fail collection; format, units, precision, and timezone are sent. | Availability mapping matches, but periods with zero/reversed time bounds pass; `AUD-014`. |
|
|
| Narrative forecast | Optional. Missing/null and malformed data use configured policy; issue/update timestamps and forecast periods are normalized. An empty period list remains checked data. | Matches the documented source contract. Semantic forecast use remains Stage 7. |
|
|
| Active alerts | Optional. An absent member is missing, explicit `null` is checked empty data with a hash and no warning, and malformed non-null data uses configured policy. Alert items remain raw payloads at this boundary. | Matches the explicit alerts exception. Full duplicate raw payload is routed to Stage 23. |
|
|
| Forecast discussion | Optional. Missing/null and malformed data use configured policy; issue/update times, key messages, and short/long sections are normalized. | Matches the source contract. |
|
|
| Weather story | Optional. Uses format only; missing/null and malformed data use configured policy; start/update provenance is retained when available. | Matches the source contract. |
|
|
| SPC convective outlooks | Optional. Uses format/timezone without units; missing/null uses configured policy; non-null empty outlook/discussion lists are checked data; issue time prefers `issuedAt` and falls back to `asOf`; GeoJSON remains raw. | Matches the explicit SPC empty-data and provenance contract. |
|
|
| Optional-source policy | `error` aborts with no partial bundle, `warn` records the same stable warning in source and bundle, and `none` records a missing source without a warning. Transport/status/envelope failures remain direct request errors. | Matches the documented division. Unsupported policy keys remain the Stage 4 finding `AUD-004`. |
|
|
| Endpoint construction | Joins every fixed endpoint to a base path prefix and applies only the required format/units/precision/timezone matrix. | Query/path behavior matches for HTTP(S); unsupported absolute schemes survive until runtime in `AUD-011`. Base URL query/user-info shape is routed to Stage 24. |
|
|
| HTTP attempt lifecycle | Requests carry context and configured client timeout; response bodies close after bounded reads; source attempts retry transport/read failures and only the documented transient statuses. | Mostly coherent. Unsafe response text and silent truncation are `AUD-012`/`AUD-013`; warmup classification divergence is `AUD-015`. |
|
|
| Normalization and collection ownership | `weatherdata` contains no HTTP/config/filesystem behavior. The adapter translates wire envelopes into project types. `collect.Run` constructs one adapter, forwards context, and distinguishes setup from fetch errors. | Matches architecture; the narrow collection seam remains retained under `RET-004`. Source-specific normalization remains explicit under `RET-013`. |
|
|
| Test assets and ownership | Adapter tests use `httptest.Server` for HTTP/query/status/retry/cancellation/policy behavior and eight small checked-in JSON fixtures for translation. `weatherdata` owns a focused GeoJSON round trip; `collect` owns three real local composition/error cases. Fixtures are synthetic, credential-free, and total under 5 KiB. | Ownership is distinct and offline. Missing high-risk cases are attached to `AUD-011` through `AUD-015`, not inferred from coverage alone. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph architecture, symbol search, source snippets, inbound traces, and
|
|
graph-augmented use searches for adapter construction, all eight source
|
|
methods, HTTP attempts and retries, body lifecycle, endpoint/query assembly,
|
|
missing/malformed policy, hashes and timestamps, normalized source consumers,
|
|
`collect.Run`, and propagation toward app/CLI boundaries.
|
|
- Compared implementation and fixtures with
|
|
`docs/integrations/weatherapi.md`, `docs/internal/weather-data.md`,
|
|
`docs/internal/collect.md`, the architecture policy, and the testing policy.
|
|
Bounded text inspection was used for documentation, JSON fixtures, literal
|
|
credential markers, and known test assertions outside graph discovery.
|
|
- Ran
|
|
`go test -coverprofile=/tmp/weatherreporter-stage6-cover.out ./internal/adapters/weatherapi ./internal/weatherdata ./internal/collect`;
|
|
all focused packages passed. Adapter statement coverage was 84.9% and
|
|
collection coverage 100%; coverage was used only to guide branch inspection.
|
|
- Findings: `AUD-011`, `AUD-012`, `AUD-013`, `AUD-014`, and `AUD-015`.
|
|
- Retained decisions: existing `RET-004` was revalidated and `RET-013` records
|
|
explicit source normalization.
|
|
- Open questions: the four leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 7: Audit Forecast And Fact Derivation
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all production code and focused tests in
|
|
`internal/forecast` and `internal/facts`; the forecast derivation and facts
|
|
internal documents; normalized forecast types; and immediate briefing/app
|
|
consumers needed to determine unit meaning, mutation ownership, and whether
|
|
derived facts participate in normal workflows.
|
|
- Exclusions: Detailed source-facing, derived, planning, formatting, and SPC
|
|
briefing behavior remains assigned to Stages 8-9. Cross-cutting test
|
|
portfolio and cleanup remain assigned to Stages 19-23, documentation
|
|
reconciliation to Stage 24, and adversarial dynamic checks to Stage 25.
|
|
|
|
#### Derivation Rule And Edge-Case Accounting
|
|
|
|
| Derivation rule or risk | Boundary, ordering, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Valid-period selection | Hourly, narrative, and daily runs use half-open overlap, retain complete source periods, and stable-sort by start time. Nil runs return no facts. Focused tests cover unsorted input, partial overlap, and exact-boundary exclusion. | Matches the source-period contract; complete-period selection is retained as `RET-014`. Structurally invalid required periods remain `AUD-014`. |
|
|
| Civil-day summaries and dayparts | Daily summaries require hourly data, resolve configured local windows, and select each daypart independently. Empty dayparts remain explicit summaries; overnight hourly selection crosses midnight. | Ordinary and empty behavior matches. DST window construction remains `AUD-002`; after-midnight alert selection is incorrect in `AUD-017`. |
|
|
| Temperature and wind aggregation | Min/max and peak selection is deterministic for populated values; numeric heat/cold thresholds distinguish Fahrenheit and Celsius. Wind and all unitless summary fields fall back between explicit unit fields without conversion. | Incorrect for metric fallback values; `AUD-016`. Paired unit-system regressions are absent. |
|
|
| Condition indicators | Text matching is case-insensitive; numerical heat, cold, and wind signals merge monotonically across periods. Dominant-condition ties resolve lexically and notable conditions sort. | Deterministic, but common “icy” text is missed; `AUD-019`. Broader briefing interpretation remains Stage 9. |
|
|
| Precipitation timing | Input is copied and stable-sorted. Exact threshold inclusion, dry input, gaps, maxima, text aggregation, thunder mentions, and intentionally open final windows have focused tests. | Window mechanics match. The percentage domain is unenforced in `AUD-018`; the open final window is an intentional horizon signal. |
|
|
| Alert parsing and overlap | Supported time-field fallbacks, malformed omission, instruction whitespace, half-open clipping, ordering, and direct boundary cases have focused tests. | Direct behavior matches. Daily prefiltering loses alerts in wrapping dayparts; `AUD-017`. |
|
|
| Collected facts and mutation | Collection projects every normalized product and copies top-level sources/warnings. Bundle reconstruction repeats those top-level copies; derivation reads shared nested values without mutation. | Fits immutable workflow ownership and is retained as `RET-015`; no speculative deep-copy requirement was added. |
|
|
| Report-family derived facts | Every report selects valid-period hourly, narrative, daily, alert, precipitation, and SPC facts. Daily/Today/Tomorrow additionally build one daily/daypart summary; Hourly does not; unknown reports and invalid periods fail. | Matches the implemented report-family contract with focused cases. The stale multi-day documentation name is routed to Stage 24. |
|
|
| SPC outlooks and discussions | Only valid overlapping outlook periods are selected. Output ordering uses day, type, severity rank, validity, label, and ID; discussions are retained only for selected days and sorted deterministically. Nil and checked-empty runs remain empty facts. | Matches documented checked-empty and deterministic ordering behavior with focused tests. Detailed briefing meaning remains Stage 9. |
|
|
| Repeated work and realistic size | One derived build performs several linear selections, then each configured daypart scans the hourly run and its alert candidates; selected precipitation and SPC slices are sorted. With the normal four dayparts and service-sized runs of a few hundred periods, this is only low-thousands of simple overlap checks. | No Stage 7 performance finding. Reassess in Stage 22 only if horizons, daypart counts, or alert volumes become materially larger or derivation is repeated per module. |
|
|
| Test ownership | `forecast` owns selection, summaries, thresholds, precipitation, parsing, overlap, ordering, and fixture integration. `facts` owns projection, report-family derivation, SPC filtering/order, and failure cases. `timeutil` owns the underlying period rules. | Ownership is coherent. Missing high-risk unit, overnight-alert, percentage-domain, and icy-text cases are attached to `AUD-016` through `AUD-019`, not inferred from coverage alone. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph architecture, symbol search, source snippets, call traces, and
|
|
graph queries for every scoped derivation function, report-family consumers,
|
|
unit-bearing normalized fields, comparison helpers, loops, sorting, and
|
|
immediate unit interpretations. Text search was limited to documents,
|
|
literals, fixtures, and focused assertions where graph discovery was not
|
|
applicable.
|
|
- Compared implementation and tests with
|
|
`docs/internal/forecast-derivation.md`, `docs/internal/facts.md`, and the
|
|
architecture, documentation, and testing policies. Existing Stage 3 and
|
|
Stage 6 findings were treated as dependencies rather than duplicated.
|
|
- Ran
|
|
`go test -coverprofile=/tmp/weatherreporter-stage7-cover.out ./internal/forecast ./internal/facts ./internal/timeutil`;
|
|
all focused packages passed. Statement coverage was 89.7% for forecast,
|
|
73.4% for facts, and 54.5% for timeutil; coverage guided branch inspection
|
|
but was not itself a finding.
|
|
- Findings: `AUD-016`, `AUD-017`, `AUD-018`, and `AUD-019`.
|
|
- Retained decisions: `RET-014` and `RET-015`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 8: Audit Module Contracts, Registry, And Source-Facing Briefing Modules
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: `internal/module`; briefing registry and package contracts;
|
|
metadata, current conditions, narrative forecast, hourly forecast, alert
|
|
digest, area forecast discussion, and weather story builders and their
|
|
directly related tests; report default composition; configuration module
|
|
normalization/validation; and the relevant module, briefing, and prompt-input
|
|
internal documentation.
|
|
- Exclusions: Derived daily/daypart, precipitation, outdoor/planning, detailed
|
|
formatting, and SPC builders remain assigned to Stage 9. Complete prompt
|
|
serialization and asset contracts remain Stage 10, application preparation
|
|
remains Stage 14, cross-cutting cleanup remains Stage 23, and documentation
|
|
coherence remains Stage 24.
|
|
|
|
#### Registry And Source-Module Accounting
|
|
|
|
| Contract or module | Registry, transformation, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Neutral module envelope | Stable IDs and fact/option vocabulary live in `internal/module`; snapshots preserve caller order, omit runtime prompt values from JSON, reject missing identities and duplicates, and support typed rich-stanza lookup. | Separation is retained as `RET-016`. Exact version compatibility is not enforced; `AUD-022`. |
|
|
| Registry initialization | Default definitions have unique IDs/stanzas, builders, supported reports, typed defaults, missing-data behavior, requirements, and prompt-export policy. Constructor tests cover duplicates, absent builders, and unsupported `warn`; default report compositions validate. | Default metadata is coherent. Unknown custom missing-data values fail only at build time; low-impact constructor hardening is left for Stage 23 unless the registry becomes extensible. |
|
|
| Configuration and options | Configuration normalizes raw YAML into registry-declared option types, validates compositions through the same registry, and preserves report order. Only AFD has nonempty default selections and consumes a section list. | IDs/defaults/order agree. Accepted pointer/value option shapes diverge at the AFD builder; `AUD-020`. |
|
|
| BuildModule policy | Lookup, report support, option shape, fact availability, omit/error/empty behavior, output identity, and prompt export execute in one boundary. Exporter failures receive module/stanza context and outputs are identity-checked before export. | Coherent for defaults. The branching is policy-driven rather than per-module switch testing; builders are exercised through public registry behavior. |
|
|
| Metadata | Emits report identity, variant, prompt, generation/period context, configured units/timezone/location, and bounded source-warning summaries without endpoint/hash provenance or alert duplication. It remains available with empty source metadata. | Matches the documented prompt-safe module contract with focused field/omission tests. Parallel non-module metadata construction is routed to Stage 23. |
|
|
| Current conditions | Optional source; rounds explicit unit fields, retains day/condition/humidity, derives compass labels, omits a wholly empty value, and uses a curated prompt projection without lowercase/display helpers. | Matches source and prompt-export contracts with focused rich/prompt tests. |
|
|
| Narrative forecast | Daily/Today/Tomorrow only; requires both source presence and selected derived periods, maps only valid-period values, copies unit-bearing fields, and produces local friendly bounds and compass wind labels. Missing or empty selections omit the module. | Matches support, selection, transformation, and omission contracts with focused tests. Invalid normalized percentages remain `AUD-018`. |
|
|
| Hourly forecast | All reports; requires source presence and selected periods, maps the complete selected weather fields, derives labels/compass direction, and marks precipitation at the exact 20-percent threshold. Its prompt export omits rich-only labels and mention helpers. | Matches support, boundary, rich-value, and curated-export contracts with focused tests. Invalid period/percentage inputs remain `AUD-014`/`AUD-018`. |
|
|
| Alert digest | All reports with explicit empty behavior; distinguishes checked-empty and source-missing states, counts raw active and relevant overlaps separately, preserves derived order, and formats full alert periods with guidance. | Matches availability and value contracts; distinction retained as `RET-018`. Overnight relevance remains the upstream `AUD-017`. |
|
|
| Area forecast discussion | Optional for all reports; empty section lists mean all sections, report defaults select long-term for Daily and key messages/short-term for Hourly, unsupported names fail, and selected slices are copied. | Section policy and defaults agree, but pointer-shaped accepted options fail construction; `AUD-020`. |
|
|
| Weather story | Optional for all reports; copies story content, flags, URL, update time, and friendly valid-period labels. | Populated and missing behavior is tested. Structurally empty data is incorrectly emitted as available; `AUD-021`. Complete prompt curation remains Stage 10. |
|
|
| Mutation and ordering | Snapshot outputs copy the outer slice; source modules copy scalar pointers and AFD key-message slices. Prepared-report construction later deep-copies complete inputs. Registry and report order flow unchanged into snapshots. | No shared-mutation finding under immutable preparation ownership. Rich/prompt separation is retained as `RET-017`. |
|
|
| Complexity drivers | `BuildModule` centralizes validation/missing/export branching; fact availability uses parallel requirement switches; default definitions form a long policy table; current/hourly prompt projections copy many fields. Builders otherwise perform small linear maps over service-sized periods/alerts. | No Stage 8 performance finding. Table/switch drift, parallel metadata, and projection duplication are routed to Stage 23 with concrete owners. |
|
|
| Test ownership | Module tests own snapshot structure/order and rich-versus-prompt selection. Briefing registry tests own default composition, support, options, missing behavior, output/export validation, and assembled snapshots. Source-module tests own observable transformations and omissions; config tests own YAML normalization. | Mostly sufficient and behavioral. Missing pointer-consumption, empty-story, and unsupported-schema cases are attached to `AUD-020` through `AUD-022`. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph architecture, symbol search, source snippets, inbound traces, and
|
|
graph queries for snapshot consumers, registry/config validation, all scoped
|
|
builders and exporters, source fields, default report compositions, tests,
|
|
shared-state copying, and complexity drivers. Text inspection was limited to
|
|
scoped documentation and targeted source/test bodies.
|
|
- Compared implementation and focused tests with `docs/internal/module.md`,
|
|
relevant portions of `docs/internal/briefing.md` and
|
|
`docs/internal/prompt-input.md`, and the architecture, documentation, and
|
|
testing policies.
|
|
- Ran
|
|
`go test -coverprofile=/tmp/weatherreporter-stage8-cover.out ./internal/module ./internal/briefing ./internal/config ./internal/report`;
|
|
all focused packages passed. Statement coverage was 86.5% for module, 82.5%
|
|
for briefing, 86.1% for config, and 48.8% for report; coverage guided risk
|
|
inspection but was not itself a finding.
|
|
- Findings: `AUD-020`, `AUD-021`, and `AUD-022`.
|
|
- Retained decisions: `RET-016`, `RET-017`, and `RET-018`.
|
|
- Open questions: the four leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 9: Audit Derived, Planning, Formatting, And SPC Briefing Modules
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: the remaining production code and focused tests in
|
|
`internal/briefing`: daily and daypart summaries, precipitation timing,
|
|
outdoor windows, Daily/Today/Tomorrow planning, shared summary and formatting
|
|
helpers, SPC outlooks and discussions, and the embedded SPC definition
|
|
asset; plus relevant briefing, facts, configuration, and authoritative SPC
|
|
references needed to judge the domain rules.
|
|
- Exclusions: Complete prompt serialization and embedded prompt assets remain
|
|
Stage 10; generated-text schemas and render contexts remain Stages 12-13;
|
|
application preparation remains Stage 14; test-portfolio and cross-cutting
|
|
cleanup remain Stages 19-23; canonical documentation reconciliation remains
|
|
Stage 24; and adversarial dynamic checks remain Stage 25.
|
|
|
|
#### Derived And Planning Rule Accounting
|
|
|
|
| Module or risk | Threshold, ordering, missing-data, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Daily summary | Selects narrative high/low and daily precipitation when present, falls back to daypart aggregates, preserves deterministic conditions/hazards, and emits timing/gust facts. | Ordinary and fallback selection are exercised. Apparent temperature is incorrectly relabeled as heat index in `AUD-023`; unit fallback remains upstream `AUD-016`. |
|
|
| Daypart summaries | Emit local periods, ranges, condition/hazard flags, precipitation and gust maxima, temperature phrases/trends, and configured normalized keys; date prefixes prevent ordinary multi-day collisions. | Field and positive-temperature behavior is exercised. Canonical key collisions are `AUD-024`, subzero wording is `AUD-027`, and unit interpretation remains `AUD-016`. |
|
|
| Precipitation timing | Maps dry and rainy windows, open-ended periods, expectation thresholds, type phrases, thunder mentions, and local labels without mutating derived order. | Matches the reviewed briefing contract with focused rainy, dry, threshold, phrase, and missing-daypart cases. Percentage-domain defects remain `AUD-018`. |
|
|
| Outdoor windows | Scores precipitation, gusts, alerts, heat, and cold; preserves deterministic input-order ties and exposes best/worst periods and reasons to Today planning. | Incomplete hazard policy: snow, ice, and fog can score as quiet; `AUD-026`. |
|
|
| Planning variants | Daily and Tomorrow share morning, workday, and overnight sections; Today intentionally adds outdoor and late-day sections. Empty daily summaries produce empty modules or defined fallback prose according to builder context. | Report variation is intentional and retained as `RET-019`. Configurable name classification is inconsistent in `AUD-025`; upstream unit/alert/text defects are not duplicated. |
|
|
| User-facing formatting | Time/date labels use the requested location with UTC fallback, wind compass boundaries are focused-tested, strings are deduplicated in stable order, and sentence casing is Unicode-aware. | Ordinary formatting is coherent. Signed temperature bands are incorrect in `AUD-027`; byte-oriented title casing is routed to Stage 23 after the identity defects. |
|
|
| SPC outlooks | Preserve selected overlapping records and checked/missing status, attach definitions only for recognized normalized keys, and compute a location-specific categorical risk digest at severity rank 3 or higher. | Selection, ordering, threshold equality, type, location, overlap, unknown definitions, and empty/missing states have focused tests; separation retained as `RET-020`. Official categorical descriptions have drifted in `AUD-028`. |
|
|
| SPC discussion | Includes selected discussions only for days with an overlapping categorical outlook meeting the rank threshold; ignores high-rank noncategorical products and nonoverlapping outlooks. | Matches the explicit regional discussion policy with focused threshold, day, type, overlap, missing-text, and missing-source cases. |
|
|
| Complexity and repetition | Scoped builders perform small linear maps over configured dayparts, precipitation windows, outlooks, and discussions. The highest production cognitive scores are in planning, precipitation type selection, and outdoor scoring, with no material service-sized cost. | No Stage 9 performance finding. Repeated daypart role/display logic is routed to Stage 23 with concrete evidence. |
|
|
| Test ownership | One derived-module suite owns daily/daypart/precipitation/planning behavior, format-helper tests own compass mapping, and dedicated SPC suites own risk/discussion/definition behavior. | Mostly behavioral. Missing semantic regressions are attached to `AUD-023` through `AUD-028`, not inferred from coverage alone. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph symbol and augmented-code search, source snippets, inbound traces,
|
|
and complexity queries for every remaining builder/helper, configured
|
|
daypart identities, apparent-temperature flow, outdoor hazard inputs, signed
|
|
temperature bands, SPC selection/definition lookup, and their focused tests.
|
|
Bounded text inspection was used for scoped documents, configuration
|
|
examples, and the embedded JSON asset.
|
|
- Compared SPC vocabulary and externally attributed definitions with NOAA/NWS
|
|
Storm Prediction Center's current convective-outlook reference and the
|
|
[2026 Conditional Intensity service-change notice](https://www.weather.gov/media/notification/pdf_2026/scn26-11_SPC_conditional-intensity.pdf).
|
|
Categorical definition drift is recorded as `AUD-028`; supported CIG
|
|
identifiers and the distinction between occurrence probability and
|
|
conditional intensity remain coherent.
|
|
- Ran
|
|
`go test -coverprofile=/tmp/weatherreporter-stage9-cover.out ./internal/briefing ./internal/config ./internal/forecast ./internal/facts`;
|
|
all focused packages passed. Statement coverage was 82.5% for briefing,
|
|
86.1% for config, 89.7% for forecast, and 73.4% for facts; coverage guided
|
|
missing-edge inspection but was not itself a finding.
|
|
- Findings: `AUD-023`, `AUD-024`, `AUD-025`, `AUD-026`, `AUD-027`, and
|
|
`AUD-028`.
|
|
- Retained decisions: `RET-019` and `RET-020`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 10: Audit Prompt Inputs, Assets, And Neutral Execution Contracts
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all production and focused-test code in
|
|
`internal/promptinput`, `internal/promptassets`, and `internal/promptexec`;
|
|
every embedded prompt, Promptkit profile, input schema, and output schema;
|
|
prompt-input and generated-text internal documents; the Promptkit integration
|
|
contract; and immediate briefing/application consumers needed to establish
|
|
the provider-boundary data flow.
|
|
- Exclusions: Promptkit adaptation, provider cancellation/error translation,
|
|
and secure debug files remain Stage 11; output schemas and generated-text
|
|
semantics remain Stage 12; application preparation/provider preflight remains
|
|
Stage 14; cross-cutting API cleanup and documentation coherence remain Stages
|
|
23-24; and adversarial repository-wide checks remain Stage 25.
|
|
|
|
#### Prompt Boundary And Execution Accounting
|
|
|
|
| Contract or risk | Serialization, asset, caller, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Curated work-package construction | Application preparation reaches `promptinput.Build` and `MarshalYAML` from ordered module `DataPackageValue` exports, not normalized bundle serialization. Report identity, local date, period, and module categories are explicit. | Module values follow the curated path, but the top-level raw warning copy violates it in `AUD-029`. |
|
|
| Category order and normalization | One fixed table groups known stanzas as metadata, applicable risk, derived, narrative, and raw data; module order is retained within groups. Values normalize through JSON before YAML, and repeated marshals are byte-identical in focused tests. | Coherent and retained as `RET-021`; missing failure-branch coverage is `AUD-033`. |
|
|
| Package validation and loading | Current schema, report/run metadata, ordered stanza presence, duplicate order entries, known category membership, and misplaced stanzas are rejected. A temporary table-driven probe exercised additional malformed shapes. | Incomplete for required metadata, orphan values, unknown struct fields, and trailing documents; `AUD-032`. |
|
|
| Embedded prompt identities | Exactly four report definitions bind the registry's prompt ID/version, embedded profile, `weatherreporter.data_package.v4` input schema, and matching generated-text output schema. Real offline Promptkit inspection succeeds for each. | Identity is coherent; common prose references nonexistent serialized paths in `AUD-030`. |
|
|
| Safety-sensitive prompt semantics | The shared prompt prioritizes risk, forbids invention, and asks regional discussion limitations to be preserved. | Its blanket assertion that all retained products are location matched contradicts SPC module semantics; `AUD-031`. |
|
|
| Embedded profiles and schemas | Exactly three profiles carry the intended backend/model/timeouts/reasoning/tier settings; tests reject unsafe/incidental profile settings and retired runtime strings. Embedded schema bytes are independently copied. | Matches the scoped identity and immutability contracts. Output-schema field meaning remains Stage 12. |
|
|
| Inspection and execution lifecycle | Dependency-neutral requests carry context and exact prompt/profile identity. Execution runs validation callbacks before provider work and distinguishes completed validation rejection from operational failure without a result. | Coherent and retained as `RET-022`; adapter realization remains Stage 11. |
|
|
| Safe errors, debug, and result isolation | All stable error categories are covered; safe errors preserve causes while omitting cause text; messages and invalid UTF-8 diagnostics are bounded; debug is opt-in; returned output/debug bytes and maps are copied. | Sufficient at the neutral contract boundary. Dependency diagnostics remain Stage 11. |
|
|
| Complexity and realistic input size | The highest scoped production path is stanza YAML marshaling, with two shallow loops over five fixed categories and the report's service-sized module list. Asset lookup and neutral result construction are constant or small linear work. | No Stage 10 performance finding; explicit grouping remains cheaper to reason about than generic reflection. |
|
|
| Test ownership | Prompt-input tests own construction, validation, grouping, ordering, category placement, and YAML round trips; prompt-assets tests own embedded inventories/identities and real inspection; prompt-exec tests own lifecycle, category, error, debug, bound, and copy semantics. | Mostly behavioral and nonduplicated. Missing provider-boundary warning/path/locality and serialization/load cases are attached to `AUD-029` through `AUD-033`. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph architecture, symbol and augmented-code search, snippets,
|
|
inbound/outbound traces, data-flow traces, and complexity queries for
|
|
`Build`, `Validate`, `MarshalYAML`, `LoadYAML`, stanza grouping/normalization,
|
|
prompt asset lookup, execution request/result construction, and application
|
|
preparation callers. Bounded text inspection covered embedded Markdown,
|
|
YAML, and JSON assets and their focused assertions.
|
|
- Compared implementation and tests with `docs/internal/prompt-input.md`,
|
|
`docs/internal/generatedtext.md`, `docs/internal/module.md`,
|
|
`docs/integrations/promptkit.md`, and the architecture, documentation, and
|
|
testing policies. Every embedded prompt/profile/schema reference was
|
|
inventoried and cross-checked against the report registry.
|
|
- Ran a temporary focused malformed-package probe proving acceptance of
|
|
missing metadata, orphan stanza values, unknown fields, and a trailing YAML
|
|
document, then removed the probe without retaining test or product changes.
|
|
- Ran
|
|
`go test -coverprofile=/tmp/weatherreporter-stage10-cover.out ./internal/promptinput ./internal/promptassets ./internal/promptexec`;
|
|
all focused packages passed. Coverage guided branch inspection but was not
|
|
itself treated as a finding. Ran `go test ./...`, `go vet ./...`, and
|
|
`git diff --check`; all passed.
|
|
- Findings: `AUD-029`, `AUD-030`, `AUD-031`, `AUD-032`, and `AUD-033`.
|
|
- Retained decisions: `RET-021` and `RET-022`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 11: Audit Promptkit Adaptation And Secure Prompt Debugging
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all production and focused-test code in
|
|
`internal/adapters/promptkit` and `internal/promptdebug`; Promptkit integration
|
|
and adapter internals; relevant operations, architecture, documentation, and
|
|
testing rules; Promptkit v0.5.0's local public concurrency/prepared-execution
|
|
contract; and immediate app/CLI consumers needed to trace credential checks,
|
|
debug authorization, callback failure, and result projection.
|
|
- Exclusions: Generated-text schema semantics remain Stage 12; application
|
|
prompt inspection/preparation order remains Stage 14; comparison execution
|
|
scheduling remains Stage 18; suite-wide test hygiene and cross-cutting
|
|
cleanup remain Stages 19-23; documentation-wide coherence remains Stage 24;
|
|
and broader adversarial diagnostics remain Stage 25.
|
|
|
|
#### Adapter And Debug-Security Accounting
|
|
|
|
| Contract or risk | Adapter, filesystem, caller, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Dependency containment and configuration | Promptkit production imports occur only in its adapter. CLI-owned settings map to profile directory/file, local endpoint/capacity, and timeout; mutually exclusive sources and inactive local capacity fail construction. | Matches the architecture boundary. Profile resolution remains dependency-owned under `RET-023`. |
|
|
| Profile source precedence | Explicit in-memory definitions, configured file/directory, embedded fallback, and built-ins provide complete definitions in the documented order. Absence falls through; a malformed matching configured definition fails. | Coherent with offline focused tests and retained as part of `RET-023`. |
|
|
| Exact inspection and credential preflight | Prompt inspection returns exact ID/version/hash, one input, and output contract. Profile inspection returns only logical ID, backend/model, direct-key requirement, and environment-variable name; app checks direct keys and nonblank environment values before collection. | Safe values and missing-credential categories match. Environment names are identifiers, not secret values. |
|
|
| Prepared inline execution | The adapter copies the YAML bytes into one inline artifact, prepares once, invokes the callback before generation, and runs the same opaque one-use handle. Callback failure prevents the provider call and deferred discard clears unused private state. | Matches the integration contract with exact-provenance and callback tests. |
|
|
| Cancellation and concurrency | Preparation and execution receive caller context independently; canceled/deadline causes map before generic generation errors. Promptkit documents `Engine` as concurrency-safe with capacity owned per engine, and comparison uses one adapter concurrently without wrapper state. | Coherent; race-enabled adapter/app tests pass. No redundant application-wide limit or adapter mutex is warranted. |
|
|
| Error classification and normal redaction | Promptkit configuration, prompt/profile, credential, artifact/render, capacity, generation, validation, request, cancellation, and deadline errors map to bounded repository categories. Underlying causes remain discoverable but their content does not enter `Error()`, summaries, or logs. | Sufficient at this boundary; focused marker and category tests protect normal output. |
|
|
| Debug authorization and isolation | Empty root disables all filesystem access; an enabled root must be absolute and not `/`. Report/date/run segments reject separators and traversal, comparison derives distinct deterministic run segments, and content-rich values appear only when capture is requested. | Matches explicit opt-in and per-run isolation. Separate ownership retained as `RET-024`. |
|
|
| Debug credential projection | User information, fragments, known secret query keys, and known nested parameter keys are removed or redacted before preparation JSON is written. | The denylist misses common aliases, signed/path credentials, and arbitrary provider vocabulary; `AUD-034`. |
|
|
| Directory and symlink containment | Static symlinks in the root or any visible descendant component are rejected; directory modes are tightened to `0700`, and unsafe references fail before a file write. | Path checks are separated from later name-based creation/replacement and can be raced; `AUD-035`. |
|
|
| File collision, modes, and partial writes | JSON is marshaled before creation; same-name artifacts use a mode-`0600` temporary in the final directory and atomic rename; non-regular existing targets are rejected; distinct concurrent run directories and shared missing ancestors are tested. | Coherent for non-adversarial paths. A failed requested callback stops generation, and a failed execution write stops publication. |
|
|
| Complexity and test ownership | Adapter mapping/classification is straight-line or small linear copying. Debug recursion follows bounded JSON-compatible parameter trees; directory traversal is linear in path components; realistic artifact sizes are dominated by intentionally captured prompt/output content. Adapter tests own dependency translation; debug tests own filesystem/security behavior; app tests own callback consequences. | No Stage 11 efficiency or duplication finding. Redaction and race gaps are attached to their narrow owners rather than inferred from coverage. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph architecture, symbol and augmented-code search, snippets,
|
|
inbound/data-flow traces, and hotspot inspection for adapter construction,
|
|
exact inspection, preparation/execution, classification, debug writes,
|
|
path validation, redaction, application callbacks, and CLI debug-root flow.
|
|
Text inspection was limited to scoped documents, tests, string-based
|
|
security policy, and the locally installed Promptkit v0.5.0 contract.
|
|
- Confirmed Promptkit's public engine concurrency guarantee, one-use prepared
|
|
handle state, independent preparation/execution contexts, credential-redacted
|
|
details, and upstream concurrent engine contract test without contacting a
|
|
provider.
|
|
- Ran a temporary deterministic redaction probe using only synthetic markers;
|
|
it proved that common endpoint-query and nested-parameter aliases survive
|
|
serialization, then was removed without retaining source/test changes.
|
|
- Ran
|
|
`go test -race -coverprofile=/tmp/weatherreporter-stage11-cover.out ./internal/adapters/promptkit ./internal/promptdebug ./internal/app`;
|
|
all focused packages passed with 90.0%, 73.6%, and 76.0% statement coverage
|
|
respectively. Coverage guided branch inspection but was not itself a
|
|
finding. Ran `go test ./...`, `go vet ./...`, and `git diff --check`; all
|
|
passed.
|
|
- Findings: `AUD-034` and `AUD-035`.
|
|
- Retained decisions: `RET-023` and `RET-024`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 12: Audit Generated-Text Validation And Catalog Contracts
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all production and focused validation/catalog tests in
|
|
`internal/generatedtext` except `render_context.go` and its focused tests;
|
|
all four embedded generated-text schemas and their `promptassets` owner;
|
|
schema assertions currently placed in `internal/reporttemplate`; the
|
|
generated-text and Promptkit integration contracts; and the immediate
|
|
Promptkit/application path needed to establish schema validation, raw-output
|
|
dispatch, and error propagation.
|
|
- Exclusions: Render-context assembly, template/partial semantics, and Markdown
|
|
rendering remain Stage 13; application preparation and preflight ordering
|
|
remain Stage 14; publication consequences remain Stages 15-18; suite-wide
|
|
durability and refactoring remain Stages 19-23 except for duplication
|
|
explicitly assigned to this stage; documentation-wide coherence remains
|
|
Stage 24; and repository-wide adversarial checks remain Stage 25.
|
|
|
|
#### Generated-Text Contract Accounting
|
|
|
|
| Contract or risk | Schema, decoder, catalog, caller, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Raw output to typed value | Promptkit validates raw provider JSON against the report definition's embedded schema and reports passed/failed status. After a pass, app dispatches the same raw bytes through the prepared handler; the report validator strictly decodes shape, trims prose, removes blank day-style paragraphs, applies semantic required-field rules, and returns a typed value plus canonical JSON before context construction. | The path and ownership are explicit. Retain both validation layers under `RET-026`; their exact-name disagreement is `AUD-037`. |
|
|
| Required and empty fields | Every schema requires summary, forecast discussion, and precipitation timing and rejects additional properties. Go requires nonblank trimmed summary/discussion; day-style discussion retains one or more nonblank paragraphs; precipitation timing must be present and string-valued but may normalize to empty. Null and wrong structural types fail. | Coherent aside from case-insensitive aliases. The schema intentionally permits lexical blanks that repository semantic validation rejects. |
|
|
| JSON document strictness | Typed decoding rejects malformed JSON, unknown canonical additions, wrong field types, and multiple top-level values. It accepts surrounding whitespace. Duplicate names follow Go/parsed-JSON last-value semantics because no stronger canonical rule exists. | Sufficient for documented document shape, with exact-key enforcement missing in `AUD-037`. Duplicate policy is routed to Stage 25 remediation validation rather than declared retroactively. |
|
|
| Array and size behavior | Day-style arrays require one schema item and one nonblank normalized paragraph but have no item-count or aggregate bound. No schema or Go string/total-output bound exists, and decoder unknown-field errors quote arbitrary names. | Unbounded accepted prose and diagnostics are `AUD-038`. |
|
|
| Report-specific types and shared mechanics | Daily, Today, and Tomorrow are distinct types and validators over one private field projection and validation helper; Hourly deliberately uses a single discussion string. Typed context builders can therefore distinguish reports even when day-style fields match. | Appropriate separation retained as `RET-025`; tests duplicate rather than merely protect that choice in `AUD-039`. |
|
|
| Catalog compatibility | Four entries bind schema/template pairs to validators and builders; current registry iteration resolves all entries, schema/template lookup returns nonempty assets, and unknown IDs or mismatched known pairs fail. | Entries omit report identity, so a complete known pair can be assigned to the wrong report; `AUD-036`. |
|
|
| Asset ownership and immutability | `promptassets` embeds and returns independent copies of all four schema byte slices. `generatedtext.Handler.Schema` adds report context; `reporttemplate` production code owns templates and does not own schemas. | Production ownership and copy isolation are coherent. Schema tests placed in `reporttemplate` duplicate the asset owner's assertions; `AUD-039`. |
|
|
| Error safety | Semantic required-field errors and catalog errors contain stable report/schema/template identifiers. JSON library errors are wrapped with report kind and preserve causes. | Arbitrary unknown-field names are emitted without a diagnostic bound and can be wrapped into normal application errors; part of `AUD-038`. No raw value is otherwise deliberately included. |
|
|
| Complexity and realistic cost | Scoped validators are straight-line except for one pass over discussion paragraphs; catalog lookup scans four fixed entries. Normal work is linear in provider output and paragraph count. | Algorithms are proportionate, but absent input/array limits make their allocation and normalization cost attacker/provider-controlled; `AUD-038`. |
|
|
| Test ownership and durability | Focused tests cover all four dispatches, canonical normalization, malformed/type/multiple-value cases, required semantics, retired fields, catalog lookup, schema inventory/identity, and independent schema bytes. | Exact report pairing, schema/Go agreement, and bounds are unprotected. Day-style semantics and schema shapes are redundantly asserted across packages; `AUD-036` through `AUD-039`. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph architecture, symbol and augmented-code search, inbound/data-flow
|
|
tracing, snippets, and complexity queries for catalog lookup, schema access,
|
|
typed decoding/normalization, report validators, application execution, and
|
|
their production/test callers. Bounded text inspection covered the four JSON
|
|
schemas, scoped tests, generated-text/Promptkit documentation, and policy.
|
|
- Compared each schema's exact required fields and property shapes with Daily,
|
|
Today, Tomorrow, and Hourly structs and semantic validators. Traced schema
|
|
validation status through Promptkit adaptation into app's handler dispatch
|
|
and verified schema byte-copy isolation and fixed catalog scan cost.
|
|
- Ran temporary offline probes showing that a Daily definition accepts the
|
|
known Hourly pair, Go accepts case-varied summary/discussion keys while real
|
|
embedded-schema validation rejects them, both layers accept a two-MiB valid
|
|
summary, and a 64-KiB synthetic unknown key appears in full in the decoder
|
|
error. The probes were removed without retaining source or test changes.
|
|
- Ran
|
|
`go test -coverprofile=/tmp/weatherreporter-stage12-cover.out ./internal/generatedtext ./internal/promptassets ./internal/reporttemplate`;
|
|
all packages passed with 81.3%, 73.7%, and 86.2% statement coverage
|
|
respectively. Coverage guided branch inspection but was not itself treated
|
|
as a finding. Ran `go test ./...`, `go vet ./...`, and `git diff --check`;
|
|
all passed.
|
|
- Findings: `AUD-036`, `AUD-037`, `AUD-038`, and `AUD-039`.
|
|
- Retained decisions: `RET-025` and `RET-026`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 13: Audit Render Contexts, Templates, And Markdown Rendering
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: `internal/generatedtext/render_context.go` and all focused
|
|
context tests; all production code, tests, four top-level templates, and four
|
|
partials in `internal/reporttemplate`; the generated-text and report-template
|
|
internal documents; the complete maintainer template guide; and the narrow
|
|
catalog/application caller path needed to establish report identity,
|
|
template pairing, and in-memory render behavior.
|
|
- Exclusions: Generated-text JSON decoding and schema semantics remain Stage
|
|
12 except where their validated strings enter Markdown; application
|
|
preparation, immutable copies, and execution ordering remain Stage 14;
|
|
publication and notification consequences remain Stages 15-18; portfolio-
|
|
wide test and complexity work remains Stages 19-23; documentation-wide
|
|
coherence remains Stage 24; and broader adversarial rendering remains Stage
|
|
25.
|
|
|
|
#### Context And Markdown Accounting
|
|
|
|
| Contract or risk | Context, template, partial, caller, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Report metadata and identity | Hourly builds location and rolling-period labels; day-style reports derive civil-date/day-name labels and preserve distinct titles. Timezones, generated times, and periods are validated before construction. | Builders do not validate the report ID or reconcile the separately decoded metadata stanza; `AUD-040`. Error branches for invalid time metadata also lack focused tests, to be covered with the identity remediation rather than as a second finding. |
|
|
| Module-to-context mapping | Snapshot stanzas are selected by canonical stanza name, JSON-normalized through `module.StanzaValue`, and projected into typed optional pointers. Common day-style extraction is followed by report-specific Daily, Today, or Tomorrow planning fields; Hourly has its own surface. Extraction errors name the failing module. | Coherent under valid snapshots and retained as `RET-027`. Snapshot schema-version permissiveness remains the upstream `AUD-022`; duplicate stanzas are rejected by `module.NewSnapshot`. |
|
|
| Deterministic daypart ordering | Typed row slices follow `DerivedFacts.DaypartSummaries`, try canonical and date-qualified keys, suppress already selected keys, then append remaining module keys lexically. Templates range only the ordered slices, not the source map. | Byte-stable and semantically protected for configured and remaining rows. Upstream key collisions remain `AUD-024`; no new ordering defect. |
|
|
| Generated prose slots | Summary and forecast discussion occupy explicit top-level prose positions. Precipitation prose renders only when deterministic precipitation windows exist, preventing that slot from creating a dry-period timing section by itself. | Slot selection is coherent, but raw Markdown lets any prose slot create unauthorized report structure; `AUD-041`. Unbounded prose remains `AUD-038`. |
|
|
| Deterministic sections | Alert/SPC, current conditions, hourly/daypart forecast, precipitation windows, and their numeric values come from typed module fields. Alert instructions/descriptions and unused planning/narrative modules do not silently enter current output. Enhanced-or-higher SPC policy is centralized in helpers. | Current field ownership is explicit. Dynamic module strings share the Markdown-escaping defect in `AUD-041`; Today's all-filtered daypart case is `AUD-042`. |
|
|
| Missing values and report variation | Optional module pointers are guarded. Hourly provides explicit current/hourly fallbacks; day-style reports provide daypart fallbacks; Today alone includes current conditions and deliberately suppresses conditionless rows. Daily/Tomorrow share one richer daypart partial. | Intentional variation is preserved, except Today's row filter can leave a bare heading; `AUD-042`. |
|
|
| Template and partial lookup | Four fixed IDs resolve independent embedded source strings. `Render` applies `missingkey=error`, parses the chosen top-level asset and every shared partial, and attributes lookup, top-level parse, partial read/parse, and execution failures. All partials being parsed for every report makes a broken shared asset fail closed. | Coherent and retained as `RET-028`. Schema ownership remains in `promptassets`; catalog report pairing remains `AUD-036` and misplaced schema tests remain `AUD-039`. |
|
|
| Escaping and formatting | Repository literals define headings, bullets, ordering, and spacing. Templates use `text/template`; dynamic strings are not recursively evaluated as template actions. | There is no Markdown/HTML structural escaping for provider or source-derived strings; `AUD-041`. |
|
|
| Output determinism | Embedded assets, typed slices, fixed partial order, and absence of current map iteration make repeated rendering of an unchanged context byte-identical. A temporary repeat-render probe confirmed identical output. | Sufficient. Context aliasing/immutability before rendering belongs to Stage 14. |
|
|
| Test review value | Tests cover real context construction for all reports, rich/omitted modules, module decoding errors, ordering, partial thresholds, conditional sections, fallback rows, open-ended precipitation, missing context, actionable unknown IDs, and semantically important output order. They use focused fragments rather than full-output goldens. | Good semantic review value under `RET-028`. Identity mismatch, Markdown structure, and all-filtered Today rows are missing and attach directly to `AUD-040` through `AUD-042`; cross-suite fixture cost remains Stages 20-21. |
|
|
| Complexity and cost | Builders are straight-line projections plus linear stanza/daypart passes; remaining daypart keys sort deterministically. Template helpers scan small alert/SPC slices, and rendering parses eight embedded assets per call. | Proportionate at current service-sized inputs. Output-size amplification remains `AUD-038`; cross-cutting render performance remains Stage 22. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph architecture, symbol and augmented-code search, snippets,
|
|
inbound traces, and complexity queries for all context builders, module
|
|
extraction, daypart ordering, template lookup/rendering, partial helpers, and
|
|
application callers. Text inspection covered every scoped Go file, template,
|
|
partial, focused test, and canonical template/internal document.
|
|
- Cross-checked every current template field against its typed context
|
|
producer, confirmed no template ranges over a map, inventoried optional
|
|
guards and fallbacks, compared the Today/Daily/Tomorrow variations, and
|
|
traced normalized snapshot values and validated generated prose into
|
|
repository-owned Markdown.
|
|
- Ran temporary offline probes showing that a Daily builder accepts outer
|
|
Hourly metadata alongside a Daily metadata stanza, a schema-valid generated
|
|
summary can inject an Alert Digest heading and fabricated warning bullet,
|
|
repeated rendering is byte-identical, and an all-filtered Today daypart slice
|
|
produces a bare heading. The probes were removed without retaining source or
|
|
test changes.
|
|
- Ran focused coverage for `./internal/generatedtext` and
|
|
`./internal/reporttemplate`; both passed with 81.3% and 86.2% statement
|
|
coverage respectively. Coverage identified untested metadata failure paths
|
|
but was not itself treated as a finding. Ran `go test ./...`, `go vet ./...`,
|
|
`go run ./cmd/weatherreporter --help`, and `git diff --check`; all passed.
|
|
- Findings: `AUD-040`, `AUD-041`, and `AUD-042`.
|
|
- Retained decisions: `RET-027` and `RET-028`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 14: Audit Application Preparation And Prompt Preflight
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all production code and focused tests in
|
|
`internal/app/prepared_report.go`, `prompt_inspection.go`,
|
|
`prompt_generate.go`, and `profile_execution.go`; the narrow ordinary,
|
|
batch, and comparison callers needed to establish collection order,
|
|
preparation count, cancellation, and partial-result consequences; and the
|
|
prepared-report and application-orchestration internal contracts.
|
|
- Exclusions: Single-report destination resolution and atomic publication
|
|
remain Stage 15; batch continuation and notification remain Stage 16;
|
|
comparison identity/publication and scheduling remain Stages 17-18;
|
|
portfolio-wide test consolidation and application refactoring remain Stages
|
|
20-23; documentation-wide reconciliation remains Stage 24; and broader
|
|
adversarial execution remains Stage 25.
|
|
|
|
#### Preparation And Execution Accounting
|
|
|
|
| Contract or risk | Ordering, copy, execution, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Pre-collection validation | Ordinary and comparison flows resolve report and destination before optional debug initialization, then inspect prompt/profile and credentials before `collectWeather`; batch inspects all needed prompt/profile pairs before its one collection. Exact prompt ID/version, one required YAML input, JSON Schema output path, selected profile ID, and environment credential availability fail before collection. | The generated-text template/catalog binding is absent from this preflight and is checked late; `AUD-043`. Output publication preflight remains with the assigned later stages. |
|
|
| Prompt/profile inspection | Inspection is side-effect-free, caches one effective batch profile, preserves explicit comparison order, and returns a successfully inspected comparison prefix on profile failure. Unclassified dependency failures become content-safe configuration errors. | Coherent and retained as `RET-030`, except that safe provenance can be incomplete and is not later reconciled; `AUD-044`. |
|
|
| Post-collection preparation | Preparation requires a bundle, builds facts once, builds one ordered module snapshot and metadata value, constructs and deterministically serializes the curated package, resolves a generated-text handler, and wraps every failure with its operation. No model or normal publication occurs on a preparation error. | The construction boundary is explicit. Catalog lookup order is `AUD-043`; missing serialization/no-executor regression remains `AUD-033`. |
|
|
| Immutable prepared state | Facts, snapshots, metadata, source warnings, resolved slice-bearing fields, and YAML bytes are copied before retention. Executor requests receive byte copies and rendering receives fresh typed clones. Focused tests mutate the source bundle and returned render values; concurrent comparison tests observe equal package bytes for every profile. | Demonstrably isolated for current types and retained as `RET-029`. The manual future option-shape risk is routed to Stage 23, not treated as a current alias defect. |
|
|
| Single-build reuse | Ordinary generation prepares once for its one selected profile. Comparison collects and prepares once, then passes the same immutable logical package through independent copies to every profile; it does not recompute facts or modules per provider. | Matches the prepared-report contract and avoids cross-profile derivation drift. |
|
|
| Executor request and cancellation | Execution receives the inspected prompt ID/version, selected profile ID, caller context, independent YAML bytes, and debug capture only when explicitly enabled. Classified provider errors retain their category; otherwise they become generation errors. The reusable executor/render core performs no durable writes. | Request shape and context propagation are coherent. Missing provenance checks are `AUD-044`; post-render publication cancellation and comparison aggregation remain Stages 15 and 18. |
|
|
| Debug callbacks | Preparation debug is written only through the explicit writer. A requested callback write failure is marked local and returned directly, preventing provider work under the executor contract; execution-debug failures stop before rendering/publication. Normal results retain only safe identities and an artifact path. | Matches Stage 11's secure-debug ownership. Callback invocation and identity are nevertheless trusted rather than verified; part of `AUD-044`. |
|
|
| Generated text and rendering | Only completed `passed` validation proceeds; `failed` becomes validation rejection and incomplete states become operational validation errors. The prepared handler independently validates raw JSON, clones render inputs, builds a typed context, and renders in memory. | Correct layer order under coherent provenance. Schema/mode and execution identity are not tied to the inspected contract; `AUD-044`. Existing schema, size, context, and Markdown findings remain `AUD-036` through `AUD-042`. |
|
|
| Errors and partial results | Preparation and execution wrappers preserve actionable operation names. Ordinary generation copies source warnings after preparation and records resolved profile identity, validation status, and debug path before returning execution errors. Comparison outcomes independently retain ordered safe execution status. | Coherent for reachable callers. Durable-path truthfulness and aggregation policy remain explicitly assigned to Stages 15-18. |
|
|
| Complexity and test ownership | The preparation path is a guarded straight-line pipeline over service-sized typed values; its dominant cost is deliberate deep copy and serialization. Execution is another guarded pipeline with no loop. Focused suites own inspection, mutation isolation, debug callbacks, validation states, rendering without publication, and concurrent shared-package reuse. | No Stage 14 performance finding. Missing invalid-catalog ordering, provenance mismatch, and serialization consequence tests attach directly to `AUD-043`, `AUD-044`, and `AUD-033`. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph symbol and augmented-code search, exact source snippets,
|
|
inbound/data-flow traces, and scoped complexity metadata for preparation,
|
|
inspection, ordinary generation, profile execution, collection, and
|
|
comparison reuse. Bounded text inspection covered the four scoped production
|
|
files, focused tests, task-specific internal documents, audit plan, and all
|
|
repository policies.
|
|
- Enumerated every pre-collection failure (report/output/debug setup, executor,
|
|
exact prompt shape, profile selection, and credentials), post-collection
|
|
preparation failure (bundle, facts, modules, package construction,
|
|
serialization, catalog, and immutable copies), and post-model boundary
|
|
(completed schema status, typed validation, context construction, rendering,
|
|
debug callback, cancellation handoff, and partial outcome).
|
|
- Ran temporary offline probes proving that prompt preflight accepts an unknown
|
|
generated-text template plus blank safe provenance, and that mutually
|
|
inconsistent callback/execution identities and schema-validation provenance
|
|
still render valid Daily-shaped output. The probes were removed without
|
|
retaining source or test changes and contacted no provider.
|
|
- Ran
|
|
`go test -race -coverprofile=/tmp/weatherreporter-stage14-cover.out ./internal/app`;
|
|
the focused package passed with 76.0% statement coverage. Coverage guided
|
|
missing-branch inspection but was not itself treated as a finding. Ran
|
|
`go test ./...`, `go vet ./...`, `go run ./cmd/weatherreporter --help`, and
|
|
`git diff --check`; all passed.
|
|
- Findings: `AUD-043` and `AUD-044`; application consequences also extend the
|
|
existing test finding `AUD-033`.
|
|
- Retained decisions: `RET-029` and `RET-030`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 15: Audit Single-Report Generation And Atomic Output
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: the complete single-report path in `internal/app/app.go`,
|
|
single-report destination logic in `internal/app/output.go`, publication in
|
|
`prompt_generate.go`, all production code and tests in `internal/fileutil`,
|
|
generation/output tests, the application-orchestration and operations
|
|
contracts, and the narrow CLI result projection needed to trace committed
|
|
paths and errors without re-auditing parsing.
|
|
- Exclusions: Batch output planning, continuation, and notification remain
|
|
Stage 16; comparison destination identity and transactional bundle
|
|
publication remain Stage 17; comparison cancellation remains Stage 18;
|
|
suite-wide durability and refactoring remain Stages 19-23; documentation-wide
|
|
coherence remains Stage 24; and adversarial pathname races remain Stage 25.
|
|
|
|
#### Single-Report Publication Accounting
|
|
|
|
| Contract or risk | Destination, filesystem, result, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Stage order and destination precedence | Report resolution precedes output resolution; an explicit file wins over the configured directory, which wins over the captured absolute working directory. Existing configured non-directories fail before prompt inspection or collection, while missing directories are deferred until publication. | Coherent for ordinary paths. Catalog preflight remains `AUD-043`; CLI executor-before-date ordering remains `AUD-009`. |
|
|
| Path normalization and target type | Relative paths are anchored to the captured working directory, absolute paths are cleaned, roots and directories are rejected, and result paths are absolute. Final validation otherwise accepts every non-directory object and follows symlinks. | Destructive special-file and ambiguous symlink behavior is `AUD-045`. |
|
|
| Atomic write and linearization | The writer creates a temporary file in the destination directory, writes and closes complete bytes, then uses same-directory `os.Rename`. Successful rename is the single visibility/commit point. Existing regular output is untouched on all earlier application failures and writer failures before rename. | Mechanism retained as `RET-031`. Temporary naming can reject a valid final component in `AUD-046`; crash durability is not claimed. |
|
|
| File and directory modes | `os.CreateTemp` creates new report files as mode `0600`; `MkdirAll` requests `0755` for missing operator-selected parents and leaves existing modes unchanged. Replacement takes the temporary file's restrictive mode. | Safe for report content. Focused tests do not assert the final mode, but the standard-library primitive directly defines it and no conflicting user contract exists; no coverage-only finding. |
|
|
| Temporary and failure cleanup | A deferred remove covers write, close, and rename failures after temporary creation; the rename-error test proves no matching temp remains. Parent directories created before a later create/write failure are not rolled back. | Temp cleanup is coherent. The valid-long-name failure and directory residue are `AUD-046`; broad directory rollback would need concurrency-safe ownership. |
|
|
| Cancellation | Cancellation/deadline before publication entry returns the stable category, leaves `OutputPath` empty, preserves previous bytes, and skips notification; focused app tests cover both. | The check precedes all temporary I/O rather than the rename boundary, so later-observable cancellation can still commit; `AUD-047`. |
|
|
| Pre-publication preservation | Prompt inspection, collection, generation, schema/typed validation, render, debug-write, destination, cancellation, and atomic-write failures occur before `OutputPath` is set. Tests preserve an existing file across generation, render, cancellation, deadline, and directory-collision failures; fileutil tests cover rename error and temp cleanup. | Sufficient for regular files aside from the final cancellation gap. Stage 14 owns preparation/execution provenance and Stage 16 owns multi-output consequences. |
|
|
| Post-publication semantics | Immediately after rename, the result records the absolute path. Optional notification is then built from and sent with that exact file. Notification failure returns a failed action while retaining the committed path and Markdown; disabled notification returns success without a notification object. | Coherent and retained as `RET-032`. Single-report identity rendering gaps remain `AUD-005`; Distributor adapter/batch policy remain Stage 16. |
|
|
| Errors and CLI result truth | Cancellation is wrapped with report/run/operation context and preserves its category. Filesystem errors carry the exact affected directory, temp operation, or final path; notification errors carry report/run/output context. CLI projection copies active safe fields, committed path, notification, and returned error, marking any result-bearing error failed. | Actionable and truthful at the reviewed boundary. Assembled non-comparison CLI status/exit tests remain the existing `AUD-010`. |
|
|
| Complexity and test ownership | Resolution and publication are short guarded pipelines; file publication is constant-pass work over rendered bytes and has no service-sized loop beyond the write itself. App tests own workflow preservation/result ordering; fileutil tests own atomic replacement and temporary cleanup. | No Stage 15 complexity finding. Missing special-target, long-name, and final cancellation cases attach directly to `AUD-045` through `AUD-047`. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph symbol and augmented-code search, exact snippets, inbound/outbound
|
|
traces, and scoped complexity/coverage data for `GenerateDetailed`, output
|
|
resolution and validation, publication/cancellation, atomic writing,
|
|
notification, and CLI result projection. Bounded text inspection covered all
|
|
scoped production/tests, task-specific documents, policies, and the exact
|
|
Stage 15 boundary.
|
|
- Traced every consequential result through the CLI projection: pre-result
|
|
resolution failure, resolved but unpublished destination/preflight,
|
|
collection/generation/render/cancellation/write failure, successful commit,
|
|
and committed output followed by notification failure. CLI parsing and
|
|
aggregate batch behavior were not re-audited.
|
|
- Ran temporary offline filesystem probes showing that complete generation
|
|
replaces a FIFO with a regular Markdown file and that a filesystem-valid
|
|
248-byte basename fails after model work because the derived temp component
|
|
is too long, leaves its new parent directory, and remains directly writable.
|
|
The probes were removed without retaining source/test changes.
|
|
- Ran focused coverage for `./internal/app` and `./internal/fileutil`; both
|
|
passed with 76.0% and 68.4% statement coverage respectively. Coverage guided
|
|
failure-branch inspection but was not itself treated as a finding. Ran
|
|
`go test ./...`, `go vet ./...`, `go run ./cmd/weatherreporter --help`, and
|
|
`git diff --check`; all passed.
|
|
- Findings: `AUD-045`, `AUD-046`, and `AUD-047`.
|
|
- Retained decisions: `RET-031` and `RET-032`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 16: Audit Batch Orchestration And Distributor Notification
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: batch inspection and planning in `internal/app/app.go` and
|
|
`batch_plan.go`, destination preflight, the complete sequential report loop,
|
|
batch request construction and notification in `batch_notification.go`,
|
|
report/notification result and CLI projection, all focused batch tests, the
|
|
production and test code in `internal/adapters/distributor`, the pinned
|
|
Distributor upload and bundle boundaries, and all canonical app,
|
|
Distributor, operations, and policy documents assigned by the stage.
|
|
- Exclusions: Comparison publication and execution remain Stages 17-18;
|
|
repository-wide test portfolio and complexity work remain Stages 19-23;
|
|
documentation-wide reconciliation remains Stage 24; adversarial pathname
|
|
races remain Stage 25; and finding remediation remains outside this audit
|
|
stage.
|
|
|
|
#### Batch Outcome And Delivery Accounting
|
|
|
|
| Contract or risk | Planning, publication, delivery, result, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Pre-collection validation | The application resolves the selected batch, output directory, debug writer, and one representative of every possible prompt/profile identity before collection. The Daily candidate uses the first possible post-Tomorrow date; later Daily reports share its definition and execution profile. | Coherent. Generated-text catalog timing remains `AUD-043`; unsupported Distributor endpoint forms are not included in this gate and are `AUD-048`. |
|
|
| Collect-once and data-dependent plan | One collected result is retained for every item. Morning plans Today then Tomorrow; Evening plans Tomorrow; both append only complete post-Tomorrow local civil days in ascending order. DST transition days are checked by instant coverage rather than assuming 24 rows. | Coherent and retained as `RET-033`. Daypart clock semantics remain the separate `AUD-002`. |
|
|
| Complete output preflight | After collection determines membership, every final absolute report path is resolved and validated before any executor call. A later invalid destination prevents all report replacement; an existing earlier file remains untouched. | Coherent for the fixed distinct registry outputs. Each final path retains the single-file limitations in `AUD-045` through `AUD-047`. |
|
|
| Sequential execution and partial success | Reports execute one at a time in plan order through the shared generation/publication core. Per-report notification is suppressed. An independent item failure is recorded and later items continue; successful files and their output paths remain. Counters match appended report statuses. | Coherent and retained as `RET-034`. Shared cancellation is incorrectly treated as repeatable independent failure in `AUD-051`. |
|
|
| Notification gate and request membership | Disabled integration yields no notification object. Any report failure yields one skipped result and no notifier call. All-success execution builds one request after all sources exist, matches results to unique planned run IDs, rejects duplicate bundle paths, and maps only each report's committed output path in report/template order. | Coherent and retained as `RET-035`. Dynamic single-report identity emptiness remains `AUD-005`; active endpoint admission remains `AUD-048`. |
|
|
| Exactly-once ownership and retries | The application invokes the batch notifier once; the adapter constructs one client, calls `UploadFiles` once at its boundary, then performs status lookup/polling. The pinned dependency alone owns replay-safe retries and reuses the supplied idempotency key. Status failures retain accepted-upload diagnostics; terminal run failure fails notification. | Ownership is coherent. Local tests do not traverse the production HTTP path (`AUD-052`), and remote response safety/bounds are `AUD-049` and `AUD-050`. |
|
|
| Report versus notification accounting | Report totals count only report items. A skipped notification follows a report failure; a failed notification leaves every report succeeded and every committed path intact but makes `RunBatch`, the CLI summary, and exit status fail with notification-specific wording. Accepted-but-unconfirmed status remains separate diagnostic state. | Coherent and retained as `RET-035`. Normal remote error text is not safe merely because it is notification-specific (`AUD-049`). |
|
|
| Cancellation and timeout | Context reaches inspection, collection, every prompt execution/publication, Distributor upload, retry delay, status request, and polling timer. Adapter timeout returns the latest accepted/run status plus a diagnostic when confirmation does not finish. | Adapter propagation is coherent. The batch loop continues after shared cancellation and loses cause identity in aggregate/CLI errors (`AUD-051`); final report rename timing remains `AUD-047`. |
|
|
| Response lifecycle and resource safety | The pinned HTTP client closes upload and status bodies and the adapter copies returned JSON so results do not alias dependency buffers. Conflict type identity and exact bearer-token replacement survive local wrapping. | Body closure and typed conflict handling are sound. Remote body text reaches normal output (`AUD-049`) and all response classes are unbounded (`AUD-050`). |
|
|
| Test ownership and cost | App tests own preflight, ordering, partial files, all-output-before-notify, one notifier call, counters, and notification-failure exit semantics. Adapter fakes cover mapping, polling, conflicts, and token replacement without importing app policy. Planning is linear over the forecast horizon and report/file mappings. | App ownership is appropriate. Adapter tests substitute below the production HTTP translation and need a narrow realistic local boundary under `AUD-052`; broader duplication/cost review remains Stages 20-22. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph symbol and augmented-code search, exact snippets, and caller/callee
|
|
traces for batch execution, planning, destination preparation, notification
|
|
request construction, Distributor upload/status translation, and CLI result
|
|
projection. Bounded text inspection covered every scoped production/test
|
|
file, all assigned canonical documents, policies, and the exact Stage 16
|
|
boundary.
|
|
- Traced every consequential batch outcome: setup/preflight failure before a
|
|
result; independent report failure before or after a peer publication; full
|
|
report success with notification disabled, skipped, accepted/unconfirmed,
|
|
terminally succeeded, or failed; request-construction failure; upload
|
|
conflict; status error; timeout; and shared cancellation. Cross-checked
|
|
report paths, counters, notification state, summary wording, and exit status.
|
|
- Inspected the pinned Distributor `v0.5.0` client and bundle contracts to
|
|
distinguish repository wiring from dependency-owned retries, idempotency,
|
|
archive construction, endpoint grammar, body closure, and response parsing.
|
|
This showed the production-only endpoint mismatch and unbounded/raw response
|
|
paths without treating upstream unit tests as local adapter coverage.
|
|
- Ran focused race/coverage checks for `./internal/app` and
|
|
`./internal/adapters/distributor`; both passed with 76.0% and 70.5% statement
|
|
coverage respectively. Coverage guided missing cancellation and production
|
|
boundary inspection but was not itself treated as a finding. Ran
|
|
`go test ./...`, `go vet ./...`, `go run ./cmd/weatherreporter --help`, and
|
|
`git diff --check`; all passed.
|
|
- Findings: `AUD-048` through `AUD-052`.
|
|
- Retained decisions: `RET-033` through `RET-035`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 17: Audit Comparison Contracts And Transactional Publication
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all production code and focused tests in
|
|
`internal/comparison`, the durable comparison bundle contract, comparison
|
|
publication internals, and the narrow architecture/operations rules needed
|
|
to establish replacement authorization and recovery semantics.
|
|
- Exclusions: Application preparation, concurrent profile execution, partial
|
|
result construction, and CLI integration remain Stage 18; repository-wide
|
|
test, efficiency, and refactoring review remain Stages 19-23;
|
|
documentation-wide reconciliation remains Stage 24; adversarial pathname and
|
|
crash robustness remain Stage 25; and remediation remains outside this audit
|
|
stage.
|
|
|
|
#### Durable Bundle And Filesystem Transaction Accounting
|
|
|
|
| Contract or risk | Model, destination, mutation, recovery, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Logical identity and names | Profile IDs require at least two exact distinct nonblank values. Comparison IDs derive from report run IDs; directory names derive from report output names; profile slugs retain the documented ASCII set and byte bound; ordinal width expands at 100 and higher. | Pure helpers are deterministic and focused tests cover boundaries. Manifest validation does not connect successful paths back to `ReportFilename`; `AUD-054`. |
|
|
| Manifest schema and encoding | One current version validates nonzero ordered timestamps, valid period, identity/hashes, at least two contiguous distinct profile results, exact counters, success/failure shapes, bounded UTF-8 messages, and unique safe report basenames. Encoding is two-space indented in declaration order with one newline. | Core shape is coherent and retained as `RET-036`. Recognition accepts duplicate/case-variant JSON fields (`AUD-053`) and arbitrary safe report names (`AUD-054`). |
|
|
| Logical payload agreement | `LogicalBundle.Validate` hashes the exact data-package bytes, requires one report for each and only each successful manifest result, and reconciles report order, positions, and paths before any filesystem mutation. Failed results carry no Markdown. | Coherent aside from the shared canonical-name gap in `AUD-054`. Report-content hashes are deliberately not part of the current schema. |
|
|
| Recognition and compatibility | Recognition requires a real directory containing exactly the declared manifest, data package, and successful reports as regular non-symlink files; rejects unknown fields, trailing JSON, unsupported versions, unsafe paths, extra/missing entries, and a data digest mismatch. | Exact file/type/digest protection is strong. Decoder ambiguity is `AUD-053`; full report reads are routed to Stage 22. |
|
|
| Read-only destination authorization | Planning requires clean absolute working/target paths, rejects roots, the exact working directory, final symlinks, non-directories, unreadable states, and nonempty content without explicit recognized replacement; missing-parent inspection creates nothing. Empty targets remain usable without `--replace`. | Coherent and retained as `RET-037`. Existing parent symlinks and adversarial path changes are routed to Stage 25. |
|
|
| Staging mutations | After bundle validation and replanning, publication creates missing parents, a mode-`0700` sibling staging directory, and mode-`0600` data/report/manifest files with exclusive creation; manifest is written last and deferred cleanup removes uncommitted staging. | Restrictive complete staging is coherent. Derived staging/backup components reject valid long targets and can leave new parents; `AUD-057`. Mid-write failure seams are simple and remain pre-target. |
|
|
| Initial commit | Immediately before installing an absent target, publication replans, exposes a deterministic final test hook, checks context, and uses one sibling rename as the visibility linearization point. Rename failure retains no target and deferred staging cleanup runs. | Coherent for ordinary filesystem state and retained in `RET-038`. Syscall races remain Stage 25. |
|
|
| Replacement authorization and rollback | Replacement moves the current empty or recognized target to a unique sibling, re-runs destination/recognition policy against the moved entry, and restores it if authorization or staging installation fails. A reappeared target is not overwritten during restoration; the error retains the backup path. | Strong ordinary-state defense, retained as `RET-037` and `RET-038`. Cancellation after the first rename is not rechecked before install; `AUD-055`. |
|
|
| Committed state and backup cleanup | Once staging is renamed to target, the new bundle remains authoritative. Successful backup removal returns clean commit; removal failure returns `Committed: true`, a typed error, and a sibling path rather than rolling back the new bundle. | Correct commit truth is retained as `RET-038`. Recursive cleanup may already have damaged or removed the old bundle, so the reported retained-backup state is overstated in `AUD-056`. |
|
|
| Safe and recovery errors | Destination kinds and `ErrUnrecognizedBundle` are inspectable; pre-commit restoration errors join the triggering cause with target/backup recovery context; post-commit cleanup errors unwrap the filesystem cause. Manifest failures remain bounded through `SafeError`. | Error identity is actionable. `AUD-056` requires surviving cleanup state to be classified truthfully; Stage 18 owns safe CLI projection of these package errors. |
|
|
| Test ownership and transaction cost | Model tests own naming, invariants, encoding, hashing, and logical reconciliation. Real `t.TempDir` tests own layout, modes, recognition, destination states, commit, restore, reauthorization, retained paths, and cancellation; injected rename/remove operations deterministically reach recovery states. Work is linear in profiles and artifact bytes. | Good package-boundary ownership. Missing strict decoding, canonical-name, post-backup cancellation, partial-cleanup, and long-name cases attach directly to `AUD-053` through `AUD-057`; broader cost and suite maintenance remain Stages 20-22. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used the codebase graph architecture view, symbol search, exact snippets, and
|
|
caller/callee metadata for manifest validation/encoding, identity helpers,
|
|
destination planning, recognition, publication, restoration, and cleanup.
|
|
Bounded text inspection covered all four scoped Go files, every focused test,
|
|
the exact Stage 17 boundary, and all assigned canonical documents/policies.
|
|
- Traced every filesystem mutation from parent creation through staging files,
|
|
absent-target commit, old-target backup, moved-entry reauthorization,
|
|
installation, restoration, staging cleanup, and post-commit backup cleanup.
|
|
Cross-checked which state authorizes each mutation and which path/result/error
|
|
remains after every failure branch.
|
|
- Ran temporary deterministic package probes confirming acceptance of ambiguous
|
|
manifest fields and a noncanonical safe report path, committed replacement
|
|
after cancellation immediately following backup rename, an advertised backup
|
|
made unrecognizable by partial cleanup, and late staging failure plus parent
|
|
residue for a valid 240-byte target basename. The probe file was removed and
|
|
no production/test source change was retained.
|
|
- Ran the focused comparison race/coverage command with its profile at
|
|
`/tmp/weatherreporter-stage17-comparison-cover.out`; it passed with 83.5%
|
|
statement coverage. Coverage guided inspection of staging and transaction
|
|
failure branches but was not itself treated as a finding. Ran
|
|
`go test ./...`, `go vet ./...`,
|
|
`go run ./cmd/weatherreporter --help`, and `git diff --check`; all passed.
|
|
- Findings: `AUD-053` through `AUD-057`.
|
|
- Retained decisions: `RET-036` through `RET-038`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 18: Audit Comparison Execution And CLI Integration
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: the complete comparison orchestration and result projection
|
|
in `internal/app/comparison.go`, concurrent profile execution in
|
|
`comparison_execution.go`, prompt/profile inspection and prepared-report
|
|
seams needed to establish preflight and input identity, all focused
|
|
comparison application tests, comparison-specific CLI parsing, wiring,
|
|
summaries, output, and tests, and the canonical comparison-execution,
|
|
application-orchestration, CLI, operations, and bundle contracts.
|
|
- Exclusions: Repository-wide hermeticity, portfolio coverage, maintenance,
|
|
efficiency, and refactoring remain Stages 19-23; documentation-wide
|
|
reconciliation remains Stage 24; adversarial runtime robustness remains
|
|
Stage 25; and finding triage/remediation remain Stages 26-27. Stage 17 owns
|
|
bundle-model and filesystem-transaction defects; Stage 14 owns prepared
|
|
execution provenance.
|
|
|
|
#### Comparison Execution And Result Accounting
|
|
|
|
| Contract or risk | Execution, publication, CLI, and test evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Ordered explicit selection and sequential preflight | CLI parsing preserves repeated profile order and validates exact distinct IDs before config/executor work. Application inspection resolves the prompt once, then profiles sequentially in supplied order, stopping at the first failure before collection. Resolved prompt identity survives later preflight failures in the partial result. | Coherent. Generated-text catalog timing and provenance reconciliation remain `AUD-043` and `AUD-044`. |
|
|
| Collect/prepare once and identical immutable input | After destination, debug, prompt, profile, and credential preflight, comparison calls weather collection and `prepareReport` once. Every execution receives an independent byte copy of the same serialized YAML plus cloned prepared values, so provider mutation cannot affect peers. Focused concurrent tests compare the captured package bytes. | Coherent and retained as `RET-039`. |
|
|
| One executor, concurrency, and backend capacity | CLI constructs one Promptkit adapter and passes it into one comparison request. Application starts one goroutine per selected profile; the backend owns configured provider capacity rather than an application semaphore. Production execution creates a private prepared handle per call. | Ownership matches the contract. Extreme profile-count cost is routed to Stage 22, and the interface comment gap to Stage 24. |
|
|
| Goroutine lifecycle and shared state | Each goroutine captures its own loop index/profile, writes only that preallocated slice element, and decrements one wait group. The caller waits for every started goroutine before reading outcomes or returning. Immutable prepared state and concurrency-safe executor/debug boundaries are shared; the scoped race suites pass. | Joined and race-clean for covered production boundaries; retained as `RET-039`. |
|
|
| Profile isolation and cancellation | Generation, validation, render, and debug failures stay in their own safe bounded outcome while peers continue. Context reaches every execution, prevents later starts when observed, joins started work, and prevents publication. The post-join sweep nevertheless overwrites an independent completed failure when any peer makes cancellation observable. | Mixed failure identity is incorrect in `AUD-058`; all-canceled joining and failure-without-cancellation have focused tests. |
|
|
| Deterministic result and debug identity | Results are initialized and stored by caller position, copied to the application result in that order, encoded into the manifest in that order, and projected by CLI without sorting. Filenames use position/count/profile identity; debug run IDs include comparison identity, ordinal width, and profile slug. Inverse completion and distinct-debug tests cover both properties. | Completion order cannot change durable or CLI order; retained as `RET-039`. Canonical filename validation remains `AUD-054`. |
|
|
| Partial and all-failed publication | After all noncanceled work joins, one logical bundle contains the exact shared package and only successful Markdown reports. Independent failures do not suppress publication; both partial-success and all-failed manifests commit, then the application returns a comparison aggregate error. Paths become visible only after commit. | Coherent and retained as `RET-040`. Execution provenance remains `AUD-044`. |
|
|
| Publication and post-commit cleanup failures | Pre-commit destination/publication errors leave manifest, data-package, and report paths empty. A committed bundle followed by backup-cleanup failure retains authoritative target paths and profile report paths while returning a typed failure. CLI maps destination and cleanup errors to stable content-safe categories. | End-to-end projection is truthful for the currently reported package state. Underlying cancellation/partial-cleanup defects remain `AUD-055` and `AUD-056`. |
|
|
| Never-notify policy | `ComparisonRequest`, comparison orchestration, and CLI comparison wiring expose no notifier. The complete call path ends at bundle publication and aggregate result construction, never Distributor delivery. | Matches policy and retained in `RET-040`. |
|
|
| Safe summaries and exit behavior | Result-bearing outcomes emit one ordered JSON summary unless quiet. Per-profile errors are already bounded `SafeError` values; top-level aggregate, prompt, destination, cleanup, cancellation, and deadline errors map to fixed safe categories/messages. Success requires all profiles plus committed manifest/data paths; partial, all-failed, publication, cleanup, and cancellation paths return an error and failed status. | Coherent and retained as `RET-041`. Pre-result input/config failures deliberately remain ordinary CLI diagnostics. |
|
|
| Test ownership and cost | Application tests own order under concurrent completion, equal input bytes, capacity observation, failure isolation, cancellation joining, debug identities, collect/prepare/publish ordering, partial/all-failed bundles, destination changes, preservation, and cleanup paths. CLI tests own parsing/wiring, one executor, safe structured outcomes, quiet mode, committed cleanup, and status. | Focused ownership is strong except for the mixed cancellation overlap in `AUD-058`; broader suite and cost review remain Stages 19-22. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used codebase graph symbol and augmented-code search, exact snippets, and
|
|
caller/data-flow inspection for comparison orchestration, profile execution,
|
|
cancellation marking, bundle/result copying, CLI request construction, safe
|
|
summaries, and comparison tests. Bounded text inspection covered the scoped
|
|
production and test paths, exact Stage 18 boundary, task-specific canonical
|
|
documents, and all repository policies.
|
|
- Traced pre-result validation, destination/debug/prompt/profile preflight,
|
|
collection/preparation, concurrent success/failure/cancellation, logical
|
|
bundle validation, pre-commit publication failure, committed cleanup failure,
|
|
aggregate failure, quiet output, and CLI result/exit projection. Cross-checked
|
|
ordered identities and artifact paths at every handoff.
|
|
- Ran a temporary deterministic two-profile probe in which one provider failure
|
|
completed before its still-running peer was canceled. It proved the joined
|
|
cancellation sweep changed the completed outcome from `generation` to
|
|
`canceled`; the probe was removed without retaining source or test changes.
|
|
- Ran `go test -race` with focused coverage for `./internal/app` and
|
|
`./internal/cli`; both passed with 76.0% and 75.8% statement coverage.
|
|
Coverage guided missing-overlap inspection but was not itself treated as a
|
|
finding. Ran `go test ./...`, `go vet ./...`,
|
|
`go run ./cmd/weatherreporter --help`, and `git diff --check`; all passed.
|
|
- Findings: `AUD-058`.
|
|
- Retained decisions: `RET-039` through `RET-041`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 19: Audit Test Hermeticity And Execution Hygiene
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all 50 Go test files and 418 top-level tests; all nine JSON
|
|
files under testdata; maintained examples and production embedded assets read
|
|
by tests; external-network, subprocess, environment, process-global, clock,
|
|
randomness, sleep/deadline, filesystem, permission, parallelism, goroutine,
|
|
fixture, and golden-update mechanisms; and the assigned execution, asset,
|
|
test-double, and Go-specific testing-policy sections.
|
|
- Exclusions: Whether important risks have sufficient or correctly owned
|
|
coverage remains Stage 20; test value, duplication, brittleness, and
|
|
maintenance cost remain Stage 21; cross-cutting runtime efficiency remains
|
|
Stage 22; refactoring remains Stage 23; documentation-wide coherence remains
|
|
Stage 24; broader diagnostic/adversarial execution remains Stage 25; and
|
|
remediation remains Stages 26-27.
|
|
|
|
#### Nondeterminism And Machine-State Accounting
|
|
|
|
| Mechanism | Inventory and contextual evidence | Classification |
|
|
| --- | --- | --- |
|
|
| Live networking and external infrastructure | Four `httptest.NewServer` sites serve loopback Weather API fixtures, cancellation, and timeout behavior. Promptkit executions inject a mutex-protected fake LLM client; Distributor injects a fake factory/client; app and CLI use project-owned doubles. External-looking URLs are inert fixture/config values. A full uncached run passed with external HTTP/HTTPS/all proxies forced to a closed loopback port while loopback was exempted. | Controlled and offline; retained as `RET-042`. No live Weather API, Promptkit provider, Distributor, DNS, or mutable service dependency was found. |
|
|
| Subprocesses and host tools | No test imports `os/exec`, calls `exec.Command`, invokes a shell/tool, or uses a helper script. The repository has no test script, Make/Task/just test wrapper, or test-only generated executable. | Absent. Toolchain commands used by the audit are validation, not test-suite behavior. |
|
|
| Environment and credentials | Seventeen `t.Setenv` calls establish synthetic Distributor, Promptkit, and secrets-directory values and restore them automatically; none are parallel. A scrubbed-environment suite passed. One missing-credential test reads `WEATHERREPORTER_TEST_MISSING_KEY` without establishing absence and fails when it is set. | Controlled except `AUD-059`; real-credential independence is narrowly disproved for that test's outcome, though its fake prevents a live call. |
|
|
| Other process-global mutation | No test changes working directory, `time.Local`, command-line globals, logging globals, random seed, scheduler settings, standard streams, or signal handlers. CLI streams are local buffers. Secrets tests are serial and test-owned environment changes restore through the testing package. | Controlled; retained as `RET-044`. |
|
|
| Clocks and calendar state | Workflow/domain tests overwhelmingly use literal times, fixed zones/locations, or injected fixed clocks. One `time.Now` reaches only an early missing-hourly-data error whose result is date-independent. Expired/immediate contexts establish cancellation categories without calendar assumptions. | Controlled. Host tzdata supplies named zones but all asserted dates/zones are explicit and the scrubbed-environment run passed. |
|
|
| Sleeps, deadlines, and elapsed time | One Weather API timeout handler sleeps 50 ms behind a nanosecond client timeout. Retry cancellation measures only that a canceled hour-long delay returns within a generous one-second ceiling. Two five-second timers guard channel/wait-group liveness; they do not order successful execution. The assembled collect test incurs one configured one-second warmup. | Bounded and justified for current behavior; not treated as flakiness. Runtime/failure-path cleanup leads are routed to Stages 21-22. |
|
|
| Randomness and unordered execution | Tests import no random package, seed no generator, and expose no random update choice. Promptkit may assign opaque execution run IDs, but assertions do not depend on them. Map traversal is used for membership/setup rather than asserted incidental order. Ten shuffled repetitions of every package passed. | Controlled. No test-order or randomized-output dependency was reproduced. |
|
|
| Writable filesystem roots and path assumptions | There are 111 `t.TempDir` calls; all consequential writes, replacements, permission changes, and debug artifacts stay beneath those roots. Testdata and examples are tracked read-only inputs. `/tmp/report.md` and `/tmp/debug` occurrences are values passed to doubles/request construction and are not accessed. Relative testdata/example paths follow Go's package working-directory contract. | Controlled and repeatable; retained as `RET-043`. |
|
|
| Permissions, file types, and platforms | Exact sensitive modes are asserted only outside Windows; unreadable file/directory cases skip when effective privilege defeats mode denial. Symlink safety cases use real links, but six sites across app, comparison, and config fatal on any host creation failure instead of checking capability. | Partly machine-specific; `AUD-060`. Current Linux user is non-root and all cases execute successfully. |
|
|
| Parallel tests and goroutine cleanup | Ten `t.Parallel` calls are confined to pure comparison model tests. Six explicit goroutine sites use buffered result channels, cancellation, and wait groups for comparison/debug concurrency. Normal success/failure paths release barriers and join; a repository-wide race run passed. | Controlled; retained as `RET-044`. Failure-only timeout cleanup is routed to Stage 21. |
|
|
| Fixtures, embedded assets, and credentials | Nine tracked JSON fixtures total 6,594 bytes and contain fixed synthetic weather shapes/timestamps. Tests also read maintained secret-free examples and immutable embedded prompt/profile/schema/template assets. Credential search found only environment-variable mechanism names and synthetic values. | Minimal, versioned, local, and credential-free. Fixture realism/value remains a Stage 20-21 question, not a hermeticity defect. |
|
|
| Goldens and update mechanisms | No `.golden`, snapshot, approval file, updater flag/environment variable, or golden-writing code exists. Stable schema/manifest/template fragments are asserted inline or semantically; ordinary validation cannot rewrite repository assets. | Absent; the explicit-update requirement is satisfied vacuously. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph-augmented code search and symbol/snippet inspection to enumerate
|
|
tests that touch HTTP servers, provider/upload doubles, environment,
|
|
deadlines, goroutines, parallel execution, permissions, and filesystem
|
|
helpers. Targeted text searches covered test-only imports/calls, literals,
|
|
non-code fixtures, embedded assets, scripts, and golden mechanisms that are
|
|
not fully represented in the graph.
|
|
- Counted 50 `_test.go` files, 418 top-level tests, no fuzz tests or benchmarks,
|
|
111 temporary-directory calls, 17 environment mutations, four local HTTP
|
|
server constructions, ten parallel calls, six explicit goroutine sites, one
|
|
sleep, and two liveness timers. Inspected every candidate category in context
|
|
rather than treating the raw mechanism as a violation.
|
|
- Ran the full suite with a clean synthetic home, temporary/cache roots, no
|
|
inherited environment, and the existing read-only module cache; it passed.
|
|
Ran another uncached full suite with all external proxies pointed to a closed
|
|
loopback port and only loopback exempted; it passed, confirming external
|
|
network independence at execution time.
|
|
- Ran `go test -shuffle=on -count=10 ./...`; every ordering/repetition passed.
|
|
Ran `go test -race ./...`; all packages passed without a race report. A
|
|
targeted synthetic ambient credential probe reproduced `AUD-059`; it used an
|
|
injected fake and contacted no provider.
|
|
- Ran `go test ./...`, `go vet ./...`,
|
|
`go run ./cmd/weatherreporter --help`, and `git diff --check`; all passed.
|
|
- Findings: `AUD-059` and `AUD-060`.
|
|
- Retained decisions: `RET-042` through `RET-044`.
|
|
- Open questions: the three leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 20: Audit Test Risk Coverage And Ownership
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: the complete 50-file, 418-test Go suite as a risk portfolio;
|
|
architecture invariants; CLI, configuration, domain, adapter, prompt,
|
|
generated-text, schema, template, publication, comparison, concurrency,
|
|
cancellation, partial-success, notification, compatibility, security,
|
|
idempotency, and recovery contracts; meaningful negative paths; test-layer
|
|
ownership; representative cross-boundary repetition; and statement coverage
|
|
only as a gap-finding diagnostic.
|
|
- Exclusions: test brittleness, fixture and helper cost, exact duplication, and
|
|
consolidation remain Stage 21; runtime and algorithmic cost remain Stage 22;
|
|
refactoring remains Stage 23; documentation coherence remains Stage 24;
|
|
broader adversarial execution remains Stage 25; and remediation remains
|
|
Stages 26-27. No production code or tests were changed.
|
|
|
|
#### Consequential Risk-To-Test Matrix
|
|
|
|
| Risk | Narrowest credible owner and meaningful negative paths | Assessment and higher-level repetition |
|
|
| --- | --- | --- |
|
|
| Architecture invariants | Package import/build checks and assembled `app`/`cli` workflows protect dependency direction, adapter containment, collect/prepare/execute/publish order, and stateless normal operation. | Sufficient for active paths. Higher layers repeat only collaboration; dormant persistence remains `AUD-001`. |
|
|
| CLI | `internal/cli` owns parsing, required and conflicting flags, working/output paths, quiet/JSON streams, safe summaries, one-executor wiring, and action status. | Partly insufficient under `AUD-008` through `AUD-010`; lower workflow failures need no exhaustive CLI repetition. |
|
|
| Configuration | `internal/config` owns defaults, strict YAML, precedence, active/inactive validation, secrets, durations, URLs, templates, module options, and maintained examples. | Partly insufficient under `AUD-004` through `AUD-007`; adapter/app tests check only consumed mappings and consequences. |
|
|
| Domain rules | `internal/report`, `timeutil`, `forecast`, `facts`, `module`, and `briefing` own identity, civil/elapsed periods, selection, derivation, availability, ordering, and meteorological thresholds with boundary and malformed cases. | Partly insufficient under `AUD-002`, `AUD-003`, and `AUD-014` through `AUD-028`; app/template cases remain representative projections. |
|
|
| Weather adapter | `internal/adapters/weatherapi` uses local HTTP servers for query shape, all source products, required/optional/checked-empty policy, malformed JSON, statuses, retries, timeouts, cancellation, and provenance. | Partly insufficient under `AUD-011` through `AUD-015`; collection tests need only normalized bundle/error behavior. |
|
|
| Promptkit adapter | `internal/adapters/promptkit` owns exact inspection/preparation/execution mapping, credential absence, callback order, cancellation, copies, provenance fields, and safe classified provider errors through an injected provider client. | Partly insufficient under `AUD-034` and `AUD-044`; app tests appropriately prove lifecycle use rather than provider variants. |
|
|
| Distributor adapter | `internal/adapters/distributor` owns request/path mapping, one upload, idempotency-key forwarding, accepted/conflict states, polling, timeout/cancellation, and token replacement through injected clients. | Partly insufficient under `AUD-048` through `AUD-052`, especially the absent production HTTP boundary; app tests own only notification policy. |
|
|
| Prompt contracts | `promptassets`, `promptinput`, and `promptexec` own embedded identities, category/order rules, validation, deterministic YAML, neutral lifecycle, error categories, bounds, and copy isolation. | Partly insufficient under `AUD-029` through `AUD-033`; complete app inputs are representative composition proofs, not duplicate serializers. |
|
|
| Schemas and generated text | `internal/generatedtext` owns catalog lookup, schema validation, typed decoding, normalization, required fields, and immutable asset copies. | Partly insufficient under `AUD-036` through `AUD-039`; application tests check selected schema/result consequences only. |
|
|
| Templates and Markdown | `internal/reporttemplate` and typed generated-text context tests own lookup failures, report-specific sections, missing values, semantic output, dynamic prose placement, and deterministic repeated renders. | Partly insufficient under `AUD-040` through `AUD-042`; app tests prove publication of one rendered result rather than restating template matrices. |
|
|
| Atomic single output | `internal/fileutil` owns same-directory temporary write/close/rename and cleanup; `internal/app` owns destination preflight, prior-output preservation, committed paths, and notification ordering. | Partly insufficient under `AUD-045` through `AUD-047`; the split correctly separates mechanism from workflow policy. |
|
|
| Comparison replacement | `internal/comparison` owns destination classification, exact recognition, sibling staging, reauthorization, install, rollback, backup cleanup, restrictive modes, and committed-state errors on a real filesystem. | Partly insufficient under `AUD-053` through `AUD-057`; app/CLI repeat only durable outcome projection. |
|
|
| Concurrency | `internal/app` owns one immutable preparation, concurrent profile-local work, backend capacity observation, joining, isolation, stable caller order, and distinct debug identities; the race suite supplies dynamic corroboration. | Partly insufficient under `AUD-058`; pure package tests need no concurrency repetition. |
|
|
| Cancellation | Adapter tests own propagation through requests/retries/providers/uploads; app tests own stop/start, preservation, joining, and no-publication outcomes; CLI should own signal conversion. | Partly insufficient under `AUD-008`, `AUD-047`, `AUD-051`, `AUD-055`, and `AUD-058`; each gap is attached to its observable boundary. |
|
|
| Partial success | App tests own independent batch report results and comparison profile outcomes, counters, partial artifacts, aggregate errors, and deterministic ordering; CLI owns structured partial summaries and failure status. | Sufficient for implemented semantics except the cancellation overlaps in `AUD-051` and `AUD-058`; package negative cases are not repeated. |
|
|
| Notification | App tests own publish-before-notify, all-success batch gating, one call, request identities/paths, notification failure, and never-notify comparisons; the Distributor adapter owns delivery mechanics. | Partly insufficient under `AUD-005` and `AUD-048` through `AUD-052`; ordinary comparison omission is an explicit contract. |
|
|
| Compatibility | Report/prompt/generated-text/comparison packages own stable IDs, versions, catalog pairs, strict decoding, round trips, canonical encodings, and unsupported-version behavior; asset tests exercise embedded current pairs. | Partly insufficient under `AUD-022`, `AUD-032`, `AUD-036`, `AUD-037`, `AUD-053`, and `AUD-054`; dormant JSON/persistence surfaces do not justify workflow tests. |
|
|
| Security | Config, Promptkit, prompt-debug, Weather API, Distributor, file publication, comparison, and CLI tests own secret handling, bounded safe errors, explicit debug, redaction, symlink/type checks, restrictive modes, and content-safe summaries. | Partly insufficient under `AUD-007`, `AUD-012`, `AUD-034`, `AUD-035`, `AUD-038`, `AUD-045`, `AUD-049`, `AUD-050`, and `AUD-053`; higher layers verify safe projection only. |
|
|
| Idempotency | App tests own one batch notification and stable run-derived key construction; Distributor tests own exact key forwarding and already-accepted conflict mapping without replaying app workflow. | Credibly owned at two distinct contracts. Production HTTP traversal remains the broader adapter gap `AUD-052`, not a separate idempotency finding. |
|
|
| Recovery | File and comparison tests own pre-commit cleanup, preservation, replacement rollback, concurrent destination changes, restoration failure, post-commit cleanup truth, and retained recovery paths. | Partly insufficient under `AUD-046`, `AUD-056`, and `AUD-057`; app/CLI tests appropriately assert only user-visible committed and recovery state. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph search, inbound/outbound relationship traces, and symbol snippets to
|
|
map active production paths to their package and higher-level tests. Used
|
|
targeted text search for non-code assets, test names, configuration literals,
|
|
and finding references that are not completely represented in the graph.
|
|
- Reviewed every prior stage's test-ownership accounting and reconciled all
|
|
missing consequential cases with the existing findings. No uncovered risk
|
|
required a new finding; the accepted omissions are thin entry/delegation
|
|
wrappers, defensive error accessors, impossible-under-current-type branches,
|
|
and dormant helpers already routed for removal or justification.
|
|
- Ran `go test -coverprofile=/tmp/weatherreporter-stage20-cover.out ./...`; all
|
|
packages passed and total statement coverage was 79.4%. Inspected the
|
|
function report for unexpectedly weak CLI, app, adapter, domain, prompt,
|
|
filesystem, and comparison areas rather than treating uncovered lines or
|
|
package percentages as targets.
|
|
- Ran `go test ./...`, `go vet ./...`,
|
|
`go run ./cmd/weatherreporter --help`, and `git diff --check`; all passed.
|
|
- Findings: none new. Existing missing-test or defect-specific regression gaps
|
|
remain `AUD-003`, `AUD-010`, `AUD-033`, `AUD-045` through `AUD-047`,
|
|
`AUD-052`, and `AUD-053` through `AUD-060`.
|
|
- Retained decisions: `RET-045` and `RET-046`.
|
|
- Open questions: the four leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 21: Audit Test Durability, Duplication, And Maintenance Cost
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all 50 Go test files and 418 top-level tests, totaling 16,876
|
|
lines; the largest and highest-change test files; exact and fragment error
|
|
assertions; defaults and thresholds; direct private-symbol assertions;
|
|
external-boundary and orchestration fakes; repeated report/workflow fixtures;
|
|
canonical byte assertions and rendered-output checks; table ownership;
|
|
helper size; historical regression guards; concurrent failure cleanup; fresh
|
|
test timing; graph test/call/co-change relationships; and test-file history.
|
|
- Exclusions: production runtime/algorithmic efficiency remains Stage 22;
|
|
implementation and package refactoring remain Stage 23; documentation
|
|
coherence remains Stage 24; fuzz/adversarial execution remains Stage 25; and
|
|
remediation remains Stages 26-27. No production code or tests were changed.
|
|
|
|
#### Durability And Maintenance Accounting
|
|
|
|
| Candidate | Protected behavior and maintenance evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Exact error strings | Six direct equality sites cover one CLI unknown-command diagnostic, three shared generated-text validation messages, and two comparison aggregate summaries. Most other cases use error identity/category, structured fields, safe omissions, or the smallest actionable fragment. | Retain CLI and comparison boundary checks. Generated-text equality participates in the broader duplicated policy finding `AUD-039`; no repository-wide exact-string smell. |
|
|
| Defaults and thresholds | Config tests assert documented defaults/examples; adapter mapping tests use synthetic nondefault values; forecast/briefing tests mostly compare outputs relationally or reference the production threshold where the literal is not itself the contract. One test directly repeats private SPC constants despite an adjacent stronger behavior table. | Retain contractual defaults and relational mechanisms. Remove only the redundant private assertion under `AUD-061`. |
|
|
| Private-helper coupling | Direct tests of batch planning, render projection, classification, and path helpers express dense stable package rules more clearly than assembled workflows. The SPC constant case instead protects representation with no marginal behavior. | Focused helper tests are justified; `AUD-061` is the narrow exception. |
|
|
| Mock choreography | Promptkit and Distributor fakes, Weather API local servers, and app/CLI doubles observe calls only for exact external mapping, one construction/upload, publish-before-notify, no-work-on-preflight, cancellation, capacity, and ordering contracts. | Retain under `RET-048`; no oversized mocking framework or noncontractual call graph was found. |
|
|
| Repeated workflow fixtures | Small `generationBundle`/`generationTime` helpers have 28/30 graph callers and keep representative app/CLI workflows concise. The 156-line briefing context supplies many independent module cases but is a plain deterministic value, not a behavioral framework. | Retain. Shared fixtures reduce rather than amplify incidental setup, and callers assert distinct outcomes. |
|
|
| Render and report-family fixtures | Typed context tests own projection but also render all four reports and repeat reporttemplate's semantic fragments. The two suites changed together 11 times with 0.73 graph coupling; large day-style fixtures amplify that overlap. | Consolidate only the cross-owner Markdown assertions under `AUD-063`; retain report-specific typed projections, template semantics, and one real composition proof. |
|
|
| Complete-output snapshots | No goldens or approval snapshots exist. Exact bytes protect canonical manifest/JSON/YAML encodings, atomic payload preservation, or copy isolation. Markdown assertions use selected fragments, ordering, and omissions. | Retain under `RET-049`; these exact values are compatibility/state contracts rather than incidental output dumps. |
|
|
| Redundant tables | Generated-text day-style/schema duplication is already `AUD-039`. The SPC output filter table subsumes its constant test. Config unknown/retired fields, module compatibility, weather source policy, and comparison destination tables describe distinct inputs or states. | `AUD-039` and `AUD-061` own the proven redundancies. Keep the remaining explicit matrices under `RET-047`. |
|
|
| Helper frameworks | Sixty-seven test helper functions/types matching common helper roles are package-local and mostly small; the largest are literal context/snapshot builders. No shared test package, fluent fixture API, auto-updater, or reflection-heavy assertion framework exists. | No finding. Prefer local direct helpers; do not create a common framework merely to reduce line count. |
|
|
| Regression guards | Removed workspace, recent-change, report-alias, Distributor-path, Scriptorium, and prompt-runtime cases protect strict configuration, statelessness, or migration diagnostics. Destructive publication and comparison regressions retain distinct state outcomes. | Retain. The removed surfaces remain plausible to reintroduce and are not all implied by one generic unknown-field case. |
|
|
| Execution waits | Fresh JSON timing found a 1.00-second collection wrapper test and 0.25-second Distributor polling test; no other top-level test exceeded 0.04 second. Neither case is intended to measure elapsed time. | Avoidable cost under `AUD-062`; preserve the same error/polling behavior with nonretryable input or a controlled wait seam. |
|
|
| Concurrent failure diagnostics | Normal comparison/debug concurrency completes, joins, and passes the race suite. `waitForProfileStarts` has four callers and releases its barrier only for callback failure, not all fatal exits. Prompt-debug filesystem workers have no artificial barrier to release. | Add comparison cleanup under `AUD-064`; retain bounded five-second liveness diagnostics. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph search, symbol snippets, and relationship traces to inspect the
|
|
largest tests/helpers, their production targets, call counts, and direct
|
|
behavioral owners. Graph history ranked current test-file change counts and
|
|
identified 11 co-changes (0.73 coupling) between generated-text context and
|
|
reporttemplate tests.
|
|
- Used targeted text search for exact error equality, whole-byte comparisons,
|
|
defaults/durations, liveness timers, legacy regression names, and test-only
|
|
helper declarations. Counted 50 test files, 418 top-level tests, 16,876 test
|
|
lines, six exact error equality sites, and 67 package-local helper
|
|
declarations matching common fixture/assertion roles.
|
|
- Reviewed Git history for dense/current test files rather than inferring cost
|
|
from size alone. The graph records 37 changes to `config_test.go`, 19 to
|
|
`reporttemplate_test.go`, and 15 to `render_context_test.go`; the first
|
|
remains primarily explicit user-contract coverage, while the latter pair has
|
|
the proven cross-owner overlap in `AUD-063`.
|
|
- Ran an uncached `go test -count=1 -json ./...` timing pass. All packages
|
|
passed; `TestRunWrapsFetchError` took 1.00 second and
|
|
`TestUploadPollsUntilTerminalStatus` took 0.25 second, while every other
|
|
top-level test completed in at most 0.04 second on the audit host.
|
|
- Ran `go test ./...`, `go vet ./...`,
|
|
`go run ./cmd/weatherreporter --help`, and `git diff --check`; all passed.
|
|
- Findings: `AUD-061` through `AUD-064`; existing duplication finding
|
|
`AUD-039` remains applicable.
|
|
- Retained decisions: `RET-047` through `RET-049`.
|
|
- Open questions: the four leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 22: Audit Cross-Cutting Efficiency And Complexity
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: production graph complexity, fan-in/fan-out, loop depth, and
|
|
repeated work; Weather API collection; forecast/fact/module construction;
|
|
prompt-input serialization; generated-text validation and rendering;
|
|
prepare-once execution; single and fixed-batch workflows; comparison profile
|
|
fan-out; comparison bundle construction, recognition, and transactional
|
|
publication; fixed registry construction; allocation and I/O boundaries; and
|
|
realistic service, report, profile, and artifact cardinalities.
|
|
- Exclusions: structural extraction, API simplification, and code deduplication
|
|
remain Stage 23; documentation promises remain Stage 24; adversarial and
|
|
dynamic robustness remain Stage 25; no remediation was implemented. No
|
|
production code or tests were changed.
|
|
|
|
#### Efficiency And Workload Accounting
|
|
|
|
| Candidate | Realistic workload and measured/defensible cost | Disposition |
|
|
| --- | --- | --- |
|
|
| Weather source acquisition | Every workflow collects once. Healthy collection performs a discarded `/conditions/current` warmup, fetches current again, and serially fetches seven other independent products. With equal latency `L`, the transport floor is about `9L`. Five controlled 25-millisecond-delay runs took 236-240 milliseconds, made nine requests, and hit current twice. | Credible user-visible latency and redundant upstream work under `AUD-065`. |
|
|
| Forecast and fact derivation | Weather runs contain a few hundred periods; four default dayparts cause low-thousands of simple linear overlap/selection checks. The graph's propagated depth of three or four reflects composed scans, not one polynomial nested loop over the same growing collection. | Proportionate under `RET-050`; no finding. |
|
|
| Registry and catalog construction | Report/generated-text catalogs contain four definitions and the module catalog is fixed and small. Construction occurs per command/report boundary but does not grow with weather data or generated output. | Retain direct construction under `RET-050`; Stage 23 owns API/duplication questions. |
|
|
| Prompt input and generated validation | One package is built and YAML-serialized per prepared report. Generated output is decoded, semantically normalized, and encoded once per execution; no runtime schema compiler or repeated schema parse exists. Payload-size risk is already `AUD-038`. | Linear necessary work; no separate efficiency finding. |
|
|
| Template rendering | Every render reparses one template and four partials. A temporary parse-through-execution benchmark measured 0.283-0.291 ms and about 91 KiB/1,898 allocations per render. Fixed batches contain three reports and ordinary comparisons a handful of profiles. | Measurable but immaterial at supported volume; retain under `RET-051`. |
|
|
| Prepared comparison fan-out | Facts/modules/input are prepared once. Each explicit profile receives isolated JSON-cloned render inputs, one render, and one result slot; one goroutine starts per profile while Promptkit owns provider capacity. Documentation and examples describe human-selected sets beginning at two profiles. | Proportionate at current scale under `RET-052`; watch tens/hundreds rather than inventing a metric-only limit. |
|
|
| Fixed batch sequencing | Morning/evening batches collect once and sequentially prepare, execute, and publish a fixed three-report set before one notification. Provider calls dominate, but concurrency would change backend-capacity and partial-publication behavior for a bounded gain. | Retain under `RET-052`; no finding from sequentiality alone. |
|
|
| Comparison replacement recognition | Existing recognized destinations are fully scanned four times. Each scan performs `N+4` full reads for `N` successful reports, so ordinary replacement performs `4N+16` reads: 24 for two profiles and 56 for ten. Reports are read four times; manifest and data package eight times. | Credible scaling and filesystem-I/O issue under `AUD-066`; preserve early and commit-time authorization. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Used graph complexity queries, search, symbol snippets, and call traces to
|
|
inspect the highest propagated loop depths and production fan-in/fan-out,
|
|
then followed the concrete generate, batch, comparison, collection,
|
|
derivation, rendering, and publication paths rather than treating graph
|
|
scores as findings.
|
|
- Reconciled prior-stage efficiency leads: service-sized derivation remained
|
|
bounded; template reparsing was measured and retained; extreme profile
|
|
counts remained a watch condition; full comparison reads became `AUD-066`;
|
|
and production waits were separated from the test-only delays in `AUD-062`.
|
|
- Ran a temporary local Weather API diagnostic with 25 milliseconds of
|
|
deterministic server latency for every healthy endpoint. Five runs took
|
|
236-240 milliseconds, issued nine serial requests, and fetched current
|
|
conditions twice. Removed the diagnostic file immediately afterward.
|
|
- Ran a temporary report-template benchmark five times. Parsing through the
|
|
execution boundary took 0.283-0.291 milliseconds with about 91 KiB and 1,898
|
|
allocations per call. Removed the benchmark file immediately afterward.
|
|
- Ran `go test ./...`, `go vet ./...`,
|
|
`go run ./cmd/weatherreporter --help`, and `git diff --check`; all passed.
|
|
- Findings: `AUD-065` and `AUD-066`.
|
|
- Retained decisions: `RET-050` through `RET-052`.
|
|
- Open questions: the four leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 23: Audit Cross-Cutting Refactoring And Deduplication Opportunities
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: graph exact-fingerprint and semantic-similarity candidates;
|
|
repeated report/date policies, mappings, validation, path handling, result
|
|
and error projections, report and module variants, workflow stages, template
|
|
parsing, canonicalization, value copying, prompt curation, generated-text
|
|
APIs, render contexts, registry construction, fact requirements, dormant
|
|
compatibility surfaces, test ownership already established in Stage 21, and
|
|
production/test-file change history.
|
|
- Exclusions: no structural remediation was implemented; documentation
|
|
coherence remains Stage 24; adversarial/dynamic validation remains Stage 25;
|
|
consolidation, priority, and remediation sequencing remain Stages 26-27. No
|
|
production code or tests were changed.
|
|
|
|
#### Refactoring And Duplication Accounting
|
|
|
|
| Candidate | Shared behavior, drift/change evidence, and required stable boundary | Disposition |
|
|
| --- | --- | --- |
|
|
| Daypart canonical keys | Briefing and generated-text contain exact 17-line normalizers introduced in different commits. Both must map accepted configuration names to the same snapshot/render key; collisions are already consequential under `AUD-024`. | One domain owner under `AUD-067`; preserve display labels and semantic roles separately. |
|
|
| Distributor template grammar | Single and batch rendering contain exact 27-line brace parsers and duplicated malformed-input tests. Batch copied the grammar later; typed values and variable sets differ, syntax does not. | One private parser callback under `AUD-068`; retain typed public wrappers and family policies. |
|
|
| Prepared/report metadata | Two briefing builders repeat ten identity/context fields; app retains outer metadata plus a metadata stanza and projects prompt metadata again. Templates use neither complete facts nor module metadata exposed by all contexts. Existing conflict is `AUD-040`. | Establish one prepared identity and narrow projections under `AUD-069`; preserve prompt YAML and Markdown contracts. |
|
|
| Fact requirements | Names live in `module`, lists in briefing definitions, and availability in two switches; constructor accepts unknown or miscategorized strings. `modules.go` has 21 revisions across continued module growth. | Define descriptor and predicate once under `AUD-070`; keep per-module lists/missing policy explicit. |
|
|
| CLI report dates | Generate and comparison each repeat flag eligibility plus required/default/parse logic. Policies arrived in different commits, and a cross-action test now guards agreement in a 36-change file. | One private CLI descriptor/resolver under `AUD-071`; preserve action-specific errors and request types. |
|
|
| Dormant compatibility surfaces | YAML loading, three forecast threshold exports, complete alert envelopes, and canonical normalized generated JSON have no production consumer; some retain tests or runtime copies. | Retire together with related `AUD-001`/`AUD-032` work under `AUD-072`, unless Stage 26 identifies a concrete owner. |
|
|
| Capitalization | Rune-safe `sentenceCase` and byte-slicing `titleWord` express the same presentation mechanic; the latter has four prompt-facing callers and corrupts accepted multibyte labels. | One display-only rune-safe helper under `AUD-073`; canonical identity remains `AUD-067`. |
|
|
| Report variants | Daily/Tomorrow context builders are fingerprint-identical and catalog/planning wrappers are similar, but distinct types, schema/template pairs, IDs, and planning fields are intentional contracts. | Retain thin explicit variants under `RET-053`; existing private day-style helpers own shared mechanics. |
|
|
| Prompt projections | Rich current/hourly/daypart values require lengthy field copies into narrower prompt types. Omitted fields are an authority allowlist, not incidental mapping repetition. | Retain reviewable projections under `RET-054`; no reflection, embedding, or generated mega-mapper. |
|
|
| Maps and absolute paths | Three packages locally copy string maps; app/comparison perform similar absolute-clean checks. Copies protect separate mutation boundaries, while path errors and downstream policy differ. | Retain tiny local helpers under `RET-055`; no generic utility package. |
|
|
| Config and briefing registry | Config calls the runtime briefing registry for module IDs, option types, supported reports, and composition validation. | Retain under `RET-056`; duplicating a config descriptor would increase drift. |
|
|
| Workflow and result translation | Generate, batch, and comparison share phase names but differ in preflight scope, continuation, concurrency, transaction, notification, partial results, and safe CLI projection. | Retain visible state machines and boundary-specific result/error projections under `RET-057`; extract only local phases after open correctness findings settle. |
|
|
| Prepared clone | `cloneResolved` manually copies slices and the sole reference-bearing option; current mutation tests cover it. | Retain under `RET-058`; introduce an option-owned clone contract only when another reference-bearing shape appears. |
|
|
| Test fixtures and duplicate assertions | Stage 21 already identified the concrete cross-owner and implementation-coupled cases in `AUD-039`, `AUD-061`, and `AUD-063`; remaining local fixtures/tables protect distinct contracts. | Do not create a shared fixture framework or duplicate Stage 21 findings. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Enumerated all indexed function nodes and grouped exact fingerprints, then
|
|
manually separated tests, thin report variants, local safety helpers, and
|
|
genuine shared policy. Used semantic graph search, symbol snippets, and
|
|
inbound traces for metadata, requirements, projections, date handling,
|
|
workflows, and dormant APIs.
|
|
- Used targeted text search where assets and field references are outside graph
|
|
coverage. Confirmed no template references `.Collected`, `.Derived`, or
|
|
`.Modules.Metadata`; no production caller consumes validator normalized JSON,
|
|
`AlertRun.Raw`, the three forecast thresholds, or `LoadYAML`.
|
|
- Reviewed history rather than inferring burden from line similarity alone.
|
|
`root.go` has 36 changes, `briefing/modules.go` 21, and
|
|
`config/notify_templates.go` seven. The two daypart normalizers and the two
|
|
CLI action policies were introduced in distinct commits; batch notification
|
|
copied the existing template grammar when that feature arrived.
|
|
- Reconciled every earlier Stage 23 lead: unused persistence/loader surfaces
|
|
join `AUD-072`; metadata and render-context breadth become `AUD-069`;
|
|
requirements become `AUD-070`; date policy becomes `AUD-071`; Unicode and
|
|
daypart identity become `AUD-067`/`AUD-073`; projections, config coupling,
|
|
workflow metrics, and the current clone remain retained decisions.
|
|
- Ran `go test ./...`, `go vet ./...`,
|
|
`go run ./cmd/weatherreporter --help`, and `git diff --check`; all passed.
|
|
- Findings: `AUD-067` through `AUD-073`.
|
|
- Retained decisions: `RET-053` through `RET-058`.
|
|
- Open questions: the four leads recorded above are routed to their assigned
|
|
later stages.
|
|
|
|
### Stage 24: Audit Documentation Coherence And Executable Contracts
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: `README.md`, all current-state, policy, integration,
|
|
internal, ADR, release, release-note, and roadmap documents under `docs/`,
|
|
all three maintained examples, executable CLI help, configuration/report/time
|
|
parsing, embedded prompt/profile/schema/template assets, focused test
|
|
inventories, local and external links, fenced examples, sensitive-value
|
|
patterns, and every documentation lead routed from Stages 3-23.
|
|
- Exclusions: no canonical document, executable help, interface comment,
|
|
production code, test, example, or completed roadmap was remediated. Dynamic
|
|
adversarial validation remains Stage 25; consolidation and remediation
|
|
sequencing remain Stages 26-27.
|
|
|
|
#### Documentation And Executable-Authority Accounting
|
|
|
|
| Topic | Executable authority, canonical documentation owner, and evidence | Disposition |
|
|
| --- | --- | --- |
|
|
| Product orientation and contributor routing | `cmd/weatherreporter` and the assembled workflow establish the product; `README.md` owns the concise orientation and `docs/development.md` owns task routing/package inventory. Their summaries link onward and do not reproduce complete volatile references. | Coherent linked summaries under `RET-059`. |
|
|
| CLI commands, flags, output, and dispatch | `internal/cli/root.go` and `writeActionResult` are executable authority; `docs/cli.md` owns the public contract and `docs/internal/cli.md` owns mechanics. Date/quiet help and comparison-dispatch statements disagree under `AUD-074`; accepted actions, report/date matrix, paths, summary fields, and failure exits otherwise match. | Partly insufficient; exact owners recorded in `AUD-074`. |
|
|
| Configuration fields, defaults, and examples | Config structs/defaults/loading/validation, `report.IDForConfigKey`, `timeutil.LoadLocation`, and the Promptkit adapter are executable authority; `docs/config.md` owns field behavior and `examples/` owns complete files. Defaults, precedence, fields, active/inactive validation, report/module IDs, profile precedence, and all three loaded/inspected examples match. Report-key normalization and numeric-offset bounds are incomplete under `AUD-078`. | Partly insufficient; exact owners recorded in `AUD-078`. |
|
|
| Report, module, prompt, profile, schema, and template identities | Report/briefing/generated-text registries and embedded prompt/profile/schema/template assets are executable authority; focused internal docs, `docs/templates.md`, and `docs/integrations/promptkit.md` own their reader-facing explanations. The four report/prompt/schema/template pairs, prompt version `2.0.0`, profile defaults/models, module catalog/compositions, partials, and schema fields match. Dormant/broad surfaces remain truthful current code until `AUD-069`/`AUD-072` remediation. | Coherent; no duplicate retirement finding. |
|
|
| Weather API and Distributor integrations | Config and both adapters plus the pinned Distributor module are executable authority; `docs/integrations/weatherapi.md`, `docs/integrations/distributor/`, and `docs/config.md` own wire and selection contracts. Endpoint paths/queries, envelopes, availability, retry statuses, bundle mappings, pinned version, upload/status flow, and logical paths match. Unsupported schemes and unsafe diagnostics violate intended contracts under `AUD-011`, `AUD-012`, and `AUD-048` through `AUD-050`; those executable owners should change, with canonical docs reviewed together under `RET-060`. | Earlier implementation findings retained; no duplicate documentation finding. |
|
|
| Prompt and rendering safety contracts | Embedded prompt/schema/template assets, prompt-input/validator/catalog code, and app preparation are executable authority; Promptkit integration, prompt-input/generated-text internals, template docs, and architecture own the written contracts. `AUD-029` through `AUD-031`, `AUD-036` through `AUD-040`, and `AUD-043`/`AUD-044` identify executable violations of correctly stated curation, identity, schema, and preflight expectations. | Repair executable owners and review canonical owners together under `RET-060`; do not document defects as supported behavior. |
|
|
| Output and comparison publication contracts | `internal/fileutil`, app publication, and `internal/comparison` are executable authority; `docs/cli.md`, `docs/operations.md`, `docs/integrations/comparison-bundle.md`, comparison internals, and architecture are canonical owners. `AUD-045`, `AUD-047`, and `AUD-053` through `AUD-056` identify executable violations of stated target-type, cancellation, recognition, filename, and recovery guarantees. Current atomic language promises namespace visibility, not power-loss durability. | Repair executable owners under `RET-060`; bounded terminology retained under `RET-061`. |
|
|
| Comparison execution and result identity | App scheduling/result projection is executable authority; CLI, Promptkit/comparison integration, and comparison internals own the written behavior. `AUD-058` owns the existing mixed failure/cancellation violation; `AUD-077` records the missing concurrent-call substitutability requirement. Current docs correctly avoid an arbitrary profile-count performance promise. | Partly insufficient under `AUD-077`; prior defect remains `AUD-058`; scaling scope retained under `RET-061`. |
|
|
| Focused internal flows and verification claims | Package symbols, callers, and `go test -list` inventories are executable authority; matching files under `docs/internal/` own implementation descriptions. Forecast, collection, report, and prompt-input claims name absent APIs/flows/tests under `AUD-075`; remaining internal flows agree with caller traces and focused suites. | Partly insufficient; exact owners recorded in `AUD-075`. |
|
|
| Roadmap, ADR, release-note, and release lifecycle | Documentation policy is the lifecycle authority. The accepted stateless ADR and versioned release notes are clearly historical, the release procedure owns current publication steps, and `docs/roadmap/future.md` labels proposals unimplemented. The implemented comparison roadmap and completed plan instead duplicate current volatile references under `AUD-076`. The active audit plan/ledger remain temporary coordination records. | Partly insufficient under `AUD-076`; historical distinctions otherwise coherent. |
|
|
| Links, examples, fences, and sensitive content | Every repository-relative Markdown link resolves; both external reference links returned HTTP 200; every fence is balanced and language-tagged. Maintained examples use synthetic `.example.com` endpoints or loopback, contain no credential values, and passed their owning suites. Security-pattern review found mechanisms/placeholders and the intentionally public module host, not private credentials, keys, environment dumps, or undisclosed infrastructure. | Coherent. |
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Inventoried 46 documentation/example files and read every current-state
|
|
canonical owner, internal guide, maintained example, policy, ADR, release
|
|
procedure/note, and roadmap status relevant to Stage 24. Used prior-stage
|
|
findings for code behavior and graph search/snippets/traces only to resolve
|
|
disputed symbols, callers, and concurrency requirements.
|
|
- Compared root and comparison help with the public/internal CLI owners; ran
|
|
focused CLI and report test inventories; confirmed the missing forecast
|
|
symbol and absent explicit collection flow; and checked embedded prompt,
|
|
profile, schema, and template identities against their documentation.
|
|
- Ran focused configuration, Promptkit-adapter, report, and CLI suites; all
|
|
passed, including the maintained-example checks. A local-link checker found
|
|
no missing targets, a fence checker found no untagged or unclosed fences,
|
|
and both external NOAA/SPC references returned HTTP 200.
|
|
- Searched docs/examples for credentials, keys, tokens, authorization values,
|
|
endpoints, and private infrastructure. Only documented placeholders,
|
|
environment-variable names, synthetic endpoints, loopback, public upstream
|
|
references, and the repository's intentionally public module host appeared.
|
|
- Ran `go test ./...`, `go vet ./...`,
|
|
`go run ./cmd/weatherreporter --help`,
|
|
`go run ./cmd/weatherreporter compare --help`, and `git diff --check`; all
|
|
passed.
|
|
- Findings: `AUD-074` through `AUD-078`.
|
|
- Retained decisions: `RET-059` through `RET-061`.
|
|
- Open questions: none; remediation and dynamic-validation leads are routed
|
|
above without changing later-stage scope.
|
|
|
|
### Stage 25: Run Dynamic Robustness And Final Diagnostic Validation
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: the complete default suite and CLI help path, race-enabled
|
|
execution, three shuffled repetitions, vet, statement coverage, critical
|
|
weak branches, whitespace validity, and worktree artifact cleanup.
|
|
- Exclusions: no live service, real credential, production mutation, or
|
|
remediation was introduced. Coverage remained diagnostic rather than a
|
|
percentage target.
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Ran `GOWORK=off go test -count=1 ./...`; all packages passed.
|
|
- Ran `GOWORK=off go test -race -count=1 ./...`; all packages passed with no
|
|
race report.
|
|
- Ran `GOWORK=off go test -shuffle=on -count=3 ./...`; all three shuffled
|
|
repetitions passed with no order-dependent or intermittent failure.
|
|
- Ran `GOWORK=off go vet ./...` and
|
|
`GOWORK=off go run ./cmd/weatherreporter --help`; both passed.
|
|
- Ran `GOWORK=off go test -coverprofile=coverage.out ./...` and inspected the
|
|
function/package report. Total statement coverage was 79.4%. Lower coverage
|
|
remained concentrated in thin production wrappers, defensive accessors,
|
|
report/time helpers, and dormant surfaces already owned by `AUD-003`,
|
|
`AUD-052`, and `AUD-072`; no additional consequential unowned branch became
|
|
a finding.
|
|
- Ran `git diff --check`; it passed. Removed `coverage.out` and verified no
|
|
generated diagnostic artifact remained.
|
|
- Findings: none new. Dynamic execution strengthened the existing evidence for
|
|
the test, cancellation, concurrency, diagnostic-boundary, and publication
|
|
findings without changing their ownership.
|
|
- Retained decisions: all existing decisions remained consistent with the
|
|
dynamic results.
|
|
- Open questions: none.
|
|
|
|
### Stage 26: Verify, Consolidate, And Triage Findings
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all 78 candidate findings, all 61 retained decisions, every
|
|
routed open question, finding-specific validation and test implications, the
|
|
stage-coverage table, and every risk-to-test coverage entry.
|
|
- Exclusions: no production code, test code, canonical documentation,
|
|
implementation priority, acceptance decision, or remediation sequence was
|
|
changed; those remain Stage 27 work. Stage 25's interrupted evidence was
|
|
rerun and recorded during final audit closure rather than inferred.
|
|
|
|
#### Final Disposition
|
|
|
|
| Disposition | Count | Severity | Rationale |
|
|
| --- | ---: | --- | --- |
|
|
| Confirmed | 74 | 2 high, 43 medium, 29 low | Current assets, code paths, tests, graph/caller evidence, or focused probes support a distinct contract, safety, maintenance, or efficiency gap. Each entry names a corrective direction, test implications, and validation outcome. |
|
|
| Rejected as standalone | 4 | 4 low | The evidence is real, but a separate remediation item has insufficient cost-benefit because the work and validation are wholly owned by a broader confirmed root cause. Each rejected entry names that owner. |
|
|
| Deferred | 0 | — | No candidate lacks enough evidence to classify, and remediation deferral is a Stage 27 product-priority decision rather than a finding-confidence status. |
|
|
|
|
All confidence ratings remain high after recheck. Severity remains calibrated
|
|
to plausible user, security, data-integrity, compatibility, or maintenance
|
|
impact rather than implementation cost: the two prompt defects remain high;
|
|
43 operationally consequential defects remain medium; structural,
|
|
documentation, test-cost, and dormant-surface issues remain low. Passing tests
|
|
do not lower the severity of missing boundary cases whose present behavior was
|
|
directly established.
|
|
|
|
#### Consolidation And Recommendation Resolution
|
|
|
|
| Root cause or sequence | Final ownership |
|
|
| --- | --- |
|
|
| Dormant compatibility and persistence | `AUD-072` owns the workflow-owner decision. `AUD-001`, `AUD-022`, and `AUD-032` are rejected only as standalone items and retain their evidence there. |
|
|
| Generated-text test ownership | `AUD-039` owns repeated day-style semantics, misplaced schema assertions, and duplicate render-output assertions. `AUD-063` is rejected as a separate slice of that same boundary. |
|
|
| Daypart identity and display | Establish one canonical key owner under `AUD-067`; use it to correct collision and planning behavior in `AUD-024`/`AUD-025`; keep Unicode-safe display formatting distinct under `AUD-073`. |
|
|
| Prepared metadata and render identity | Establish one prepared metadata authority under `AUD-069`, enforce agreement at context construction under `AUD-040`, then narrow unused render surfaces. |
|
|
| Weather collection efficiency | `AUD-065` may change acquisition scheduling only while preserving source-local outcomes and one deterministic ordered merge; it does not supersede source validation findings `AUD-011` through `AUD-015`. |
|
|
| Comparison recognition efficiency | `AUD-066` may reduce complete reads only while preserving early rejection and commit-time authorization owned by the publisher, including `AUD-053` through `AUD-056`. |
|
|
| Distributor diagnostics and body bounds | `AUD-049` and `AUD-050` remain requirements of the repository-owned adapter boundary. A future pinned-client capability may satisfy them, but dependency selection is an implementation choice and does not weaken the required bounded, redacted result. |
|
|
| Documentation repairs | `AUD-074`, `AUD-075`, `AUD-077`, and `AUD-078` must change with their executable owners; `AUD-076` removes duplicate completed roadmaps only after incoming links are checked. Current defects must not be documented as supported behavior. |
|
|
|
|
#### High-Impact Recheck
|
|
|
|
- Re-read the shared prompt against the prompt-input serializer. It still names
|
|
nonexistent `briefing.derived_daily_summary`,
|
|
`briefing.derived_daypart_summaries`, `briefing.precip_timing`,
|
|
`briefing.outdoor_windows`, and `briefing.metadata.alerts` paths while the
|
|
emitted package groups these values under `briefing.derived_summaries` and
|
|
alert data under applicable-risk products. `AUD-030` remains confirmed high.
|
|
- Rechecked the same prompt's claim that all retained hazard/risk products are
|
|
already location matched. Only point/polygon-specific products carry that
|
|
assurance; regional outlooks and discussions do not. `AUD-031` remains
|
|
confirmed high.
|
|
- Re-read diagnostic-body, prompt-debug, single-file publication, comparison
|
|
manifest/replacement/cleanup, and cancellation boundaries and reran their
|
|
focused packages. The medium security and data-integrity findings remain
|
|
distinct and reproducible from current branches or deterministic seams.
|
|
None is merely a duplicate of the high prompt-contract findings.
|
|
- Reconciled all historical open-question routes. Each terminates in a finding
|
|
or retained decision, and none presents unresolved high-impact evidence, so
|
|
no supplemental investigation stage is necessary.
|
|
|
|
#### Commands And Evidence
|
|
|
|
- Ran focused uncached tests for `internal/promptassets`,
|
|
`internal/promptinput`, `internal/adapters/promptkit`, `internal/fileutil`,
|
|
`internal/comparison`, and `internal/adapters/distributor`; all passed.
|
|
- Ran `GOWORK=off go test -count=1 ./...`, `GOWORK=off go vet ./...`,
|
|
`GOWORK=off go run ./cmd/weatherreporter --help`, and `git diff --check`; all
|
|
passed.
|
|
- Mechanically checked the final finding set: no `candidate` status remains;
|
|
all 74 confirmed findings contain recommendation, test-implication, and
|
|
validation fields; all four rejected findings contain an explicit
|
|
disposition and related consolidated owner; no finding is deferred.
|
|
- Reviewed every retained decision against the later-stage evidence. All 61
|
|
remain valid with their stated reconsideration conditions; none conflicts
|
|
with the consolidated corrective directions.
|
|
- Findings: 74 confirmed; `AUD-001`, `AUD-022`, `AUD-032`, and `AUD-063`
|
|
rejected as standalone duplicates.
|
|
- Retained decisions: `RET-001` through `RET-061` revalidated.
|
|
- Open questions: none.
|
|
|
|
### Stage 27: Produce The Remediation Roadmap And Close The Audit
|
|
|
|
- Status: Complete.
|
|
- Scope reviewed: all 74 confirmed findings, all 61 retained decisions, final
|
|
coverage and risk matrices, discovery exclusions, repository-wide validation,
|
|
canonical documentation ownership, dependency ordering, one-prompt stage
|
|
sizing, and roadmap lifecycle requirements.
|
|
- Exclusions: no finding was remediated; no production code, tests,
|
|
dependencies, embedded assets, maintained examples, canonical current-state
|
|
documents, ADRs, or release notes changed.
|
|
|
|
#### Acceptance And Allocation
|
|
|
|
- Accepted all 74 confirmed findings for implementation. None was priority-
|
|
deferred, and the evidence was sufficient to choose a corrective direction
|
|
without a separate product or architecture decision.
|
|
- Created `docs/roadmap/audit-remediation.md` with 56 independently executable
|
|
stages. Each stage names its findings, affected areas, work, locked behavior,
|
|
test expectations, canonical documentation owners, and exit commands.
|
|
- Allocated every confirmed finding exactly once. Correctness, security, data
|
|
integrity, destructive operations, cancellation, and compatibility precede
|
|
efficiency, behavior-preserving refactoring, test cleanup, and documentation
|
|
retirement. Refactors that must preserve behavior have their own stages
|
|
after related behavioral fixes.
|
|
- Kept the four rejected standalone findings out of the allocation: `AUD-001`,
|
|
`AUD-022`, and `AUD-032` remain consolidated into `AUD-072`; `AUD-063`
|
|
remains consolidated into `AUD-039`.
|
|
- Recorded a final lifecycle gate that removes temporary audit records only
|
|
after remediation is complete and useful contracts/rationale live with their
|
|
durable canonical owners.
|
|
|
|
#### Final Verification
|
|
|
|
- Re-ran and recorded the complete Stage 25 command set on the current checkout
|
|
to close the previously interrupted evidence gap. All commands passed,
|
|
coverage remained 79.4%, and `coverage.out` was removed.
|
|
- Verified mechanically that all 78 ledger entries have final statuses, every
|
|
confirmed finding has one remediation-stage reference, rejected entries have
|
|
no independent remediation assignment, and the roadmap allocation contains
|
|
each confirmed ID exactly once.
|
|
- Ran repository link/reference checks for the new roadmap, `git diff --check`,
|
|
and worktree-scope checks. Only `docs/roadmap/audit.md` and
|
|
`docs/roadmap/audit-remediation.md` changed during Stage 27.
|
|
- Findings: no new audit findings.
|
|
- Retained decisions: `RET-001` through `RET-061` remain valid and are encoded
|
|
as locked behavior or reconsideration constraints where relevant.
|
|
- Open questions and unresolved scope gaps: none.
|