Compare commits
6 Commits
47bacc7abb
...
3c35124db4
| Author | SHA1 | Date | |
|---|---|---|---|
| 3c35124db4 | |||
| e4ec521bed | |||
| 2111e01142 | |||
| a39eea7ed6 | |||
| 7bcce9953e | |||
| 9746a42e04 |
@@ -30,7 +30,8 @@ Flags:
|
||||
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are
|
||||
comma-separated and must be non-empty.
|
||||
- `--resume`: request checkpoint reuse for this invocation. See
|
||||
[Operations](operations.md#checkpoints) for prerequisites and reuse behavior.
|
||||
[Operations](operations.md#checkpoint-cache) for prerequisites and reuse
|
||||
behavior.
|
||||
- `--chunk_cache auto|bypass|refresh`: select chunk-plan reuse for this
|
||||
invocation. `auto` reuses a valid plan by canonical source digest, `bypass`
|
||||
performs no plan-cache I/O, and `refresh` regenerates and replaces a valid
|
||||
|
||||
@@ -14,8 +14,9 @@ debug roots.
|
||||
|
||||
## Output And Cache
|
||||
|
||||
The pipeline runner returns logical output files. The CLI places them beneath
|
||||
the selected output root with confined, atomic writes.
|
||||
The pipeline runner returns logical output files. After validating every
|
||||
logical name, the CLI exclusively creates the run directory beneath the
|
||||
selected output root and performs confined, atomic file writes within it.
|
||||
|
||||
`internal/framework/chunkplan` owns source-addressed plan storage, validation,
|
||||
and atomic publication. Its store is constructed only when the selected mode is
|
||||
@@ -43,6 +44,12 @@ boundaries redact sensitive metadata and credential-shaped bytes while allowing
|
||||
application-owned trace material. Debug data is never a checkpoint source or
|
||||
cache input.
|
||||
|
||||
After allocation, one CLI-owned state value accumulates the known report paths,
|
||||
pipeline outcome counts, and validation status. A single guarded terminalization
|
||||
operation writes the success report, or makes one attempt each to write the
|
||||
failure report and error log. Terminal persistence failures are reported
|
||||
separately and never replace the command's primary error.
|
||||
|
||||
## Tests To Inspect
|
||||
|
||||
- `internal/cli/state_surfaces_test.go`: debug allocation and configuration
|
||||
|
||||
@@ -24,9 +24,17 @@ Durable logical files are written under:
|
||||
<output-root>/<run-id>/
|
||||
```
|
||||
|
||||
Each output file is written atomically. Notarius never automatically removes
|
||||
output. The [JSON output contract](integrations/json-output.md) owns the
|
||||
logical file names, schemas, and media types inside a run directory.
|
||||
The CLI generates one run ID in the form
|
||||
`run-<started-at-unix-nanoseconds>-<32-lowercase-hex-characters>` and uses it
|
||||
for output, manifests, and any requested debug bundle. It validates every
|
||||
logical output name before exclusively creating the run directory. If that
|
||||
directory already exists, the invocation fails without changing it.
|
||||
|
||||
Each output file is written atomically. A later file-write failure leaves the
|
||||
newly allocated partial run directory in place for inspection; Notarius never
|
||||
automatically removes output. The
|
||||
[JSON output contract](integrations/json-output.md) owns the logical file
|
||||
names, schemas, and media types inside a run directory.
|
||||
|
||||
Remove an output run directory only after its consumer data is no longer
|
||||
needed. This is data deletion, not cache cleanup.
|
||||
@@ -132,26 +140,30 @@ supported Unix systems.
|
||||
Notarius never automatically deletes a requested bundle. If allocation
|
||||
succeeds, its path is reported on success and failure. A requested summary or
|
||||
trace write failure makes the command fail, preserving whatever bundle data was
|
||||
already written for inspection.
|
||||
already written for inspection. Every allocated bundle makes one best-effort
|
||||
attempt to record a terminal `run-report.json`.
|
||||
|
||||
## Failures And Warnings
|
||||
|
||||
Failures before debug allocation are reported on stderr without a bundle.
|
||||
Failures after allocation report the bundle path on stderr and write `error.log`
|
||||
when that summary write succeeds. An output-write failure leaves the allocated
|
||||
bundle in place. A successful run with warnings exits `0`, reports a warning
|
||||
count on stderr, and records warnings in durable output and any requested debug
|
||||
summary.
|
||||
Failures after allocation report the bundle path on stderr and make independent
|
||||
attempts to write a failure `run-report.json` and `error.log`. The report retains
|
||||
the paths and pipeline outcome fields known at the failure point. If either
|
||||
terminal write fails, the original command error remains first on stderr,
|
||||
followed by the persistence error and bundle path. An output-write failure
|
||||
leaves the allocated bundle in place. A successful run with warnings exits `0`,
|
||||
reports a warning count on stderr, and records warnings in durable output and
|
||||
any requested debug summary.
|
||||
|
||||
## Cleanup
|
||||
|
||||
Use exact paths for manual cleanup. Examples:
|
||||
|
||||
```sh
|
||||
rm -rf ./notarius-output/run-1234567890
|
||||
rm -rf ./notarius-output/run-1721300000000000000-0123456789abcdef0123456789abcdef
|
||||
rm -rf /var/cache/notarius/chunk-plans/0123abcd
|
||||
rm -rf /var/cache/notarius/checkpoints/pipeline/input-0123/pipeline-4567/identity-89ab
|
||||
rm -rf ./notarius-debug/run-1234567890
|
||||
rm -rf ./notarius-debug/run-1721300000000000000-0123456789abcdef0123456789abcdef
|
||||
```
|
||||
|
||||
Avoid broad recursive cleanup against a parent root unless it is an explicit
|
||||
|
||||
@@ -41,6 +41,12 @@ future work only.
|
||||
- Optional generated example output fixtures with a regeneration procedure.
|
||||
- Additional diagnostics or reporting views if operator workflows need them.
|
||||
|
||||
## Candidate Developer Work
|
||||
|
||||
- Reassess broad CLI and configuration regression coverage, including whether
|
||||
to retain a legacy-checkpoint compatibility fixture and adopt a coverage
|
||||
policy.
|
||||
|
||||
## Candidate Workspace Work
|
||||
|
||||
- Default-idempotent run behavior with an explicit force override.
|
||||
|
||||
@@ -1,403 +0,0 @@
|
||||
# ADR-0006 Findings 1–3 Remediation Plan
|
||||
|
||||
This document is the executable implementation plan for correcting Findings 1,
|
||||
2, and 3 from the post-implementation review of ADR-0006:
|
||||
|
||||
1. unredacted resolved-pipeline data in the debug summary;
|
||||
2. collision-prone output run directories; and
|
||||
3. missing failure `run-report.json` artifacts.
|
||||
|
||||
Finding 4—the broad loss of CLI and configuration regression coverage—is
|
||||
explicitly deferred. The focused regression tests required to prove these three
|
||||
fixes are in scope; restoring or redesigning the general test suites is not.
|
||||
|
||||
The audience is an LLM coding agent. Implement the stages in order. Each stage
|
||||
must finish with a compiling, tested repository.
|
||||
|
||||
## Execution Rules
|
||||
|
||||
Before every stage:
|
||||
|
||||
1. read [Development](../development.md), both documents under `docs/policy/`,
|
||||
[ADR-0006](../adr/0006-separate-output-cache-and-debug-state.md), this plan,
|
||||
and every stage-specific document listed below;
|
||||
2. inspect `git status` and preserve user changes;
|
||||
3. inspect the focused implementation and tests before editing; and
|
||||
4. confirm all earlier stages are complete.
|
||||
|
||||
During every stage:
|
||||
|
||||
- keep the CLI as the run-identity and physical-path composition root;
|
||||
- preserve durable logical output schemas, checkpoint identity and wire
|
||||
compatibility, chunk-plan behavior, debug layout, exit codes, and current
|
||||
configuration contracts unless this plan expressly changes them;
|
||||
- never allow a secondary debug persistence error to hide the primary command
|
||||
error;
|
||||
- add focused tests for the changed behavior without expanding into Finding 4;
|
||||
and
|
||||
- update a canonical current-behavior document only if the implemented contract
|
||||
changes or its existing statement is inaccurate.
|
||||
|
||||
At the end of every stage:
|
||||
|
||||
1. run the focused tests listed for that stage;
|
||||
2. run `go test ./...`;
|
||||
3. run `go vet ./...`;
|
||||
4. run `go build ./cmd/notarius`;
|
||||
5. run `git diff --check`; and
|
||||
6. mark the stage complete here only after all checks pass.
|
||||
|
||||
If repository reality conflicts with this plan, update the plan before making a
|
||||
materially different design choice. Do not silently weaken redaction, output
|
||||
exclusivity, or failure reporting.
|
||||
|
||||
## Fixed Decisions
|
||||
|
||||
### Redacted summary boundary
|
||||
|
||||
- Every file under `summary/` is a redacted artifact. The fact that the adjacent
|
||||
`trace/` may contain application data does not relax this boundary.
|
||||
- `resolved-pipeline.json` retains its current logical shape and useful module,
|
||||
validator, digest, capability, and reference-provenance data, but all module
|
||||
and validator option maps pass through the same recursive sensitive-key
|
||||
redaction used by `effective-config.json`.
|
||||
- The debug-bundle writer must not accept an arbitrary resolved pipeline for a
|
||||
method whose name promises redacted summary output. Introduce a narrow
|
||||
payload interface, analogous to `RedactedSummaryPayload`, that can only be
|
||||
satisfied by a producer responsible for redaction. `config.EffectiveConfig`
|
||||
implements it and returns a deep-cloned, redacted
|
||||
`pipeline.ResolvedPipeline`.
|
||||
- The CLI passes the effective configuration to that method, not
|
||||
`effective.ResolvedPipeline` directly. Do not expose the private redaction
|
||||
helper merely to keep the unsafe call shape.
|
||||
- Redaction must cover input, chunk, output, lane extract/merge/normalize,
|
||||
resolved validator-chain bindings, deprecated lane validator bindings, and
|
||||
recursively nested option maps and lists. Safe option values remain intact.
|
||||
- Reference contents remain excluded through the existing `json:"-"`
|
||||
`ReferenceSet` contract. Reference provenance and configured paths may remain
|
||||
in the summary.
|
||||
- Credentials and sensitive values must not be copied into errors while testing
|
||||
this behavior.
|
||||
|
||||
### Run identity and output allocation
|
||||
|
||||
- The CLI owns one run ID before allocating debug or executing the pipeline.
|
||||
Output, debug, manifests, metadata, stdout, and stderr use that same ID.
|
||||
- The default ID format becomes:
|
||||
|
||||
```text
|
||||
run-<started-at-unix-nanoseconds>-<32-lowercase-hex-characters>
|
||||
```
|
||||
|
||||
The suffix is 16 bytes from `crypto/rand`. Time preserves useful ordering;
|
||||
randomness makes IDs collision-resistant across processes and hosts sharing a
|
||||
root.
|
||||
- Add an injectable `RunIDGenerator func(time.Time) (string, error)` to
|
||||
`internal/cli.Options`. The production default uses `crypto/rand.Reader`.
|
||||
Tests inject deterministic valid IDs. Do not retain package-global clocks or
|
||||
random sources for this path.
|
||||
- Generate the ID after configuration and CLI validation but before debug
|
||||
allocation and pipeline resolution. A generation failure is a normal command
|
||||
failure with exit code `1` and cannot have a debug bundle because no run
|
||||
identity exists yet.
|
||||
- Change debug allocation to accept the CLI-generated run ID and `startedAt`.
|
||||
It validates that the ID is one safe path component and creates that exact
|
||||
bundle with `os.Mkdir`, failing on an existing entry. Debug no longer creates
|
||||
a different ID from its own clock or retries by changing identity.
|
||||
- Durable output allocation is exclusive. Validate every logical output name
|
||||
first, create the output parent as needed, then create
|
||||
`<output-root>/<run-id>` with `os.Mkdir`, not `os.MkdirAll`. An existing run
|
||||
directory is an error; Notarius never writes into it.
|
||||
- Nested logical output directories may still use `MkdirAll` after the exclusive
|
||||
run directory is owned by the invocation. File writes remain atomic.
|
||||
- If output allocation or a later output write fails, retain any newly created
|
||||
partial run directory for inspection. Never remove or overwrite an existing
|
||||
run directory. Document this only if current Operations wording needs
|
||||
clarification.
|
||||
- Keep output directory permissions and logical file permissions unchanged;
|
||||
this remediation addresses identity and exclusivity, not the public output
|
||||
permission contract.
|
||||
|
||||
### Terminal debug reports
|
||||
|
||||
- Every debug-enabled invocation whose bundle was successfully allocated makes
|
||||
one best-effort attempt to write a terminal `run-report.json`, whether the
|
||||
command succeeds or fails.
|
||||
- A failure report uses `Succeeded: false`. Populate fields only from state that
|
||||
is known at the failure point:
|
||||
- `run_id`, requested `pipeline_id`, and `debug_path` are always known after
|
||||
bundle allocation;
|
||||
- `output_path` is included once the effective output root and run ID are
|
||||
known;
|
||||
- output, rejection, warning, and validation fields are populated when a
|
||||
`pipeline.RunOutput` is available; otherwise their zero values are retained.
|
||||
- `error.log` remains the human-readable primary error artifact. Do not add the
|
||||
error string to `RunReport`; this avoids a second unstructured error surface
|
||||
and preserves the existing report schema.
|
||||
- Introduce one CLI-owned run-state value that accumulates terminal-report data
|
||||
as resolution and execution progress. Success and failure reporting both
|
||||
derive `debugbundle.RunReport` from this value rather than constructing
|
||||
unrelated literals.
|
||||
- Add a single summary terminalization operation that attempts
|
||||
`run-report.json` and, on failure, `error.log`. On a successful command it
|
||||
writes only the success report. On a failed command it writes the failure
|
||||
report and error log, attempting both even if one write fails and joining
|
||||
secondary persistence errors for stderr reporting.
|
||||
- Terminalization is invoked at most once. If the primary failure is itself a
|
||||
summary persistence failure, do not recursively retry the failed operation;
|
||||
make one terminalization attempt that excludes the artifact already known to
|
||||
have failed. In particular, a failure to write `run-report.json` must never
|
||||
call itself again.
|
||||
- Stderr prints the primary error first, then any terminalization error, then
|
||||
the allocated debug path. Exit code remains `1`. A secondary error never
|
||||
replaces or wraps away the primary error.
|
||||
- Runs without debug perform no summary work and retain current concise stderr
|
||||
behavior.
|
||||
|
||||
## Stage 1: Enforce redaction for resolved-pipeline summaries
|
||||
|
||||
**Status:** Not started
|
||||
|
||||
### Objective
|
||||
|
||||
Close the summary credential leak and make the redacted-payload boundary
|
||||
difficult to bypass accidentally.
|
||||
|
||||
### Read first
|
||||
|
||||
- `internal/core/config/redaction.go`
|
||||
- `internal/core/config/effective_config.go`
|
||||
- `internal/core/debugbundle/summary.go`
|
||||
- the summary-writing section of `internal/cli/run.go`
|
||||
- `internal/core/config/v3_test.go`
|
||||
- `internal/cli/state_hardening_test.go`
|
||||
- [Architecture: State, Output, And Safety](../policy/architecture.md#state-output-and-safety)
|
||||
|
||||
### Implement
|
||||
|
||||
1. Add the resolved-pipeline redacted payload interface to `debugbundle` and
|
||||
change `SummaryWriter.WriteResolvedPipeline` to accept it rather than `any`.
|
||||
2. Implement the interface on `config.EffectiveConfig` using the existing deep
|
||||
clone/redaction path.
|
||||
3. Audit recursive option redaction. Extend it only where nested list/map values
|
||||
can currently retain a value under a sensitive key.
|
||||
4. Change the CLI call to pass `effective` through the redacted interface.
|
||||
5. Keep `effective-config.json` behavior and `resolved-pipeline.json` logical
|
||||
structure unchanged apart from required redaction.
|
||||
|
||||
### Focused tests
|
||||
|
||||
- Unit-test a resolved pipeline containing distinct sentinel secrets in every
|
||||
module and validator binding listed under Fixed Decisions.
|
||||
- Include nested maps and lists, safe neighboring values, and materialized
|
||||
reference content. Assert secrets and reference contents are absent, redaction
|
||||
markers are present, and safe provenance remains.
|
||||
- Run a debug-enabled CLI invocation whose accepted fake module options include
|
||||
sensitive keys. Inspect both `effective-config.json` and
|
||||
`resolved-pipeline.json` and prove neither contains the sentinels.
|
||||
- Assert mutation of a redacted payload cannot alter the effective configuration
|
||||
or resolved pipeline.
|
||||
|
||||
Run at minimum:
|
||||
|
||||
```sh
|
||||
go test ./internal/core/config ./internal/core/debugbundle ./internal/cli
|
||||
```
|
||||
|
||||
### Exit criteria
|
||||
|
||||
No arbitrary resolved pipeline can be passed to the redacted summary writer;
|
||||
all resolved bindings are recursively redacted; focused leak tests fail against
|
||||
the old implementation and pass against the new one; and all repository checks
|
||||
pass.
|
||||
|
||||
## Stage 2: Make run identity collision-resistant and output exclusive
|
||||
|
||||
**Status:** Not started
|
||||
|
||||
### Objective
|
||||
|
||||
Ensure separate invocations can never silently share or overwrite one durable
|
||||
output run directory.
|
||||
|
||||
### Read first
|
||||
|
||||
- run initialization and output writing in `internal/cli/run.go`
|
||||
- `internal/core/debugbundle/bundle.go`
|
||||
- `internal/core/debugbundle/bundle_test.go`
|
||||
- `internal/cli/state_surfaces_test.go`
|
||||
- `internal/cli/state_hardening_test.go`
|
||||
- [Operations: Output](../operations.md#output)
|
||||
|
||||
### Implement
|
||||
|
||||
1. Add the default cryptographic run-ID generator and inject it through
|
||||
`cli.Options` as specified above. Validate generated IDs before any path use.
|
||||
2. Generate one ID from `startedAt` before debug allocation. Remove the debug
|
||||
allocator's package-global clock and identity generation.
|
||||
3. Refactor `debugbundle.Allocate` to accept the run ID and start time and
|
||||
exclusively create that exact bundle.
|
||||
4. Refactor output writing so the run directory is created exclusively after
|
||||
all logical file names validate. Return a contextual collision error without
|
||||
touching the existing directory.
|
||||
5. Preserve atomic per-file writes, partial-new-directory behavior on later
|
||||
write failures, and the single ID across manifests and reported paths.
|
||||
6. Update existing focused tests to inject deterministic run IDs rather than
|
||||
relying on timestamps or the debug allocator's global clock.
|
||||
|
||||
### Focused tests
|
||||
|
||||
- Verify the production generator's format and validate many generated IDs for
|
||||
uniqueness and path safety without asserting random bytes.
|
||||
- Inject a deterministic ID and prove debug and output use the exact same run
|
||||
directory name and manifest run ID.
|
||||
- Pre-create the target output run directory with sentinel files. Run the
|
||||
command and assert exit `1`, a collision error, and byte-for-byte preservation
|
||||
of the existing tree.
|
||||
- Execute two invocations with the same injected ID and root. The first may
|
||||
succeed; the second must fail without changing the first output.
|
||||
- Pre-create the target debug bundle and assert allocation fails without
|
||||
changing it.
|
||||
- Prove nested logical output paths still work and unsafe logical paths are
|
||||
rejected before the run directory is created.
|
||||
- Prove a later output-file failure retains only the newly allocated partial
|
||||
directory and never affects a sibling run.
|
||||
|
||||
Run at minimum:
|
||||
|
||||
```sh
|
||||
go test ./internal/core/debugbundle ./internal/cli
|
||||
```
|
||||
|
||||
### Exit criteria
|
||||
|
||||
Every invocation has one collision-resistant ID, output and debug allocation
|
||||
are exclusive, an existing run directory is never reused or overwritten, and
|
||||
all repository checks pass.
|
||||
|
||||
## Stage 3: Write terminal run reports for failures
|
||||
|
||||
**Status:** Not started
|
||||
|
||||
### Objective
|
||||
|
||||
Make every allocated debug summary record a structured terminal outcome while
|
||||
preserving primary errors and preventing recursive persistence failures.
|
||||
|
||||
### Read first
|
||||
|
||||
- `internal/core/debugbundle/summary.go`
|
||||
- failure and summary paths in `internal/cli/run.go`
|
||||
- `internal/cli/state_hardening_test.go`
|
||||
- [Run State Internals](../internal/state.md)
|
||||
- [Operations: Debug Bundles](../operations.md#debug-bundles)
|
||||
- [Operations: Failures And Warnings](../operations.md#failures-and-warnings)
|
||||
|
||||
### Implement
|
||||
|
||||
1. Add the CLI-owned run-state/report builder described under Fixed Decisions.
|
||||
Initialize it immediately after run-ID generation and update it as pipeline
|
||||
identity, output path, and `RunOutput` become available.
|
||||
2. Add a debug-summary terminalization method or focused helper that writes the
|
||||
success report, or the failure report plus error log, exactly once.
|
||||
3. Route every post-allocation failure through the common terminalization path:
|
||||
catalog and reference resolution, pipeline resolution, profile validation,
|
||||
reference materialization, registry and LLM construction, preparation, input
|
||||
read, cache/checkpoint construction, pipeline execution, summary writes,
|
||||
output allocation, and output writes.
|
||||
4. Replace the existing success-only report literal with the same report
|
||||
builder and terminalization path.
|
||||
5. Preserve failures before run-ID generation or debug allocation as stderr-only
|
||||
failures.
|
||||
6. Keep explicit guards for failures caused by summary terminalization itself so
|
||||
the command never recursively rewrites `run-report.json` or `error.log`.
|
||||
|
||||
### Focused tests
|
||||
|
||||
- For resolution, pipeline, and output-write failures, assert both
|
||||
`run-report.json` and `error.log` exist, `succeeded` is false, and every known
|
||||
field is correct.
|
||||
- For a pipeline failure with partial `RunOutput`, assert available counts,
|
||||
validation status, manifest, warnings, checkpoint events, and chunk-plan
|
||||
summary are retained.
|
||||
- Assert a success report uses `succeeded: true` and contains the same paths and
|
||||
counts printed by the CLI.
|
||||
- Inject independent failures for the run-report writer and error-log writer.
|
||||
Assert each operation is attempted no more than once, the primary error is
|
||||
printed first, the secondary error is reported, the debug path is printed,
|
||||
and exit code is `1`.
|
||||
- Assert a pre-allocation configuration failure creates no report or debug root.
|
||||
- Assert a non-debug failure performs no summary writes.
|
||||
|
||||
Run at minimum:
|
||||
|
||||
```sh
|
||||
go test ./internal/core/debugbundle ./internal/cli
|
||||
```
|
||||
|
||||
### Exit criteria
|
||||
|
||||
Every successfully allocated debug bundle has a best-effort terminal report;
|
||||
ordinary failures retain both structured and textual outcomes; summary failures
|
||||
cannot recurse or mask the primary error; and all repository checks pass.
|
||||
|
||||
## Stage 4: Final audit and close the remediation plan
|
||||
|
||||
**Status:** Not started
|
||||
|
||||
### Objective
|
||||
|
||||
Verify the three findings are closed, align any affected current-behavior
|
||||
documentation, and remove this completed planning artifact.
|
||||
|
||||
### Read first
|
||||
|
||||
- the final code and tests from Stages 1–3
|
||||
- [Documentation Policy](../policy/documentation.md)
|
||||
- [Architecture Policy](../policy/architecture.md)
|
||||
- [CLI Reference](../cli.md)
|
||||
- [Operations](../operations.md)
|
||||
- [Run State Internals](../internal/state.md)
|
||||
|
||||
### Implement
|
||||
|
||||
1. Audit every `summary/` writer call and confirm each accepts or constructs a
|
||||
redacted payload. Search for direct serialization of raw effective pipeline
|
||||
bindings into summary files.
|
||||
2. Audit every output path and confirm no code writes into a pre-existing run
|
||||
directory. Confirm debug and output use one CLI-owned ID.
|
||||
3. Audit every post-allocation return path and confirm it reaches terminal
|
||||
reporting exactly once or is itself an explicitly guarded terminalization
|
||||
failure.
|
||||
4. Update Operations or Run State Internals only where the final implementation
|
||||
adds a useful current-behavior clarification about run IDs, exclusive output
|
||||
allocation, partial output after failure, or best-effort failure reports.
|
||||
Do not duplicate volatile CLI or configuration contracts.
|
||||
5. Confirm Finding 4 remains deferred. Do not restore unrelated deleted tests,
|
||||
introduce a coverage threshold, or add a legacy-checkpoint fixture in this
|
||||
remediation.
|
||||
6. After all validation succeeds and documentation is current, delete this
|
||||
implementation plan. The accepted ADR and canonical current-behavior
|
||||
documents remain authoritative.
|
||||
|
||||
### Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
go test ./...
|
||||
go vet ./...
|
||||
go build ./cmd/notarius
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Manually inspect the changed debug artifacts from one success, one pipeline
|
||||
failure, and one output collision. Confirm no test fixture contains real
|
||||
credentials or private data.
|
||||
|
||||
### Exit criteria
|
||||
|
||||
Findings 1–3 are closed with focused regression tests; durable output cannot be
|
||||
overwritten by a run-ID collision; failure bundles have terminal reports;
|
||||
summary redaction is enforced by contract; documentation is accurate; Finding
|
||||
4 remains deferred; the plan is removed; and all checks pass.
|
||||
@@ -37,11 +37,13 @@ type Options struct {
|
||||
Catalog pipeline.ModuleCatalog
|
||||
Registries pipeline.Registries
|
||||
LLMClientFactory LLMClientFactory
|
||||
RunIDGenerator RunIDGenerator
|
||||
LookupEnv func(string) (string, bool)
|
||||
Now func() time.Time
|
||||
UserCacheDir func() (string, error)
|
||||
ChunkPlanStoreFactory pipeline.ChunkPlanStoreFactory
|
||||
DebugRecorderFactory func(string) (pipeline.DebugRecorder, error)
|
||||
DebugTerminalFactory func(*debugbundle.SummaryWriter) DebugTerminalWriter
|
||||
}
|
||||
|
||||
type LLMClientFactory func(ctx context.Context, cfg config.Config, profileID string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error)
|
||||
@@ -91,6 +93,9 @@ func normalizeOptions(opts Options) (Options, error) {
|
||||
if opts.Now == nil {
|
||||
opts.Now = time.Now
|
||||
}
|
||||
if opts.RunIDGenerator == nil {
|
||||
opts.RunIDGenerator = defaultRunIDGenerator
|
||||
}
|
||||
if opts.UserCacheDir == nil {
|
||||
opts.UserCacheDir = os.UserCacheDir
|
||||
}
|
||||
@@ -100,6 +105,9 @@ func normalizeOptions(opts Options) (Options, error) {
|
||||
if opts.DebugRecorderFactory == nil {
|
||||
opts.DebugRecorderFactory = frameworkdebug.NewFilesystemRecorder
|
||||
}
|
||||
if opts.DebugTerminalFactory == nil {
|
||||
opts.DebugTerminalFactory = func(writer *debugbundle.SummaryWriter) DebugTerminalWriter { return writer }
|
||||
}
|
||||
if isEmptyCatalog(opts.Catalog) && isEmptyRegistries(opts.Registries) {
|
||||
components, err := newProductionComponents()
|
||||
if err != nil {
|
||||
@@ -213,20 +221,36 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
||||
}
|
||||
|
||||
startedAt := opts.Now().UTC()
|
||||
runID := fmt.Sprintf("run-%d", startedAt.UnixNano())
|
||||
runID, err := opts.RunIDGenerator(startedAt)
|
||||
if err != nil {
|
||||
fmt.Fprintf(stderr, "notarius: generate run ID: %v\n", err)
|
||||
return 1
|
||||
}
|
||||
if err := validateRunID(runID); err != nil {
|
||||
fmt.Fprintf(stderr, "notarius: invalid generated run ID: %v\n", err)
|
||||
return 1
|
||||
}
|
||||
runOutputDir := filepath.Join(cfg.Output.Directory, runID)
|
||||
commandState := newPipelineCommandState(runID, pipelineID, runOutputDir)
|
||||
var summary *debugbundle.SummaryWriter
|
||||
var terminalWriter DebugTerminalWriter
|
||||
debugPath := ""
|
||||
debugRecorder := pipeline.NoopDebugRecorder()
|
||||
if *debug {
|
||||
bundle, err := debugbundle.Allocate(cfg.Debug.Directory)
|
||||
bundle, err := debugbundle.Allocate(cfg.Debug.Directory, runID, startedAt)
|
||||
if err != nil {
|
||||
fmt.Fprintf(stderr, "notarius: %v\n", err)
|
||||
return 1
|
||||
}
|
||||
runID, debugPath, summary = bundle.RunID(), bundle.Path(), bundle.Summary()
|
||||
debugPath, summary = bundle.Path(), bundle.Summary()
|
||||
commandState.setDebugPath(debugPath)
|
||||
terminalWriter = opts.DebugTerminalFactory(summary)
|
||||
if terminalWriter == nil {
|
||||
terminalWriter = summary
|
||||
}
|
||||
debugRecorder, err = opts.DebugRecorderFactory(bundle.TraceRoot())
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("create debug recorder: %w", err), true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("create debug recorder: %w", err))
|
||||
}
|
||||
debugRecorder = pipeline.SynchronizedDebugRecorder(debugRecorder)
|
||||
}
|
||||
@@ -243,16 +267,16 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
||||
StartedAt: startedAt,
|
||||
}
|
||||
if err := writeSummary(summary, func() error { return summary.WriteInvocation(invocation) }); err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug invocation metadata: %w", err), false)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("write debug invocation metadata: %w", err))
|
||||
}
|
||||
|
||||
catalog, err := effectiveCatalog(opts)
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, err, true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
referenceOverrides, referenceUnbinds, err := resolveCLIReferenceRequests(cfg, pipelineID, only, catalog, referenceRequests, referenceUnbindRequests)
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, err, true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
effective, err := cfg.Resolve(config.ResolveInput{
|
||||
PipelineID: pipelineID,
|
||||
@@ -263,43 +287,43 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
||||
ReferenceUnbinds: referenceUnbinds,
|
||||
})
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, err, true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
profileIDs := effectiveLLMProfileIDs(effective.ResolvedPipeline)
|
||||
if err := validateExplicitScriptoriumProfiles(context.Background(), effective.Config, profileIDs); err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, err, true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
workingDir, err := os.Getwd()
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("resolve working directory: %w", err), true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("resolve working directory: %w", err))
|
||||
}
|
||||
materialized, referenceWarnings, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalog, pipeline.ReferenceMaterializationOptions{
|
||||
ConfigPath: loadedConfigPath,
|
||||
WorkingDir: workingDir,
|
||||
})
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, err, true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
effective.ResolvedPipeline = materialized
|
||||
invocation.PipelineDigest = effective.ResolvedPipeline.Digest
|
||||
if err := writeSummary(summary, func() error { return summary.WriteInvocation(invocation) }); err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug invocation metadata: %w", err), false)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("write debug invocation metadata: %w", err))
|
||||
}
|
||||
if err := writeSummary(summary, func() error { return summary.WriteRedactedEffectiveConfig(effective) }); err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug effective config: %w", err), false)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("write debug effective config: %w", err))
|
||||
}
|
||||
if err := writeSummary(summary, func() error { return summary.WriteResolvedPipeline(effective.ResolvedPipeline) }); err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug resolved pipeline: %w", err), false)
|
||||
if err := writeSummary(summary, func() error { return summary.WriteResolvedPipeline(effective) }); err != nil {
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("write debug resolved pipeline: %w", err))
|
||||
}
|
||||
if err := writeSummary(summary, func() error {
|
||||
return summary.WriteResolvedReferences(pipeline.ReferenceProvenance(effective.ResolvedPipeline))
|
||||
}); err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug resolved references: %w", err), false)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("write debug resolved references: %w", err))
|
||||
}
|
||||
|
||||
registries, err := effectiveRegistries(opts)
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, err, true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
@@ -309,24 +333,24 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
||||
}
|
||||
llmClient, llmProfiles, err := opts.LLMClientFactory(ctx, effective.Config, factoryProfileID)
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("create LLM client for profile %q: %w", factoryProfileID, err), true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("create LLM client for profile %q: %w", factoryProfileID, err))
|
||||
}
|
||||
llmClient = pipeline.WithDebugLLMRecording(llmClient, debugRecorder)
|
||||
prepared, err := pipeline.Prepare(effective.ResolvedPipeline, registries, pipeline.ModuleDependencies{LLM: llmClient})
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("prepare pipeline %q: %w", pipelineID, err), true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("prepare pipeline %q: %w", pipelineID, err))
|
||||
}
|
||||
rawInput, err := os.ReadFile(strings.TrimSpace(*inputPath))
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("read input %q: %w", strings.TrimSpace(*inputPath), err), true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("read input %q: %w", strings.TrimSpace(*inputPath), err))
|
||||
}
|
||||
chunkPlans, err := chunkPlanStoreForRun(effective.Config.Cache.ChunkPlans, opts)
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, err, true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
checkpointRecorder, checkpointLoader, err := checkpointHandlersForRun(effective.Config.Cache.Checkpoints, opts, effective.ResolvedPipeline, rawInput, only, llmProfiles, strings.TrimSpace(*llmProfile), strings.TrimSpace(sessionID.value), *resume)
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, err, true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
|
||||
output, err := pipeline.New().Run(ctx, pipeline.RunInput{
|
||||
@@ -346,26 +370,25 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
||||
Debug: debugRecorder,
|
||||
ExtractWorkers: cfg.Concurrency.StageWorkers["extract"],
|
||||
})
|
||||
commandState.observeOutput(output)
|
||||
if err != nil {
|
||||
primaryErr := fmt.Errorf("run pipeline %q: %w", pipelineID, err)
|
||||
if output.Manifest.PipelineID != "" {
|
||||
if summaryErr := writePartialSummary(summary, output); summaryErr != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("run pipeline %q: %w; write debug summary: %v", pipelineID, err, summaryErr), false)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, primaryErr, fmt.Errorf("write debug summary: %w", summaryErr))
|
||||
}
|
||||
}
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("run pipeline %q: %w", pipelineID, err), true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, primaryErr)
|
||||
}
|
||||
|
||||
runOutputDir := filepath.Join(effective.Config.Output.Directory, runID)
|
||||
if err := writePartialSummary(summary, output); err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug summary: %w", err), false)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("write debug summary: %w", err))
|
||||
}
|
||||
if err := writeOutputFiles(runOutputDir, output.OutputFiles); err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, err, true)
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
if err := writeSummary(summary, func() error {
|
||||
return summary.WriteRunReport(debugbundle.RunReport{RunID: runID, PipelineID: effective.PipelineID, OutputPath: runOutputDir, DebugPath: debugPath, Succeeded: true, OutputCount: len(output.NormalizeOutputs), RejectedCount: len(output.Rejected), WarningCount: len(output.Warnings), ValidationStatus: output.Manifest.ValidationStatus})
|
||||
}); err != nil {
|
||||
return failPipelineCommand(stderr, summary, debugPath, fmt.Errorf("write debug run report: %w", err), false)
|
||||
if primaryErr, persistenceErr := commandState.terminalize(terminalWriter, nil); primaryErr != nil {
|
||||
return writePipelineCommandFailure(stderr, commandState, primaryErr, persistenceErr)
|
||||
}
|
||||
|
||||
fmt.Fprintf(stdout, "pipeline %q complete: outputs=%d rejected=%d output=%s\n", effective.PipelineID, len(output.NormalizeOutputs), len(output.Rejected), runOutputDir)
|
||||
@@ -378,19 +401,6 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
||||
return 0
|
||||
}
|
||||
|
||||
func failPipelineCommand(stderr io.Writer, summary *debugbundle.SummaryWriter, debugPath string, err error, recordError bool) int {
|
||||
fmt.Fprintf(stderr, "notarius: %v\n", err)
|
||||
if recordError && summary != nil {
|
||||
if summaryErr := summary.WriteError(err.Error()); summaryErr != nil {
|
||||
fmt.Fprintf(stderr, "notarius: write debug error log: %v\n", summaryErr)
|
||||
}
|
||||
}
|
||||
if debugPath != "" {
|
||||
fmt.Fprintf(stderr, "notarius: debug=%s\n", debugPath)
|
||||
}
|
||||
return 1
|
||||
}
|
||||
|
||||
func writeSummary(summary *debugbundle.SummaryWriter, write func() error) error {
|
||||
if summary == nil {
|
||||
return nil
|
||||
@@ -515,8 +525,15 @@ func writeOutputFiles(runOutputDir string, files []contracts.OutputFile) error {
|
||||
targets = append(targets, outputTarget{path: targetPath, file: file})
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(runOutputDir, 0o755); err != nil {
|
||||
return fmt.Errorf("create output directory %q: %w", runOutputDir, err)
|
||||
outputParent := filepath.Dir(runOutputDir)
|
||||
if err := os.MkdirAll(outputParent, 0o755); err != nil {
|
||||
return fmt.Errorf("create output parent %q: %w", outputParent, err)
|
||||
}
|
||||
if err := os.Mkdir(runOutputDir, 0o755); err != nil {
|
||||
if os.IsExist(err) {
|
||||
return fmt.Errorf("output run directory %q already exists", runOutputDir)
|
||||
}
|
||||
return fmt.Errorf("create output run directory %q: %w", runOutputDir, err)
|
||||
}
|
||||
for _, target := range targets {
|
||||
if err := os.MkdirAll(filepath.Dir(target.path), 0o755); err != nil {
|
||||
|
||||
34
internal/cli/run_id.go
Normal file
34
internal/cli/run_id.go
Normal file
@@ -0,0 +1,34 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"io"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
type RunIDGenerator func(time.Time) (string, error)
|
||||
|
||||
func defaultRunIDGenerator(startedAt time.Time) (string, error) {
|
||||
var suffix [16]byte
|
||||
if _, err := io.ReadFull(rand.Reader, suffix[:]); err != nil {
|
||||
return "", fmt.Errorf("read random run ID suffix: %w", err)
|
||||
}
|
||||
return fmt.Sprintf("run-%d-%s", startedAt.UnixNano(), hex.EncodeToString(suffix[:])), nil
|
||||
}
|
||||
|
||||
func validateRunID(runID string) error {
|
||||
if runID == "" {
|
||||
return fmt.Errorf("run ID must not be empty")
|
||||
}
|
||||
if runID != strings.TrimSpace(runID) {
|
||||
return fmt.Errorf("run ID %q must not have surrounding whitespace", runID)
|
||||
}
|
||||
if strings.ContainsAny(runID, `/\\`) || filepath.IsAbs(runID) || filepath.Clean(runID) != runID || runID == "." || runID == ".." {
|
||||
return fmt.Errorf("run ID %q must be one safe path component", runID)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
87
internal/cli/run_id_test.go
Normal file
87
internal/cli/run_id_test.go
Normal file
@@ -0,0 +1,87 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
)
|
||||
|
||||
func TestDefaultRunIDGeneratorProducesUniqueSafeIDs(t *testing.T) {
|
||||
startedAt := time.Unix(0, 123456789).UTC()
|
||||
pattern := regexp.MustCompile(`^run-123456789-[0-9a-f]{32}$`)
|
||||
seen := make(map[string]struct{}, 256)
|
||||
for i := 0; i < 256; i++ {
|
||||
runID, err := defaultRunIDGenerator(startedAt)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !pattern.MatchString(runID) {
|
||||
t.Fatalf("run ID %q does not match production format", runID)
|
||||
}
|
||||
if err := validateRunID(runID); err != nil {
|
||||
t.Fatalf("run ID %q is not path-safe: %v", runID, err)
|
||||
}
|
||||
if _, exists := seen[runID]; exists {
|
||||
t.Fatalf("duplicate run ID %q", runID)
|
||||
}
|
||||
seen[runID] = struct{}{}
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteOutputFilesSupportsNestedLogicalPaths(t *testing.T) {
|
||||
runPath := filepath.Join(t.TempDir(), "output", "run-safe")
|
||||
if err := writeOutputFiles(runPath, []contracts.OutputFile{{Name: "nested/result.json", Bytes: []byte("result")}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(runPath, "nested", "result.json"))
|
||||
if err != nil || string(data) != "result" {
|
||||
t.Fatalf("nested output = %q, %v", data, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteOutputFilesRejectsUnsafeNamesBeforeAllocatingRunDirectory(t *testing.T) {
|
||||
outputRoot := filepath.Join(t.TempDir(), "output")
|
||||
runPath := filepath.Join(outputRoot, "run-safe")
|
||||
for _, name := range []string{"", "../outside", "/absolute", `nested\\outside`, "nested/../outside"} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
if err := writeOutputFiles(runPath, []contracts.OutputFile{{Name: "safe.json"}, {Name: name}}); err == nil {
|
||||
t.Fatalf("writeOutputFiles accepted %q", name)
|
||||
}
|
||||
if _, err := os.Stat(outputRoot); !os.IsNotExist(err) {
|
||||
t.Fatalf("output root exists or stat failed after %q: %v", name, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteOutputFilesRetainsNewPartialDirectoryAndPreservesSibling(t *testing.T) {
|
||||
outputRoot := filepath.Join(t.TempDir(), "output")
|
||||
siblingPath := filepath.Join(outputRoot, "sibling")
|
||||
if err := os.MkdirAll(siblingPath, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sentinelPath := filepath.Join(siblingPath, "sentinel")
|
||||
if err := os.WriteFile(sentinelPath, []byte("preserve sibling"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
runPath := filepath.Join(outputRoot, "run-safe")
|
||||
err := writeOutputFiles(runPath, []contracts.OutputFile{
|
||||
{Name: "blocked", Bytes: []byte("partial output")},
|
||||
{Name: "blocked/nested.json", Bytes: []byte("unreachable")},
|
||||
})
|
||||
if err == nil || !strings.Contains(err.Error(), "create output directory") {
|
||||
t.Fatalf("writeOutputFiles() error = %v, want later directory failure", err)
|
||||
}
|
||||
if got, err := os.ReadFile(filepath.Join(runPath, "blocked")); err != nil || string(got) != "partial output" {
|
||||
t.Fatalf("partial output = %q, %v", got, err)
|
||||
}
|
||||
if got, err := os.ReadFile(sentinelPath); err != nil || string(got) != "preserve sibling" {
|
||||
t.Fatalf("sibling sentinel = %q, %v", got, err)
|
||||
}
|
||||
}
|
||||
90
internal/cli/run_terminal.go
Normal file
90
internal/cli/run_terminal.go
Normal file
@@ -0,0 +1,90 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/debugbundle"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
)
|
||||
|
||||
type DebugTerminalWriter interface {
|
||||
WriteRunReport(debugbundle.RunReport) error
|
||||
WriteError(string) error
|
||||
}
|
||||
|
||||
type pipelineCommandState struct {
|
||||
report debugbundle.RunReport
|
||||
terminalized bool
|
||||
}
|
||||
|
||||
func newPipelineCommandState(runID, pipelineID, outputPath string) *pipelineCommandState {
|
||||
return &pipelineCommandState{report: debugbundle.RunReport{
|
||||
RunID: runID,
|
||||
PipelineID: pipelineID,
|
||||
OutputPath: outputPath,
|
||||
}}
|
||||
}
|
||||
|
||||
func (s *pipelineCommandState) setDebugPath(debugPath string) {
|
||||
if s != nil {
|
||||
s.report.DebugPath = debugPath
|
||||
}
|
||||
}
|
||||
|
||||
func (s *pipelineCommandState) observeOutput(output pipeline.RunOutput) {
|
||||
if s == nil {
|
||||
return
|
||||
}
|
||||
s.report.OutputCount = len(output.NormalizeOutputs)
|
||||
s.report.RejectedCount = len(output.Rejected)
|
||||
s.report.WarningCount = len(output.Warnings)
|
||||
s.report.ValidationStatus = output.Manifest.ValidationStatus
|
||||
}
|
||||
|
||||
func (s *pipelineCommandState) terminalize(writer DebugTerminalWriter, primaryErr error) (error, error) {
|
||||
if s == nil || s.terminalized {
|
||||
return primaryErr, nil
|
||||
}
|
||||
s.terminalized = true
|
||||
if writer == nil {
|
||||
return primaryErr, nil
|
||||
}
|
||||
|
||||
report := s.report
|
||||
report.Succeeded = primaryErr == nil
|
||||
reportErr := writer.WriteRunReport(report)
|
||||
if reportErr != nil {
|
||||
reportErr = fmt.Errorf("write debug run report: %w", reportErr)
|
||||
if primaryErr == nil {
|
||||
primaryErr = reportErr
|
||||
reportErr = nil
|
||||
}
|
||||
}
|
||||
|
||||
var errorLogErr error
|
||||
if primaryErr != nil {
|
||||
if err := writer.WriteError(primaryErr.Error()); err != nil {
|
||||
errorLogErr = fmt.Errorf("write debug error log: %w", err)
|
||||
}
|
||||
}
|
||||
return primaryErr, errors.Join(reportErr, errorLogErr)
|
||||
}
|
||||
|
||||
func failPipelineCommand(stderr io.Writer, state *pipelineCommandState, writer DebugTerminalWriter, primaryErr error, persistenceErrs ...error) int {
|
||||
primaryErr, terminalErr := state.terminalize(writer, primaryErr)
|
||||
persistenceErrs = append(persistenceErrs, terminalErr)
|
||||
return writePipelineCommandFailure(stderr, state, primaryErr, errors.Join(persistenceErrs...))
|
||||
}
|
||||
|
||||
func writePipelineCommandFailure(stderr io.Writer, state *pipelineCommandState, primaryErr, persistenceErr error) int {
|
||||
fmt.Fprintf(stderr, "notarius: %v\n", primaryErr)
|
||||
if persistenceErr != nil {
|
||||
fmt.Fprintf(stderr, "notarius: %v\n", persistenceErr)
|
||||
}
|
||||
if state != nil && state.report.DebugPath != "" {
|
||||
fmt.Fprintf(stderr, "notarius: debug=%s\n", state.report.DebugPath)
|
||||
}
|
||||
return 1
|
||||
}
|
||||
@@ -3,6 +3,7 @@ package cli
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
@@ -15,6 +16,7 @@ import (
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/debugbundle"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/chunkplan"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
@@ -166,7 +168,7 @@ func TestRunRetainsDebugBundlesAcrossFailures(t *testing.T) {
|
||||
h.extractErr = errors.New("synthetic extraction failure")
|
||||
return h.options()
|
||||
}},
|
||||
{"output", "create output directory", func(t *testing.T, roots stateTestRoots, h *stateTestHarness) Options {
|
||||
{"output", "create output parent", func(t *testing.T, roots stateTestRoots, h *stateTestHarness) Options {
|
||||
if err := os.WriteFile(roots.output, []byte("not a directory"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -248,6 +250,369 @@ func TestRunDebugArtifactsRedactSecretsButRetainApplicationData(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunRedactsSensitiveModuleOptionsFromConfigAndPipelineSummaries(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
data, err := os.ReadFile(roots.config)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
configText := strings.Replace(string(data), " input: test/input\n", ` input:
|
||||
module: test/input
|
||||
options:
|
||||
api_key: CONFIG_SUMMARY_SECRET_SENTINEL
|
||||
safe: SAFE_OPTION_SENTINEL
|
||||
nested:
|
||||
- - password: PIPELINE_SUMMARY_SECRET_SENTINEL
|
||||
neighbor: SAFE_NESTED_OPTION_SENTINEL
|
||||
`, 1)
|
||||
if err := os.WriteFile(roots.config, []byte(configText), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
result := runStateTest(t, roots, newStateTestHarness().options(), true, false, "bypass")
|
||||
if result.code != 0 {
|
||||
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
|
||||
}
|
||||
summaryRoot := filepath.Join(onlyChildDir(t, roots.debug), "summary")
|
||||
for _, name := range []string{"effective-config.json", "resolved-pipeline.json"} {
|
||||
contents, err := os.ReadFile(filepath.Join(summaryRoot, name))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
text := string(contents)
|
||||
for _, secret := range []string{"CONFIG_SUMMARY_SECRET_SENTINEL", "PIPELINE_SUMMARY_SECRET_SENTINEL"} {
|
||||
if strings.Contains(text, secret) {
|
||||
t.Fatalf("%s contains %q: %s", name, secret, text)
|
||||
}
|
||||
}
|
||||
for _, retained := range []string{"[REDACTED]", "SAFE_OPTION_SENTINEL", "SAFE_NESTED_OPTION_SENTINEL"} {
|
||||
if !strings.Contains(text, retained) {
|
||||
t.Fatalf("%s does not contain %q: %s", name, retained, text)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunUsesOneInjectedIdentityForDebugOutputAndManifest(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
harness := newStateTestHarness()
|
||||
opts := harness.options()
|
||||
const runID = "run-1000000000-11111111111111111111111111111111"
|
||||
opts.RunIDGenerator = func(time.Time) (string, error) { return runID, nil }
|
||||
|
||||
result := runStateTest(t, roots, opts, true, false, "bypass")
|
||||
if result.code != 0 {
|
||||
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
|
||||
}
|
||||
outputPath := filepath.Join(roots.output, runID)
|
||||
debugPath := filepath.Join(roots.debug, runID)
|
||||
assertFile(t, filepath.Join(outputPath, "result.json"))
|
||||
assertFile(t, filepath.Join(debugPath, "summary", "run-manifest.json"))
|
||||
if !strings.Contains(result.stdout, "output="+outputPath) || !strings.Contains(result.stdout, "debug="+debugPath) {
|
||||
t.Fatalf("stdout=%q, want shared run identity", result.stdout)
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(debugPath, "summary", "run-manifest.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var manifest artifacts.RunManifest
|
||||
if err := json.Unmarshal(data, &manifest); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if manifest.RunID != runID {
|
||||
t.Fatalf("manifest run ID = %q, want %q", manifest.RunID, runID)
|
||||
}
|
||||
report := readStateTestRunReport(t, debugPath)
|
||||
if !report.Succeeded || report.RunID != runID || report.PipelineID != "sample" || report.OutputPath != outputPath || report.DebugPath != debugPath || report.OutputCount != 1 || report.RejectedCount != 0 || report.WarningCount != 0 || report.ValidationStatus != "approved" {
|
||||
t.Fatalf("success report = %#v", report)
|
||||
}
|
||||
if !strings.Contains(result.stdout, "outputs=1 rejected=0") {
|
||||
t.Fatalf("stdout=%q, want report counts", result.stdout)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunWritesTerminalArtifactsForResolutionPipelineAndOutputFailures(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
pipelineID string
|
||||
wantError string
|
||||
wantOutputs int
|
||||
wantValidation string
|
||||
configureFailure func(*testing.T, stateTestRoots, *stateTestHarness)
|
||||
}{
|
||||
{name: "resolution", pipelineID: "missing", wantError: `pipeline "missing"`},
|
||||
{name: "pipeline", pipelineID: "sample", wantError: "synthetic extraction failure", wantValidation: "failed", configureFailure: func(_ *testing.T, _ stateTestRoots, h *stateTestHarness) {
|
||||
h.extractErr = errors.New("synthetic extraction failure")
|
||||
}},
|
||||
{name: "output", pipelineID: "sample", wantError: "create output parent", wantOutputs: 1, wantValidation: "approved", configureFailure: func(t *testing.T, roots stateTestRoots, _ *stateTestHarness) {
|
||||
if err := os.WriteFile(roots.output, []byte("not a directory"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
harness := newStateTestHarness()
|
||||
if tc.configureFailure != nil {
|
||||
tc.configureFailure(t, roots, harness)
|
||||
}
|
||||
opts := harness.options()
|
||||
var stdout, stderr bytes.Buffer
|
||||
args := []string{"run", tc.pipelineID, "--config", roots.config, "--input", roots.input, "--chunk_cache", "bypass", "--debug"}
|
||||
code := RunWithOptions(args, &stdout, &stderr, opts)
|
||||
if code != 1 || !strings.Contains(stderr.String(), tc.wantError) {
|
||||
t.Fatalf("code=%d stderr=%q", code, stderr.String())
|
||||
}
|
||||
bundlePath := onlyChildDir(t, roots.debug)
|
||||
runID := filepath.Base(bundlePath)
|
||||
report := readStateTestRunReport(t, bundlePath)
|
||||
if report.Succeeded || report.RunID != runID || report.PipelineID != tc.pipelineID || report.OutputPath != filepath.Join(roots.output, runID) || report.DebugPath != bundlePath || report.OutputCount != tc.wantOutputs || report.RejectedCount != 0 || report.WarningCount != 0 || report.ValidationStatus != tc.wantValidation {
|
||||
t.Fatalf("failure report = %#v", report)
|
||||
}
|
||||
errorLog, err := os.ReadFile(filepath.Join(bundlePath, "summary", "error.log"))
|
||||
if err != nil || !strings.Contains(string(errorLog), tc.wantError) {
|
||||
t.Fatalf("error log = %q, %v", errorLog, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunRetainsPartialPipelineOutcomeInFailureSummary(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
harness := newStateTestHarness()
|
||||
harness.chunkWarnings = []contracts.Warning{{Scope: "chunk", ReasonCode: "partial-warning", Message: "warning retained before failure"}}
|
||||
harness.extractErr = errors.New("synthetic partial pipeline failure")
|
||||
|
||||
result := runStateTest(t, roots, harness.options(), true, true, "bypass")
|
||||
if result.code != 1 {
|
||||
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
|
||||
}
|
||||
bundlePath := onlyChildDir(t, roots.debug)
|
||||
report := readStateTestRunReport(t, bundlePath)
|
||||
if report.Succeeded || report.OutputCount != 0 || report.RejectedCount != 0 || report.WarningCount != 1 || report.ValidationStatus != "failed" {
|
||||
t.Fatalf("partial failure report = %#v", report)
|
||||
}
|
||||
|
||||
var manifest artifacts.RunManifest
|
||||
readStateTestSummaryJSON(t, bundlePath, "run-manifest.json", &manifest)
|
||||
if manifest.RunID != report.RunID || manifest.PipelineID != "sample" || manifest.ValidationStatus != "failed" {
|
||||
t.Fatalf("partial manifest = %#v", manifest)
|
||||
}
|
||||
var warnings []contracts.Warning
|
||||
readStateTestSummaryJSON(t, bundlePath, "warnings.json", &warnings)
|
||||
if len(warnings) != 1 || warnings[0].ReasonCode != "partial-warning" {
|
||||
t.Fatalf("partial warnings = %#v", warnings)
|
||||
}
|
||||
var events []pipeline.CheckpointEvent
|
||||
readStateTestSummaryJSON(t, bundlePath, "checkpoint-events.json", &events)
|
||||
if len(events) == 0 || events[0].Stage != "source" {
|
||||
t.Fatalf("partial checkpoint events = %#v, want retained source decision", events)
|
||||
}
|
||||
var chunkPlan artifacts.ChunkPlanSummary
|
||||
readStateTestSummaryJSON(t, bundlePath, "chunk-plan.json", &chunkPlan)
|
||||
if chunkPlan.Mode != "bypass" || chunkPlan.ValidationStatus == "not_run" {
|
||||
t.Fatalf("partial chunk plan = %#v", chunkPlan)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunTerminalPersistenceFailuresDoNotRecurseOrHidePrimaryError(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
reportErr error
|
||||
errorLogErr error
|
||||
wantSecondary string
|
||||
}{
|
||||
{name: "run report", reportErr: errors.New("injected run report failure"), wantSecondary: "injected run report failure"},
|
||||
{name: "error log", errorLogErr: errors.New("injected error log failure"), wantSecondary: "injected error log failure"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
harness := newStateTestHarness()
|
||||
harness.extractErr = errors.New("primary pipeline failure")
|
||||
opts := harness.options()
|
||||
var terminal *recordingTerminalWriter
|
||||
opts.DebugTerminalFactory = func(delegate *debugbundle.SummaryWriter) DebugTerminalWriter {
|
||||
terminal = &recordingTerminalWriter{delegate: delegate, reportErr: tc.reportErr, errorLogErr: tc.errorLogErr}
|
||||
return terminal
|
||||
}
|
||||
|
||||
result := runStateTest(t, roots, opts, true, false, "bypass")
|
||||
if result.code != 1 {
|
||||
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
|
||||
}
|
||||
if terminal == nil {
|
||||
t.Fatal("terminal writer was not constructed")
|
||||
}
|
||||
if terminal.reportCalls != 1 || terminal.errorLogCalls != 1 {
|
||||
t.Fatalf("terminal calls = report:%d error:%d", terminal.reportCalls, terminal.errorLogCalls)
|
||||
}
|
||||
primaryIndex := strings.Index(result.stderr, "primary pipeline failure")
|
||||
secondaryIndex := strings.Index(result.stderr, tc.wantSecondary)
|
||||
debugIndex := strings.Index(result.stderr, "debug=")
|
||||
if primaryIndex < 0 || secondaryIndex <= primaryIndex || debugIndex <= secondaryIndex {
|
||||
t.Fatalf("stderr order = %q", result.stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunReportFailureOnSuccessIsTerminalizedWithoutRetry(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
opts := newStateTestHarness().options()
|
||||
var terminal *recordingTerminalWriter
|
||||
opts.DebugTerminalFactory = func(delegate *debugbundle.SummaryWriter) DebugTerminalWriter {
|
||||
terminal = &recordingTerminalWriter{delegate: delegate, reportErr: errors.New("injected success report failure")}
|
||||
return terminal
|
||||
}
|
||||
|
||||
result := runStateTest(t, roots, opts, true, false, "bypass")
|
||||
if result.code != 1 || !strings.Contains(result.stderr, "write debug run report") || !strings.Contains(result.stderr, "injected success report failure") {
|
||||
t.Fatalf("code=%d stdout=%q stderr=%q", result.code, result.stdout, result.stderr)
|
||||
}
|
||||
if terminal == nil {
|
||||
t.Fatal("terminal writer was not constructed")
|
||||
}
|
||||
if terminal.reportCalls != 1 || terminal.errorLogCalls != 1 {
|
||||
t.Fatalf("terminal calls = report:%d error:%d", terminal.reportCalls, terminal.errorLogCalls)
|
||||
}
|
||||
if result.stdout != "" {
|
||||
t.Fatalf("stdout=%q, want no success message", result.stdout)
|
||||
}
|
||||
bundlePath := onlyChildDir(t, roots.debug)
|
||||
errorLog, err := os.ReadFile(filepath.Join(bundlePath, "summary", "error.log"))
|
||||
if err != nil || !strings.Contains(string(errorLog), "injected success report failure") {
|
||||
t.Fatalf("error log = %q, %v", errorLog, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunWithoutDebugDoesNotUseTerminalSummaryWriter(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
harness := newStateTestHarness()
|
||||
harness.extractErr = errors.New("non-debug pipeline failure")
|
||||
opts := harness.options()
|
||||
factoryCalls := 0
|
||||
opts.DebugTerminalFactory = func(delegate *debugbundle.SummaryWriter) DebugTerminalWriter {
|
||||
factoryCalls++
|
||||
return delegate
|
||||
}
|
||||
|
||||
result := runStateTest(t, roots, opts, false, false, "bypass")
|
||||
if result.code != 1 || !strings.Contains(result.stderr, "non-debug pipeline failure") {
|
||||
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
|
||||
}
|
||||
if factoryCalls != 0 {
|
||||
t.Fatalf("terminal summary factory calls = %d, want 0", factoryCalls)
|
||||
}
|
||||
assertAbsent(t, roots.debug)
|
||||
}
|
||||
|
||||
func TestRunRefusesExistingOutputDirectoryWithoutChangingIt(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
const runID = "run-1000000000-22222222222222222222222222222222"
|
||||
runPath := filepath.Join(roots.output, runID)
|
||||
if err := os.MkdirAll(filepath.Join(runPath, "nested"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(runPath, "sentinel"), []byte("existing output"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(runPath, "nested", "data"), []byte("preserve me"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before := readTree(t, runPath)
|
||||
opts := newStateTestHarness().options()
|
||||
opts.RunIDGenerator = func(time.Time) (string, error) { return runID, nil }
|
||||
|
||||
result := runStateTest(t, roots, opts, true, false, "bypass")
|
||||
if result.code != 1 || !strings.Contains(result.stderr, "output run directory") || !strings.Contains(result.stderr, "already exists") {
|
||||
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
|
||||
}
|
||||
if after := readTree(t, runPath); !sameFiles(after, before) {
|
||||
t.Fatalf("existing output changed: before=%v after=%v", before, after)
|
||||
}
|
||||
bundlePath := filepath.Join(roots.debug, runID)
|
||||
report := readStateTestRunReport(t, bundlePath)
|
||||
if report.Succeeded || report.RunID != runID || report.OutputPath != runPath || report.DebugPath != bundlePath || report.OutputCount != 1 || report.ValidationStatus != "approved" {
|
||||
t.Fatalf("output collision report = %#v", report)
|
||||
}
|
||||
errorLog, err := os.ReadFile(filepath.Join(bundlePath, "summary", "error.log"))
|
||||
if err != nil || !strings.Contains(string(errorLog), "already exists") {
|
||||
t.Fatalf("output collision error log = %q, %v", errorLog, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRepeatedRunIdentityCannotOverwriteFirstOutput(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
const runID = "run-1000000000-33333333333333333333333333333333"
|
||||
harness := newStateTestHarness()
|
||||
opts := harness.options()
|
||||
opts.RunIDGenerator = func(time.Time) (string, error) { return runID, nil }
|
||||
|
||||
first := runStateTest(t, roots, opts, false, false, "bypass")
|
||||
if first.code != 0 {
|
||||
t.Fatalf("first code=%d stderr=%q", first.code, first.stderr)
|
||||
}
|
||||
runPath := filepath.Join(roots.output, runID)
|
||||
before := readTree(t, runPath)
|
||||
second := runStateTest(t, roots, opts, false, false, "bypass")
|
||||
if second.code != 1 || !strings.Contains(second.stderr, "already exists") {
|
||||
t.Fatalf("second code=%d stderr=%q", second.code, second.stderr)
|
||||
}
|
||||
if after := readTree(t, runPath); !sameFiles(after, before) {
|
||||
t.Fatalf("first output changed: before=%v after=%v", before, after)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunRefusesExistingDebugBundleWithoutChangingIt(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
const runID = "run-1000000000-44444444444444444444444444444444"
|
||||
bundlePath := filepath.Join(roots.debug, runID)
|
||||
if err := os.MkdirAll(bundlePath, 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sentinelPath := filepath.Join(bundlePath, "sentinel")
|
||||
if err := os.WriteFile(sentinelPath, []byte("existing debug"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
opts := newStateTestHarness().options()
|
||||
opts.RunIDGenerator = func(time.Time) (string, error) { return runID, nil }
|
||||
|
||||
result := runStateTest(t, roots, opts, true, false, "bypass")
|
||||
if result.code != 1 || !strings.Contains(result.stderr, "debug bundle") || !strings.Contains(result.stderr, "already exists") {
|
||||
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
|
||||
}
|
||||
if got, err := os.ReadFile(sentinelPath); err != nil || string(got) != "existing debug" {
|
||||
t.Fatalf("sentinel = %q, %v", got, err)
|
||||
}
|
||||
assertAbsent(t, roots.output)
|
||||
}
|
||||
|
||||
func TestRunIDGenerationFailurePrecedesDebugAllocation(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
opts := newStateTestHarness().options()
|
||||
opts.RunIDGenerator = func(time.Time) (string, error) { return "", errors.New("random source unavailable") }
|
||||
|
||||
result := runStateTest(t, roots, opts, true, false, "bypass")
|
||||
if result.code != 1 || !strings.Contains(result.stderr, "generate run ID: random source unavailable") {
|
||||
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
|
||||
}
|
||||
assertAbsent(t, roots.debug)
|
||||
assertAbsent(t, roots.output)
|
||||
}
|
||||
|
||||
func TestRunRejectsUnsafeGeneratedIdentityBeforePathUse(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
opts := newStateTestHarness().options()
|
||||
opts.RunIDGenerator = func(time.Time) (string, error) { return "../outside", nil }
|
||||
|
||||
result := runStateTest(t, roots, opts, true, false, "bypass")
|
||||
if result.code != 1 || !strings.Contains(result.stderr, "invalid generated run ID") || !strings.Contains(result.stderr, "one safe path component") {
|
||||
t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
|
||||
}
|
||||
assertAbsent(t, roots.debug)
|
||||
assertAbsent(t, roots.output)
|
||||
}
|
||||
|
||||
type stateTestRoots struct{ config, input, output, plans, checkpoints, debug string }
|
||||
|
||||
func newStateTestRoots(t *testing.T) stateTestRoots {
|
||||
@@ -351,6 +716,24 @@ func readAllFiles(t *testing.T, root string) string {
|
||||
return content.String()
|
||||
}
|
||||
|
||||
func readStateTestRunReport(t *testing.T, bundlePath string) debugbundle.RunReport {
|
||||
t.Helper()
|
||||
var report debugbundle.RunReport
|
||||
readStateTestSummaryJSON(t, bundlePath, "run-report.json", &report)
|
||||
return report
|
||||
}
|
||||
|
||||
func readStateTestSummaryJSON(t *testing.T, bundlePath, name string, target any) {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(filepath.Join(bundlePath, "summary", name))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := json.Unmarshal(data, target); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func assertRestrictedTree(t *testing.T, root string) {
|
||||
t.Helper()
|
||||
if runtime.GOOS == "windows" {
|
||||
@@ -413,7 +796,9 @@ func sameFiles(left, right map[string][]byte) bool {
|
||||
type stateTestHarness struct {
|
||||
mu sync.Mutex
|
||||
chunkCalls, extractCalls int
|
||||
runIDCalls uint64
|
||||
extractErr error
|
||||
chunkWarnings []contracts.Warning
|
||||
}
|
||||
|
||||
func newStateTestHarness() *stateTestHarness { return &stateTestHarness{} }
|
||||
@@ -422,7 +807,7 @@ func (h *stateTestHarness) options() Options {
|
||||
if err := pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, stateTestCodec{}); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
if err := registries.Inputs.RegisterWithSpec(pipeline.ModuleSpec{Key: "test/input", Stage: pipeline.StageInput, Provides: []string{"source"}}, func() (contracts.InputAdapter, error) { return stateTestInput{}, nil }); err != nil {
|
||||
if err := registries.Inputs.RegisterBuilderWithSpec(pipeline.ModuleSpec{Key: "test/input", Stage: pipeline.StageInput, Provides: []string{"source"}}, func(map[string]any) error { return nil }, func(pipeline.BuildRequest) (contracts.InputAdapter, error) { return stateTestInput{}, nil }); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
if err := registries.Chunkers.RegisterWithSpec(pipeline.ModuleSpec{Key: "test/chunk", Stage: pipeline.StageChunk, Requires: []string{"source"}, Provides: []string{"chunks"}}, func() (contracts.Chunker, error) { return stateTestChunker{h}, nil }); err != nil {
|
||||
@@ -440,7 +825,12 @@ func (h *stateTestHarness) options() Options {
|
||||
if err := registries.Outputs.RegisterWithSpec(pipeline.ModuleSpec{Key: "test/output", Stage: pipeline.StageOutput, Requires: []string{"normalized"}, Provides: []string{"output"}}, func() (contracts.OutputEncoder, error) { return stateTestOutput{}, nil }); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return Options{Catalog: catalogFromRegistries(registries), Registries: registries, LookupEnv: emptyLookup, Now: func() time.Time { return time.Unix(1, 0) }, UserCacheDir: func() (string, error) { return "", errors.New("unexpected user cache lookup") }, LLMClientFactory: func(context.Context, config.Config, string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) {
|
||||
return Options{Catalog: catalogFromRegistries(registries), Registries: registries, LookupEnv: emptyLookup, Now: func() time.Time { return time.Unix(1, 0) }, RunIDGenerator: func(startedAt time.Time) (string, error) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.runIDCalls++
|
||||
return fmt.Sprintf("run-%d-%032x", startedAt.UnixNano(), h.runIDCalls), nil
|
||||
}, UserCacheDir: func() (string, error) { return "", errors.New("unexpected user cache lookup") }, LLMClientFactory: func(context.Context, config.Config, string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) {
|
||||
return nil, nil, nil
|
||||
}}
|
||||
}
|
||||
@@ -460,7 +850,7 @@ func (c stateTestChunker) Plan(_ context.Context, req contracts.ChunkRequest) (c
|
||||
c.harness.mu.Lock()
|
||||
c.harness.chunkCalls++
|
||||
c.harness.mu.Unlock()
|
||||
return contracts.ChunkPlanResult{Plan: source.ChunkPlan{SourceDigest: req.Source.Digest, Ranges: []source.ChunkRange{{StartUnitID: 1, EndUnitID: 1}}}}, nil
|
||||
return contracts.ChunkPlanResult{Plan: source.ChunkPlan{SourceDigest: req.Source.Digest, Ranges: []source.ChunkRange{{StartUnitID: 1, EndUnitID: 1}}}, Warnings: append([]contracts.Warning(nil), c.harness.chunkWarnings...)}, nil
|
||||
}
|
||||
|
||||
const stateTestArtifactKind contracts.ArtifactKind = "test/artifact"
|
||||
@@ -526,3 +916,25 @@ type failingDebugRecorder struct{}
|
||||
func (failingDebugRecorder) Enabled() bool { return true }
|
||||
func (failingDebugRecorder) WriteJSON(string, any) error { return errors.New("trace unavailable") }
|
||||
func (failingDebugRecorder) WriteBytes(string, []byte) error { return errors.New("trace unavailable") }
|
||||
|
||||
type recordingTerminalWriter struct {
|
||||
delegate DebugTerminalWriter
|
||||
reportErr, errorLogErr error
|
||||
reportCalls, errorLogCalls int
|
||||
}
|
||||
|
||||
func (w *recordingTerminalWriter) WriteRunReport(report debugbundle.RunReport) error {
|
||||
w.reportCalls++
|
||||
if w.reportErr != nil {
|
||||
return w.reportErr
|
||||
}
|
||||
return w.delegate.WriteRunReport(report)
|
||||
}
|
||||
|
||||
func (w *recordingTerminalWriter) WriteError(message string) error {
|
||||
w.errorLogCalls++
|
||||
if w.errorLogErr != nil {
|
||||
return w.errorLogErr
|
||||
}
|
||||
return w.delegate.WriteError(message)
|
||||
}
|
||||
|
||||
@@ -6,8 +6,13 @@ import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
const stateSurfaceRunID = "run-1000000000-55555555555555555555555555555555"
|
||||
|
||||
func stateSurfaceRunIDGenerator(time.Time) (string, error) { return stateSurfaceRunID, nil }
|
||||
|
||||
func TestRunRejectsDebugDirectoryWithoutDebug(t *testing.T) {
|
||||
var stdout, stderr bytes.Buffer
|
||||
code := RunWithOptions([]string{"run", "example", "--input", "source.json", "--debug-dir", t.TempDir()}, &stdout, &stderr, Options{})
|
||||
@@ -20,7 +25,7 @@ func TestRunDebugAllocatesBeforePipelineResolution(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
configPath := writeV3Config(t, "")
|
||||
var stdout, stderr bytes.Buffer
|
||||
code := RunWithOptions([]string{"run", "missing", "--config", configPath, "--input", "source.json", "--debug", "--debug-dir", root, "--chunk_cache", "bypass"}, &stdout, &stderr, Options{LookupEnv: emptyLookup})
|
||||
code := RunWithOptions([]string{"run", "missing", "--config", configPath, "--input", "source.json", "--debug", "--debug-dir", root, "--chunk_cache", "bypass"}, &stdout, &stderr, Options{LookupEnv: emptyLookup, RunIDGenerator: stateSurfaceRunIDGenerator})
|
||||
if code != 1 {
|
||||
t.Fatalf("code=%d stderr=%q", code, stderr.String())
|
||||
}
|
||||
@@ -28,7 +33,10 @@ func TestRunDebugAllocatesBeforePipelineResolution(t *testing.T) {
|
||||
if err != nil || len(entries) != 1 {
|
||||
t.Fatalf("debug bundles: %v, %v", entries, err)
|
||||
}
|
||||
bundle := filepath.Join(root, entries[0].Name())
|
||||
if entries[0].Name() != stateSurfaceRunID {
|
||||
t.Fatalf("debug bundle name = %q, want %q", entries[0].Name(), stateSurfaceRunID)
|
||||
}
|
||||
bundle := filepath.Join(root, stateSurfaceRunID)
|
||||
for _, name := range []string{"summary", "trace"} {
|
||||
if info, err := os.Stat(filepath.Join(bundle, name)); err != nil || !info.IsDir() {
|
||||
t.Fatalf("%s: %v", name, err)
|
||||
@@ -49,7 +57,7 @@ func TestRunWithoutDebugDoesNotAllocateDebugRoot(t *testing.T) {
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
code := RunWithOptions([]string{"run", "missing", "--config", configPath, "--input", "source.json", "--chunk_cache", "bypass"}, &stdout, &stderr, Options{LookupEnv: lookup})
|
||||
code := RunWithOptions([]string{"run", "missing", "--config", configPath, "--input", "source.json", "--chunk_cache", "bypass"}, &stdout, &stderr, Options{LookupEnv: lookup, RunIDGenerator: stateSurfaceRunIDGenerator})
|
||||
if code != 1 {
|
||||
t.Fatalf("code=%d stderr=%q", code, stderr.String())
|
||||
}
|
||||
|
||||
@@ -25,6 +25,10 @@ func (e EffectiveConfig) RedactedSummaryPayload() any {
|
||||
}
|
||||
}
|
||||
|
||||
func (e EffectiveConfig) RedactedResolvedPipelinePayload() pipeline.ResolvedPipeline {
|
||||
return cloneResolvedPipeline(e.ResolvedPipeline)
|
||||
}
|
||||
|
||||
func cloneResolvedPipeline(in pipeline.ResolvedPipeline) pipeline.ResolvedPipeline {
|
||||
out := in
|
||||
out.Input = redactBinding(cloneModuleBinding(in.Input))
|
||||
@@ -116,26 +120,26 @@ func redactOptions(values map[string]any) map[string]any {
|
||||
out[key] = "[REDACTED]"
|
||||
continue
|
||||
}
|
||||
switch typed := value.(type) {
|
||||
case map[string]any:
|
||||
out[key] = redactOptions(typed)
|
||||
case []any:
|
||||
items := make([]any, len(typed))
|
||||
for i, item := range typed {
|
||||
if nested, ok := item.(map[string]any); ok {
|
||||
items[i] = redactOptions(nested)
|
||||
} else {
|
||||
items[i] = item
|
||||
}
|
||||
}
|
||||
out[key] = items
|
||||
default:
|
||||
out[key] = value
|
||||
}
|
||||
out[key] = redactOptionValue(value)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func redactOptionValue(value any) any {
|
||||
switch typed := value.(type) {
|
||||
case map[string]any:
|
||||
return redactOptions(typed)
|
||||
case []any:
|
||||
items := make([]any, len(typed))
|
||||
for i, item := range typed {
|
||||
items[i] = redactOptionValue(item)
|
||||
}
|
||||
return items
|
||||
default:
|
||||
return typed
|
||||
}
|
||||
}
|
||||
|
||||
func sensitiveConfigKey(key string) bool {
|
||||
key = strings.ToLower(key)
|
||||
return strings.Contains(key, "api_key") || strings.Contains(key, "apikey") || strings.Contains(key, "authorization") || strings.Contains(key, "bearer") || strings.Contains(key, "password") || strings.Contains(key, "secret") || strings.Contains(key, "token")
|
||||
|
||||
169
internal/core/config/redaction_test.go
Normal file
169
internal/core/config/redaction_test.go
Normal file
@@ -0,0 +1,169 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
)
|
||||
|
||||
func TestRedactedResolvedPipelinePayloadRedactsEveryBinding(t *testing.T) {
|
||||
bindings := map[string]pipeline.ModuleBinding{}
|
||||
for _, name := range []string{
|
||||
"input", "chunk", "output", "extract", "merge", "normalize",
|
||||
"resolved-validator", "lane-validator",
|
||||
} {
|
||||
bindings[name] = redactionTestBinding(name)
|
||||
}
|
||||
|
||||
resolved := pipeline.ResolvedPipeline{
|
||||
ID: "redaction-test",
|
||||
Digest: "sha256:safe-digest",
|
||||
Input: bindings["input"],
|
||||
Chunk: bindings["chunk"],
|
||||
ChunkReferences: redactionTestReferenceTarget(pipeline.StageChunk, "", "chunk-reference-content"),
|
||||
ArtifactLanes: []pipeline.ResolvedArtifactLane{{
|
||||
ID: "safe-lane",
|
||||
ArtifactKind: "safe/artifact",
|
||||
Extract: bindings["extract"],
|
||||
Merge: bindings["merge"],
|
||||
Normalize: bindings["normalize"],
|
||||
Validators: []pipeline.ModuleBinding{bindings["lane-validator"]},
|
||||
ExtractReferences: redactionTestReferenceTarget(pipeline.StageExtract, "safe-lane", "extract-reference-content"),
|
||||
MergeReferences: redactionTestReferenceTarget(pipeline.StageMerge, "safe-lane", "merge-reference-content"),
|
||||
NormalizeReferences: redactionTestReferenceTarget(pipeline.StageNormalize, "safe-lane", "normalize-reference-content"),
|
||||
}},
|
||||
ValidatorChains: []pipeline.ResolvedValidatorChain{{
|
||||
Stage: pipeline.StageExtract,
|
||||
LaneID: "safe-lane",
|
||||
ModuleKey: "safe-extract-owner",
|
||||
Validators: []pipeline.ResolvedValidator{{
|
||||
Binding: bindings["resolved-validator"],
|
||||
ExecutionClass: contracts.ExecutionClassDeterministic,
|
||||
Target: pipeline.ValidatorTargetTyped,
|
||||
ArtifactKind: "safe/artifact",
|
||||
}},
|
||||
}},
|
||||
Output: bindings["output"],
|
||||
}
|
||||
effective := EffectiveConfig{
|
||||
Config: Config{Pipelines: map[string]pipeline.PipelineProfile{
|
||||
"redaction-test": {Input: bindings["input"]},
|
||||
}},
|
||||
PipelineID: "redaction-test",
|
||||
ResolvedPipeline: resolved,
|
||||
}
|
||||
|
||||
payload := effective.RedactedResolvedPipelinePayload()
|
||||
encoded, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
text := string(encoded)
|
||||
for name := range bindings {
|
||||
for _, forbidden := range []string{name + "-secret", name + "-nested-secret"} {
|
||||
if strings.Contains(text, forbidden) {
|
||||
t.Fatalf("resolved pipeline summary contains %q: %s", forbidden, text)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(text, name+"-safe") {
|
||||
t.Fatalf("resolved pipeline summary does not retain safe option for %q: %s", name, text)
|
||||
}
|
||||
}
|
||||
for _, content := range []string{
|
||||
"chunk-reference-content", "extract-reference-content",
|
||||
"merge-reference-content", "normalize-reference-content",
|
||||
} {
|
||||
if strings.Contains(text, content) {
|
||||
t.Fatalf("resolved pipeline summary contains materialized reference content %q", content)
|
||||
}
|
||||
}
|
||||
for _, safe := range []string{"[REDACTED]", "safe-reference-path", "safe-binding-source"} {
|
||||
if !strings.Contains(text, safe) {
|
||||
t.Fatalf("resolved pipeline summary does not retain %q: %s", safe, text)
|
||||
}
|
||||
}
|
||||
|
||||
payload.Input.Options["safe"] = "mutated"
|
||||
nested := payload.Input.Options["nested"].([]any)[0].([]any)[0].(map[string]any)
|
||||
nested["neighbor"] = "mutated"
|
||||
payload.ChunkReferences.ReferenceSet.Slots["safe-slot"].Items[0].Content[0] = 'X'
|
||||
payload.ValidatorChains[0].Validators[0].Binding.Options["safe"] = "mutated"
|
||||
|
||||
assertRedactionTestBindingUnchanged(t, effective.ResolvedPipeline.Input, "input")
|
||||
assertRedactionTestBindingUnchanged(t, effective.Config.Pipelines["redaction-test"].Input, "input")
|
||||
assertRedactionTestBindingUnchanged(t, effective.ResolvedPipeline.ValidatorChains[0].Validators[0].Binding, "resolved-validator")
|
||||
if got := string(effective.ResolvedPipeline.ChunkReferences.ReferenceSet.Slots["safe-slot"].Items[0].Content); got != "chunk-reference-content" {
|
||||
t.Fatalf("source reference content mutated through redacted payload: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRedactedEffectiveConfigPayloadDoesNotAliasSource(t *testing.T) {
|
||||
binding := redactionTestBinding("effective")
|
||||
effective := EffectiveConfig{
|
||||
Config: Config{Pipelines: map[string]pipeline.PipelineProfile{
|
||||
"redaction-test": {Input: binding},
|
||||
}},
|
||||
ResolvedPipeline: pipeline.ResolvedPipeline{Input: binding},
|
||||
}
|
||||
|
||||
payload := effective.RedactedSummaryPayload().(EffectiveConfig)
|
||||
payload.Config.Pipelines["redaction-test"].Input.Options["safe"] = "mutated"
|
||||
payload.ResolvedPipeline.Input.Options["safe"] = "mutated"
|
||||
|
||||
assertRedactionTestBindingUnchanged(t, effective.Config.Pipelines["redaction-test"].Input, "effective")
|
||||
assertRedactionTestBindingUnchanged(t, effective.ResolvedPipeline.Input, "effective")
|
||||
}
|
||||
|
||||
func redactionTestBinding(name string) pipeline.ModuleBinding {
|
||||
return pipeline.ModuleBinding{
|
||||
Module: "safe-" + name,
|
||||
Options: map[string]any{
|
||||
"api_key": name + "-secret",
|
||||
"safe": name + "-safe",
|
||||
"nested": []any{[]any{map[string]any{
|
||||
"password": name + "-nested-secret",
|
||||
"neighbor": name + "-nested-safe",
|
||||
}}},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func redactionTestReferenceTarget(stage pipeline.ModuleStage, laneID, content string) pipeline.ResolvedReferenceTarget {
|
||||
return pipeline.ResolvedReferenceTarget{
|
||||
Stage: stage,
|
||||
LaneID: laneID,
|
||||
Module: "safe-reference-module",
|
||||
Bindings: []pipeline.ReferenceBinding{{
|
||||
Stage: stage,
|
||||
LaneID: laneID,
|
||||
SlotName: "safe-slot",
|
||||
Source: "safe-reference-path",
|
||||
BindingSource: "safe-binding-source",
|
||||
}},
|
||||
ReferenceSet: contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
|
||||
"safe-slot": {
|
||||
Slot: contracts.ReferenceSlot{Name: "safe-slot"},
|
||||
Items: []contracts.ReferenceItem{{
|
||||
SlotName: "safe-slot",
|
||||
Content: []byte(content),
|
||||
Digest: "sha256:safe-reference-digest",
|
||||
BindingSource: "safe-binding-source",
|
||||
}},
|
||||
},
|
||||
}},
|
||||
}
|
||||
}
|
||||
|
||||
func assertRedactionTestBindingUnchanged(t *testing.T, binding pipeline.ModuleBinding, name string) {
|
||||
t.Helper()
|
||||
if got := binding.Options["safe"]; got != name+"-safe" {
|
||||
t.Fatalf("source safe option = %v, want %q", got, name+"-safe")
|
||||
}
|
||||
nested := binding.Options["nested"].([]any)[0].([]any)[0].(map[string]any)
|
||||
if got := nested["neighbor"]; got != name+"-nested-safe" {
|
||||
t.Fatalf("source nested safe option = %v, want %q", got, name+"-nested-safe")
|
||||
}
|
||||
}
|
||||
@@ -9,47 +9,49 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
const maxCreateAttempts = 16
|
||||
|
||||
var utcNow = func() time.Time { return time.Now().UTC() }
|
||||
|
||||
type Bundle struct {
|
||||
path, summaryRoot, traceRoot string
|
||||
createdAt time.Time
|
||||
}
|
||||
|
||||
func Allocate(parent string) (*Bundle, error) {
|
||||
func Allocate(parent, runID string, startedAt time.Time) (*Bundle, error) {
|
||||
parent = strings.TrimSpace(parent)
|
||||
if parent == "" {
|
||||
return nil, fmt.Errorf("debug parent must not be empty")
|
||||
}
|
||||
if err := validateRunID(runID); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := os.MkdirAll(parent, 0o700); err != nil {
|
||||
return nil, fmt.Errorf("create debug parent %q: %w", parent, err)
|
||||
}
|
||||
var last string
|
||||
for attempt := 0; attempt < maxCreateAttempts; attempt++ {
|
||||
createdAt := utcNow()
|
||||
runID := fmt.Sprintf("run-%d", createdAt.UnixNano())
|
||||
path := filepath.Join(parent, runID)
|
||||
last = path
|
||||
if err := os.Mkdir(path, 0o700); err != nil {
|
||||
if os.IsExist(err) {
|
||||
continue
|
||||
}
|
||||
return nil, fmt.Errorf("create debug bundle %q: %w", path, err)
|
||||
path := filepath.Join(parent, runID)
|
||||
if err := os.Mkdir(path, 0o700); err != nil {
|
||||
if os.IsExist(err) {
|
||||
return nil, fmt.Errorf("debug bundle %q already exists", path)
|
||||
}
|
||||
summary, trace := filepath.Join(path, "summary"), filepath.Join(path, "trace")
|
||||
if err := os.Mkdir(summary, 0o700); err != nil {
|
||||
_ = os.Remove(path)
|
||||
return nil, fmt.Errorf("create debug summary %q: %w", summary, err)
|
||||
}
|
||||
if err := os.Mkdir(trace, 0o700); err != nil {
|
||||
_ = os.RemoveAll(path)
|
||||
return nil, fmt.Errorf("create debug trace %q: %w", trace, err)
|
||||
}
|
||||
return &Bundle{path: path, summaryRoot: summary, traceRoot: trace, createdAt: createdAt}, nil
|
||||
return nil, fmt.Errorf("create debug bundle %q: %w", path, err)
|
||||
}
|
||||
return nil, fmt.Errorf("create debug bundle %q: exhausted unique run ID attempts", last)
|
||||
summary, trace := filepath.Join(path, "summary"), filepath.Join(path, "trace")
|
||||
if err := os.Mkdir(summary, 0o700); err != nil {
|
||||
_ = os.Remove(path)
|
||||
return nil, fmt.Errorf("create debug summary %q: %w", summary, err)
|
||||
}
|
||||
if err := os.Mkdir(trace, 0o700); err != nil {
|
||||
_ = os.RemoveAll(path)
|
||||
return nil, fmt.Errorf("create debug trace %q: %w", trace, err)
|
||||
}
|
||||
return &Bundle{path: path, summaryRoot: summary, traceRoot: trace, createdAt: startedAt}, nil
|
||||
}
|
||||
|
||||
func validateRunID(runID string) error {
|
||||
if runID == "" {
|
||||
return fmt.Errorf("debug run ID must not be empty")
|
||||
}
|
||||
if runID != strings.TrimSpace(runID) || strings.ContainsAny(runID, `/\\`) || filepath.IsAbs(runID) || filepath.Clean(runID) != runID || runID == "." || runID == ".." {
|
||||
return fmt.Errorf("debug run ID %q must be one safe path component", runID)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (b *Bundle) Path() string {
|
||||
if b == nil {
|
||||
|
||||
@@ -1,23 +1,28 @@
|
||||
package debugbundle
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
)
|
||||
|
||||
const testBundleRunID = "run-42-00000000000000000000000000000001"
|
||||
|
||||
func TestAllocateCreatesRestrictiveSummaryAndTrace(t *testing.T) {
|
||||
parent := t.TempDir()
|
||||
fixed := time.Unix(0, 42).UTC()
|
||||
previous := utcNow
|
||||
utcNow = func() time.Time { return fixed }
|
||||
defer func() { utcNow = previous }()
|
||||
bundle, err := Allocate(parent)
|
||||
bundle, err := Allocate(parent, testBundleRunID, fixed)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if bundle.RunID() != "run-42" || bundle.SummaryRoot() != filepath.Join(bundle.Path(), "summary") || bundle.TraceRoot() != filepath.Join(bundle.Path(), "trace") {
|
||||
if bundle.RunID() != testBundleRunID || bundle.CreatedAt() != fixed || bundle.SummaryRoot() != filepath.Join(bundle.Path(), "summary") || bundle.TraceRoot() != filepath.Join(bundle.Path(), "trace") {
|
||||
t.Fatalf("bundle=%#v", bundle)
|
||||
}
|
||||
for _, path := range []string{bundle.Path(), bundle.SummaryRoot(), bundle.TraceRoot()} {
|
||||
@@ -40,36 +45,119 @@ func TestAllocateCreatesRestrictiveSummaryAndTrace(t *testing.T) {
|
||||
t.Fatalf("file mode=%#o", info.Mode().Perm())
|
||||
}
|
||||
}
|
||||
func TestAllocateRetriesAndDoesNotDeleteBundle(t *testing.T) {
|
||||
func TestAllocateRejectsExistingBundleWithoutChangingIt(t *testing.T) {
|
||||
parent := t.TempDir()
|
||||
fixed := time.Unix(0, 9).UTC()
|
||||
previous := utcNow
|
||||
defer func() { utcNow = previous }()
|
||||
calls := 0
|
||||
utcNow = func() time.Time { calls++; return fixed.Add(time.Duration(calls-1) * time.Nanosecond) }
|
||||
if err := os.Mkdir(filepath.Join(parent, "run-9"), 0o700); err != nil {
|
||||
bundlePath := filepath.Join(parent, testBundleRunID)
|
||||
if err := os.Mkdir(bundlePath, 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
bundle, err := Allocate(parent)
|
||||
if err != nil {
|
||||
sentinelPath := filepath.Join(bundlePath, "sentinel")
|
||||
sentinel := []byte("existing bundle")
|
||||
if err := os.WriteFile(sentinelPath, sentinel, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if bundle.RunID() != "run-10" {
|
||||
t.Fatalf("run id=%q", bundle.RunID())
|
||||
|
||||
if _, err := Allocate(parent, testBundleRunID, time.Unix(0, 42)); err == nil || !strings.Contains(err.Error(), "already exists") {
|
||||
t.Fatalf("Allocate() error = %v, want collision", err)
|
||||
}
|
||||
if _, err := os.Stat(bundle.Path()); err != nil {
|
||||
t.Fatal(err)
|
||||
if got, err := os.ReadFile(sentinelPath); err != nil || !bytes.Equal(got, sentinel) {
|
||||
t.Fatalf("sentinel = %q, %v", got, err)
|
||||
}
|
||||
}
|
||||
func TestSummaryWriterConfinesArtifacts(t *testing.T) {
|
||||
bundle, err := Allocate(t.TempDir())
|
||||
|
||||
func TestAllocateRejectsUnsafeRunIDsBeforeCreatingParent(t *testing.T) {
|
||||
for _, runID := range []string{"", ".", "..", "../escape", `..\\escape`, "/absolute", " trailing "} {
|
||||
t.Run(runID, func(t *testing.T) {
|
||||
parent := filepath.Join(t.TempDir(), "debug")
|
||||
if _, err := Allocate(parent, runID, time.Time{}); err == nil {
|
||||
t.Fatalf("Allocate(%q) succeeded", runID)
|
||||
}
|
||||
if _, err := os.Stat(parent); !os.IsNotExist(err) {
|
||||
t.Fatalf("debug parent exists or stat failed: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSummaryWriterWritesEverySummaryArtifact(t *testing.T) {
|
||||
bundle, err := Allocate(t.TempDir(), testBundleRunID, time.Unix(0, 42))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := bundle.Summary().WriteJSON("../outside.json", map[string]any{}); err == nil {
|
||||
summary := bundle.Summary()
|
||||
if err := summary.WriteInvocation(Invocation{Operation: "run"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := summary.WriteRedactedEffectiveConfig(testRedactedSummaryPayload{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := summary.WriteResolvedPipeline(testRedactedResolvedPipelinePayload{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := summary.WriteResolvedReferences(nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := summary.WriteCheckpointEvents(nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := summary.WriteRunManifest(artifacts.RunManifest{RunID: bundle.RunID()}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := summary.WriteChunkPlan(artifacts.ChunkPlanSummary{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := summary.WriteRunReport(RunReport{RunID: bundle.RunID(), PipelineID: "test"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := summary.WriteWarnings([]contracts.Warning{{ReasonCode: "test"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := summary.WriteError("failed"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
for _, name := range []string{
|
||||
ArtifactInvocationMetadata,
|
||||
ArtifactEffectiveConfig,
|
||||
ArtifactResolvedPipeline,
|
||||
ArtifactResolvedReferences,
|
||||
ArtifactCheckpointEvents,
|
||||
ArtifactRunManifest,
|
||||
ArtifactChunkPlan,
|
||||
ArtifactRunReport,
|
||||
ArtifactWarnings,
|
||||
ArtifactErrorLog,
|
||||
} {
|
||||
info, err := os.Stat(filepath.Join(bundle.SummaryRoot(), name))
|
||||
if err != nil {
|
||||
t.Fatalf("summary artifact %q: %v", name, err)
|
||||
}
|
||||
if info.Mode().Perm() != 0o600 {
|
||||
t.Fatalf("summary artifact %q mode=%#o", name, info.Mode().Perm())
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestSummaryWriterInternalWritesConfineArtifacts(t *testing.T) {
|
||||
bundle, err := Allocate(t.TempDir(), testBundleRunID, time.Unix(0, 42))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := bundle.Summary().writeJSON("../outside.json", map[string]any{}); err == nil {
|
||||
t.Fatal("accepted traversal")
|
||||
}
|
||||
if err := bundle.Summary().WriteBytes(`trace\\x`, []byte("x")); err == nil {
|
||||
if err := bundle.Summary().writeBytes(`trace\\x`, []byte("x")); err == nil {
|
||||
t.Fatal("accepted backslash")
|
||||
}
|
||||
}
|
||||
|
||||
type testRedactedSummaryPayload struct{}
|
||||
|
||||
func (testRedactedSummaryPayload) RedactedSummaryPayload() any {
|
||||
return map[string]any{"redacted": true}
|
||||
}
|
||||
|
||||
type testRedactedResolvedPipelinePayload struct{}
|
||||
|
||||
func (testRedactedResolvedPipelinePayload) RedactedResolvedPipelinePayload() pipeline.ResolvedPipeline {
|
||||
return pipeline.ResolvedPipeline{ID: "redacted"}
|
||||
}
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/fileio"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
)
|
||||
|
||||
const (
|
||||
@@ -24,6 +25,9 @@ const (
|
||||
)
|
||||
|
||||
type RedactedSummaryPayload interface{ RedactedSummaryPayload() any }
|
||||
type RedactedResolvedPipelinePayload interface {
|
||||
RedactedResolvedPipelinePayload() pipeline.ResolvedPipeline
|
||||
}
|
||||
type Invocation struct {
|
||||
Operation string `json:"operation"`
|
||||
PipelineID string `json:"pipeline_id,omitempty"`
|
||||
@@ -63,37 +67,40 @@ func (w *SummaryWriter) WriteInvocation(payload Invocation) error {
|
||||
if payload.StartedAt.IsZero() {
|
||||
payload.StartedAt = w.createdAt
|
||||
}
|
||||
return w.WriteJSON(ArtifactInvocationMetadata, payload)
|
||||
return w.writeJSON(ArtifactInvocationMetadata, payload)
|
||||
}
|
||||
func (w *SummaryWriter) WriteRedactedEffectiveConfig(payload RedactedSummaryPayload) error {
|
||||
if payload == nil {
|
||||
return fmt.Errorf("redacted summary payload must not be nil")
|
||||
}
|
||||
return w.WriteJSON(ArtifactEffectiveConfig, payload.RedactedSummaryPayload())
|
||||
return w.writeJSON(ArtifactEffectiveConfig, payload.RedactedSummaryPayload())
|
||||
}
|
||||
func (w *SummaryWriter) WriteResolvedPipeline(v any) error {
|
||||
return w.WriteJSON(ArtifactResolvedPipeline, v)
|
||||
func (w *SummaryWriter) WriteResolvedPipeline(payload RedactedResolvedPipelinePayload) error {
|
||||
if payload == nil {
|
||||
return fmt.Errorf("redacted resolved pipeline payload must not be nil")
|
||||
}
|
||||
return w.writeJSON(ArtifactResolvedPipeline, payload.RedactedResolvedPipelinePayload())
|
||||
}
|
||||
func (w *SummaryWriter) WriteResolvedReferences(v any) error {
|
||||
return w.WriteJSON(ArtifactResolvedReferences, v)
|
||||
func (w *SummaryWriter) WriteResolvedReferences(v []artifacts.ReferenceProvenance) error {
|
||||
return w.writeJSON(ArtifactResolvedReferences, v)
|
||||
}
|
||||
func (w *SummaryWriter) WriteCheckpointEvents(v any) error {
|
||||
return w.WriteJSON(ArtifactCheckpointEvents, v)
|
||||
func (w *SummaryWriter) WriteCheckpointEvents(v []pipeline.CheckpointEvent) error {
|
||||
return w.writeJSON(ArtifactCheckpointEvents, v)
|
||||
}
|
||||
func (w *SummaryWriter) WriteRunManifest(v artifacts.RunManifest) error {
|
||||
return w.WriteJSON(ArtifactRunManifest, v)
|
||||
return w.writeJSON(ArtifactRunManifest, v)
|
||||
}
|
||||
func (w *SummaryWriter) WriteChunkPlan(v artifacts.ChunkPlanSummary) error {
|
||||
return w.WriteJSON(ArtifactChunkPlan, v)
|
||||
return w.writeJSON(ArtifactChunkPlan, v)
|
||||
}
|
||||
func (w *SummaryWriter) WriteRunReport(v RunReport) error { return w.WriteJSON(ArtifactRunReport, v) }
|
||||
func (w *SummaryWriter) WriteRunReport(v RunReport) error { return w.writeJSON(ArtifactRunReport, v) }
|
||||
func (w *SummaryWriter) WriteWarnings(v []contracts.Warning) error {
|
||||
return w.WriteJSON(ArtifactWarnings, v)
|
||||
return w.writeJSON(ArtifactWarnings, v)
|
||||
}
|
||||
func (w *SummaryWriter) WriteError(message string) error {
|
||||
return w.WriteBytes(ArtifactErrorLog, []byte(message+"\n"))
|
||||
return w.writeBytes(ArtifactErrorLog, []byte(message+"\n"))
|
||||
}
|
||||
func (w *SummaryWriter) WriteJSON(name string, v any) error {
|
||||
func (w *SummaryWriter) writeJSON(name string, v any) error {
|
||||
if w == nil {
|
||||
return fmt.Errorf("debug summary writer must not be nil")
|
||||
}
|
||||
@@ -102,7 +109,7 @@ func (w *SummaryWriter) WriteJSON(name string, v any) error {
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (w *SummaryWriter) WriteBytes(name string, v []byte) error {
|
||||
func (w *SummaryWriter) writeBytes(name string, v []byte) error {
|
||||
if w == nil {
|
||||
return fmt.Errorf("debug summary writer must not be nil")
|
||||
}
|
||||
|
||||
@@ -19,7 +19,7 @@ func SafePath(root, name string) (string, error) {
|
||||
if name == "" {
|
||||
return "", fmt.Errorf("artifact name must not be empty")
|
||||
}
|
||||
if strings.Contains(name, `\\`) {
|
||||
if strings.ContainsRune(name, '\\') {
|
||||
return "", fmt.Errorf("artifact name %q must use slash-separated relative paths", name)
|
||||
}
|
||||
if path.IsAbs(name) || filepath.IsAbs(name) {
|
||||
@@ -46,6 +46,9 @@ func SafePath(root, name string) (string, error) {
|
||||
if rel == "." || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
return "", fmt.Errorf("artifact name %q resolves outside file root", name)
|
||||
}
|
||||
if err := rejectSymlinkComponents(absRoot, name); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return target, nil
|
||||
}
|
||||
|
||||
@@ -62,16 +65,44 @@ func WriteBytes(root, name string, data []byte, dirMode, fileMode os.FileMode) e
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := writeAtomic(target, data, dirMode, fileMode); err != nil {
|
||||
if err := os.MkdirAll(filepath.Dir(target), dirMode); err != nil {
|
||||
return fmt.Errorf("write artifact %q: %w", name, err)
|
||||
}
|
||||
if err := rejectSymlinkComponents(root, name); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := writeAtomic(target, data, fileMode); err != nil {
|
||||
return fmt.Errorf("write artifact %q: %w", name, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func writeAtomic(target string, data []byte, dirMode, fileMode os.FileMode) error {
|
||||
if err := os.MkdirAll(filepath.Dir(target), dirMode); err != nil {
|
||||
return err
|
||||
func rejectSymlinkComponents(root, name string) error {
|
||||
absRoot, err := filepath.Abs(root)
|
||||
if err != nil {
|
||||
return fmt.Errorf("resolve file root %q: %w", root, err)
|
||||
}
|
||||
current := absRoot
|
||||
for _, component := range strings.Split(filepath.FromSlash(name), string(filepath.Separator)) {
|
||||
if component == "" || component == "." {
|
||||
continue
|
||||
}
|
||||
current = filepath.Join(current, component)
|
||||
info, err := os.Lstat(current)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("inspect artifact path %q: %w", name, err)
|
||||
}
|
||||
if info.Mode()&os.ModeSymlink != 0 {
|
||||
return fmt.Errorf("artifact path %q must not traverse symbolic links", name)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func writeAtomic(target string, data []byte, fileMode os.FileMode) error {
|
||||
temp, err := os.CreateTemp(filepath.Dir(target), "."+filepath.Base(target)+".tmp-*")
|
||||
if err != nil {
|
||||
return err
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
)
|
||||
|
||||
func TestSafePathRejectsUnsafeNames(t *testing.T) {
|
||||
for _, name := range []string{"/tmp/x", "a/../x", "a//x", `a\\x`} {
|
||||
for _, name := range []string{"/tmp/x", "a/../x", "a//x", `a\x`, `a\\x`} {
|
||||
if _, err := SafePath(t.TempDir(), name); err == nil {
|
||||
t.Fatalf("SafePath(%q) accepted unsafe path", name)
|
||||
}
|
||||
@@ -39,3 +39,18 @@ func TestWriteBytesIsAtomicAndUsesRequestedModes(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteBytesRejectsSymlinkedComponents(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
outside := t.TempDir()
|
||||
if err := os.Symlink(outside, filepath.Join(root, "link")); err != nil {
|
||||
t.Skipf("symbolic links unavailable: %v", err)
|
||||
}
|
||||
|
||||
if err := WriteBytes(root, "link/value", []byte("value"), 0o700, 0o600); err == nil {
|
||||
t.Fatal("WriteBytes accepted a symlinked directory")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(outside, "value")); !os.IsNotExist(err) {
|
||||
t.Fatalf("write escaped through symlink: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,9 +1,13 @@
|
||||
package debug
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
)
|
||||
|
||||
func TestFilesystemRecorderWritesWithinTraceRoot(t *testing.T) {
|
||||
@@ -26,3 +30,50 @@ func TestFilesystemRecorderWritesWithinTraceRoot(t *testing.T) {
|
||||
t.Fatal("accepted traversal")
|
||||
}
|
||||
}
|
||||
|
||||
func TestFilesystemRecorderPropagatesWriteFailures(t *testing.T) {
|
||||
root := filepath.Join(t.TempDir(), "trace")
|
||||
if err := os.WriteFile(root, []byte("not a directory"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
recorder, err := NewFilesystemRecorder(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := recorder.WriteBytes("attempt/data", []byte("payload")); err == nil {
|
||||
t.Fatal("WriteBytes concealed a trace-root write failure")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSynchronizedFilesystemRecorderSupportsConcurrentWrites(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
recorder, err := NewFilesystemRecorder(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
recorder = pipeline.SynchronizedDebugRecorder(recorder)
|
||||
|
||||
const writes = 16
|
||||
var wg sync.WaitGroup
|
||||
errs := make(chan error, writes)
|
||||
for i := 0; i < writes; i++ {
|
||||
i := i
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
errs <- recorder.WriteBytes(fmt.Sprintf("attempt/%02d/data", i), []byte("payload"))
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
close(errs)
|
||||
for err := range errs {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for i := 0; i < writes; i++ {
|
||||
if _, err := os.Stat(filepath.Join(root, "attempt", fmt.Sprintf("%02d", i), "data")); err != nil {
|
||||
t.Fatalf("trace artifact %d: %v", i, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user