Document chunk plan caching and provenance

This commit is contained in:
2026-07-18 00:57:51 +00:00
parent 6d0a19c94c
commit 96a49bb7cd
12 changed files with 235 additions and 44 deletions

View File

@@ -57,6 +57,10 @@ Implemented diagnostics artifacts:
binding source, without reference content.
- `checkpoint-events.json`: checkpoint steps that were reused or executed
during an explicit resume invocation.
- `chunk-plan.json`: redacted plan-cache lookup, validation, and publication
summary. It contains identifiers and decisions, never source units, plan
annotations, reference content, prompts, model responses, or invalid-file
bytes.
- `run-manifest.json`: the same run manifest written to durable output when it
is available, including top-level module metadata when present.
- `warnings.json`: warning list.
@@ -64,6 +68,54 @@ Implemented diagnostics artifacts:
- `error.log`: failure message, written after diagnostics directory creation
when a run fails.
## Chunk-Plan Cache
The chunk-plan cache is independent of the workspace and checkpoints. Its
configuration and selection precedence are defined in
[Configuration](config.md#workspace); the invocation override is documented in
the [CLI reference](cli.md#run).
When no root is configured, a normal Linux user uses
`$XDG_CACHE_HOME/notarius/chunk-plans` when `XDG_CACHE_HOME` is a valid absolute
path, or `$HOME/.cache/notarius/chunk-plans` when it is unset. A relative
`XDG_CACHE_HOME` is a configuration error. A configured
`workspace.chunk_cache.directory` is the root itself, not a parent to which
Notarius adds a suffix.
Each source digest has one file:
```text
<chunk-plan-root>/<source-sha256-hex>/plan.json
```
Directories are created with `0700` permissions and plan files with `0600`.
`auto` reuses a complete valid plan or regenerates an absent or invalid one;
`refresh` deliberately regenerates; `bypass` performs no cache I/O. A stored
plan is still validated and materialized against the current source before use,
and the current run's chunk validators always run. Invalid state is recoverable:
an `auto` run regenerates and atomically replaces it only after validation
succeeds. Delete an exact cache root or digest directory only when regeneration
cost is acceptable.
Publication uses atomic replacement. Concurrent readers observe a complete old
or new plan, and concurrent writers leave one complete valid winner; there is
no history, lock protocol, or rollback facility. Do not share a cache root
between mutually untrusted users because plans can contain source-derived
structure and annotations.
For a system-wide Linux deployment under a dedicated service account, configure
and provision a separate restrictive root such as:
```yaml
workspace:
chunk_cache:
directory: /var/cache/notarius/chunk-plans
```
`/var/cache/notarius/chunk-plans` is a recommended configured service root, not
the unprivileged default. The operator or package installer must create it with
restrictive service-account ownership and permissions before use.
## Checkpoints
When checkpoint writing is enabled for a configured workspace, runs write
@@ -158,8 +210,12 @@ failure before a candidate exists has no candidate payload. If the envelope
cannot be persisted, the run does not retry that module attempt and reports the
debug failure together with any primary attempt error.
Checkpoint-reused chunk, extract, merge, and normalize work retains the
stage-level input and output artifacts but has no retry-attempt artifacts
Chunk-plan candidates, materialized chunks, annotations, and chunk-attempt
details appear only in these opt-in debug artifacts. They are intentionally not
included in normal manifests or the `chunk-plan.json` diagnostics summary.
Checkpoint-reused extract, merge, and normalize work retains the stage-level
input and output artifacts but has no retry-attempt artifacts
because no module attempt executed. Debug artifacts may contain source
material, reference material, prompt inputs, model outputs, and other sensitive
data. Typed artifact
@@ -218,6 +274,10 @@ rm -rf /var/lib/notarius/checkpoints/dnd-session/seriatim-abcdef123456/7890abcd1
rm -rf /var/lib/notarius/debug/run-1234567890
```
Chunk-plan cache entries can likewise be removed by exact digest directory or
configured root. Removal is recoverable, but the next non-bypass run may need
to regenerate plans and repeat any chunk-stage LLM work.
Use exact run-directory paths. Avoid broad cleanup commands against parent
directories unless they are part of your own operational policy.