Document chunk plan caching and provenance
This commit is contained in:
@@ -57,6 +57,10 @@ Implemented diagnostics artifacts:
|
||||
binding source, without reference content.
|
||||
- `checkpoint-events.json`: checkpoint steps that were reused or executed
|
||||
during an explicit resume invocation.
|
||||
- `chunk-plan.json`: redacted plan-cache lookup, validation, and publication
|
||||
summary. It contains identifiers and decisions, never source units, plan
|
||||
annotations, reference content, prompts, model responses, or invalid-file
|
||||
bytes.
|
||||
- `run-manifest.json`: the same run manifest written to durable output when it
|
||||
is available, including top-level module metadata when present.
|
||||
- `warnings.json`: warning list.
|
||||
@@ -64,6 +68,54 @@ Implemented diagnostics artifacts:
|
||||
- `error.log`: failure message, written after diagnostics directory creation
|
||||
when a run fails.
|
||||
|
||||
## Chunk-Plan Cache
|
||||
|
||||
The chunk-plan cache is independent of the workspace and checkpoints. Its
|
||||
configuration and selection precedence are defined in
|
||||
[Configuration](config.md#workspace); the invocation override is documented in
|
||||
the [CLI reference](cli.md#run).
|
||||
|
||||
When no root is configured, a normal Linux user uses
|
||||
`$XDG_CACHE_HOME/notarius/chunk-plans` when `XDG_CACHE_HOME` is a valid absolute
|
||||
path, or `$HOME/.cache/notarius/chunk-plans` when it is unset. A relative
|
||||
`XDG_CACHE_HOME` is a configuration error. A configured
|
||||
`workspace.chunk_cache.directory` is the root itself, not a parent to which
|
||||
Notarius adds a suffix.
|
||||
|
||||
Each source digest has one file:
|
||||
|
||||
```text
|
||||
<chunk-plan-root>/<source-sha256-hex>/plan.json
|
||||
```
|
||||
|
||||
Directories are created with `0700` permissions and plan files with `0600`.
|
||||
`auto` reuses a complete valid plan or regenerates an absent or invalid one;
|
||||
`refresh` deliberately regenerates; `bypass` performs no cache I/O. A stored
|
||||
plan is still validated and materialized against the current source before use,
|
||||
and the current run's chunk validators always run. Invalid state is recoverable:
|
||||
an `auto` run regenerates and atomically replaces it only after validation
|
||||
succeeds. Delete an exact cache root or digest directory only when regeneration
|
||||
cost is acceptable.
|
||||
|
||||
Publication uses atomic replacement. Concurrent readers observe a complete old
|
||||
or new plan, and concurrent writers leave one complete valid winner; there is
|
||||
no history, lock protocol, or rollback facility. Do not share a cache root
|
||||
between mutually untrusted users because plans can contain source-derived
|
||||
structure and annotations.
|
||||
|
||||
For a system-wide Linux deployment under a dedicated service account, configure
|
||||
and provision a separate restrictive root such as:
|
||||
|
||||
```yaml
|
||||
workspace:
|
||||
chunk_cache:
|
||||
directory: /var/cache/notarius/chunk-plans
|
||||
```
|
||||
|
||||
`/var/cache/notarius/chunk-plans` is a recommended configured service root, not
|
||||
the unprivileged default. The operator or package installer must create it with
|
||||
restrictive service-account ownership and permissions before use.
|
||||
|
||||
## Checkpoints
|
||||
|
||||
When checkpoint writing is enabled for a configured workspace, runs write
|
||||
@@ -158,8 +210,12 @@ failure before a candidate exists has no candidate payload. If the envelope
|
||||
cannot be persisted, the run does not retry that module attempt and reports the
|
||||
debug failure together with any primary attempt error.
|
||||
|
||||
Checkpoint-reused chunk, extract, merge, and normalize work retains the
|
||||
stage-level input and output artifacts but has no retry-attempt artifacts
|
||||
Chunk-plan candidates, materialized chunks, annotations, and chunk-attempt
|
||||
details appear only in these opt-in debug artifacts. They are intentionally not
|
||||
included in normal manifests or the `chunk-plan.json` diagnostics summary.
|
||||
|
||||
Checkpoint-reused extract, merge, and normalize work retains the stage-level
|
||||
input and output artifacts but has no retry-attempt artifacts
|
||||
because no module attempt executed. Debug artifacts may contain source
|
||||
material, reference material, prompt inputs, model outputs, and other sensitive
|
||||
data. Typed artifact
|
||||
@@ -218,6 +274,10 @@ rm -rf /var/lib/notarius/checkpoints/dnd-session/seriatim-abcdef123456/7890abcd1
|
||||
rm -rf /var/lib/notarius/debug/run-1234567890
|
||||
```
|
||||
|
||||
Chunk-plan cache entries can likewise be removed by exact digest directory or
|
||||
configured root. Removal is recoverable, but the next non-bypass run may need
|
||||
to regenerate plans and repeat any chunk-stage LLM work.
|
||||
|
||||
Use exact run-directory paths. Avoid broad cleanup commands against parent
|
||||
directories unless they are part of your own operational policy.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user