Document output cache and debug state model

This commit is contained in:
2026-07-18 13:08:27 +00:00
parent 8cb11e60e4
commit 8824948910
14 changed files with 354 additions and 1478 deletions

View File

@@ -114,8 +114,8 @@ plan, while the current run still applies its configured chunk validators to
the materialized chunks.
The framework owns orchestration and handoff provenance. Modules return logical
results and warnings; they do not own CLI reporting, workspace paths, durable
file placement, checkpoints, or diagnostics.
results and warnings; they do not own CLI reporting, physical output, cache, or
debug roots, durable file placement, or checkpoint and debug lifecycle.
After pipeline-wide chunking, extraction uses bounded framework concurrency.
One run-wide worker pool receives chunk-scoped lane jobs in deterministic
@@ -188,31 +188,32 @@ record identities and summaries rather than secret or large payload content.
## State, Output, And Safety
Durable output, diagnostics, checkpoints, and debug artifacts are separate
surfaces with separate ownership:
Notarius exposes three filesystem surfaces with independent roots and
lifecycle:
- output modules define logical durable output; the application boundary owns
filesystem placement;
- diagnostics provide redacted run inspection and are not the durable output
contract;
- checkpoints support validated stage reuse and are not diagnostics;
- debug artifacts are opt-in inspection data and may contain sensitive source,
prompt, reference, and model-output content.
- output is durable user data; output modules define logical files and the CLI
owns their placement;
- cache is reconstructible state, with separate chunk-plan and checkpoint
families; and
- debug is explicitly requested inspection data, combining a redacted summary
with a detailed trace.
Chunk-plan cache state is an additional independent surface. It is keyed only
by canonical source digest, is not rooted under `workspace.directory`, and is
not a checkpoint or a diagnostic. A cache record is atomically replaced as one
complete plan envelope; it has no history, locking, or rollback interface.
Invalid records are recoverable cache misses rather than pipeline state.
Chunk plans are keyed only by canonical source digest. Checkpoints are used only
for an invocation that explicitly requests resume. Debug is never a cache input
and is never created without an explicit request. Pipeline modules receive
collaborator interfaces and never physical roots.
Writes of durable state are atomic where practical. Paths for writes, moves,
overwrites, and deletion must be narrow and explicit. Cleanup that can lose data
is opt-in.
Writes are atomic where practical. Paths for writes, moves, overwrites, and
deletion must be narrow and explicit. Notarius never automatically deletes
output or requested debug bundles; cache cleanup is explicit and recoverable.
Secrets must not appear in errors, logs, diagnostics, manifests,
documentation, examples, or redacted configuration. Default logs and
diagnostics must not include large source, prompt, reference, or artifact
payloads.
Secrets must not appear in errors, logs, output, cache, debug summaries,
traces, manifests, documentation, examples, or redacted configuration. Debug
collection is allowlisted to application-owned payloads and must not capture
unrelated process environment values or filesystem content. Trace data may
contain application data and therefore inherits its sensitivity; operators own
access controls and retention. Physical layout and operation are defined in
[Operations](../operations.md).
## Architectural Non-Goals

View File

@@ -63,7 +63,7 @@ secret values.
| Documentation organization | `docs/policy/documentation.md` | Documentation ownership, audience boundaries, maintenance rules, and ADR/document lifecycle. | Application architecture or product behavior. |
| CLI contract | `docs/cli.md` | Commands, arguments, flags, invocation semantics, and exit codes. | End-to-end operating procedures, configuration field definitions, runtime filesystem layout, module implementation details. |
| Configuration contract | `docs/config.md` | Discovery and precedence, file schema, fields, defaults, environment overrides, validation rules, and user-selectable module or validator keys. | Complete example files, CLI syntax, runtime state lifecycle, module implementation details. |
| Operations | `docs/operations.md` | Runtime workflows, physical filesystem and state layout, diagnostics use, retention, resume, cleanup, permissions, recovery, and operational limits. | CLI flag syntax, configuration field definitions, logical output schemas, implementation mechanics. |
| Operations | `docs/operations.md` | Runtime workflows, physical filesystem and state layout, output, cache, and debug handling, resume, cleanup, permissions, recovery, and operational limits. | CLI flag syntax, configuration field definitions, logical output schemas, implementation mechanics. |
| Public HTTP contract, if introduced | `docs/api.md` | Routes, authentication, media types, request and response schemas, status codes, pagination, caching, idempotency, rate limits, and HTTP retry semantics. | Client walkthroughs, upstream or downstream integration internals, implementation detail. |
| Consumer guidance, if a public package or API is introduced | `docs/consumers/` | Task-oriented use of the public interface, minimal client examples, and consumer responsibilities. | HTTP wire semantics, external protocol contracts, internal implementation detail. |
| External and durable integration contracts | `docs/integrations/` | External file formats and protocols, upstream and downstream contracts, logical output bundle paths and schemas, media types, and compatibility behavior. | Physical runtime placement and lifecycle, internal transformations, CLI syntax, configuration defaults. |