Document chunk plan remediation

This commit is contained in:
2026-07-18 02:12:28 +00:00
parent 561d65a505
commit a0f5e6e2b9
7 changed files with 26 additions and 14 deletions

View File

@@ -32,7 +32,10 @@ digests, requested module, lookup decision, materialization action, validation
decision, and publication decision. Its closed decision values make failures decision, and publication decision. Its closed decision values make failures
and recoverable invalid records inspectable without serializing plan ranges, and recoverable invalid records inspectable without serializing plan ranges,
annotations, source content, reference content, prompts, model responses, or annotations, source content, reference content, prompts, model responses, or
raw invalid-file bytes. raw invalid-file bytes. Lookup reasons are derived only from the lookup status:
`stored chunk plan is valid`, `chunk plan not found`, `stored chunk plan is
invalid`, or `chunk plan lookup skipped`. Store-provided reasons and malformed
record details never enter this artifact.
JSON methods indent their payload and append a newline. All artifact writes use JSON methods indent their payload and append a newline. All artifact writes use
a temporary file in the target directory, apply the requested permissions, and a temporary file in the target directory, apply the requested permissions, and

View File

@@ -71,7 +71,8 @@ Chunkers implement `contracts.Chunker.Plan`. A plan identifies ordered source
unit ranges and may carry optional namespaced JSON annotations; it does not unit ranges and may carry optional namespaced JSON annotations; it does not
contain materialized chunk content. The framework canonicalizes annotations, contain materialized chunk content. The framework canonicalizes annotations,
validates ranges against the current source, and materializes chunk IDs, validates ranges against the current source, and materializes chunk IDs,
indexes, references, content, units, and generic metadata. Annotation indexes, references, content, units, and generic metadata. Materialized source
unit metadata is independently owned. Annotation
namespaces remain optional data: generic framework code and downstream modules namespaces remain optional data: generic framework code and downstream modules
must not require D&D scene annotations or import `dnd/scenes`. must not require D&D scene annotations or import `dnd/scenes`.

View File

@@ -34,7 +34,7 @@ normalize continuations that may overlap across lanes.
| `internal/core/artifacts` | Run-manifest and provenance models. | | `internal/core/artifacts` | Run-manifest and provenance models. |
| `internal/core/config` | Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline resolution. | | `internal/core/config` | Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline resolution. |
| `internal/core/diagnostics` | Scoped run directories, diagnostics writers, atomic writes, and retention decisions. | | `internal/core/diagnostics` | Scoped run directories, diagnostics writers, atomic writes, and retention decisions. |
| `internal/core/source` | Generic source documents, units, chunks, canonical references, lookup, validation, and deterministic source digests. | | `internal/core/source` | Generic source documents, units, chunks, canonical references, validation, deterministic source digests, and independent metadata materialization. |
| `internal/core/workspace` | Effective workspace settings, confined paths and writes, and checkpoint identity and manifest models. | | `internal/core/workspace` | Effective workspace settings, confined paths and writes, and checkpoint identity and manifest models. |
## Framework Packages ## Framework Packages

View File

@@ -214,7 +214,10 @@ Plan canonicalization requires canonical JSON annotations, a matching source
digest, at least one range, existing ordered boundaries, and increasing range digest, at least one range, existing ordered boundaries, and increasing range
starts. Ranges may overlap or leave gaps; a chunker may impose stricter policy. starts. Ranges may overlap or leave gaps; a chunker may impose stricter policy.
Materialization deterministically reconstructs each range from the current Materialization deterministically reconstructs each range from the current
source document and copies annotations without interpreting their namespaces. source document, deep-clones JSON-shaped source-unit metadata, and copies
annotations without interpreting their namespaces. Materialized chunks and
separate materializations do not share mutable unit metadata; unsupported or
cyclic metadata fails materialization with context.
Before lane execution, generic chunk validation checks the materialized chunks' Before lane execution, generic chunk validation checks the materialized chunks'
identities, order, source references, content, media type, units, and metadata. identities, order, source references, content, media type, units, and metadata.

View File

@@ -97,6 +97,11 @@ an `auto` run regenerates and atomically replaces it only after validation
succeeds. Delete an exact cache root or digest directory only when regeneration succeeds. Delete an exact cache root or digest directory only when regeneration
cost is acceptable. cost is acceptable.
The configured root is the cache trust boundary. An operator-supplied root path
may itself resolve through a symlink, but cache-owned digest directories and
plan files must be real directory and regular-file entries. Links or other
unexpected entry types are rejected rather than followed.
Publication uses atomic replacement. Concurrent readers observe a complete old Publication uses atomic replacement. Concurrent readers observe a complete old
or new plan, and concurrent writers leave one complete valid winner; there is or new plan, and concurrent writers leave one complete valid winner; there is
no history, lock protocol, or rollback facility. Do not share a cache root no history, lock protocol, or rollback facility. Do not share a cache root

View File

@@ -1,9 +1,8 @@
# ADR-0005 Feature Roadmap # ADR-0005 Feature Roadmap
This roadmap defines the intended end state for This roadmap records the implemented target state for
[ADR-0005](../adr/0005-cache-canonical-chunk-plans-by-source.md). The feature [ADR-0005](../adr/0005-cache-canonical-chunk-plans-by-source.md). The
is not implemented. The ordered coding work belongs in the [implementation record](implementation.md) preserves the completed work.
[implementation plan](implementation.md).
## Intent ## Intent
@@ -156,11 +155,11 @@ This feature does not provide:
## Completion Outcomes ## Completion Outcomes
The feature is complete when independent runs over the same canonical source **Achieved.** Independent runs over the same canonical source
reuse byte-stable materialized chunks across pipeline, lane, reference, reuse byte-stable materialized chunks across pipeline, lane, reference,
chunk-module, and LLM configuration changes; bypass and refresh obey their chunk-module, and LLM configuration changes; bypass and refresh obey their
documented state semantics; provenance identifies the effective producer; documented state semantics; provenance identifies the effective producer;
legacy chunk checkpoints cannot compete with plan reuse; and all focused, legacy chunk checkpoints cannot compete with plan reuse; and all focused,
integration, compatibility, CLI, and repository-wide validation passes. The integration, compatibility, CLI, and repository-wide validation pass. The
configuration and operations references must also document both the per-user configuration and operations references must also document both the per-user
default and the recommended system-wide Linux setting. default and the recommended system-wide Linux setting.

View File

@@ -5,10 +5,11 @@ This document records the implementation of the target state in
[ADR-0005](../adr/0005-cache-canonical-chunk-plans-by-source.md). [ADR-0005](../adr/0005-cache-canonical-chunk-plans-by-source.md).
Stages 1 through 8 summarize the completed initial implementation. Stages 9 Stages 1 through 8 summarize the completed initial implementation. Stages 9
through 12 track pending remediation identified by the post-implementation through 12 record remediation identified by the post-implementation review and
review. Current behavior is documented in the canonical references linked from retain their individual completion status. Current behavior is documented in the
[Development](../development.md); those references must not describe the Stages canonical references linked from [Development](../development.md); those
9 through 12 target state until the corresponding stage is complete. references must not describe a remediation target state until its corresponding
work is complete.
## Execution Rules ## Execution Rules