Document chunk plan remediation
This commit is contained in:
@@ -32,7 +32,10 @@ digests, requested module, lookup decision, materialization action, validation
|
||||
decision, and publication decision. Its closed decision values make failures
|
||||
and recoverable invalid records inspectable without serializing plan ranges,
|
||||
annotations, source content, reference content, prompts, model responses, or
|
||||
raw invalid-file bytes.
|
||||
raw invalid-file bytes. Lookup reasons are derived only from the lookup status:
|
||||
`stored chunk plan is valid`, `chunk plan not found`, `stored chunk plan is
|
||||
invalid`, or `chunk plan lookup skipped`. Store-provided reasons and malformed
|
||||
record details never enter this artifact.
|
||||
|
||||
JSON methods indent their payload and append a newline. All artifact writes use
|
||||
a temporary file in the target directory, apply the requested permissions, and
|
||||
|
||||
@@ -71,7 +71,8 @@ Chunkers implement `contracts.Chunker.Plan`. A plan identifies ordered source
|
||||
unit ranges and may carry optional namespaced JSON annotations; it does not
|
||||
contain materialized chunk content. The framework canonicalizes annotations,
|
||||
validates ranges against the current source, and materializes chunk IDs,
|
||||
indexes, references, content, units, and generic metadata. Annotation
|
||||
indexes, references, content, units, and generic metadata. Materialized source
|
||||
unit metadata is independently owned. Annotation
|
||||
namespaces remain optional data: generic framework code and downstream modules
|
||||
must not require D&D scene annotations or import `dnd/scenes`.
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ normalize continuations that may overlap across lanes.
|
||||
| `internal/core/artifacts` | Run-manifest and provenance models. |
|
||||
| `internal/core/config` | Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline resolution. |
|
||||
| `internal/core/diagnostics` | Scoped run directories, diagnostics writers, atomic writes, and retention decisions. |
|
||||
| `internal/core/source` | Generic source documents, units, chunks, canonical references, lookup, validation, and deterministic source digests. |
|
||||
| `internal/core/source` | Generic source documents, units, chunks, canonical references, validation, deterministic source digests, and independent metadata materialization. |
|
||||
| `internal/core/workspace` | Effective workspace settings, confined paths and writes, and checkpoint identity and manifest models. |
|
||||
|
||||
## Framework Packages
|
||||
|
||||
@@ -214,7 +214,10 @@ Plan canonicalization requires canonical JSON annotations, a matching source
|
||||
digest, at least one range, existing ordered boundaries, and increasing range
|
||||
starts. Ranges may overlap or leave gaps; a chunker may impose stricter policy.
|
||||
Materialization deterministically reconstructs each range from the current
|
||||
source document and copies annotations without interpreting their namespaces.
|
||||
source document, deep-clones JSON-shaped source-unit metadata, and copies
|
||||
annotations without interpreting their namespaces. Materialized chunks and
|
||||
separate materializations do not share mutable unit metadata; unsupported or
|
||||
cyclic metadata fails materialization with context.
|
||||
|
||||
Before lane execution, generic chunk validation checks the materialized chunks'
|
||||
identities, order, source references, content, media type, units, and metadata.
|
||||
|
||||
@@ -97,6 +97,11 @@ an `auto` run regenerates and atomically replaces it only after validation
|
||||
succeeds. Delete an exact cache root or digest directory only when regeneration
|
||||
cost is acceptable.
|
||||
|
||||
The configured root is the cache trust boundary. An operator-supplied root path
|
||||
may itself resolve through a symlink, but cache-owned digest directories and
|
||||
plan files must be real directory and regular-file entries. Links or other
|
||||
unexpected entry types are rejected rather than followed.
|
||||
|
||||
Publication uses atomic replacement. Concurrent readers observe a complete old
|
||||
or new plan, and concurrent writers leave one complete valid winner; there is
|
||||
no history, lock protocol, or rollback facility. Do not share a cache root
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
# ADR-0005 Feature Roadmap
|
||||
|
||||
This roadmap defines the intended end state for
|
||||
[ADR-0005](../adr/0005-cache-canonical-chunk-plans-by-source.md). The feature
|
||||
is not implemented. The ordered coding work belongs in the
|
||||
[implementation plan](implementation.md).
|
||||
This roadmap records the implemented target state for
|
||||
[ADR-0005](../adr/0005-cache-canonical-chunk-plans-by-source.md). The
|
||||
[implementation record](implementation.md) preserves the completed work.
|
||||
|
||||
## Intent
|
||||
|
||||
@@ -156,11 +155,11 @@ This feature does not provide:
|
||||
|
||||
## Completion Outcomes
|
||||
|
||||
The feature is complete when independent runs over the same canonical source
|
||||
**Achieved.** Independent runs over the same canonical source
|
||||
reuse byte-stable materialized chunks across pipeline, lane, reference,
|
||||
chunk-module, and LLM configuration changes; bypass and refresh obey their
|
||||
documented state semantics; provenance identifies the effective producer;
|
||||
legacy chunk checkpoints cannot compete with plan reuse; and all focused,
|
||||
integration, compatibility, CLI, and repository-wide validation passes. The
|
||||
integration, compatibility, CLI, and repository-wide validation pass. The
|
||||
configuration and operations references must also document both the per-user
|
||||
default and the recommended system-wide Linux setting.
|
||||
|
||||
@@ -5,10 +5,11 @@ This document records the implementation of the target state in
|
||||
[ADR-0005](../adr/0005-cache-canonical-chunk-plans-by-source.md).
|
||||
|
||||
Stages 1 through 8 summarize the completed initial implementation. Stages 9
|
||||
through 12 track pending remediation identified by the post-implementation
|
||||
review. Current behavior is documented in the canonical references linked from
|
||||
[Development](../development.md); those references must not describe the Stages
|
||||
9 through 12 target state until the corresponding stage is complete.
|
||||
through 12 record remediation identified by the post-implementation review and
|
||||
retain their individual completion status. Current behavior is documented in the
|
||||
canonical references linked from [Development](../development.md); those
|
||||
references must not describe a remediation target state until its corresponding
|
||||
work is complete.
|
||||
|
||||
## Execution Rules
|
||||
|
||||
|
||||
Reference in New Issue
Block a user