eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document chunk plan remediation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-18 02:12:28 +00:00
parent 561d65a505
commit a0f5e6e2b9
7 changed files with 26 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/internal/diagnostics.md
View File

@@ -32,7 +32,10 @@ digests, requested module, lookup decision, materialization action, validation
decision, and publication decision. Its closed decision values make failures
and recoverable invalid records inspectable without serializing plan ranges,
annotations, source content, reference content, prompts, model responses, or
raw invalid-file bytes.
raw invalid-file bytes. Lookup reasons are derived only from the lookup status:
`stored chunk plan is valid`, `chunk plan not found`, `stored chunk plan is
invalid`, or `chunk plan lookup skipped`. Store-provided reasons and malformed
record details never enter this artifact.
JSON methods indent their payload and append a newline. All artifact writes use
a temporary file in the target directory, apply the requested permissions, and

3
docs/internal/modules.md
View File

@@ -71,7 +71,8 @@ Chunkers implement `contracts.Chunker.Plan`. A plan identifies ordered source
unit ranges and may carry optional namespaced JSON annotations; it does not
contain materialized chunk content. The framework canonicalizes annotations,
validates ranges against the current source, and materializes chunk IDs,
indexes, references, content, units, and generic metadata. Annotation
indexes, references, content, units, and generic metadata. Materialized source
unit metadata is independently owned. Annotation
namespaces remain optional data: generic framework code and downstream modules
must not require D&D scene annotations or import `dnd/scenes`.

2
docs/internal/overview.md
View File

@@ -34,7 +34,7 @@ normalize continuations that may overlap across lanes.
| `internal/core/artifacts` | Run-manifest and provenance models. |
| `internal/core/config` | Defaults, YAML parsing, environment overrides, validation, redaction, and effective pipeline resolution. |
| `internal/core/diagnostics` | Scoped run directories, diagnostics writers, atomic writes, and retention decisions. |
| `internal/core/source` | Generic source documents, units, chunks, canonical references, lookup, validation, and deterministic source digests. |
| `internal/core/source` | Generic source documents, units, chunks, canonical references, validation, deterministic source digests, and independent metadata materialization. |
| `internal/core/workspace` | Effective workspace settings, confined paths and writes, and checkpoint identity and manifest models. |
## Framework Packages

5
docs/internal/pipeline.md
View File

@@ -214,7 +214,10 @@ Plan canonicalization requires canonical JSON annotations, a matching source
digest, at least one range, existing ordered boundaries, and increasing range
starts. Ranges may overlap or leave gaps; a chunker may impose stricter policy.
Materialization deterministically reconstructs each range from the current
source document and copies annotations without interpreting their namespaces.
source document, deep-clones JSON-shaped source-unit metadata, and copies
annotations without interpreting their namespaces. Materialized chunks and
separate materializations do not share mutable unit metadata; unsupported or
cyclic metadata fails materialization with context.
Before lane execution, generic chunk validation checks the materialized chunks'
identities, order, source references, content, media type, units, and metadata.