eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement final fixes and close out the implemetation roadmap

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-17 23:16:16 -05:00
parent 604c7a7945
commit 9fa9154dda
14 changed files with 463 additions and 879 deletions
Download Patch File Download Diff File Expand all files Collapse all files

165
docs/roadmap/adr0005.md
View File

@@ -1,165 +0,0 @@
# ADR-0005 Feature Roadmap
This roadmap records the implemented target state for
[ADR-0005](../adr/0005-cache-canonical-chunk-plans-by-source.md). The
[implementation record](implementation.md) preserves the completed work.
## Intent
Notarius may run multiple extraction passes over the same source. Chunking,
especially LLM-backed chunking over a large transcript, can be substantially
more expensive than later per-chunk operations. Stable chunk material also
improves the opportunity for provider-side prompt-cache reads across passes.
Notarius therefore prioritizes stable, aggressively reused source partitions
over automatically applying later chunk-module, reference, prompt, model, or
pipeline changes. An operator must explicitly request repartitioning.
## Target State
### Canonical source plan
Each validated generic source document has at most one stored chunk plan. The
source document's canonical digest selects that plan. Pipeline identity,
selected lanes, configured chunk module and options, references, validators,
prompts, schemas, and LLM settings do not participate in lookup.
In the default mode, the first accepted plan for a source is reused by later
pipelines and invocations. A configured chunk module runs only when no valid
stored plan exists, when reuse is bypassed for one invocation, or when the
operator requests refresh.
### Plan and annotation contract
A plan contains an ordered, non-empty collection of inclusive source-unit
ranges. The framework validates these boundaries and deterministically
materializes runtime chunks from the current source document.
Chunk modules may attach optional domain annotations at plan or range scope.
Annotations are namespaced, validated JSON values. They are preserved through
storage and materialization but are never a hard cross-domain capability.
Downstream modules may rely on the generic chunk contract only.
The store contains boundaries, annotations, warnings, and provenance. It does
not contain fully materialized chunks or duplicated source-unit content.
### Persistence
Chunk plans use a dedicated cache root that is independent of
`workspace.directory` and the locations of checkpoints, diagnostics, debug
artifacts, and durable output. The default is the platform-appropriate per-user
cache directory: on Linux, `$XDG_CACHE_HOME/notarius/chunk-plans` when
`XDG_CACHE_HOME` is set to a valid absolute path, otherwise
`$HOME/.cache/notarius/chunk-plans` when it is unset, through the platform
cache-directory resolver. An invalid relative `XDG_CACHE_HOME` is an error, not
a fallback.
Operators may set `workspace.chunk_cache.directory` to replace that root. The
recommended system-wide setting for a dedicated Linux service account is
`/var/cache/notarius/chunk-plans`. This is an operational recommendation, not
the application default; the service account must own the directory and it must
not be shared across mutually untrusted users.
One mutable, versioned plan file is stored beneath the chunk-plan root under the
full canonical source digest. Normal publication and refresh replace that file
atomically; history, rollback, multiple variants, and content deduplication are
outside this feature.
Plan state is potentially sensitive. Paths are confined, directories and files
use restrictive permissions, default diagnostics omit annotation and source
payloads, and debug output remains explicitly opt-in.
### Runtime policy
The effective chunk-cache mode is one of:
- `auto`: load a valid stored plan; otherwise generate, validate, and publish
one;
- `bypass`: do not read or write plan state for this invocation; or
- `refresh`: generate and validate a plan, then atomically replace stored state.
`auto` is the default. Invalid or incompatible stored state is a reported miss
in `auto`; it is replaced only after a newly generated plan is accepted. A
failed generation never overwrites prior state. Concurrent writes must never
expose partial data and use last-successful-atomic-write semantics.
The public CLI override is
`--chunk_cache <auto|bypass|refresh>`. Configuration and environment values use
the same three modes, with CLI taking highest precedence.
### Validation and execution
Framework plan validation and deterministic materialization run for generated
and reused plans. The current pipeline's configured chunk-validator chain then
validates the materialized chunks.
On a cache hit, the configured chunk module is constructed during normal
pipeline preparation but its operation is not invoked and it makes no LLM call.
A validator rejection of a structurally valid reused plan is a run outcome; it
does not implicitly authorize rechunking.
Only the generic `chunks` capability is hard. Domain annotations such as D&D
scene information are opportunistic and cannot be required solely because the
current pipeline selected the module that normally produces them.
Canonical plans are the sole owner of chunk reuse. Invocation-scoped
checkpoints no longer load or record chunk outputs. Downstream checkpoint
identity continues to depend on the digest of the effective materialized
chunks, so refresh invalidates affected downstream work.
### Provenance
Every run distinguishes the chunk module requested by the resolved pipeline
from the producer of the effective stored plan. Durable provenance records the
source and plan digests, plan schema version, effective cache mode and action,
producer module, producer references and LLM profile where applicable, and
non-sensitive producer metadata.
Default diagnostics record lookup, validation, generation, and publication
decisions without source or annotation payloads. Opt-in debug artifacts may
contain complete plan and annotation material and are treated as sensitive.
### Deployment documentation
The implemented configuration reference identifies the dedicated directory
field, its environment override, and the per-user default. The operations guide
documents the resulting filesystem layout and permissions, and recommends
`/var/cache/notarius/chunk-plans` for a system-wide Linux deployment running as
a dedicated service account. It also explains that cache deletion is
recoverable but may repeat expensive chunk generation. Neither document
presents the system-wide path as the default for an ordinary unprivileged
invocation.
## Compatibility Policy
Existing fully materialized chunk checkpoint files are not migrated or promoted
to canonical plans. They are ignored for chunk reuse after this feature lands.
The first `auto` run generates the source's plan through the configured chunk
module.
The existing source digest is the canonical lookup identity. Any source change
that alters that digest creates a separate plan. Earlier plan schema versions,
malformed files, and digest mismatches are incompatible and follow the invalid
`auto`-miss behavior.
## Non-Goals
This feature does not provide:
- plan editing, comparison, history, rollback, or garbage collection;
- remote or shared plan storage;
- multiple active or automatically selected plan variants for one source;
- mandatory domain annotation contracts;
- automatic rechunking because configuration or model inputs changed; or
- a guarantee that an LLM provider will report cache hits.
## Completion Outcomes
**Achieved.** Independent runs over the same canonical source
reuse byte-stable materialized chunks across pipeline, lane, reference,
chunk-module, and LLM configuration changes; bypass and refresh obey their
documented state semantics; provenance identifies the effective producer;
legacy chunk checkpoints cannot compete with plan reuse; and all focused,
integration, compatibility, CLI, and repository-wide validation pass. The
configuration and operations references must also document both the per-user
default and the recommended system-wide Linux setting.

578
docs/roadmap/implementation.md
View File

@@ -1,578 +0,0 @@
# ADR-0005 Implementation Record
This document records the implementation of the target state in
[ADR-0005 Feature Roadmap](adr0005.md), governed by
[ADR-0005](../adr/0005-cache-canonical-chunk-plans-by-source.md).
All stages are complete. This document remains a concise historical
implementation record. Current behavior is documented in the canonical
references linked from [Development](../development.md).
## Execution Rules
Before every stage:
1. read [Development](../development.md), both documents under `docs/policy/`,
ADR-0005, this plan, and every stage-specific document named below;
2. inspect `git status` and preserve user changes;
3. inspect the focused package contracts and tests before editing; and
4. confirm that all earlier stages are complete.
During every stage:
- implement only that stage and prerequisites discovered to be inseparable;
- use existing package boundaries and helpers before introducing abstractions;
- keep framework and core packages independent of concrete D&D modules;
- preserve cancellation, deterministic ordering, redaction, path confinement,
and atomic-write invariants;
- add or update focused tests with every behavior change;
- do not update current-behavior documentation before the final documentation
stage, except for ADR acceptance in Stage 1; and
- do not begin the next stage after meeting the current exit criteria.
At the end of every stage:
1. run the focused tests listed for that stage;
2. run `go test ./...`;
3. run `go vet ./...`;
4. run `go build ./cmd/notarius`;
5. run `git diff --check`; and
6. mark the stage complete in this document only after all checks pass.
If repository reality conflicts with this plan, stop and update the plan in the
same change before implementing a materially different design. Do not silently
invent a new contract.
## Fixed Decisions
These choices are settled for this implementation:
- `source.SourceDocument.Digest` is the cache lookup identity.
- The chunk-plan root is independent of `workspace.directory` and every other
state root.
- Its default joins the directory returned by `os.UserCacheDir` with
`notarius/chunk-plans`.
- `workspace.chunk_cache.directory` overrides that default and names the
chunk-plan root itself. The recommended value for a system-wide Linux service
is `/var/cache/notarius/chunk-plans`.
- The plan file is
`<chunk-plan-root>/<64-character-lowercase-source-sha256-hex>/plan.json`. The
validated `sha256:` prefix remains part of the logical digest but is removed
from the path segment.
- The plan envelope schema is `notarius.chunk-plan.v1`.
- There is one mutable plan file per source, atomically replaced. There are no
immutable plan objects, pointers, history, locks, or rollback.
- Concurrent writers use last-successful-atomic-write semantics; they must
never expose a partial file.
- Chunk modules retain the `contracts.Chunker` name but replace `Chunk` with a
`Plan` operation. No legacy interface remains after Stage 2.
- Plans and ranges use `map[string]json.RawMessage` annotations. Annotation
values are canonical valid JSON and namespaces are non-empty trimmed strings.
- Materialized chunks expose range annotations through
`source.Chunk.Annotations` and plan annotations through
`source.Chunk.PlanAnnotations`, never through `Metadata`.
- Framework materialization owns chunk IDs, indexes, references, content,
media type, units, and generic metadata.
- Only the generic `chunks` capability is hard; `chunks.scenes` is removed.
- Cached-plan hits still construct the configured chunker during preparation
but never call its operation or the chunk-stage LLM.
- The current pipeline's configured chunk validators run against materialized
chunks on every path.
- Canonical plan storage is the only chunk-reuse mechanism; chunk checkpoint
APIs and payloads are removed.
- The mode values are `auto`, `bypass`, and `refresh`. They are valid in config,
environment, and CLI, including persistent `refresh` configuration.
- The YAML field is `workspace.chunk_cache.mode`.
- The YAML directory field is `workspace.chunk_cache.directory`.
- The environment variable is `NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE`.
- The root environment variable is `NOTARIUS_WORKSPACE_CHUNK_CACHE_DIR`.
- The CLI flag is `--chunk_cache <auto|bypass|refresh>`.
- Precedence is CLI, environment, file, then default `auto`.
- Chunk-plan-root precedence is environment, file, then the per-user default;
there is no CLI root override.
- Every supplied file, environment, and CLI value is parsed strictly even when
a higher-precedence source would override it; precedence selects among valid
values and does not mask malformed configuration.
- Existing chunk checkpoint files are ignored and never migrated.
- A structurally invalid stored plan is an `auto` miss. A configured-validator
rejection of a structurally valid hit does not trigger regeneration.
## Planned Contracts
Use these names unless an existing collision requires the smallest obvious
adjustment.
### Source-zone types
Add to `internal/core/source`:
```go
type ChunkAnnotations map[string]json.RawMessage
type ChunkPlan struct {
SourceDigest string `json:"source_digest"`
Ranges []ChunkRange `json:"ranges"`
Annotations ChunkAnnotations `json:"annotations,omitempty"`
}
type ChunkRange struct {
StartUnitID int `json:"start_unit_id"`
EndUnitID int `json:"end_unit_id"`
Annotations ChunkAnnotations `json:"annotations,omitempty"`
}
```
Add `Annotations ChunkAnnotations` and
`PlanAnnotations ChunkAnnotations` to `source.Chunk`, with JSON names
`annotations` and `plan_annotations` and `omitempty` on both.
The plan digest covers `SourceDigest`, ordered ranges, and canonical annotation
bytes. It excludes producer provenance, warnings, timestamps, and storage
schema. Materialization produces:
- IDs `chunk-%06d`, numbered from one;
- zero-based indexes in plan order;
- references spanning the first and last units in each range;
- cloned contiguous source units;
- media type `application/json`;
- canonical JSON content shaped as `{"units":[...]}`;
- generic metadata containing `start_unit_id`, `end_unit_id`, and `unit_count`;
- cloned range annotations in `Chunk.Annotations`; and
- cloned plan annotations in `Chunk.PlanAnnotations`.
The plan and range scopes remain distinct even when both use the same namespace.
Copying plan annotations into each runtime chunk is intentional: it makes them
available to downstream chunk-scoped operations without duplicating source
content in durable storage.
Framework validation permits gaps and overlap between ranges, but requires
strictly increasing start positions and rejects duplicate or backward ranges.
Producing modules may impose stricter policy before returning a plan.
### Module contract
Replace the current chunk result and operation with:
```go
type ChunkPlanResult struct {
Plan source.ChunkPlan
Warnings []Warning
}
type Chunker interface {
Key() string
ReferenceSlots() []ReferenceSlot
Plan(context.Context, ChunkRequest) (ChunkPlanResult, error)
}
```
Keep `ChunkRequest`, chunk module keys, stage name `chunk`, configuration
bindings, registries, and preparation ownership unchanged.
### Durable plan record
Define framework-facing record and store contracts beside other pipeline
collaborator contracts:
```go
type ChunkPlanProducer struct {
InputModule string
ChunkModule string
LLMProfile string
References []artifacts.ReferenceProvenance
Metadata map[string]any
}
type ChunkPlanRecord struct {
SchemaVersion string
SourceDigest string
PlanDigest string
Plan source.ChunkPlan
Producer ChunkPlanProducer
Warnings []contracts.Warning
CreatedAt time.Time
}
type ChunkPlanStore interface {
Load(sourceDigest string) (ChunkPlanRecord, ChunkPlanDecision, error)
Save(ChunkPlanRecord) error
}
type ChunkPlanStoreFactory func(root string) (ChunkPlanStore, error)
type ChunkPlanDecision struct {
Status ChunkPlanStatus
Reason string
}
type ChunkPlanStatus string
const (
ChunkPlanHit ChunkPlanStatus = "hit"
ChunkPlanMissing ChunkPlanStatus = "missing"
ChunkPlanInvalid ChunkPlanStatus = "invalid"
)
```
`ChunkPlanDecision` reports `hit`, `missing`, or `invalid` plus a redacted
reason. A miss or invalid result returns a zero record and nil error. Missing
and structurally invalid records are recoverable in `auto`. Filesystem access
failures other than absence return an error and are framework failures.
The concrete constructor is:
```go
// Package chunkplan
func NewFilesystemStore(root string) (pipeline.ChunkPlanStore, error)
```
It treats `root` as the exact chunk-plan root. CLI composition injects this
constructor through `cli.Options.ChunkPlanStoreFactory`; tests may inject a
recording factory. `cli.Options.UserCacheDir` has signature
`func() (string, error)` and defaults to `os.UserCacheDir`.
The JSON envelope uses snake-case equivalents of the record fields. It stores
references as provenance only, never reference content. Before returning a hit,
the loader validates schema version, source digest, plan digest, plan structure,
annotation JSON, and materializability against the current source in the runner.
### Cache policy and provenance
Add a `ChunkCacheMode` enum with `auto`, `bypass`, and `refresh`, plus strict
parsing and validation.
Add to `RunInput`:
```go
ChunkCacheMode ChunkCacheMode
ChunkPlans ChunkPlanStore
```
An empty mode used by direct framework callers means `bypass`; the production
CLI always supplies the effective non-empty mode.
Add a run-manifest summary:
```go
type ChunkPlanManifest struct {
Mode string `json:"mode"`
Action string `json:"action,omitempty"`
SourceDigest string `json:"source_digest,omitempty"`
PlanDigest string `json:"plan_digest,omitempty"`
PlanSchemaVersion string `json:"plan_schema_version,omitempty"`
RequestedModule string `json:"requested_module"`
ProducerInputModule string `json:"producer_input_module,omitempty"`
ProducerModule string `json:"producer_module,omitempty"`
ProducerLLMProfile string `json:"producer_llm_profile,omitempty"`
ProducerReferences []ReferenceProvenance `json:"producer_references,omitempty"`
ProducerMetadata map[string]any `json:"producer_metadata,omitempty"`
CreatedAt *time.Time `json:"created_at,omitempty"`
}
```
`Action` is one of `reused`, `generated`, `refreshed`, or `bypassed`. Retain
the existing top-level `input_module` and `chunker` fields as the requested
pipeline modules. Add this field to `artifacts.RunManifest`:
```go
ChunkPlan *ChunkPlanManifest `json:"chunk_plan,omitempty"`
```
The runner initializes it with the effective mode and requested chunk module,
then fills candidate and producer fields when a plan reaches materialization.
## Completed Stage Summaries
### Stage 1: Finalize ADR and add the source-zone plan model
**Status:** Complete
Accepted ADR-0005 and added the domain-neutral chunk-plan, range, annotation,
validation, digest, cloning, and deterministic materialization primitives in
`internal/core/source`.
### Stage 2: Replace the chunk operation with plan generation
**Status:** Complete
Replaced chunk-returning module operations with `Chunker.Plan`, converted the
generic and D&D scene chunkers to produce ranges and optional annotations, and
made the runner materialize and validate generated plans.
### Stage 3: Add dedicated cache configuration and the plan store
**Status:** Complete
Added strict cache modes, independent per-user or explicitly configured
chunk-plan roots, the source-addressed filesystem store, restrictive
permissions, envelope validation, and atomic replacement.
### Stage 4: Integrate cache policy and remove chunk checkpoints
**Status:** Complete
Implemented runner semantics for `auto`, `bypass`, and `refresh`; retained
current chunk-validator behavior; made accepted canonical plans the sole chunk
reuse mechanism; and removed chunk checkpoint loading and recording.
### Stage 5: Wire persistent policy into the CLI
**Status:** Complete
Added CLI, environment, and file configuration precedence; per-user root
resolution; persistent store construction; bypass isolation; and cross-run
reuse through normal CLI execution.
### Stage 6: Expose producer provenance and safe diagnostics
**Status:** Complete
Added requested-versus-producer chunk-plan provenance to manifests and a
payload-free chunk-plan decision summary to normal diagnostics, with complete
plan material confined to opt-in debug artifacts.
### Stage 7: Add cross-run, corruption, and concurrency hardening
**Status:** Complete
Added integration, corruption, interruption, concurrency, compatibility,
cross-domain annotation, import-boundary, and race coverage for the implemented
cache design.
### Stage 8: Publish current-behavior documentation
**Status:** Complete
Updated architecture, pipeline, module, configuration, CLI, operations,
diagnostics, integration, and overview documentation for the implemented
feature and its per-user and system-wide deployment guidance.
## Stage 9: Redact invalid cache-record diagnostics
**Status:** Complete
### Objective
Ensure malformed or incompatible cache contents cannot enter normal errors,
warnings, manifests, or diagnostics while retaining useful closed lookup
decisions.
### Read first
- [Architecture Policy](../policy/architecture.md), especially secret handling
- [Diagnostics Internals](../internal/diagnostics.md)
- `internal/framework/chunkplan/store.go` and its focused tests
- `internal/framework/pipeline/runner_chunk_plan.go` and its focused tests
- `internal/core/artifacts/artifacts.go`
- the CLI path that writes `chunk-plan.json`
### Implement
1. Stop interpolating decoder errors, unknown field names, schema values,
annotation namespaces, timestamps, paths, or any other stored content into
recoverable `ChunkPlanDecision.Reason` values.
2. At the normal-diagnostic boundary, ignore the reason supplied by a
`ChunkPlanStore` and derive `ChunkPlanSummary.LookupReason` solely from
status using exactly:
- `hit`: `stored chunk plan is valid`;
- `missing`: `chunk plan not found`;
- `invalid`: `stored chunk plan is invalid`; and
- `skipped`: `chunk plan lookup skipped`.
3. Keep operational read failures as framework errors rather than recoverable
invalid-record decisions. Their normal error text must identify the failed
operation without including cache-file content.
4. Do not add raw invalid-record details to another default surface. Direct
operator inspection or a future explicitly sensitive debug facility may
expose them, but this stage does not need to add such a facility.
### Tests
Add focused tests proving that sentinel secrets placed in unknown field names,
schema values, annotation namespaces, timestamps, and malformed trailing JSON
appear in neither the store decision nor `chunk-plan.json`. Add a runner test
with a custom store whose reason contains a sentinel and prove that the summary
uses the fixed status-derived reason. Preserve useful missing, hit, invalid, and
operational-error classification.
Run focused tests for `internal/framework/chunkplan`,
`internal/framework/pipeline`, and `internal/cli`, then perform the
repository-wide validation in the execution rules.
### Exit criteria
Every normal lookup reason belongs to the fixed safe vocabulary, hostile
invalid-file content cannot reach default diagnostics, operational failures
remain correctly classified, and all focused and repository-wide checks pass.
## Stage 10: Enforce rooted chunk-plan filesystem access
**Status:** Complete
### Objective
Make the cache's path-confinement guarantee resistant to symlinked cache entries
while preserving atomicity, permissions, and concurrent access semantics.
### Read first
- [Architecture Policy](../policy/architecture.md), especially state and path
invariants
- [Operations](../operations.md)
- `internal/framework/chunkplan/store.go` and all store tests
- Go's `os.Root` API for the repository's declared Go version
### Implement
1. Treat the configured cache root as the filesystem trust boundary. A symlink
in the operator-supplied root path itself may be followed when the root is
opened, but every store-owned operation after that point must remain beneath
the opened root.
2. Perform digest-directory, plan-file, temporary-file, and rename operations
through `os.Root` or an equivalently race-resistant directory-relative
mechanism. Do not rely on `filepath.Join` plus a preflight `Lstat` as the
confinement boundary.
3. Reject symlinks and unexpected file types at the store-owned digest-directory
and `plan.json` positions. A missing cache root on `Load` remains a normal
miss and is not created; `Save` creates the root and digest directory as
needed.
4. Preserve the exact
`<root>/<full-source-sha256-hex>/plan.json` layout, `0700` directories,
`0600` plan files, strict digest validation, and no implicit path suffix.
5. Publish using an exclusively created random temporary file inside the digest
directory, sync and close it, then atomically rename it through the same
rooted handle. Remove temporary files on every pre-rename failure. Preserve
last-successful-write and complete-reader semantics.
6. Keep store access serial at the runner boundary and safe for concurrent
independent store callers. Do not introduce a lock, history, or rollback
protocol.
### Tests
Add load and save cases for digest-directory and plan-file symlinks that resolve
outside the configured root. Prove that they are rejected and that no outside
file is read, created, chmodded, or replaced. Include unexpected file types and
failed-publication preservation.
Retain and run tests for absent-root behavior, exact paths, permissions, round
trip, malformed digests, atomic replacement, interrupted writes, temporary-file
cleanup, concurrent readers, and concurrent writers. Run the race detector for
`internal/framework/chunkplan`, then perform the repository-wide validation in
the execution rules.
### Exit criteria
No store-owned operation can escape the opened root through a cache entry,
normal layout and permissions remain unchanged, publication remains atomic,
concurrency tests and the race detector pass, and the repository is green.
## Stage 11: Deep-clone source metadata during materialization
**Status:** Complete
### Objective
Ensure source documents, materialized chunks, and separate materializations
never share mutable metadata storage, regardless of the concrete JSON-shaped
map, slice, array, interface, raw-message, or byte-slice types used by an input
adapter.
### Read first
- `internal/core/source/source.go`
- `internal/core/source/chunk_plan.go` and its focused tests
- `internal/core/source/digest.go` and validation behavior
- every metadata clone helper and ownership boundary found with `rg`
- focused runner tests that hand chunks to validators and extractors
### Implement
1. Replace the concrete-type allowlist used by plan materialization with one
reusable source-owned deep-clone implementation for supported
JSON-serializable metadata.
2. Recursively clone string-keyed maps, slices, arrays, interfaces,
`json.RawMessage`, and byte slices without sharing mutable backing storage.
Preserve scalar values and the logical JSON representation. Typed composite
containers must not fall through as shared values.
3. Detect cyclic or unsupported values and return a contextual error from
materialization rather than retaining an alias, panicking, or recursing
indefinitely.
4. Use the source-owned helper at pipeline ownership boundaries where doing so
removes a duplicate incomplete implementation. Do not introduce a dependency
from `internal/core/source` to a framework package and do not expand this
stage into unrelated metadata-schema redesign.
5. Preserve deterministic materialized content, chunk digests, annotations, IDs,
references, and metadata fields for all currently supported production
inputs.
### Tests
Cover typed nested maps, slices of typed maps, arrays, interfaces,
`json.RawMessage`, and byte slices. Mutate each materialized value and prove
that neither the source document nor a separately materialized chunk changes.
Add cyclic and unsupported-value cases that fail deterministically with
contextual errors. Retain the existing exact-materialization and digest tests.
Run focused tests for `internal/core/source`,
`internal/framework/pipeline`, and affected module packages, then perform the
repository-wide validation in the execution rules.
### Exit criteria
Every supported mutable metadata value has independent ownership across source,
chunk, validation, and extraction boundaries; invalid metadata fails safely;
current materialized bytes remain stable; and all checks pass.
## Stage 12: Finalize remediation documentation and roadmap status
**Status:** Complete
### Objective
Make the roadmap and canonical documentation accurately describe the completed
feature after Stages 9 through 11 land.
### Read first
- [Documentation Policy](../policy/documentation.md)
- [ADR-0005 Feature Roadmap](adr0005.md)
- [Architecture Policy](../policy/architecture.md)
- [Diagnostics Internals](../internal/diagnostics.md)
- [Operations](../operations.md)
- the final code and tests from Stages 9 through 11
### Implement
1. Remove the feature roadmap's statement that ADR-0005 is unimplemented and
describe it as the implemented target-state record. Mark its completion
outcomes as achieved without adding staged implementation detail.
2. Recheck architecture, diagnostics, operations, pipeline, and module
documentation against the remediated behavior. Update a canonical document
only where Stages 9 through 11 changed a current operator or developer
contract; do not duplicate implementation detail across documents.
3. Mark Stages 9 through 12 complete only after their respective code,
documentation, focused tests, and repository-wide validation have passed.
Update this document's introduction to state that all stages are complete and
that it remains as a concise historical implementation record.
4. Validate every changed link, heading, symbol, diagnostic field, permission,
and filesystem claim against the implementation.
### Tests and validation
Run:
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
git diff --check
```
Run any repository documentation or link checker if present, and manually
inspect the reading map in [Development](../development.md) and component links
in [Internal Overview](../internal/overview.md).
### Exit criteria
The feature roadmap and canonical references describe implemented behavior,
this plan accurately records completion, documentation ownership remains clear,
all links and names are current, and the repository-wide validation is green.