notarius/docs/roadmap/implementation.md

Deployment-Managed Cache Permissions Implementation Plan

Purpose

Implement the target contract in Deployment-Managed Cache Permissions: Notarius must preserve safe, confined, atomic cache I/O while allowing normal Unix creation semantics to determine who can access newly created chunk-plan and checkpoint state.

This document is the execution sequence. The feature roadmap is authoritative for user intent, policy, scope, and the target end state. Each numbered stage below is sized for one gpt-5.6-terra implementation prompt and must be completed in order.

Cross-Stage Requirements

Every stage must preserve these decisions:

• Do not add a cache.access field, environment variable, CLI flag, or an application-level private/group policy.
• New cache directories request mode 0777; new cache files request mode 0666. The kernel applies umask and default ACL policy.
• Do not chmod, chown, chgrp, or recursively migrate existing cache entries to enforce access policy.
• Keep chunk-plan and checkpoint confinement, symlink rejection, identity, validation, atomic replacement, cleanup, and recovery behavior intact.
• Do not change output or debug permissions as an incidental consequence of a shared-helper change.
• Do not add cross-process locking or promise transactional concurrent writes to one checkpoint identity.
• Follow docs/policy/testing.md: protect the observable permission boundary and meaningful filesystem risks without snapshotting helper structure or duplicating all cases at every layer.
• Keep the repository buildable and the tests relevant to the completed stage passing. Do not retire either roadmap document during implementation; they remain review evidence until the completed feature is accepted.

Stage 1: Record The Decision And Add An Umask-Respecting Atomic Writer

Completed: 2026-09-11

Goal

Record the durable ownership boundary and add the narrow filesystem primitive needed by checkpoint storage, without changing any production caller yet.

Required Changes

1. Add docs/adr/0017-defer-cache-access-policy-to-deployment.md using the repository's Nygard ADR format, with status Accepted and the implementation date. The decision must:
   • assign cache access policy to deployment provisioning, process umask, setgid inheritance, and default ACLs;
   • distinguish access policy from Notarius-owned confinement, validation, atomicity, and recovery;
   • reject both forced private modes and a Notarius-specific private/group setting;
   • state that source-derived cache contents may be sensitive and that the operator owns access and retention;
   • state that API credentials must remain outside cache payloads under the existing secret boundary;
   • explain that existing cache modes are not migrated automatically;
   • reference ADR-0005, ADR-0006, and ADR-0007 without superseding them; and
   • record application-enforced 0700/0600, configurable permission modes, and recursive permission convergence as rejected alternatives.
2. Extend internal/core/fileio with exactly two cache-oriented entry points:
   • WriteBytesWithProcessPermissions(root, name string, data []byte) error;
   • WriteJSONWithProcessPermissions(root, name string, payload any) error.
3. The new byte writer must reuse SafePath and symlink-component rejection, call os.MkdirAll with 0777, and atomically replace the target. The JSON wrapper must retain the existing indented JSON plus trailing-newline format.
4. Implement the new atomic path independently from the existing exact-mode WriteBytes path:
   • generate a 16-byte cryptographically random suffix;
   • attempt at most 32 unique sibling names of the form .<target-base>.tmp-<hex-suffix>;
   • open with os.O_WRONLY|os.O_CREATE|os.O_EXCL and mode 0666 so the kernel applies umask and default ACLs at creation;
   • retry only name collisions;
   • write, close, rename, and remove an unpublished temporary file on every failure path; and
   • do not call chmod or inspect/change the process umask in production code.
5. Leave WriteBytes, WriteJSON, and their existing callers unchanged so output and debug behavior cannot drift in this stage.

Tests

• Add supported-Unix (linux || darwin) behavioral tests using syscall.Umask. Each test must restore the prior umask with defer and must not use t.Parallel.
• Prove a restrictive umask (0077) produces 0700 nested directories and a 0600 final file.
• Prove a group-oriented umask (0007) produces 0770 nested directories and a 0660 final file.
• Pre-create the selected root with a distinctive mode, write beneath it, and prove the root mode is unchanged.
• Replace an existing file created under a different mode and prove the new inode follows the current process policy and contains the complete new content.
• Prove no temporary sibling remains after success. Retain all existing confinement, symlink, and exact-mode writer tests.

Validation

Run:

gofmt -w internal/core/fileio/*.go
go test ./internal/core/fileio
go test -race ./internal/core/fileio
git diff --check

Stage Acceptance

The ADR records the agreed boundary; the new writer demonstrates both target umasks without chmod; existing file-writing behavior has not changed; and the stage is small enough for one implementation prompt.

Stage 2: Convert Canonical Chunk-Plan Storage

Completed: 2026-09-11

Goal

Make every chunk-plan creation and replacement path follow deployment-managed permissions while preserving the rooted filesystem and cache-recovery protocol.

Required Changes

1. In internal/framework/chunkplan:
   • request 0777 when creating the configured root;
   • remove the chmod of an opened existing root;
   • request 0777 when creating a source-digest directory;
   • remove the chmod branch for an opened existing digest directory;
   • retain the before/open/after identity checks that defend against entry replacement races;
   • change the existing rooted O_CREATE|O_EXCL temporary-file open mode from 0600 to 0666; and
   • remove the subsequent temp.Chmod(0600) call while preserving write, sync, close, rename, cleanup, and interruption hooks.
2. Do not replace os.Root operations with ordinary path operations, weaken unexpected-entry handling, or create a missing root during read-only load.
3. Rewrite TestFilesystemStorePermissions into a behavioral Unix test that controls and restores umask. Consolidate permission cases rather than retaining a stale forced-private assertion alongside the new contract.

Tests

• Under umask 0077, prove the root, digest directory, and published plan are 0700, 0700, and 0600.
• Under umask 0007, prove they are 0770, 0770, and 0660.
• Because the published plan is the renamed temporary inode, its final mode proves the temporary-file creation policy; use the existing pre-rename hook only if a separate assertion is needed to prevent a plausible regression.
• Pre-create both the root and digest directory with deliberate modes, save a plan, and prove neither directory is modified.
• Retain the existing missing-root side-effect, symlink, entry-replacement, interruption, atomic-replacement, record-validation, and recovery tests.

Validation

Run:

gofmt -w internal/framework/chunkplan/*.go
go test ./internal/framework/chunkplan
go test -race ./internal/framework/chunkplan
git diff --check

Stage Acceptance

Chunk-plan writes honor both controlled deployment policies, existing directories are not converged, and all rooted atomicity and recovery coverage passes. This stage is small enough for one implementation prompt.

Stage 3: Convert Checkpoints And Prove Assembled Cache Behavior

Goal

Move checkpoints to the new shared writer, remove stale private-cache expectations at the CLI boundary, and prove one assembled run applies the same deployment policy to both cache families.

Required Changes

1. Change the checkpoint recorder's JSON write boundary in internal/framework/checkpoint to call fileio.WriteJSONWithProcessPermissions. Do not thread modes through the recorder or introduce a cache permission configuration value.
2. Replace the checkpoint round-trip test's forced 0700/0600 subsection with focused POSIX permission coverage. Use a cache root that does not exist before the recorder writes so root creation is covered.
3. Preserve checkpoint JSON representation, path layout, manifests, payloads, identity, recording enablement, reuse eligibility, loader decisions, and corrupt/incompatible-state behavior.
4. In internal/cli/state_hardening_test.go, stop applying assertRestrictedTree to checkpoints. Retain that helper and assertion for debug state, whose permission contract is unchanged.
5. Add one supported-Unix CLI integration test using the existing offline state-test harness:
   • configure both chunk-plan caching and checkpoint recording beneath previously absent roots;
   • set and reliably restore umask 0007 around the run;
   • prove representative directories and final files in both cache families are group-accessible (0770/0660);
   • prove the run completes and the stored state remains reusable; and
   • do not duplicate every package-level permission or recovery case.
6. Keep output and debug mode assertions unchanged and confirm that no cache.access parsing or composition code exists.

Tests

• Add checkpoint-level cases for umask 0077 and 0007, covering representative nested directories and both manifest and payload files.
• Prove a pre-existing checkpoint root is not chmod modified.
• Retain the existing all-stage round trip as the owner of serialization and loader behavior rather than cloning it into each permission case.
• Ensure every umask-changing test is non-parallel and restores global state.

Validation

Run:

gofmt -w internal/framework/checkpoint/*.go internal/cli/*.go
go test ./internal/framework/checkpoint ./internal/cli
go test -race ./internal/framework/checkpoint ./internal/cli
go test ./internal/framework/chunkplan ./internal/core/fileio
git diff --check

Stage Acceptance

Both cache families follow process/filesystem policy through the real CLI composition, checkpoint reuse remains intact, and output/debug permission contracts have not changed. This stage is small enough for one implementation prompt.

Stage 4: Update Canonical Documentation

Goal

Make the implemented behavior discoverable without duplicating volatile details across documentation owners.

Required Changes

1. Update docs/policy/architecture.md with the normative boundary: deployments own cache access policy; Notarius owns confined paths, atomic publication, validation, and recovery. Do not generalize the rule to output or debug state in this work.
2. Update docs/operations.md, the canonical operational owner, to:
   • replace claims that cache entries are forcibly 0700/0600;
   • explain requested maximum modes 0777/0666, umask filtering, default ACLs, and setgid group inheritance without claiming literal mode-bit inheritance;
   • provide concise private (0077) and trusted-group (0007 plus a deployment-provisioned setgid root, or an equivalent default ACL) examples;
   • state that cache data is reconstructible but may contain source-derived content and operators own access and retention;
   • explain that pre-feature cache trees retain their existing inode modes and must be deliberately recreated or administratively migrated; and
   • accurately limit concurrent-use claims to the existing atomicity and validation guarantees.
3. Update docs/config.md only to clarify that selecting chunk-plan or checkpoint directories does not configure their filesystem access policy. Do not document a nonexistent permission field.
4. Update docs/internal/state.md with the two implemented writer paths: rooted chunk-plan atomic creation and checkpoint use of the process-permission file writer. Link to Operations for operator-facing mode and provisioning guidance instead of duplicating it.
5. Review docs/internal/overview.md and docs/development.md; update links or routing only if the existing navigation no longer reaches the canonical state and operations documents.
6. Do not edit a historical release note. The release procedure will summarize this change when the next release candidate is prepared.

Validation

• Verify every changed relative Markdown link and heading anchor.
• Search current-behavior documentation for stale assertions that cache modes are always 0700/0600; distinguish cache from still-valid debug claims.
• Confirm current-behavior documentation outside docs/roadmap/ contains neither cache.access nor instructions for Notarius-managed chmod/chown/chgrp/ACL mutation.
• Run git diff --check.

Stage Acceptance

Each policy and operational fact has one canonical owner, current docs describe only implemented behavior, and the feature roadmap remains the owner of future status until final review. This stage is small enough for one implementation prompt.

Stage 5: Repository-Wide Verification And Contract Audit

Goal

Prove the complete target state and catch accidental permission, safety, or documentation drift before review.

Required Review

1. Inspect production cache code for remaining 0700, 0600, or Chmod operations. Every remaining occurrence must be a fixture, an unrelated output/debug contract, or otherwise justified; neither cache writer may enforce those values.
2. Search configuration, CLI, examples, and documentation to confirm no cache.access surface or private/group application policy was introduced.
3. Confirm output and debug production paths and their contract tests are unchanged except for necessary documentation disambiguation.
4. Confirm cache reads remain side-effect free where previously guaranteed and every cache write remains confined and atomic.
5. Review all new tests against docs/policy/testing.md. Consolidate redundant mode assertions while retaining one owner for each realistic regression: kernel filtering, existing-root preservation, each cache writer, and assembled CLI wiring.
6. Review the ADR and canonical docs for the agreed scope. Do not describe Unix permissions as literal inheritance from a parent directory.

Validation

Run the complete offline suite:

gofmt -w internal/core/fileio/*.go internal/framework/chunkplan/*.go internal/framework/checkpoint/*.go internal/cli/*.go
go test ./...
go test -race ./internal/core/fileio ./internal/framework/chunkplan ./internal/framework/checkpoint ./internal/cli
go vet ./...
go build ./cmd/notarius
go run ./cmd/notarius config validate --config examples/dnd-minimal.config.yml --pipeline dnd-session
OPENROUTER_API_KEY=validation-placeholder go run ./cmd/notarius config validate --config examples/dnd-complete.config.yml --pipeline dnd-session
git diff --check

The config validations must perform no provider requests. Also inspect git status --short --untracked-files=all for unintended files, generated artifacts, or unrelated edits.

Final Acceptance

Every acceptance criterion in the feature roadmap is supported by direct code, test, or documentation evidence; all prescribed commands pass; the working diff contains only this feature; and the two roadmap documents remain in place for the post-implementation review. This stage is small enough for one final verification prompt.