Plan deployment-managed cache permissions

This commit is contained in:
2026-09-11 14:24:51 +00:00
parent 77ae132e56
commit 7d9adc77ee
3 changed files with 593 additions and 0 deletions

View File

@@ -0,0 +1,236 @@
# Deployment-Managed Cache Permissions
Status: proposed
## Summary
Stop treating cache access permissions as application policy. Notarius should
create chunk-plan and checkpoint state through conventional Unix filesystem
semantics and leave the effective owner, group, and access policy to the
operator's provisioned directories, process umask, setgid inheritance, and
default ACLs.
Notarius continues to own safe paths, symlink rejection, atomic publication,
cache validation, and recovery. It does not decide whether a configured cache
is private to one account, shared by a trusted group, or readable under some
other deployment policy.
## Motivation And Current Problem
The current cache implementations enforce a policy that was stronger than the
application requires:
- the chunk-plan store creates and repeatedly converges cache directories to
`0700` and atomic files to `0600`; and
- the checkpoint recorder supplies `0700` directory and `0600` file modes to
the shared confined writer, which explicitly applies the file mode during
atomic publication.
This prevents an administrator from using ordinary Unix provisioning to share
one cache between a service account and trusted operator accounts. Adding a
second Notarius-specific `private` or `group` configuration policy would solve
that immediate symptom, but it would preserve the underlying ownership error:
Notarius would still decide access policy instead of respecting the deployment.
Routine Notarius operation does not require an application-enforced private
cache. Cache records can contain source-derived data, so operators must be told
what they are storing, but sensitivity does not imply that Notarius should
override a deliberately configured filesystem policy. API credentials remain
outside cache payloads and continue to follow their existing configuration and
redaction boundaries.
## Target Policy
The target contract is:
> Notarius creates cache entries with conventional maximum creation modes and
> allows the operating system to determine effective access through the
> process umask, inherited group ownership, and default ACLs.
Concretely:
- newly created cache directories request `0777`;
- newly created cache files, including atomic temporary files, request `0666`;
- the kernel applies the invoking process's umask and any applicable default
ACL;
- setgid directories provisioned by the operator control inherited group
ownership and, on supported Unix systems, setgid propagation to descendants;
- Notarius does not call `chmod`, `chown`, or `chgrp` to impose or repair a
cache access policy; and
- Notarius does not reject a cache merely because its effective modes are more
or less permissive than a Notarius-selected default.
This is policy inheritance, not literal copying of a parent directory's mode
bits. Directories and non-executable data files appropriately have different
maximum modes. A parent mode alone also does not determine child modes: an
operator who needs a particular result must configure the service umask and,
where appropriate, a setgid parent or default ACL.
For example, a service started with umask `0077` ordinarily creates `0700`
directories and `0600` files. A trusted-group deployment can provision a
setgid cache root and use umask `0007`, or an equivalent default ACL, to obtain
`0770` directories and `0660` files. These are deployment choices rather than
configuration values interpreted by Notarius.
## Public Contract
No new Notarius configuration field, environment variable, or CLI flag is
introduced. In particular, do not add the previously considered
`cache.access: private|group` setting.
Existing cache enablement, locations, chunk-plan modes, resume controls,
identity, and compatibility rules remain unchanged. The behavioral change is
that effective permissions on newly created cache entries are controlled by
the deployment environment rather than forcibly converged by Notarius.
The policy applies uniformly to both cache families:
- canonical chunk plans; and
- invocation checkpoints.
Output and debug storage are outside this feature's scope. Their current
contracts must not change incidentally through a shared helper refactor. A
separate review may later consider whether the same deployment-owned policy is
appropriate for those surfaces.
## Filesystem And Atomic-Publication Requirements
Permission deference must cover every cache write path, not only final files:
- configured cache-family roots created by Notarius;
- chunk-plan source-digest directories;
- checkpoint identity and stage directories;
- atomic temporary files; and
- final files published by rename.
Atomic publication must not bypass the deployment policy. Go's
`os.CreateTemp` creates a file with `0600`, and applying `chmod(0666)` afterward
would bypass the process umask. Cache writers therefore need an atomic
temporary-file creation path that chooses a collision-resistant name and opens
it with `O_CREATE|O_EXCL` and requested mode `0666`, allowing the kernel to
apply umask and default ACL policy at creation time.
Replacement writes publish a newly created temporary inode. Its effective
permissions and ACL are therefore those selected for new entries by the
current parent and process policy; Notarius does not copy mode bits or ACLs
from the replaced file. Operators who require stable replacement permissions
must express that policy through a stable service umask and directory default
ACL.
Opening an existing cache for writing must not alter its root or descendant
modes. Read-only lookup of a missing cache must remain side-effect free.
## Preserved Safety And Correctness Boundaries
Deferring access policy does not weaken Notarius's filesystem correctness
rules. The implementation must preserve:
- confined resolution beneath the selected cache root;
- rejection of symlinked or unexpected cache entries;
- validation of source digests, checkpoint identities, schemas, and recorded
provenance;
- atomic replacement, existing synchronization behavior, cleanup of abandoned
temporary files, and complete-record visibility;
- existing missing, malformed, incompatible, and operational-error recovery
classifications; and
- separation of output, cache, and debug roots.
An administrator's decision to permit multiple accounts to reach one cache
does not create a new Notarius guarantee for simultaneous writers. Preserve
the existing atomicity and validation behavior, but do not add locking or claim
transactional coordination across concurrent processes as part of this work.
Concurrent writes to the same multi-file checkpoint identity remain subject to
the currently documented cache behavior; the implementation documentation
must state any existing limitation accurately.
## Architectural Boundaries
The target removes application-owned cache modes without creating a general
permission-policy framework. Chunk-plan and checkpoint storage may use
different filesystem APIs internally, but both must implement the same
deployment-owned result. Pipeline modules and the runner remain unaware of
physical roots and filesystem access policy.
Any shared writer API must distinguish an application-enforced exact mode from
an umask-filtered maximum creation mode. The cache change must not silently
alter output or debug behavior merely because checkpoints currently share
filesystem helpers with those surfaces.
## Existing Cache Migration
Stopping permission convergence does not change existing inode modes. A cache
tree created by an older Notarius version may therefore remain `0700`/`0600`
and inaccessible to another account after upgrade.
Operators who want a different policy must either:
- remove the reconstructible cache and allow it to be recreated beneath a
correctly provisioned parent and service environment; or
- stop writers and deliberately update ownership, modes, and ACLs using normal
system-administration tools.
Notarius must not perform a recursive startup migration. The same rule applies
when an operator wants to make a previously shared cache private: change the
deployment policy and deliberately migrate or recreate existing state when an
immediate whole-tree change is required.
## Documentation And Decision Records
The final documentation set includes an accepted ADR titled along the lines of
"Defer cache access policy to the deployment." The ADR records the durable
ownership boundary so later maintenance does not reintroduce
application-enforced modes solely because cache data can contain source-derived
content.
The new ADR need not supersede ADR-0005 or ADR-0007: those decisions govern
canonical chunk-plan reuse and checkpoint recording/reuse, while leaving exact
storage permissions to operational contracts. It should reference those ADRs
and the output/cache/debug separation established by ADR-0006 and refined by
ADR-0007.
In the final current-behavior documentation, `docs/operations.md` owns cache
provisioning, umask, setgid, default ACL, migration, sensitivity, and
concurrent-use guidance. Architecture owns the deployment/application boundary;
the configuration reference clarifies that selecting a cache path does not
select its permissions; and the internal state guide explains the implemented
write mechanics. The release that first contains the feature summarizes the
operational change without becoming its canonical reference.
Current-behavior documentation must not describe this target behavior before
the implementation lands.
## Verification Expectations
Behavioral tests protect kernel-selected permissions for both cache families,
non-mutation of existing roots, atomic replacement, and representative
assembled CLI wiring. Existing confinement, symlink, validation, identity,
reuse, and recovery coverage remains authoritative. Umask tests are POSIX-only,
serialized, and restore process-global state. The default suite does not create
privileged accounts or groups; setgid and default-ACL examples may be verified
in a purpose-built Linux deployment environment.
## Acceptance Criteria
- No cache access-mode field or application-specific private/group policy is
added.
- Both chunk-plan and checkpoint writers request `0777` for new directories
and `0666` for new files, including atomic temporary files.
- Effective permissions reflect the process umask and applicable default ACLs
rather than a later Notarius chmod.
- Existing cache roots and descendants are not chmod-converged during reads or
writes.
- A private deployment can obtain private cache state through its filesystem
provisioning and umask without a Notarius-specific setting.
- A trusted-group deployment can obtain shared cache state through setgid,
umask, and/or default ACL policy without a Notarius-specific setting.
- Existing cache trees are not recursively migrated.
- Chunk-plan and checkpoint enablement, lookup, recording, resume, identity,
validation, confinement, atomicity, and recovery semantics remain unchanged.
- Output and debug permission behavior does not change as an incidental effect.
- Notarius performs no user/group management, ownership mutation, ACL
management, privilege escalation, or cross-process locking.
- The ADR and canonical operations, architecture, configuration, and internal
documentation accurately describe the implemented boundary; the next
release note summarizes the change when that release is prepared.
- Focused tests and repository-wide `go test ./...`, `go vet ./...`, and
`go build ./cmd/notarius` checks pass.

View File

@@ -150,6 +150,16 @@ address Go dependency cycles, registrar ownership, stable public module keys,
and supersession of the affected ADR-0004 decision. Conceptual artifact-family
ownership does not by itself require this move.
## Near-Term Operations
### Deployment-Managed Cache Permissions
- Implement the proposed
[deployment-managed cache permissions](deployment-managed-cache-permissions.md)
contract so Notarius respects the access policy established by the cache
root, process umask, setgid inheritance, and default ACLs instead of imposing
application-owned private or group modes.
## Blue-Sky Platform And Operations
These ideas are intentionally less specified. Promote one into an earlier

View File

@@ -0,0 +1,347 @@
# Deployment-Managed Cache Permissions Implementation Plan
## Purpose
Implement the target contract in
[Deployment-Managed Cache Permissions](deployment-managed-cache-permissions.md):
Notarius must preserve safe, confined, atomic cache I/O while allowing normal
Unix creation semantics to determine who can access newly created chunk-plan
and checkpoint state.
This document is the execution sequence. The feature roadmap is authoritative
for user intent, policy, scope, and the target end state. Each numbered stage
below is sized for one `gpt-5.6-terra` implementation prompt and must be
completed in order.
## Cross-Stage Requirements
Every stage must preserve these decisions:
- Do not add a `cache.access` field, environment variable, CLI flag, or an
application-level private/group policy.
- New cache directories request mode `0777`; new cache files request mode
`0666`. The kernel applies umask and default ACL policy.
- Do not chmod, chown, chgrp, or recursively migrate existing cache entries to
enforce access policy.
- Keep chunk-plan and checkpoint confinement, symlink rejection, identity,
validation, atomic replacement, cleanup, and recovery behavior intact.
- Do not change output or debug permissions as an incidental consequence of a
shared-helper change.
- Do not add cross-process locking or promise transactional concurrent writes
to one checkpoint identity.
- Follow `docs/policy/testing.md`: protect the observable permission boundary
and meaningful filesystem risks without snapshotting helper structure or
duplicating all cases at every layer.
- Keep the repository buildable and the tests relevant to the completed stage
passing. Do not retire either roadmap document during implementation; they
remain review evidence until the completed feature is accepted.
## Stage 1: Record The Decision And Add An Umask-Respecting Atomic Writer
### Goal
Record the durable ownership boundary and add the narrow filesystem primitive
needed by checkpoint storage, without changing any production caller yet.
### Required Changes
1. Add `docs/adr/0017-defer-cache-access-policy-to-deployment.md` using the
repository's Nygard ADR format, with status `Accepted` and the implementation
date. The decision must:
- assign cache access policy to deployment provisioning, process umask,
setgid inheritance, and default ACLs;
- distinguish access policy from Notarius-owned confinement, validation,
atomicity, and recovery;
- reject both forced private modes and a Notarius-specific private/group
setting;
- state that source-derived cache contents may be sensitive and that the
operator owns access and retention;
- state that API credentials must remain outside cache payloads under the
existing secret boundary;
- explain that existing cache modes are not migrated automatically;
- reference ADR-0005, ADR-0006, and ADR-0007 without superseding them; and
- record application-enforced `0700`/`0600`, configurable permission modes,
and recursive permission convergence as rejected alternatives.
2. Extend `internal/core/fileio` with exactly two cache-oriented entry points:
- `WriteBytesWithProcessPermissions(root, name string, data []byte) error`;
- `WriteJSONWithProcessPermissions(root, name string, payload any) error`.
3. The new byte writer must reuse `SafePath` and symlink-component rejection,
call `os.MkdirAll` with `0777`, and atomically replace the target. The JSON
wrapper must retain the existing indented JSON plus trailing-newline format.
4. Implement the new atomic path independently from the existing exact-mode
`WriteBytes` path:
- generate a 16-byte cryptographically random suffix;
- attempt at most 32 unique sibling names of the form
`.<target-base>.tmp-<hex-suffix>`;
- open with `os.O_WRONLY|os.O_CREATE|os.O_EXCL` and mode `0666` so the kernel
applies umask and default ACLs at creation;
- retry only name collisions;
- write, close, rename, and remove an unpublished temporary file on every
failure path; and
- do not call `chmod` or inspect/change the process umask in production code.
5. Leave `WriteBytes`, `WriteJSON`, and their existing callers unchanged so
output and debug behavior cannot drift in this stage.
### Tests
- Add supported-Unix (`linux || darwin`) behavioral tests using
`syscall.Umask`. Each test must restore the prior umask with `defer` and must
not use `t.Parallel`.
- Prove a restrictive umask (`0077`) produces `0700` nested directories and a
`0600` final file.
- Prove a group-oriented umask (`0007`) produces `0770` nested directories and
a `0660` final file.
- Pre-create the selected root with a distinctive mode, write beneath it, and
prove the root mode is unchanged.
- Replace an existing file created under a different mode and prove the new
inode follows the current process policy and contains the complete new
content.
- Prove no temporary sibling remains after success. Retain all existing
confinement, symlink, and exact-mode writer tests.
### Validation
Run:
```sh
gofmt -w internal/core/fileio/*.go
go test ./internal/core/fileio
go test -race ./internal/core/fileio
git diff --check
```
### Stage Acceptance
The ADR records the agreed boundary; the new writer demonstrates both target
umasks without chmod; existing file-writing behavior has not changed; and the
stage is small enough for one implementation prompt.
## Stage 2: Convert Canonical Chunk-Plan Storage
### Goal
Make every chunk-plan creation and replacement path follow deployment-managed
permissions while preserving the rooted filesystem and cache-recovery
protocol.
### Required Changes
1. In `internal/framework/chunkplan`:
- request `0777` when creating the configured root;
- remove the chmod of an opened existing root;
- request `0777` when creating a source-digest directory;
- remove the chmod branch for an opened existing digest directory;
- retain the before/open/after identity checks that defend against entry
replacement races;
- change the existing rooted `O_CREATE|O_EXCL` temporary-file open mode from
`0600` to `0666`; and
- remove the subsequent `temp.Chmod(0600)` call while preserving write,
sync, close, rename, cleanup, and interruption hooks.
2. Do not replace `os.Root` operations with ordinary path operations, weaken
unexpected-entry handling, or create a missing root during read-only load.
3. Rewrite `TestFilesystemStorePermissions` into a behavioral Unix test that
controls and restores umask. Consolidate permission cases rather than
retaining a stale forced-private assertion alongside the new contract.
### Tests
- Under umask `0077`, prove the root, digest directory, and published plan are
`0700`, `0700`, and `0600`.
- Under umask `0007`, prove they are `0770`, `0770`, and `0660`.
- Because the published plan is the renamed temporary inode, its final mode
proves the temporary-file creation policy; use the existing pre-rename hook
only if a separate assertion is needed to prevent a plausible regression.
- Pre-create both the root and digest directory with deliberate modes, save a
plan, and prove neither directory is modified.
- Retain the existing missing-root side-effect, symlink, entry-replacement,
interruption, atomic-replacement, record-validation, and recovery tests.
### Validation
Run:
```sh
gofmt -w internal/framework/chunkplan/*.go
go test ./internal/framework/chunkplan
go test -race ./internal/framework/chunkplan
git diff --check
```
### Stage Acceptance
Chunk-plan writes honor both controlled deployment policies, existing
directories are not converged, and all rooted atomicity and recovery coverage
passes. This stage is small enough for one implementation prompt.
## Stage 3: Convert Checkpoints And Prove Assembled Cache Behavior
### Goal
Move checkpoints to the new shared writer, remove stale private-cache
expectations at the CLI boundary, and prove one assembled run applies the same
deployment policy to both cache families.
### Required Changes
1. Change the checkpoint recorder's JSON write boundary in
`internal/framework/checkpoint` to call
`fileio.WriteJSONWithProcessPermissions`. Do not thread modes through the
recorder or introduce a cache permission configuration value.
2. Replace the checkpoint round-trip test's forced `0700`/`0600` subsection
with focused POSIX permission coverage. Use a cache root that does not exist
before the recorder writes so root creation is covered.
3. Preserve checkpoint JSON representation, path layout, manifests, payloads,
identity, recording enablement, reuse eligibility, loader decisions, and
corrupt/incompatible-state behavior.
4. In `internal/cli/state_hardening_test.go`, stop applying
`assertRestrictedTree` to checkpoints. Retain that helper and assertion for
debug state, whose permission contract is unchanged.
5. Add one supported-Unix CLI integration test using the existing offline
state-test harness:
- configure both chunk-plan caching and checkpoint recording beneath
previously absent roots;
- set and reliably restore umask `0007` around the run;
- prove representative directories and final files in both cache families
are group-accessible (`0770`/`0660`);
- prove the run completes and the stored state remains reusable; and
- do not duplicate every package-level permission or recovery case.
6. Keep output and debug mode assertions unchanged and confirm that no
`cache.access` parsing or composition code exists.
### Tests
- Add checkpoint-level cases for umask `0077` and `0007`, covering
representative nested directories and both manifest and payload files.
- Prove a pre-existing checkpoint root is not chmod modified.
- Retain the existing all-stage round trip as the owner of serialization and
loader behavior rather than cloning it into each permission case.
- Ensure every umask-changing test is non-parallel and restores global state.
### Validation
Run:
```sh
gofmt -w internal/framework/checkpoint/*.go internal/cli/*.go
go test ./internal/framework/checkpoint ./internal/cli
go test -race ./internal/framework/checkpoint ./internal/cli
go test ./internal/framework/chunkplan ./internal/core/fileio
git diff --check
```
### Stage Acceptance
Both cache families follow process/filesystem policy through the real CLI
composition, checkpoint reuse remains intact, and output/debug permission
contracts have not changed. This stage is small enough for one implementation
prompt.
## Stage 4: Update Canonical Documentation
### Goal
Make the implemented behavior discoverable without duplicating volatile
details across documentation owners.
### Required Changes
1. Update `docs/policy/architecture.md` with the normative boundary:
deployments own cache access policy; Notarius owns confined paths, atomic
publication, validation, and recovery. Do not generalize the rule to output
or debug state in this work.
2. Update `docs/operations.md`, the canonical operational owner, to:
- replace claims that cache entries are forcibly `0700`/`0600`;
- explain requested maximum modes `0777`/`0666`, umask filtering, default
ACLs, and setgid group inheritance without claiming literal mode-bit
inheritance;
- provide concise private (`0077`) and trusted-group (`0007` plus a
deployment-provisioned setgid root, or an equivalent default ACL)
examples;
- state that cache data is reconstructible but may contain source-derived
content and operators own access and retention;
- explain that pre-feature cache trees retain their existing inode modes and
must be deliberately recreated or administratively migrated; and
- accurately limit concurrent-use claims to the existing atomicity and
validation guarantees.
3. Update `docs/config.md` only to clarify that selecting chunk-plan or
checkpoint directories does not configure their filesystem access policy.
Do not document a nonexistent permission field.
4. Update `docs/internal/state.md` with the two implemented writer paths:
rooted chunk-plan atomic creation and checkpoint use of the
process-permission file writer. Link to Operations for operator-facing mode
and provisioning guidance instead of duplicating it.
5. Review `docs/internal/overview.md` and `docs/development.md`; update links or
routing only if the existing navigation no longer reaches the canonical
state and operations documents.
6. Do not edit a historical release note. The release procedure will summarize
this change when the next release candidate is prepared.
### Validation
- Verify every changed relative Markdown link and heading anchor.
- Search current-behavior documentation for stale assertions that cache modes
are always `0700`/`0600`; distinguish cache from still-valid debug claims.
- Confirm current-behavior documentation outside `docs/roadmap/` contains
neither `cache.access` nor instructions for Notarius-managed
chmod/chown/chgrp/ACL mutation.
- Run `git diff --check`.
### Stage Acceptance
Each policy and operational fact has one canonical owner, current docs describe
only implemented behavior, and the feature roadmap remains the owner of future
status until final review. This stage is small enough for one implementation
prompt.
## Stage 5: Repository-Wide Verification And Contract Audit
### Goal
Prove the complete target state and catch accidental permission, safety, or
documentation drift before review.
### Required Review
1. Inspect production cache code for remaining `0700`, `0600`, or `Chmod`
operations. Every remaining occurrence must be a fixture, an unrelated
output/debug contract, or otherwise justified; neither cache writer may
enforce those values.
2. Search configuration, CLI, examples, and documentation to confirm no
`cache.access` surface or private/group application policy was introduced.
3. Confirm output and debug production paths and their contract tests are
unchanged except for necessary documentation disambiguation.
4. Confirm cache reads remain side-effect free where previously guaranteed and
every cache write remains confined and atomic.
5. Review all new tests against `docs/policy/testing.md`. Consolidate redundant
mode assertions while retaining one owner for each realistic regression:
kernel filtering, existing-root preservation, each cache writer, and
assembled CLI wiring.
6. Review the ADR and canonical docs for the agreed scope. Do not describe
Unix permissions as literal inheritance from a parent directory.
### Validation
Run the complete offline suite:
```sh
gofmt -w internal/core/fileio/*.go internal/framework/chunkplan/*.go internal/framework/checkpoint/*.go internal/cli/*.go
go test ./...
go test -race ./internal/core/fileio ./internal/framework/chunkplan ./internal/framework/checkpoint ./internal/cli
go vet ./...
go build ./cmd/notarius
go run ./cmd/notarius config validate --config examples/dnd-minimal.config.yml --pipeline dnd-session
OPENROUTER_API_KEY=validation-placeholder go run ./cmd/notarius config validate --config examples/dnd-complete.config.yml --pipeline dnd-session
git diff --check
```
The config validations must perform no provider requests. Also inspect
`git status --short --untracked-files=all` for unintended files, generated
artifacts, or unrelated edits.
### Final Acceptance
Every acceptance criterion in the feature roadmap is supported by direct code,
test, or documentation evidence; all prescribed commands pass; the working
diff contains only this feature; and the two roadmap documents remain in place
for the post-implementation review. This stage is small enough for one final
verification prompt.