# File Operations Simplification ## Status Accepted feature roadmap. This document defines the desired filesystem policy and target state. It is not an implementation plan and does not describe implemented behavior. It refines the filesystem conclusions in [`audit.md`](audit.md). ## Goal Make Narratio's filesystem code proportionate to its real operating model: single-invocation work under a session lock in a trusted, group-owned workspace containing intentionally shareable D&D material. Use the Go standard library directly for ordinary filesystem work and retain custom code only where it expresses a concrete Narratio requirement. The change should materially reduce production code, platform-specific code, test machinery, full-tree copying, and the number of filesystem guarantees that maintainers must preserve. ## Underlying Issue `internal/fileops` has accumulated a general confined-filesystem abstraction, two directory-promotion implementations, adversarial race hooks and inode identity rechecks, handle-relative no-replace promotion, explicit file and directory syncing, platform capability errors, universal symlink and regular- file restrictions, and tests for each mechanism. Those mechanisms are internally coherent, but they solve a stronger problem than Narratio has. Narratio does not operate on a hostile multi-writer tree, and its ordinary workspace data is not private. The session lock prevents normal concurrent mutation by Narratio, while run IDs make durable extract destinations unique. Treating every current mechanism as a permanent policy has therefore created a self-reinforcing contract: implementation detail drives tests and documentation, which in turn appear to require the implementation detail. This roadmap resets that contract around required outcomes. It does not assume that a replacement must preserve identical mechanics or failure behavior. ## Policy Decisions ### Operating and threat model - Campaign and session workspaces are trusted operational inputs, not hostile filesystems. Narratio does not defend ordinary reads and writes against an attacker racing directory entries during an invocation. - Workspace transcripts, artifacts, manifests, logs, and configuration are intended to be shareable. World readability is acceptable. Group-owned, group-writable operation remains a requirement. - API keys are the exception. Narratio must not write credentials into runtime artifacts or diagnostics, and any filesystem path that can contain API-key material retains appropriately strict handling. - Narratio must still prevent accidental destructive action or restore outside the selected application-owned root. Trusted input does not justify an unbounded delete or an escaped restore target. ### Guarantees to retain - Corruption-sensitive single-file updates use same-directory temporary files and replacement so readers do not observe partially written content. This applies to such state as manifests and generated configuration; it is an atomic-visibility requirement, not a power-loss durability promise. - Files downloaded from remote storage are completely received and validated before they become the canonical local file. - Adapter and configuration input remains bounded by centralized, discoverable constants. Limits remain deliberately far above expected workloads and produce unmistakable errors naming both the limit and the enforcing adapter. - Workspace files and directories preserve the repository's group-owned, group-writable mode policy, including after temporary-file replacement or directory rename. - Cleanup and restore operate relative to an explicitly opened root and reject targets outside that scope. Recursive removal uses the standard-library root primitive available at the maintained Go baseline. - Restore treats leaf or ancestor symlinks as non-regular conflicts rather than following them to classify or overwrite a target. This is a restore boundary, not a universal rule for ordinary workspace access. - Staging cleanup is limited to paths that Narratio created and verified as descendants of the applicable invocation or session workspace. ### Guarantees to retire - Narratio does not promise that a successful write or directory install has been forced through every operating-system and storage cache before return. File and directory `fsync` calls, directory-sync capability probes, and unsupported-durability errors are not part of the target contract. - Narratio does not defend against a concurrent adversary replacing a file or directory after it has been inspected. Inode identity snapshots, `SameFile` rechecks, and race-injection hooks are not part of the target contract. - Ordinary workspace operations do not universally reject symlinks or require every source to be a regular file. Components may validate a concrete external contract where needed, but there is no package-wide no-follow policy. - Directory installation does not require a custom atomic no-replace primitive or an exact cross-platform race guarantee. The session lock, absent-destination check, and unique run ID are the intended concurrency controls. - “Durable” in Narratio documentation means managed persistent application state. It does not mean guaranteed survival of every acknowledged update across immediate power loss. ## Required Changes ### Narrow the filesystem API Replace the broad `internal/fileops` facade with direct `os`, `io`, and `path/filepath` operations at ordinary call sites. Retain only small helpers that enforce a shared product rule rather than wrapping the standard library. Expected shared concerns are: - corruption-resistant single-file replacement with explicit final mode; - bounded reads or copies with centralized limits and contextual errors; - application workspace file and directory modes; and - narrowly scoped root-relative cleanup and restore operations. The target is not a one-for-one rewrite of existing exported functions. Remove unused or single-purpose wrappers when the calling package can express the operation clearly with the standard library. Keep platform-specific code only if it is necessary for the small atomic-file replacement helper to provide the supported cross-platform behavior. Do not add a general filesystem framework. Same-directory atomic replacement is a generic abstraction, but a small local helper is the preferred target unless implementation research identifies a maintained cross-platform dependency that removes more code than it adds and preserves the required mode behavior. ### Simplify ordinary reads, writes, and copies Use ordinary standard-library path operations for trusted workspace data. Preserve contextual errors and the existing centralized safety limits, but do not route every access through an `os.Root`, reopen files to prove identity, or reject symlinks merely because the current facade does so. Callers that consume a defined external format may still validate type, size, checksum, structure, or placement because their own contract requires it. Those checks should live with the consumer and must not reconstitute a universal FileOps policy indirectly. ### Replace file updates without power-loss machinery Use a minimal same-directory temporary-file flow for corruption-sensitive file updates: create, write, close, apply the intended mode, and replace the canonical path. Ensure temporary files are removed after pre-commit failures. Preserve context cancellation checks where a caller can still abandon an update before replacement. Remove file and parent-directory syncing, sync-related platform probes and sentinel errors, and tests that exist only to assert power-loss guarantees. If a small operating-system-specific replace shim is still required, keep it private to this helper and test its observable replacement behavior rather than its internal syscall sequence. ### Simplify download installation Keep remote downloads in a temporary file until transfer, size, checksum, and format validation succeeds. Install the validated file using the same minimal replacement helper or a direct rename when the destination is known absent. Remove standalone download-install APIs that have no production caller and avoid duplicating replacement policy between download and ordinary writes. ### Replace Notarius directory promotion Have Notarius write its output bundle to a unique staging directory within the invocation workspace. Validate that the reported bundle is inside the requested output root and satisfies the Notarius integration contract. Normalize required workspace modes, confirm that the unique durable destination is absent, and use `os.Rename` to move the validated directory directly to that destination. Delete the production and test-only directory tree-copy implementations, promotion hooks, handle-relative no-replace code, directory syncing, and their platform files. A destination collision must return a clear ordinary error; it does not require adversarial race recovery. The successfully renamed bundle becomes the sole retained artifact tree. The pre-promotion Notarius tree is not duplicated for diagnostics. Existing receipt and log records remain the troubleshooting evidence, and failed staging trees are cleaned up only after their scope is verified. ### Keep destructive and restore operations root-scoped Retain `os.Root` at boundaries where containment is the requirement. Use the maintained Go version's root-relative recursive removal for cleanup. Continue to require proper-descendant relationships and application-owned eligibility before destructive operations. Rework restore classification and installation around a session root. Inspect targets without following leaf or ancestor symlinks, classify such targets as non-regular conflicts in both force and non-force plans, and hash only a target verified within that root. Do not add inode-stability or hostile-race rechecks. ### Preserve group-owned workspace operation and protect secrets Centralize the intended workspace modes in one discoverable location and apply them consistently to created directories, ordinary files, atomic replacements, downloads, and renamed Notarius bundles. Do not tighten general workspace permissions on the assumption that transcripts or artifacts are sensitive. Audit API-key ingress separately from general workspace operations. Secret values must remain out of files unless an existing explicit credential contract requires a file, must never enter logs or errors, and must not inherit permissive workspace modes. The simplification must not weaken existing credential redaction or environment handling. ### Align documentation and tests with outcomes Revise the canonical architecture, operations, troubleshooting, and focused internal documents to remove power-loss durability, universal confinement, universal no-follow, and atomic directory-promotion claims. Keep volatile modes, paths, limits, and operational behavior in their existing canonical owners rather than duplicating them across documents. Delete tests for removed algorithms, syscall ordering, race hooks, directory- sync capability behavior, and guarantees no longer offered. Retain or add focused tests for: - successful and failed atomic file replacement on every supported platform; - cleanup of temporary files before replacement; - bounded input and clear adapter-specific limit errors; - group-owned file and directory modes; - validated download installation; - validated Notarius bundle rename, collision, and cleanup behavior; - root-scoped cleanup and refusal of escaped destructive targets; - restore handling of leaf and ancestor symlinks; and - API-key redaction and any credential-file permission contract. Tests should assert externally meaningful results and failure boundaries, not recreate the removed implementation through new private hooks. ## Compatibility and Migration This work changes internal guarantees rather than CLI, configuration, manifest, or external artifact schemas. Existing valid workspaces should remain usable. Users may observe fewer failures on filesystems that cannot sync directories, faster extract installation, less temporary disk consumption, and ordinary symlink behavior outside explicitly scoped restore and destructive operations. The removal of the duplicate pre-promotion Notarius output tree is an intentional operational change and must be reflected in the operations and troubleshooting documentation. No compatibility layer should preserve the old FileOps API solely for internal callers; migrate callers and delete obsolete surfaces together. ## Target End State At completion: - ordinary workspace file operations are clear standard-library code located near their domain owners; - shared filesystem code is small and limited to atomic file replacement, bounded I/O and modes, plus explicitly root-scoped destructive or restore helpers; - directory promotion, tree-copy duplication, adversarial race defenses, directory syncing, and unsupported-durability machinery no longer exist; - extract validates and renames one Notarius staging bundle without copying the complete tree or retaining a duplicate source tree; - manifests and other corruption-sensitive files are never intentionally exposed partially, without claiming power-loss durability; - accidental out-of-root deletion and restore remain prevented; - group-owned workspaces remain first-class, and API keys remain the only data requiring secret-specific filesystem treatment; - tests cover the reduced observable contract rather than removed mechanisms; and - canonical documentation accurately distinguishes persistent application state, atomic visibility, trusted-workspace assumptions, and destructive-operation confinement. ## Out of Scope - Changing CLI commands, configuration schemas, artifact schemas, or manifest formats. - Adding multi-process writes to one session or removing the session lock. - Defending trusted workspaces against malicious concurrent mutation. - Adding encryption or restrictive permissions for shareable campaign data. - Redesigning remote publish commit ordering or object-store concurrency. - Adding user-configurable size limits in this feature; constants should remain easy to promote to configuration later.