Plan ConfigKit adoption and clarify implementation policy
This commit is contained in:
@@ -1,2 +1,7 @@
|
|||||||
Please review `docs/development.md` for initial orientation in this repository
|
Please review `docs/development.md` for initial orientation in this repository
|
||||||
and follow its task-specific reading guide.
|
and follow its task-specific reading guide.
|
||||||
|
|
||||||
|
For every planning and implementation task, follow the
|
||||||
|
[`Implementation Discipline`](docs/policy/architecture.md#implementation-discipline):
|
||||||
|
keep machinery proportionate, do not invent restrictions, and do not elevate
|
||||||
|
incidental implementation details into repository policy.
|
||||||
|
|||||||
@@ -38,6 +38,36 @@ Notarius is contract-first without being abstraction-heavy. Interfaces and
|
|||||||
extension points should protect demonstrated boundaries. New abstraction is not
|
extension points should protect demonstrated boundaries. New abstraction is not
|
||||||
itself an architectural goal.
|
itself an architectural goal.
|
||||||
|
|
||||||
|
## Implementation Discipline
|
||||||
|
|
||||||
|
Simplify the mechanism, not the user's choices.
|
||||||
|
|
||||||
|
**Keep implementation machinery proportionate.** Use straightforward,
|
||||||
|
idiomatic code and introduce only the machinery needed to support the requested
|
||||||
|
behavior and flexibility. Optimize for conceptual simplicity and
|
||||||
|
maintainability, not for the fewest lines, fewest options, or narrowest product
|
||||||
|
surface.
|
||||||
|
|
||||||
|
**Do not invent restrictions.** Do not reject, prohibit, constrain, cap,
|
||||||
|
harden, or otherwise narrow behavior unless required by the user, an explicit
|
||||||
|
repository contract, or an unavoidable correctness constraint. Preserve
|
||||||
|
operator and caller choice where policy has not been established. If an
|
||||||
|
additional restriction appears advisable, present it as a decision rather than
|
||||||
|
silently implementing it.
|
||||||
|
|
||||||
|
**An implementation detail is not repository policy.** Tests and documentation
|
||||||
|
must not elevate incidental behavior into a durable guarantee or restriction
|
||||||
|
unless that behavior was expressly selected as part of the contract. Existing
|
||||||
|
explicit contracts remain authoritative; changing or weakening one requires a
|
||||||
|
deliberate decision rather than being inferred from this principle.
|
||||||
|
|
||||||
|
[ADR-0017](../adr/0017-defer-cache-access-policy-to-deployment.md) illustrates
|
||||||
|
these principles. Notarius previously elevated private cache creation modes
|
||||||
|
into application policy even though Unix deployments already express access
|
||||||
|
policy through directory permissions, process umask, setgid inheritance, and
|
||||||
|
default ACLs. The revised design retains application-owned correctness
|
||||||
|
boundaries while leaving deployment security policy to the operator.
|
||||||
|
|
||||||
## Layers And Dependency Direction
|
## Layers And Dependency Direction
|
||||||
|
|
||||||
The application boundary is the composition root and may depend on concrete
|
The application boundary is the composition root and may depend on concrete
|
||||||
|
|||||||
@@ -30,6 +30,18 @@ that the behavior exists. The roadmap owns implementation status and sequencing
|
|||||||
until the decision is implemented. Current architecture, user, operator,
|
until the decision is implemented. Current architecture, user, operator,
|
||||||
integration, and internal documentation are updated when the behavior lands.
|
integration, and internal documentation are updated when the behavior lands.
|
||||||
|
|
||||||
|
### Implementation Details And Policy
|
||||||
|
|
||||||
|
Documentation must distinguish an explicit contract from an incidental
|
||||||
|
implementation choice. Do not describe a restriction, guarantee, security
|
||||||
|
policy, default, or limit as authoritative merely because the current code
|
||||||
|
implements it. Documentation may accurately describe current behavior without
|
||||||
|
making it a permanent requirement; distinguish descriptive behavior from
|
||||||
|
normative policy. The
|
||||||
|
[implementation-discipline rules](architecture.md#implementation-discipline)
|
||||||
|
own the criteria for adopting product restrictions and for elevating behavior
|
||||||
|
into repository policy.
|
||||||
|
|
||||||
### Audience And Detail
|
### Audience And Detail
|
||||||
|
|
||||||
Write for the document's stated audience and include only the detail needed for
|
Write for the document's stated audience and include only the detail needed for
|
||||||
|
|||||||
@@ -87,6 +87,13 @@ Do not force all behavior through oversized end-to-end tests. Do not test every
|
|||||||
|
|
||||||
A test should protect a decision, contract, or invariant—not memorialize the current implementation.
|
A test should protect a decision, contract, or invariant—not memorialize the current implementation.
|
||||||
|
|
||||||
|
Tests must not create a stronger product contract than the requirement they
|
||||||
|
protect. In particular, do not turn an incidental limit, restriction,
|
||||||
|
permission choice, validation rule, or implementation default into policy by
|
||||||
|
asserting it without an expressly selected contract. Follow the
|
||||||
|
[implementation-discipline rules](architecture.md#implementation-discipline)
|
||||||
|
when deciding what behavior deserves durable protection.
|
||||||
|
|
||||||
Before adding or retaining a test, ask:
|
Before adding or retaining a test, ask:
|
||||||
|
|
||||||
> What realistic defect would this test catch?
|
> What realistic defect would this test catch?
|
||||||
|
|||||||
615
docs/roadmap/configkit.md
Normal file
615
docs/roadmap/configkit.md
Normal file
@@ -0,0 +1,615 @@
|
|||||||
|
# ConfigKit Adoption
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Replace Notarius's application-owned YAML parsing, presence bookkeeping,
|
||||||
|
default application, and configuration-inspection machinery with ConfigKit,
|
||||||
|
while preserving Notarius ownership of pipeline semantics, module catalogs,
|
||||||
|
runtime resolution, and invocation behavior.
|
||||||
|
|
||||||
|
This roadmap is based on ConfigKit v0.2.0, the current stable release in the
|
||||||
|
local `../ConfigKit` checkout. ConfigKit's maintained consumer guide includes a
|
||||||
|
Notarius-shaped example, and its complete test suite passes locally. The
|
||||||
|
dependency starts at v0.2.0; completing source-aware normalization requires a
|
||||||
|
reviewed release with the extension below. Do not follow an unreviewed branch
|
||||||
|
tip. Ordered implementation work belongs in
|
||||||
|
[the implementation plan](implementation.md).
|
||||||
|
|
||||||
|
### Source Metadata Dependency
|
||||||
|
|
||||||
|
Review for implementation identified an API gap in v0.2.0: `Resolution`
|
||||||
|
exposes input presence and change recording, but not declaration origins or
|
||||||
|
source paths. Those are available only on the completed `Result`, whose
|
||||||
|
effective and explanation snapshots are already frozen. The target of
|
||||||
|
recording declaration-relative path normalization in those snapshots therefore
|
||||||
|
requires an upstream extension. The selected approach is to request source
|
||||||
|
origins and canonical source records during resolution, then adopt a reviewed
|
||||||
|
release providing that API. The implementation plan records the dependency
|
||||||
|
gate. Resolving paths only after loading with a separate inspection report was
|
||||||
|
considered but not selected; the agreed effective/explanation contract remains
|
||||||
|
the target. ConfigKit v0.2.0 alone is insufficient for that part of the target.
|
||||||
|
|
||||||
|
## Motivation
|
||||||
|
|
||||||
|
The current configuration boundary under `internal/core/config` works, but it
|
||||||
|
reimplements a substantial general-purpose configuration system. In
|
||||||
|
particular, Notarius currently owns:
|
||||||
|
|
||||||
|
- strict YAML setup, duplicate-key and trailing-document checks;
|
||||||
|
- custom YAML decoders for module bindings, references, pipelines, steps, and
|
||||||
|
lanes;
|
||||||
|
- pointer and private-flag bookkeeping to distinguish omitted values from
|
||||||
|
explicit zero, false, empty, or null values;
|
||||||
|
- a large file-to-runtime application function that combines normalization,
|
||||||
|
validation, conversion, and default derivation;
|
||||||
|
- operational environment overlays and their separate validation path; and
|
||||||
|
- bespoke cloning and redaction for effective configuration summaries.
|
||||||
|
|
||||||
|
ConfigKit already provides strict single-document YAML parsing, typed decode,
|
||||||
|
input presence, typed defaults, recorded normalization and derived defaults,
|
||||||
|
semantic diagnostics with source locations, detached results, redacted
|
||||||
|
effective rendering, and value explanation. Adopting those facilities should
|
||||||
|
reduce application-specific code and make configuration failures easier for an
|
||||||
|
operator to understand without moving domain policy into the dependency.
|
||||||
|
|
||||||
|
This is a configuration-contract migration, not a mechanical parser swap.
|
||||||
|
ConfigKit deliberately rejects several conveniences supported by version 4,
|
||||||
|
and its dynamic integer representation differs from `gopkg.in/yaml.v3`.
|
||||||
|
|
||||||
|
## Target End State
|
||||||
|
|
||||||
|
After this work:
|
||||||
|
|
||||||
|
- every command uses one ConfigKit-backed loading path;
|
||||||
|
- Notarius configuration files declare schema version 5 using the existing
|
||||||
|
integer YAML convention;
|
||||||
|
- the file schema uses canonical object forms rather than YAML type-dependent
|
||||||
|
shorthand;
|
||||||
|
- operators can import whole pipeline definitions and select a configuration
|
||||||
|
profile for development, testing, or deployment;
|
||||||
|
- defaults, surviving-input presence, normalization, derived values, and
|
||||||
|
configuration-only validation are expressed through one ConfigKit
|
||||||
|
`Definition`;
|
||||||
|
- ConfigKit diagnostics are rendered safely with semantic paths and available
|
||||||
|
source locations at the CLI boundary;
|
||||||
|
- the successful load result retains ConfigKit metadata for safe effective
|
||||||
|
configuration rendering and explanation;
|
||||||
|
- dynamic module-option integers have one documented `int64` configuration
|
||||||
|
representation and are range-checked when converted to runtime `int` values;
|
||||||
|
- pipeline and module resolution remains in Notarius and still fails before
|
||||||
|
module preparation when a configured composition is invalid;
|
||||||
|
- configuration discovery and invocation-only CLI state remain application
|
||||||
|
responsibilities;
|
||||||
|
- `NOTARIUS_CONFIG` remains a configuration-discovery input, while the seven
|
||||||
|
field-level operational environment overlays are retired;
|
||||||
|
- read-only CLI commands expose ConfigKit's redacted effective configuration
|
||||||
|
and per-path provenance explanations; and
|
||||||
|
- the direct `gopkg.in/yaml.v3` dependency and the superseded custom parsing,
|
||||||
|
presence, environment-overlay, and configuration-redaction code are removed.
|
||||||
|
|
||||||
|
The migration must preserve the fixed pipeline architecture, deterministic
|
||||||
|
resolved-pipeline identity, detached ownership, fail-before-execution checks,
|
||||||
|
secret exclusion, source-relative reference behavior, and offline validation
|
||||||
|
requirements defined by the architecture and current integration contracts.
|
||||||
|
|
||||||
|
## Ownership Boundaries
|
||||||
|
|
||||||
|
### ConfigKit owns
|
||||||
|
|
||||||
|
- reading and bounding the selected YAML source, its declared imports, and the
|
||||||
|
selected configuration profile after Notarius chooses them;
|
||||||
|
- composing imported definitions and applying the selected profile overlay;
|
||||||
|
- enforcing a single strict YAML document, supported scalar forms, unique
|
||||||
|
mapping keys, and known typed fields;
|
||||||
|
- decoding into the application-defined configuration schema;
|
||||||
|
- distinguishing omitted input from explicit zero, false, empty, and null;
|
||||||
|
- applying typed application defaults;
|
||||||
|
- recording application normalization and derived defaults;
|
||||||
|
- attaching application validation diagnostics to semantic paths and source
|
||||||
|
locations;
|
||||||
|
- detaching the successful configuration and its metadata;
|
||||||
|
- rendering the redacted effective configuration; and
|
||||||
|
- explaining the provenance of an effective or historical configuration path.
|
||||||
|
|
||||||
|
### Notarius continues to own
|
||||||
|
|
||||||
|
- configuration-file discovery through `--config`, `NOTARIUS_CONFIG`, and the
|
||||||
|
installed default path;
|
||||||
|
- the version policy and the application configuration structs;
|
||||||
|
- the meaning and validation of every Notarius field;
|
||||||
|
- pipeline, module, validator, artifact, and reference catalogs;
|
||||||
|
- conversion from configuration-facing values to runtime pipeline types;
|
||||||
|
- source-relative interpretation of configured external reference paths;
|
||||||
|
- run-time selectors and actions such as lane selection, input, resume,
|
||||||
|
recomputation, session identity, and CLI reference changes;
|
||||||
|
- PromptKit profile discovery and validation; and
|
||||||
|
- redaction of resolved runtime structures that are outside ConfigKit's typed
|
||||||
|
configuration snapshot, including exclusion of materialized reference
|
||||||
|
content.
|
||||||
|
|
||||||
|
ConfigKit configuration profiles must not be confused with PromptKit LLM
|
||||||
|
profiles or Notarius named pipeline definitions. ConfigKit does not become a
|
||||||
|
module registry, pipeline resolver, command framework, or secret manager for
|
||||||
|
PromptKit credentials.
|
||||||
|
|
||||||
|
## Configuration Contract Version 5
|
||||||
|
|
||||||
|
### Version spelling
|
||||||
|
|
||||||
|
Use the following root declaration:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
version: 5
|
||||||
|
```
|
||||||
|
|
||||||
|
ConfigKit v0.2.0 accepts string or strict decimal-integer input at the exact
|
||||||
|
path selected by its `VersionRule`. It passes the original spelling to the
|
||||||
|
application checker and can decode an accepted integer into a string-backed Go
|
||||||
|
field. This narrow exception lets Notarius retain its existing numeric YAML
|
||||||
|
convention without weakening strict decoding elsewhere. Quoted `version: "5"`
|
||||||
|
is also accepted by ConfigKit, but the maintained Notarius spelling is the
|
||||||
|
unquoted integer shown above.
|
||||||
|
|
||||||
|
The version checker must reject absent, null, and unsupported identifiers with
|
||||||
|
safe, value-free diagnostics at `/version`. Version 5 is a clean break: the
|
||||||
|
loader must not decode or rewrite a version 4 document and must not retain the
|
||||||
|
legacy loader as a fallback.
|
||||||
|
|
||||||
|
### Canonical module bindings
|
||||||
|
|
||||||
|
Every module and validator binding must use an object with a `module` field.
|
||||||
|
For example:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
input:
|
||||||
|
module: seriatim
|
||||||
|
extract:
|
||||||
|
module: dnd/spells
|
||||||
|
merge:
|
||||||
|
module: appendorder
|
||||||
|
normalize:
|
||||||
|
module: dnd/spells
|
||||||
|
```
|
||||||
|
|
||||||
|
Version 5 does not accept scalar binding shorthand. Preserving shorthand would
|
||||||
|
require a dynamic subtree or a compatibility pre-parser and would retain much
|
||||||
|
of the complexity this adoption is intended to remove.
|
||||||
|
|
||||||
|
### Canonical reference sources
|
||||||
|
|
||||||
|
Reference sources must also use explicit objects. An external file reference
|
||||||
|
uses a `path` field; a generated reference retains its typed `artifact`
|
||||||
|
selector. For example:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
references:
|
||||||
|
party:
|
||||||
|
path: ./dnd-party.txt
|
||||||
|
npc_registry:
|
||||||
|
artifact:
|
||||||
|
step: describe-session
|
||||||
|
lane: npc-registry
|
||||||
|
```
|
||||||
|
|
||||||
|
Pipeline-level references remain external-file-only. Step- and binding-local
|
||||||
|
references may select earlier artifacts according to the existing ordered
|
||||||
|
handoff rules. Relative external paths resolve from the file that declared the
|
||||||
|
surviving value, including imported fragments and configuration profiles. CLI
|
||||||
|
reference paths continue to resolve from the process working directory.
|
||||||
|
|
||||||
|
Remove the lane-level `references` compatibility alias in version 5. New files
|
||||||
|
must place a lane's extraction references under `extract.references`. This
|
||||||
|
leaves one canonical location and removes merge-precedence behavior from the
|
||||||
|
file converter.
|
||||||
|
|
||||||
|
### Strictness and dynamic options
|
||||||
|
|
||||||
|
Keep ConfigKit's strict YAML rules. Version 5 therefore rejects aliases, merge
|
||||||
|
keys, unsupported tags and scalar forms, duplicate mapping keys, unknown typed
|
||||||
|
fields, trailing documents, and incompatible YAML types. Do not recreate an
|
||||||
|
application pre-parser to weaken these rules.
|
||||||
|
|
||||||
|
Module `options` remain `map[string]any` where the selected module determines
|
||||||
|
the valid keys. ConfigKit represents YAML integers in dynamic values as
|
||||||
|
`int64`. Module option validators must accept that canonical representation,
|
||||||
|
check range before conversion, and hand runtime modules a safe `int` only where
|
||||||
|
their APIs require it. Do not marshal and decode dynamic subtrees through YAML
|
||||||
|
to recover native `int` values.
|
||||||
|
|
||||||
|
## ConfigKit-Backed Loading Model
|
||||||
|
|
||||||
|
Define a plain, typed configuration document for ConfigKit. It should mirror
|
||||||
|
the public file schema, use ordinary YAML tags, and contain no custom YAML
|
||||||
|
unmarshal hooks. Keep it separate from framework runtime types where doing so
|
||||||
|
avoids YAML concerns, presence flags, or configuration-only unions leaking into
|
||||||
|
`internal/framework/pipeline`.
|
||||||
|
|
||||||
|
The reusable ConfigKit definition should provide:
|
||||||
|
|
||||||
|
- fresh built-in defaults for each load;
|
||||||
|
- the `/version` rule;
|
||||||
|
- recorded whitespace normalization for identifiers, paths, profile IDs,
|
||||||
|
endpoints, and other fields whose current contract normalizes input;
|
||||||
|
- recorded derivation of `concurrency.stage_workers.extract` from the effective
|
||||||
|
`concurrency.total_llm` only when the worker value was omitted;
|
||||||
|
- collision checks before normalizing map keys or cross-step identifiers;
|
||||||
|
- configuration-only semantic validation with precise paths and related paths
|
||||||
|
where useful; and
|
||||||
|
- a dynamic secret predicate that covers credential-shaped keys below module
|
||||||
|
option maps, preserving the current defensive redaction boundary even though
|
||||||
|
ordinary Notarius configuration contains no credential values.
|
||||||
|
|
||||||
|
Use ConfigKit's resolver only for deterministic configuration normalization and
|
||||||
|
derived defaults. Use its read-only semantic validator for domain constraints.
|
||||||
|
Do not put module lookup, PromptKit profile lookup, filesystem resource checks,
|
||||||
|
or runtime pipeline construction in either callback.
|
||||||
|
|
||||||
|
After a successful load, perform one explicit, independently tested conversion
|
||||||
|
from the canonical configuration document to the Notarius runtime `Config` and
|
||||||
|
pipeline profile types. The converter should be a narrow ownership boundary,
|
||||||
|
not a second parser or validation framework. It may translate enums and checked
|
||||||
|
numeric values, but it must not silently change already validated meaning.
|
||||||
|
|
||||||
|
Retain the ConfigKit result alongside the runtime configuration in an
|
||||||
|
application-owned loaded-configuration value. Commands that need only the
|
||||||
|
runtime `Config` should not gain access to parser internals; commands and debug
|
||||||
|
writers that expose safe configuration provenance may use the retained result.
|
||||||
|
|
||||||
|
## Imports And Configuration Profiles
|
||||||
|
|
||||||
|
Enable ConfigKit's explicit imports using its existing `configkit.imports`
|
||||||
|
syntax, including path, glob, and optional declarations. Operators can keep
|
||||||
|
whole pipeline definitions in separate files and compose them through the
|
||||||
|
selected root file. The root file alone declares `version`.
|
||||||
|
|
||||||
|
Use ConfigKit's strict composition semantics. Imports combine disjoint
|
||||||
|
definitions; they do not provide ordered override layers or concatenate
|
||||||
|
sequences. In particular, one pipeline's ordered `steps` list remains together
|
||||||
|
in a single definition. Do not add Notarius-specific step merging or a second
|
||||||
|
import mechanism.
|
||||||
|
|
||||||
|
Expose `--config-profile <name>` on every command that loads configuration:
|
||||||
|
`run`, `config validate`, `pipelines list`, `config effective`, and
|
||||||
|
`config explain`. Pass the selection to ConfigKit. A missing selection uses
|
||||||
|
the composed base configuration. ConfigKit's reserved selection `default`
|
||||||
|
also means no overlay; it does not load a default profile file.
|
||||||
|
|
||||||
|
Use ConfigKit's profile location `<configuration-root>/profiles/<name>.yml`
|
||||||
|
and existing selection rules. A selected missing profile fails loading.
|
||||||
|
Profiles recursively overlay mappings and replace scalars, sequences, and
|
||||||
|
nulls as specified upstream. Development profiles can change pipeline-level
|
||||||
|
`llm_profile`, concurrency, and output settings without repeating the pipeline.
|
||||||
|
Changing an individual binding inside `steps` requires replacing the complete
|
||||||
|
steps sequence. Document this limitation prominently in examples.
|
||||||
|
|
||||||
|
Configuration profiles and PromptKit profiles are independent. The former
|
||||||
|
select a configuration overlay with `--config-profile`; the latter select LLM
|
||||||
|
settings through `llm_profile` and the existing `--llm-profile` override. Neither
|
||||||
|
selection implicitly chooses the other.
|
||||||
|
|
||||||
|
Use the selected file's canonical directory as ConfigKit's configuration root.
|
||||||
|
Document the upstream requirement that imports and configuration profiles stay
|
||||||
|
within that root, along with composition conflicts, cycle/repeated-import
|
||||||
|
errors, and profile sequence replacement. This containment rule does not apply
|
||||||
|
to Notarius input, reference, output, cache, debug, or PromptKit profile paths.
|
||||||
|
|
||||||
|
### Paths And Source Ownership
|
||||||
|
|
||||||
|
Resolve relative external reference paths and `promptkit.profile_file` and
|
||||||
|
`promptkit.profile_dir` relative to the file that supplied the surviving value.
|
||||||
|
This deliberately changes PromptKit profile-source paths from their current
|
||||||
|
working-directory convention in version 5. Absolute paths remain absolute.
|
||||||
|
Keep output, cache, and debug relative-directory semantics unchanged; they are
|
||||||
|
runtime destinations rather than files bundled with configuration fragments.
|
||||||
|
|
||||||
|
Use ConfigKit's source metadata to identify each surviving declaration's
|
||||||
|
origin, including profile replacements and values inherited from the base.
|
||||||
|
Record path normalization so effective output and explanation reflect the
|
||||||
|
paths used at runtime. Preserve attribution when identifier normalization
|
||||||
|
renames an enclosing map key. Do not resolve overwritten or removed path
|
||||||
|
values or infer source ownership from diagnostic display labels. Explicit CLI
|
||||||
|
paths retain their working-directory interpretation.
|
||||||
|
|
||||||
|
## Discovery, Precedence, And Overrides
|
||||||
|
|
||||||
|
Keep the existing file-discovery order: explicit `--config`, then non-empty
|
||||||
|
`NOTARIUS_CONFIG`, then the installed default path. `NOTARIUS_CONFIG` selects a
|
||||||
|
source and is not a field-level configuration override, so it remains outside
|
||||||
|
ConfigKit.
|
||||||
|
|
||||||
|
Retire these seven field-level environment overlays in version 5:
|
||||||
|
|
||||||
|
- `NOTARIUS_TOTAL_LLM_CONCURRENCY`;
|
||||||
|
- `NOTARIUS_STAGE_WORKERS_EXTRACT`;
|
||||||
|
- `NOTARIUS_OUTPUT_DIR`;
|
||||||
|
- `NOTARIUS_CACHE_CHUNK_PLANS_MODE`;
|
||||||
|
- `NOTARIUS_CACHE_CHUNK_PLANS_DIR`;
|
||||||
|
- `NOTARIUS_CACHE_CHECKPOINTS_DIR`; and
|
||||||
|
- `NOTARIUS_DEBUG_DIR`.
|
||||||
|
|
||||||
|
ConfigKit deliberately does not model general environment overlays. Applying
|
||||||
|
them after a successful load would make the retained effective configuration,
|
||||||
|
origins, and explanations disagree with the values used at runtime. Operators
|
||||||
|
must put durable values in YAML and use the supported explicit CLI flags for
|
||||||
|
invocation-local changes.
|
||||||
|
|
||||||
|
Classify the current run flags by responsibility. A flag that directly
|
||||||
|
overrides a field in the typed configuration document should become a
|
||||||
|
ConfigKit `Override` record only when the operator explicitly supplied it;
|
||||||
|
parser defaults must not be submitted as overrides. Input selection, lane
|
||||||
|
selection, session identity, resume/recompute actions, command-line references,
|
||||||
|
and similar invocation state remain outside ConfigKit even when they affect
|
||||||
|
runtime resolution. The documented precedence becomes built-in defaults,
|
||||||
|
composed YAML sources, the selected configuration profile, explicit
|
||||||
|
configuration-valued CLI overrides, and then
|
||||||
|
invocation-only resolution actions at their existing application boundary.
|
||||||
|
|
||||||
|
## Validation And Diagnostics
|
||||||
|
|
||||||
|
Move configuration-only checks currently split among parsing, file
|
||||||
|
application, and `Config.Validate` into the ConfigKit
|
||||||
|
definition when they can be expressed against the typed file contract. Report
|
||||||
|
all independently discoverable semantic errors in one validation pass rather
|
||||||
|
than failing at the first field, while avoiding cascaded diagnostics whose only
|
||||||
|
cause is an earlier invalid parent.
|
||||||
|
|
||||||
|
Keep catalog-dependent checks in `Config.Resolve` and
|
||||||
|
`pipeline.ResolvePipeline`. These include registered module and validator
|
||||||
|
existence, stage capabilities, artifact compatibility, reference-slot
|
||||||
|
compatibility, PromptKit profile availability, and the concrete resolved
|
||||||
|
pipeline digest. Their existing error classification and fail-before-prepare
|
||||||
|
behavior must remain intact.
|
||||||
|
|
||||||
|
At the CLI boundary, recognize `*configkit.LoadError` and use ConfigKit's
|
||||||
|
canonical diagnostic renderer. Operators should receive the diagnostic code,
|
||||||
|
semantic path, safe message, and source coordinates when available. Do not
|
||||||
|
append raw YAML parser errors, source fragments, configuration values, or
|
||||||
|
secret values to the outward error chain. File discovery/read context should
|
||||||
|
remain clear without wrapping the same failure repeatedly.
|
||||||
|
|
||||||
|
Nonfatal ConfigKit diagnostics, if Notarius defines any, must not be silently
|
||||||
|
dropped. They should enter the existing process-warning model only when they
|
||||||
|
represent an actionable configuration degradation. Ordinary normalization,
|
||||||
|
defaulting, and provenance records are explanation data, not run warnings.
|
||||||
|
|
||||||
|
## Effective Configuration And Redaction
|
||||||
|
|
||||||
|
Use `Result.WriteEffective` as the canonical renderer for the loaded effective
|
||||||
|
configuration. The requested debug bundle should store this deterministic,
|
||||||
|
redacted YAML rather than a separately maintained JSON clone of the raw
|
||||||
|
configuration. Update the artifact name and documentation together if the
|
||||||
|
format changes.
|
||||||
|
|
||||||
|
Use ConfigKit's secret predicate to redact credential-shaped dynamic option
|
||||||
|
paths. PromptKit credentials remain environment references in PromptKit-owned
|
||||||
|
profile files and must not be copied into Notarius configuration or debug
|
||||||
|
state.
|
||||||
|
|
||||||
|
Do not remove Notarius's redaction of the resolved pipeline merely because the
|
||||||
|
loaded configuration is ConfigKit-backed. The resolved value contains
|
||||||
|
application-owned structures, selection results, and reference provenance that
|
||||||
|
ConfigKit does not own; materialized reference bytes must continue to be
|
||||||
|
excluded from ordinary summaries. Remove only manual configuration cloning and
|
||||||
|
redaction made redundant by the frozen ConfigKit result.
|
||||||
|
|
||||||
|
## Configuration Inspection Commands
|
||||||
|
|
||||||
|
Add these read-only commands as first-class ConfigKit-backed inspection
|
||||||
|
surfaces:
|
||||||
|
|
||||||
|
```text
|
||||||
|
notarius config effective [--config path/to/config.yml] [--config-profile name]
|
||||||
|
notarius config explain [--config path/to/config.yml] [--config-profile name] <json-pointer>
|
||||||
|
```
|
||||||
|
|
||||||
|
`config effective` loads the selected configuration through the production
|
||||||
|
path and writes only ConfigKit's deterministic redacted YAML to standard
|
||||||
|
output. `config explain` accepts one canonical JSON Pointer, loads through the
|
||||||
|
same path, and writes ConfigKit's deterministic redacted explanation for that
|
||||||
|
effective or historical path. Neither command resolves a pipeline, prepares a
|
||||||
|
module, requires provider credentials, or contacts a provider.
|
||||||
|
|
||||||
|
Both commands must use ordinary configuration discovery, ConfigKit diagnostic
|
||||||
|
rendering, the same import and profile selection as execution, and the existing
|
||||||
|
CLI exit-status conventions. An unknown or
|
||||||
|
noncanonical explanation path is a command failure, not an empty successful
|
||||||
|
result. Standard output contains only the requested inspection output;
|
||||||
|
diagnostics go to standard error. Effective output is YAML and explanation is
|
||||||
|
ConfigKit's human-readable text; neither is promised to be reloadable
|
||||||
|
configuration or a new durable JSON API. There is no unredacted mode.
|
||||||
|
|
||||||
|
## Required Code And Contract Changes
|
||||||
|
|
||||||
|
The implementation must address the following coherent work areas:
|
||||||
|
|
||||||
|
- add and pin a reviewed `gitea.maximumdirect.net/eric/ConfigKit` release with
|
||||||
|
v0.2.0's semantics and the required source-origin API;
|
||||||
|
- introduce the version 5 typed document and its ConfigKit definition;
|
||||||
|
- replace `LoadFileConfig`, custom `UnmarshalYAML` implementations, manual
|
||||||
|
strict-parser checks, presence flags, `ApplyFileConfig`, and redundant
|
||||||
|
configuration cloning with the ConfigKit load boundary;
|
||||||
|
- consolidate defaults, normalization, and configuration-only validation into
|
||||||
|
the definition while retaining runtime catalog validation;
|
||||||
|
- update CLI configuration loading and error rendering without changing file
|
||||||
|
discovery;
|
||||||
|
- enable ConfigKit imports and configuration profiles, wire `--config-profile`
|
||||||
|
through all configuration-loading commands, and retain declaration origins
|
||||||
|
for relative file paths;
|
||||||
|
- retire field-level environment overlays while retaining `NOTARIUS_CONFIG`;
|
||||||
|
- classify current run flags so only true configuration overrides use
|
||||||
|
ConfigKit override records and invocation-only state remains outside the
|
||||||
|
configuration loader;
|
||||||
|
- update dynamic module-option decoding and range checks for `int64`;
|
||||||
|
- integrate ConfigKit effective rendering and metadata into requested debug
|
||||||
|
bundles and the new `config effective` and `config explain` commands;
|
||||||
|
- remove superseded code, tests, and the direct `gopkg.in/yaml.v3` dependency
|
||||||
|
after no production or test-only use remains; and
|
||||||
|
- update maintained examples and canonical documentation in the same change as
|
||||||
|
each externally visible contract adjustment.
|
||||||
|
|
||||||
|
## Documentation And Migration
|
||||||
|
|
||||||
|
Update `docs/config.md` as the canonical owner of version 5 syntax, defaults,
|
||||||
|
precedence, canonical bindings and references, strict YAML behavior, and the
|
||||||
|
retirement of field-level environment overrides. Replace the version 3
|
||||||
|
migration section with a concise version 4-to-5 migration guide covering:
|
||||||
|
|
||||||
|
- changing the numeric version value from 4 to 5;
|
||||||
|
- expanding scalar module bindings into `{module: ...}` objects;
|
||||||
|
- expanding scalar external references into `{path: ...}` objects;
|
||||||
|
- moving lane-level reference aliases to `extract.references`;
|
||||||
|
- resolving configured PromptKit profile-source paths from their declaring
|
||||||
|
file instead of the working directory;
|
||||||
|
- using imports for whole pipeline definitions and configuration profiles for
|
||||||
|
overlays, including complete replacement of step sequences;
|
||||||
|
- replacing retired operational environment overrides with YAML or supported
|
||||||
|
explicit CLI flags; and
|
||||||
|
- any effective/debug artifact or inspection-command changes.
|
||||||
|
|
||||||
|
Update `docs/internal/configuration.md` to describe the ConfigKit boundary,
|
||||||
|
typed document-to-runtime conversion, application-owned validation, retained
|
||||||
|
metadata, and catalog-resolution handoff. Update `docs/cli.md` only for changed
|
||||||
|
commands or diagnostic behavior, `docs/operations.md` only for operator-facing
|
||||||
|
configuration and debug workflows, and the relevant architecture text if the
|
||||||
|
adopted precedence model changes.
|
||||||
|
|
||||||
|
Convert both maintained D&D configurations and every configuration snippet to
|
||||||
|
version 5 canonical syntax. Examples must remain secret-free and load through
|
||||||
|
the same production path exercised by the CLI. Do not keep a second set of
|
||||||
|
version 4 examples.
|
||||||
|
|
||||||
|
Keep the minimal D&D example self-contained. Extend the complete D&D example
|
||||||
|
with an imported whole pipeline definition and a selectable development
|
||||||
|
configuration profile. Keep configuration overlays physically distinct from
|
||||||
|
the maintained PromptKit profile files so readers can see which system owns
|
||||||
|
each file. Test the complete example both with and without the development
|
||||||
|
overlay, and update subprocess guidance to show optional `--config-profile`
|
||||||
|
selection. All configuration inspection commands must document what the
|
||||||
|
selected profile changes and that runtime pipeline resolution is separate.
|
||||||
|
|
||||||
|
Because this change establishes a durable dependency and reallocates ownership
|
||||||
|
at a core boundary, add an ADR recording the decision to use ConfigKit, the
|
||||||
|
responsibilities retained by Notarius, the deliberate version 5 break, and the
|
||||||
|
retirement of the application environment-overlay layer. The rejected
|
||||||
|
alternatives should include continuing the custom loader, retaining version 4
|
||||||
|
temporarily, or wrapping ConfigKit in a compatibility pre-parser. The ADR
|
||||||
|
should record rationale, not duplicate the current configuration reference or
|
||||||
|
implementation details.
|
||||||
|
|
||||||
|
## Testing And Verification
|
||||||
|
|
||||||
|
Follow the repository's behavior-focused testing policy. Reuse ConfigKit's own
|
||||||
|
tests for generic YAML and metadata mechanics; Notarius tests should protect
|
||||||
|
only its contract and integration responsibilities.
|
||||||
|
|
||||||
|
Required Notarius coverage includes:
|
||||||
|
|
||||||
|
- successful loading of omitted, explicit zero/false/empty, and derived values
|
||||||
|
that materially affect Notarius behavior;
|
||||||
|
- numeric and quoted version 5 acceptance and safe rejection of absent, null,
|
||||||
|
version 4, unsupported identifiers, and non-integer numeric forms;
|
||||||
|
- canonical binding and reference decoding plus rejection of former shorthand;
|
||||||
|
- identifier normalization and normalized-collision rejection;
|
||||||
|
- aggregation and source location of representative application validation
|
||||||
|
errors;
|
||||||
|
- unchanged configuration discovery and source-relative reference resolution;
|
||||||
|
- imported whole pipeline definitions, representative composition conflicts,
|
||||||
|
and profile precedence through the real ConfigKit loader;
|
||||||
|
- profile sequence replacement and explicit clearing, so Notarius does not
|
||||||
|
accidentally append or merge ordered steps;
|
||||||
|
- relative reference and PromptKit profile-source paths from root files,
|
||||||
|
imported fragments, and overlays, including an unchanged inherited value,
|
||||||
|
a replaced value, and an enclosing normalized identifier;
|
||||||
|
- matching configuration-profile selection across execution, validation,
|
||||||
|
pipeline listing, effective output, and explanation, without provider calls;
|
||||||
|
- checked dynamic integer conversion at each production module option boundary;
|
||||||
|
- detached ownership of loaded configuration and resolved pipelines;
|
||||||
|
- safe CLI rendering of representative ConfigKit parse, decode, version,
|
||||||
|
resolution, and validation failures;
|
||||||
|
- successful redacted `config effective` output and effective, historical,
|
||||||
|
unknown, and noncanonical `config explain` path behavior;
|
||||||
|
- effective-configuration redaction, including nested credential-shaped dynamic
|
||||||
|
option keys, without exposing materialized references;
|
||||||
|
- maintained minimal and complete example validation through the production
|
||||||
|
loader and production module catalog; and
|
||||||
|
- representative end-to-end CLI validation and pipeline resolution without
|
||||||
|
network access or credentials.
|
||||||
|
|
||||||
|
Delete or rewrite legacy parser tests whose only purpose is to retest behavior
|
||||||
|
owned by ConfigKit. Retain a small integration sample for each strictness rule
|
||||||
|
that is itself part of the Notarius public contract, and avoid assertions on
|
||||||
|
incidental diagnostic prose.
|
||||||
|
|
||||||
|
The completed implementation must pass:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test ./...
|
||||||
|
go vet ./...
|
||||||
|
go build ./cmd/notarius
|
||||||
|
```
|
||||||
|
|
||||||
|
It must also validate both maintained example files with the production CLI.
|
||||||
|
The complete example may use a placeholder credential for offline PromptKit
|
||||||
|
profile validation, as documented today; no live provider call is required.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- Moving pipeline resolution, module catalogs, or PromptKit profiles into
|
||||||
|
ConfigKit.
|
||||||
|
- Adding arbitrary environment-to-field binding.
|
||||||
|
- Retaining the retired field-level operational environment variables as a
|
||||||
|
parallel precedence layer.
|
||||||
|
- Loading credentials into the Notarius configuration document.
|
||||||
|
- Supporting remote configuration sources or formats other than YAML.
|
||||||
|
- Adding a generic dynamic-subtree conversion layer.
|
||||||
|
- Composing ordered steps across fragments or adding application-specific
|
||||||
|
sequence merge, profile inheritance, or profile stacking semantics.
|
||||||
|
- Preserving scalar shorthand with an application pre-parser.
|
||||||
|
- Changing the fixed pipeline shape or durable extraction artifacts.
|
||||||
|
- Changing run input, output publication, cache, checkpoint, retry, or warning
|
||||||
|
semantics except where configuration loading or debug rendering directly
|
||||||
|
exposes them.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- All configuration-loading commands use a pinned ConfigKit release providing
|
||||||
|
the required source-origin API, building on v0.2.0, and no production code
|
||||||
|
directly imports a YAML parser.
|
||||||
|
- Version 5 is the only accepted file contract, documents `version: 5` as its
|
||||||
|
canonical spelling, and has one canonical object representation for module
|
||||||
|
bindings and references.
|
||||||
|
- The ConfigKit definition is the sole owner of typed defaults, input presence,
|
||||||
|
recorded normalization, derived configuration defaults, and
|
||||||
|
configuration-only semantic diagnostics.
|
||||||
|
- Notarius remains the sole owner of catalog-dependent resolution and runtime
|
||||||
|
conversion.
|
||||||
|
- Dynamic option integers are handled without unchecked narrowing or YAML
|
||||||
|
round-trips.
|
||||||
|
- CLI diagnostics and effective configuration output are deterministic,
|
||||||
|
redacted, value-safe, and retain useful path/source context.
|
||||||
|
- `config effective` and `config explain` expose only ConfigKit's redacted
|
||||||
|
inspection views and perform no pipeline or provider work.
|
||||||
|
- Imports compose whole pipeline definitions and selected configuration
|
||||||
|
profiles apply ConfigKit's documented overlay semantics consistently across
|
||||||
|
all configuration-loading commands.
|
||||||
|
- Relative external references and PromptKit profile-source paths use the
|
||||||
|
surviving declaration's source directory, and safe inspection agrees with
|
||||||
|
the paths used at runtime.
|
||||||
|
- The resolved pipeline remains detached, reference content remains excluded
|
||||||
|
from ordinary summaries, and its identity changes under the same semantic
|
||||||
|
conditions as before.
|
||||||
|
- Maintained examples, current documentation, focused tests, and repository-wide
|
||||||
|
validation agree with the implemented version 5 contract.
|
||||||
|
- `NOTARIUS_CONFIG` still participates in discovery, while the seven
|
||||||
|
field-level operational environment variables no longer affect loaded
|
||||||
|
configuration.
|
||||||
|
- Superseded custom parser, presence, environment, clone, redaction, and test
|
||||||
|
code is removed rather than left as an alternate loading path.
|
||||||
|
|
||||||
|
## Deferred Composition Work
|
||||||
|
|
||||||
|
Step-level composition may be revisited after operators use imported whole
|
||||||
|
pipeline definitions. It is not part of this adoption: ConfigKit does not
|
||||||
|
concatenate sequences or merge their elements by application-defined IDs.
|
||||||
|
Any future extension must define ordering, duplicate step IDs, dependency
|
||||||
|
validation, profile overrides, and provenance before changing that contract.
|
||||||
@@ -1,236 +0,0 @@
|
|||||||
# Deployment-Managed Cache Permissions
|
|
||||||
|
|
||||||
Status: proposed
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Stop treating cache access permissions as application policy. Notarius should
|
|
||||||
create chunk-plan and checkpoint state through conventional Unix filesystem
|
|
||||||
semantics and leave the effective owner, group, and access policy to the
|
|
||||||
operator's provisioned directories, process umask, setgid inheritance, and
|
|
||||||
default ACLs.
|
|
||||||
|
|
||||||
Notarius continues to own safe paths, symlink rejection, atomic publication,
|
|
||||||
cache validation, and recovery. It does not decide whether a configured cache
|
|
||||||
is private to one account, shared by a trusted group, or readable under some
|
|
||||||
other deployment policy.
|
|
||||||
|
|
||||||
## Motivation And Current Problem
|
|
||||||
|
|
||||||
The current cache implementations enforce a policy that was stronger than the
|
|
||||||
application requires:
|
|
||||||
|
|
||||||
- the chunk-plan store creates and repeatedly converges cache directories to
|
|
||||||
`0700` and atomic files to `0600`; and
|
|
||||||
- the checkpoint recorder supplies `0700` directory and `0600` file modes to
|
|
||||||
the shared confined writer, which explicitly applies the file mode during
|
|
||||||
atomic publication.
|
|
||||||
|
|
||||||
This prevents an administrator from using ordinary Unix provisioning to share
|
|
||||||
one cache between a service account and trusted operator accounts. Adding a
|
|
||||||
second Notarius-specific `private` or `group` configuration policy would solve
|
|
||||||
that immediate symptom, but it would preserve the underlying ownership error:
|
|
||||||
Notarius would still decide access policy instead of respecting the deployment.
|
|
||||||
|
|
||||||
Routine Notarius operation does not require an application-enforced private
|
|
||||||
cache. Cache records can contain source-derived data, so operators must be told
|
|
||||||
what they are storing, but sensitivity does not imply that Notarius should
|
|
||||||
override a deliberately configured filesystem policy. API credentials remain
|
|
||||||
outside cache payloads and continue to follow their existing configuration and
|
|
||||||
redaction boundaries.
|
|
||||||
|
|
||||||
## Target Policy
|
|
||||||
|
|
||||||
The target contract is:
|
|
||||||
|
|
||||||
> Notarius creates cache entries with conventional maximum creation modes and
|
|
||||||
> allows the operating system to determine effective access through the
|
|
||||||
> process umask, inherited group ownership, and default ACLs.
|
|
||||||
|
|
||||||
Concretely:
|
|
||||||
|
|
||||||
- newly created cache directories request `0777`;
|
|
||||||
- newly created cache files, including atomic temporary files, request `0666`;
|
|
||||||
- the kernel applies the invoking process's umask and any applicable default
|
|
||||||
ACL;
|
|
||||||
- setgid directories provisioned by the operator control inherited group
|
|
||||||
ownership and, on supported Unix systems, setgid propagation to descendants;
|
|
||||||
- Notarius does not call `chmod`, `chown`, or `chgrp` to impose or repair a
|
|
||||||
cache access policy; and
|
|
||||||
- Notarius does not reject a cache merely because its effective modes are more
|
|
||||||
or less permissive than a Notarius-selected default.
|
|
||||||
|
|
||||||
This is policy inheritance, not literal copying of a parent directory's mode
|
|
||||||
bits. Directories and non-executable data files appropriately have different
|
|
||||||
maximum modes. A parent mode alone also does not determine child modes: an
|
|
||||||
operator who needs a particular result must configure the service umask and,
|
|
||||||
where appropriate, a setgid parent or default ACL.
|
|
||||||
|
|
||||||
For example, a service started with umask `0077` ordinarily creates `0700`
|
|
||||||
directories and `0600` files. A trusted-group deployment can provision a
|
|
||||||
setgid cache root and use umask `0007`, or an equivalent default ACL, to obtain
|
|
||||||
`0770` directories and `0660` files. These are deployment choices rather than
|
|
||||||
configuration values interpreted by Notarius.
|
|
||||||
|
|
||||||
## Public Contract
|
|
||||||
|
|
||||||
No new Notarius configuration field, environment variable, or CLI flag is
|
|
||||||
introduced. In particular, do not add the previously considered
|
|
||||||
`cache.access: private|group` setting.
|
|
||||||
|
|
||||||
Existing cache enablement, locations, chunk-plan modes, resume controls,
|
|
||||||
identity, and compatibility rules remain unchanged. The behavioral change is
|
|
||||||
that effective permissions on newly created cache entries are controlled by
|
|
||||||
the deployment environment rather than forcibly converged by Notarius.
|
|
||||||
|
|
||||||
The policy applies uniformly to both cache families:
|
|
||||||
|
|
||||||
- canonical chunk plans; and
|
|
||||||
- invocation checkpoints.
|
|
||||||
|
|
||||||
Output and debug storage are outside this feature's scope. Their current
|
|
||||||
contracts must not change incidentally through a shared helper refactor. A
|
|
||||||
separate review may later consider whether the same deployment-owned policy is
|
|
||||||
appropriate for those surfaces.
|
|
||||||
|
|
||||||
## Filesystem And Atomic-Publication Requirements
|
|
||||||
|
|
||||||
Permission deference must cover every cache write path, not only final files:
|
|
||||||
|
|
||||||
- configured cache-family roots created by Notarius;
|
|
||||||
- chunk-plan source-digest directories;
|
|
||||||
- checkpoint identity and stage directories;
|
|
||||||
- atomic temporary files; and
|
|
||||||
- final files published by rename.
|
|
||||||
|
|
||||||
Atomic publication must not bypass the deployment policy. Go's
|
|
||||||
`os.CreateTemp` creates a file with `0600`, and applying `chmod(0666)` afterward
|
|
||||||
would bypass the process umask. Cache writers therefore need an atomic
|
|
||||||
temporary-file creation path that chooses a collision-resistant name and opens
|
|
||||||
it with `O_CREATE|O_EXCL` and requested mode `0666`, allowing the kernel to
|
|
||||||
apply umask and default ACL policy at creation time.
|
|
||||||
|
|
||||||
Replacement writes publish a newly created temporary inode. Its effective
|
|
||||||
permissions and ACL are therefore those selected for new entries by the
|
|
||||||
current parent and process policy; Notarius does not copy mode bits or ACLs
|
|
||||||
from the replaced file. Operators who require stable replacement permissions
|
|
||||||
must express that policy through a stable service umask and directory default
|
|
||||||
ACL.
|
|
||||||
|
|
||||||
Opening an existing cache for writing must not alter its root or descendant
|
|
||||||
modes. Read-only lookup of a missing cache must remain side-effect free.
|
|
||||||
|
|
||||||
## Preserved Safety And Correctness Boundaries
|
|
||||||
|
|
||||||
Deferring access policy does not weaken Notarius's filesystem correctness
|
|
||||||
rules. The implementation must preserve:
|
|
||||||
|
|
||||||
- confined resolution beneath the selected cache root;
|
|
||||||
- rejection of symlinked or unexpected cache entries;
|
|
||||||
- validation of source digests, checkpoint identities, schemas, and recorded
|
|
||||||
provenance;
|
|
||||||
- atomic replacement, existing synchronization behavior, cleanup of abandoned
|
|
||||||
temporary files, and complete-record visibility;
|
|
||||||
- existing missing, malformed, incompatible, and operational-error recovery
|
|
||||||
classifications; and
|
|
||||||
- separation of output, cache, and debug roots.
|
|
||||||
|
|
||||||
An administrator's decision to permit multiple accounts to reach one cache
|
|
||||||
does not create a new Notarius guarantee for simultaneous writers. Preserve
|
|
||||||
the existing atomicity and validation behavior, but do not add locking or claim
|
|
||||||
transactional coordination across concurrent processes as part of this work.
|
|
||||||
Concurrent writes to the same multi-file checkpoint identity remain subject to
|
|
||||||
the currently documented cache behavior; the implementation documentation
|
|
||||||
must state any existing limitation accurately.
|
|
||||||
|
|
||||||
## Architectural Boundaries
|
|
||||||
|
|
||||||
The target removes application-owned cache modes without creating a general
|
|
||||||
permission-policy framework. Chunk-plan and checkpoint storage may use
|
|
||||||
different filesystem APIs internally, but both must implement the same
|
|
||||||
deployment-owned result. Pipeline modules and the runner remain unaware of
|
|
||||||
physical roots and filesystem access policy.
|
|
||||||
|
|
||||||
Any shared writer API must distinguish an application-enforced exact mode from
|
|
||||||
an umask-filtered maximum creation mode. The cache change must not silently
|
|
||||||
alter output or debug behavior merely because checkpoints currently share
|
|
||||||
filesystem helpers with those surfaces.
|
|
||||||
|
|
||||||
## Existing Cache Migration
|
|
||||||
|
|
||||||
Stopping permission convergence does not change existing inode modes. A cache
|
|
||||||
tree created by an older Notarius version may therefore remain `0700`/`0600`
|
|
||||||
and inaccessible to another account after upgrade.
|
|
||||||
|
|
||||||
Operators who want a different policy must either:
|
|
||||||
|
|
||||||
- remove the reconstructible cache and allow it to be recreated beneath a
|
|
||||||
correctly provisioned parent and service environment; or
|
|
||||||
- stop writers and deliberately update ownership, modes, and ACLs using normal
|
|
||||||
system-administration tools.
|
|
||||||
|
|
||||||
Notarius must not perform a recursive startup migration. The same rule applies
|
|
||||||
when an operator wants to make a previously shared cache private: change the
|
|
||||||
deployment policy and deliberately migrate or recreate existing state when an
|
|
||||||
immediate whole-tree change is required.
|
|
||||||
|
|
||||||
## Documentation And Decision Records
|
|
||||||
|
|
||||||
The final documentation set includes an accepted ADR titled along the lines of
|
|
||||||
"Defer cache access policy to the deployment." The ADR records the durable
|
|
||||||
ownership boundary so later maintenance does not reintroduce
|
|
||||||
application-enforced modes solely because cache data can contain source-derived
|
|
||||||
content.
|
|
||||||
|
|
||||||
The new ADR need not supersede ADR-0005 or ADR-0007: those decisions govern
|
|
||||||
canonical chunk-plan reuse and checkpoint recording/reuse, while leaving exact
|
|
||||||
storage permissions to operational contracts. It should reference those ADRs
|
|
||||||
and the output/cache/debug separation established by ADR-0006 and refined by
|
|
||||||
ADR-0007.
|
|
||||||
|
|
||||||
In the final current-behavior documentation, `docs/operations.md` owns cache
|
|
||||||
provisioning, umask, setgid, default ACL, migration, sensitivity, and
|
|
||||||
concurrent-use guidance. Architecture owns the deployment/application boundary;
|
|
||||||
the configuration reference clarifies that selecting a cache path does not
|
|
||||||
select its permissions; and the internal state guide explains the implemented
|
|
||||||
write mechanics. The release that first contains the feature summarizes the
|
|
||||||
operational change without becoming its canonical reference.
|
|
||||||
|
|
||||||
Current-behavior documentation must not describe this target behavior before
|
|
||||||
the implementation lands.
|
|
||||||
|
|
||||||
## Verification Expectations
|
|
||||||
|
|
||||||
Behavioral tests protect kernel-selected permissions for both cache families,
|
|
||||||
non-mutation of existing roots, atomic replacement, and representative
|
|
||||||
assembled CLI wiring. Existing confinement, symlink, validation, identity,
|
|
||||||
reuse, and recovery coverage remains authoritative. Umask tests are POSIX-only,
|
|
||||||
serialized, and restore process-global state. The default suite does not create
|
|
||||||
privileged accounts or groups; setgid and default-ACL examples may be verified
|
|
||||||
in a purpose-built Linux deployment environment.
|
|
||||||
|
|
||||||
## Acceptance Criteria
|
|
||||||
|
|
||||||
- No cache access-mode field or application-specific private/group policy is
|
|
||||||
added.
|
|
||||||
- Both chunk-plan and checkpoint writers request `0777` for new directories
|
|
||||||
and `0666` for new files, including atomic temporary files.
|
|
||||||
- Effective permissions reflect the process umask and applicable default ACLs
|
|
||||||
rather than a later Notarius chmod.
|
|
||||||
- Existing cache roots and descendants are not chmod-converged during reads or
|
|
||||||
writes.
|
|
||||||
- A private deployment can obtain private cache state through its filesystem
|
|
||||||
provisioning and umask without a Notarius-specific setting.
|
|
||||||
- A trusted-group deployment can obtain shared cache state through setgid,
|
|
||||||
umask, and/or default ACL policy without a Notarius-specific setting.
|
|
||||||
- Existing cache trees are not recursively migrated.
|
|
||||||
- Chunk-plan and checkpoint enablement, lookup, recording, resume, identity,
|
|
||||||
validation, confinement, atomicity, and recovery semantics remain unchanged.
|
|
||||||
- Output and debug permission behavior does not change as an incidental effect.
|
|
||||||
- Notarius performs no user/group management, ownership mutation, ACL
|
|
||||||
management, privilege escalation, or cross-process locking.
|
|
||||||
- The ADR and canonical operations, architecture, configuration, and internal
|
|
||||||
documentation accurately describe the implemented boundary; the next
|
|
||||||
release note summarizes the change when that release is prepared.
|
|
||||||
- Focused tests and repository-wide `go test ./...`, `go vet ./...`, and
|
|
||||||
`go build ./cmd/notarius` checks pass.
|
|
||||||
@@ -150,16 +150,6 @@ address Go dependency cycles, registrar ownership, stable public module keys,
|
|||||||
and supersession of the affected ADR-0004 decision. Conceptual artifact-family
|
and supersession of the affected ADR-0004 decision. Conceptual artifact-family
|
||||||
ownership does not by itself require this move.
|
ownership does not by itself require this move.
|
||||||
|
|
||||||
## Near-Term Operations
|
|
||||||
|
|
||||||
### Deployment-Managed Cache Permissions
|
|
||||||
|
|
||||||
- Implement the proposed
|
|
||||||
[deployment-managed cache permissions](deployment-managed-cache-permissions.md)
|
|
||||||
contract so Notarius respects the access policy established by the cache
|
|
||||||
root, process umask, setgid inheritance, and default ACLs instead of imposing
|
|
||||||
application-owned private or group modes.
|
|
||||||
|
|
||||||
## Blue-Sky Platform And Operations
|
## Blue-Sky Platform And Operations
|
||||||
|
|
||||||
These ideas are intentionally less specified. Promote one into an earlier
|
These ideas are intentionally less specified. Promote one into an earlier
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user