Implement main-relative companion inputs

The feature roadmap owns the target behavior and the four settled decisions. The Audita request supplies the consumer workflow. Implement in order as one sprint. Intermediate stages may leave code or tests incomplete; run focused checks when their dependencies are in place, and require a coherent, validated tree at the end. Stage boundaries are handoffs for Luna, not required commit boundaries.

Start with the development guide and its applicable agent, architecture, testing, and documentation policies. Keep the public facade root-owned. Keep ordinary loading, file source semantics, and existing explicit layered overlays compatible. No dependency change, new YAML syntax, second discovery pass, application callback for context selection, or new release gate is part of this work.

Stage 1 — Define the Go API and preflight rules

Status: Complete. Main-relative descriptions and layered preflight are in place.

In root and engine layered.go, add matching descriptions and carry them into loadInput and loadRequest in the engine. Use these public names:

type MainRelativeContext struct {
    Fallback    *FilesystemContext
    RequireBase bool
}

func MainRelative() *MainRelativeContext

MainRelative() returns a fresh pointer to a zero-valued description. Add Input.MainRelative *MainRelativeContext and LayeredOptions[T].OverlayFallback *FilesystemContext. The latter only needs a concrete context: a selected overlay already requires a base. Use keyed literals in new examples. Copy the description and its concrete fallback into the private request using the existing filesystem-context conversion in lifecycle.go; preserve the caller's native namespace and reader ownership rules. Store resolved state only in the current load.

In layered preflight, apply the concrete-context override before checking binding scope. Reject an active main binding and active bindings on File, Filesystem.File, and SearchConfig underlays with CodeOptionsInvalid. Keep ordinary source validation, including zero-source rejection. A valid NoSource underlay remains absent; binding does not make it contribute content or evidence, and RequireBase still applies to an active binding. Suppress the synthetic-source missing-context preflight error only for a valid active underlay binding. Binding does not enable imports.

Check the static shape of a fallback that can participate, using the existing non-file context rules (both base and root, host absolute paths or native namespace paths). Defer existence, directory, access, and resolved containment checks until selection. Ignore the shadowed binding entirely when Input.Filesystem is supplied; likewise ignore the overlay fallback when the overlay is inactive or an explicit OverlayFilesystem wins. Keep overlay name and explicit-context preflight checks. Move the missing-context error for active layered overlays to Stage 5, where selection can see the main. Do not change preflight for ordinary Load.

Cover active binding scope, explicit overrides (including invalid explicit contexts), and fallback shape through public option tests. Reuse existing option diagnostic codes; avoid promising order among unrelated errors.

Stage 2 — Bind companion inputs with a usable context

Status: Complete. Focused host, native, symlink, fallback, and grouped-load checks passed.

In assembleLayeredInput, pass the main's assembledInput.resolvedFilesystem to acquireUnderlayGroupScopes. Reuse the acquisition handoff in acquisition.go and resolution helpers in filesystem_context.go; do not select or open the main again. Verify the handoff covers a file or selected search without imports, a native file, explicit non-file contexts, and NoSource or exhausted optional search with explicit context. Preserve the absent host root-only rule and the native context requirements. A main acquisition/context error is terminal, never an absent-base signal.

For each underlay, use its explicit filesystem first; otherwise, for active bindings, use the resolved main context, then its fallback. For main-derived contexts copy the resolved base and root and keep the native namespace object intact. Clear mainPath in the copy: the companion is still a synthetic source, not a second copy of the main. Use the receiving tier's existing host identity token and visited state; the main's host location and scope carry over, not its tier bookkeeping. A selected fallback uses the same resolution/validation rules as a concrete non-file context; pass it through the normal acquisition path where possible. Validate selected contexts even when imports are inactive as required by the existing context contract; do not probe the fallback when main derivation wins.

Carry a pre-resolved derived context through the private acquisition request to acquireImportedInput and the standalone/context completion path. Use existing import access, traversal and closure; keep root/base checks, parse and version processing, budget accounting, and source evidence. Bound synthetic sources can expose the same passive context as explicitly located synthetic sources but keep their original kind, label and empty physical path. The imported files use their own paths and directory bases.

Make the existing native namespace anchor selection in acquireUnderlayGroupScopes use the selected namespace, including derived or fallback contexts. Determine anchors from selected descriptors without opening directories or validating later inputs out of turn. Keep the first input position anchor for each namespace and the existing host and synthetic anchoring and tier boundaries. Preserve independent sources and repetition semantics, including two different native namespaces with the same label and paths.

Cover successful bindings for host and native contexts, an unrelated working directory, canonical host symlink target directory, a wider root that permits contained parent-relative imports, and reuse of one description across loads. Test selected and unused fallbacks. Defer the no-base path to Stages 3–4.

In import_acquisition.go, extract the context-independent first pass of walkDeclarations so both normal traversal and the contextless path can use it. Prepare from the already parsed declaration specs, apply resolveImportPolicy with the load-wide defaults, and retain existing placement checks and diagnostic sites. For an input placed as a sequence item, prepare against its local mapping scope and keep the existing diagnostic path projection. Use the already processed root directives from processAcquiredInputVersion; do not reparse YAML, repeat version checks, or infer a filesystem path from a source label.

Keep selection and admission in the existing normal walk. This stage changes no bound import behavior. Reuse import specification, defaults, placement, selection, and diagnostic tests for verification.

Stage 4 — Finish companion handling when no base exists

Handle the no-base outcome only for an active underlay binding with no override, usable main context, or fallback. RequireBase fails with CodeOptionsInvalid regardless of import enablement or declaration count. Otherwise retain normal source parsing, version handling, disabled-import validation, and contribution assembly. When imports are enabled, use Stage 3 to validate their declarations even if none will be expanded.

For required declarations use CodeImportMissing at the first named target site for path, or the selector field site for dir and glob. Say that no base is available; do not report a file search that did not happen. A required named-path declaration with nonempty targets fails, as do required directory or glob selections. Empty path: [] has no targets even if required. Explicit per-declaration requiredness continues to override the application default. Skip optional expansion without calling a filesystem selector.

Assemble the declaring payload once, with its original source record, placement, sequence declarations, and version evidence. Share the existing root contribution assembly and accounting where needed for imports-enabled payloads, including the pending local scope for sequence-item placement. Do not turn imports off to bypass validation or introduce a dummy access object, dummy filesystem, acquired child, resource context, or EmptyImportSelections entry. Charge the declaring payload as usual; charge no traversal or child work for skipped imports. Normal searches with a selected context still produce their normal empty-selection evidence.

Test absent optional discovery and explicit absence, mixed payload with optional and required declarations, application defaults and explicit overrides, empty paths under both requiredness values, require-base with and without enabled imports, malformed declarations and invalid placement, source version policy, mounted payloads, and absence of synthetic search evidence. Required discovery and failed selected main processing must not activate this path.

Stage 5 — Derive layered overlay context and lookup base

In the existing overlay branch of assembleLayeredInput, select in order: explicit OverlayFilesystem, usable main context, OverlayFallback. Skip selection entirely for an inactive name. For an active name without a base, return CodeOptionsInvalid; if context validation fails, propagate its normal diagnostics without trying another context. Keep overlay selection after source assembly and main version handling, and preserve existing failures for missing files, invalid overlays, and repeated sources.

In selected_overlay.go, pass a lookup directory separately to the existing selected overlay loader (for example on selectedOverlayRequest). Ordinary Load and explicit layered contexts use the resolved root as before. Derived and fallback contexts use the resolved base. Open access at the containment root; join the lookup directory with overlays/<name>.yml using the selected namespace's path operations, then use normal canonicalization and containment. Keep the namespace, source identities, version processing, limits, source contexts, and handle ownership on the existing path. Do not use the lookup directory as a replacement root or construct a new namespace.

Cover canonical host symlink targets, wider roots with distinct overlay files at base and root, native filesystems, context on a non-file or absent main, inactive overlay with an invalid fallback, explicit override with no fallback on error, base-relative fallback, and missing selected overlays without recovery. Retain ordinary and explicit root-relative regression coverage, including existing namespace and repeated-source cases.

Stage 6 — Exercise the combined workflow

Add a maintained executable example in examples/consumers for a single layered load using SearchConfig, a mapping companion with optional named files and recursive directory imports, an independently located embedded tier, and one selected overlay. Use actual fixture files and the existing *.yml and *.yaml pattern list; show destination placement, transitive imports, and strict peers. Demonstrate precedence from embedded catalog through operator underlay, main, and overlay. Include a registered identity case for whole-record upsert across tiers and a same-tier duplicate failure. No change to Audita or PromptKit is needed.

Stage 7 — Close remaining integration and evidence gaps

Fill remaining public integration gaps without repeating every engine test: first and later search candidates, explicit file and mapping/bytes/reader sources, only the file, only the directory, both, neither and an empty directory, invalid or inaccessible selected fragments (use a deterministic filesystem error where permissions are unreliable), and a selected main failure that cannot resume search or use the fallback. Use focused fixtures to check layer/index, physical paths, passive resource contexts, normal source-version evidence, shared acquisition budgets and cleanup on failure. Check that callbacks execute only in the existing lifecycle, and that a reused description binds anew. Preserve existing ownership and detached inspection checks. Do not require new global diagnostic ordering or test incidental call counts.

Stage 8 — Document and verify the completed sprint

Update the public declaration comments and the discovery, layered loading, imports, resource context and selected overlay sections of the configuration contract. Document lookup versus containment, canonical host symlink behavior, override and fallback precedence, and absent-base behavior. Link the maintained examples from consumer guidance. Update the internal overview and import internals for the actual handoff, contextless path, and ownership.

Record the deferred context decision in the next numbered ADR, extending ADR 0019 without rewriting it. Clarify in the architecture policy that the layered loader binds application-selected dependent locations after main selection; applications still own search conventions, and an explicit overlay selection is required. Preserve existing scope boundaries. Use the documentation policy's canonical owners rather than restating the full contract in every document.

Run the focused engine, facade, and consumer tests added or affected above, then the everyday ./scripts/check.sh. Run the race suite for the load-local ownership and description reuse changes. Check changed Markdown links, public facade boundaries and examples. Use the declared Go toolchain and existing dependencies. No release or benchmark is part of completion. Report validation and any remaining limitations; commit the completed sprint under the agent policy.

14 KiB Raw Blame History Unescape Escape