promptkit/docs/roadmap/implementation.md

Application Fallback Profiles Implementation Plan

Status: Ready for implementation.

Purpose

This document is the decision-complete implementation plan for application fallback profiles. It is written for a gpt-5.6-terra coding agent that will implement each stage in order.

The feature roadmap owns the motivation, policy choices, compatibility boundary, non-goals, and target end state. This document owns the concrete design, file-level work, test ownership, documentation updates, validation, and completion gates.

Implementation Rules

• Complete the stages in order. Every stage must leave the repository buildable, tested for the behavior changed in that stage, and accurately documented for its implemented state.
• Preserve unrelated working-tree changes. Inspect git status --short and the relevant diffs before editing, and do not overwrite or reformat pre-existing user work.
• Follow every policy under docs/policy/, the task-specific reading guide in docs/development.md, and the accepted behavior in fallback-profiles.md.
• Keep the module root as the public facade. The root package owns profile source selection and composition; internal/profile owns repository, parsing, validation, and error-preserving overlay behavior; and internal/profile/builtin owns only the embedded built-in catalog.
• Add only the public WithFallbackProfileFS option. Do not add an exported repository type, source-provenance value, fallback-specific error, config field, programmatic fallback-profile option, or public package.
• Reuse profile.NewFSRepository and profile.NewOverlayRepository. Do not add another parser, validator, repository implementation, or merge model.
• Preserve lazy loading. Engine construction validates the option arguments, not every file in the supplied filesystem. A profile source is read only when resolution reaches it.
• Preserve existing error identities and mappings. Only profile.ErrProfileNotFound permits an overlay to consult its next layer; every other error from a higher layer must be returned and mapped through the existing public profile-load path.
• Keep tests classical and behavior-focused. Public source precedence and exported option semantics belong in external-package root tests; generic overlay behavior remains owned by internal/profile tests.
• Update GoDoc and current-state documentation in the same stage that exposes the public option. Do not describe the feature as implemented before that stage is complete.
• Do not add release notes, change a module version, commit, tag, push, or publish a release as part of this plan.

Fixed Design

Public API

Add this function to engine.go beside the existing profile-source options:

func WithFallbackProfileFS(fsys fs.FS, root string) Option

The option accepts the same filesystem and root forms as WithProfileFS and constructs its repository with profile.NewFSRepository(fsys, root). It must return ErrInvalidConfig from option application when fsys is nil or when strings.TrimSpace(root) is empty. Do not normalize or replace a valid root before passing it to the repository.

The fallback source is its own last-value-wins option category. Add these two private fields to engineOptions:

fallbackProfiles     profile.Repository
fallbackProfileSource bool

Use the repository field for the selected source and the boolean only to distinguish an unapplied option from an applied option. A later valid WithFallbackProfileFS replaces both values. Option application remains sequential, so an invalid option fails NewEngine immediately even if a later option could otherwise replace it.

The exact GoDoc for WithFallbackProfileFS must state:

• that it supplies application-owned fallback profile definitions;
• the full four-layer lookup order;
• that definitions are whole profiles and are not field-merged;
• that only a missing ID falls through, while a matching read, parse, duplicate, validation, or credential-format failure stops resolution;
• that loading and validation are lazy;
• that files use the ordinary strict profile YAML and api_key_env rules;
• that nil filesystems and blank roots cause NewEngine to match ErrInvalidConfig;
• that repeated calls use the last valid fallback source; and
• that this is definition lookup, not provider or generation failover.

Update the Option, Config.ProfileDir, WithProfileFS, WithProfileFile, and WithProfiles GoDoc in engine.go where necessary so their relative precedence is unambiguous. Exact declarations and behavior remain owned by GoDoc; consumer documentation should summarize the workflow and link readers back to the API rather than reproduce every error clause.

Repository Composition

Move all profile-source composition into one private root helper in engine.go:

func newProfileRepository(profileDir string, options engineOptions) profile.Repository

NewEngine must call this helper once and pass its returned repository to the runner. The helper must build from lowest to highest precedence:

1. begin with builtin.NewRepository();
2. if options.fallbackProfileSource is true, overlay options.fallbackProfiles over the built-in repository;
3. select exactly one ordinary configured source: use options.profiles when options.profileSource is true; otherwise, when profileDir is nonblank, use profile.NewFilesystemRepository(profileDir); overlay that selected source over the current repository;
4. if options.memorySource is true, overlay options.memoryProfiles over the current repository; and
5. return the resulting chain.

This preserves the existing rule that WithProfileFS or WithProfileFile replaces Config.ProfileDir; those sources are alternatives in one ordinary configured-source category, not two independent layers. WithProfiles remains a distinct highest-precedence category.

The final lookup order is therefore:

WithProfiles
    -> WithProfileFile / WithProfileFS / Config.ProfileDir
        -> WithFallbackProfileFS
            -> Promptkit built-ins

Every arrow is a whole-profile, not-found-only fallback. Do not inspect or copy profile fields in the composition helper.

Built-In Package Boundary

Reduce internal/profile/builtin/repository.go to the embedded catalog leaf:

func NewRepository() profile.Repository

Remove NewRepositoryWithPrimary and NewRepositoryWithDirectory. Remove their now-obsolete tests from internal/profile/builtin/repository_test.go and remove imports used only by those helpers or tests. Do not move their tests to another private helper: existing root public-behavior tests own assembled precedence, and internal/profile.TestOverlayRepository owns not-found-only overlay semantics.

Do not change built-in YAML assets, built-in validation, the profile.Repository interface, profile.NewOverlayRepository, or filesystem repository behavior.

Resolution And Error Semantics

The runner receives one assembled profile.Repository; do not add fallback logic to InspectProfile, Prepare, PrepareExecution, Run, or RunPrepared. Those paths must continue to resolve through the runner's one repository dependency.

The existing overlay contract is authoritative:

• a successful lookup returns the complete higher-layer profile;
• profile.ErrProfileNotFound consults the next layer;
• cancellation, filesystem read failures, malformed YAML, duplicate matches, raw api_key, invalid profiles, and all other errors stop lookup; and
• the root facade maps failures through the existing public identities such as ErrProfileNotFound and ErrProfileLoad.

Do not add eager filesystem walking in the option or NewEngine. A malformed asset unrelated to the requested ID retains the existing ordinary FSRepository behavior; this plan does not strengthen that package's global validation guarantees.

Test Ownership

Use the following test boundaries and avoid duplicating the profile parser's existing case matrix.

In public_contract_test.go:

• Add TestFallbackProfileSourcePrecedence. Use minimal synthetic fstest.MapFS profiles and, where Config.ProfileDir is under test, a t.TempDir. Cover these distinct relationships: an in-memory profile beats both ordinary and fallback definitions; an ordinary WithProfileFS source beats a fallback definition; Config.ProfileDir beats a fallback definition when no ordinary source option replaces it; a fallback definition beats a built-in definition with the same ID; and an ID absent from the fallback source still resolves from the built-in catalog. Assert the selected model or another stable complete-profile field rather than internal repository structure.
• Extend TestRepeatedOptionsUseLastValueInEachCategory with a fallback profile source subtest proving that the later valid fallback filesystem is selected.
• Add TestFallbackProfileSourcePreservesLazyLoadingAndErrors. Prove that engine construction succeeds without reading malformed fallback YAML, that an unrelated malformed file does not prevent a valid requested fallback definition from resolving under existing FS-repository semantics, that a malformed fallback file whose stem matches a built-in profile ID yields ErrProfileLoad instead of silently reaching the built-in, and that a malformed ordinary configured definition yields ErrProfileLoad instead of reaching a valid application fallback definition. Use errors.Is; do not assert complete error strings.
• Add one representative workflow test that supplies a fallback-only profile and verifies the same effective model through InspectProfile, Prepare, a PrepareExecution followed by RunPrepared, and direct Run. Use the existing deterministic injected-client style, no live provider, and no real credential. This test owns the cross-workflow repository wiring; do not repeat the full precedence matrix through every method.

In engine_test.go, extend TestSourceOptionsRejectInvalidInputs with nil filesystem and blank-root cases for WithFallbackProfileFS. Both must make NewEngine match ErrInvalidConfig.

Retain internal/profile.TestOverlayRepository unchanged unless a genuine existing defect is found. It already owns success, not-found fallback, and non-not-found error preservation. Do not add package-private tests for the new root helper, snapshots, golden files, provider calls, or one test per profile format error already covered by internal/profile.

Canonical Documentation

Update current-state documentation when the option is implemented:

• In docs/formats.md, make the source-precedence section the canonical four-layer definition lookup order. State that ordinary configured sources override application fallbacks, application fallbacks override built-ins, profiles are whole values, and only a missing ID falls through. Retain this document's ownership of strict YAML, credentials, validation, and source discovery details.
• In docs/consumers/pkg-promptkit.md, add a short task-oriented section that shows an illustrative embed.FS declaration and WithFallbackProfileFS. Explain that application defaults belong in the embedded fallback and operator overrides belong in the ordinary configured source. Link to docs/formats.md for exact format and precedence rules, and do not turn the snippet into a second complete maintained application.
• In docs/internal/sources.md, describe the root-owned four-layer composition and the existing not-found-only overlay mechanism. Remove any claim that the built-in package composes caller-selected repositories.
• In docs/internal/overview.md, keep the root facade responsible for source assembly, describe internal/profile/builtin only as the embedded catalog, and reflect the implemented fallback layer without duplicating the exact public API contract.
• In engine.go, apply the GoDoc changes under Public API. GoDoc owns the exact option signature, validation, category, and public semantics.

The architecture policy already assigns assembly to the root facade and the built-in catalog to internal/profile/builtin; do not edit it unless the implementation reveals an actual contradiction. No integration protocol, outbound request body, profile YAML shape, README orientation, or maintained example changes as part of this feature.

Stage 1: Move Existing Profile Composition To The Root Facade

Objective

Establish the intended ownership boundary and a single root composition point without changing public behavior or adding the fallback option.

Implementation Prompt

Implement only Stage 1 of docs/roadmap/implementation.md. Read the complete feature roadmap, implementation rules, and fixed design above before editing.

1. In engine.go, add newProfileRepository(profileDir string, options engineOptions) profile.Repository and move the existing three-layer assembly into it: built-ins, then the selected ordinary configured source, then WithProfiles. Do not add fallback fields or the public option yet.
2. Replace the inline profile assembly in NewEngine with one call to the helper. Preserve the existing replacement relationship between Config.ProfileDir and WithProfileFS/WithProfileFile.
3. In internal/profile/builtin/repository.go, remove NewRepositoryWithPrimary and NewRepositoryWithDirectory, leaving NewRepository as the only constructor.
4. Remove the three tests dedicated to the deleted built-in composition helpers from internal/profile/builtin/repository_test.go. Preserve tests that validate the embedded catalog itself.
5. Update docs/internal/sources.md and docs/internal/overview.md so they describe the implemented Stage 1 ownership accurately. At this boundary the source order is still in-memory, ordinary configured source, built-ins; do not document the application fallback as implemented yet.
6. Run the focused validation below. Fix in-scope regressions without adding fallback behavior early.

Do not add or mention an implemented WithFallbackProfileFS in Stage 1. Do not change exported declarations, profile parsing, profile assets, error mapping, runner behavior, or consumer and format documentation.

Focused Validation

Run from the repository root:

gofmt -w engine.go internal/profile/builtin/repository.go \
  internal/profile/builtin/repository_test.go
go test . -run \
  'Test(PrepareUsesBuiltInProfileWithoutProfileDir|CustomProfileOverridesBuiltInProfile|InMemoryProfilesOverrideBuiltInsAndProfileSources)$'
go test ./internal/profile/...
go test . ./internal/profile/...
go vet . ./internal/profile/...
git diff --check

If a focused expression does not match an existing test name, inspect the current suite and run the narrowest equivalent public precedence coverage; do not silently skip the intended relationship.

Completion Gate

Stage 1 is complete only when:

• the root facade assembles the unchanged three-layer profile chain in one private helper;
• ordinary source options still replace Config.ProfileDir and in-memory profiles still have highest precedence;
• built-in profiles remain available and remain lower than consumer sources;
• only internal/profile owns generic overlay behavior and the built-in package owns only its embedded catalog;
• no public API or behavior changed;
• internal current-state documentation matches that boundary; and
• all focused tests, vet, formatting, and whitespace checks pass.

Stage 2: Add Application Fallback Profiles And Public Contracts

Objective

Add the public option, insert the application fallback into the root-owned repository chain, prove its precedence and failure behavior through public workflows, and publish the canonical current-state documentation.

Implementation Prompt

Implement only Stage 2 of docs/roadmap/implementation.md after Stage 1 satisfies its completion gate.

1. Add fallbackProfiles and fallbackProfileSource to engineOptions, then implement WithFallbackProfileFS exactly as specified under Public API.
2. Extend newProfileRepository so it constructs the fixed four-layer chain in the prescribed low-to-high order. Do not alter generic overlay logic or add fallback branches to runner methods.
3. Update all affected engine.go GoDoc, including the option category list and the relative precedence descriptions for existing profile sources.
4. Add and extend the external-package root tests exactly as specified under Test Ownership. Reuse small existing fakes and fixture helpers where they remain clear; add only minimal synthetic YAML helpers needed by these tests.
5. Extend TestSourceOptionsRejectInvalidInputs with the two fallback option validation cases.
6. Update docs/formats.md, docs/consumers/pkg-promptkit.md, docs/internal/sources.md, and docs/internal/overview.md according to Canonical Documentation.
7. Run the focused validation below. Repair in-scope failures without weakening existing parser, error-identity, prepared-execution, or profile precedence guarantees.

Do not add a config field, in-memory fallback API, source provenance, profile inheritance, provider failover, eager validation, application-specific assets, or backend/model policy. Do not edit internal/profile/builtin/assets/.

Focused Validation

Run from the repository root:

gofmt -w engine.go engine_test.go public_contract_test.go
go test . -run \
  'Test(FallbackProfileSource|RepeatedOptionsUseLastValueInEachCategory|SourceOptionsRejectInvalidInputs)'
go test ./internal/profile/...
go test . ./internal/profile/... ./internal/usecase
go vet . ./internal/profile/... ./internal/usecase
git diff --check

The focused root expression must execute the precedence, lazy/error, cross-workflow, repeated-option, and invalid-input coverage described above. If the implemented names differ slightly, run explicit equivalent expressions and record no skipped contract category.

Completion Gate

Stage 2 is complete only when:

• WithFallbackProfileFS is the sole new public declaration and has complete, accurate GoDoc;
• nil filesystem and blank root inputs fail construction with ErrInvalidConfig, and the last valid repeated fallback option wins;
• the assembled order is in-memory, ordinary configured, application fallback, built-ins;
• existing ordinary source options still replace Config.ProfileDir;
• lookup falls through only on a missing ID and never after a matching higher-layer failure;
• profiles remain whole values and loading remains lazy;
• inspection, preparation, prepared execution, and ordinary execution resolve the same fallback definition through one runner repository;
• no built-in asset, profile format, public error identity, provider request, or existing consumer behavior changed unintentionally;
• GoDoc and all affected canonical documents describe implemented behavior without duplicating ownership; and
• all focused tests, vet, formatting, links, and whitespace checks pass.

Stage 3: Audit Compatibility And Validate The Repository

Objective

Confirm that the implementation is complete, minimal, and consistent across the public facade, internal boundaries, tests, and documentation, then mark the temporary planning documents complete.

Implementation Prompt

Implement only Stage 3 of docs/roadmap/implementation.md after Stage 2 satisfies its completion gate.

1. Search tracked Go and Markdown files for NewRepositoryWithPrimary, NewRepositoryWithDirectory, profile source precedence lists, and descriptions of built-in repository composition. Remove stale references and correct only feature-owned contradictions.
2. Review newProfileRepository directly and confirm it has exactly four possible layers in the required order, selects only one ordinary configured source, and contains no profile field merging or eager I/O.
3. Review the public tests as a suite. Confirm that each distinct risk in Test Ownership is protected once, generic parser and overlay cases remain with internal/profile, and no test depends on private helper shape.
4. Confirm that internal/profile/builtin/assets/, external wire behavior, backend configuration, credential resolution, public result shapes, and stable JSON tags have no feature-related changes.
5. Follow every added or changed Markdown link and confirm that its target and relevant heading exist. Verify that exact API details live in GoDoc, exact profile format and precedence details live in docs/formats.md, consumer guidance remains task-oriented, and internal documents describe only implementation responsibility.
6. Run the complete validation sequence below and repair only in-scope failures.
7. After every check passes, change the status of fallback-profiles.md and this document to Complete. Do not delete or retire either roadmap; retirement is a separate maintainer action.
8. Re-run git diff --check, inspect git status --short, and review the full diff while distinguishing pre-existing user changes from this feature.

Do not add release notes, change versions, or create a commit, tag, push, or release during this stage.

Full Validation

Run from the repository root:

gofmt -w engine.go engine_test.go public_contract_test.go \
  internal/profile/builtin/repository.go \
  internal/profile/builtin/repository_test.go
gofmt -l $(git ls-files '*.go')
go test ./...
go test -race ./...
go vet ./...
go build ./...
go run ./examples/go-library/prepare
git diff --check
git status --short

The gofmt -l command must print no paths. The maintained example must remain offline and require no real credential or provider.

Inspect the final diff and confirm:

• only this feature's files and pre-existing user changes are present;
• no built-in asset, public value shape, stable JSON tag, provider payload, workspace file, local module replacement, generated binary, or unrelated formatting changed;
• deleted built-in composition helpers have no remaining references;
• the fallback option reuses the ordinary FS repository and generic overlay;
• the root constructs one repository used by every resolution workflow;
• the feature roadmap and this plan are both complete; and
• no commit, tag, push, or release was created.

Completion Gate

The implementation is complete only when:

• every Stage 1 and Stage 2 gate remains satisfied;
• the ordinary and race-enabled suites pass;
• vet, build, formatting, the maintained offline example, Markdown links, and whitespace checks pass;
• the four-layer precedence and not-found-only fallthrough are consistent in code, GoDoc, public tests, format reference, consumer guidance, and internal documentation;
• engines without WithFallbackProfileFS retain their previous behavior;
• the public surface contains no speculative companion API or provenance;
• both temporary roadmap statuses are Complete; and
• the repository is ready for maintainer review without a commit or release having been created by this plan.

Open Questions

None. The feature roadmap and fixed design above fully specify the public API, repository composition, error and validation behavior, compatibility boundary, documentation ownership, test strategy, and staged implementation sequence.