promptkit/docs/roadmap/implementation.md

# Application Fallback Profiles Implementation Plan

**Status:** Ready for implementation.

## Purpose

This document is the decision-complete implementation plan for
[application fallback profiles](fallback-profiles.md). It is written for a
`gpt-5.6-terra` coding agent that will implement each stage in order.

The feature roadmap owns the motivation, policy choices, compatibility
boundary, non-goals, and target end state. This document owns the concrete
design, file-level work, test ownership, documentation updates, validation,
and completion gates.

## Implementation Rules

- Complete the stages in order. Every stage must leave the repository
  buildable, tested for the behavior changed in that stage, and accurately
  documented for its implemented state.
- Preserve unrelated working-tree changes. Inspect `git status --short` and
  the relevant diffs before editing, and do not overwrite or reformat
  pre-existing user work.
- Follow every policy under `docs/policy/`, the task-specific reading guide in
  `docs/development.md`, and the accepted behavior in
  `fallback-profiles.md`.
- Keep the module root as the public facade. The root package owns profile
  source selection and composition; `internal/profile` owns repository,
  parsing, validation, and error-preserving overlay behavior; and
  `internal/profile/builtin` owns only the embedded built-in catalog.
- Add only the public `WithFallbackProfileFS` option. Do not add an exported
  repository type, source-provenance value, fallback-specific error, config
  field, programmatic fallback-profile option, or public package.
- Reuse `profile.NewFSRepository` and `profile.NewOverlayRepository`. Do not
  add another parser, validator, repository implementation, or merge model.
- Preserve lazy loading. Engine construction validates the option arguments,
  not every file in the supplied filesystem. A profile source is read only
  when resolution reaches it.
- Preserve existing error identities and mappings. Only
  `profile.ErrProfileNotFound` permits an overlay to consult its next layer;
  every other error from a higher layer must be returned and mapped through
  the existing public profile-load path.
- Keep tests classical and behavior-focused. Public source precedence and
  exported option semantics belong in external-package root tests; generic
  overlay behavior remains owned by `internal/profile` tests.
- Update GoDoc and current-state documentation in the same stage that exposes
  the public option. Do not describe the feature as implemented before that
  stage is complete.
- Do not add release notes, change a module version, commit, tag, push, or
  publish a release as part of this plan.

## Fixed Design

### Public API

Add this function to `engine.go` beside the existing profile-source options:

```go
func WithFallbackProfileFS(fsys fs.FS, root string) Option
```

The option accepts the same filesystem and root forms as `WithProfileFS` and
constructs its repository with `profile.NewFSRepository(fsys, root)`. It must
return `ErrInvalidConfig` from option application when `fsys` is nil or when
`strings.TrimSpace(root)` is empty. Do not normalize or replace a valid root
before passing it to the repository.

The fallback source is its own last-value-wins option category. Add these two
private fields to `engineOptions`:

```go
fallbackProfiles     profile.Repository
fallbackProfileSource bool
```

Use the repository field for the selected source and the boolean only to
distinguish an unapplied option from an applied option. A later valid
`WithFallbackProfileFS` replaces both values. Option application remains
sequential, so an invalid option fails `NewEngine` immediately even if a later
option could otherwise replace it.

The exact GoDoc for `WithFallbackProfileFS` must state:

- that it supplies application-owned fallback profile definitions;
- the full four-layer lookup order;
- that definitions are whole profiles and are not field-merged;
- that only a missing ID falls through, while a matching read, parse,
  duplicate, validation, or credential-format failure stops resolution;
- that loading and validation are lazy;
- that files use the ordinary strict profile YAML and `api_key_env` rules;
- that nil filesystems and blank roots cause `NewEngine` to match
  `ErrInvalidConfig`;
- that repeated calls use the last valid fallback source; and
- that this is definition lookup, not provider or generation failover.

Update the `Option`, `Config.ProfileDir`, `WithProfileFS`, `WithProfileFile`,
and `WithProfiles` GoDoc in `engine.go` where necessary so their relative
precedence is unambiguous. Exact declarations and behavior remain owned by
GoDoc; consumer documentation should summarize the workflow and link readers
back to the API rather than reproduce every error clause.

### Repository Composition

Move all profile-source composition into one private root helper in
`engine.go`:

```go
func newProfileRepository(profileDir string, options engineOptions) profile.Repository
```

`NewEngine` must call this helper once and pass its returned repository to the
runner. The helper must build from lowest to highest precedence:

1. begin with `builtin.NewRepository()`;
2. if `options.fallbackProfileSource` is true, overlay
   `options.fallbackProfiles` over the built-in repository;
3. select exactly one ordinary configured source: use `options.profiles` when
   `options.profileSource` is true; otherwise, when `profileDir` is nonblank,
   use `profile.NewFilesystemRepository(profileDir)`; overlay that selected
   source over the current repository;
4. if `options.memorySource` is true, overlay `options.memoryProfiles` over
   the current repository; and
5. return the resulting chain.

This preserves the existing rule that `WithProfileFS` or `WithProfileFile`
replaces `Config.ProfileDir`; those sources are alternatives in one ordinary
configured-source category, not two independent layers. `WithProfiles` remains
a distinct highest-precedence category.

The final lookup order is therefore:

```text
WithProfiles
    -> WithProfileFile / WithProfileFS / Config.ProfileDir
        -> WithFallbackProfileFS
            -> Promptkit built-ins
```

Every arrow is a whole-profile, not-found-only fallback. Do not inspect or
copy profile fields in the composition helper.

### Built-In Package Boundary

Reduce `internal/profile/builtin/repository.go` to the embedded catalog leaf:

```go
func NewRepository() profile.Repository
```

Remove `NewRepositoryWithPrimary` and `NewRepositoryWithDirectory`. Remove
their now-obsolete tests from `internal/profile/builtin/repository_test.go` and
remove imports used only by those helpers or tests. Do not move their tests to
another private helper: existing root public-behavior tests own assembled
precedence, and `internal/profile.TestOverlayRepository` owns not-found-only
overlay semantics.

Do not change built-in YAML assets, built-in validation, the `profile.Repository`
interface, `profile.NewOverlayRepository`, or filesystem repository behavior.

### Resolution And Error Semantics

The runner receives one assembled `profile.Repository`; do not add fallback
logic to `InspectProfile`, `Prepare`, `PrepareExecution`, `Run`, or
`RunPrepared`. Those paths must continue to resolve through the runner's one
repository dependency.

The existing overlay contract is authoritative:

- a successful lookup returns the complete higher-layer profile;
- `profile.ErrProfileNotFound` consults the next layer;
- cancellation, filesystem read failures, malformed YAML, duplicate matches,
  raw `api_key`, invalid profiles, and all other errors stop lookup; and
- the root facade maps failures through the existing public identities such as
  `ErrProfileNotFound` and `ErrProfileLoad`.

Do not add eager filesystem walking in the option or `NewEngine`. A malformed
asset unrelated to the requested ID retains the existing ordinary
`FSRepository` behavior; this plan does not strengthen that package's global
validation guarantees.

### Test Ownership

Use the following test boundaries and avoid duplicating the profile parser's
existing case matrix.

In `public_contract_test.go`:

- Add `TestFallbackProfileSourcePrecedence`. Use minimal synthetic
  `fstest.MapFS` profiles and, where `Config.ProfileDir` is under test, a
  `t.TempDir`. Cover these distinct relationships: an in-memory profile beats
  both ordinary and fallback definitions; an ordinary `WithProfileFS` source
  beats a fallback definition; `Config.ProfileDir` beats a fallback
  definition when no ordinary source option replaces it; a fallback
  definition beats a built-in definition with the same ID; and an ID absent
  from the fallback source still resolves from the built-in catalog. Assert
  the selected model or another stable complete-profile field rather than
  internal repository structure.
- Extend `TestRepeatedOptionsUseLastValueInEachCategory` with a
  `fallback profile source` subtest proving that the later valid fallback
  filesystem is selected.
- Add `TestFallbackProfileSourcePreservesLazyLoadingAndErrors`. Prove that
  engine construction succeeds without reading malformed fallback YAML, that
  an unrelated malformed file does not prevent a valid requested fallback
  definition from resolving under existing FS-repository semantics, that a
  malformed fallback file whose stem matches a built-in profile ID yields
  `ErrProfileLoad` instead of silently reaching the built-in, and that a
  malformed ordinary configured definition yields `ErrProfileLoad` instead of
  reaching a valid application fallback definition. Use `errors.Is`; do not
  assert complete error strings.
- Add one representative workflow test that supplies a fallback-only profile
  and verifies the same effective model through `InspectProfile`, `Prepare`,
  a `PrepareExecution` followed by `RunPrepared`, and direct `Run`. Use the
  existing deterministic injected-client style, no live provider, and no real
  credential. This test owns the cross-workflow repository wiring; do not
  repeat the full precedence matrix through every method.

In `engine_test.go`, extend `TestSourceOptionsRejectInvalidInputs` with nil
filesystem and blank-root cases for `WithFallbackProfileFS`. Both must make
`NewEngine` match `ErrInvalidConfig`.

Retain `internal/profile.TestOverlayRepository` unchanged unless a genuine
existing defect is found. It already owns success, not-found fallback, and
non-not-found error preservation. Do not add package-private tests for the new
root helper, snapshots, golden files, provider calls, or one test per profile
format error already covered by `internal/profile`.

### Canonical Documentation

Update current-state documentation when the option is implemented:

- In `docs/formats.md`, make the source-precedence section the canonical
  four-layer definition lookup order. State that ordinary configured sources
  override application fallbacks, application fallbacks override built-ins,
  profiles are whole values, and only a missing ID falls through. Retain this
  document's ownership of strict YAML, credentials, validation, and source
  discovery details.
- In `docs/consumers/pkg-promptkit.md`, add a short task-oriented section that
  shows an illustrative `embed.FS` declaration and
  `WithFallbackProfileFS`. Explain that application defaults belong in the
  embedded fallback and operator overrides belong in the ordinary configured
  source. Link to `docs/formats.md` for exact format and precedence rules, and
  do not turn the snippet into a second complete maintained application.
- In `docs/internal/sources.md`, describe the root-owned four-layer
  composition and the existing not-found-only overlay mechanism. Remove any
  claim that the built-in package composes caller-selected repositories.
- In `docs/internal/overview.md`, keep the root facade responsible for source
  assembly, describe `internal/profile/builtin` only as the embedded catalog,
  and reflect the implemented fallback layer without duplicating the exact
  public API contract.
- In `engine.go`, apply the GoDoc changes under Public API. GoDoc owns the
  exact option signature, validation, category, and public semantics.

The architecture policy already assigns assembly to the root facade and the
built-in catalog to `internal/profile/builtin`; do not edit it unless the
implementation reveals an actual contradiction. No integration protocol,
outbound request body, profile YAML shape, README orientation, or maintained
example changes as part of this feature.

## Stage 1: Move Existing Profile Composition To The Root Facade

### Objective

Establish the intended ownership boundary and a single root composition point
without changing public behavior or adding the fallback option.

### Implementation Prompt

Implement only Stage 1 of `docs/roadmap/implementation.md`. Read the complete
feature roadmap, implementation rules, and fixed design above before editing.

1. In `engine.go`, add `newProfileRepository(profileDir string, options
   engineOptions) profile.Repository` and move the existing three-layer
   assembly into it: built-ins, then the selected ordinary configured source,
   then `WithProfiles`. Do not add fallback fields or the public option yet.
2. Replace the inline profile assembly in `NewEngine` with one call to the
   helper. Preserve the existing replacement relationship between
   `Config.ProfileDir` and `WithProfileFS`/`WithProfileFile`.
3. In `internal/profile/builtin/repository.go`, remove
   `NewRepositoryWithPrimary` and `NewRepositoryWithDirectory`, leaving
   `NewRepository` as the only constructor.
4. Remove the three tests dedicated to the deleted built-in composition
   helpers from `internal/profile/builtin/repository_test.go`. Preserve tests
   that validate the embedded catalog itself.
5. Update `docs/internal/sources.md` and `docs/internal/overview.md` so they
   describe the implemented Stage 1 ownership accurately. At this boundary
   the source order is still in-memory, ordinary configured source, built-ins;
   do not document the application fallback as implemented yet.
6. Run the focused validation below. Fix in-scope regressions without adding
   fallback behavior early.

Do not add or mention an implemented `WithFallbackProfileFS` in Stage 1. Do
not change exported declarations, profile parsing, profile assets, error
mapping, runner behavior, or consumer and format documentation.

### Focused Validation

Run from the repository root:

```sh
gofmt -w engine.go internal/profile/builtin/repository.go \
  internal/profile/builtin/repository_test.go
go test . -run \
  'Test(PrepareUsesBuiltInProfileWithoutProfileDir|CustomProfileOverridesBuiltInProfile|InMemoryProfilesOverrideBuiltInsAndProfileSources)$'
go test ./internal/profile/...
go test . ./internal/profile/...
go vet . ./internal/profile/...
git diff --check
```

If a focused expression does not match an existing test name, inspect the
current suite and run the narrowest equivalent public precedence coverage; do
not silently skip the intended relationship.

### Completion Gate

Stage 1 is complete only when:

- the root facade assembles the unchanged three-layer profile chain in one
  private helper;
- ordinary source options still replace `Config.ProfileDir` and in-memory
  profiles still have highest precedence;
- built-in profiles remain available and remain lower than consumer sources;
- only `internal/profile` owns generic overlay behavior and the built-in
  package owns only its embedded catalog;
- no public API or behavior changed;
- internal current-state documentation matches that boundary; and
- all focused tests, vet, formatting, and whitespace checks pass.

## Stage 2: Add Application Fallback Profiles And Public Contracts

### Objective

Add the public option, insert the application fallback into the root-owned
repository chain, prove its precedence and failure behavior through public
workflows, and publish the canonical current-state documentation.

### Implementation Prompt

Implement only Stage 2 of `docs/roadmap/implementation.md` after Stage 1
satisfies its completion gate.

1. Add `fallbackProfiles` and `fallbackProfileSource` to `engineOptions`, then
   implement `WithFallbackProfileFS` exactly as specified under Public API.
2. Extend `newProfileRepository` so it constructs the fixed four-layer chain
   in the prescribed low-to-high order. Do not alter generic overlay logic or
   add fallback branches to runner methods.
3. Update all affected `engine.go` GoDoc, including the option category list
   and the relative precedence descriptions for existing profile sources.
4. Add and extend the external-package root tests exactly as specified under
   Test Ownership. Reuse small existing fakes and fixture helpers where they
   remain clear; add only minimal synthetic YAML helpers needed by these tests.
5. Extend `TestSourceOptionsRejectInvalidInputs` with the two fallback option
   validation cases.
6. Update `docs/formats.md`, `docs/consumers/pkg-promptkit.md`,
   `docs/internal/sources.md`, and `docs/internal/overview.md` according to
   Canonical Documentation.
7. Run the focused validation below. Repair in-scope failures without
   weakening existing parser, error-identity, prepared-execution, or profile
   precedence guarantees.

Do not add a config field, in-memory fallback API, source provenance, profile
inheritance, provider failover, eager validation, application-specific assets,
or backend/model policy. Do not edit `internal/profile/builtin/assets/`.

### Focused Validation

Run from the repository root:

```sh
gofmt -w engine.go engine_test.go public_contract_test.go
go test . -run \
  'Test(FallbackProfileSource|RepeatedOptionsUseLastValueInEachCategory|SourceOptionsRejectInvalidInputs)'
go test ./internal/profile/...
go test . ./internal/profile/... ./internal/usecase
go vet . ./internal/profile/... ./internal/usecase
git diff --check
```

The focused root expression must execute the precedence, lazy/error,
cross-workflow, repeated-option, and invalid-input coverage described above.
If the implemented names differ slightly, run explicit equivalent expressions
and record no skipped contract category.

### Completion Gate

Stage 2 is complete only when:

- `WithFallbackProfileFS` is the sole new public declaration and has complete,
  accurate GoDoc;
- nil filesystem and blank root inputs fail construction with
  `ErrInvalidConfig`, and the last valid repeated fallback option wins;
- the assembled order is in-memory, ordinary configured, application
  fallback, built-ins;
- existing ordinary source options still replace `Config.ProfileDir`;
- lookup falls through only on a missing ID and never after a matching
  higher-layer failure;
- profiles remain whole values and loading remains lazy;
- inspection, preparation, prepared execution, and ordinary execution resolve
  the same fallback definition through one runner repository;
- no built-in asset, profile format, public error identity, provider request,
  or existing consumer behavior changed unintentionally;
- GoDoc and all affected canonical documents describe implemented behavior
  without duplicating ownership; and
- all focused tests, vet, formatting, links, and whitespace checks pass.

## Stage 3: Audit Compatibility And Validate The Repository

### Objective

Confirm that the implementation is complete, minimal, and consistent across
the public facade, internal boundaries, tests, and documentation, then mark
the temporary planning documents complete.

### Implementation Prompt

Implement only Stage 3 of `docs/roadmap/implementation.md` after Stage 2
satisfies its completion gate.

1. Search tracked Go and Markdown files for
   `NewRepositoryWithPrimary`, `NewRepositoryWithDirectory`, profile source
   precedence lists, and descriptions of built-in repository composition.
   Remove stale references and correct only feature-owned contradictions.
2. Review `newProfileRepository` directly and confirm it has exactly four
   possible layers in the required order, selects only one ordinary configured
   source, and contains no profile field merging or eager I/O.
3. Review the public tests as a suite. Confirm that each distinct risk in Test
   Ownership is protected once, generic parser and overlay cases remain with
   `internal/profile`, and no test depends on private helper shape.
4. Confirm that `internal/profile/builtin/assets/`, external wire behavior,
   backend configuration, credential resolution, public result shapes, and
   stable JSON tags have no feature-related changes.
5. Follow every added or changed Markdown link and confirm that its target and
   relevant heading exist. Verify that exact API details live in GoDoc, exact
   profile format and precedence details live in `docs/formats.md`, consumer
   guidance remains task-oriented, and internal documents describe only
   implementation responsibility.
6. Run the complete validation sequence below and repair only in-scope
   failures.
7. After every check passes, change the status of
   `fallback-profiles.md` and this document to `Complete`. Do not delete or
   retire either roadmap; retirement is a separate maintainer action.
8. Re-run `git diff --check`, inspect `git status --short`, and review the full
   diff while distinguishing pre-existing user changes from this feature.

Do not add release notes, change versions, or create a commit, tag, push, or
release during this stage.

### Full Validation

Run from the repository root:

```sh
gofmt -w engine.go engine_test.go public_contract_test.go \
  internal/profile/builtin/repository.go \
  internal/profile/builtin/repository_test.go
gofmt -l $(git ls-files '*.go')
go test ./...
go test -race ./...
go vet ./...
go build ./...
go run ./examples/go-library/prepare
git diff --check
git status --short
```

The `gofmt -l` command must print no paths. The maintained example must remain
offline and require no real credential or provider.

Inspect the final diff and confirm:

- only this feature's files and pre-existing user changes are present;
- no built-in asset, public value shape, stable JSON tag, provider payload,
  workspace file, local module replacement, generated binary, or unrelated
  formatting changed;
- deleted built-in composition helpers have no remaining references;
- the fallback option reuses the ordinary FS repository and generic overlay;
- the root constructs one repository used by every resolution workflow;
- the feature roadmap and this plan are both complete; and
- no commit, tag, push, or release was created.

### Completion Gate

The implementation is complete only when:

- every Stage 1 and Stage 2 gate remains satisfied;
- the ordinary and race-enabled suites pass;
- vet, build, formatting, the maintained offline example, Markdown links, and
  whitespace checks pass;
- the four-layer precedence and not-found-only fallthrough are consistent in
  code, GoDoc, public tests, format reference, consumer guidance, and internal
  documentation;
- engines without `WithFallbackProfileFS` retain their previous behavior;
- the public surface contains no speculative companion API or provenance;
- both temporary roadmap statuses are `Complete`; and
- the repository is ready for maintainer review without a commit or release
  having been created by this plan.

## Open Questions

None. The feature roadmap and fixed design above fully specify the public API,
repository composition, error and validation behavior, compatibility boundary,
documentation ownership, test strategy, and staged implementation sequence.