promptkit/docs/roadmap/implementation.md

# Profile Inheritance Implementation Plan

## Purpose

Implement the profile-inheritance feature defined by the
[profile inheritance roadmap](profile-inheritance.md). This plan is the
decision-complete execution sequence for a coding agent. Follow the repository
[architecture](../policy/architecture.md), [testing](../policy/testing.md), and
[documentation](../policy/documentation.md) policies and the task-specific
reading guide in [`docs/development.md`](../development.md) throughout the
work.

## Target Outcome

A downstream consumer can define a YAML or in-memory profile whose
`base_profile`/`BaseProfileID` names another profile, including a Promptkit
built-in. Promptkit resolves the complete one-parent chain through the assembled
profile catalog, merges root-to-leaf profile settings, resolves the final
backend, and then applies request overrides exactly as it does for a standalone
profile.

The selected leaf ID remains the public profile identity. Ordinary operations
resolve sources afresh, while prepared execution freezes the resolved target.
Missing dependencies, cycles, excessive depth, and incomplete resolved profiles
fail as profile-load errors without changing the distinct not-found contract for
an absent directly selected profile.

## Fixed Decisions

- YAML uses `base_profile`; the public Go fields are
  `Profile.BaseProfileID` and
  `OpenAICompatibleProfileConfig.BaseProfileID`.
- One profile may name one base. Bases may form a linear chain of at most 32
  profiles, including the selected leaf.
- The resolver performs every lookup through the complete raw composite
  repository. Each base therefore observes the ordinary in-memory, configured,
  fallback, and built-in precedence.
- The resolving repository is stateless and does not cache results across
  calls. It is applied once, outside the fully assembled raw overlay chain.
- Profile definitions with a base may omit `model` and both connection fields.
  Standalone definitions retain their current completeness requirements.
- Root-to-leaf merge behavior follows existing profile presence semantics:
  nonblank strings and nonzero numbers override; blank strings and zero numbers
  inherit; a nonempty `extra_params` map replaces the complete inherited map;
  and `APIKeyRequired` is a sticky logical OR.
- `id` always comes from the selected leaf. `base_profile` is cleared from the
  resolved profile before it reaches the use-case layer.
- Backend and endpoint remain independent fields. A child backend replaces the
  inherited backend ID without implicitly clearing an inherited endpoint; a
  child endpoint replaces only the endpoint.
- This work introduces no profile-level clearing syntax, presence pointers,
  multiple inheritance, source-qualified references, public resolution graph,
  new public error sentinel, or provider-wire change.
- A missing directly selected profile continues to expose
  `ErrProfileNotFound`. Once the leaf exists, a missing base is an invalid
  selected profile and must expose `ErrProfileLoad` without also matching
  `ErrProfileNotFound`.
- Existing raw-source failures and context cancellation remain discoverable
  through `errors.Is` where they are currently preserved. The missing-base
  branch is the one deliberate exception: its diagnostic names the missing ID
  and chain but does not wrap the internal not-found sentinel.
- No release notes, version tags, commits, or pushes are part of these stages.

## Execution Rules

- Complete the stages in numerical order. Each stage is sized for one
  gpt-5.6-terra implementation prompt and must finish its focused verification
  before the next begins.
- At the start of every stage, inspect the working tree and preserve unrelated
  changes. The feature roadmap is intentional planning state and must remain.
- Treat intermediate stages as an unreleased partial implementation. Do not
  describe the feature in durable current-state prose until Stage 4.
- Keep inheritance logic in `internal/profile`. Do not add recursive profile
  lookup, chain merging, or source-precedence logic to `internal/usecase` or the
  root public facade.
- Use real in-memory repositories and synthetic `fs.FS` sources where practical.
  Tests must remain deterministic, offline, parallel-safe, and independent of
  provider credentials.
- Update existing test owners when their validation contract changes; do not
  retain contradictory expectations or duplicate the full inheritance matrix
  at higher layers.
- Use `apply_patch` for edits, format every changed Go file, and review the
  complete stage diff before proceeding.

## Stage 1: Add Profile Definition Surfaces And Separate Local Validation

### Objective

Represent a base-profile reference in every supported profile input while
preserving strict decoding, standalone-profile compatibility, and eager local
validation. At the end of this stage repositories can publish locally valid
derived definitions, but the engine does not yet resolve them.

### Implementation

1. Add `BaseProfileID string` to `internal/domain.ExecutionProfile` with the
   YAML tag `base_profile`. Keep it out of execution targets and provider
   request types.
2. Add `BaseProfileID string` to the public `Profile` immediately after `ID`.
   Its GoDoc must state that:
   - `WithProfiles` trims the value;
   - a nonblank value permits required target fields to be inherited;
   - reference existence and resolved completeness are checked only when the
     profile is selected or inspected; and
   - blank means the profile remains standalone.
3. Add the matching field to `OpenAICompatibleProfileConfig`, document that it
   maps to `Profile.BaseProfileID`, and copy it in
   `OpenAICompatibleProfile`. Preserve the constructor's shallow-copy and
   deferred deep-validation behavior for `ExtraParams`.
4. Map and trim `Profile.BaseProfileID` in `toDomainProfile`. Ensure the memory
   repository continues to return caller-independent `ExtraParams` and does not
   expose shared mutable profile state.
5. Centralize the duplicated file-profile and in-memory-profile normalization
   rules in `internal/profile` rather than adding a third validation path:
   - provide an internal-package exported helper named
     `NormalizeAndValidateDefinition` so the root facade can validate a
     converted in-memory `domain.ExecutionProfile` without owning profile
     parsing rules;
   - trim `ID`, `BaseProfileID`, `BackendID`, and `Endpoint` in that helper;
   - require a nonblank `ID` and validate a supplied endpoint and all execution
     setting bounds for every definition;
   - when `BaseProfileID` is blank, retain the existing requirements for a
     nonblank model and at least one nonblank backend or endpoint;
   - when `BaseProfileID` is nonblank, allow model, backend, and endpoint to be
     omitted so they can be inherited; and
   - do not add unrelated normalization, environment-name policy, clearing
     behavior, or backend registry lookup.
6. Replace `normalizeAndValidateProfile` in
   `internal/profile/filesystem_repository.go` and
   `normalizeAndValidatePublicProfile` in `profiles.go` with the shared helper.
   Preserve each caller's current public/internal error wrapping, selected-file
   path diagnostic, raw-key prohibition, strict single-document rule, and
   duplicate-ID behavior. Remove imports and helpers made obsolete by the
   centralization.
7. Keep the raw repositories raw: file, `fs.FS`, memory, overlay, and built-in
   repositories return `BaseProfileID` but do not recursively load it in this
   stage.

### Tests

Update the narrow existing owners:

1. In `internal/profile/repository_test.go`, extend the filesystem/`fs.FS`
   source-parity coverage to prove that:
   - strict YAML accepts `base_profile`, trims it, and publishes it;
   - an alias containing only `id` and `base_profile` is locally valid;
   - a derived profile still rejects an invalid supplied endpoint, negative or
     out-of-range setting, malformed `extra_params`, raw `api_key`, duplicate
     ID, and extra YAML document through their existing identities; and
   - a standalone profile missing connection fields or model remains invalid.
   Modify existing missing-field expectations rather than duplicating their
   entire matrices.
2. In root package tests, extend the existing `WithProfiles` validation owner
   to show that an incomplete profile with a nonblank `BaseProfileID` is
   accepted by `NewEngine`, while the equivalent standalone profile retains its
   current `ErrInvalidConfig` result. Do not attempt to select the derived
   profile until the resolver is implemented.
3. Extend `TestOpenAICompatibleProfileMapsEveryField` so
   `BaseProfileID` is included in the constructor's exact field mapping.
4. Preserve existing tests for defensive copying, duplicates, standalone
   profiles, overlay precedence, built-ins, and strict decoding.

### Verification

```sh
gofmt -w internal/domain/domain.go internal/profile/*.go profiles.go types.go
go test ./internal/profile
go test ./internal/profile/builtin
go test ./... -run 'Profile|OpenAICompatibleProfile'
go test ./...
git diff --check
```

Stage 1 is complete when all supported definition surfaces carry the normalized
base ID, locally valid aliases can be stored, standalone validation is
unchanged, and no repository performs inheritance resolution yet.

**Status:** Complete.

## Stage 2: Implement The Stateless Resolving Repository

### Objective

Add the internal component that resolves one complete profile chain, merges it
deterministically, enforces safety bounds, and returns one ordinary resolved
profile to its caller.

### Implementation

1. Add `internal/profile/resolving_repository.go` with:
   - an unexported `resolvingRepository` containing one underlying raw
     `Repository`;
   - `NewResolvingRepository(source Repository) Repository`; and
   - an unexported constant setting the maximum chain length to 32, counting
     the selected leaf.
2. Make the wrapper safe for concurrent calls by keeping the source immutable
   and all chain, visited-ID, and merge state local to `GetProfile`. Do not add a
   cache, mutex, process-global registry, or source snapshot.
3. Resolve one request using this exact flow:
   - reject a nil underlying repository or blank requested ID as
     `ErrInvalidProfile`;
   - trim and look up the directly requested ID through the underlying raw
     repository;
   - if that first lookup fails, return its error unchanged so direct
     `ErrProfileNotFound` behavior remains intact;
   - reject a nil profile returned without an error as `ErrInvalidProfile`;
   - follow each normalized nonblank `BaseProfileID` through the same underlying
     repository, recording IDs and profiles from leaf toward root;
   - check `ctx.Err()` between lookups;
   - detect a repeated ID before loading it again and return an
     `ErrInvalidProfile` cycle diagnostic containing the `a -> b -> a` chain;
   - reject a reference that would make the chain exceed 32 profiles with an
     `ErrInvalidProfile` diagnostic containing the traversed chain and next ID;
   - when a base lookup returns `ErrProfileNotFound`, return a new
     `ErrInvalidProfile` error naming the missing base and chain without
     wrapping `ErrProfileNotFound`; and
   - for any other base error, add `ErrInvalidProfile`, base ID, and chain
     context while preserving the source error and cancellation identity with
     wrapping.
4. Merge the collected definitions from root to leaf into a new value. Use one
   focused helper with these rules:
   - retain the selected leaf `ID` and clear final `BaseProfileID`;
   - replace `BackendID`, `Endpoint`, `Model`, `ServiceTier`,
     `ReasoningEffort`, and `APIKeyEnv` only for a nonblank child value;
   - replace `Temperature`, `MaxTokens`, `TopP`, and `TimeoutSeconds` only for a
     nonzero child value;
   - set `APIKeyRequired` to the logical OR of every definition in the chain;
   - replace `ExtraParams` only when the child map is nonempty; otherwise
     inherit the current map; and
   - defensively deep-copy every retained or replacement `ExtraParams` map by
     using the existing bounded `internal/jsonvalue` owner. Never mutate a
     repository-returned profile or map. Treat an unexpected copy failure as
     `ErrInvalidProfile`, preserve the copy error, and include the selected
     chain in the diagnostic.
5. Add an unexported final validator in `internal/profile` that requires a
   nonblank model and at least one backend or endpoint on the merged profile,
   and reuses the common endpoint and execution-setting validation. Wrap final
   failures with `ErrInvalidProfile` and the selected chain. The resolver must
   not perform backend registry membership lookup; that remains in
   `internal/usecase` after profile resolution.
6. Return a caller-owned resolved profile on every successful call. A
   standalone profile passes through the same final validation and copy path,
   ensuring the wrapper does not create separate standalone and inherited
   execution contracts.

### Tests

Create `internal/profile/resolving_repository_test.go` and use small synthetic
repositories plus the real overlay repository where it materially protects
composition:

1. Prove a three-level root/middle/leaf chain merges representative values for
   every rule: inherited values, string and numeric replacements, independent
   backend/endpoint behavior, sticky `APIKeyRequired`, complete-map
   `ExtraParams` replacement, leaf identity, and cleared resolution metadata.
   Use one well-structured table or fixture rather than one test per field.
2. Prove an empty child `ExtraParams` inherits the base map, a nonempty child
   map replaces rather than key-merges it, and mutating the returned nested
   values cannot mutate the raw repository or a later result.
3. Prove the same composite repository resolves a base from a lower-precedence
   source and honors a higher-precedence shadow of that base ID. Do not add
   source-specific resolution to the wrapper.
4. Cover the safety/error matrix through the resolving repository boundary:
   - directly selected missing ID preserves `ErrProfileNotFound`;
   - missing base matches `ErrInvalidProfile` and does not match
     `ErrProfileNotFound`;
   - direct and indirect cycles are rejected with useful chains;
   - exactly 32 profiles are accepted and a 33rd is rejected;
   - nil repository results and incomplete final profiles are invalid;
   - non-not-found base errors retain their original identity; and
   - cancellation before or during traversal remains discoverable and stops
     later lookups.
5. Prove freshness and concurrency without timing sleeps: change a controlled
   raw repository between sequential calls and observe the new base value, then
   issue concurrent independent resolutions and verify results do not share
   mutable maps. Use synchronization owned by the fake; do not race on a plain
   map.
6. Preserve the existing raw repository and overlay tests unchanged except
   where Stage 1 intentionally changed local completeness rules.

### Verification

```sh
gofmt -w internal/profile/*.go
go test ./internal/profile -run 'Resolving|Inheritance|Profile'
go test -race ./internal/profile
go test ./...
git diff --check
```

Stage 2 is complete when a standalone internal wrapper can safely and freshly
resolve every accepted chain into one complete, copied profile with the fixed
merge and error contracts.

## Stage 3: Assemble Inheritance And Prove Public Workflows

### Objective

Place the resolver at the correct engine boundary and prove that YAML,
in-memory, built-in, inspection, ordinary preparation, runtime overrides, and
prepared execution compose as intended through the public facade.

### Implementation

1. Update `newProfileRepository` in `engine.go` to:
   - assemble the raw repository in its current precedence order—built-ins,
     fallback, configured source or `Config.ProfileDir`, then in-memory;
   - wrap that complete raw repository exactly once with
     `profile.NewResolvingRepository`; and
   - return the wrapper to `usecase.NewRunner`.
   Do not wrap individual sources, because that would prevent a higher-level
   child from referencing a lower-level base and would change shadowing rules.
2. Keep `internal/usecase.resolveProfileSelection` structurally unchanged. It
   should continue to request one profile, preserve the requested normalized ID
   as selection identity, resolve the final backend once, and then apply the
   ordinary backend/profile/request precedence. Do not pass `BaseProfileID`
   into `ExecutionTarget`, `PreparedRun`, `RunResult`, injected `LLMClient`
   requests, stable JSON, hashes, or provider payloads.
3. Confirm `APIKeyRequired` behavior after inheritance without introducing a
   special credential path: a true value in any ancestor reaches the existing
   profile-to-target merge, clears inherited profile/backend `APIKeyEnv`, and
   still permits a direct request key or explicit request environment override.
4. Preserve the current error mapper. The resolver's use of
   `ErrInvalidProfile` must make a missing/invalid base cross the use-case and
   root boundaries as `ErrProfileLoad`; no new root sentinel or special-case
   string matching is permitted.
5. Do not modify built-in profile assets merely to demonstrate inheritance.
   Built-ins serve as valid bases without changing their own definitions.

### Tests

Use the smallest public tests that protect assembled behavior without repeating
the resolver's matrix:

1. Add one external-package contract workflow that defines `weather-light` as
   an in-memory alias/refinement of the built-in `deepseek-4-flash`. Compare
   its inspected target with an inspection of the base rather than duplicating
   every built-in literal. Assert:
   - the child inspection reports `weather-light`;
   - backend and model are inherited;
   - explicit child reasoning and timeout values win;
   - `Prepare` reports the child as `SelectedProfileID`; and
   - a per-request reasoning or timeout override still wins over the child.
2. Exercise YAML through `WithProfileFS` with a minimal alias of a built-in and
   prove it resolves through inspection or preparation. Parser tests own YAML
   edge cases, so do not repeat them here.
3. Add a focused public error-identity table:
   - a directly selected absent profile still matches `ErrProfileNotFound` and
     not `ErrProfileLoad`;
   - an existing child with an absent base matches `ErrProfileLoad` and not
     `ErrProfileNotFound`; and
   - one representative cycle matches `ErrProfileLoad` and includes its IDs.
   Reuse or extend the existing profile-inspection public error owner where
   that keeps the suite lean.
4. Prove fresh ordinary resolution and frozen prepared execution with one
   controlled mutable `fs.FS` workflow:
   - prepare an execution while its base supplies model A;
   - change the base source to model B after preparation;
   - run the prepared handle through an injected deterministic client and
     observe model A; and
   - perform a new inspection or ordinary preparation and observe model B.
   Keep all mutation sequential or synchronized and make no provider call.
5. Add only a narrow credential assertion if existing use-case tests do not
   already prove the final inherited `APIKeyRequired` target behavior. The
   resolver test owns sticky merging; current credential tests own availability,
   precedence, redaction, and prepared rechecking.
6. Run all existing source-precedence, fallback, malformed-profile, built-in,
   public JSON, runtime-override, credential, and prepared-execution tests. Fix
   production behavior rather than weakening those contracts.

### Verification

```sh
gofmt -w engine.go engine_test.go public_contract_test.go prepared_execution_contract_test.go profiles.go types.go
go test ./... -run 'Profile|Inheritance|PreparedExecution|RuntimeOverride'
go test ./internal/profile
go test ./internal/usecase
go test -race ./...
go vet ./...
go build ./...
git diff --check
```

Stage 3 is complete when the engine resolves aliases and refinements across all
supported sources, public identities and errors match the roadmap, ordinary
operations remain fresh, and prepared execution remains frozen.

## Stage 4: Publish Canonical Documentation And Complete Validation

### Objective

Align every canonical contract owner with the implemented feature, remove
temporary ambiguity, and validate the release candidate comprehensively.

### Documentation

1. Finalize exact GoDoc for `Profile.BaseProfileID` and
   `OpenAICompatibleProfileConfig.BaseProfileID`, plus nearby `Profile`,
   `OpenAICompatibleProfile`, and `WithProfiles` text whose standalone
   completeness or validation-timing statements changed. GoDoc owns the public
   fields and construction contract; do not reproduce internal traversal.
2. Update `docs/formats.md` as the exact YAML contract owner:
   - add `base_profile` to the field table and example;
   - distinguish locally complete standalone profiles from derived aliases;
   - document root-to-leaf override rules, complete-map `extra_params`
     replacement, sticky `APIKeyRequired`, independent backend/endpoint
     behavior, the 32-profile limit, and lack of clearing syntax;
   - document cross-source lookup precedence and shadowing; and
   - distinguish a missing selected profile from an invalid selected chain.
3. Update `docs/internal/sources.md` to describe the raw composite catalog,
   the one outer resolving repository, fresh per-operation traversal,
   cycle/depth protection, final validation, and prepared-state freezing. Link
   to the format reference for exact consumer syntax and merge rules.
4. Update the `internal/profile` entry in `docs/internal/overview.md` because
   the concrete package now resolves inherited definitions in addition to
   loading, validating, and overlaying raw sources. Do not change architecture
   policy because package ownership and dependency direction are unchanged.
5. Add one concise task-oriented example to
   `docs/consumers/pkg-promptkit.md` showing an application-owned alias of a
   built-in and a selected child override. Link to the format reference and
   public GoDoc for details rather than duplicating the complete contract.
6. Do not update the README, examples, release documents, release procedure,
   future/deferred catalogs, or built-in catalog unless the implementation has
   made a concrete current-state statement there inaccurate. Do not copy the
   roadmap into durable documentation.

### Final Validation

Run the complete maintainer workflow from
[`docs/development.md#maintainer-validation`](../development.md#maintainer-validation):

```sh
go test ./...
go test -race ./...
go vet ./...
go build ./...
go run ./examples/go-library/prepare
go run ./examples/go-library/run
```

Then run the documented tracked-Go formatting check, repository-relative
Markdown link checker, workspace/vendor/module-replacement checks, unstaged and
staged whitespace checks, ignored-file check, credential scan, and
`git diff --check`. Inspect the complete diff and confirm:

- only `base_profile`/`BaseProfileID` were added to consumer configuration;
- raw repositories remain source-local and the resolver wraps the full catalog
  once;
- chain resolution is bounded, cycle-safe, fresh, and concurrency-safe;
- merge behavior covers every profile field without mutating source values;
- direct missing and dependent missing profiles preserve distinct public error
  identities;
- selected IDs, backend resolution, runtime overrides, credentials, and
  prepared freezing retain their stated ownership;
- resolution metadata never reaches stable execution JSON, hashes, injected
  generation targets, or provider payloads;
- tests remain lean, deterministic, offline, and owned at the narrowest stable
  boundary; and
- durable documentation describes only implemented behavior with one canonical
  owner per exact contract.

Stage 4 is complete when all focused and repository-wide checks pass and the
implementation, GoDoc, format reference, internal documentation, and consumer
guidance agree.

## Open Questions

None. Naming, supported sources, lookup precedence, merge behavior, connection
semantics, credential inheritance, validation timing, chain bounds, error
identity, freshness, prepared freezing, package ownership, test boundaries, and
documentation ownership are fixed by the feature roadmap and this plan.