Refine the external catalog implementation plan

docs/roadmap/implementation.md

@@ -47,6 +47,11 @@ Apply these constraints throughout every stage:
 - Keep all default tests deterministic, offline, and credential-free. Remote
   publication and module-resolution checks are release gates, not test-suite
   behavior.
+- Treat copying the Promptkit-maintained catalog assets into the LGPL-3.0
+  external repositories as an intentional copyright-holder relicensing
+  decision. Preserve source provenance and record that decision in both
+  catalog READMEs; do not imply that an ordinary dependency extraction alone
+  changes an asset's license.
 - Update the canonical current-state documentation in the same commit that
   introduces or changes the behavior it describes. In particular, do not
   defer an implemented-package inventory or source-boundary update to a later
@@ -204,7 +209,10 @@ commit, and release immutable tag `v1.0.0` before Promptkit depends on it.
 2. Add `catalog/backend.json` using the exact OpenRouter manifest from the
    shared contract. Copy every current OpenRouter-owned YAML asset from
    `promptkit/internal/profile/builtin/assets/` into
-   `catalog/profiles/<provider>/` without editing its bytes.
+   `catalog/profiles/<provider>/` without editing its bytes. Record in the
+   README that the Promptkit-maintained source assets are intentionally being
+   distributed under this repository's LGPL-3.0 terms with authorization from
+   their copyright holder, and identify Promptkit as their source provenance.
 3. Add the private embedded filesystem and the exact `Root`/`FS` public
    surface. Embed only `catalog`, return the embedded filesystem by value
    behind `fs.FS`, and add package GoDoc explaining that the module supplies
@@ -271,7 +279,9 @@ release history or package implementation to the OpenRouter module.
    name `rakestrawhome`, and Go version `1.25.5`.
 2. Add the exact Rakestrawhome manifest from the shared contract. Copy only
    `google/rakestrawhome-gemma-4-31b.yml` into
-   `catalog/profiles/google/`, without editing its bytes.
+   `catalog/profiles/google/`, without editing its bytes. Apply the same
+   explicit LGPL-3.0 relicensing and Promptkit source-provenance statement as
+   Stage 2.
 3. Apply the same focused asset tests, changing the expected backend to
    `rakestrawhome` and the initial profile set to the single owned profile.
    Keep the module independent: do not import or share code with either
@@ -312,27 +322,39 @@ point-in-time semantics of consumer-configured profile sources.
 1. In `internal/profile`, add:
 
    ```go
+   type LoadedProfileMetadata struct {
+       ID             string
+       Path           string
+       ExplicitFields []string
+   }
+
    func LoadFSRepository(
        ctx context.Context,
        fsys fs.FS,
        root string,
-   ) (Repository, []string, error)
+   ) (Repository, []LoadedProfileMetadata, error)
    ```
 
    It must discover sorted YAML paths through `internal/filecatalog`, read each
    file once, require exactly one document, strictly decode the existing
    profile schema, reject raw API keys, normalize and validate each raw
    definition through the existing owners, and reject duplicate trimmed IDs.
-   Return an immutable in-memory raw repository plus a newly allocated sorted
+   Return an immutable in-memory raw repository plus a newly allocated metadata
-   ID slice. An empty source returns an empty repository and ID slice; the
+   slice sorted by normalized profile ID. Each metadata entry contains that ID,
-   catalog adapter, not this generic primitive, decides whether emptiness is
+   the safe root-relative source path, and a newly allocated sorted list of the
-   invalid.
+   exact top-level YAML field names present in the source document. An empty
+   source returns an empty repository and metadata slice; the catalog adapter,
+   not this generic primitive, decides whether emptiness is invalid. All three
+   fields, the type, and the function are internal to the Promptkit module but
+   require accurate GoDoc because they cross internal package boundaries.
 2. Refactor existing private decode/metadata logic only as needed so eager and
    point lookup share strict decoding, source-path context, raw-key rejection,
    normalization, and defensive JSON-value copying. Do not change
    `NewFSRepository`: configured and fallback consumer sources must retain
    fresh point-in-time reads and their current error-preserving fallback
-   semantics.
+   semantics. Derive `ExplicitFields` during the same source read and through
+   the existing YAML-node metadata path; do not make the later catalog adapter
+   reread or independently parse profile YAML.
 3. The returned repository must honor context cancellation before lookup,
    return `ErrProfileNotFound` for absence, and publish a fresh profile value
    with a deeply copied `ExtraParams` tree on every successful lookup. Do not
@@ -345,10 +367,12 @@ point-in-time semantics of consumer-configured profile sources.
 
 ### Tests And Validation
 
-1. Add focused profile-package tests for sorted IDs, strict malformed-input
+1. Add focused profile-package tests for sorted metadata, exact explicit-field
-   rejection, duplicate IDs, raw API-key rejection, empty input, cancellation,
+   presence including explicitly empty values, safe relative paths, strict
-   and defensive copies. Reuse representative existing fixtures and avoid
+   malformed-input rejection, duplicate IDs, raw API-key rejection, empty
-   repeating the entire profile rule matrix already owned by point lookup.
+   input, cancellation, and defensive copies of metadata and profile values.
+   Reuse representative existing fixtures and avoid repeating the entire
+   profile rule matrix already owned by point lookup.
 2. Add a parity test showing that eager and ordinary FS repositories publish
    the same raw semantic value for representative standalone and derived
    profiles.
@@ -359,7 +383,9 @@ point-in-time semantics of consumer-configured profile sources.
 
 - Promptkit can eagerly load and validate all raw profiles from an `fs.FS`
   without creating a second YAML contract implementation.
-- Returned IDs and profile values are caller-independent.
+- Returned metadata and profile values are caller-independent, and the
+  metadata preserves the distinction between an absent field and an explicitly
+  empty field.
 - Existing configured, fallback, in-memory, and built-in runtime behavior is
   unchanged.
 - The internal source document accurately distinguishes eager immutable loads
@@ -409,18 +435,24 @@ source as the active implementation.
    `json.Decoder.DisallowUnknownFields`, require exactly one JSON value, use
    presence-aware raw fields so missing required fields differ from zero or
    `null`, reject unsupported schema versions, and check the manifest ID
-   against `ExpectedBackendID`.
+   against `ExpectedBackendID`. When constructing `domain.Backend`, set
+   `QueueCapacitySet` to `true` because schema version 1 requires an explicit
+   `queue_capacity`; this ensures normalization preserves later compatible
+   releases that intentionally select a non-default capacity.
 4. Rename the existing private backend normalizer to the internal exported
    `backend.NormalizeDefinition` and have both `Registry` and the catalog
    adapter call it. This remains inside Go's `internal` boundary and is not a
    Promptkit public API. Do not duplicate endpoint, environment-name,
    capacity, reserved-field, or bounded JSON-value policy in the adapter.
 5. For each source, call `profile.LoadFSRepository` on `<root>/profiles`.
-   Reject an empty profile set. Validate every raw profile for prohibited
+   Reject an empty profile set. Use the returned explicit-field metadata to
-   connection/credential fields and forbidden nested secret keys. Resolve
+   reject `endpoint` or `api_key_env` whenever the key is present, including
-   every ID through a source-local `profile.NewResolvingRepository`; this both
+   when its YAML value is explicitly empty; do not infer source presence from
-   proves inheritance is self-contained and verifies that the final backend ID
+   the decoded profile's zero values. Validate every raw profile's nested extra
-   equals the manifest owner.
+   parameters for forbidden secret keys. Resolve every metadata ID through a
+   source-local `profile.NewResolvingRepository`; this both proves inheritance
+   is self-contained and verifies that the final backend ID equals the manifest
+   owner.
 6. Reject duplicate backend IDs and duplicate raw profile IDs across sources.
    Compose the already validated raw repositories in source order only after
    duplicate checks pass. Do not pre-resolve the returned composite: the root
@@ -585,12 +617,14 @@ across all three clean repositories.
    external catalog or maintained built-in changes. Route contributors to the
    internal source document, the format reference, testing policy, both module
    repositories, and each module's release procedure.
-3. Add `docs/releases/external-backend-catalogs.md` as short supplemental
+3. Do not add a versionless supplemental release document during feature
-   release guidance. State that the adopting Promptkit release adds two
+   implementation. Record for the later Promptkit release-preparation pass that
-   independently versioned data dependencies, preserves the public API and
+   its versioned `docs/releases/vMAJOR.MINOR.PATCH.md` document should state
-   configuration, requires no consumer migration, and guarantees only the
+   that the release adds two independently versioned data dependencies,
-   catalog versions selected and tested by that Promptkit release. Link to
+   preserves the public API and configuration, requires no consumer migration,
-   canonical current-state documents rather than restating their contracts.
+   and guarantees only the catalog versions selected and tested by that
+   Promptkit release. That document must link to canonical current-state
+   documentation rather than restating its contracts.
 4. Update each external repository README only if the final implemented paths
    or links changed during integration. Do not turn either README into a
    parallel Promptkit consumer manual.
@@ -616,8 +650,8 @@ across all three clean repositories.
 
 - Durable current-state documents accurately describe the external asset
   boundary without duplicating implementation-plan detail.
-- Supplemental release guidance clearly states the compatibility and
+- The later versioned release-document requirements are explicit without
-  dependency impact.
+  creating a release note before a Promptkit version has been selected.
 - All completion criteria in the feature roadmap hold, all three repositories
   are clean, and the complete offline validation passes without credentials,
   a workspace, a replacement, or provider network access.