Plan external catalog extraction

docs/roadmap/external-backend-catalogs.md

@@ -0,0 +1,238 @@
+# External Backend Catalogs
+
+## Status
+
+Accepted for future implementation. This roadmap defines the intended end
+state for extracting Promptkit's maintained built-in backend and profile data
+into independently versioned Go modules. It describes future behavior only;
+the current implementation and canonical documentation remain authoritative
+until the extraction is complete.
+
+## Purpose
+
+Move the rapidly changing built-in backend and model-profile catalogs out of
+the Promptkit repository without changing how consumers configure or use
+Promptkit.
+
+Provider model catalogs change much more frequently than Promptkit's engine,
+public API, and internal execution architecture. Independent catalog modules
+allow model additions, deprecations, and metadata corrections to be developed,
+tested, reviewed, and released without placing that content churn in the core
+repository's history. Promptkit can adopt tested catalog releases through
+ordinary Go module version updates.
+
+This extraction is an ownership and release-boundary improvement. It does not
+eliminate all Promptkit maintenance: Promptkit must still pin and test the
+catalog versions that it supplies by default, and adopting a newer default
+catalog ordinarily requires a Promptkit dependency update and release.
+
+## Target End State
+
+The OpenRouter and Rakestrawhome catalogs reside in separate, independently
+versioned Go repositories and modules. Each module owns exactly one built-in
+backend definition and every built-in profile maintained for that backend.
+
+Promptkit imports both modules and assembles their contents as its built-in
+catalog. Downstream consumers continue to receive the same built-in backends
+and profiles automatically when they construct an engine. They do not need to
+import a catalog module, call a registration function, or change existing
+configuration.
+
+Promptkit no longer contains duplicate embedded copies of the extracted
+profile assets or hard-coded copies of the extracted backend definitions.
+Promptkit continues to own:
+
+- the public backend constants and their exact string values;
+- catalog decoding, normalization, and validation;
+- conversion into Promptkit's internal domain values;
+- built-in and consumer-source precedence;
+- engine assembly, capacity management, credentials, and execution; and
+- the canonical consumer documentation for the built-ins supplied by each
+  Promptkit release.
+
+The catalog modules supply immutable data. They do not own Promptkit runtime
+behavior or become general-purpose provider SDKs.
+
+## External Catalog Modules
+
+Maintain one repository and Go module for OpenRouter and one for the
+Rakestrawhome inference backend. Repository and module names make both the
+Promptkit relationship and owning backend unambiguous. Every catalog version
+selected by Promptkit must be available from the same build environment used
+to test and release Promptkit.
+
+Each module should contain:
+
+- a versioned backend manifest containing the backend ID, OpenAI-compatible
+  base endpoint, API-key environment-variable name, concurrency policy, queue
+  capacity, and any backend-wide extra parameters;
+- the backend's existing profile YAML assets, retaining their current IDs and
+  behavior;
+- a minimal public package that exposes the embedded, read-only catalog as an
+  `fs.FS` together with the stable root needed to read it;
+- focused package documentation describing the asset contract and release
+  responsibility;
+- validation tests appropriate to data that the module can validate without
+  importing Promptkit; and
+- an independent semantic version and release history.
+
+The exported Go surface should remain limited to access to immutable embedded
+assets. The modules should use only the Go standard library at runtime. They
+must not import Promptkit, duplicate Promptkit's domain types, perform global
+registration in `init`, expose mutable registries, read process environment,
+or contain credentials.
+
+The backend manifest format should include an explicit schema version.
+Promptkit should strictly reject unsupported versions, unknown fields,
+malformed values, and files outside the catalog contract. A structured
+standard-library format such as JSON is preferred for the backend manifest;
+the existing profile YAML format remains unchanged and continues to be parsed
+by Promptkit.
+
+## Promptkit Integration
+
+Add an internal catalog adapter that accepts an external catalog's `fs.FS` and
+root, decodes its backend manifest, exposes its profile assets through the
+existing profile repository boundary, and converts validated data into the
+existing internal domain types.
+
+Engine construction should explicitly assemble the imported catalogs. Catalog
+packages must not register themselves through package initialization or other
+process-global mutable state. The internal backend registry should receive the
+assembled built-ins separately from engine-scoped consumer additions so that
+the distinction between maintained and consumer-owned IDs remains clear.
+
+The assembled catalog must be validated before use. Validation should cover at
+least:
+
+- a supported catalog-manifest schema version;
+- all existing backend invariants, including endpoint, environment-variable,
+  concurrency, queue-capacity, and extra-parameter rules;
+- duplicate backend IDs across imported catalogs;
+- duplicate profile IDs within or across imported catalogs;
+- profile references to the backend owned by their catalog;
+- profile-format and inheritance validity under Promptkit's existing rules;
+  and
+- absence of raw API keys or other secret material.
+
+Invalid maintained catalog data should fail engine construction through the
+existing configuration-error boundary with enough source context for a
+maintainer to identify the catalog, while avoiding content or credential
+leakage.
+
+Imported profile repositories remain the lowest-precedence profile source.
+Configured, fallback, and in-memory consumer sources retain their existing
+precedence and override behavior. Consumer backend registration continues to
+accept only new IDs and cannot replace a maintained built-in.
+
+## Compatibility Requirements
+
+The extraction must preserve consumer-visible behavior. In particular:
+
+- `BackendOpenRouter` and `BackendRakestrawHome` retain their current names and
+  values in the root Promptkit package;
+- every current built-in profile retains its ID, model, backend, settings,
+  output behavior, and inheritance behavior;
+- each backend retains its endpoint, API-key environment variable,
+  concurrency limit, queue capacity, and extra parameters;
+- built-ins remain available without additional consumer options;
+- consumer-defined backend and profile behavior remains unchanged;
+- preparation, execution, errors, capacity, and credential handling remain
+  Promptkit responsibilities; and
+- no new catalog implementation types appear in Promptkit's public API.
+
+Promptkit should record the complete pre-extraction built-in catalog in a
+reviewable compatibility fixture or equivalent test-owned snapshot before
+switching data sources. Integration tests must prove that the imported
+catalogs reproduce it exactly and that source precedence and reserved-ID
+behavior remain unchanged.
+
+## Migration Safety Requirements
+
+The migration must not publish a Promptkit state with either zero runtime
+owners or two overlaid runtime owners for maintained catalog data. Every
+external module version selected in `go.mod` must already be published and
+resolvable through ordinary Go module tooling. The frozen pre-extraction
+compatibility baseline remains test-owned migration evidence rather than a
+runtime fallback.
+
+At cutover, Promptkit must use only the imported modules for runtime catalog
+assembly and must remove its embedded profiles and hard-coded backend
+definitions in the same accepted source state. That source state must pass the
+ordinary and race-enabled test suites, static analysis, builds, maintained
+examples, compatibility checks, and repository-hygiene validation. The final
+tree has one authoritative runtime data owner for each built-in catalog.
+
+## Catalog Version And Release Policy
+
+Promptkit's `go.mod` should pin catalog versions that have passed Promptkit's
+full integration suite. Catalog dependencies should not use local replacement
+directives or require a committed Go workspace. The complete module graph and
+checksums remain the reproducible identity of a consumer build.
+
+Catalog repositories should adopt an explicit compatibility policy before
+their first release:
+
+- adding a new profile is ordinarily backward compatible;
+- stable profile IDs should not silently be reassigned to unrelated models;
+- metadata corrections should preserve the documented meaning of an ID;
+- removals should account for consumer reliance and upstream model
+  deprecation, with notice or an appropriate catalog version boundary when
+  practicable; and
+- manifest-schema changes must remain compatible with the Promptkit versions
+  that consume that module version.
+
+Go module version selection may allow an advanced consumer to select a newer
+compatible transitive catalog release. Promptkit guarantees only the versions
+it pins and tests. This possibility must not require a new Promptkit API or
+weaken the catalog compatibility policy.
+
+## Documentation And Release Impact
+
+When the extraction lands, update current-state documentation to reflect that
+Promptkit imports rather than embeds its maintained catalog data. The framework
+format reference remains the canonical consumer owner of built-in backend and
+profile behavior. Internal architecture and component documentation should
+describe the external asset boundary without duplicating the catalog's
+contents.
+
+The Promptkit release adopting the modules should include a short consumer
+release note explaining the new dependency boundary and confirming that no
+configuration migration is required. Catalog repository documentation should
+link consumers back to Promptkit for runtime behavior and configuration rather
+than becoming a parallel Promptkit manual.
+
+## Non-Goals
+
+This feature does not:
+
+- change Promptkit's root public API or require consumers to import catalogs;
+- add runtime plugin discovery, network catalog downloads, or dynamic module
+  loading;
+- allow a consumer to replace a maintained backend ID;
+- move backend execution, transport, credential, or concurrency logic out of
+  Promptkit;
+- define a general public catalog-authoring SDK;
+- automatically select untested catalog releases at runtime;
+- guarantee that an upstream provider continues to serve every cataloged
+  model; or
+- extract consumer-defined backends or profiles from their owning
+  applications.
+
+## Completion Criteria
+
+The feature is complete when:
+
+- both external catalog repositories have released usable, tested modules;
+- Promptkit imports and explicitly assembles both catalogs through a private
+  adapter;
+- compatibility tests prove parity with every previously built-in backend and
+  profile;
+- ordinary consumer construction and configuration require no changes;
+- consumer overlays and reserved backend IDs retain their behavior;
+- Promptkit contains no duplicate embedded catalog assets or hard-coded
+  definitions for the extracted backends;
+- all required Promptkit validation succeeds without a workspace, replacement
+  directive, real credential, or provider network call; and
+- current-state and release documentation accurately describe the resulting
+  dependency boundary.