Verify scoped collection behavior and complete roadmap

This commit is contained in:
2026-10-17 06:39:24 +00:00
parent dfd8da843e
commit 0221cd4610
3 changed files with 138 additions and 36 deletions

View File

@@ -1,10 +1,10 @@
# Configuration Collections, Namespaces, And Scoped Lookup # Configuration Collections, Namespaces, And Scoped Lookup
Status: accepted feature roadmap; not implemented in PromptKit. This document Status: implementation complete; roadmap retirement remains for normal release
consolidates the namespace roadmap and selected collection proposal 3. The preparation. This document consolidates the namespace roadmap and selected
configuration and namespace design decisions are settled. The companion collection proposal 3. The configuration and namespace design decisions are
[implementation plan](implementation.md) defines concrete API delivery choices settled. The companion [implementation plan](implementation.md) records concrete
and the staged work needed to reach this target state. API delivery choices and the completed work.
## Purpose ## Purpose
@@ -27,14 +27,14 @@ and provenance guarantees.
## Prerequisite: Adopt ConfigKit v0.8.0 ## Prerequisite: Adopt ConfigKit v0.8.0
The target selects released ConfigKit **v0.8.0** for the collection schema and The target selects released ConfigKit **v0.8.0** for the collection schema and
loading APIs. PromptKit currently selects v0.7.1. Dependency adoption and its loading APIs. At planning time PromptKit selected v0.7.1; the
verification belong to the [implementation plan](implementation.md); local [implementation plan](implementation.md) records adoption and verification of
filesystem replacements are not part of the distributed module. v0.8.0. Local filesystem replacements are not part of the distributed module.
The v0.8.0 tag was reviewed at `137769f` on 2026-10-17. It supplies input The v0.8.0 tag was reviewed at `137769f` on 2026-10-17. It supplies input
placement, relative import destinations, and shared source-version checks needed placement, relative import destinations, and shared source-version checks needed
by this design. No additional upstream extension is required. Recording this by this design. No additional upstream extension is required. The implementation
prerequisite does not itself upgrade PromptKit's dependency. plan records adoption of this prerequisite.
ConfigKit's public `docs/config.md`, ADRs 0011–0014, and v0.8.0 release notes own ConfigKit's public `docs/config.md`, ADRs 0011–0014, and v0.8.0 release notes own
the upstream contracts. The capabilities used here are: the upstream contracts. The capabilities used here are:
@@ -175,15 +175,16 @@ weather, err := engine.WithNamespaces(Namespaces{
// Handle err, then use local prompt and profile names through weather. // Handle err, then use local prompt and profile names through weather.
``` ```
These declarations do not exist yet. Empty scope fields mean no scope for that The concrete declarations are recorded in the
resource type. Keep name qualification shared across the view and engine; do not [implementation plan](implementation.md). Empty scope fields mean no scope for
duplicate selection logic in each operation. Initially omit a one-string shorthand that resource type. Keep name qualification shared across the view and engine;
that implicitly scopes all resource types. do not duplicate selection logic in each operation. Omit a one-string
shorthand that implicitly scopes all resource types.
The view would share the engine's captured configuration, clients, and execution The implemented views share the engine's captured configuration, clients, and
machinery. Selecting a namespace would not rebuild the engine or duplicate its execution machinery. Selecting a namespace does not rebuild the engine or
backend capacity accounting. Consumers could use different views concurrently duplicate its backend capacity accounting. Consumers can use different views
without changing a shared current-namespace setting. concurrently without changing a shared current-namespace setting.
The accepted resolution rules, independent of the final scope API, are: The accepted resolution rules, independent of the final scope API, are:
@@ -680,10 +681,10 @@ automatically relocate schema or artifact stores or rewrite references when a
collection is renamed. collection is renamed.
Retain the accepted removal of mandatory catalog modules and built-in profiles. Retain the accepted removal of mandatory catalog modules and built-in profiles.
This is a breaking configuration/API change under the This was a breaking configuration/API change under the
[release policy](../policy/release.md); implementation must include consumer [release policy](../policy/release.md); the implementation includes consumer
migration guidance and update current contracts when it lands. The proposal migration guidance and updates current contracts. The completed work changes
does not change today's supported configuration or API. the supported configuration and API as described here.
Existing unscoped exact lookup remains available for IDs valid under the new Existing unscoped exact lookup remains available for IDs valid under the new
grammar. Compatibility with arbitrary slash-containing resource IDs is grammar. Compatibility with arbitrary slash-containing resource IDs is
@@ -703,8 +704,8 @@ intentionally broken. The migration must:
catalog adoption under the [release policy](../policy/release.md). Its catalog catalog adoption under the [release policy](../policy/release.md). Its catalog
profile-data exception does not cover this general format/API behavior change; profile-data exception does not cover this general format/API behavior change;
deliberately retire its built-in-profile preservation promise when the deliberately retire its built-in-profile preservation promise when the
accepted removal lands. Current policy and format documents remain unchanged accepted removal landed. Current policy and format documents now reflect the
until implementation. implemented behavior.
Renaming a backend changes its identity; duplicating it under another ID can Renaming a backend changes its identity; duplicating it under another ID can
separate capacity accounting. Do not use blanket aliases or duplicate catalog separate capacity accounting. Do not use blanket aliases or duplicate catalog
@@ -728,11 +729,11 @@ can be reconsidered for concrete needs; none blocks implementation.
## Target Acceptance ## Target Acceptance
The target includes the collection schema and final configuration adapter, The completed target includes the collection schema and final configuration
catalog and built-in removal, ID-only prompt selection, immutable scoped views, adapter, catalog and built-in removal, ID-only prompt selection, immutable
integrated filtering, and collection inventory. Runtime indexes use full scoped views, integrated filtering, and collection inventory. Runtime indexes
identities after ConfigKit handoff. Current contracts and consumer migration use full identities after ConfigKit handoff. Current contracts and consumer
guidance describe the implemented behavior when it lands. The migration guidance describe the implemented behavior. The
[implementation plan](implementation.md) owns sequencing, concrete public shapes, [implementation plan](implementation.md) owns sequencing, concrete public shapes,
documentation tasks, and focused offline validation under the documentation tasks, and focused offline validation under the
[testing policy](../policy/testing.md). [testing policy](../policy/testing.md).
@@ -798,15 +799,17 @@ review of release v0.8.0 at `137769f` passed maintained consumer examples and
focused public API probes for sequence assembly, duplicate rejection, whole-list focused public API probes for sequence assembly, duplicate rejection, whole-list
replacement, inheritance limits, header consumption, and final identity validation replacement, inheritance limits, header consumption, and final identity validation
boundaries. The latter review informed the decision to retain mappings. These boundaries. The latter review informed the decision to retain mappings. These
checks establish upstream behavior, not an implemented PromptKit feature. checks establish the upstream behavior relied on by the implemented PromptKit
feature.
## Delivery Reference ## Delivery Reference
No substantive design question remains. ConfigKit v0.8.0 is the selected No substantive design question remains. ConfigKit v0.8.0 is the selected
dependency. The [implementation plan](implementation.md) specifies helper dependency. The [implementation plan](implementation.md) records helper
signatures, inventory and optional-label representations, identifier migration signatures, inventory and optional-label representations, identifier migration
mappings, and consumer migration work. Its delivery choices implement these mappings, and completed consumer migration work. Its delivery choices implement
accepted contracts without adding speculative options or reopening workflows. these accepted contracts without adding speculative options or reopening
workflows.
## Consolidation And Retired Alternatives ## Consolidation And Retired Alternatives
@@ -838,5 +841,6 @@ part of that request is a prerequisite here.
The accepted configuration choices remain: optional main declarations and The accepted configuration choices remain: optional main declarations and
if-present checks; source-root `version` reserved without banning that resource if-present checks; source-root `version` reserved without banning that resource
name; removal of request-version machinery; current ConfigKit inheritance; and name; removal of request-version machinery; current ConfigKit inheritance; and
flexible source assembly with a small convenience API. This consolidation does flexible source assembly with a small convenience API. This consolidation
not change those decisions or implement the feature. records those decisions and their implementation scope; current behavior is
maintained in the canonical format and consumer documents.

View File

@@ -1,6 +1,7 @@
# Configuration Collections Implementation Plan # Configuration Collections Implementation Plan
Status: ready for implementation; no feature changes are implemented by this plan. Status: implementation complete; roadmap retirement remains for normal release
preparation.
Implement the accepted [feature roadmap](configuration-collections.md). That Implement the accepted [feature roadmap](configuration-collections.md). That
document owns intent, scope, and selected policy; this document owns API delivery document owns intent, scope, and selected policy; this document owns API delivery
@@ -828,6 +829,20 @@ Scope: focused integration gaps and final regression validation, not new feature
release preparation. Do not publish, wait for CI, probe providers, or introduce release preparation. Do not publish, wait for CI, probe providers, or introduce
a catalog-release dependency to finish implementation. a catalog-release dependency to finish implementation.
Completion: acceptance coverage was traced across public collection assembly,
scoped execution and inspection, full-ID listings and inventories, exact stored
defaults, optional prompt labels, prepared selection capture, source headers and
origins, final-value validation, namespaced redaction, and retained effective
configuration shape. Added a controlled concurrent regression test proving an
ordinary call and a differently scoped call share one collection backend's
capacity. Audited current source, tests, and docs for obsolete selectors,
automatic catalogs, built-in profile promises, and local module replacements;
remaining version references are descriptive labels or migration guidance.
Marked the feature roadmap complete; normal release preparation owns retirement.
Validation passed: `GOWORK=off go test -race ./...`, `GOWORK=off go vet ./...`,
`GOWORK=off python3 scripts/check-examples.py`, `gofmt`, local Markdown link
checks, and `git diff --check`.
## Open Questions ## Open Questions
None required to begin implementation. The feature roadmap's policy decisions None required to begin implementation. The feature roadmap's policy decisions

View File

@@ -59,6 +59,89 @@ func TestSmokeTestSharesBackendCapacityWithRunAndReleasesAfterCancellation(t *te
} }
} }
func TestScopedCollectionRunsShareBackendCapacity(t *testing.T) {
release := make(chan struct{})
client := newCapacityGateClient(release, 2)
var inputs []configkit.Input
for _, namespace := range []string{"north", "south"} {
input, err := promptkit.CollectionInput(namespace, configkit.Mapping(namespace+" resources", map[string]any{
"profiles": map[string]any{
"shared": map[string]any{"backend": "capacity/backend", "model": namespace + "-model"},
},
"prompts": map[string]any{
"shared": map[string]any{
"default_profile": namespace + "/shared",
"messages": []any{map[string]any{"role": "user", "content": "Hello"}},
"output": map[string]any{"format": "text", "validation_mode": "none"},
},
},
}))
if err != nil {
t.Fatal(err)
}
input.Imports = false
inputs = append(inputs, input)
}
capacityInput, err := promptkit.CollectionInput("capacity", configkit.Mapping("shared capacity", map[string]any{
"backends": map[string]any{
"backend": map[string]any{
"endpoint": "https://example.invalid/v1",
"concurrency_limit": 1,
"queue_capacity": 0,
},
},
}))
if err != nil {
t.Fatal(err)
}
capacityInput.Imports = false
inputs = append(inputs, capacityInput)
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
Underlays: []configkit.Underlay{{Inputs: inputs}},
}}, promptkit.WithLLMClient(client))
if err != nil {
t.Fatalf("construct collection engine: %v", err)
}
south, err := engine.WithNamespaces(promptkit.Namespaces{Prompt: "south", Profile: "south"})
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
results := make(chan capacityRunResult, 1)
go runCapacityRequest(engine, ctx, promptkit.RunRequest{
PromptID: "north/shared",
}, results)
select {
case <-client.started:
case <-time.After(5 * time.Second):
cancel()
t.Fatal("ordinary Run did not reach the shared backend")
}
scopedContext, cancelScoped := context.WithCancel(context.Background())
defer cancelScoped()
if _, err := south.Run(scopedContext, promptkit.RunRequest{
PromptID: "shared", ProfileID: "shared",
}); !errors.Is(err, promptkit.ErrCapacityExceeded) {
cancel()
cancelScoped()
t.Fatalf("concurrent scoped Run error = %v, want ErrCapacityExceeded", err)
}
cancel()
outcome := awaitCapacityRun(t, results)
if outcome.result != nil || !errors.Is(outcome.err, context.Canceled) {
t.Fatalf("ordinary Run outcome=(%+v, %v), want context cancellation", outcome.result, outcome.err)
}
_, peak, calls := client.snapshot()
if calls != 1 || peak != 1 {
t.Fatalf("shared backend calls=%d peak=%d, want one admitted call", calls, peak)
}
}
type failOnceSmokeClient struct { type failOnceSmokeClient struct {
mu sync.Mutex mu sync.Mutex
calls int calls int