Verify scoped collection behavior and complete roadmap
This commit is contained in:
@@ -1,10 +1,10 @@
|
|||||||
# Configuration Collections, Namespaces, And Scoped Lookup
|
# Configuration Collections, Namespaces, And Scoped Lookup
|
||||||
|
|
||||||
Status: accepted feature roadmap; not implemented in PromptKit. This document
|
Status: implementation complete; roadmap retirement remains for normal release
|
||||||
consolidates the namespace roadmap and selected collection proposal 3. The
|
preparation. This document consolidates the namespace roadmap and selected
|
||||||
configuration and namespace design decisions are settled. The companion
|
collection proposal 3. The configuration and namespace design decisions are
|
||||||
[implementation plan](implementation.md) defines concrete API delivery choices
|
settled. The companion [implementation plan](implementation.md) records concrete
|
||||||
and the staged work needed to reach this target state.
|
API delivery choices and the completed work.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
@@ -27,14 +27,14 @@ and provenance guarantees.
|
|||||||
## Prerequisite: Adopt ConfigKit v0.8.0
|
## Prerequisite: Adopt ConfigKit v0.8.0
|
||||||
|
|
||||||
The target selects released ConfigKit **v0.8.0** for the collection schema and
|
The target selects released ConfigKit **v0.8.0** for the collection schema and
|
||||||
loading APIs. PromptKit currently selects v0.7.1. Dependency adoption and its
|
loading APIs. At planning time PromptKit selected v0.7.1; the
|
||||||
verification belong to the [implementation plan](implementation.md); local
|
[implementation plan](implementation.md) records adoption and verification of
|
||||||
filesystem replacements are not part of the distributed module.
|
v0.8.0. Local filesystem replacements are not part of the distributed module.
|
||||||
|
|
||||||
The v0.8.0 tag was reviewed at `137769f` on 2026-10-17. It supplies input
|
The v0.8.0 tag was reviewed at `137769f` on 2026-10-17. It supplies input
|
||||||
placement, relative import destinations, and shared source-version checks needed
|
placement, relative import destinations, and shared source-version checks needed
|
||||||
by this design. No additional upstream extension is required. Recording this
|
by this design. No additional upstream extension is required. The implementation
|
||||||
prerequisite does not itself upgrade PromptKit's dependency.
|
plan records adoption of this prerequisite.
|
||||||
|
|
||||||
ConfigKit's public `docs/config.md`, ADRs 0011–0014, and v0.8.0 release notes own
|
ConfigKit's public `docs/config.md`, ADRs 0011–0014, and v0.8.0 release notes own
|
||||||
the upstream contracts. The capabilities used here are:
|
the upstream contracts. The capabilities used here are:
|
||||||
@@ -175,15 +175,16 @@ weather, err := engine.WithNamespaces(Namespaces{
|
|||||||
// Handle err, then use local prompt and profile names through weather.
|
// Handle err, then use local prompt and profile names through weather.
|
||||||
```
|
```
|
||||||
|
|
||||||
These declarations do not exist yet. Empty scope fields mean no scope for that
|
The concrete declarations are recorded in the
|
||||||
resource type. Keep name qualification shared across the view and engine; do not
|
[implementation plan](implementation.md). Empty scope fields mean no scope for
|
||||||
duplicate selection logic in each operation. Initially omit a one-string shorthand
|
that resource type. Keep name qualification shared across the view and engine;
|
||||||
that implicitly scopes all resource types.
|
do not duplicate selection logic in each operation. Omit a one-string
|
||||||
|
shorthand that implicitly scopes all resource types.
|
||||||
|
|
||||||
The view would share the engine's captured configuration, clients, and execution
|
The implemented views share the engine's captured configuration, clients, and
|
||||||
machinery. Selecting a namespace would not rebuild the engine or duplicate its
|
execution machinery. Selecting a namespace does not rebuild the engine or
|
||||||
backend capacity accounting. Consumers could use different views concurrently
|
duplicate its backend capacity accounting. Consumers can use different views
|
||||||
without changing a shared current-namespace setting.
|
concurrently without changing a shared current-namespace setting.
|
||||||
|
|
||||||
The accepted resolution rules, independent of the final scope API, are:
|
The accepted resolution rules, independent of the final scope API, are:
|
||||||
|
|
||||||
@@ -680,10 +681,10 @@ automatically relocate schema or artifact stores or rewrite references when a
|
|||||||
collection is renamed.
|
collection is renamed.
|
||||||
|
|
||||||
Retain the accepted removal of mandatory catalog modules and built-in profiles.
|
Retain the accepted removal of mandatory catalog modules and built-in profiles.
|
||||||
This is a breaking configuration/API change under the
|
This was a breaking configuration/API change under the
|
||||||
[release policy](../policy/release.md); implementation must include consumer
|
[release policy](../policy/release.md); the implementation includes consumer
|
||||||
migration guidance and update current contracts when it lands. The proposal
|
migration guidance and updates current contracts. The completed work changes
|
||||||
does not change today's supported configuration or API.
|
the supported configuration and API as described here.
|
||||||
|
|
||||||
Existing unscoped exact lookup remains available for IDs valid under the new
|
Existing unscoped exact lookup remains available for IDs valid under the new
|
||||||
grammar. Compatibility with arbitrary slash-containing resource IDs is
|
grammar. Compatibility with arbitrary slash-containing resource IDs is
|
||||||
@@ -703,8 +704,8 @@ intentionally broken. The migration must:
|
|||||||
catalog adoption under the [release policy](../policy/release.md). Its catalog
|
catalog adoption under the [release policy](../policy/release.md). Its catalog
|
||||||
profile-data exception does not cover this general format/API behavior change;
|
profile-data exception does not cover this general format/API behavior change;
|
||||||
deliberately retire its built-in-profile preservation promise when the
|
deliberately retire its built-in-profile preservation promise when the
|
||||||
accepted removal lands. Current policy and format documents remain unchanged
|
accepted removal landed. Current policy and format documents now reflect the
|
||||||
until implementation.
|
implemented behavior.
|
||||||
|
|
||||||
Renaming a backend changes its identity; duplicating it under another ID can
|
Renaming a backend changes its identity; duplicating it under another ID can
|
||||||
separate capacity accounting. Do not use blanket aliases or duplicate catalog
|
separate capacity accounting. Do not use blanket aliases or duplicate catalog
|
||||||
@@ -728,11 +729,11 @@ can be reconsidered for concrete needs; none blocks implementation.
|
|||||||
|
|
||||||
## Target Acceptance
|
## Target Acceptance
|
||||||
|
|
||||||
The target includes the collection schema and final configuration adapter,
|
The completed target includes the collection schema and final configuration
|
||||||
catalog and built-in removal, ID-only prompt selection, immutable scoped views,
|
adapter, catalog and built-in removal, ID-only prompt selection, immutable
|
||||||
integrated filtering, and collection inventory. Runtime indexes use full
|
scoped views, integrated filtering, and collection inventory. Runtime indexes
|
||||||
identities after ConfigKit handoff. Current contracts and consumer migration
|
use full identities after ConfigKit handoff. Current contracts and consumer
|
||||||
guidance describe the implemented behavior when it lands. The
|
migration guidance describe the implemented behavior. The
|
||||||
[implementation plan](implementation.md) owns sequencing, concrete public shapes,
|
[implementation plan](implementation.md) owns sequencing, concrete public shapes,
|
||||||
documentation tasks, and focused offline validation under the
|
documentation tasks, and focused offline validation under the
|
||||||
[testing policy](../policy/testing.md).
|
[testing policy](../policy/testing.md).
|
||||||
@@ -798,15 +799,17 @@ review of release v0.8.0 at `137769f` passed maintained consumer examples and
|
|||||||
focused public API probes for sequence assembly, duplicate rejection, whole-list
|
focused public API probes for sequence assembly, duplicate rejection, whole-list
|
||||||
replacement, inheritance limits, header consumption, and final identity validation
|
replacement, inheritance limits, header consumption, and final identity validation
|
||||||
boundaries. The latter review informed the decision to retain mappings. These
|
boundaries. The latter review informed the decision to retain mappings. These
|
||||||
checks establish upstream behavior, not an implemented PromptKit feature.
|
checks establish the upstream behavior relied on by the implemented PromptKit
|
||||||
|
feature.
|
||||||
|
|
||||||
## Delivery Reference
|
## Delivery Reference
|
||||||
|
|
||||||
No substantive design question remains. ConfigKit v0.8.0 is the selected
|
No substantive design question remains. ConfigKit v0.8.0 is the selected
|
||||||
dependency. The [implementation plan](implementation.md) specifies helper
|
dependency. The [implementation plan](implementation.md) records helper
|
||||||
signatures, inventory and optional-label representations, identifier migration
|
signatures, inventory and optional-label representations, identifier migration
|
||||||
mappings, and consumer migration work. Its delivery choices implement these
|
mappings, and completed consumer migration work. Its delivery choices implement
|
||||||
accepted contracts without adding speculative options or reopening workflows.
|
these accepted contracts without adding speculative options or reopening
|
||||||
|
workflows.
|
||||||
|
|
||||||
## Consolidation And Retired Alternatives
|
## Consolidation And Retired Alternatives
|
||||||
|
|
||||||
@@ -838,5 +841,6 @@ part of that request is a prerequisite here.
|
|||||||
The accepted configuration choices remain: optional main declarations and
|
The accepted configuration choices remain: optional main declarations and
|
||||||
if-present checks; source-root `version` reserved without banning that resource
|
if-present checks; source-root `version` reserved without banning that resource
|
||||||
name; removal of request-version machinery; current ConfigKit inheritance; and
|
name; removal of request-version machinery; current ConfigKit inheritance; and
|
||||||
flexible source assembly with a small convenience API. This consolidation does
|
flexible source assembly with a small convenience API. This consolidation
|
||||||
not change those decisions or implement the feature.
|
records those decisions and their implementation scope; current behavior is
|
||||||
|
maintained in the canonical format and consumer documents.
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
# Configuration Collections Implementation Plan
|
# Configuration Collections Implementation Plan
|
||||||
|
|
||||||
Status: ready for implementation; no feature changes are implemented by this plan.
|
Status: implementation complete; roadmap retirement remains for normal release
|
||||||
|
preparation.
|
||||||
|
|
||||||
Implement the accepted [feature roadmap](configuration-collections.md). That
|
Implement the accepted [feature roadmap](configuration-collections.md). That
|
||||||
document owns intent, scope, and selected policy; this document owns API delivery
|
document owns intent, scope, and selected policy; this document owns API delivery
|
||||||
@@ -828,6 +829,20 @@ Scope: focused integration gaps and final regression validation, not new feature
|
|||||||
release preparation. Do not publish, wait for CI, probe providers, or introduce
|
release preparation. Do not publish, wait for CI, probe providers, or introduce
|
||||||
a catalog-release dependency to finish implementation.
|
a catalog-release dependency to finish implementation.
|
||||||
|
|
||||||
|
Completion: acceptance coverage was traced across public collection assembly,
|
||||||
|
scoped execution and inspection, full-ID listings and inventories, exact stored
|
||||||
|
defaults, optional prompt labels, prepared selection capture, source headers and
|
||||||
|
origins, final-value validation, namespaced redaction, and retained effective
|
||||||
|
configuration shape. Added a controlled concurrent regression test proving an
|
||||||
|
ordinary call and a differently scoped call share one collection backend's
|
||||||
|
capacity. Audited current source, tests, and docs for obsolete selectors,
|
||||||
|
automatic catalogs, built-in profile promises, and local module replacements;
|
||||||
|
remaining version references are descriptive labels or migration guidance.
|
||||||
|
Marked the feature roadmap complete; normal release preparation owns retirement.
|
||||||
|
Validation passed: `GOWORK=off go test -race ./...`, `GOWORK=off go vet ./...`,
|
||||||
|
`GOWORK=off python3 scripts/check-examples.py`, `gofmt`, local Markdown link
|
||||||
|
checks, and `git diff --check`.
|
||||||
|
|
||||||
## Open Questions
|
## Open Questions
|
||||||
|
|
||||||
None required to begin implementation. The feature roadmap's policy decisions
|
None required to begin implementation. The feature roadmap's policy decisions
|
||||||
|
|||||||
@@ -59,6 +59,89 @@ func TestSmokeTestSharesBackendCapacityWithRunAndReleasesAfterCancellation(t *te
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestScopedCollectionRunsShareBackendCapacity(t *testing.T) {
|
||||||
|
release := make(chan struct{})
|
||||||
|
client := newCapacityGateClient(release, 2)
|
||||||
|
var inputs []configkit.Input
|
||||||
|
for _, namespace := range []string{"north", "south"} {
|
||||||
|
input, err := promptkit.CollectionInput(namespace, configkit.Mapping(namespace+" resources", map[string]any{
|
||||||
|
"profiles": map[string]any{
|
||||||
|
"shared": map[string]any{"backend": "capacity/backend", "model": namespace + "-model"},
|
||||||
|
},
|
||||||
|
"prompts": map[string]any{
|
||||||
|
"shared": map[string]any{
|
||||||
|
"default_profile": namespace + "/shared",
|
||||||
|
"messages": []any{map[string]any{"role": "user", "content": "Hello"}},
|
||||||
|
"output": map[string]any{"format": "text", "validation_mode": "none"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
input.Imports = false
|
||||||
|
inputs = append(inputs, input)
|
||||||
|
}
|
||||||
|
capacityInput, err := promptkit.CollectionInput("capacity", configkit.Mapping("shared capacity", map[string]any{
|
||||||
|
"backends": map[string]any{
|
||||||
|
"backend": map[string]any{
|
||||||
|
"endpoint": "https://example.invalid/v1",
|
||||||
|
"concurrency_limit": 1,
|
||||||
|
"queue_capacity": 0,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
capacityInput.Imports = false
|
||||||
|
inputs = append(inputs, capacityInput)
|
||||||
|
|
||||||
|
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
|
||||||
|
Underlays: []configkit.Underlay{{Inputs: inputs}},
|
||||||
|
}}, promptkit.WithLLMClient(client))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("construct collection engine: %v", err)
|
||||||
|
}
|
||||||
|
south, err := engine.WithNamespaces(promptkit.Namespaces{Prompt: "south", Profile: "south"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
results := make(chan capacityRunResult, 1)
|
||||||
|
go runCapacityRequest(engine, ctx, promptkit.RunRequest{
|
||||||
|
PromptID: "north/shared",
|
||||||
|
}, results)
|
||||||
|
select {
|
||||||
|
case <-client.started:
|
||||||
|
case <-time.After(5 * time.Second):
|
||||||
|
cancel()
|
||||||
|
t.Fatal("ordinary Run did not reach the shared backend")
|
||||||
|
}
|
||||||
|
|
||||||
|
scopedContext, cancelScoped := context.WithCancel(context.Background())
|
||||||
|
defer cancelScoped()
|
||||||
|
if _, err := south.Run(scopedContext, promptkit.RunRequest{
|
||||||
|
PromptID: "shared", ProfileID: "shared",
|
||||||
|
}); !errors.Is(err, promptkit.ErrCapacityExceeded) {
|
||||||
|
cancel()
|
||||||
|
cancelScoped()
|
||||||
|
t.Fatalf("concurrent scoped Run error = %v, want ErrCapacityExceeded", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
cancel()
|
||||||
|
outcome := awaitCapacityRun(t, results)
|
||||||
|
if outcome.result != nil || !errors.Is(outcome.err, context.Canceled) {
|
||||||
|
t.Fatalf("ordinary Run outcome=(%+v, %v), want context cancellation", outcome.result, outcome.err)
|
||||||
|
}
|
||||||
|
_, peak, calls := client.snapshot()
|
||||||
|
if calls != 1 || peak != 1 {
|
||||||
|
t.Fatalf("shared backend calls=%d peak=%d, want one admitted call", calls, peak)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
type failOnceSmokeClient struct {
|
type failOnceSmokeClient struct {
|
||||||
mu sync.Mutex
|
mu sync.Mutex
|
||||||
calls int
|
calls int
|
||||||
|
|||||||
Reference in New Issue
Block a user