eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add scoped identity listings and backend inventory

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-17 06:22:22 +00:00
parent 15f79ce7b1
commit c9b623b374
10 changed files with 438 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs/consumers/pkg-promptkit.md
View File

@@ -148,9 +148,13 @@ does not watch or reload sources automatically.
## Discover Available Profiles And Prompts
Use [`Engine.ListProfiles`](../../listing.go) and
[`Engine.ListPrompts`](../../listing.go) to discover exact identities from the
engine's final captured configuration, then inspect a selected prompt:
Use [`Engine.ListProfiles`](../../listing.go),
[`Engine.ListPrompts`](../../listing.go), and
[`Engine.ListBackends`](../../listing.go) to discover exact identities from the
engine's final captured configuration. A scoped view filters by its respective
namespace by default; pass `NamespaceNamed`, `NamespaceUnqualified`, or
`NamespaceAll` to override that listing filter without changing execution
scope. Returned IDs remain full identities and can be used for exact selection:
```go
profiles, err := engine.ListProfiles(ctx)

2
docs/internal/overview.md
View File

@@ -16,7 +16,7 @@ contributor workflow and validation.
| `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, programmatic profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
| `examples/go-library/observe` | Demonstrates offline built-in HTTP retry and synchronous metadata observation across structured proposal and assessment runs using embedded sources, two profiles on one finite backend, and an application decision. It is not a public library package. | [Example program](../../examples/go-library/observe/main.go) |
| `internal/backend` | Holds a defensive, immutable lookup of final backend definitions, including diagnostic metadata, plus capacity policies validated by the configuration adapter. | [Backend registry](../../internal/backend/registry.go) |
| `internal/configuration` | Loads keyed backend, profile, and prompt configuration through ConfigKit; records effective field presence, validates final values including optional diagnostic metadata, captures indexed prompt definitions, referenced message content, compiled templates, input metadata snapshots, sorted identity-only profile and prompt inventories, backend and presence-aware profile snapshots, and an explicit read-only inspection view. | [Typed loader](../../internal/configuration/load.go), [identity listing](../../internal/configuration/listing.go), [internal sources](sources.md#typed-configuration-loading) |
| `internal/configuration` | Loads keyed backend, profile, and prompt configuration through ConfigKit; records effective field presence, validates final values including optional diagnostic metadata, captures indexed prompt definitions, referenced message content, compiled templates, input metadata snapshots, sorted identity-only backend, profile, and prompt inventories, backend and presence-aware profile snapshots, and an explicit read-only inspection view. | [Typed loader](../../internal/configuration/load.go), [identity listing](../../internal/configuration/listing.go), [internal sources](sources.md#typed-configuration-loading) |
| `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) |
| `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, requirement inspection, profiles, execution targets, rendering, generation, validation, and source-neutral smoke diagnostics, and owns source-neutral invariants for shared execution settings, OpenAI-compatible base endpoints, session identifiers, output contracts, and resource-identity syntax and qualification. Source parsing, required fields, other source-specific normalization, defaulting, and boundary-specific error classification remain with their callers. | [Domain declarations](../../internal/domain/domain.go), [requirement values](../../internal/domain/requirements.go), [endpoint invariant](../../internal/domain/endpoint.go) |
| `internal/engine` | Assembles caller-selected configuration into one snapshot, then constructs shared capacity, rendering, validation, generation, repair, the ordinary runner, and the smoke tester for each engine. Immutable namespace views share those collaborators and qualify caller-supplied execution, inspection, and diagnostic selections without rewriting captured references. Built-in engines provide the diagnostic HTTP collaborator; injected-client engines do not. It delegates identity listing directly to the captured configuration inventory rather than the runner. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#typed-configuration-loading), [internal runner](runner.md) |

12
docs/roadmap/implementation.md
View File

@@ -681,6 +681,18 @@ names, full-ID round trips (and unqualified selection through the original
engine), invalid filters, ownership, and canceled contexts. Use a compact matrix
across resource types, not three copies of all tests. Run configuration/root tests.
Completion: prompt and profile listings now accept the documented optional
namespace filters, and backend identities are captured and listed through the
same path. Default filters use the view's per-resource scope, explicit filters
replace it, results retain full IDs and detached prompt labels, and ordinary
no-filter calls still return the full inventory. Contract tests cover three
resource types, exact namespace boundaries, round trips, invalid filters,
ownership, and cancellation. Validation passed: `GOWORK=off go test ./...`,
focused `GOWORK=off go test -race . -run
'TestScopedIdentityListingsApplyNamespaceFilters|TestListIdentitiesAreOwnedAndSafeForConcurrentUse|TestListIdentitiesPreserveContextErrorsWithoutPartialResults' -count=1`,
`GOWORK=off go vet ./...`, `GOWORK=off python3 scripts/check-examples.py`,
and `git diff --check`.
## Stage 15: Expose Namespace And Effective Collection Inventories
Scope: the two discovery APIs and construction-time inventory capture.

3
engine.go
View File

@@ -88,7 +88,8 @@ var (
// [Engine.InspectConfiguration],
// [Engine.InspectPrompt],
// [Engine.InspectRequirements], [Engine.InspectProfile], [Engine.Prepare], [Engine.PrepareExecution],
// [Engine.Run], [Engine.RunPrepared], and [Engine.SmokeTest]. Each Engine owns
// [Engine.Run], [Engine.RunPrepared], [Engine.SmokeTest], [Engine.ListProfiles],
// [Engine.ListPrompts], and [Engine.ListBackends]. Each Engine owns
// independent backend-capacity pools that coordinate Run, RunPrepared, and
// inference SmokeTest admission and model generation. Injected collaborators
// may still be invoked concurrently across different backend pools or for

11
internal/configuration/listing.go
View File

@@ -17,6 +17,17 @@ func (state *loaded) ListProfiles(ctx context.Context) ([]string, error) {
return profiles, nil
}
func (state *loaded) ListBackends(ctx context.Context) ([]string, error) {
if err := ctx.Err(); err != nil {
return nil, err
}
backends := append([]string(nil), state.backendIDs...)
if err := ctx.Err(); err != nil {
return nil, err
}
return backends, nil
}
func (state *loaded) ListPrompts(ctx context.Context) ([]domain.PromptIdentity, error) {
if err := ctx.Err(); err != nil {
return nil, err

26
internal/configuration/listing_test.go
View File

@@ -25,6 +25,14 @@ func TestLoadedListsSortedEffectiveIdentities(t *testing.T) {
if err != nil {
t.Fatal(err)
}
backends, err := state.ListBackends(context.Background())
if err != nil {
t.Fatal(err)
}
wantBackends := []string{"local"}
if !reflect.DeepEqual(backends, wantBackends) {
t.Fatalf("backend IDs = %#v, want %#v", backends, wantBackends)
}
profiles, err := state.ListProfiles(context.Background())
if err != nil {
@@ -51,6 +59,7 @@ func TestLoadedListsSortedEffectiveIdentities(t *testing.T) {
func TestLoadedListsAreIndependent(t *testing.T) {
state, err := Load(mapping("identities", map[string]any{
"backends": map[string]any{"backend": map[string]any{"endpoint": "https://example.test/v1"}},
"profiles": map[string]any{"profile": map[string]any{}},
"prompts": map[string]any{"example": listingPromptDefinition()},
}), nil)
@@ -64,6 +73,11 @@ func TestLoadedListsAreIndependent(t *testing.T) {
}
profiles = append(profiles, "caller profile")
profiles[0] = "changed profile"
backends, err := state.ListBackends(context.Background())
if err != nil {
t.Fatal(err)
}
backends[0] = "changed backend"
prompts, err := state.ListPrompts(context.Background())
if err != nil {
@@ -79,6 +93,10 @@ func TestLoadedListsAreIndependent(t *testing.T) {
if !reflect.DeepEqual(profiles, []string{"profile"}) {
t.Fatalf("caller mutation changed profile inventory: %#v", profiles)
}
backends, err = state.ListBackends(context.Background())
if err != nil || !reflect.DeepEqual(backends, []string{"backend"}) {
t.Fatalf("caller mutation changed backend inventory: %#v, %v", backends, err)
}
prompts, err = state.ListPrompts(context.Background())
if err != nil {
t.Fatal(err)
@@ -98,6 +116,10 @@ func TestLoadedListsEmptyInventory(t *testing.T) {
if err != nil || len(profiles) != 0 {
t.Fatalf("empty profile listing = %#v, %v", profiles, err)
}
backends, err := state.ListBackends(context.Background())
if err != nil || len(backends) != 0 {
t.Fatalf("empty backend listing = %#v, %v", backends, err)
}
prompts, err := state.ListPrompts(context.Background())
if err != nil || len(prompts) != 0 {
t.Fatalf("empty prompt listing = %#v, %v", prompts, err)
@@ -119,6 +141,10 @@ func TestLoadedListsHonorCanceledContext(t *testing.T) {
if !errors.Is(err, context.Canceled) || len(profiles) != 0 {
t.Fatalf("canceled profile listing = %#v, %v", profiles, err)
}
backends, err := state.ListBackends(ctx)
if !errors.Is(err, context.Canceled) || len(backends) != 0 {
t.Fatalf("canceled backend listing = %#v, %v", backends, err)
}
prompts, err := state.ListPrompts(ctx)
if !errors.Is(err, context.Canceled) || len(prompts) != 0 {
t.Fatalf("canceled prompt listing = %#v, %v", prompts, err)

2
internal/configuration/load.go
View File

@@ -124,6 +124,7 @@ type loaded struct {
prompts map[string]*promptEntry
registry *backend.Registry
indexedProfiles map[string]domain.ExecutionProfile
backendIDs []string
profileIDs []string
promptIDs []domain.PromptIdentity
}
@@ -260,6 +261,7 @@ func load(main configkit.Input, underlays []configkit.Underlay) (*loaded, error)
if err := state.buildExecutionProfiles(); err != nil {
return nil, fmt.Errorf("load configuration: %w", err)
}
state.backendIDs = sortedKeys(indexedBackends(&result.Config))
state.profileIDs = sortedKeys(state.indexedProfiles)
for _, id := range sortedKeys(state.prompts) {
state.promptIDs = append(state.promptIDs, domain.PromptIdentity{ID: id, Version: stringPointerValue(state.prompts[id].definition.Version)})

112
internal/engine/engine.go
View File

@@ -94,10 +94,27 @@ type resourceNamespaces struct {
}
type identityInventory interface {
ListBackends(context.Context) ([]string, error)
ListProfiles(context.Context) ([]string, error)
ListPrompts(context.Context) ([]domain.PromptIdentity, error)
}
// ListingFilterMode selects namespace membership for one identity listing.
type ListingFilterMode uint8
const (
ListingDefault ListingFilterMode = iota
ListingNamed
ListingUnqualified
ListingAll
)
// ListingFilter optionally overrides the namespace selected by an engine view.
type ListingFilter struct {
Mode ListingFilterMode
Name string
}
// New assembles an engine from facade-selected configuration and collaborators.
func New(cfg Config, options Options) (*Engine, error) {
defaultRetries, err := domain.ResolveTransientRetries(2, cfg.TransientRetries)
@@ -231,12 +248,99 @@ func (e *Engine) InspectProfile(ctx context.Context, id string) (*domain.Profile
return e.runner.InspectProfile(ctx, id)
}
func (e *Engine) ListProfiles(ctx context.Context) ([]string, error) {
return e.listing.ListProfiles(ctx)
func (e *Engine) ListBackends(ctx context.Context, filters ...ListingFilter) ([]string, error) {
return filterIdentityList(ctx, filters, e.namespaces.backend, e.listing.ListBackends)
}
func (e *Engine) ListPrompts(ctx context.Context) ([]domain.PromptIdentity, error) {
return e.listing.ListPrompts(ctx)
func (e *Engine) ListProfiles(ctx context.Context, filters ...ListingFilter) ([]string, error) {
return filterIdentityList(ctx, filters, e.namespaces.profile, e.listing.ListProfiles)
}
func (e *Engine) ListPrompts(ctx context.Context, filters ...ListingFilter) ([]domain.PromptIdentity, error) {
mode, namespace, err := resolveListingFilter(filters, e.namespaces.prompt)
if err != nil {
return nil, err
}
prompts, err := e.listing.ListPrompts(ctx)
if err != nil {
return nil, err
}
var filtered []domain.PromptIdentity
for _, prompt := range prompts {
if matchesListingNamespace(prompt.ID, mode, namespace) {
filtered = append(filtered, prompt)
}
}
if err := ctx.Err(); err != nil {
return nil, err
}
return filtered, nil
}
func filterIdentityList(
ctx context.Context,
filters []ListingFilter,
scope string,
list func(context.Context) ([]string, error),
) ([]string, error) {
mode, namespace, err := resolveListingFilter(filters, scope)
if err != nil {
return nil, err
}
ids, err := list(ctx)
if err != nil {
return nil, err
}
var filtered []string
for _, id := range ids {
if matchesListingNamespace(id, mode, namespace) {
filtered = append(filtered, id)
}
}
if err := ctx.Err(); err != nil {
return nil, err
}
return filtered, nil
}
func resolveListingFilter(filters []ListingFilter, scope string) (ListingFilterMode, string, error) {
if len(filters) > 1 {
return 0, "", fmt.Errorf("%w: at most one listing filter is allowed", usecase.ErrInvalidRequest)
}
filter := ListingFilter{}
if len(filters) == 1 {
filter = filters[0]
}
switch filter.Mode {
case ListingDefault:
if scope == "" {
return ListingAll, "", nil
}
return ListingNamed, scope, nil
case ListingNamed:
if !domain.IsValidResourceNamespace(filter.Name) {
return 0, "", fmt.Errorf("%w: invalid listing namespace", usecase.ErrInvalidRequest)
}
return ListingNamed, filter.Name, nil
case ListingUnqualified, ListingAll:
return filter.Mode, "", nil
default:
return 0, "", fmt.Errorf("%w: unknown listing filter mode", usecase.ErrInvalidRequest)
}
}
func matchesListingNamespace(id string, mode ListingFilterMode, namespace string) bool {
switch mode {
case ListingNamed:
idNamespace, _, qualified := strings.Cut(id, "/")
return qualified && idNamespace == namespace
case ListingUnqualified:
return !strings.Contains(id, "/")
case ListingAll:
return true
default:
return false
}
}
func (e *Engine) Prepare(ctx context.Context, request domain.RunRequest) (*domain.PreparedRun, error) {
request, err := e.qualifyRunRequest(request)

99
listing.go
View File

@@ -3,8 +3,32 @@ package promptkit
import (
"context"
"fmt"
internalengine "gitea.maximumdirect.net/eric/promptkit/internal/engine"
)
// NamespaceFilterMode selects namespace membership for an identity listing.
type NamespaceFilterMode uint8
const (
// NamespaceDefault uses the engine view's scope, or all IDs if it is unset.
NamespaceDefault NamespaceFilterMode = iota
// NamespaceNamed selects IDs whose exact namespace matches Name.
NamespaceNamed
// NamespaceUnqualified selects IDs without a namespace.
NamespaceUnqualified
// NamespaceAll selects IDs from every namespace, overriding the view scope.
NamespaceAll
)
// NamespaceFilter optionally overrides the namespace used for one listing.
// NamespaceNamed requires a nonempty valid namespace in Name; other modes
// ignore Name.
type NamespaceFilter struct {
Mode NamespaceFilterMode
Name string
}
// PromptIdentity identifies one prompt ID and its optional descriptive version
// label in the effective configuration captured when an Engine is constructed.
// A nil Version means no label was declared; a non-nil empty string is an
@@ -17,43 +41,57 @@ type PromptIdentity struct {
// ListProfiles returns profile IDs in the effective configuration captured
// when the engine was constructed. Maintained and consumer profiles are
// included; an inherited profile retains its own ID. IDs are preserved exactly
// and sorted lexically. The returned slice is caller-owned and safe to mutate
// while the engine is used concurrently. Empty inventories succeed.
// and sorted lexically. An omitted filter or NamespaceDefault uses the view's
// profile namespace, or all profiles when unset. NamespaceNamed selects the
// exact named namespace, NamespaceUnqualified selects IDs without a slash, and
// NamespaceAll selects every profile. An explicit filter replaces the view's
// default for this call. At most one filter is accepted.
//
// The returned slice is caller-owned and safe to mutate while the engine is
// used concurrently. Empty inventories succeed.
//
// Listing reports membership, not execution readiness. It does not reload
// configuration or perform resource, credential, provider, or execution work.
// Profile IDs are application data and may be sensitive. A nil or zero-value
// engine returns an error matching ErrInvalidConfig. Cancellation or deadline
// expiration returns the context error and no partial result.
func (e *Engine) ListProfiles(ctx context.Context) ([]string, error) {
// expiration returns the context error and no partial result. Multiple
// filters, an unknown mode, or an invalid named namespace returns an error
// matching ErrInvalidRequest.
func (e *Engine) ListProfiles(ctx context.Context, filters ...NamespaceFilter) ([]string, error) {
if e == nil || e.internal == nil {
return nil, fmt.Errorf("%w: engine is nil", ErrInvalidConfig)
}
profiles, err := e.internal.ListProfiles(ctx)
profiles, err := e.internal.ListProfiles(ctx, internalListingFilters(filters)...)
if err != nil {
return nil, mapPublicError(err)
}
return profiles, nil
}
// ListPrompts returns every prompt ID in the effective configuration captured
// when the engine was constructed, sorted lexically by ID. Each identity may
// include its descriptive version label. The returned slice and each Version
// pointer are caller-owned and safe to mutate while the engine is used
// concurrently. Empty inventories succeed.
// ListPrompts returns prompt IDs in the effective configuration captured when
// the engine was constructed, sorted lexically by ID. Each identity may include
// its descriptive version label. Its optional filter follows the same
// NamespaceDefault, NamespaceNamed, NamespaceUnqualified, and NamespaceAll
// behavior as [Engine.ListProfiles], using the prompt namespace by default.
// At most one filter is accepted.
//
// The returned slice and each Version pointer are caller-owned and safe to
// mutate while the engine is used concurrently. Empty inventories succeed.
//
// Listing reports membership, not execution readiness. It does not reload
// configuration or perform resource, credential, provider, or execution work.
// Prompt identities are application data and may be sensitive. A nil or
// zero-value engine returns an error matching ErrInvalidConfig. Cancellation
// or deadline expiration returns the context error and no partial result.
func (e *Engine) ListPrompts(ctx context.Context) ([]PromptIdentity, error) {
// Multiple filters, an unknown mode, or an invalid named namespace returns an
// error matching ErrInvalidRequest.
func (e *Engine) ListPrompts(ctx context.Context, filters ...NamespaceFilter) ([]PromptIdentity, error) {
if e == nil || e.internal == nil {
return nil, fmt.Errorf("%w: engine is nil", ErrInvalidConfig)
}
identities, err := e.internal.ListPrompts(ctx)
identities, err := e.internal.ListPrompts(ctx, internalListingFilters(filters)...)
if err != nil {
return nil, mapPublicError(err)
}
@@ -66,3 +104,40 @@ func (e *Engine) ListPrompts(ctx context.Context) ([]PromptIdentity, error) {
}
return public, nil
}
// ListBackends returns backend IDs in the effective configuration captured
// when the engine was constructed, sorted lexically by full ID. Its optional
// filter follows the same namespace behavior as [Engine.ListProfiles], using
// the backend namespace by default. At most one filter is accepted. The
// returned slice is caller-owned and safe to mutate while the engine is used
// concurrently. Empty inventories succeed.
//
// Listing reports membership, not execution readiness, and performs no
// provider or execution work. A nil or zero-value engine returns an error
// matching ErrInvalidConfig. Cancellation or deadline expiration returns the
// context error and no partial result. Multiple filters, an unknown mode, or
// an invalid named namespace returns an error matching ErrInvalidRequest.
func (e *Engine) ListBackends(ctx context.Context, filters ...NamespaceFilter) ([]string, error) {
if e == nil || e.internal == nil {
return nil, fmt.Errorf("%w: engine is nil", ErrInvalidConfig)
}
backends, err := e.internal.ListBackends(ctx, internalListingFilters(filters)...)
if err != nil {
return nil, mapPublicError(err)
}
return backends, nil
}
func internalListingFilters(filters []NamespaceFilter) []internalengine.ListingFilter {
if filters == nil {
return nil
}
internal := make([]internalengine.ListingFilter, len(filters))
for index, filter := range filters {
internal[index] = internalengine.ListingFilter{
Mode: internalengine.ListingFilterMode(filter.Mode),
Name: filter.Name,
}
}
return internal
}

182
listing_contract_test.go
View File

@@ -60,6 +60,185 @@ func TestListIdentitiesBasicContract(t *testing.T) {
}
}
func TestScopedIdentityListingsApplyNamespaceFilters(t *testing.T) {
engine := newNamespaceListingEngine(t)
scoped, err := engine.WithNamespaces(promptkit.Namespaces{
Prompt: "team", Profile: "team", Backend: "team",
})
if err != nil {
t.Fatal(err)
}
all := promptkit.NamespaceFilter{Mode: promptkit.NamespaceAll}
named := promptkit.NamespaceFilter{Mode: promptkit.NamespaceNamed, Name: "team-blue"}
unqualified := promptkit.NamespaceFilter{Mode: promptkit.NamespaceUnqualified}
defaultOverride := promptkit.NamespaceFilter{Mode: promptkit.NamespaceDefault, Name: "ignored"}
absent := promptkit.NamespaceFilter{Mode: promptkit.NamespaceNamed, Name: "missing"}
type listing struct {
name string
list func(*promptkit.Engine, ...promptkit.NamespaceFilter) ([]string, error)
all []string
team []string
blue []string
root []string
}
listings := []listing{
{name: "profiles", list: func(e *promptkit.Engine, f ...promptkit.NamespaceFilter) ([]string, error) {
return e.ListProfiles(context.Background(), f...)
}, all: []string{"root-profile", "team-blue/profile", "team/profile"}, team: []string{"team/profile"}, blue: []string{"team-blue/profile"}, root: []string{"root-profile"}},
{name: "prompts", list: func(e *promptkit.Engine, f ...promptkit.NamespaceFilter) ([]string, error) {
prompts, err := e.ListPrompts(context.Background(), f...)
ids := make([]string, len(prompts))
for index := range prompts {
ids[index] = prompts[index].ID
}
return ids, err
}, all: []string{"root-prompt", "team-blue/prompt", "team/prompt"}, team: []string{"team/prompt"}, blue: []string{"team-blue/prompt"}, root: []string{"root-prompt"}},
{name: "backends", list: func(e *promptkit.Engine, f ...promptkit.NamespaceFilter) ([]string, error) {
return e.ListBackends(context.Background(), f...)
}, all: []string{"root-backend", "team-blue/backend", "team/backend"}, team: []string{"team/backend"}, blue: []string{"team-blue/backend"}, root: []string{"root-backend"}},
}
for _, listing := range listings {
t.Run(listing.name, func(t *testing.T) {
for _, check := range []struct {
name string
engine *promptkit.Engine
filter []promptkit.NamespaceFilter
want []string
}{
{name: "scoped default", engine: scoped, want: listing.team},
{name: "explicit default override", engine: scoped, filter: []promptkit.NamespaceFilter{defaultOverride}, want: listing.team},
{name: "named override", engine: scoped, filter: []promptkit.NamespaceFilter{named}, want: listing.blue},
{name: "unqualified override", engine: scoped, filter: []promptkit.NamespaceFilter{unqualified}, want: listing.root},
{name: "all override", engine: scoped, filter: []promptkit.NamespaceFilter{all}, want: listing.all},
{name: "absent namespace", engine: scoped, filter: []promptkit.NamespaceFilter{absent}, want: []string{}},
{name: "unscoped default", engine: engine, want: listing.all},
} {
got, err := listing.list(check.engine, check.filter...)
if err != nil || (check.name == "absent namespace" && len(got) != 0) ||
(check.name != "absent namespace" && !reflect.DeepEqual(got, check.want)) {
t.Errorf("%s = %#v, %v; want %#v", check.name, got, err, check.want)
}
}
for _, invalid := range [][]promptkit.NamespaceFilter{
{{Mode: promptkit.NamespaceNamed}},
{{Mode: promptkit.NamespaceNamed, Name: "team/child"}},
{{Mode: promptkit.NamespaceFilterMode(99)}},
{{Mode: promptkit.NamespaceAll}, {Mode: promptkit.NamespaceUnqualified}},
} {
got, err := listing.list(scoped, invalid...)
if !errors.Is(err, promptkit.ErrInvalidRequest) || len(got) != 0 {
t.Errorf("invalid filter %#v = %#v, %v; want ErrInvalidRequest without results", invalid, got, err)
}
}
})
}
prompts, err := scoped.ListPrompts(context.Background(), promptkit.NamespaceFilter{Mode: promptkit.NamespaceNamed, Name: "team"})
if err != nil || len(prompts) != 1 || prompts[0].ID != "team/prompt" || prompts[0].Version == nil || *prompts[0].Version != "team-v1" {
t.Fatalf("named prompt identity = %#v, %v", prompts, err)
}
if _, err := scoped.InspectPrompt(context.Background(), prompts[0].ID); err != nil {
t.Fatalf("listed full prompt ID did not round trip: %v", err)
}
*prompts[0].Version = "caller change"
prompts[0].ID = "caller change"
prompts, err = scoped.ListPrompts(context.Background(), promptkit.NamespaceFilter{Mode: promptkit.NamespaceNamed, Name: "team"})
if err != nil || len(prompts) != 1 || prompts[0].ID != "team/prompt" || prompts[0].Version == nil || *prompts[0].Version != "team-v1" {
t.Fatalf("mutating filtered prompt inventory escaped: %#v, %v", prompts, err)
}
profiles, err := scoped.ListProfiles(context.Background(), promptkit.NamespaceFilter{Mode: promptkit.NamespaceNamed, Name: "team"})
if err != nil || len(profiles) != 1 || profiles[0] != "team/profile" {
t.Fatalf("named profile inventory = %#v, %v", profiles, err)
}
if _, err := scoped.InspectProfile(context.Background(), profiles[0]); err != nil {
t.Fatalf("listed full profile ID did not round trip: %v", err)
}
profiles[0] = "caller change"
profiles, err = scoped.ListProfiles(context.Background(), promptkit.NamespaceFilter{Mode: promptkit.NamespaceNamed, Name: "team"})
if err != nil || len(profiles) != 1 || profiles[0] != "team/profile" {
t.Fatalf("mutating filtered profile inventory escaped: %#v, %v", profiles, err)
}
backends, err := scoped.ListBackends(context.Background(), promptkit.NamespaceFilter{Mode: promptkit.NamespaceNamed, Name: "team"})
if err != nil || len(backends) != 1 || backends[0] != "team/backend" {
t.Fatalf("named backend inventory = %#v, %v", backends, err)
}
if report, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{
Kind: promptkit.SmokeTestReachability, BackendID: backends[0],
}); err != nil || report.Target.BackendID != backends[0] {
t.Fatalf("listed full backend ID did not round trip: %+v, %v", report, err)
}
backends[0] = "caller change"
backends, err = scoped.ListBackends(context.Background(), promptkit.NamespaceFilter{Mode: promptkit.NamespaceNamed, Name: "team"})
if err != nil || len(backends) != 1 || backends[0] != "team/backend" {
t.Fatalf("mutating filtered backend inventory escaped: %#v, %v", backends, err)
}
rootPrompts, err := engine.ListPrompts(context.Background(), unqualified)
if err != nil || len(rootPrompts) != 1 || rootPrompts[0].ID != "root-prompt" {
t.Fatalf("unqualified prompt listing = %#v, %v", rootPrompts, err)
}
if _, err := engine.InspectPrompt(context.Background(), rootPrompts[0].ID); err != nil {
t.Fatalf("unqualified prompt did not select through original engine: %v", err)
}
rootProfiles, err := engine.ListProfiles(context.Background(), unqualified)
if err != nil || len(rootProfiles) != 1 || rootProfiles[0] != "root-profile" {
t.Fatalf("unqualified profile listing = %#v, %v", rootProfiles, err)
}
if _, err := engine.InspectProfile(context.Background(), rootProfiles[0]); err != nil {
t.Fatalf("unqualified profile did not select through original engine: %v", err)
}
rootBackends, err := engine.ListBackends(context.Background(), unqualified)
if err != nil || len(rootBackends) != 1 || rootBackends[0] != "root-backend" {
t.Fatalf("unqualified backend listing = %#v, %v", rootBackends, err)
}
if report, err := engine.SmokeTest(context.Background(), promptkit.SmokeTestRequest{
Kind: promptkit.SmokeTestReachability, BackendID: rootBackends[0],
}); err != nil || report.Target.BackendID != rootBackends[0] {
t.Fatalf("unqualified backend did not select through original engine: %+v, %v", report, err)
}
canceled, cancel := context.WithCancel(context.Background())
cancel()
if got, err := scoped.ListProfiles(canceled); !errors.Is(err, context.Canceled) || len(got) != 0 {
t.Errorf("canceled profile list = %#v, %v", got, err)
}
if got, err := scoped.ListPrompts(canceled); !errors.Is(err, context.Canceled) || len(got) != 0 {
t.Errorf("canceled prompt list = %#v, %v", got, err)
}
if got, err := scoped.ListBackends(canceled); !errors.Is(err, context.Canceled) || len(got) != 0 {
t.Errorf("canceled backend list = %#v, %v", got, err)
}
}
func newNamespaceListingEngine(t *testing.T) *promptkit.Engine {
t.Helper()
collection := func(namespace string) map[string]any {
return map[string]any{
"backends": map[string]any{"backend": map[string]any{"endpoint": "https://example.test/v1"}},
"profiles": map[string]any{"profile": map[string]any{"backend": namespace + "/backend", "model": "example-model"}},
"prompts": map[string]any{"prompt": promptDefinitionWithVersion("team-v1", listingContractPrompt())},
}
}
input := configkit.Input{Source: configkit.Mapping("namespaced listing", map[string]any{
"collections": map[string]any{
"team": collection("team"),
"team-blue": collection("team-blue"),
},
"backends": map[string]any{"root-backend": map[string]any{"endpoint": "https://root.example.test/v1"}},
"profiles": map[string]any{"root-profile": map[string]any{"backend": "root-backend", "model": "root-model"}},
"prompts": map[string]any{"root-prompt": promptDefinitionWithVersion("root-v1", listingContractPrompt())},
})}
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{Main: &input}})
if err != nil {
var loadErr *configkit.LoadError
if errors.As(err, &loadErr) {
t.Fatalf("construct namespaced listing engine: %v: %+v", err, loadErr.Diagnostics())
}
t.Fatalf("construct namespaced listing engine: %v", err)
}
return engine
}
func TestListIdentitiesInvalidEngine(t *testing.T) {
var nilEngine *promptkit.Engine
var zeroEngine promptkit.Engine
@@ -70,6 +249,9 @@ func TestListIdentitiesInvalidEngine(t *testing.T) {
if _, err := engine.ListPrompts(context.Background()); !errors.Is(err, promptkit.ErrInvalidConfig) {
t.Errorf("ListPrompts on invalid engine: %v", err)
}
if _, err := engine.ListBackends(context.Background()); !errors.Is(err, promptkit.ErrInvalidConfig) {
t.Errorf("ListBackends on invalid engine: %v", err)
}
}
}