diff --git a/docs/consumers/pkg-promptkit.md b/docs/consumers/pkg-promptkit.md index 52bad63..ff9a130 100644 --- a/docs/consumers/pkg-promptkit.md +++ b/docs/consumers/pkg-promptkit.md @@ -81,14 +81,15 @@ prepared, err := scoped.Prepare(ctx, promptkit.RunRequest{ }) ``` -Ordinary execution and preparation qualify slashless caller selections with -their respective namespace. Already-qualified IDs pass through unchanged; -configured defaults and references are not rewritten. Each call replaces the -view's scope defaults, including clearing a scope with an empty field, without -changing the receiver or rebuilding its captured configuration and execution -collaborators. Namespace syntax is checked when creating the view; resource -existence is checked when selected. See [`WithNamespaces`](../../namespaces.go) -for the exact public contract. +Execution, preparation, prompt/profile inspection, requirement inspection, and +smoke diagnostics qualify slashless caller selections with their respective +namespaces. Already-qualified IDs pass through unchanged; configured defaults +and references are not rewritten. Each call replaces the view's scope defaults, +including clearing a scope with an empty field, without changing the receiver +or rebuilding its captured configuration and execution collaborators. +Namespace syntax is checked when creating the view; resource existence is +checked when selected. Configuration inspection remains global. See +[`WithNamespaces`](../../namespaces.go) for the exact public contract. ## Supply Embedded Application Defaults diff --git a/docs/internal/overview.md b/docs/internal/overview.md index 3ddb72f..1cccdc3 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -19,7 +19,7 @@ contributor workflow and validation. | `internal/configuration` | Loads keyed backend, profile, and prompt configuration through ConfigKit; records effective field presence, validates final values including optional diagnostic metadata, captures indexed prompt definitions, referenced message content, compiled templates, input metadata snapshots, sorted identity-only profile and prompt inventories, backend and presence-aware profile snapshots, and an explicit read-only inspection view. | [Typed loader](../../internal/configuration/load.go), [identity listing](../../internal/configuration/listing.go), [internal sources](sources.md#typed-configuration-loading) | | `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) | | `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, requirement inspection, profiles, execution targets, rendering, generation, validation, and source-neutral smoke diagnostics, and owns source-neutral invariants for shared execution settings, OpenAI-compatible base endpoints, session identifiers, output contracts, and resource-identity syntax and qualification. Source parsing, required fields, other source-specific normalization, defaulting, and boundary-specific error classification remain with their callers. | [Domain declarations](../../internal/domain/domain.go), [requirement values](../../internal/domain/requirements.go), [endpoint invariant](../../internal/domain/endpoint.go) | -| `internal/engine` | Assembles caller-selected configuration into one snapshot, then constructs shared capacity, rendering, validation, generation, repair, the ordinary runner, and the smoke tester for each engine. Immutable namespace views share those collaborators and qualify ordinary request selections. Built-in engines provide the diagnostic HTTP collaborator; injected-client engines do not. It delegates identity listing directly to the captured configuration inventory rather than the runner. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#typed-configuration-loading), [internal runner](runner.md) | +| `internal/engine` | Assembles caller-selected configuration into one snapshot, then constructs shared capacity, rendering, validation, generation, repair, the ordinary runner, and the smoke tester for each engine. Immutable namespace views share those collaborators and qualify caller-supplied execution, inspection, and diagnostic selections without rewriting captured references. Built-in engines provide the diagnostic HTTP collaborator; injected-client engines do not. It delegates identity listing directly to the captured configuration inventory rather than the runner. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#typed-configuration-loading), [internal runner](runner.md) | | `internal/generation` | Coordinates frozen built-in HTTP attempts and one-shot injected calls through active backend permits, including bounded status retries, cancellable backoff, ordinary attempt observations, and invocation-local evidence for diagnostics. | [Internal generation coordination](generation.md) | | `internal/defaults` | Defines application-neutral framework constants and constructs the default execution target. It contains no CLI, server, or inbound HTTP limits. | [Framework defaults](../../internal/defaults/defaults.go) | | `internal/filecatalog` | Provides deterministic YAML discovery and path helpers for operating-system filesystems and `fs.FS` sources. | [File catalog](../../internal/filecatalog/catalog.go) | diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index ff131d5..c33fd5a 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -650,6 +650,18 @@ prove shared behavior and full identities. Reuse naming-helper tests instead of duplicating the full syntax matrix in every API test. Run relevant root/engine tests and focused race checks. +Completion: prompt, profile, and requirement inspections now qualify each +explicit caller selection independently, and smoke diagnostics qualify supplied +prompt, profile, and backend IDs before applying their unchanged selection +precedence. Captured references and global configuration inspection remain +unchanged. Contract tests verify full returned identities, exact configured +prompt defaults and backend references, malformed caller IDs, and ordinary +not-found behavior. +Validation passed: `GOWORK=off go test ./...`, focused +`GOWORK=off go test -race . -run 'TestWithNamespaces' -count=1`, +`GOWORK=off go vet ./...`, `GOWORK=off python3 scripts/check-examples.py`, +and `git diff --check`. + ## Stage 14: Add Backend Listing And Namespace Filters Scope: the existing identity-inventory path, not the execution runner. diff --git a/internal/engine/engine.go b/internal/engine/engine.go index 51a8226..8a95b0d 100644 --- a/internal/engine/engine.go +++ b/internal/engine/engine.go @@ -153,7 +153,9 @@ func New(cfg Config, options Options) (*Engine, error) { } // WithNamespaces returns an immutable view sharing this engine's assembled -// collaborators and configuration snapshot. +// collaborators and configuration snapshot. Caller-supplied selection IDs are +// qualified at each operation boundary; captured configuration references are +// not rewritten. func (e *Engine) WithNamespaces(prompt, profile, backend string) *Engine { view := *e view.namespaces = resourceNamespaces{prompt: prompt, profile: profile, backend: backend} @@ -198,15 +200,34 @@ func fileSource(name string) (fs.FS, string, string, error) { } func (e *Engine) InspectPrompt(ctx context.Context, id string) (*domain.PromptInspection, error) { + var err error + id, err = e.qualifyResourceID(id, e.namespaces.prompt, "prompt") + if err != nil { + return nil, err + } return e.runner.InspectPrompt(ctx, id) } func (e *Engine) InspectRequirements( ctx context.Context, request usecase.RequirementInspectionRequest, ) (*domain.RequirementInspection, error) { + var err error + request.PromptID, err = e.qualifyResourceID(request.PromptID, e.namespaces.prompt, "prompt") + if err != nil { + return nil, err + } + request.ProfileID, err = e.qualifyResourceID(request.ProfileID, e.namespaces.profile, "profile") + if err != nil { + return nil, err + } return e.runner.InspectRequirements(ctx, request) } func (e *Engine) InspectProfile(ctx context.Context, id string) (*domain.ProfileInspection, error) { + var err error + id, err = e.qualifyResourceID(id, e.namespaces.profile, "profile") + if err != nil { + return nil, err + } return e.runner.InspectProfile(ctx, id) } @@ -240,21 +261,29 @@ func (e *Engine) Run(ctx context.Context, request domain.RunRequest) (*domain.Ru } func (e *Engine) qualifyRunRequest(request domain.RunRequest) (domain.RunRequest, error) { - var ok bool - if request.PromptID != "" { - request.PromptID, ok = domain.QualifyResourceID(request.PromptID, e.namespaces.prompt) - if !ok { - return domain.RunRequest{}, fmt.Errorf("%w: invalid prompt ID", usecase.ErrInvalidRequest) - } + var err error + request.PromptID, err = e.qualifyResourceID(request.PromptID, e.namespaces.prompt, "prompt") + if err != nil { + return domain.RunRequest{}, err } - if request.ProfileID != "" { - request.ProfileID, ok = domain.QualifyResourceID(request.ProfileID, e.namespaces.profile) - if !ok { - return domain.RunRequest{}, fmt.Errorf("%w: invalid profile ID", usecase.ErrInvalidRequest) - } + request.ProfileID, err = e.qualifyResourceID(request.ProfileID, e.namespaces.profile, "profile") + if err != nil { + return domain.RunRequest{}, err } return request, nil } + +func (e *Engine) qualifyResourceID(id, namespace, kind string) (string, error) { + if id == "" { + return "", nil + } + qualified, ok := domain.QualifyResourceID(id, namespace) + if !ok { + return "", fmt.Errorf("%w: invalid %s ID", usecase.ErrInvalidRequest, kind) + } + return qualified, nil +} + func (e *Engine) RunPrepared(ctx context.Context, prepared *usecase.PreparedExecution) (*domain.RunResult, error) { return e.runner.RunPrepared(ctx, prepared) } @@ -263,5 +292,18 @@ func (e *Engine) SmokeTest(ctx context.Context, request domain.SmokeTestRequest) if e == nil || e.smokeTester == nil { return nil, fmt.Errorf("%w: smoke tester is not configured", usecase.ErrInvalidRequest) } + var err error + request.PromptID, err = e.qualifyResourceID(request.PromptID, e.namespaces.prompt, "prompt") + if err != nil { + return nil, err + } + request.ProfileID, err = e.qualifyResourceID(request.ProfileID, e.namespaces.profile, "profile") + if err != nil { + return nil, err + } + request.BackendID, err = e.qualifyResourceID(request.BackendID, e.namespaces.backend, "backend") + if err != nil { + return nil, err + } return e.smokeTester.Run(ctx, request) } diff --git a/namespaces.go b/namespaces.go index 74786cf..0cb952c 100644 --- a/namespaces.go +++ b/namespaces.go @@ -14,12 +14,14 @@ type Namespaces struct { Backend string } -// WithNamespaces returns an engine view that qualifies slashless prompt and -// profile selections with the supplied namespaces. Qualified selections pass -// through unchanged. It replaces all namespace defaults on the new view and -// leaves the receiver unchanged; resource existence is checked when selected. -// An empty field clears that resource's scope. Invalid namespace syntax or a -// nil engine returns an error matching ErrInvalidConfig. +// WithNamespaces returns an engine view that qualifies slashless caller-supplied +// prompt, profile, and backend selections with the supplied namespaces. +// Qualified selections pass through unchanged. It replaces all namespace +// defaults on the new view and leaves the receiver unchanged; resource +// existence is checked when selected. An empty field clears that resource's +// scope. Captured configuration references, defaults, and global configuration +// inspection are not rewritten. Invalid namespace syntax or a nil engine +// returns an error matching ErrInvalidConfig. func (e *Engine) WithNamespaces(namespaces Namespaces) (*Engine, error) { if e == nil || e.internal == nil { return nil, fmt.Errorf("%w: engine is nil", ErrInvalidConfig) diff --git a/namespaces_contract_test.go b/namespaces_contract_test.go index ead65b3..fed9341 100644 --- a/namespaces_contract_test.go +++ b/namespaces_contract_test.go @@ -164,6 +164,104 @@ func TestWithNamespacesViewsCanPrepareConcurrently(t *testing.T) { wait.Wait() } +func TestWithNamespacesRoutesInspectionsAndSmokeSelections(t *testing.T) { + engine := newNamespacedContractEngine(t, &fakeLLMClient{response: &promptkit.GenerateResponse{Content: "reply"}}) + scoped, err := engine.WithNamespaces(promptkit.Namespaces{ + Prompt: "alpha", Profile: "beta", Backend: "alpha", + }) + if err != nil { + t.Fatal(err) + } + + prompt, err := scoped.InspectPrompt(context.Background(), "shared") + if err != nil || prompt.PromptID != "alpha/shared" { + t.Fatalf("scoped prompt inspection = %+v, error %v", prompt, err) + } + profile, err := scoped.InspectProfile(context.Background(), "shared") + if err != nil || profile.ProfileID != "beta/shared" { + t.Fatalf("scoped profile inspection = %+v, error %v", profile, err) + } + requirements, err := scoped.InspectRequirements(context.Background(), promptkit.RequirementInspectionRequest{ + PromptID: "shared", ProfileID: "shared", + }) + if err != nil || requirements.PromptID != "alpha/shared" || requirements.ProfileID != "beta/shared" { + t.Fatalf("scoped requirements inspection = %+v, error %v", requirements, err) + } + qualifiedPrompt, err := scoped.InspectPrompt(context.Background(), "beta/shared") + if err != nil || qualifiedPrompt.PromptID != "beta/shared" { + t.Fatalf("qualified prompt inspection = %+v, error %v", qualifiedPrompt, err) + } + + // Explicit profile selection keeps precedence over the supplied backend, + // while prompt, profile, and backend caller references are independently + // validated/qualified. The profile's configured backend remains exact. + report, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{ + Kind: promptkit.SmokeTestReachability, PromptID: "shared", ProfileID: "shared", BackendID: "backend", + }) + if err != nil { + t.Fatal(err) + } + if report.Target.PromptID != "alpha/shared" || report.Target.ProfileID != "beta/shared" || + report.Target.BackendID != "beta/backend" || report.Target.SelectionSource != "profile" { + t.Fatalf("scoped smoke selection = %+v", report.Target) + } + backendReport, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{ + Kind: promptkit.SmokeTestReachability, BackendID: "backend", + }) + if err != nil { + t.Fatal(err) + } + if backendReport.Target.BackendID != "alpha/backend" || backendReport.Target.SelectionSource != "backend" { + t.Fatalf("scoped backend diagnostic selection = %+v", backendReport.Target) + } + defaultedReport, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{ + Kind: promptkit.SmokeTestReachability, PromptID: "shared", + }) + if err != nil { + t.Fatal(err) + } + if defaultedReport.Target.PromptID != "alpha/shared" || defaultedReport.Target.ProfileID != "root-only" || + defaultedReport.Target.SelectionSource != "prompt" { + t.Fatalf("configured prompt default was rewritten by scope: %+v", defaultedReport.Target) + } + + for _, run := range []func() error{ + func() error { _, err := scoped.InspectPrompt(context.Background(), "alpha/beta/shared"); return err }, + func() error { _, err := scoped.InspectProfile(context.Background(), "alpha/beta/shared"); return err }, + func() error { + _, err := scoped.InspectRequirements(context.Background(), promptkit.RequirementInspectionRequest{ + PromptID: "alpha/beta/shared", ProfileID: "shared", + }) + return err + }, + func() error { + _, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{ + Kind: promptkit.SmokeTestReachability, PromptID: "alpha/beta/shared", + }) + return err + }, + func() error { + _, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{ + Kind: promptkit.SmokeTestReachability, ProfileID: "alpha/beta/shared", + }) + return err + }, + func() error { + _, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{ + Kind: promptkit.SmokeTestReachability, ProfileID: "shared", BackendID: "alpha/beta/backend", + }) + return err + }, + } { + if err := run(); !errors.Is(err, promptkit.ErrInvalidRequest) { + t.Errorf("malformed supplied ID error = %v, want ErrInvalidRequest", err) + } + } + if _, err := scoped.InspectPrompt(context.Background(), "missing"); !errors.Is(err, promptkit.ErrPromptNotFound) { + t.Errorf("well-formed missing prompt error = %v, want ErrPromptNotFound", err) + } +} + func newNamespacedContractEngine(t *testing.T, client promptkit.LLMClient) *promptkit.Engine { t.Helper() var inputs []configkit.Input @@ -175,9 +273,10 @@ func newNamespacedContractEngine(t *testing.T, client promptkit.LLMClient) *prom "backend": namespace + "/backend", "model": namespace + "-model", }}, "prompts": map[string]any{"shared": map[string]any{ - "version": "1", - "messages": []any{map[string]any{"role": "user", "content": "Hello"}}, - "output": map[string]any{"format": "text", "validation_mode": "none"}, + "version": "1", + "default_profile": "root-only", + "messages": []any{map[string]any{"role": "user", "content": "Hello"}}, + "output": map[string]any{"format": "text", "validation_mode": "none"}, }}, })) if err != nil {