Apply namespace scopes to inspections and diagnostics

This commit is contained in:
2026-10-17 06:17:16 +00:00
parent aa54b8d1c4
commit 15f79ce7b1
6 changed files with 186 additions and 30 deletions

View File

@@ -81,14 +81,15 @@ prepared, err := scoped.Prepare(ctx, promptkit.RunRequest{
}) })
``` ```
Ordinary execution and preparation qualify slashless caller selections with Execution, preparation, prompt/profile inspection, requirement inspection, and
their respective namespace. Already-qualified IDs pass through unchanged; smoke diagnostics qualify slashless caller selections with their respective
configured defaults and references are not rewritten. Each call replaces the namespaces. Already-qualified IDs pass through unchanged; configured defaults
view's scope defaults, including clearing a scope with an empty field, without and references are not rewritten. Each call replaces the view's scope defaults,
changing the receiver or rebuilding its captured configuration and execution including clearing a scope with an empty field, without changing the receiver
collaborators. Namespace syntax is checked when creating the view; resource or rebuilding its captured configuration and execution collaborators.
existence is checked when selected. See [`WithNamespaces`](../../namespaces.go) Namespace syntax is checked when creating the view; resource existence is
for the exact public contract. checked when selected. Configuration inspection remains global. See
[`WithNamespaces`](../../namespaces.go) for the exact public contract.
## Supply Embedded Application Defaults ## Supply Embedded Application Defaults

View File

@@ -19,7 +19,7 @@ contributor workflow and validation.
| `internal/configuration` | Loads keyed backend, profile, and prompt configuration through ConfigKit; records effective field presence, validates final values including optional diagnostic metadata, captures indexed prompt definitions, referenced message content, compiled templates, input metadata snapshots, sorted identity-only profile and prompt inventories, backend and presence-aware profile snapshots, and an explicit read-only inspection view. | [Typed loader](../../internal/configuration/load.go), [identity listing](../../internal/configuration/listing.go), [internal sources](sources.md#typed-configuration-loading) | | `internal/configuration` | Loads keyed backend, profile, and prompt configuration through ConfigKit; records effective field presence, validates final values including optional diagnostic metadata, captures indexed prompt definitions, referenced message content, compiled templates, input metadata snapshots, sorted identity-only profile and prompt inventories, backend and presence-aware profile snapshots, and an explicit read-only inspection view. | [Typed loader](../../internal/configuration/load.go), [identity listing](../../internal/configuration/listing.go), [internal sources](sources.md#typed-configuration-loading) |
| `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) | | `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) |
| `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, requirement inspection, profiles, execution targets, rendering, generation, validation, and source-neutral smoke diagnostics, and owns source-neutral invariants for shared execution settings, OpenAI-compatible base endpoints, session identifiers, output contracts, and resource-identity syntax and qualification. Source parsing, required fields, other source-specific normalization, defaulting, and boundary-specific error classification remain with their callers. | [Domain declarations](../../internal/domain/domain.go), [requirement values](../../internal/domain/requirements.go), [endpoint invariant](../../internal/domain/endpoint.go) | | `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, requirement inspection, profiles, execution targets, rendering, generation, validation, and source-neutral smoke diagnostics, and owns source-neutral invariants for shared execution settings, OpenAI-compatible base endpoints, session identifiers, output contracts, and resource-identity syntax and qualification. Source parsing, required fields, other source-specific normalization, defaulting, and boundary-specific error classification remain with their callers. | [Domain declarations](../../internal/domain/domain.go), [requirement values](../../internal/domain/requirements.go), [endpoint invariant](../../internal/domain/endpoint.go) |
| `internal/engine` | Assembles caller-selected configuration into one snapshot, then constructs shared capacity, rendering, validation, generation, repair, the ordinary runner, and the smoke tester for each engine. Immutable namespace views share those collaborators and qualify ordinary request selections. Built-in engines provide the diagnostic HTTP collaborator; injected-client engines do not. It delegates identity listing directly to the captured configuration inventory rather than the runner. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#typed-configuration-loading), [internal runner](runner.md) | | `internal/engine` | Assembles caller-selected configuration into one snapshot, then constructs shared capacity, rendering, validation, generation, repair, the ordinary runner, and the smoke tester for each engine. Immutable namespace views share those collaborators and qualify caller-supplied execution, inspection, and diagnostic selections without rewriting captured references. Built-in engines provide the diagnostic HTTP collaborator; injected-client engines do not. It delegates identity listing directly to the captured configuration inventory rather than the runner. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#typed-configuration-loading), [internal runner](runner.md) |
| `internal/generation` | Coordinates frozen built-in HTTP attempts and one-shot injected calls through active backend permits, including bounded status retries, cancellable backoff, ordinary attempt observations, and invocation-local evidence for diagnostics. | [Internal generation coordination](generation.md) | | `internal/generation` | Coordinates frozen built-in HTTP attempts and one-shot injected calls through active backend permits, including bounded status retries, cancellable backoff, ordinary attempt observations, and invocation-local evidence for diagnostics. | [Internal generation coordination](generation.md) |
| `internal/defaults` | Defines application-neutral framework constants and constructs the default execution target. It contains no CLI, server, or inbound HTTP limits. | [Framework defaults](../../internal/defaults/defaults.go) | | `internal/defaults` | Defines application-neutral framework constants and constructs the default execution target. It contains no CLI, server, or inbound HTTP limits. | [Framework defaults](../../internal/defaults/defaults.go) |
| `internal/filecatalog` | Provides deterministic YAML discovery and path helpers for operating-system filesystems and `fs.FS` sources. | [File catalog](../../internal/filecatalog/catalog.go) | | `internal/filecatalog` | Provides deterministic YAML discovery and path helpers for operating-system filesystems and `fs.FS` sources. | [File catalog](../../internal/filecatalog/catalog.go) |

View File

@@ -650,6 +650,18 @@ prove shared behavior and full identities. Reuse naming-helper tests instead of
duplicating the full syntax matrix in every API test. Run relevant root/engine duplicating the full syntax matrix in every API test. Run relevant root/engine
tests and focused race checks. tests and focused race checks.
Completion: prompt, profile, and requirement inspections now qualify each
explicit caller selection independently, and smoke diagnostics qualify supplied
prompt, profile, and backend IDs before applying their unchanged selection
precedence. Captured references and global configuration inspection remain
unchanged. Contract tests verify full returned identities, exact configured
prompt defaults and backend references, malformed caller IDs, and ordinary
not-found behavior.
Validation passed: `GOWORK=off go test ./...`, focused
`GOWORK=off go test -race . -run 'TestWithNamespaces' -count=1`,
`GOWORK=off go vet ./...`, `GOWORK=off python3 scripts/check-examples.py`,
and `git diff --check`.
## Stage 14: Add Backend Listing And Namespace Filters ## Stage 14: Add Backend Listing And Namespace Filters
Scope: the existing identity-inventory path, not the execution runner. Scope: the existing identity-inventory path, not the execution runner.

View File

@@ -153,7 +153,9 @@ func New(cfg Config, options Options) (*Engine, error) {
} }
// WithNamespaces returns an immutable view sharing this engine's assembled // WithNamespaces returns an immutable view sharing this engine's assembled
// collaborators and configuration snapshot. // collaborators and configuration snapshot. Caller-supplied selection IDs are
// qualified at each operation boundary; captured configuration references are
// not rewritten.
func (e *Engine) WithNamespaces(prompt, profile, backend string) *Engine { func (e *Engine) WithNamespaces(prompt, profile, backend string) *Engine {
view := *e view := *e
view.namespaces = resourceNamespaces{prompt: prompt, profile: profile, backend: backend} view.namespaces = resourceNamespaces{prompt: prompt, profile: profile, backend: backend}
@@ -198,15 +200,34 @@ func fileSource(name string) (fs.FS, string, string, error) {
} }
func (e *Engine) InspectPrompt(ctx context.Context, id string) (*domain.PromptInspection, error) { func (e *Engine) InspectPrompt(ctx context.Context, id string) (*domain.PromptInspection, error) {
var err error
id, err = e.qualifyResourceID(id, e.namespaces.prompt, "prompt")
if err != nil {
return nil, err
}
return e.runner.InspectPrompt(ctx, id) return e.runner.InspectPrompt(ctx, id)
} }
func (e *Engine) InspectRequirements( func (e *Engine) InspectRequirements(
ctx context.Context, ctx context.Context,
request usecase.RequirementInspectionRequest, request usecase.RequirementInspectionRequest,
) (*domain.RequirementInspection, error) { ) (*domain.RequirementInspection, error) {
var err error
request.PromptID, err = e.qualifyResourceID(request.PromptID, e.namespaces.prompt, "prompt")
if err != nil {
return nil, err
}
request.ProfileID, err = e.qualifyResourceID(request.ProfileID, e.namespaces.profile, "profile")
if err != nil {
return nil, err
}
return e.runner.InspectRequirements(ctx, request) return e.runner.InspectRequirements(ctx, request)
} }
func (e *Engine) InspectProfile(ctx context.Context, id string) (*domain.ProfileInspection, error) { func (e *Engine) InspectProfile(ctx context.Context, id string) (*domain.ProfileInspection, error) {
var err error
id, err = e.qualifyResourceID(id, e.namespaces.profile, "profile")
if err != nil {
return nil, err
}
return e.runner.InspectProfile(ctx, id) return e.runner.InspectProfile(ctx, id)
} }
@@ -240,21 +261,29 @@ func (e *Engine) Run(ctx context.Context, request domain.RunRequest) (*domain.Ru
} }
func (e *Engine) qualifyRunRequest(request domain.RunRequest) (domain.RunRequest, error) { func (e *Engine) qualifyRunRequest(request domain.RunRequest) (domain.RunRequest, error) {
var ok bool var err error
if request.PromptID != "" { request.PromptID, err = e.qualifyResourceID(request.PromptID, e.namespaces.prompt, "prompt")
request.PromptID, ok = domain.QualifyResourceID(request.PromptID, e.namespaces.prompt) if err != nil {
if !ok { return domain.RunRequest{}, err
return domain.RunRequest{}, fmt.Errorf("%w: invalid prompt ID", usecase.ErrInvalidRequest)
}
} }
if request.ProfileID != "" { request.ProfileID, err = e.qualifyResourceID(request.ProfileID, e.namespaces.profile, "profile")
request.ProfileID, ok = domain.QualifyResourceID(request.ProfileID, e.namespaces.profile) if err != nil {
if !ok { return domain.RunRequest{}, err
return domain.RunRequest{}, fmt.Errorf("%w: invalid profile ID", usecase.ErrInvalidRequest)
}
} }
return request, nil return request, nil
} }
func (e *Engine) qualifyResourceID(id, namespace, kind string) (string, error) {
if id == "" {
return "", nil
}
qualified, ok := domain.QualifyResourceID(id, namespace)
if !ok {
return "", fmt.Errorf("%w: invalid %s ID", usecase.ErrInvalidRequest, kind)
}
return qualified, nil
}
func (e *Engine) RunPrepared(ctx context.Context, prepared *usecase.PreparedExecution) (*domain.RunResult, error) { func (e *Engine) RunPrepared(ctx context.Context, prepared *usecase.PreparedExecution) (*domain.RunResult, error) {
return e.runner.RunPrepared(ctx, prepared) return e.runner.RunPrepared(ctx, prepared)
} }
@@ -263,5 +292,18 @@ func (e *Engine) SmokeTest(ctx context.Context, request domain.SmokeTestRequest)
if e == nil || e.smokeTester == nil { if e == nil || e.smokeTester == nil {
return nil, fmt.Errorf("%w: smoke tester is not configured", usecase.ErrInvalidRequest) return nil, fmt.Errorf("%w: smoke tester is not configured", usecase.ErrInvalidRequest)
} }
var err error
request.PromptID, err = e.qualifyResourceID(request.PromptID, e.namespaces.prompt, "prompt")
if err != nil {
return nil, err
}
request.ProfileID, err = e.qualifyResourceID(request.ProfileID, e.namespaces.profile, "profile")
if err != nil {
return nil, err
}
request.BackendID, err = e.qualifyResourceID(request.BackendID, e.namespaces.backend, "backend")
if err != nil {
return nil, err
}
return e.smokeTester.Run(ctx, request) return e.smokeTester.Run(ctx, request)
} }

View File

@@ -14,12 +14,14 @@ type Namespaces struct {
Backend string Backend string
} }
// WithNamespaces returns an engine view that qualifies slashless prompt and // WithNamespaces returns an engine view that qualifies slashless caller-supplied
// profile selections with the supplied namespaces. Qualified selections pass // prompt, profile, and backend selections with the supplied namespaces.
// through unchanged. It replaces all namespace defaults on the new view and // Qualified selections pass through unchanged. It replaces all namespace
// leaves the receiver unchanged; resource existence is checked when selected. // defaults on the new view and leaves the receiver unchanged; resource
// An empty field clears that resource's scope. Invalid namespace syntax or a // existence is checked when selected. An empty field clears that resource's
// nil engine returns an error matching ErrInvalidConfig. // scope. Captured configuration references, defaults, and global configuration
// inspection are not rewritten. Invalid namespace syntax or a nil engine
// returns an error matching ErrInvalidConfig.
func (e *Engine) WithNamespaces(namespaces Namespaces) (*Engine, error) { func (e *Engine) WithNamespaces(namespaces Namespaces) (*Engine, error) {
if e == nil || e.internal == nil { if e == nil || e.internal == nil {
return nil, fmt.Errorf("%w: engine is nil", ErrInvalidConfig) return nil, fmt.Errorf("%w: engine is nil", ErrInvalidConfig)

View File

@@ -164,6 +164,104 @@ func TestWithNamespacesViewsCanPrepareConcurrently(t *testing.T) {
wait.Wait() wait.Wait()
} }
func TestWithNamespacesRoutesInspectionsAndSmokeSelections(t *testing.T) {
engine := newNamespacedContractEngine(t, &fakeLLMClient{response: &promptkit.GenerateResponse{Content: "reply"}})
scoped, err := engine.WithNamespaces(promptkit.Namespaces{
Prompt: "alpha", Profile: "beta", Backend: "alpha",
})
if err != nil {
t.Fatal(err)
}
prompt, err := scoped.InspectPrompt(context.Background(), "shared")
if err != nil || prompt.PromptID != "alpha/shared" {
t.Fatalf("scoped prompt inspection = %+v, error %v", prompt, err)
}
profile, err := scoped.InspectProfile(context.Background(), "shared")
if err != nil || profile.ProfileID != "beta/shared" {
t.Fatalf("scoped profile inspection = %+v, error %v", profile, err)
}
requirements, err := scoped.InspectRequirements(context.Background(), promptkit.RequirementInspectionRequest{
PromptID: "shared", ProfileID: "shared",
})
if err != nil || requirements.PromptID != "alpha/shared" || requirements.ProfileID != "beta/shared" {
t.Fatalf("scoped requirements inspection = %+v, error %v", requirements, err)
}
qualifiedPrompt, err := scoped.InspectPrompt(context.Background(), "beta/shared")
if err != nil || qualifiedPrompt.PromptID != "beta/shared" {
t.Fatalf("qualified prompt inspection = %+v, error %v", qualifiedPrompt, err)
}
// Explicit profile selection keeps precedence over the supplied backend,
// while prompt, profile, and backend caller references are independently
// validated/qualified. The profile's configured backend remains exact.
report, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{
Kind: promptkit.SmokeTestReachability, PromptID: "shared", ProfileID: "shared", BackendID: "backend",
})
if err != nil {
t.Fatal(err)
}
if report.Target.PromptID != "alpha/shared" || report.Target.ProfileID != "beta/shared" ||
report.Target.BackendID != "beta/backend" || report.Target.SelectionSource != "profile" {
t.Fatalf("scoped smoke selection = %+v", report.Target)
}
backendReport, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{
Kind: promptkit.SmokeTestReachability, BackendID: "backend",
})
if err != nil {
t.Fatal(err)
}
if backendReport.Target.BackendID != "alpha/backend" || backendReport.Target.SelectionSource != "backend" {
t.Fatalf("scoped backend diagnostic selection = %+v", backendReport.Target)
}
defaultedReport, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{
Kind: promptkit.SmokeTestReachability, PromptID: "shared",
})
if err != nil {
t.Fatal(err)
}
if defaultedReport.Target.PromptID != "alpha/shared" || defaultedReport.Target.ProfileID != "root-only" ||
defaultedReport.Target.SelectionSource != "prompt" {
t.Fatalf("configured prompt default was rewritten by scope: %+v", defaultedReport.Target)
}
for _, run := range []func() error{
func() error { _, err := scoped.InspectPrompt(context.Background(), "alpha/beta/shared"); return err },
func() error { _, err := scoped.InspectProfile(context.Background(), "alpha/beta/shared"); return err },
func() error {
_, err := scoped.InspectRequirements(context.Background(), promptkit.RequirementInspectionRequest{
PromptID: "alpha/beta/shared", ProfileID: "shared",
})
return err
},
func() error {
_, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{
Kind: promptkit.SmokeTestReachability, PromptID: "alpha/beta/shared",
})
return err
},
func() error {
_, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{
Kind: promptkit.SmokeTestReachability, ProfileID: "alpha/beta/shared",
})
return err
},
func() error {
_, err := scoped.SmokeTest(context.Background(), promptkit.SmokeTestRequest{
Kind: promptkit.SmokeTestReachability, ProfileID: "shared", BackendID: "alpha/beta/backend",
})
return err
},
} {
if err := run(); !errors.Is(err, promptkit.ErrInvalidRequest) {
t.Errorf("malformed supplied ID error = %v, want ErrInvalidRequest", err)
}
}
if _, err := scoped.InspectPrompt(context.Background(), "missing"); !errors.Is(err, promptkit.ErrPromptNotFound) {
t.Errorf("well-formed missing prompt error = %v, want ErrPromptNotFound", err)
}
}
func newNamespacedContractEngine(t *testing.T, client promptkit.LLMClient) *promptkit.Engine { func newNamespacedContractEngine(t *testing.T, client promptkit.LLMClient) *promptkit.Engine {
t.Helper() t.Helper()
var inputs []configkit.Input var inputs []configkit.Input
@@ -175,9 +273,10 @@ func newNamespacedContractEngine(t *testing.T, client promptkit.LLMClient) *prom
"backend": namespace + "/backend", "model": namespace + "-model", "backend": namespace + "/backend", "model": namespace + "-model",
}}, }},
"prompts": map[string]any{"shared": map[string]any{ "prompts": map[string]any{"shared": map[string]any{
"version": "1", "version": "1",
"messages": []any{map[string]any{"role": "user", "content": "Hello"}}, "default_profile": "root-only",
"output": map[string]any{"format": "text", "validation_mode": "none"}, "messages": []any{map[string]any{"role": "user", "content": "Hello"}},
"output": map[string]any{"format": "text", "validation_mode": "none"},
}}, }},
})) }))
if err != nil { if err != nil {