Add offline proposal assessment observation example

docs/development.md

@@ -40,7 +40,7 @@ Start with:
 | Model-client behavior | The [OpenAI-compatible integration contract](integrations/openai-compatible-chat.md), [internal model-client document](internal/llm.md), and owning package tests. |
 | Internal package implementation | The [architecture policy](policy/architecture.md), [internal component overview](internal/overview.md), and focused internal document listed for that package. |
 | Tests or test fixtures | The [testing policy](policy/testing.md), owning package, and focused internal document listed by the component overview. |
-| Maintained example | The [example](../examples/go-library/prepare/main.go), [consumer guide](consumers/pkg-promptkit.md), [framework format reference](formats.md), and [documentation policy](policy/documentation.md). |
+| Maintained example | The [preparation](../examples/go-library/prepare/main.go), [execution](../examples/go-library/run/main.go), and [observation](../examples/go-library/observe/main.go) examples, [consumer guide](consumers/pkg-promptkit.md), [framework format reference](formats.md), and [documentation policy](policy/documentation.md). |
 | Documentation | The [documentation policy](policy/documentation.md) and canonical owner of every affected contract. |
 | Release preparation or publication | The [release policy](policy/release.md) and [release procedure](release.md). |
 
@@ -59,7 +59,7 @@ release-candidate checks have one executable entry point from the repository roo
 ```
 
 The [script](../scripts/release-check.sh) owns the executable checks:
-race-enabled tests, vet, both offline consumer examples with semantic output
+race-enabled tests, vet, the three offline consumer examples with semantic output
 assertions, tracked Go formatting, local Markdown links, module/workspace
 hygiene, whitespace, and the credential-pattern scan. It does not publish,
 change files, validate release-note structure, or retire roadmaps. It requires Go,

docs/internal/overview.md

@@ -14,6 +14,7 @@ contributor workflow and validation.
 | Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, profile-inspection, and ordinary-run observation values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, and typed capacity and generation error mapping. | [Package GoDoc](../../doc.go), [observations](../../observations.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine facade](../../engine.go) |
 | `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
 | `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
+| `examples/go-library/observe` | Demonstrates offline built-in HTTP retry and synchronous metadata observation across structured proposal and assessment runs using embedded sources, two profiles on one finite backend, and an application decision. It is not a public library package. | [Example program](../../examples/go-library/observe/main.go) |
 | `internal/backend` | Constructs each engine's immutable registry from maintained definitions and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) |
 | `internal/catalog` | Strictly validates imported immutable maintained backend and profile catalog assets before engine assembly uses them. | [Catalog adapter](../../internal/catalog/catalog.go), [internal sources](sources.md#profiles-and-built-ins) |
 | `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) |

docs/policy/testing.md

@@ -64,13 +64,15 @@ Introducing hosted CI later would supplement, not silently redefine, this
 documented validation model.
 
 Full release-candidate validation includes race-enabled package tests, explicit
-static analysis, and execution with automated semantic checks of both maintained
+static analysis, and execution with automated semantic checks of the three maintained
 offline consumer examples. The test run builds the packages; the examples
 also exercise ordinary compilation. Separate ordinary tests and a standalone
 build are not additional release gates. Ordinary tests remain useful during
 development. The preparation example protects assembled preparation and
 inspection behavior. The execution example separately protects assembled
-`Run`, injected-client, validation, usage, and result behavior.
+`Run`, injected-client, validation, usage, and result behavior. The observation
+example protects assembled built-in retries, structured proposal and assessment
+runs, and application-owned interpretation of metadata.
 
 Tests in the default suite must be deterministic, offline, and independent of
 real credentials. They must not invoke paid APIs, use live network
@@ -96,7 +98,7 @@ Use each test type where it protects a distinct risk:
   interaction, while replacing live or nondeterministic external boundaries.
 - External-package root tests exercise the public facade as a Go consumer,
   while internal package tests own focused implementation behavior.
-- The maintained offline preparation and execution examples protect distinct
+- The maintained offline preparation, execution, and observation examples protect distinct
   representative assembled consumer workflows without contacting a model
   provider.
 - Fixtures should be minimal, synthetic, versioned with the behavior they

docs/roadmap/implementation.md

@@ -451,6 +451,8 @@ Run `go test -race -vet=off ./...` and `go vet ./...`; commit the verified scope
 
 ## Stage 11 — Add The Offline Proposal And Assessment Example
 
+**Completed:** 2026-09-23.
+
 Add `examples/go-library/observe` with embedded prompt/schema assets, two
 application-owned profiles sharing a finite custom backend, and one engine.
 Use a controlled RoundTripper under the built-in client: return one eligible

examples/go-library/observe/main.go

@@ -0,0 +1,157 @@
+package main
+
+import (
+	"context"
+	"embed"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"os"
+	"strings"
+
+	"gitea.maximumdirect.net/eric/promptkit"
+)
+
+//go:embed prompts/*.yaml schemas/*.json
+var assets embed.FS
+
+type scriptedTransport struct {
+	proposalCalls int
+}
+
+func (t *scriptedTransport) RoundTrip(req *http.Request) (*http.Response, error) {
+	if err := req.Context().Err(); err != nil {
+		return nil, err
+	}
+	var request struct {
+		Model string `json:"model"`
+	}
+	if err := json.NewDecoder(req.Body).Decode(&request); err != nil {
+		return nil, err
+	}
+	switch request.Model {
+	case "proposal-model":
+		t.proposalCalls++
+		if t.proposalCalls == 1 {
+			return httpResponse(http.StatusServiceUnavailable, `{"error":{}}`), nil
+		}
+		return modelResponse(`{"title":"Queue cleanup","risk":"low"}`, 10, 8), nil
+	case "assessment-model":
+		return modelResponse(`{"recommendation":"accept","confidence":0.9}`, 7, 5), nil
+	default:
+		return nil, fmt.Errorf("unexpected model in offline transport")
+	}
+}
+
+func modelResponse(content string, promptTokens, completionTokens int) *http.Response {
+	body, _ := json.Marshal(map[string]any{
+		"choices": []any{map[string]any{"message": map[string]string{"content": content}}},
+		"usage":   map[string]int{"prompt_tokens": promptTokens, "completion_tokens": completionTokens},
+	})
+	return httpResponse(http.StatusOK, string(body))
+}
+
+func httpResponse(status int, body string) *http.Response {
+	return &http.Response{StatusCode: status, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(body))}
+}
+
+type eventLog struct{ events []promptkit.ExecutionEvent }
+
+func (l *eventLog) record(event promptkit.ExecutionEvent) { l.events = append(l.events, event) }
+
+func (l *eventLog) terminal() (promptkit.ExecutionEvent, error) {
+	if len(l.events) == 0 || l.events[len(l.events)-1].Kind != promptkit.ExecutionFinished {
+		return promptkit.ExecutionEvent{}, fmt.Errorf("execution did not finish")
+	}
+	return l.events[len(l.events)-1], nil
+}
+
+type summary struct {
+	ProposalAttempts           uint64 `json:"proposal_attempts"`
+	ProposalTransientRetries   uint64 `json:"proposal_transient_retries"`
+	ProposalUsageComplete      bool   `json:"proposal_usage_complete"`
+	AssessmentAttempts         uint64 `json:"assessment_attempts"`
+	AssessmentTransientRetries uint64 `json:"assessment_transient_retries"`
+	AssessmentOutcome          string `json:"assessment_outcome"`
+}
+
+func run() (summary, error) {
+	transport := &scriptedTransport{}
+	engine, err := promptkit.NewEngine(promptkit.Config{HTTPClient: &http.Client{Transport: transport}},
+		promptkit.WithPromptFS(assets, "prompts"),
+		promptkit.WithSchemaFS(assets, "schemas"),
+		promptkit.WithBackend(promptkit.Backend{ID: "offline-backend", Endpoint: "https://example.invalid/v1", ConcurrencyLimit: 1}),
+		promptkit.WithProfiles(
+			promptkit.Profile{ID: "proposal-profile", BackendID: "offline-backend", Model: "proposal-model"},
+			promptkit.Profile{ID: "assessment-profile", BackendID: "offline-backend", Model: "assessment-model"},
+		))
+	if err != nil {
+		return summary{}, err
+	}
+
+	var proposalLog eventLog
+	proposal, err := engine.Run(context.Background(), promptkit.RunRequest{
+		PromptID: "example.proposal", Inputs: map[string]promptkit.ArtifactRef{"idea": promptkit.Inline("Reduce queue cleanup toil.")}, Observer: proposalLog.record,
+	})
+	if err != nil {
+		return summary{}, err
+	}
+	if !proposal.Validation.IsValid {
+		return summary{}, fmt.Errorf("proposal failed structural validation")
+	}
+	proposalTerminal, err := proposalLog.terminal()
+	if err != nil {
+		return summary{}, err
+	}
+	var assessmentLog eventLog
+	assessment, err := engine.Run(context.Background(), promptkit.RunRequest{
+		PromptID: "example.assessment", Inputs: map[string]promptkit.ArtifactRef{"proposal": promptkit.Inline(proposal.RawOutput)}, Observer: assessmentLog.record,
+	})
+	if err != nil {
+		return summary{}, err
+	}
+	if !assessment.Validation.IsValid {
+		return summary{}, fmt.Errorf("assessment failed structural validation")
+	}
+	assessmentTerminal, err := assessmentLog.terminal()
+	if err != nil {
+		return summary{}, err
+	}
+	var proposalValue struct {
+		Risk string `json:"risk"`
+	}
+	var assessmentValue struct {
+		Recommendation string `json:"recommendation"`
+	}
+	if err := json.Unmarshal([]byte(proposal.RawOutput), &proposalValue); err != nil {
+		return summary{}, err
+	}
+	if err := json.Unmarshal([]byte(assessment.RawOutput), &assessmentValue); err != nil {
+		return summary{}, err
+	}
+	decision := "rejected"
+	if proposalValue.Risk == "low" && assessmentValue.Recommendation == "accept" {
+		decision = "accepted"
+	}
+	return summary{
+		ProposalAttempts:           proposalTerminal.Attempts,
+		ProposalTransientRetries:   proposalTerminal.TransientRetries,
+		ProposalUsageComplete:      proposalTerminal.Usage.PromptTokens.Complete,
+		AssessmentAttempts:         assessmentTerminal.Attempts,
+		AssessmentTransientRetries: assessmentTerminal.TransientRetries,
+		AssessmentOutcome:          decision,
+	}, nil
+}
+
+func main() {
+	value, err := run()
+	if err != nil {
+		fmt.Fprintln(os.Stderr, err)
+		os.Exit(1)
+	}
+	if err := json.NewEncoder(os.Stdout).Encode(value); err != nil {
+		fmt.Fprintln(os.Stderr, err)
+		os.Exit(1)
+	}
+}

examples/go-library/observe/prompts/assessment.yaml

@@ -0,0 +1,17 @@
+id: example.assessment
+version: "1.0.0"
+default_profile: assessment-profile
+description: Assess a structured proposal independently of schema validity.
+inputs:
+  - name: proposal
+    required: true
+    content_type: application/json
+messages:
+  - role: system
+    content: Return a concise assessment as JSON.
+  - role: user
+    content: '{{input "proposal"}}'
+output:
+  format: json
+  validation_mode: json_schema
+  schema_path: assessment.schema.json

examples/go-library/observe/prompts/proposal.yaml

@@ -0,0 +1,17 @@
+id: example.proposal
+version: "1.0.0"
+default_profile: proposal-profile
+description: Propose a bounded improvement from a synthetic idea.
+inputs:
+  - name: idea
+    required: true
+    content_type: text/plain
+messages:
+  - role: system
+    content: Return a concise proposal as JSON.
+  - role: user
+    content: '{{input "idea"}}'
+output:
+  format: json
+  validation_mode: json_schema
+  schema_path: proposal.schema.json

examples/go-library/observe/schemas/assessment.schema.json

@@ -0,0 +1,10 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "type": "object",
+  "required": ["recommendation", "confidence"],
+  "properties": {
+    "recommendation": {"type": "string"},
+    "confidence": {"type": "number"}
+  },
+  "additionalProperties": false
+}

examples/go-library/observe/schemas/proposal.schema.json

@@ -0,0 +1,10 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "type": "object",
+  "required": ["title", "risk"],
+  "properties": {
+    "title": {"type": "string"},
+    "risk": {"type": "string"}
+  },
+  "additionalProperties": false
+}

scripts/release-check.sh

@@ -24,6 +24,13 @@ assert result["is_valid"] is True, result
 assert result["validation_status"] == "passed", result
 assert result["model"] == "offline-model", result
 assert result["total_tokens"] == 18, result
+observed = example("observe")
+assert observed["proposal_attempts"] == 2, observed
+assert observed["proposal_transient_retries"] == 1, observed
+assert observed["proposal_usage_complete"] is False, observed
+assert observed["assessment_attempts"] == 1, observed
+assert observed["assessment_transient_retries"] == 0, observed
+assert observed["assessment_outcome"] == "accepted", observed
 PY
 
 unformatted=$(