Add offline proposal assessment observation example

This commit is contained in:
2026-09-23 17:23:16 +00:00
parent 93e53d1199
commit 1cc754c29e
10 changed files with 228 additions and 5 deletions

View File

@@ -40,7 +40,7 @@ Start with:
| Model-client behavior | The [OpenAI-compatible integration contract](integrations/openai-compatible-chat.md), [internal model-client document](internal/llm.md), and owning package tests. |
| Internal package implementation | The [architecture policy](policy/architecture.md), [internal component overview](internal/overview.md), and focused internal document listed for that package. |
| Tests or test fixtures | The [testing policy](policy/testing.md), owning package, and focused internal document listed by the component overview. |
| Maintained example | The [example](../examples/go-library/prepare/main.go), [consumer guide](consumers/pkg-promptkit.md), [framework format reference](formats.md), and [documentation policy](policy/documentation.md). |
| Maintained example | The [preparation](../examples/go-library/prepare/main.go), [execution](../examples/go-library/run/main.go), and [observation](../examples/go-library/observe/main.go) examples, [consumer guide](consumers/pkg-promptkit.md), [framework format reference](formats.md), and [documentation policy](policy/documentation.md). |
| Documentation | The [documentation policy](policy/documentation.md) and canonical owner of every affected contract. |
| Release preparation or publication | The [release policy](policy/release.md) and [release procedure](release.md). |
@@ -59,7 +59,7 @@ release-candidate checks have one executable entry point from the repository roo
```
The [script](../scripts/release-check.sh) owns the executable checks:
race-enabled tests, vet, both offline consumer examples with semantic output
race-enabled tests, vet, the three offline consumer examples with semantic output
assertions, tracked Go formatting, local Markdown links, module/workspace
hygiene, whitespace, and the credential-pattern scan. It does not publish,
change files, validate release-note structure, or retire roadmaps. It requires Go,

View File

@@ -14,6 +14,7 @@ contributor workflow and validation.
| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, profile-inspection, and ordinary-run observation values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, and typed capacity and generation error mapping. | [Package GoDoc](../../doc.go), [observations](../../observations.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine facade](../../engine.go) |
| `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
| `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
| `examples/go-library/observe` | Demonstrates offline built-in HTTP retry and synchronous metadata observation across structured proposal and assessment runs using embedded sources, two profiles on one finite backend, and an application decision. It is not a public library package. | [Example program](../../examples/go-library/observe/main.go) |
| `internal/backend` | Constructs each engine's immutable registry from maintained definitions and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) |
| `internal/catalog` | Strictly validates imported immutable maintained backend and profile catalog assets before engine assembly uses them. | [Catalog adapter](../../internal/catalog/catalog.go), [internal sources](sources.md#profiles-and-built-ins) |
| `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) |

View File

@@ -64,13 +64,15 @@ Introducing hosted CI later would supplement, not silently redefine, this
documented validation model.
Full release-candidate validation includes race-enabled package tests, explicit
static analysis, and execution with automated semantic checks of both maintained
static analysis, and execution with automated semantic checks of the three maintained
offline consumer examples. The test run builds the packages; the examples
also exercise ordinary compilation. Separate ordinary tests and a standalone
build are not additional release gates. Ordinary tests remain useful during
development. The preparation example protects assembled preparation and
inspection behavior. The execution example separately protects assembled
`Run`, injected-client, validation, usage, and result behavior.
`Run`, injected-client, validation, usage, and result behavior. The observation
example protects assembled built-in retries, structured proposal and assessment
runs, and application-owned interpretation of metadata.
Tests in the default suite must be deterministic, offline, and independent of
real credentials. They must not invoke paid APIs, use live network
@@ -96,7 +98,7 @@ Use each test type where it protects a distinct risk:
interaction, while replacing live or nondeterministic external boundaries.
- External-package root tests exercise the public facade as a Go consumer,
while internal package tests own focused implementation behavior.
- The maintained offline preparation and execution examples protect distinct
- The maintained offline preparation, execution, and observation examples protect distinct
representative assembled consumer workflows without contacting a model
provider.
- Fixtures should be minimal, synthetic, versioned with the behavior they

View File

@@ -451,6 +451,8 @@ Run `go test -race -vet=off ./...` and `go vet ./...`; commit the verified scope
## Stage 11 — Add The Offline Proposal And Assessment Example
**Completed:** 2026-09-23.
Add `examples/go-library/observe` with embedded prompt/schema assets, two
application-owned profiles sharing a finite custom backend, and one engine.
Use a controlled RoundTripper under the built-in client: return one eligible