Document workload-oriented LLM profile deployment

This commit is contained in:
2026-08-03 17:21:42 +00:00
parent 5cd7f8e737
commit 394278e1f2
16 changed files with 230 additions and 87 deletions

View File

@@ -88,55 +88,6 @@ not as committed release dates.
secret unless a concrete multi-tenant or privacy requirement justifies
sacrificing deterministic identity across installations.
### Pipeline-Level LLM Profile Defaults
- Add an optional pipeline-level `llm_profile` default so an operator can
select one PromptKit execution policy for the pipeline without repeating the
same profile ID on every LLM-backed module binding.
- Apply the following precedence consistently: an explicit run-wide
`--llm-profile` override, then a binding-specific `llm_profile`, then the
pipeline-level default, then the prompt definition's embedded
`default_profile`.
- Apply inheritance only to bindings whose resolved modules are LLM-backed,
including applicable chunk, extraction, merge, normalization, and validation
bindings. Do not attach an inherited profile to deterministic modules or
weaken existing validation that rejects profiles where generation is not
supported.
- Resolve inherited profiles before effective-pipeline validation, digest and
checkpoint construction, execution, and provenance capture. Validate every
resulting explicit profile ID against the selected PromptKit profile source,
and ensure manifests and debug output report the profile actually selected
for each generation target.
- Preserve binding-specific profiles as intentional exceptions for modules
that require a different quality, latency, cost, provider, or reasoning
policy. Preserve `--llm-profile` as the convenient highest-precedence
experiment or incident-response override for an entire run.
- Treat PromptKit profiles as execution-policy configuration and prefer stable,
workload-oriented IDs such as `dnd-extraction` over model names. A pipeline
should express the kind of work it performs rather than encode a particular
provider, model, or deployment environment.
- Establish deployment-managed PromptKit profile files as the recommended
environment-specific configuration mechanism. Production, development, and
local deployments may each define the same logical `dnd-extraction` ID with
different model and generation settings, while retaining one unchanged
Notarius pipeline definition.
- Keep deployment profiles distinct from both Notarius prompt assets embedded
in the application binary and PromptKit's built-in profile catalog. Document
that `promptkit.profile_dir` or `promptkit.profile_file` selects an external
filesystem source whose definitions overlay PromptKit built-ins, and
recommend application-owned IDs rather than silently replacing built-in
profile IDs.
- Document an operator-friendly layout in which the Notarius configuration and
a profile subdirectory are deployed together. Until profile paths are
explicitly resolved relative to the configuration file, clearly state that
relative paths use the process working directory and recommend absolute
paths for services and containers.
- Update the configuration reference, operations guide, PromptKit integration
boundary, relevant internal configuration and pipeline documentation, and
maintained D&D examples together. Include one concrete external-profile
example and explain the profile precedence and environment-neutral pipeline
pattern without duplicating PromptKit's complete profile-format reference.
### Raise The Default Application-Wide LLM Limit
- Raise the default `concurrency.total_llm` value from 1 to 16 so ordinary