diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index fd1c7a2..2ba0426 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -53,6 +53,120 @@ not as committed release dates. spell, combat, interaction, and scene-description lanes after real-world use. Add more complex chunking only in response to demonstrated failures. +## Cross-Cutting LLM Runtime + +### Deterministic Prompt Session Identity + +- Replace the source-document-ID default for prompt sessions with one + predictable, procedurally generated session ID for the complete + source-processing workload. +- Preserve an explicit non-empty `--session-id` as the highest-precedence + override. Otherwise, derive the default only from the effective input module + identity and the exact raw input bytes. +- Use a versioned, bounded representation such as + `notarius:v1:`. The exact encoding + must fit PromptKit's session length contract and must not embed source + content. +- Keep the derived session stable across runs, pipelines, selected lanes, + ordered steps, retries, resume, recomputation, LLM profiles, reasoning + overrides, and output, debug, or cache settings. +- Do not include file-backed references, generated references, reference + contents, or the composition of a reference bundle in session derivation. + References may change between prompt calls within one pipeline without + changing routing affinity. +- Resolve the authoritative session before checkpoint construction and use the + same value for checkpoint runtime identity, every prompt-facing module, + PromptKit's direct session field, the compatibility `session_id` prompt + variable, run-manifest metadata, and debug metadata. +- Keep routing identity separate from cache and checkpoint content identity. + Exact prompt prefixes, reference contents, model settings, and other + generation-affecting inputs must continue to participate in their existing + hashes and checkpoint fingerprints even though they do not change the + session. +- Treat the generated value as a provider-visible, stable pseudonymous + correlation identifier. Do not introduce an installation-specific HMAC or + secret unless a concrete multi-tenant or privacy requirement justifies + sacrificing deterministic identity across installations. + +### Pipeline-Level LLM Profile Defaults + +- Add an optional pipeline-level `llm_profile` default so an operator can + select one PromptKit execution policy for the pipeline without repeating the + same profile ID on every LLM-backed module binding. +- Apply the following precedence consistently: an explicit run-wide + `--llm-profile` override, then a binding-specific `llm_profile`, then the + pipeline-level default, then the prompt definition's embedded + `default_profile`. +- Apply inheritance only to bindings whose resolved modules are LLM-backed, + including applicable chunk, extraction, merge, normalization, and validation + bindings. Do not attach an inherited profile to deterministic modules or + weaken existing validation that rejects profiles where generation is not + supported. +- Resolve inherited profiles before effective-pipeline validation, digest and + checkpoint construction, execution, and provenance capture. Validate every + resulting explicit profile ID against the selected PromptKit profile source, + and ensure manifests and debug output report the profile actually selected + for each generation target. +- Preserve binding-specific profiles as intentional exceptions for modules + that require a different quality, latency, cost, provider, or reasoning + policy. Preserve `--llm-profile` as the convenient highest-precedence + experiment or incident-response override for an entire run. +- Treat PromptKit profiles as execution-policy configuration and prefer stable, + workload-oriented IDs such as `dnd-extraction` over model names. A pipeline + should express the kind of work it performs rather than encode a particular + provider, model, or deployment environment. +- Establish deployment-managed PromptKit profile files as the recommended + environment-specific configuration mechanism. Production, development, and + local deployments may each define the same logical `dnd-extraction` ID with + different model and generation settings, while retaining one unchanged + Notarius pipeline definition. +- Keep deployment profiles distinct from both Notarius prompt assets embedded + in the application binary and PromptKit's built-in profile catalog. Document + that `promptkit.profile_dir` or `promptkit.profile_file` selects an external + filesystem source whose definitions overlay PromptKit built-ins, and + recommend application-owned IDs rather than silently replacing built-in + profile IDs. +- Document an operator-friendly layout in which the Notarius configuration and + a profile subdirectory are deployed together. Until profile paths are + explicitly resolved relative to the configuration file, clearly state that + relative paths use the process working directory and recommend absolute + paths for services and containers. +- Update the configuration reference, operations guide, PromptKit integration + boundary, relevant internal configuration and pipeline documentation, and + maintained D&D examples together. Include one concrete external-profile + example and explain the profile precedence and environment-neutral pipeline + pattern without duplicating PromptKit's complete profile-format reference. + +### Raise The Default Application-Wide LLM Limit + +- Raise the default `concurrency.total_llm` value from 1 to 16 so ordinary + single-backend runs can use PromptKit's expected OpenRouter capacity and + lower-capacity local backends without an unnecessarily narrower Notarius + limit. +- Keep the Notarius application-wide scheduler mandatory and require + `total_llm` to remain a positive integer. Do not make the default unlimited: + endpoint-only profiles, an unrestricted local backend, injected clients, and + aggregate work across several backends may have no narrower PromptKit limit. +- Continue defaulting `concurrency.stage_workers.extract` to the effective + `total_llm`, making its default 16 as part of the same change. Preserve an + explicit lower extract-worker setting when an operator wants less queued or + concurrent extraction work. +- Define effective provider concurrency as the intersection of the Notarius + application-wide limit, the selected PromptKit backend limit when present, + and the work made available by stage execution. A Notarius limit of 16 does + not narrow a backend already limited to 16, while a local backend limited to + 4 remains bounded at 4. +- Treat the default as an application-wide safety ceiling across profiles, + backends, modules, retries, and validators. A run that intentionally needs + the combined capacity of several backends may configure a higher + `total_llm` and an appropriate extract-worker count explicitly. +- Retain the existing configuration and environment override surfaces. Update + canonical configuration, operations, and internal documentation together + when the default changes. +- Reconsider decoupling the extract-worker default from `total_llm` only after + mixed-backend workloads demonstrate a need for a high global emergency + ceiling with a lower default work-production rate. + ## Shared Normalization And Quality Work ### Generic LLM-Assisted Deduplication