Add optional execution groups to conversation roadmap
This commit is contained in:
@@ -1,8 +1,9 @@
|
||||
# Conversation Continuation And Application-Directed Interaction
|
||||
|
||||
Status: behavioral decisions accepted; ready for API and staged implementation
|
||||
planning. No implementation is authorized by this document. Exact public types
|
||||
and signatures remain for implementation planning.
|
||||
Status: continuation and coordinator behavioral decisions accepted; optional
|
||||
execution groups accepted as a separable addition with open decisions below.
|
||||
No implementation is authorized by this document. Exact public types and
|
||||
signatures remain for implementation planning.
|
||||
|
||||
## North Star: Content-Neutral, Generally Useful Capabilities
|
||||
|
||||
@@ -51,6 +52,8 @@ The proposed capabilities are:
|
||||
- Allow independent continuations from one captured exchange.
|
||||
- Provide an optional sequential coordinator for bounded looping using
|
||||
application-supplied next actions and follow-up requests.
|
||||
- Provide optional execution groups for explicitly selected runs and summaries
|
||||
of their available execution facts, independently of conversation continuation.
|
||||
|
||||
Continuation has direct value by removing consumer capture, reconstruction,
|
||||
prefix comparison, and history assembly. The coordinator centralizes turn
|
||||
@@ -471,9 +474,10 @@ the operational-error behavior does not reclassify it.
|
||||
|
||||
**Status: accepted; exact placement of relationship metadata remains for
|
||||
implementation planning.** Reuse ordinary per-run accounting, expose a minimal
|
||||
parent relationship for continuations, and leave grouping of other calls and
|
||||
overall totals to applications. This applies to both standalone continuations
|
||||
and coordinated turns.
|
||||
parent relationship for continuations, and let applications select accounting
|
||||
membership explicitly. The separable execution-group addition below supplies
|
||||
optional summaries without expanding the coordinator. This applies to both
|
||||
standalone continuations and coordinated turns.
|
||||
|
||||
The current [observation boundary](../internal/observation.md) keeps nested
|
||||
invocations independent. A callback's arbitrary model calls cannot simply be
|
||||
@@ -504,10 +508,13 @@ result.
|
||||
**Accepted application boundary:** model calls made inside a callback remain
|
||||
independently observed and accounted for. PromptKit does not infer that they
|
||||
belong to a continuation or aggregate merely because they execute inside its
|
||||
callback. Applications explicitly group those existing observations when useful,
|
||||
including concurrent calls. Audita retains assembly of its overall usage report;
|
||||
PromptKit supplies execution facts without owning an application-wide reporting
|
||||
model or promising complete cost accounting.
|
||||
callback. Applications explicitly select which runs belong together, including
|
||||
concurrent calls, and may use the optional execution groups below to collect and
|
||||
summarize those facts. This revises the earlier choice to leave all aggregation
|
||||
to applications; membership and domain interpretation remain application-owned.
|
||||
Audita retains assembly of its overall usage report and mappings to its domain
|
||||
identifiers. PromptKit supplies execution facts without owning an
|
||||
application-wide reporting model or promising complete cost accounting.
|
||||
|
||||
**Alternatives considered, not selected:**
|
||||
|
||||
@@ -518,10 +525,11 @@ model or promising complete cost accounting.
|
||||
consumers would have to reconstruct continuation relationships themselves.
|
||||
|
||||
**Implementation planning:** place the minimal parent reference and preserve
|
||||
ordinary per-request observation behavior for every continuation. Broader scoped
|
||||
observation facilities or aggregate reporting require a separate demonstrated
|
||||
need. The additional-turn budget continues to bound continuation executions,
|
||||
not all provider calls performed inside those executions or application callbacks.
|
||||
ordinary per-request observation behavior for every continuation. Execution
|
||||
groups address the separately demonstrated need for explicit accounting across
|
||||
related calls; broader reporting remains outside this scope. The additional-turn
|
||||
budget continues to bound continuation executions, not all provider calls
|
||||
performed inside those executions or application callbacks.
|
||||
|
||||
### 6. Where Should Continuation Inspection Live?
|
||||
|
||||
@@ -561,12 +569,118 @@ but this feature does not select that registry. Explicit access through an
|
||||
already-held exchange satisfies the present need. Richer reporting or registry
|
||||
capabilities require separate consumer justification.
|
||||
|
||||
## Separable Addition: Optional Execution Groups
|
||||
|
||||
**Status: accepted direction; the four decisions below remain open.** Audita's
|
||||
follow-up request identifies a distinct need: a validator call can belong to the
|
||||
same logical operation as a continuation without extending its conversation.
|
||||
Conversation ancestry describes which exchange supplies history; accounting
|
||||
membership describes which executions the application chooses to report together.
|
||||
Neither relationship implies the other.
|
||||
|
||||
An application creates a group for a logical operation and explicitly associates
|
||||
selected ordinary runs or coordinated continuations with it. Calls made by a
|
||||
validator or other callback join only when the application chooses. Groups use
|
||||
existing run identities and observation/usage semantics to summarize available
|
||||
execution facts. They are usable without capture or a coordinator, and continuation
|
||||
and coordination remain usable without them. Selecting a group does not change
|
||||
execution scheduling, capacity, cancellation, retry, or validation behavior.
|
||||
|
||||
For example, an application may group a continuation and a separate validator
|
||||
run while keeping their conversations independent. It may also include the
|
||||
initial run shared by several branches. Replaying that run's captured history
|
||||
does not enroll it or count its usage again; newly reported usage for sending
|
||||
history in later executions still belongs to those later runs.
|
||||
|
||||
The group does not infer content meaning, automatically capture arbitrary nested
|
||||
calls, schedule work, determine application outcomes, or introduce a reporting
|
||||
framework. Applications retain domain identifiers, presentation, and any broader
|
||||
report assembly. Group summaries are obtained separately from the coordinator's
|
||||
minimal termination result. They preserve unavailable versus explicit zero usage
|
||||
and do not add terminal totals to the attempts they already summarize.
|
||||
|
||||
### Group Decision 1: How Does An Application Select Membership?
|
||||
|
||||
**Recommendation:** first evaluate a group collector compatible with the existing
|
||||
observer API. Attaching it explicitly to a request would enroll the observed run;
|
||||
a coordinator could receive an explicit selection for its own continuation runs.
|
||||
Callback calls would still require their own selection. This reuses the existing
|
||||
source of execution facts and avoids a second accounting path.
|
||||
|
||||
The design must allow the application to keep its existing observer alongside
|
||||
the group. Specify ordinary and prepared execution attachment and coordinator
|
||||
propagation without inheriting membership into arbitrary nested calls. Preparation
|
||||
alone is not an executed member; discarded handles and failed claims do not
|
||||
currently emit execution observations.
|
||||
|
||||
**Alternative:** explicit group options on requests and the coordinator, backed
|
||||
by ordinary observation internally. This makes membership clearer and separates
|
||||
it from application event delivery, but adds public API plumbing. Choose after
|
||||
sketching the smallest workable interfaces for ordinary, prepared, coordinated,
|
||||
and callback-initiated calls. Do not select a context-based implicit scope.
|
||||
|
||||
### Group Decision 2: What Is The Group's Lifetime?
|
||||
|
||||
**Recommendation:** expose summary snapshots from a caller-held group, with no
|
||||
mandatory close or wait operation. Applications already own the work and decide
|
||||
when to read or discard the group. Support concurrent selected executions and
|
||||
summary reads without imposing serial scheduling.
|
||||
|
||||
A snapshot must distinguish observed terminal runs from members whose terminal
|
||||
facts are not yet available. A missing terminal event can reflect ongoing work
|
||||
or interruption such as a panic; it is not evidence of zero usage or success.
|
||||
Do not promise terminal observations beyond the existing observation contract.
|
||||
|
||||
**Alternative:** an explicit seal/close or wait lifecycle could provide a fixed
|
||||
membership boundary or completion wait. It adds lifecycle rules and overlaps
|
||||
with application orchestration; defer unless a consumer demonstrates that need.
|
||||
|
||||
### Group Decision 3: May Groups Overlap?
|
||||
|
||||
**Recommendation:** allow a run to belong to more than one explicitly selected
|
||||
group, counting each run identity once within each group. This accommodates both
|
||||
an operation summary and a broader application summary without requiring a
|
||||
hierarchical group model. Overlapping group totals are not necessarily additive.
|
||||
|
||||
**Alternative:** require one group per run. This simplifies membership plumbing,
|
||||
but restricts useful accounting views and makes applications rebuild the broader
|
||||
aggregation themselves. There is no demonstrated need for that restriction.
|
||||
Do not add group nesting, automatic parent membership, or group scheduling merely
|
||||
to support overlap.
|
||||
|
||||
### Group Decision 4: Which Facts Does A Summary Expose And Aggregate?
|
||||
|
||||
**Recommendation:** start with member run identities, terminal-observation status,
|
||||
and a small selection of existing execution counters and presence-aware usage
|
||||
totals. Select the exact fields during API design; avoid a comprehensive new
|
||||
report schema or new outcome hierarchy. Summaries describe execution facts, not
|
||||
application acceptance or monetary cost. Summed run durations, if included,
|
||||
must not be presented as elapsed group time when runs overlap.
|
||||
|
||||
Prefer aggregating one terminal event per run. Terminal observations already
|
||||
summarize retries, structural repairs, reported usage, and failure-path facts,
|
||||
including executions that return no ordinary result. This is simpler than
|
||||
reconstructing those totals from attempt events. Preserve each usage field's
|
||||
known/completeness distinctions and expose missing terminal coverage separately,
|
||||
so a total over finished members cannot imply complete accounting for all members.
|
||||
|
||||
**Alternative:** accumulate attempt events for live partial totals, reconciling
|
||||
terminal events without counting the same work twice. This exposes more facts
|
||||
before completion or after a missing terminal event, but needs additional
|
||||
reconciliation rules. Prefer terminal-based totals unless live attempt accounting
|
||||
has demonstrated value. Whichever representation is selected, absent evidence
|
||||
must not become a known zero, and terminal summaries must never be added to their
|
||||
already-counted attempts.
|
||||
|
||||
## Proposed Delivery And Validation
|
||||
|
||||
All six behavioral decisions are accepted. Remaining work is API and staged
|
||||
implementation planning: exact types and signatures, capture opt-in syntax,
|
||||
message assembly details, placement of relationship metadata, and the concrete
|
||||
inspection representation. The recommended delivery sequence is:
|
||||
The six continuation/coordinator behavioral decisions are accepted. Their
|
||||
remaining work is API and staged implementation planning: exact types and
|
||||
signatures, capture opt-in syntax, message assembly details, placement of
|
||||
relationship metadata, and the concrete inspection representation. Execution
|
||||
groups have four open decisions above and a separate delivery track; resolving
|
||||
or implementing them is not a prerequisite for continuation. The recommended
|
||||
continuation delivery sequence is:
|
||||
|
||||
1. Deliver capture and independent continuation through the public facade,
|
||||
allowing Audita to remove its reconstruction and prefix-verification work.
|
||||
@@ -575,6 +689,10 @@ inspection representation. The recommended delivery sequence is:
|
||||
both consumer sketches against the selected public contracts to verify that
|
||||
the same mechanism removes useful repeated work in each application.
|
||||
|
||||
Plan execution groups independently after resolving their open decisions,
|
||||
reusing ordinary observations and preserving existing application observers.
|
||||
They can ship separately without changing the coordinator termination contract.
|
||||
|
||||
Use [Audita's acceptance scenarios](audita-request.md#suggested-delivery-and-acceptance-scenarios)
|
||||
as consumer checks, alongside analysis and branching-conversation examples that
|
||||
exercise the same APIs without repair-specific concepts. Additional proposed
|
||||
@@ -595,14 +713,23 @@ coverage should establish:
|
||||
- Assessment calls against the same backend at capacity one, without capacity
|
||||
retained by the coordinator blocking them.
|
||||
- Per-run usage and failure diagnostics available through ordinary observations,
|
||||
including incomplete usage. Consumer examples explicitly group callback calls
|
||||
and count shared initial work once, without coordinator-provided totals.
|
||||
including incomplete usage. Consumer examples explicitly select accounting
|
||||
membership and count shared initial work once, without coordinator-provided
|
||||
totals; optional groups may supply those summaries when available.
|
||||
- Immediate parent relationships distinguish successive turns and independent
|
||||
branches using existing run identities.
|
||||
- Explicit exchange details expose captured logical history and minimal identity
|
||||
metadata through ordinary encoding. Mutating returned details does not affect
|
||||
retained branch state, and inspection requires no rendering or external reads.
|
||||
|
||||
Execution-group validation should separately cover explicit ordinary, prepared,
|
||||
coordinator, and callback membership; unrelated nested calls remaining outside
|
||||
the group; independent ancestry and membership; concurrent collection alongside
|
||||
an existing application observer; failure-path terminal facts; unavailable and
|
||||
explicit-zero usage; missing terminal observations; and no attempt/terminal
|
||||
double counting. If overlapping groups are selected, verify once-per-run counting
|
||||
within each group without implying that overlapping totals can be added.
|
||||
|
||||
These are planning targets, not new implemented guarantees. Keep additional
|
||||
mechanisms and constraints tied to the selected decisions and demonstrated
|
||||
consumer needs.
|
||||
|
||||
Reference in New Issue
Block a user