Preserve comparison failures during cancellation

This commit is contained in:
2026-08-13 03:26:11 +00:00
parent 79cba800ee
commit 5e492cf1fb
10 changed files with 167 additions and 15 deletions

View File

@@ -47,7 +47,8 @@ stop its peers, while caller cancellation applies to every in-flight execution.
Weatherreporter starts selected profile executions concurrently and does not
add an application-level concurrency limit. Promptkit owns backend capacity and
any profile or backend concurrency policy. The durable comparison output and
any profile or backend concurrency policy. A shared Weatherreporter executor
must safely accept those concurrent `Execute` calls. The durable comparison output and
its compatibility rules are defined by the
[comparison bundle contract](comparison-bundle.md); the user-facing command
contract is in the [CLI reference](../cli.md).

View File

@@ -45,8 +45,10 @@ and hash. Artifact paths are added only after publication commits.
The comparison execution core starts each inspected profile independently,
keeps results in selection order, and waits for all started work. Every profile
reconciles its callback and completion provenance before its JSON can be
rendered. Independent profile failures are recorded and do not stop peers.
Context cancellation marks unfinished work and prevents publication. Details of
rendered. Independent profile failures are recorded and do not stop peers; a
completed profile failure remains recorded if cancellation happens later.
Context cancellation marks only unfinished or cancellation-terminated work and
prevents publication. Details of
prepared values, execution and debugging, and publication are documented in [prepared report
internals](prepared-report.md), [comparison execution
internals](comparison-execution.md), and [comparison publication

View File

@@ -9,9 +9,12 @@ selection order even though execution completes in an arbitrary order.
Every profile uses the exact inspected prompt identity and a private copy of
the same prepared data package. Provider, generated-text validation, rendering,
or debug-write failure becomes that profile's safe failed outcome and does not
cancel its peers. The application deliberately imposes no additional semaphore:
Promptkit owns backend capacity. Cancellation or a deadline marks unfinished
outcomes as skipped or failed, joins work, and prevents bundle publication.
cancel its peers. The shared executor must support those concurrent `Execute`
calls. The application deliberately imposes no additional semaphore: Promptkit
owns backend capacity. A profile failure completed before a later cancellation
remains its original safe outcome; cancellation or a deadline marks only
unfinished or cancellation-terminated outcomes as skipped or failed, joins work,
and prevents bundle publication.
When debugging is enabled, each execution receives a deterministic reference
derived from the comparison identity, ordered profile position, and safe