With load_balance enabled, a run was pinned to a participant at queue time
by comparing worker queue depths, so a faster GPU went idle while a slower
one still held a backlog. Runs queued while every participant is busy now
stay unassigned in the master's prompt queue; a dispatcher watches the
candidate workers and hands the oldest waiting run to whichever one frees
up first, re-queueing the master side as a delegate prompt under the same
prompt id. A participating master takes whatever waiting run its executor
reaches first. Idle participants are still used immediately.
Upscale workflows and serverless pools keep the previous queue-depth
selection.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
