Concurrency keys consolidated to prevent master queue starvation
Queues with concurrency keys now use a single wildcard entry in the master queue, preventing high-volume tenants from locking out others on the same shard.
Previously, tenants utilizing thousands of concurrency keys (CKs) could flood the master queue. This behavior consumed the entire parentQueueLimit capacity, inadvertently starving other tenants assigned to the same shard.
The run engine now consolidates all concurrency keys for a base queue into a single wildcard entry (e.g., :ck:*) within the master queue. A dedicated, per-queue index tracks the active sub-queues behind the scenes. When processing the master queue, the system dequeues from the wildcard entry by round-robining across these active sub-queues.
This change guarantees fair processing resources for all tenants in the run engine, regardless of their concurrency key volume. Because existing entries drain naturally, the new queuing logic rolls out without requiring database migrations or downtime.
View Original GitHub Description
Queues with concurrency keys now appear as a single entry in the master queue instead of one entry per key. This prevents high-CK-count tenants from consuming the entire parentQueueLimit window and starving other tenants on the same shard.
A new per-queue CK index (sorted set) tracks active concurrency key sub-queues. The master queue gets one :ck:* wildcard entry per base queue. Dequeuing from that entry round-robins across sub-queues, maintaining per-CK concurrency tracking and fairness.
All existing operations (enqueue, dequeue, ack, nack, DLQ, TTL expiry) are CK-index-aware and keep the index consistent. Old-format entries drain naturally during rollout — no migration step needed, single deploy.