A constraint-aware GPU allocator improved cluster utilization by up to 33 percentage points over first-in, first-out scheduling while doubling priority-weighted output on identical hardware, according to benchmark data published by Dharma AI on Hugging Face.
Researchers Gabriel Pimenta de Freitas Cardoso, Francisco de Almeida Rocha Alves, and Bruno Duarte evaluated the system across seven test scenarios on clusters ranging from 8 to 64 GPUs. Across five contended benchmark environments, cluster utilization increased from a 52–85% baseline band to a 72–88% band, while priority-weighted output rose between 24.6% and 105.1%.
Allocation constraints
Traditional first-in, first-out schedulers reserve a static block of GPUs covering each application's maximum daily peak demand, leaving idle capacity locked away from batch workloads during traffic troughs. The baseline configurations held roughly half the cluster in static reservations, recording 51.6% utilization in a mixed control test and 53.6% in a training-heavy setup.
Dharma AI structured the scheduling problem around five legal operating rules:
- A GPU serves at most one job per timestep.
- Every job adheres to its demand range while active jobs are inherited and preserved.
- Batch workloads occupy contiguous GPU blocks sized to powers of two.
- Real-time jobs obey a hard limit on GPU swaps between consecutive timesteps.
- Running jobs cannot be interrupted.
The scheduler evaluates real-time inference as an elastic demand curve rather than a static peak, allowing batch workloads to run during demand troughs. Its objective function prices unmet real-time demand with a penalty 5 to 10 times higher than the reward weight for scheduling batch tasks, enforcing latency requirements within the same mathematical optimization.
Benchmark performance
The largest efficiency improvement occurred in a training-heavy scenario with 16 jobs on 8 GPUs. Utilization rose from 53.6% under FIFO to 87.0% with the allocator, while priority-weighted value increased by 105.1%.
In an 8-GPU mixed control benchmark with 10 jobs, utilization moved from 51.6% to 72.4%, yielding a 54.8% increase in priority value. An oversubscribed setup with 9 jobs on 8 GPUs saw utilization climb from 85.4% to 87.5%, lifting value by 33.6%.
In a scale test across 64 GPUs running 30 jobs, both FIFO and the allocator recorded identical 44.9% utilization and completed 27 jobs. However, the allocator delivered 15.9% more priority-weighted value by placing higher-value tasks first. In a 14-GPU uniform-priority test where all job values were identical, the system still increased utilization from 76.8% to 87.5% and value by 23.1% through placement planning alone.
Forecasting and execution
The scheduler feeds allocation models with distinct workload predictors. A training forecaster uses 22 features across 10 training variants to capture differences between parameter-efficient fine-tuning and full model runs. A quantization forecaster models requirements based on calibration tiers and algorithms like bitsandbytes, AWQ, and GPTQ, while real-time inference demand is projected using hourly traffic profiles.
The allocator runs in 1 to 2 milliseconds on 8-GPU scenarios and 15 milliseconds on a 64-GPU cluster. The architecture optimizes a 24-hour horizon, commits allocations for the immediate timestep, and re-executes every 30 to 60 minutes to absorb forecasting variance.
