Kubernetes 1.37 Garhwal: GPU Scheduling Overhaul for AI Workloads

Marcus Chen Staff Engineer (Backend & Cloud), YuSMP Group · Cloud infrastructure and Kubernetes for US and EU engineering teams

The short answer

What shipped in Kubernetes 1.37

Kubernetes releases roughly every four months; 1.37 "Garhwal" (named after the Himalayan region in Uttarakhand) arrived on 26 August 2026. The 67-enhancement count breaks down as 16 Stable, 23 Beta, and 27 Alpha, plus one deprecation announcement. Kubernetes tracks enhancement maturity carefully — Stable means the API is locked and safe for production, Beta means the API may change in minor ways but is ready for broad testing.

The release theme is GPU and accelerator workload efficiency. Both headline features — DRA device taints and gang scheduling — address failure modes that teams running large language model training or inference workloads hit repeatedly: a broken GPU node silently blocking job progress, and partial pod placement wasting accelerator budget. The Kubernetes engineering community has treated these as first-class problems since the AI compute wave; 1.37 is the release where the control-plane answers graduate out of Alpha.

DRA device taints: routing around broken GPUs at scale

Dynamic Resource Allocation (DRA) is Kubernetes's framework for managing accelerators and specialty hardware. In 1.37, DRA gains device taints at Stable — the same taint/toleration mechanism that cluster operators already use for node-level scheduling constraints, applied directly to individual GPU or accelerator devices.

In practice, a device plugin (or the DRA driver) can mark a specific GPU as NoSchedule or NoExecute when it detects hardware errors, ECC memory failures, or thermal throttling. The scheduler then refuses to place new pods that request that device on the affected node, and existing pods without a matching toleration can be evicted. The broken GPU is bypassed automatically; no human needs to drain the node or manually cordon it.

Why does this matter at scale? In large training clusters — think 64 to 256 H100 or B200 nodes — a single degraded GPU can cause an entire multi-node training job to stall or produce incorrect results silently. Before DRA device taints, operators had to monitor GPU health out-of-band, manually intervene, and requeue jobs. With Stable DRA device taints, the control plane handles it: the faulty device is flagged, new work is routed around it, and the job continues on healthy hardware. For teams paying $2–3 per GPU-hour, automating this check translates directly into reclaimed compute budget.

Gang scheduling: all-or-nothing pod placement

Gang scheduling — or workload-aware preemption — addresses the "partial placement" problem: a distributed training job requests N GPU worker pods, but only N-2 can be scheduled because two nodes are busy. Without gang scheduling, those N-2 workers start, claim their GPU memory, and sit idle waiting for the stragglers. With gang scheduling, the scheduler either places all N pods simultaneously or none of them, queuing the job until capacity is available.

In Kubernetes 1.37, gang scheduling via workload-aware preemption graduated to Beta on scheduling.k8s.io/v1beta1. The PodGroup API lets teams annotate a set of pods as a logical unit; the scheduler treats their placement as atomic. If a higher-priority job needs the cluster's resources, the pending PodGroup is preempted as a whole rather than having individual pods scattered across the eviction queue.

The operational benefit is significant: training jobs no longer leave GPUs idle while waiting for straggler pods, and the cluster's overall accelerator utilization improves because resources are not silently held by half-placed jobs. Previously, achieving this required deploying external schedulers like Volcano or Yunikorn. In 1.37, it is available in-core, meaning no additional control-plane component to version, monitor, or upgrade alongside the cluster.

Deprecations with hard deadlines

Kubernetes 1.37 ships two deprecations that carry firm removal timelines — not "may be removed eventually" language, but actual release targets.

kube-dns deprecated; removal in Kubernetes 1.40. CoreDNS has been the default DNS implementation since Kubernetes 1.13 (2019). kube-dns survived as an optional alternative for clusters that pre-date that switch. With 1.37, the project formally deprecates it and sets 1.40 as the removal release. For teams on managed Kubernetes services — EKS, GKE, AKS — this is likely a non-issue; those services have defaulted to CoreDNS for years. Self-managed clusters running 1.12-era kube-dns installations should audit their DNS deployment before the next upgrade past 1.39.

IPVS kube-proxy backend deprecated; removal in Kubernetes 1.43. IPVS replaced iptables as a high-performance option for kube-proxy at large cluster scale. The nftables backend, which graduated to Stable in Kubernetes 1.33, is now the recommended successor. nftables requires Linux 5.13 or newer; teams should verify their kernel version before migrating. The 1.43 removal timeline gives several release cycles, but migrating nftables requires a kube-proxy restart on every node — plan the change, do not let it sneak up.

What it means for US & EU software teams

For teams building AI/ML products on cloud infrastructure, the Kubernetes 1.37 GPU improvements land at exactly the right moment. Enterprise AI spending on Kubernetes-based inference and training infrastructure has grown sharply through 2026, and the bottleneck has shifted from model capability to operational efficiency: how to keep expensive GPUs busy, how to handle hardware failures without manual intervention, and how to minimize idle time in training queues.

DRA device taints and gang scheduling together close two of the most common causes of GPU waste. One automates the failure response (broken device flagged, workload rerouted); the other removes the scheduling fragmentation that leaves partial jobs idling. Teams that run distributed training in production will feel both improvements without writing any new infrastructure code — upgrade the cluster, configure the DRA driver to emit taints, and define PodGroups for batch training workloads.

The deprecation news is more about operational hygiene than immediate action. A cluster still running kube-dns has two more release cycles before 1.40 arrives; the right response is to add the migration to the next infrastructure sprint, not treat it as an emergency. The nftables migration for IPVS users is even further out (1.43), but the kernel version requirement (Linux 5.13+) means some older VMs or bare-metal hosts will need an OS upgrade first — start that dependency check now.

For FinTech and regulated industries in the EU, the Kubernetes release cycle also intersects with DORA compliance: Article 28 requires documented change-management procedures for critical ICT infrastructure. Kubernetes upgrades that touch DNS or networking components are the kind of change that needs to be logged, tested in a lower environment, and signed off — not rushed. 1.37 gives enough runway to do that properly for both the kube-dns and IPVS migrations.

Three action items before 1.38

1. Audit your DNS stack. Run kubectl get pods -n kube-system -l k8s-app=kube-dns. If you see kube-dns pods, plan the CoreDNS migration before your next upgrade past 1.39. The migration guide in the Kubernetes docs is straightforward for most clusters; the risk is in clusters with custom stub-zone configurations that need to be ported.
2. Enable DRA device taints in your GPU node pools. Check whether your NVIDIA, AMD, or Google device plugin version supports emitting DRA taints. NVIDIA GPU Operator 24.x and later support the Stable DRA API. Update the device plugin before upgrading to 1.37 so taint emission is active from the first day on the new release.
3. Test gang scheduling for your largest training jobs. Deploy a staging cluster on 1.37 and annotate your multi-worker training jobs with scheduling.k8s.io/pod-group. Measure GPU utilization before and after enabling gang scheduling — the idle reduction is typically measurable within the first full job cycle.

Frequently asked questions

Sources

Kubernetes Project — Kubernetes v1.37 Release, 26 August 2026 (primary source)
The Register — Kubernetes cleans house, bins legacy kube-dns, IPVS and cgroup v1, 26 August 2026
Cloud Native Now — Kubernetes v1.37 Enhances Dynamic Resource Allocation, 2026