GPU burst
GPUs from wherever they are free, inside the borders you set
When the home region has no GPUs of the type your pods need, the GPU optimizer looks in the locations a policy allows: other regions, other clouds and idle GPUs in your own data center. It attaches GPU nodes there, runs only the pods that opted in, and moves them home when capacity returns.
GPU burst workflow
Who does what
From a region with no GPUs to a pod that runs
Follow the inputs, decision and output for this workflow.
- KubeOn, automated
- Your decision
- Your systems and people
- Only path to a change or purchase
- Step 1, Detect, by GPU optimizer:
GPU pods that opted into burst stay pending because the home region has no capacity of their GPU type.
- Step 2, Compare, by KubeOn hub:
Check availability and GPU-hour price, data transfer included, in each location the policy allows.
- Step 3, Approve, by Your reviewers:
Approve the location and node count, or let an Automatic policy choose within its limits.
- Step 4, Attach + return, by GPU optimizer:
Add GPU nodes there, joined to the cluster, then move pods home and remove them when capacity returns.
Burst uses only the locations a policy lists, and only for pods that tolerate the burst taint. Pods drain home when capacity returns.
Read the workflowWho does what
From a region with no GPUs to a pod that runs
Follow the inputs, decision and output for this workflow.
- 01
Detect
GPU pods that opted into burst stay pending because the home region has no capacity of their GPU type.
GPU optimizer
- 02
Compare
Check availability and GPU-hour price, data transfer included, in each location the policy allows.
KubeOn hub
- 03
Approve
Approve the location and node count, or let an Automatic policy choose within its limits.
Your reviewers
- 04
Attach + return
Add GPU nodes there, joined to the cluster, then move pods home and remove them when capacity returns.
GPU optimizer
Burst uses only the locations a policy lists, and only for pods that tolerate the burst taint.
Read the workflowThe problem
The GPUs you need are not where you run
Regional shortages
Popular GPU types sell out in a region, and pods stay pending while demand waits.
Owned GPUs sit idle
A data center has spare A100s while the cloud cluster next to it cannot get a GPU.
Residency rules
Capacity in any region is not an option when data must stay in the EU.
Hidden transfer cost
A cheaper GPU elsewhere can cost more once data transfer back is counted.
Only the pods that opt in
Burst nodes run only pods that tolerate them
Burst nodes carry a taint, so only pods with the matching toleration can run there. Stateless inference and batch opt in; databases and anything that must stay in the home region do not. Each pod's GPU requirements, such as GPU memory, decide which locations can serve it.
- Taint gpu.kubeon.io/burst on every burst node
- Requirements from the pod's GPU request and affinity
- Nothing moves that did not opt in
apiVersion: gpu.kubeon.io/v1kind: GpuPolicymetadata: name: inference-burst-euspec: kind: Burst mode: Approve workloads: tolerate: gpu.kubeon.io/burst # only pods that opt in locations: # nothing outside this list, ever - aws/eu-central-1 - gcp/europe-west4 - onprem/dc1 limits: maxNodes: 4 maxGpuHour: 4.50 # USD, data transfer included returnHome: true # drain burst nodes when home capacity is backAllowed locations
Cheapest allowed location, data transfer included
The optimizer compares the locations the policy lists: availability of the GPU type, GPU-hour price at your rates or rate card, and an estimate of data transfer back to the home region. Owned GPUs that sit idle come first, then the cheapest cloud location that has capacity.
- Other regions, other clouds and your data centers
- Locations outside the policy are never used
- EU-only policies keep every node in the EU
Attach and return
Nodes join your cluster, then leave when you are home again
Burst nodes are created with the cloud credentials you grant for each location, scoped to the instances the optimizer creates, and join your cluster over an encrypted tunnel. Your control plane, workloads and tooling stay where they are. When home capacity returns, pods drain back and the burst nodes are removed.
- Your existing cluster, no second control plane
- Encrypted connection between locations
- Burst nodes removed when no longer needed
Burst rules
When the optimizer bursts, and where it stops
| Rule | Detail |
|---|---|
| Trigger | Pods that tolerate the burst taint are pending because no allowed GPU type is available in the home region |
| Order | Idle owned GPUs, then the cheapest allowed cloud location with capacity, data transfer included |
| Limits | Maximum burst nodes and maximum GPU-hour price per policy |
| Residency | Only the locations the policy lists; a location missing from the list is never used |
| Return | When home capacity is available again, pods drain back within PodDisruptionBudgets and burst nodes are removed |
| Approval | Approve mode asks before each burst; Automatic acts within the limits |
Latency and data transfer between locations are real costs. Burst suits stateless inference and batch work best, and each burst shows its transfer estimate before it starts.
Capabilities
More GPU capacity without a second cluster
Cross-location nodes
Other regions, clouds and your data centers.
Residency by policy
Locations outside the list are never used.
Transfer priced
GPU-hour with data transfer back included.
Owned GPUs first
Idle data center GPUs before new cloud spend.
Opt-in workloads
Only pods that tolerate the burst taint move.
Return home
Burst nodes removed once home capacity is back.
FAQ
GPU burst, answered
No. Burst nodes join your existing cluster, so your control plane, deployments, monitoring and tooling stay the same. Only pods that tolerate the burst taint are scheduled onto them.
Related
Evaluate GPU burst with your team.
Bring your platform and FinOps priorities. We will walk through the data, controls and evidence for your evaluation.