KubeOn GPU overview
The two parts of KubeOn GPU, what each capability needs, and the order to roll it out in.
Early access
KubeOn GPU has two parts. The read-only agent you already run for KubeOn measures GPU capacity, requests, utilization and cost. The GPU optimizer is optional: a separate Helm release, installed only in clusters where you want GPU optimization, that acts through the policies you enable. Without the optimizer, every GPU finding is a proposal your pipeline applies.
What each capability needs
| Capability | Needs | Changes your cluster |
|---|---|---|
| GPU cost and idle GPU time | The agent, and your billing export or a rate card | No |
| GPU utilization and memory | The NVIDIA DCGM exporter, scraped by Prometheus. See connect GPU metrics | No |
| Proposals for sharing, sizing and schedules | The two rows above | No |
| GPU sharing: MIG and time-slicing | The GPU optimizer and the NVIDIA GPU Operator | Yes, within a Sharing policy |
| GPU autoscaling, packing and spot | The GPU optimizer, and Karpenter or your cloud's managed node groups | Yes, within an Autoscaling policy |
| Scale to zero | The GPU optimizer and DCGM metrics | Yes, within a ScaleToZero policy |
| GPU burst | The GPU optimizer, credentials for each location, and the burst gateway. See set up GPU burst | Yes, within a Burst policy |
Modes
| Mode | What happens |
|---|---|
| Recommend | The optimizer writes the change as a proposal with its evidence and patch. Your pipeline applies it, as with every other KubeOn saving. Every policy starts here. |
| Approve | Each change waits for a named reviewer. Once approved, the optimizer applies it in the policy's change window and records the outcome. |
| Automatic | The optimizer acts on its own, but only on the node pools and namespaces the policy names, within its node, cost and time limits. |
Recommend mode is in every edition. Approve and Automatic modes need the Optimize edition, which includes burst to other regions of the same cloud. Burst across clouds and into your own data centers needs Enterprise. See KubeOn GPU pricing.
Roll it out in this order
- Connect GPU metrics and let KubeOn collect at least 14 days of history, so peaks and idle periods are real.
- Review GPU cost, idle GPU time and the proposals for each GPU node pool with the teams that own them.
- Install the optimizer in one non-production cluster. Every policy starts in Recommend mode.
- Write the first policies with tight limits. Compare their proposals with what your team would have done.
- Move one policy at a time to Approve, then to Automatic, starting with scale to zero in development namespaces.
- Repeat for production node pools, and add burst last, once its locations and credentials are agreed.
Guardrails
- Installed only in clusters you choose, as its own Helm release, separate from the read-only agent
- Acts only on the node pools and namespaces a policy names; everything else is out of reach
- Hard limits per policy: minimum and maximum nodes, monthly cost, GPU-hour price
- Disruptive steps such as drains and repartitions run only in the policy's change window
- Evicts through the Eviction API, so PodDisruptionBudgets are always respected
- No access to Secrets or ConfigMaps outside the GPU Operator config it manages
- Pulls policies and approvals from the hub over HTTPS; the hub never connects into your cluster
- Every action in the audit log with the policy that allowed it, and a one-click revert where the change can be undone
Next: how the optimizer decides, with the rule behind every action.