GPU cost and optimization
How KubeOn prices and allocates GPU nodes, measures utilization, and finds sharing, capacity and data center savings.
KubeOn GPU uses the same agent, billing data and proposals as the rest of KubeOn. This guide covers what it reads, how GPU cost is allocated, and the rule behind each GPU finding.
What KubeOn reads
| Input | Source | Needed for |
|---|---|---|
| GPU capacity per node | nvidia.com/gpu and MIG resources in node status, from the Kubernetes API | Cost, idle GPU time |
| GPU requests per pod | Container limits for nvidia.com/gpu and nvidia.com/mig-* resources | Cost, idle GPU time, capacity |
| GPU node cost | Net amortized cost from your billing export, or a GPU-hour rate card | Cost |
| Utilization and GPU memory | NVIDIA DCGM exporter series in your Prometheus | Utilization, sharing, GPU type fit |
Allocation
- Each GPU node's billed, net amortized cost is split by the GPUs pods requested, hour by hour.
- A MIG slice carries the share of the GPU's compute slices it holds: a
2g.10gbslice on an A100 40GB carries two sevenths of that GPU's hourly cost. - GPU capacity no pod requested is idle GPU time, priced at the node's rate and kept in its own column. Your idle policy decides whether it stays with the platform team or is spread.
- Ownership, effective dates and reports work exactly as they do for CPU and memory. See the cost allocation methodology.
Utilization
KubeOn reads DCGM_FI_DEV_GPU_UTIL, DCGM_FI_DEV_FB_USED and DCGM_FI_DEV_FB_FREE per GPU, and DCGM_FI_PROF_SM_ACTIVE where profiling metrics are enabled, and maps each GPU to the pod that held it.
Read GPU utilization with care
DCGM_FI_DEV_GPU_UTIL is the share of time any kernel was running, so a GPU can report high utilization while most of its streaming multiprocessors are idle. Where SM activity is available, KubeOn shows it next to GPU utilization.Sharing
| Finding | Rule |
|---|---|
| MIG partition | Workloads holding a whole MIG-capable GPU whose peak GPU memory, plus headroom, fits a smaller valid profile for that GPU model. Workloads are packed onto as few GPUs as possible; the GPUs freed are the saving. |
| Time-slicing | Non-production workloads holding a whole GPU at low utilization, on any NVIDIA GPU. Replicas per GPU are chosen so the workloads' peak GPU memory fits together. |
- MIG proposals carry the GPU Operator node label (
nvidia.com/mig.config) and the pods' new MIG resources. Applying a MIG profile reconfigures the GPU, so the rollout steps drain the node's GPU pods first. - Time-slicing proposals carry the device plugin config map and node label. Pods sharing a time-sliced GPU share its memory with no isolation, so the risk is marked on every proposal.
- On owned hardware there is no bill to cut. The saving is reported as GPUs freed, at your rate card.
Capacity
| Finding | Rule | Saving |
|---|---|---|
| Scale a GPU node pool | Peak hourly GPU requests over 14 days fit on fewer nodes, leaving at least one whole node free | Billed cost of the nodes removed |
| Smaller GPU type | Peak GPU memory plus 30% fits a smaller GPU class | Not counted until your team benchmarks it |
| GPU off-hours | GPU nodes in dev, test and sandbox clusters, scaled down outside 07:00 to 20:00 on weekdays | 103 of 168 hours a week of the node's cost |
| Idle GPU node | No GPU requested on the node for 7 days | The node's full billed cost |
A 14-day window does not contain a quarterly training run or a seasonal peak. Reject proposals for pools whose busiest days fall outside it, and check commitment coverage: removing a node covered by a Savings Plan or reservation moves the commitment to other usage rather than cutting the bill.
Data centers
GPU servers you own are priced with a rate card: a gpuHour rate per node pool or label, or amortized hardware (purchase price over the depreciation period, plus monthly power, space and support). See on-premises pricing for the rate card format.
- Owned GPUs are allocated by request like cloud GPUs, with idle owned GPUs reported at the rate card.
- Sharing and capacity findings on owned servers report GPUs freed, which is capacity to use before the next hardware order.
- A self-hosted hub runs in the data center with images from your registry and needs no outbound access.
Who applies changes
KubeOn never changes a GPU node, a MIG profile or a node pool. Each finding is a change proposal with evidence, a risk level and a patch. Your team approves it, your pipeline applies it, and next month's GPU cost shows the result.