KubeOn GPU
Every GPU-hour priced, owned and put to work
KubeOn GPU splits GPU node cost by the GPUs each team's pods requested, prices the GPUs nobody used, and measures how hard the rest worked. Then it proposes sharing, smaller node pools and schedules, for cloud GPUs and the ones in your own data center.
How KubeOn GPU works
Who does what
From a GPU-hour to a GPU put to work
Billing or a rate card prices every GPU. Requests and DCGM metrics show who held it and how hard it worked.
- KubeOn, automated
- Your systems and people
- Step 1, Price, by Billing or rate card:
Billed, net amortized GPU node cost in the cloud, or a GPU-hour rate card for servers you own.
- Step 2, Measure, by Agent + Prometheus:
GPU requests per pod from the Kubernetes API, and utilization and GPU memory from DCGM.
- Step 3, Propose, by KubeOn hub:
Idle GPU time, sharing, node pool size and schedules, each priced and written as a proposal.
- Step 4, Apply + verify, by Your pipeline:
Your pipeline applies the approved change. The next month's GPU cost shows what it saved.
KubeOn reads and proposes. Sharing, node pool and schedule changes go through your team and pipeline.
Read the workflowWho does what
From a GPU-hour to a GPU put to work
Billing or a rate card prices every GPU. Requests and DCGM metrics show who held it and how hard it worked.
- 01
Price
Billed, net amortized GPU node cost in the cloud, or a GPU-hour rate card for servers you own.
Billing or rate card
- 02
Measure
GPU requests per pod from the Kubernetes API, and utilization and GPU memory from DCGM.
Agent + Prometheus
- 03
Propose
Idle GPU time, sharing, node pool size and schedules, each priced and written as a proposal.
KubeOn hub
- 04
Apply + verify
Your pipeline applies the approved change. The next month's GPU cost shows what it saved.
Your pipeline
KubeOn reads and proposes. Sharing, node pool and schedule changes go through your team and pipeline.
Read the workflowThe problem
Allocated is not used
GPUs are the fastest-growing line on many cloud bills, and published research on shared clusters shows how much of that capacity sits idle or barely busy.
14%
of EC2 compute spend went to GPU instances across Datadog's customers, up from 10% a year earlier
Datadog, State of Cloud Costs 2024~52%
average utilization of the GPUs in use in a large multi-tenant training cluster at Microsoft
Jeon et al., USENIX ATC 20190.042
of a GPU: the median task's use in Alibaba's production cluster of over 6,000 GPUs
Weng et al., USENIX NSDI 202274% / 18%
of GPUs allocated, against average GPU utilization, on one group of Alibaba machines
Weng et al., USENIX NSDI 2022Measure
Three numbers for every GPU node
Each has a different fix, so KubeOn reports them separately, per node pool, team and day.
01
Allocated GPU-hours
How many GPUs each team's pods held, and for how long, priced at the node's billed rate or your rate card.
02
Idle GPU-hours
GPU capacity no pod requested. A scheduling problem, fixed with fewer nodes, a smaller pool or a schedule.
03
Utilization of allocated GPUs
Whether the GPUs pods hold are doing work, from DCGM. A sizing problem, fixed with sharing or a smaller GPU.
Product
Four areas, one GPU cost model
GPU sharing
Give small workloads a slice, not a whole card
Workloads whose peak GPU memory fits in part of a card get a sharing proposal: a MIG partition on GPUs that support it, where isolation matters, or time-slicing for notebooks and development. Each one states what is isolated and how many GPUs it frees.
- Smallest valid MIG profile for each workload
- Time-slicing for non-production by default
- GPU Operator changes written out for review
GPU capacity
Node pools sized from peak GPU requests
Pools where whole GPU nodes stay empty at the busiest hour get a smaller size, models that fit a smaller GPU are flagged for a benchmark, and development GPU nodes are switched off outside working hours.
- 14 days of peak hourly GPU requests
- GPU memory fit with 30% headroom
- Off-hours for non-production GPU nodes
Where it works
Cloud GPUs and the ones you own
Cloud GPUs are priced from your billing export. GPU servers in your own data center are priced from a rate card, and a self-hosted hub keeps every snapshot in your network.
- Amazon EKS
- Azure Kubernetes Service
- Google Kubernetes Engine
- Red Hat OpenShift
- Kubernetes on VMware or bare metal
- NVIDIA GPU Operator and device plugin
- NVIDIA DCGM exporter
- Karpenter and Cluster Autoscaler
Control
KubeOn measures and proposes. Your pipeline applies.
Read-only agent
GPU capacity and requests come from the same read-only data the agent already collects. DCGM metrics are read from your Prometheus.
Proposals, not automation
MIG profiles, time-slicing, node pool sizes and schedules arrive as proposals with a risk level, evidence and a patch.
Self-hosted when needed
Run the hub in your own data center, air-gapped if required, with rate cards instead of a cloud bill.
FAQ
KubeOn GPU, answered
Shares owners, budgets and sign-in with KubeOn and KubeOn AI.
No. It uses the same read-only agent and workspace as KubeOn. GPU cost appears in cost allocation, and GPU findings appear with the rest of your savings, as change proposals.
See what your GPUs cost, and how hard they work.
A 30-minute walkthrough on your own GPU node pools, in the cloud or in your data center.