Install the GPU optimizer

Prerequisites, the Helm install, values, verification, permissions and network access for the KubeOn GPU optimizer.

On this page

Prerequisites
1. Create a cluster token
2. Install with Helm
3. Verify
Permissions
Network

Install the optimizer in each cluster where you want GPU optimization. It is separate from the read-only agent, which stays as it is. Every policy starts in Recommend mode, so installing it changes nothing until you move a policy to Approve or Automatic.

Prerequisites

Requirement	Why
The KubeOn agent in the same cluster	Cost, requests and history come from the agent's snapshots
NVIDIA GPU Operator	For sharing: MIG profiles and the time-slicing config are applied through it
Karpenter, or managed node groups on EKS, AKS or GKE	For autoscaling and packing: GPU capacity is added and removed through them
DCGM exporter and Prometheus	For scale to zero, sharing and GPU type fit. See connect GPU metrics

1. Create a cluster token

In KubeOn, open GPU > Optimizer > Add cluster. Each cluster gets its own token, shown once when it is created: copy it then. It lets that cluster's optimizer pull its own policies and approvals and report its actions, nothing else. Store it as a Secret in the optimizer's namespace.

Store the token

kubectl create namespace kubeon-gpukubectl -n kubeon-gpu create secret generic kubeon-gpu-optimizer-token \  --from-literal=token="<cluster token>"

Rotate or revoke the token

Rotate: in GPU > Optimizer, choose the cluster and Rotate token. Update the Secret with the new token, then restart the optimizer. The old token stops working once the new one is first used, or after 24 hours.
Revoke: choose Revoke token. That cluster's optimizer stops at once: it can no longer pull policies or approvals, so it takes no further action, and the cluster shows as disconnected.
Every token creation, rotation and revocation is recorded in the audit log with who did it.

2. Install with Helm

Install

helm upgrade --install kubeon-gpu-optimizer kubeon/kubeon-gpu-optimizer \  --namespace kubeon-gpu --create-namespace \  --set cluster=prod-use1-a \  --set hub.url=https://acme.kubeon.io \  --set hub.existingSecret=kubeon-gpu-optimizer-token \  --set defaultMode=Recommend     # every policy starts as a proposal

Values

# kubeon-gpu-optimizer.yamlcluster: prod-use1-ahub:  url: https://acme.kubeon.io          # or your self-hosted hub  existingSecret: kubeon-gpu-optimizer-token  pollInterval: 60s                    # policies are pulled, never pusheddefaultMode: Recommend                 # mode for policies that do not set onecapacity:  provider: karpenter                  # karpenter | eks-nodegroups | aks-nodepools | gke-nodepoolsprometheus:  url: http://prometheus-server.monitoring.svc:9090gpuOperator:  namespace: gpu-operator  timeSlicingConfigMap: time-slicing-configburst:  enabled: false                       # see "Set up GPU burst"resources:  requests: { cpu: 100m, memory: 256Mi }  limits: { memory: 512Mi }

Value	Default	Meaning
`hub.pollInterval`	`60s`	How often the optimizer pulls policies and approvals from the hub
`defaultMode`	`Recommend`	The mode for policies that do not set one
`capacity.provider`	`karpenter`	How GPU nodes are added and removed: Karpenter, or the managed node groups of EKS, AKS or GKE
`prometheus.url`	none	Where DCGM metrics are read. Without it, scale to zero and sharing stay in Recommend mode
`burst.enabled`	`false`	Turns on the burst gateway. See set up GPU burst

3. Verify

Check the optimizer

kubectl -n kubeon-gpu get deploy kubeon-gpu-optimizerkubectl get gpupolicies                 # every policy the optimizer enforces, with its modekubectl -n kubeon-gpu logs deploy/kubeon-gpu-optimizer | grep "policies synced"

Policies are defined in KubeOn, in the UI or through the API, and synced into the cluster as GpuPolicy resources, so kubectl get gpupolicies always shows exactly what the optimizer enforces. Edits made directly to those resources are overwritten on the next sync.

Permissions

ClusterRole kubeon-gpu-optimizer

apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata:  name: kubeon-gpu-optimizerrules:  # Read what it optimizes  - apiGroups: [""]    resources: [nodes, pods, namespaces]    verbs: [get, list, watch]  - apiGroups: [apps]    resources: [deployments, statefulsets, replicasets]    verbs: [get, list, watch]  - apiGroups: [policy]    resources: [poddisruptionbudgets]    verbs: [get, list, watch]  # Label, taint and cordon GPU nodes; drain through the Eviction API  - apiGroups: [""]    resources: [nodes]    verbs: [patch]  - apiGroups: [""]    resources: [pods/eviction]    verbs: [create]  # Add and remove GPU capacity through Karpenter  - apiGroups: [karpenter.sh]    resources: [nodepools, nodeclaims]    verbs: [get, list, watch, create, patch, delete]  # Its own policies, synced from the hub, and their status  - apiGroups: [gpu.kubeon.io]    resources: [gpupolicies, gpupolicies/status]    verbs: [get, list, watch, create, update, patch, delete]---# Created only in namespaces named by a ScaleToZero policykind: Rolerules:  - apiGroups: [apps]    resources: [deployments/scale, statefulsets/scale]    verbs: [get, patch]---# In the gpu-operator namespace, for time-slicing onlykind: Rolerules:  - apiGroups: [""]    resources: [configmaps]    resourceNames: [time-slicing-config]    verbs: [get, patch]

No access to Secrets, apart from the token Secret mounted into its own pod.
ConfigMaps only in the GPU Operator's namespace, and only the time-slicing config named in its values.
Workload scaling only in namespaces a ScaleToZero policy names, through a Role created in each of them.
The pod runs as a non-root user with a read-only root filesystem and every capability dropped.

Network

From	To	Port	When
Optimizer	Your hub, or your workspace on `kubeon.io`	443	Every poll: pull policies and approvals, report actions
Optimizer	Kubernetes API server	443	Watching nodes and pods, applying changes
Optimizer	Prometheus in the cluster	9090	Reading DCGM metrics
Optimizer	Your cloud's APIs (EC2, Azure Compute, Compute Engine)	443	Only with managed node groups or burst
Burst nodes	The burst gateway in your cluster	Set in `burst.gateway`	Only with burst enabled

The hub never connects to the optimizer: policies and approvals are pulled. To stop the optimizer at once, set its policies to Recommend; to remove it, see remove KubeOn.

Need help with this guide?Contact support

PreviousConnect GPU metrics Next GPU policy reference