Skip to content
NewKubeOn AI: a managed LLM gateway with cost per 1M tokens for every model
KubeOn docs
API referenceReleases

Automation and control boundaries

What KubeOn runs automatically, what requires a decision, and which system applies each change across Kubernetes and AI.

Process + responsibility

From infrastructure to an approved change

  1. 01

    Collect

    The read-only agent sends cluster snapshots. Billing exports or rate cards provide the cost basis.

    Agent + billing

  2. 02

    Allocate

    The hub assigns cost to workloads and teams, separates shared and idle spend, and reconciles the totals.

    KubeOn hub

  3. 03

    Review

    A proposal brings together the saving, evidence, risk and patch. An authorized reviewer records a decision.

    Your reviewers

  4. 04

    Apply + verify

    Your pipeline applies the approved patch. Your team checks service health and measures the result.

    Your pipeline

KubeOn reads and recommends. Your team and pipeline change the cluster.

Read the workflow

Process + responsibility

One cost model, with control in the request path

  1. 01

    Connect

    Read provider billing, token usage and provisioned capacity. Map deployments and keys to their owners.

    Provider data

  2. 02

    Understand

    Compare effective cost per 1M tokens, including provisioned and reserved capacity, for the same time window.

    Tokenomics

  3. 03

    Route + control

    Applications call the gateway. It selects permitted targets, handles configured fallback and enforces key policies.

    AI gateway

  4. 04

    Measure + tune

    Review request costs, errors, latency and capacity utilization. Adjust policies and approve commitments with evidence.

    Your AI + FinOps team

Gateway policies control routed requests. Capacity purchases and renewals require named approval.

Read the workflow
On this page

The control model

WorkflowAutomatic behaviorWho applies changes
Kubernetes visibilityScheduled snapshots, billing ingestion, allocation, reconciliation and alertsNo workload changes
Kubernetes savingsComputes findings and generates proposals with evidence, risk and patch textYour team approves; your pipeline applies
AI tokenomicsImports billing and token telemetry; computes cost and capacity recommendationsRead-only analysis
AI gatewayRoutes, retries, falls back and enforces configured key policies on incoming requestsGateway, within the policies an administrator enables
AI reservationsPrepares purchase and renewal decisionsNamed approval required before purchase or renewal

Kubernetes: evidence to deployment

  1. Check Diagnostics, billing coverage and the recommendation's usage window before accepting a saving.
  2. Open the proposal and review the current state, proposed state, evidence, risk and rollout steps.
  3. An authorized reviewer approves or rejects the proposal. Approval is recorded in the audit log.
  4. Apply the patch through your own repository or pipeline, with your usual change window and service checks.
  5. Validate workload health and subsequent cost data, then mark the proposal Applied. Roll back through your deployment system if needed.

Approval does not execute a patch

The Kubernetes agent only has get, list and watch permissions. KubeOn does not provision nodes, evict pods, change resource requests or execute proposal commands. See change proposals.

AI gateway: policies in the request path

The gateway can route provisioned capacity first, spill over to pay-as-you-go and fall back after configured failures. A key can warn at budget thresholds or reject requests when its hard-stop policy is reached. These controls apply only to traffic using the gateway; direct provider calls are outside that enforcement path.

  • Scope each key to a team, product and permitted routes.
  • Set retry bounds and confirm fallback targets support the application's request and response requirements.
  • Test budgets, rate limits, residency restrictions and redaction with non-production traffic.
  • Review latency, errors, token usage and cost before expanding traffic. Keep a tested application routing rollback.

Capacity commitments

Purchases and renewals wait for a named approver with their term, scope and quantity recorded. Approval for routing does not imply approval to purchase capacity. See provisioned capacity sizing.

Evaluation evidence

Record who owns the policy, which workloads or routes it covers, the baseline, the validation result and the rollback procedure. Use the enterprise evaluation guide to agree on acceptance criteria with platform engineering, FinOps and security.

Need help with this guide?Contact support