K8s-native AI Inference: The Ecosystem of Kthena, Grove and KAI Scheduler

In 2026, the Cloud Native Computing Foundation (CNCF) Community Accelerate will elevate LLM inference to a first-class citizen in Kubernetes. The CNCF Volcano subproject Kthena offers KV Cache-aware routing and Prefill-Decode separated scheduling; NVIDIA open-sourced Grove and KAI Scheduler to support topology-aware gang scheduling for multi-node inference; the CNCF Sandbox project KAITO simplifies vLLM model deployment. Based on cross-verification of official blogs and GitHub repositories, this article sorts out the responsibility boundaries and collaboration modes of the three projects across model deployment, traffic routing, and GPU scheduling layers, and provides a minimal YAML example for PD separation, providing a reference for platform engineering teams to build a unified K8s AI infrastructure layer.