Model Fine-Tuning on Kubernetes: Google’s OpenRL Explained

Fine-tuning models through reinforcement learning is quickly becoming one of the most challenging tasks in AI infrastructure. It combines training, inference and sandbox agent work in a single loop, often with research code entangled in the underlying systems.

Google LLC addressed this problem within its Kubernetes engineering group and recently donated the result, OpenRL, to the Cloud Native Computing Foundation. According to Sunil Arora (pictured left), lead software engineer at Google Cloud, the GKE Labs team achieved this goal by building an AI agent for Kubernetes tasks and then a benchmark to evaluate models for that work.

“If you look at reinforcement learning, I think of it as kind of a super app of AI because it involves training. It involves sampling. It involves agents actually doing things in a sandbox and sometimes evaluating the work in some way,” Arora said. “It’s a very varied workload that I think pushes the boundaries of our infrastructure.”

Arora and Justin Santa Barbara (right), developers at Google, spoke with John Furrier and Rebecca Knight of theCUBE Research at KubeCon + CloudNativeCon NA during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed fine-tuning models and their work in GKE Labs, which puts experimental projects in the hands of users early on. (*Disclosure below.)

Fine-tune the model without infrastructure clutter

OpenRL debuted as a research preview in June 2026, offering a self-hosted training API for a company’s own Kubernetes cluster. It separates the reinforcement learning loop from computing power, allowing infrastructure engineers to do the scaling while researchers retain control of experiments, Arora explained.

“One of the things we’re trying to do with OpenRL is to abstract the infrastructure as a kind of core primitive so that researchers and AI engineers can just focus on their RL loop,” Arora said.

In sandboxes, this loop hits the agent infrastructure, as calculating rewards can require the execution of millions of isolated commands. According to Arora, Google’s Agent Substrate packs more of these sandboxes into limited computing power. For Santa Barbara, the need emerged little by little.

“We built kubectl-ai, and in theory kubectl-ai could have formatted my hard drive,” Santa Barbara said. “I would like to have a sandbox for this, so that when a hard drive is formatted, this sandbox is also formatted and no damage is caused.”

Here’s the full video interview, part of SiliconANGLE and theCUBE’s coverage of KubeCon + CloudNativeCon NA:

(*Disclosure: Google Cloud sponsored this segment of theCUBE. Neither Google Cloud nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE