AI native applications require a distributed cloud, not just GPUs

Companies developing AI-native applications are finding that raw GPU capacity is only half the battle. As Kubernetes users move from cloud-native to AI-native stacks, the question is shifting from who has the most accelerators to who can run distributed, secure workloads on a global scale.

The first wave of so-called neoclouds provided massive GPU clusters for a handful of the largest buyers, but enterprise developers need much more than just data center capacity. Constant Company LLC, which does business under the name Vultr, has relied on AMD chips and open software stacks to secure AI infrastructure contracts. According to Kevin Cochrane (pictured), Vultr’s chief marketing officer, it is positioning itself as a full-stack cloud that aims to bridge this gap.

“The early market … was about putting large GPU clusters wherever you could get land, power and shell, and then just resell them to a hyperscaler,” Cochrane said. “And somehow in this mix, people have forgotten that you still need a cloud, right? You still need to run this thing.”

Cochrane spoke with John Furrier and Rebecca Knight of theCUBE Research at KubeCon + CloudNativeCon NA, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed why enterprise AI requires distributed cloud infrastructure, how global networks are shaping GPU adoption, and Vultr’s new programs for open source developers. (*Disclosure below.)

Why AI native applications need a distributed cloud

According to Cochrane, each computing supercycle changes the architecture from the data center to the application layer. In AI, the first iteration is aimed at a small group of the largest GPU buyers, he explained. Companies have different requirements and to meet them they have to resort to the longer route of data processing.

“We took a small step back because in this early version of the new architecture, as we over-indexed a limited number of buyers, we went back to relying more on centralized systems,” Cochrane said. “And we must remember that the long road of data processing leads to decentralized distributed systems based on open standards and open source.”

Distributed AI is also changing the way inference is delivered. With many users asking the same question, Vultr aims to maximize token reuse through a global namespace that shares the KV cache across all nodes. This is being pursued with partners such as Vast Data Inc., Cochrane said. Equally important is connectivity between data centers, as companies want private networks isolated from all other tenants in a public cloud.

“In the Neocloud moment, I think some of the complexity of what it takes to run workloads at a global scale for enterprises has been forgotten or just assumed that it wouldn’t matter,” Cochrane said. “But… it’s the biggest barrier to getting enterprises to actually use GPUs from a new cloud because the network connectivity just doesn’t work.”

Vultr also provides resources for building the AI ​​stack in the open source community. The company has joined the Cloud Native Computing Foundation as a platinum member and introduced its open source credits program at KubeCon, which provides infrastructure for maintainers of new CNCF projects.

“All of these projects need computing power, right? The maintainers aren’t going to suddenly open their own personal checkbooks,” Cochrane said. “How can you create the next great CNCF project if you don’t have computing power?”

Here’s the full video interview, part of SiliconANGLE and theCUBE’s coverage of KubeCon + CloudNativeCon NA:

(*Disclosure: Vultr sponsored this segment from theCUBE. Neither Vultr nor other sponsors have editorial control over the content of theCUBE or SiliconANGLE.)

Photo: SiliconANGLE