The AI ​​Data Layer Drives Inference as Data Centers Distribute – SiliconANGLE Inference moves AI to smaller locations and makes the data layer the deciding factor

Inference is moving to the center of enterprise AI spending, and the AI ​​data layer is becoming the part that determines whether those investments pay off. As GPU costs rise and component shortages continue, the architecture behind AI is shifting from a handful of massive facilities to a distributed network of smaller sites.

This shift puts storage companies with roots in high-performance computing at the center of the AI ​​discussion. According to Alex Bouzari (pictured), co-founder and CEO of DataDirect Networks Inc., turning data into insights at an acceptable cost now relies on a robust, secure data engine designed for GPU efficiency.

“Because inference is outside the norm in terms of usage, it is the data layer that makes the difference,” Bouzari said. “The costs are skyrocketing. GPUs are becoming more and more powerful, but also more and more expensive.”

Bouzari spoke with John Furrier and Rebecca Knight of theCUBE Research at SC26 during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how inferences, cost pressures and bottlenecks are changing AI infrastructure. (*Disclosure below.)

Why the AI ​​data layer is distributed

The first wave of AI expansions concentrated GPUs in large, power-hungry facilities. Inference changes that math because it needs to be done close to users and their data, Bouzari explained. He added that communities are also pushing back against massive monolithic data centers.

“What happens now with inference needs to be complemented by smaller footprints. Let’s call them data centers, 1 megawatt, 2 megawatt, 5 megawatt. And those need to be located where the action is happening, where the autonomous driving is happening, where physical AI is happening, where the sensor data is,” Bouzari said.

Connecting small pods to large AI factories requires a different approach to data. DDN is building a software layer that spans many types of hardware, a direction reflected in its recent decision to free its EXAScaler file system from DDN’s own hardware. Bouzari says the goal is to increase the use of very expensive equipment.

“It’s a software layer that has to accommodate the different types of hardware because because the cost is so high, people want to be able to maximize utilization over time, just like it did with high performance computing,” Bouzari said.

The level of spending has changed dramatically. A few hundred million dollars once funded the world’s fastest supercomputer, but today it requires 15 to 20 racks of equipment, Bouzari noted. Shortages in GPUs, memory and solid-state drives are meeting tighter budget controls.

“On the one hand, there are budget constraints and the need to get as much value as possible from your investment,” Bouzari said. “On the other hand, there are bottlenecks, so a data layer that can do much more with much less infrastructure and less space is extremely valuable.”

This thinking extends to the company. DDN’s Enterprise AI HyperPOD bundles CPUs, GPUs, networking and storage into a rack that can be installed into existing enterprise locations with available power. Managing the pod on-premise also gives companies greater control over their data, Bouzari explained.

“I think we’re entering an era where more and more smaller footprint pods are being deployed in companies around the world,” Bouzari said.

Bouzari viewed the AI ​​data layer as the counterpart to accelerated computing, with Nvidia Corp. the processing and DDN provided the data. As inference and agents take over, the data side takes on more weight, he said: “You need both to be successful. But the data engine, I think, will become more and more important as inference and agents really become the dominant way to use AI and get value from it.”

Here’s the full video interview, part of SiliconANGLE and theCUBE’s coverage of SC26:

(*Disclosure: NYSE Wired sponsored this segment of theCUBE. Neither NYSE Wired nor other sponsors have editorial control over the content of theCUBE or SiliconANGLE.)

Photo: SiliconANGLE

Support our mission to keep content open and free by interacting with theCUBE community. Join theCUBE Alumni Trust Networkwhere technology leaders connect, share information and create opportunities.

  • Over 15 million viewers of theCUBE videosto spark conversations about AI, cloud, cybersecurity and more
  • Over 11.4k theCUBE alumni — Connect with more than 11,400 technology and business leaders shaping the future through a unique, trusted network
SiliconANGLE Media is a recognized leader in digital media innovation, combining breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios – with flagship locations in Silicon Valley and the New York Stock Exchange – SiliconANGLE Media operates at the intersection of media, technology and AI.

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands, reaching over 15 million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is a game-changer in audience engagement, leveraging the theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of the industry conversation.

Avatar photo
Written by

Mira Edora

Mira Edora is a writer and contributor at CKSOR, creating clear and engaging articles on current topics, technology, science, lifestyle, and stories of interest to readers. She enjoys researching new developments and presenting useful information in a simple, accessible way. Through her writing, Mira aims to keep readers informed with timely, informative, and easy-to-understand content.

Leave a Comment