Caterpillar and CoreWeave shorten the learning loop for physical AI

Demand for new data centers, power plants and highways is fueling a construction boom, even as the industry struggles with declining productivity and a shortage of skilled machine operators. Physical AI is emerging as one answer, enabling machines to perceive and respond to their environment.

Caterpillar Inc. has years of experience with autonomous mining equipment. Expanding this technology into construction requires systems that can adapt to more variable conditions, according to Brandon Hootman (pictured right), vice president of physical AI platforms and construction autonomy at Caterpillar.

“Once that mine location is instantiated, it does change, but not frequently,” he said. “You go to construction as an industry. The exact opposite of mining. When you think about the whole dynamic of what needs to happen, taking a structured system and adapting it to an unstructured environment, it’s really a real challenge.”

Hootman and Richard Ahlfeld (left), senior vice president of physical AI at CoreWeave Inc., spoke with Dave Vellante and John Furrier of theCUBE Research at the Fully Connected event during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed why physical AI is redesigning the infrastructure behind autonomy and how the two companies are shortening the learning loop for construction equipment. (*Disclosure below.)

Physical AI is transforming data center performance

AI clouds were first designed to train base models and then optimized for large-scale agent inference. To train an autonomous excavator, telemetry and image data must be recorded, a digging scenario must be simulated millions of times, and reinforcement learning must also be carried out. According to Ahlfeld, CoreWeave recently launched a Physical AI Field Engineering service that connects its engineers with customers’ subject matter experts.

“Physical AI is a whole different beast now,” he said. “First of all, it requires a lot of storage. It requires a different infrastructure.”

Hootman noted that Caterpillar’s digital ecosystem already contains approximately 18 petabytes of composite data from machines, dealers and customers. Training autonomous devices also requires perception data synchronized with machine control and performance data.

“The 18 petabytes we have collected so far is just a drop in the ocean compared to the amount of data actually required to train physical AI to operate in a world like a construction site,” he said. “But you take [Light Detection and Ranging] Data, camera data, that multi-second control data and performance data that comes back, you’re talking terabytes of data within a given day for just one machine.”

CoreWeave listed Caterpillar among its enterprise customers in its second-quarter earnings call. Hootman said Caterpillar began working with the company this year because it was confident in both the graphics processing unit’s capacity and expertise. Working with Nvidia, the partners use AI models to annotate and label incoming field data.

“Now you’re taking on things that took months or maybe weeks,” Hootman said. “And now it only takes a few hours, and your feedback loop between what’s happening and being able to use it in your simulation environment or your training environment happens within the given workday.”

Here is the full video interview, part of SiliconANGLE and theCUBE’s coverage of the Fully Connected event:

(*Disclosure: TheCUBE is a paid media partner for the Fully Connected event. Neither CoreWeave, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over theCUBE or SiliconANGLE content.)

Photo: SiliconANGLE