Google Cloud introduces Agent Compute Engine to host AI agents

Google LLC’s cloud unit today introduced Agent Compute Engine, an infrastructure service optimized for hosting artificial intelligence agents.

AI agents run in technology environments called sandboxes. They host the applications, data, and user instructions that agents need to do their work. Sandboxes also contain cybersecurity guardrails that block malicious activity.

Google refers to the sandbox implementation of Agent Compute Engine as microVMs. The company says the technology combines the hardware efficiency of containers with the security of traditional virtualization. In particular, Google introduced the ability of hypervisors to isolate virtual machines from each other to prevent the spread of malware.

The company runs MicroVMs on specially configured appliances. The appliances are stateless, meaning they do not permanently store the files used by customer agents. The data stored by the systems is stored in an immutable file system that cannot be modified or deleted.

Google verifies the integrity of the appliances’ firmware using its custom Titan cybersecurity chips. The appliances’ central processing units, in turn, have defenses against side-channel attacks. These are cyberattacks that analyze data about the physical state of a CPU to obtain the customer information it processes.

The microVMs running on an Agent Compute Engine appliance have limited access to its storage resources. This prevents hackers from reading or deleting data if they take over one of the sandboxes. Another set of guardrails protects the network that connects the appliance to the rest of the host data center. Micro VM traffic must pass through multiple proxies before it can reach its destination.

“All inbound and outbound traffic must cross an isolated point-to-point IPC boundary,” renowned Google Cloud engineers Ramanathan Venkatapathy and Rohit Jnagal explained in one Blog post. “This boundary ends at a local, unprivileged switching proxy, which forwards packets to an external proxy. The external gateway then validates request scopes and enforces strict, policy-driven outbound brokering.”

AI applications often interrupt the agents they use to automate tasks for users. When an agent is shut down, the sandbox that hosts it must remain available because the agent may be reactivated later. Such inactive sandboxes can consume a significant amount of hardware resources, increasing costs.

According to Google, Agent Compute Engine detects when a sandbox becomes inactive and shuts it down. This prevents unnecessary hardware consumption. When the sandbox is needed, the service quickly reactivates it.

“This allows a single physical node to host orders of magnitude more micro-VMs with live agents than traditional VMs,” write Venkatapathy and Jnagal. “We combine this with agent-integrated capabilities such as cloning, snapshot/forking, rewind and rollback, which enable harnesses to use the system efficiently without resource duplication.”

Some agents use credentials to access the applications they use to do their work. According to Google, Agent Compute Engine stores these credentials outside of the sandbox that hosts an agent. This means that hackers cannot steal sensitive data if they take over the sandbox. When an agent needs to log in to an application, a Google Cloud service called Agent Gateway retrieves the relevant credentials and submits them on behalf of the agent.

Google will introduce Agent Compute Engine along with a new version of Agent Gateway in the coming months. The latter service, which helps customers block unsafe agent traffic, is set to receive a model routing function. This feature allows each agent request to be sent to the model that can process it most efficiently. Agent Gateway also receives a tool to regulate agent inference costs.

“Token governance enables control over token consumption across agents and models, enables spend allocation, budget enforcement, and provides a single source of truth for token metrics,” said Google Cloud vice presidents Nirav Mehta, Michael Bachman and Kevin Hutchins wrote in a blog post.

Picture: Google