GCP News - 2026-10-17

In production systems today, most AI agents operate as lightweight dispatchers. They parse a prompt, make an API call, retrieve a record, and hand back control. Their execution is fast, stateless and cheap. Biotechnology does not work this way. 

Folding a protein target, screening a 50,000 compound ligand library, or evaluating structure-activity relationships (SAR) are not quick API lookups. They are sustained, stateful, GPU-bound simulations that run for minutes or even hours across specialized silicon. When you ask a general-purpose large language model (LLM) endpoint to coordinate a complex drug discovery workflow, the standard paradigm breaks: Timeouts occur, context windows overflow with atomic coordinates, and proprietary chemical Intellectual Property (IP) risks exposure to external networks. 

Building enterprise grade research and development (R&D) agents requires moving past the monolithic assistant model. In this blog post, we will discuss how to use a three-layer architecture: separating cognitive orchestration (the brain) from computational execution (the muscle), and anchoring both inside an enterprise governance plane to build scalable agents for drug discovery use cases in biotech.

A three-layer pattern

The reference architecture below shows how to architect, optimize, and govern high-compute autonomous agents in biotechnology, specifically Drug Discovery.

The execution layer: scaling the muscle - high-compute execution 

The execution layer is responsible for the heavy scientific computations. The first architectural decision is determining where workloads should run:

• Strategic, proprietary pipelines require dedicated GPU infrastructure within your security perimeter

• Routine or standardized workloads can safely be delegated to managed cloud endpoints

In this architecture, protein folding is driven by ESMFold and generative design by GenMol. Both run as NVIDIA Inference Microservices or NIMs deployed on Google Kubernetes Engine (GKE) using G4 VMs backed by NVIDIA RTX PRO 6000 Blackwell GPUs.

A critical hurdle in containerized GPU workloads is the cold-start penalty, the latency delay required to pull multi-gigabyte model weights from an image registry and stage them into GPU memory (VRAM).

To ensure low latency responsiveness, we build the following as a recommendation into the reference architecture:

• Maintain warm pools: Pin the GKE node pool to a minimum of one active node.

• Preload weights: Model weights are loaded directly into VRAM at pod initialization, ensuring containers are immediately ready for inference.

• Resilient scheduling: GKE manages pod lifecycles and restarts failed instances without tearing down the underlying VM infrastructure. 

This approach trades custom, manual CUDA kernel tuning for NVIDIA's pre-compiled, containerized runtimes, dramatically reducing infrastructure engineering overhead while ensuring deterministic throughput.

In biotech, chemical structures represent millions of dollars in proprietary IP. A docking request transmits SMILE strings or 3D coordinate matrices that must never leave company custody.

Containerized NIMs eliminate the friction between rapid prototyping and rigorous data isolation. During early exploration, researchers can query NVIDIA hosted endpoints. When transitioning to proprietary lead candidates, the exact same container is deployed to GKE inside a private VPC. The agent’s orchestration logic remains identical; only the target URI updates from an external proxy to an internal VPC endpoint. No structural data leaves the enterprise perimeter.

The cognitive layer: Multi-agent reasoning

With compute execution isolated, the cognitive layer manages agentic workflow sequencing. Built on Agent Runtime, within the Gemini platform, this reference architecture shows a system that deploys a hierarchical multi-agent structure, one lead orchestrator, and two specialized domain sub-agents (Lead Optimization and Safety Screening) . The multi-agent design can be modified to suit the use case at hand. 

Based on this design, when an end-user submits a high-level task such as “ Analyze Candidate 4b against Target X and produce a SAR report”, the system applies four core orchestration patterns discussed below; 

• In-process native agent handoffs: The lead orchestrator delegates tasks to specialist sub-agents using Agent Runtime’s native SDK rather than network-serialized agent-to-agent HTTP protocols. This eliminates intermediate network hops and keeps internal handoffs instantaneous. Only the outermost boundary communicates outward to user-facing surfaces like the Gemini agent.
• Passing bulky payloads by reference: LLMs degrade quickly when forced to ingest massive arrays of 3D spatial coordinates. In this architecture, raw structural data (PDB files, docking poses, trajectory logs) is written directly to Cloud Storage. Agents pass lightweight, signed object URIs and high-level structural summaries, keeping context windows compact and inference costs down.
• Decoupling workflow logic from biochemical reasoning: General-purpose LLMs excel at task planning and synthesis, but are prone to hallucinating biochemical nuances. To control this behavior, this architecture maintains strong separation of concerns, namely, workflow and tasks decomposition are handled by Gemini. Domain chemistry and pharmacological reasoning are handled by specialized Nemotron models hosted on dedicated GPU nodes. 
• Context-driven evaluation over rigid cutoffs: Traditional computational screening enforces arbitrary numerical cutoffs (e.g., discard if binding affinity ,-7.5 kcal/mol) However, therapeutic context fundamentally alters risk tolerance. Preventive cardiovascular therapy taken daily by healthy patients requires extremely strict safety margins, or a novel oncology candidate targeting an intractable cancer may accept higher toxicity if efficacy is demonstrated.

By passing disease context directly into the specialized reasoning models, drug candidates are evaluated against dynamic, target-specific profiles rather than static thresholds, preventing the premature culling of viable drug leads.

Performance best practices for high-compute pipelines

Optimizing an agentic system that interfaces with heavy GPU compute requires challenging common engineering instincts:

The enterprise governance and asymmetric autonomy

Deploying high-compute agents in enterprise environments requires robust data sovereignty, audit trails, and process control. The Gemini platform establishes these safeguards natively with the following components: 

• Agent Identity: Every autonomous agent operates with a cryptographically verifiable identity. Thus, for a biotech use case, when a candidate molecule is advanced or rejected, the event is permanently logged to that specific agent principal. 

• Agent Gateway: This serves the single ingress/egress point for every interaction within the perimeter. Centralized IAM and regional residency rules help ensure sensitive molecular data remains within designated cloud regions. 

• Agent Registry: This feature maintains a curated catalog of approved agents, validated tool schemas, and authorized Model Context Protocol (MCP) servers, preventing rogue tools from executing in production. 

• Model Armor: This functionality inspects incoming prompts and outgoing payloads in real time to prevent prompt injection and data exfiltration. 

The payoff is clear. Identity, RBAC, auditing, residency, and data protection are native properties of the Gemini platform; this allows enterprise customers to focus on science rather than having to build a compliance stack for every project. 

In high-stakes research and development, full autonomy introduces unacceptable liability, while requiring approval for trivial steps can create frictions. The solution addressing this to use asymmetric gating in the following ways;

Automate reversible operations: For example, allow the agent to resolve chemical naming synonyms, pull structural PDBs, and run preliminary Lipinski screens autonomously without interrupting the researcher.

Gate irreversible operations: For instance, initiating multi-GPU docking run, advancing a lead compound to wet-lab synthesis, or archiving a therapeutic program must require explicit human sign-off.

The pattern, and the path forward

High-compute autonomous agents succeed when the boundaries between cognitive planning, numerical computation, and system governance are clean in the following ways:

1. Keep the orchestrator focused on decomposition and synthesis, leaving intensive biophysical simulation to specialized, containerized microservices and models.

2. Eliminate context bloat by passing complex chemical data by reference through cloud storage.

3. Anchor the workflow in enterprise governance, pairing cryptographic agent identity with asymmetric human oversight.

As generative and structural biology models expand toward whole-cell modeling and complex dynamic simulations, this decoupled architecture ensures research systems scale efficiently while keeping proprietary IP secure and auditable.

Reach out to your Google sales representative if you have any requirements to build a use case like this and we will be happy to talk and support you in your agent development journey on Google Cloud.

Everything is moving faster in the age of AI, and that includes security threats. What used to take attackers weeks to exploit can now happen in just a few minutes.

Standard vulnerability scans that run on artifacts at rest can catch basic setup issues, but to detect an attack that's actively happening you need real-time runtime protection.

Today, we’re introducing the Wiz Runtime Sensor add-on for Google Kubernetes Engine (GKE), in preview. This built-in integration brings low-touch deployment, automated lifecycle management, enterprise credential security, and fleet-wide governance to Wiz's industry-leading eBPF runtime threat detection for both Standard and Autopilot GKE clusters.

The operational friction of runtime security

Security and platform engineering teams share common dilemmas when deploying in-cluster runtime defenses:

• Deployment and lifecycle friction: Deploying runtime agents often requires managing installations from third-party sources across dozens or hundreds of clusters.

• Credential sprawl and security risks: Passing and rotating API keys as Kubernetes Secrets introduces administrative overhead and security risk.

• Fleet-wide enforcement gaps: Platform teams lack built-in policy controls to enforce that every new cluster has runtime sensors enabled before workloads are deployed and start receiving traffic.

• Performance and operational overhead: Without explicit and strategic resource limits, runtime agents can impact how much CPU and memory is available for your workloads and system functionality.

With the Wiz Runtime Sensor for GKE, Google Cloud and Wiz have minimized these operational barriers. The Runtime Sensor is now available in GKE as a first-party, managed platform add-on.

Bridging the gap with Wiz Runtime Sensor for GKE

The Wiz Runtime Sensor for GKE connects central security and platform engineering workflows by simplifying and automating the sensor lifecycle.

With the add-on, GKE manages the sensor deployment, credential injection, and lifecycle updates. Building runtime protection directly into the platform reduces administrative overhead and provides the following benefits for your Kubernetes fleet:

Native Secret Manager integration: Security teams no longer need to distribute Wiz API keys across cluster namespaces. The add-on uses GKE's Secret Manager secret sync integration to store your Wiz credentials in an off-cluster, centralized credential vault that only a dedicated service account can access. 

GKE automatically retrieves the credentials and injects them directly into the Runtime Sensor pods. Updating a key in Secret Manager automatically syncs to the sensor without requiring cluster restarts or manual intervention.

• Simplified enablement: After you add your Wiz credentials to Secret Manager, platform operators can enable the Runtime Sensor add-on by using the gcloud CLI or the GKE API. The GKE control plane handles deploying the sensor components, managing the lifecycle of pods, and configuring the nodes.
• Secure egress and proxy support: For environments that restrict public internet access, the managed sensor supports standard HTTP and mTLS proxy configurations. Telemetry is routed back to Wiz without violating your egress firewall policies.
• Fleet-wide governance with organization policy: Platform security leads can enforce compliance across their entire organization using custom organization policy constraints. You can use a constraint to reject any attempts to create a cluster that doesn't enable the Runtime Sensor add-on, which can prevent malicious actors from spinning up rogue infrastructure to evade runtime detection.
• Kernel-level observability with minimal overhead: The Wiz Runtime Sensor continuously monitors system calls, running processes, network connections, and file access. It catches suspicious activity, including reverse shells, unauthorized privilege escalations, container escapes, and unsafe outbound connections while keeping CPU usage low even under heavy workload activity. 
  
  The sensor uses a fixed memory limit on each node to ensure consistent performance as workload density changes. This is especially valuable for AI workloads and web services where application activity might fluctuate quickly.
• Independent auto-updates: Security threats and zero-day vulnerabilities don’t wait for scheduled infrastructure maintenance windows. The lifecycle of the Wiz sensor add-on for GKE is independent of your GKE cluster and node lifecycle. 
  
  The sensor periodically rolls out updates of new sensor versions published by Wiz, to help ensure that your clusters always have the latest detection rules without requiring GKE version bumps, node pool recreations, or workload disruption.

How it works: Deploying and enforcing the sensor

Enabling the Wiz sensor add-on follows the same process that platform administrators are already familiar with on GKE. Using standard gcloud commands, you can activate runtime protection during cluster creation, turn it on for existing clusters, and enforce it across your fleet while keeping credentials securely managed in Secret Manager. 

Check the setup documentation for prerequisites, such as granting GKE access to Secret Manager.

Provisioning a cluster with the Wiz Sensor add-on

To provision a new GKE cluster with the managed Wiz sensor enabled using credentials stored in Secret Manager:

To enable the sensor on an existing running cluster with Workload Identity Federation for GKE and Secret Manager secret sync enabled, run the following:

Enforcing fleet-wide protection with organization policy

You can require all new GKE clusters across your organization to install the runtime sensor by deploying a custom constraint:

Intelligent threat correlation for AI security

The true power of this integration emerges when the managed sensor connects to the broader Wiz AI Application Protection Platform (AI-APP). By feeding GKE runtime telemetry into the Wiz Security Graph, you unlock a security toolkit that goes beyond static alerting, with:

• Real-time risk assessment and threat verification: By pairing background security scans with live activity monitoring, you get a clear view of real risks. The sensor checks whether vulnerable code is actually running in memory, so you can focus on active security threats and ignore inactive ones.
• Real-time threat detection and response: Identify suspicious activity like malware, unauthorized access attempts, and software vulnerabilities in real time. Security teams can set rules to automatically block or stop threats before they can spread.
• Network activity and detailed security insights: Track system activity, network connections, and data flows to uncover hidden risks and unauthorized software usage. If a threat is detected, the sensor automatically gathers diagnostic data to help teams quickly investigate what happened.

Together, these capabilities ensure that mission-critical applications and AI agents deployed on GKE are monitored for attacks and data exfiltration without compromising developer velocity.

Get started today

The Wiz Runtime Sensor add-on for GKE delivers a standard for container security: deep, kernel-level eBPF protection without operational complexity or deployment friction.

By combining GKE's managed platform automation with Wiz's comprehensive security intelligence, platform and security teams can innovate at cloud speed while maintaining strong runtime defense. Check out the Wiz Sensor for GKE documentation to learn more.

The real challenge of building an agentic system isn't the construction, it's the measurement. 

Evaluation engineering is a critical foundation for building reliable AI systems. Setting up systematic, repeatable tests to measure your agent’s true behavior can be the difference between shipping a tool you can trust and one that silently breaks, fails to call APIs, or makes mistakes. 

I’ve run evaluation engineering initiatives for years at Google and developing the newly released Google Cloud Developer plugin served as a live testbed to stress-test these key principles across real-world developer workflows.

Here are four eval lessons we learned from building the new plugin that you can apply to your own builds.

1. Grade the execution steps instead of the summary

When we first evaluated agents, we set up an LLM-as-judge to inspect the agent’s final text reply and scored it against a checklist. However, we quickly realized it’s fundamentally flawed because the grader only sees what the agent wrote, not what it actually did. When we ran our initial 11-prompt evaluation suite for the Google Cloud Developer plugin, the automated grader marked it as failing three prompts. But when we opened the raw tool-call logs and read the transcripts, the plugin had actually passed all three. On a Cloud Run cleanup prompt, for instance, our rubric required checking command syntax before running anything. The agent ran gcloud help run services list as its very first step. It did the work, but its final message was focused on warning the user about permanent data loss rather than reciting that it had checked the help docs.

The grader penalized the agent for doing the right thing simply because it didn’t brag about it in the summary. If your rubric line says "did it run X?", a LLM judge is guessing unless you evaluate the actual tool-call trajectory.

Takeaway: Don't grade an agent on its closing pitch. Score the raw execution trajectory to verify what commands were actually executed, validated, and scoped.

2. Design evals for asynchronous operations

When you start reading tool-call trajectories, how you extract that telemetry matters just as much as what you're reading. Extracting agent telemetry into a flattened, sequential list of events obscures concurrent tool dispatch, which is one of the critical causes of behavioral failure. Modern agent harnesses often batch multiple tool invocations into a single assistant turn using an array of calls.

In our evaluations, when an agent was asked to inspect an environment and look up documentation, the harness dispatched the skill-loading call and two documentation searches simultaneously within a single tool_calls payload. Because activating a skill is a multi-stage process, where the initial call returns a short launch stub and the subsequent turn injects the full instruction body, the instructions meant to govern the search arrived in the prompt context after the system had already executed the searches.

The system never gave the skill a dedicated turn to let its guidance influence the agent's actions. If telemetry pipelines strip out turn structures and merge calls into a flat timeline, engineers will waste days wondering why an agent disregarded unambiguous instructions that a developer embedded in a skill.

Takeaway: Preserve turn structure in telemetry extractors so you can identify when client-side parallel dispatch issues tool calls before injected guidance enters the context window. 

3. Test live runs and "dry runs" side-by-side

You need to inspect real API calls to catch harness quirks, like parallel dispatch. But if you only test your agent in one environment condition, you're flying half-blind. From day one, you need to run two conditions side-by-side: a live run and a plan-only run. In the live condition, the agent has real, sandboxed credentials and actually executes commands. This is how you see if syntax works, parameters serialize cleanly, and the agent exercises restraint before doing something destructive. 

Live runs frequently hit an unexpected permission error on step two and abort before you ever reach the behavior you’re trying to test. That's why you need the plan-only condition, where you strip credentials and tell the agent to explain what it would do without running anything. This strips out permission roadblocks and shows you the model’s pure reasoning and routing intent. Roughly half our most important findings came from the delta between the two.

For example, on one of our onboarding prompts, a live run was blocked immediately on a project list call. The plan-only run sailed right through and produced an 8,000-character plan that completely ignored organization policies, admin approvals, and governance. The live run masked a major cognitive flaw that the plan-only run exposed in seconds.

Takeaway: Run live and plan-only evaluations side by side. Live runs test actual execution and safety restraint; plan-only runs expose the model's raw reasoning and routing without getting tripped up by permission errors.

4. Use “golden sets” as durable baselines

Frontier model architectures change every few months, agent harnesses improve, and plugin manifest standards will inevitably evolve. When building the Google Cloud Developer plugin, we treated evaluations as continuous integration testing for these non-deterministic environments. That means we stress-tested the composite stack, including the harness, the live model, and the tools running together on realistic tasks.

Instead of guessing from the final text output, this approach captures the whole trajectory: what the agent thought, what commands it actually ran, and whether it solved the problem without breaking anything. Then you score that run against golden sets curated by real human experts who know what the ideal path looks like.

Takeaway: Build golden sets based on what the entire system should actually do in the wild, including your harness, model, and plugins. Models change, but those baseline runs are your safety net.

Reminder: Use evaluation suites over “one-shot” demos

A common mistake I see among engineers is what I like to call the one-shot demo. This is when someone feeds a prompt into a LLM, watches it build a complete 3D game or solves a Rubik’s cube in a single pass and calls it an eval. But in reality, this is just a vanity metric. 

To do quality work with AI, I truly believe that you need evaluation suites. Without evaluations, teams are like literally running into a dark cave blind. Engineering an enterprise-grade agent means testing the system's ability to handle multi-step interactions, manage permissions, and avoid destructive outcomes like accidentally deleting a project because you asked it to "find opportunities to save costs."

Where to go from here

Building an agent plugin or MCP server is the beginning. Shipping something you can actually trust with real cloud infrastructure requires significantly more work in evaluations to ensure it’s doing what you expect it to do. At the end of the day, the only thing that outlasts model churn is a stress-tested eval suite. Check out the Google Cloud Developer plugin documentation or explore this codelab to get started with the plugin in Antigravity.

Google Cloud Consulting works with organizations every day to build generative media that’s trustworthy, brand-aligned, and stands out. Instead of relying on clever or complicated prompting, we’ve identified four recurring basics to generate consistent results, what we call the “SAVE” method. 

In this blog, we’ll share more about this method and how you can apply it to your own work. 

S - Separate the researcher from the storyteller

If in a single prompt, if you’re requesting a technical summary, key product features, and an engaging video script all at once, you’re setting the model up to fail. Creative tasks require fluid language and pacing, while factual output requires constraint. Forcing both jobs into one context window will likely end with the model fabricating details to make the narrative flow better.

Instead of one prompt, consider splitting the work into specialized steps, like a researcher and a storyteller. Here’s how they might work together: 

The researcher reads the source documents, extracts core claims, aligns with approved messaging, and outputs a structured brief (like a clean checklist or bulleted factsheet).

The storyteller then takes that verified brief and focuses purely on style, pacing, visual metaphors, and tone.

By decoupling research from your creative generation, your agent is never tasked with deciding what’s true, but how to make the truth more engaging and effective.

A - Approve verified facts only

If you hand a creative generative model documents that are unedited, it might mix up speculative ideas with business facts. The media generator should never have direct access to raw source files. It should only use verified, structured inputs. To avoid this: 

• Lock down core claims: Approved statements, product names, and concrete business outcomes should be fixed upstream rather than left for the model to extract on the spot.

• Champion contextual packaging: Pass each fact along with its explicit constraints included. Examples may include things like "use this point in the opening statement" or "do not extrapolate beyond this approved bullet point."

This way, you ground the generation in verified facts and reduce the risk of unsupported claims entering the creative generation stage.

V - Verify before you render

Discovering a factual error in an animated video or rendered asset after the final export wastes both compute budget and turnaround time. The most effective human-in-the-loop checkpoint is right before generation begins.

To fix this, review the script and storyboard. Have product leads or legal teams sign off on the extracted facts, narrative outline, and voiceover text while it’s still simple, editable text. This way, you can catch errors early. Swapping an outdated feature name or fixing a typo takes two seconds in a text file. 

Treat text as your testing ground. Once approved, generating the final media becomes a low-risk, predictable step.

E - Engineer guardrails into the system

Telling a model to “keep voiceover word count short, never reference unapproved topics, and format links correctly" in a prompt works until it doesn’t, and an edge case slips through. Relying on prompt obedience for brand safety is a gamble and can damage trust.

Where requirements can be checked deterministically, enforce them in the system rather than relying only on prompt instructions. For example, try these three actions first: 

• Programmatic boundary checks: Use standard code to validate word counts, verify that URLs are live and authorized, check colors against your brand palette, and format text strings.

• Independent safety gates: Run automated filters outside the generation pipeline to catch policy violations or out-of-bounds claims before an asset is ever delivered.

• Model modularity: When your safety and brand rules are set, you don't have to rewrite your entire governance playbook every time a newer, faster generative model is launched. You simply plug the new model into your existing, verified pipeline.

Generating studio-quality media shouldn't mean constantly worrying about what the model might fabricate next. The SAVE method shifts the burden of accuracy away from prompt phrasing and into the architecture where it belongs. 

Apply the SAVE method yourself

• Check out our generative media deep dive and learn how to architect trust across your generative media using Google Cloud tools and technologies. 

• Explore Google Cloud’s generative media suite, featuring cutting-edge models for studio-grade image, video, and sound generation.

Today, Reflection released Beam, its first open-weight model built to give developers, researchers, and enterprises open control and customization for frontier software engineering and long-horizon agentic tasks. Released under a permissive Apache 2.0 license with BF16, FP8, and NVFP4 checkpoints available at launch, Beam sets a new standard for inference efficiency in the 500B-parameter class by solving complex reasoning and coding problems with 3x to 4x higher reasoning compute efficiency than comparable open models.

Alongside today’s release, Google Cloud is announcing day 0 support and ready-to-run deployment recipes for serving Beam on Google Cloud AI Hypercomputer and NVIDIA AI Infrastructure.

To help developers get the full benefit of Beam’s token efficiency in production, we worked with Reflection to optimize performance of the underlying serving stack using llm-d (with vLLM) and NVIDIA Dynamo (with SGLang) on Google Cloud:

"By deploying our Beam model on Google Cloud, we achieved up to 5x higher performance and 8x lower latency to optimize serving of agentic workloads." - Joe Spisak, VP of Product & Head of Open Source, Reflection

With today’s day 0 release on Google Cloud, developers get:

• Compounding model and system efficiency: Combining Beam’s 3x to 4x reasoning compute efficiency with system-level optimizations on A4X Max instances (NVIDIA GB300 NVL72) for up to 5x higher throughput and 8x lower latency,* and G4 VMs (NVIDIA RTX PRO 6000) for up to 2x higher throughput and 8x lower latency.*

• High-efficiency multi-turn serving: KV-cache-aware routing, prefill/decode disaggregation, speculative decoding and hierarchical KV cache offloading to Host DRAM and Local SSD reduce redundant prompt recomputation across long coding sessions.

• Turnkey optimized deployment: Our recommended performance optimized recipes for agentic workloads with llm-d and NVIDIA Dynamo are available in AI-Hypercomputer/gpu-recipes, llm-d repo and Model Garden in Gemini platform with a few clicks. 

*Based on customer benchmarking

Scaling inference requires system-level optimization

When evaluating models for agentic coding and tool use, inference efficiency happens at two distinct layers: how efficiently the model reasons and how efficiently the serving system executes multi-turn loops.

Model-level efficiency: Reflection’s Beam

Reflection designed Beam (~501B total parameters with ~23B active parameters per token, and an approximately 1M-token context window) specifically for text-native software engineering and multi-step tool use. It plugs directly into the coding harnesses developers already use, including Reflection’s Mirror CLI, Claude Code, Hermes, OpenCode, and , to resolve GitHub issues, run CLI and DevOps workflows, and complete long-horizon tasks.

Because Beam activates only 23B parameters per token and reaches solutions using 3x to 4x fewer reasoning tokens than other comparable open weight models, it dramatically reduces the inference-time compute required for each reasoning step.

System-level efficiency: AI Hypercomputer optimized serving

Once developers deploy an agent into production, a second set of bottlenecks could appear at the serving layer. In an agentic coding loop, the model is called dozens of times in a single session. With every turn, the coding harness appends new file reads, code diffs, compiler logs, and tool outputs to a shared conversation history that quickly grows to tens or hundreds of thousands of tokens.

Even when the model itself generates 3x to 4x fewer reasoning tokens on each turn, the accumulated context window across turns creates two major infrastructure challenges:

First, prefill compute becomes a bottleneck. Standard Kubernetes load balancers distribute requests using stateless round-robin routing. When a coding harness sends Turn 15 of a session, it often lands on a different replica than Turn 14. That replica is forced to recompute the entire historical prompt from scratch, wasting GPU compute and creating long prefill queues that delay other users.

Second, decode becomes bottlenecked by KV cache capacity. Holding long-context KV caches for dozens of concurrent agent sessions quickly fills up GPU High-Bandwidth Memory (HBM). Once HBM fills up, the serving engine has to evict older KV blocks. As soon as those blocks are evicted, cache hit rates drop and GPU throughput stalls.

To preserve Beam’s model-level efficiency at cluster scale, the serving stack has to be optimized as a distributed system. That is why our recipes pair Beam with llm-d (with vLLM) and NVIDIA Dynamo (with SGLang).

Inside the AI Hypercomputer inference architecture

To run Beam’s BF16, FP8, and NVFP4 checkpoints at peak efficiency on Day 0, we designed a vertically integrated architecture on AI Hypercomputer where each layer handles a specific part of the agentic serving lifecycle.

Rather than treating inference pods as stateless endpoints, each layer of the stack is co-designed around the multi-turn agent lifecycle:

• Prefix- and load-aware routing (GKE + llm-d EPP / Dynamo router): Tracks cached prefixes and queue depth across worker pods, steering each follow-up turn from a coding harness to the replica holding its warm KV prefix while preventing queue saturation via turn-priority flow control.

• Disaggregated compute and speculative decoding (A4X Max & G4 instance): Runs vLLM and SGLang with native Blackwell NVFP4, Expert Parallelism (EP) for Beam’s 23B active experts, Data-Parallel Attention, and DFlash2 speculative decoding. Separating Prefill and Decode (P/D) pools ensures heavy prompt bursts never stall active token generation.

• Tiered KV cache (GPU Memory -> DRAM & Local SSD) Active KV blocks stay in high-bandwidth GPU memory (Tier 1), while warm KV blocks evicted during long-horizon sessions overflow into host CPU DRAM and local NVMe SSDs (Tier 2) via vLLM’s OffloadingConnector or SGLang's HiCache, allowing historical context to be fetched in milliseconds instead of recomputed from scratch.

System optimizations for multi-turn agentic workloads

When serving multi-turn coding and tool-use workloads like the AgentX trajectory dataset across rising concurrency levels, each layer in the serving stack contributes to the performance and latency improvements for multi-turn, long context traffic patterns of agentic workloads:

A note on reading these charts. The two configurations are not comparable to each other. NVIDIA Dynamo ran on A4X Max instances with 32 x NVIDIA GB300 NVL72GPUs; llm-d ran on G4 VMs with 8 x NVIDIA RTX PRO 6000 GPUs. Each chart shows the gain over a baseline on the same hardware. More replicas give the router more warm KV caches to hit and more room to split prefill from decode, so the uplift grows with the deployment. Both llm-d and NVIDIA Dynamo scale up and down to match the underlying infrastructure.

Across both platforms, layering these optimizations shifts the Pareto frontier to deliver higher output throughput and faster per-user interactivity simultaneously:

• Baseline deployment (Stateless round-robin routing). In a standard Kubernetes deployment without prefix awareness, consecutive turns from the same coding session land on different replicas, forcing constant prefill recomputation on historical context and causing throughput to plateau as concurrency scales.

• KV-cache-aware & flow control routing. Enabling prefix-aware routing in NVIDIA Dynamo or llm-d’s Sticky EPP with turn-priority flow control keeps multi-turn agent trajectories pinned to the replica holding their warm KV cache, immediately cutting Time-to-First-Token (TTFT) latency and freeing up GPU compute, resulting in a significant increase in token throughput.

• Host DRAM and Local SSD (LSSD) KV offloading. Offloading evicted KV blocks from GPU memory into host DRAM and local NVMe SSDs prevents cache thrashing when dozens of long-context coding sessions exceed on-device GPU memory capacity.

• Disaggregated serving, speculative decoding, and KV pool tuning. Tuning hybrid KV cache memory allocation, enabling speculative decoding, and disaggregating prefill and decode workers isolates token generation from heavy prompt bursts while the draft model multiplies output throughput to keep per-agent token interactivity fast.

Get started and meet us at PyTorchCon and GTC Berlin 

This blog explored how you can leverage system level optimizations to scale performance to serve Reflection's Beam family of models for agentic workloads. All of the Kubernetes manifests, router configurations, and benchmarking scripts from this post are open-source and ready to run on your own GKE cluster.

• Deploy the optimized stack: Explore the day 0 guides for running Reflection’s Beam with llm-d and NVIDIA Dynamo in the AI-Hypercomputer/gpu-recipes repository on A4X Max and G4, and Model Garden in Gemini platform.
• Download the weights: Grab the Apache 2.0 BF16, FP8, and NVFP4 checkpoints on Reflection’s Hugging Face page.
• Visit us at the llm-d and PyTorch Conference in San Jose: If you are attending PyTorch Conference North America in San Jose this October, stop by the llm-d community sessions and the Google Cloud booth #. We will be walking through live demos of KV-cache-aware routing, P/D disaggregation, and KV cache tiering for agentic workloads.
• Come see us at NVIDIA GTC Berlin: If you are attending NVIDIA GTC Berlin, come connect with us at our booth #3026, or our Google Cloud Hub at w3.hub: just a short 5 minute walk from the venue. Learn more about our sessions.

※この投稿は米国時間 2026 年 9 月 24 日に、Google Cloud blog に投稿されたものの抄訳です。

真の弾力性は、クラウドネイティブ エンジニアリングの究極の目標として長年追求されてきました。Kubernetes はリソース管理に革命をもたらしましたが、散発的に実行されるワークロード（バッチ プロセッサ、イベント ドリブン ワーカー、開発環境など）は、処理を待機している間もコンピューティング リソースを消費するため、費用の増加を招きます。

Google Kubernetes Engine（GKE）1.37 では、ゼロへのスケールダウンとゼロからのスケールアップをネイティブに実現することで、この問題に正面から取り組んでいます。新しい一連の機能を使用すると、ワークロードを完全にゼロレプリカにスケールダウンして、リソースの消費を停止できます。同時に、需要が戻ったときには、GKE のキャパシティ バッファでこれらのワークロードを迅速かつ簡単に再起動できるため、インフラストラクチャの無駄を減らすことができます。これは単に費用を削減するだけでなく、常時稼働のインフラストラクチャの費用をワークロードの準備状況から切り離すことにもつながります。

進化: HPA ベースのゼロへのスケーリングと KEDA

長年にわたり、Kubernetes Event-Driven Autoscaling（KEDA）というオプションの Kubernetes コンポーネントが、ゼロへのスケーリングの定番ソリューションでした。KEDA は強力ですが、環境が複雑になります。

ゼロへのスケーリングを GKE コントロール プレーンに直接組み込むことで、アドオン オペレーターや数千行の構成が不要になります。ロジックは「サイドカー管理」からワークロードのネイティブ属性に移行します。

仕組み: AutoscalingMetric と KEP-2021 を使用した HPA

GKE 内でゼロにスケールする鍵は、2 つの重要なコンポーネントの統合にあります。

1. AutoscalingMetric を使用した HPA: これは、Google Cloud Managed Service for Prometheus からの外部シグナルの直接読み取りをサポートするマネージド指標シグナル パイプラインです。AutoscalingMetric を使用した HorizontalPodAutoscaler（HPA）は、Pub/Sub、Cloud Monitoring、ロードバランサのシグナルからオートスケーラーに指標を送信するための、統合された高パフォーマンスのパスを提供します。アダプターの煩雑さを伴わずに済みます。

2. KEP-2021: HPA で minReplicas: 0 を可能にする Kubernetes Enhancement Proposal 

   に基づいて構築されたこのメカニズムにより、指標がしきい値を下回ると、HPA はすべての Pod を停止できます。また、指標が保留中の作業を示したらすぐに、HPA が Deployment を「起動」できるようにします。

最初の「ゼロへのスケーリング」ワークロードの構成

ネイティブなゼロへのスケーリングを実装するには、指標定義と HPA という 2 つの主要なオブジェクトが必要です。次の例では、Pub/Sub サブスクリプションの未配信メッセージ数に基づいてワーカーをスケーリングします。

指標のソースを定義する

AutoscalingMetric CRD を使用して、外部の Cloud Monitoring 指標をクラスタにマッピングします。

minReplicas: 0 を使用して HPA を構成する

HPA で指標を参照し、最小レプリカ数を明示的にゼロに設定します。

これで、外部指標に基づいてワークロードをゼロにスケールダウンしたり、ゼロからスケールアップしたりできるようになりました。

ゼロへのスケーリング機能は、Cloud Monitoring からの外部指標を GKE でサポートすることで実現しています。AutoscalingMetric カスタム リソースを拡張することで、複雑なサードパーティ アダプタを使用せずに、Google Managed Service for Prometheus から指標をクエリできるようになりました。これにより、レイテンシが短縮され、セキュリティが簡素化されます。また、ネイティブな「ゼロへのスケーリング」ワークロードを構成するための重要な基盤となります。このインテグレーションについて詳しくは、GKE での外部指標のネイティブ サポートに関するブログ投稿をご覧ください。

キャパシティ バッファによる起動レイテンシの管理

ゼロからのスケーリングにおける最大の課題は、いわゆるコールド スタートです。これは、GKE がノードをプロビジョニングし、コンテナがそれを pull して起動するまでにかかる時間です。そこで役立つのが GKE キャパシティ バッファです。

キャパシティ バッファは、プールされたウォーム状態のキャパシティとして機能します。GKE は、ゼロにスケールできる複数のワークロードで共有できる少量のウォーム コンピューティング リソースを維持することで、HPA が 0 から 1 にジャンプしたときに、Pod がすぐに要求できるリソースを確保します。これにより、新しい GKE ノードがスピンアップするまでの 60～90 秒の待ち時間がなくなり、起動レイテンシが数分から瞬時に短縮されます。その間、ワークロードの費用はゼロに抑えられます。

キャパシティ バッファには、アクティブとスタンバイの 2 種類があります。小さなアクティブ バッファは、ゼロにスケールされた数百のワークロードに対応できます。各ワークロードがレプリカを維持するのではなく、アクティブ バッファがクラスタ全体にサービスを提供するワイルドカード キャパシティとして機能します。スタンバイ バッファはアクティブ バッファに比べて大幅に低コストで、サイズが大きくなるため、クラスタで持続的な負荷が発生した場合にアクティブ バッファを迅速に補充できます。これらを併用することで、即時のスケーリングを実現しながら、低コストも維持できます。

今後について

Google は、GKE の弾力性に関するロードマップを継続的に拡大しています。たとえば、開発環境を午後 8 時にゼロにスケールダウンし、午前 7 時にスケールアップするとします。繰り返し発生するスケーリングをより細かく制御する方法に注目し、ゼロへのスケーリングの時間枠を事前に定義できるようにしましょう。

ゼロへのスケーリングを今すぐ始めましょう

アイドル状態のリソースに料金を支払う時代は終わりを迎えようとしています。イベント ドリブンなワークロードや散発的なワークロードに対して GKE のネイティブなゼロへのスケール機能を有効にすることで、起動パフォーマンスを犠牲にすることなく費用を削減できます。ゼロへのスケールを開始するには、次の手順を行います。

1. アイドル期間を伴う、需要が変動するワークロードを特定します。

2. AutoscalingMetric を構成し、minReplicas をゼロに設定します。

3. クラスタまたはワークロードにキャパシティ バッファを追加して、応答時間を短縮します。

詳細は、HPA を使用した GKE ワークロードのゼロへのスケールダウンとゼロからのスケールアップに関するドキュメントをご覧ください。

- Google Kubernetes Engine、プロダクト マネージャー、Eyal Yablonka

- Google Kubernetes Engine、シニア ソフトウェア エンジニア、Scott Funkenhauser

In production systems today, most AI agents operate as lightweight dispatchers. They parse a prompt, make an API call, retrieve a record, and hand back control. Their execution is fast, stateless and cheap. Biotechnology does not work this way. 

Folding a protein target, screening a 50,000 compound ligand library, or evaluating structure-activity relationships (SAR) are not quick API lookups. They are sustained, stateful, GPU-bound simulations that run for minutes or even hours across specialized silicon. When you ask a general-purpose large language model (LLM) endpoint to coordinate a complex drug discovery workflow, the standard paradigm breaks: Timeouts occur, context windows overflow with atomic coordinates, and proprietary chemical Intellectual Property (IP) risks exposure to external networks. 

Building enterprise grade research and development (R&D) agents requires moving past the monolithic assistant model. In this blog post, we will discuss how to use a three-layer architecture: separating cognitive orchestration (the brain) from computational execution (the muscle), and anchoring both inside an enterprise governance plane to build scalable agents for drug discovery use cases in biotech.

A three-layer pattern

The reference architecture below shows how to architect, optimize, and govern high-compute autonomous agents in biotechnology, specifically Drug Discovery.

The execution layer: scaling the muscle - high-compute execution 

The execution layer is responsible for the heavy scientific computations. The first architectural decision is determining where workloads should run:

• Strategic, proprietary pipelines require dedicated GPU infrastructure within your security perimeter

• Routine or standardized workloads can safely be delegated to managed cloud endpoints

In this architecture, protein folding is driven by ESMFold and generative design by GenMol. Both run as NVIDIA Inference Microservices or NIMs deployed on Google Kubernetes Engine (GKE) using G4 VMs backed by NVIDIA RTX PRO 6000 Blackwell GPUs.

A critical hurdle in containerized GPU workloads is the cold-start penalty, the latency delay required to pull multi-gigabyte model weights from an image registry and stage them into GPU memory (VRAM).

To ensure low latency responsiveness, we build the following as a recommendation into the reference architecture:

• Maintain warm pools: Pin the GKE node pool to a minimum of one active node.

• Preload weights: Model weights are loaded directly into VRAM at pod initialization, ensuring containers are immediately ready for inference.

• Resilient scheduling: GKE manages pod lifecycles and restarts failed instances without tearing down the underlying VM infrastructure. 

This approach trades custom, manual CUDA kernel tuning for NVIDIA's pre-compiled, containerized runtimes, dramatically reducing infrastructure engineering overhead while ensuring deterministic throughput.

In biotech, chemical structures represent millions of dollars in proprietary IP. A docking request transmits SMILE strings or 3D coordinate matrices that must never leave company custody.

Containerized NIMs eliminate the friction between rapid prototyping and rigorous data isolation. During early exploration, researchers can query NVIDIA hosted endpoints. When transitioning to proprietary lead candidates, the exact same container is deployed to GKE inside a private VPC. The agent’s orchestration logic remains identical; only the target URI updates from an external proxy to an internal VPC endpoint. No structural data leaves the enterprise perimeter.

The cognitive layer: Multi-agent reasoning

With compute execution isolated, the cognitive layer manages agentic workflow sequencing. Built on Agent Runtime, within the Gemini platform, this reference architecture shows a system that deploys a hierarchical multi-agent structure, one lead orchestrator, and two specialized domain sub-agents (Lead Optimization and Safety Screening) . The multi-agent design can be modified to suit the use case at hand. 

Based on this design, when an end-user submits a high-level task such as “ Analyze Candidate 4b against Target X and produce a SAR report”, the system applies four core orchestration patterns discussed below; 

• In-process native agent handoffs: The lead orchestrator delegates tasks to specialist sub-agents using Agent Runtime’s native SDK rather than network-serialized agent-to-agent HTTP protocols. This eliminates intermediate network hops and keeps internal handoffs instantaneous. Only the outermost boundary communicates outward to user-facing surfaces like the Gemini agent.
• Passing bulky payloads by reference: LLMs degrade quickly when forced to ingest massive arrays of 3D spatial coordinates. In this architecture, raw structural data (PDB files, docking poses, trajectory logs) is written directly to Cloud Storage. Agents pass lightweight, signed object URIs and high-level structural summaries, keeping context windows compact and inference costs down.
• Decoupling workflow logic from biochemical reasoning: General-purpose LLMs excel at task planning and synthesis, but are prone to hallucinating biochemical nuances. To control this behavior, this architecture maintains strong separation of concerns, namely, workflow and tasks decomposition are handled by Gemini. Domain chemistry and pharmacological reasoning are handled by specialized Nemotron models hosted on dedicated GPU nodes. 
• Context-driven evaluation over rigid cutoffs: Traditional computational screening enforces arbitrary numerical cutoffs (e.g., discard if binding affinity ,-7.5 kcal/mol) However, therapeutic context fundamentally alters risk tolerance. Preventive cardiovascular therapy taken daily by healthy patients requires extremely strict safety margins, or a novel oncology candidate targeting an intractable cancer may accept higher toxicity if efficacy is demonstrated.

By passing disease context directly into the specialized reasoning models, drug candidates are evaluated against dynamic, target-specific profiles rather than static thresholds, preventing the premature culling of viable drug leads.

Performance best practices for high-compute pipelines

Optimizing an agentic system that interfaces with heavy GPU compute requires challenging common engineering instincts:

The enterprise governance and asymmetric autonomy

Deploying high-compute agents in enterprise environments requires robust data sovereignty, audit trails, and process control. The Gemini platform establishes these safeguards natively with the following components: 

• Agent Identity: Every autonomous agent operates with a cryptographically verifiable identity. Thus, for a biotech use case, when a candidate molecule is advanced or rejected, the event is permanently logged to that specific agent principal. 

• Agent Gateway: This serves the single ingress/egress point for every interaction within the perimeter. Centralized IAM and regional residency rules help ensure sensitive molecular data remains within designated cloud regions. 

• Agent Registry: This feature maintains a curated catalog of approved agents, validated tool schemas, and authorized Model Context Protocol (MCP) servers, preventing rogue tools from executing in production. 

• Model Armor: This functionality inspects incoming prompts and outgoing payloads in real time to prevent prompt injection and data exfiltration. 

The payoff is clear. Identity, RBAC, auditing, residency, and data protection are native properties of the Gemini platform; this allows enterprise customers to focus on science rather than having to build a compliance stack for every project. 

In high-stakes research and development, full autonomy introduces unacceptable liability, while requiring approval for trivial steps can create frictions. The solution addressing this to use asymmetric gating in the following ways;

Automate reversible operations: For example, allow the agent to resolve chemical naming synonyms, pull structural PDBs, and run preliminary Lipinski screens autonomously without interrupting the researcher.

Gate irreversible operations: For instance, initiating multi-GPU docking run, advancing a lead compound to wet-lab synthesis, or archiving a therapeutic program must require explicit human sign-off.

The pattern, and the path forward

High-compute autonomous agents succeed when the boundaries between cognitive planning, numerical computation, and system governance are clean in the following ways:

1. Keep the orchestrator focused on decomposition and synthesis, leaving intensive biophysical simulation to specialized, containerized microservices and models.

2. Eliminate context bloat by passing complex chemical data by reference through cloud storage.

3. Anchor the workflow in enterprise governance, pairing cryptographic agent identity with asymmetric human oversight.

As generative and structural biology models expand toward whole-cell modeling and complex dynamic simulations, this decoupled architecture ensures research systems scale efficiently while keeping proprietary IP secure and auditable.

Reach out to your Google sales representative if you have any requirements to build a use case like this and we will be happy to talk and support you in your agent development journey on Google Cloud.

The real challenge of building an agentic system isn't the construction, it's the measurement. 

Evaluation engineering is a critical foundation for building reliable AI systems. Setting up systematic, repeatable tests to measure your agent’s true behavior can be the difference between shipping a tool you can trust and one that silently breaks, fails to call APIs, or makes mistakes. 

I’ve run evaluation engineering initiatives for years at Google and developing the newly released Google Cloud Developer plugin served as a live testbed to stress-test these key principles across real-world developer workflows.

Here are four eval lessons we learned from building the new plugin that you can apply to your own builds.

1. Grade the execution steps instead of the summary

When we first evaluated agents, we set up an LLM-as-judge to inspect the agent’s final text reply and scored it against a checklist. However, we quickly realized it’s fundamentally flawed because the grader only sees what the agent wrote, not what it actually did. When we ran our initial 11-prompt evaluation suite for the Google Cloud Developer plugin, the automated grader marked it as failing three prompts. But when we opened the raw tool-call logs and read the transcripts, the plugin had actually passed all three. On a Cloud Run cleanup prompt, for instance, our rubric required checking command syntax before running anything. The agent ran gcloud help run services list as its very first step. It did the work, but its final message was focused on warning the user about permanent data loss rather than reciting that it had checked the help docs.

The grader penalized the agent for doing the right thing simply because it didn’t brag about it in the summary. If your rubric line says "did it run X?", a LLM judge is guessing unless you evaluate the actual tool-call trajectory.

Takeaway: Don't grade an agent on its closing pitch. Score the raw execution trajectory to verify what commands were actually executed, validated, and scoped.

2. Design evals for asynchronous operations

When you start reading tool-call trajectories, how you extract that telemetry matters just as much as what you're reading. Extracting agent telemetry into a flattened, sequential list of events obscures concurrent tool dispatch, which is one of the critical causes of behavioral failure. Modern agent harnesses often batch multiple tool invocations into a single assistant turn using an array of calls.

In our evaluations, when an agent was asked to inspect an environment and look up documentation, the harness dispatched the skill-loading call and two documentation searches simultaneously within a single tool_calls payload. Because activating a skill is a multi-stage process, where the initial call returns a short launch stub and the subsequent turn injects the full instruction body, the instructions meant to govern the search arrived in the prompt context after the system had already executed the searches.

The system never gave the skill a dedicated turn to let its guidance influence the agent's actions. If telemetry pipelines strip out turn structures and merge calls into a flat timeline, engineers will waste days wondering why an agent disregarded unambiguous instructions that a developer embedded in a skill.

Takeaway: Preserve turn structure in telemetry extractors so you can identify when client-side parallel dispatch issues tool calls before injected guidance enters the context window. 

3. Test live runs and "dry runs" side-by-side

You need to inspect real API calls to catch harness quirks, like parallel dispatch. But if you only test your agent in one environment condition, you're flying half-blind. From day one, you need to run two conditions side-by-side: a live run and a plan-only run. In the live condition, the agent has real, sandboxed credentials and actually executes commands. This is how you see if syntax works, parameters serialize cleanly, and the agent exercises restraint before doing something destructive. 

Live runs frequently hit an unexpected permission error on step two and abort before you ever reach the behavior you’re trying to test. That's why you need the plan-only condition, where you strip credentials and tell the agent to explain what it would do without running anything. This strips out permission roadblocks and shows you the model’s pure reasoning and routing intent. Roughly half our most important findings came from the delta between the two.

For example, on one of our onboarding prompts, a live run was blocked immediately on a project list call. The plan-only run sailed right through and produced an 8,000-character plan that completely ignored organization policies, admin approvals, and governance. The live run masked a major cognitive flaw that the plan-only run exposed in seconds.

Takeaway: Run live and plan-only evaluations side by side. Live runs test actual execution and safety restraint; plan-only runs expose the model's raw reasoning and routing without getting tripped up by permission errors.

4. Use “golden sets” as durable baselines

Frontier model architectures change every few months, agent harnesses improve, and plugin manifest standards will inevitably evolve. When building the Google Cloud Developer plugin, we treated evaluations as continuous integration testing for these non-deterministic environments. That means we stress-tested the composite stack, including the harness, the live model, and the tools running together on realistic tasks.

Instead of guessing from the final text output, this approach captures the whole trajectory: what the agent thought, what commands it actually ran, and whether it solved the problem without breaking anything. Then you score that run against golden sets curated by real human experts who know what the ideal path looks like.

Takeaway: Build golden sets based on what the entire system should actually do in the wild, including your harness, model, and plugins. Models change, but those baseline runs are your safety net.

Reminder: Use evaluation suites over “one-shot” demos

A common mistake I see among engineers is what I like to call the one-shot demo. This is when someone feeds a prompt into a LLM, watches it build a complete 3D game or solves a Rubik’s cube in a single pass and calls it an eval. But in reality, this is just a vanity metric. 

To do quality work with AI, I truly believe that you need evaluation suites. Without evaluations, teams are like literally running into a dark cave blind. Engineering an enterprise-grade agent means testing the system's ability to handle multi-step interactions, manage permissions, and avoid destructive outcomes like accidentally deleting a project because you asked it to "find opportunities to save costs."

Where to go from here

Building an agent plugin or MCP server is the beginning. Shipping something you can actually trust with real cloud infrastructure requires significantly more work in evaluations to ensure it’s doing what you expect it to do. At the end of the day, the only thing that outlasts model churn is a stress-tested eval suite. Check out the Google Cloud Developer plugin documentation or explore this codelab to get started with the plugin in Antigravity.