GCP News - 2026-10-16

Editor's note: Generative AI startups are moving rapidly beyond static clip generation to build interactive, real-time worlds. In this technical deep dive, we explore how AI startup Visko collaborated with Google Cloud TPU engineers to deploy Orbis — its continuously streaming Live World Model—on Cloud TPUs, going from initial kickoff to production handover in just 30 days.

What makes a live video world model so challenging to serve in production? The bottleneck isn't average throughput — it's beating a relentless real-time clock. Unlike offline video generators where a user submits a prompt and waits in a queue, a live world model streams continuously. Every new chunk of video and synchronized audio must finish rendering before the viewer finishes watching the previous one, or the world freezes.

Today, we're excited to share how Visko, an AI startup pioneering interactive world models, teamed up with Google Cloud to solve this exact challenge. Working side by side, Visko and Google Cloud engineers deployed Visko's end-to-end live inference pipeline — combining video generation, super-resolution upscaling, and synchronized audio—onto a single Cloud TPU slice. Together, the teams brought end-to-end generation down, rendering at 1.07x playback speed.

Here is an inside look at how we engineered the pipeline to hit that milestone in a single month, the architectural optimizations that moved the needle, and practical lessons for other startups building ultra-low-latency generative AI on Google Cloud.

Redefining interactive media with Live World Models

While most generative video platforms focus on rendering fixed, pre-baked clips, Visko is building what it calls Live Models: video world models that generate continuously and respond on the fly.

Visko's flagship model, Orbis, streams live video starting from a single text or image prompt. As the stream plays, viewers can enter new steering prompts in real time, dynamically evolving the scene without restarting or breaking continuity. For instance, in one test session, the livestream opens on a cat napping beside a crackling fireplace; when the viewer steers the scene to introduce a bowl of food, the bowl materializes seamlessly, and the cat stretches, stands up, and meows.

Since launching its public demo on September 1, 2026, Orbis has unlocked new possibilities across interactive entertainment, live commerce, AI companions, immersive education, and robotics simulation. To scale these interactive sessions reliably, Visko partnered with Google Cloud TPU engineers in August 2026 to port the Orbis live pipeline to Google's Tensor Processing Unit (TPU). Within 30 days, the full multimodal pipeline was running in real time on TPUs.

Visko’s ambition is to power a new generation of interactive worlds that people can step into and shape in real time. Streaming continuous video and audio under a strict clock is a fundamentally different system challenge than offline clip generation. Working side by side with Google Cloud's TPU engineers, we brought the full Orbis pipeline to TPUs in just 30 days — giving us the low-latency compute and memory bandwidth required to bring Live Models to creators and developers at scale.

"Visko is bringing an exciting new class of real-time, steerable workloads to the Cloud TPU ecosystem. By co-engineering their live multimodal pipeline on a singleTPU slice using JAX, torchax, and custom Pallas kernels, we proved how quickly a lean startup team can go from PyTorch prototype to render at 1.07x playback speed on Google Cloud's AI Hypercomputer."
– Dave Elliott, Head of Developer Advocacy for AI, Google Cloud 

Live video is a hard-deadline problem

When you serve an offline clip generator, latency is forgiving — the user waits once while the backend processes frames. A live world model has no waiting room.

Every single chunk must be denoised, decoded from latents, upscaled to 1664x936 (at 18 fps), and paired with generated audio faster than realtime playback speed, for as long as the session runs.

Two structural constraints make live world models far more demanding than standard batch inference:

• Batch size of one: Each live stream represents an individual viewer's interactive session. You cannot mask latency by grouping dozens of concurrent requests into a massive batch — the standard playbook for keeping AI accelerators saturated.

• Strict sequential dependency: Every chunk depends directly on the one before it. Because the video model maintains a rolling memory of recent video latents so that chunk N+1 smoothly continues the action from chunk N, you cannot pre-render chunks out of order or parallelize across future timestamps.

For startup infrastructure engineers coming from LLM serving, it helps to think of live world modeling not as a tokens-per-second throughput benchmark, but as an unforgiving per-step latency SLO.

Architecting the multimodal pipeline on a single TPU slice

To keep infrastructure lean and cost-effective as Visko scales, the team designed the live pipeline to chain three distinct generative models on a single Cloud TPU slice running in BF16 precision.

When a viewer submits an initial prompt or a live steering update, all three generative models — the core video diffusion model, the super-resolution upscaler, and the audio generator — share the same chips, taking turns on the hardware for each chunk.

To combine rapid startup iteration with bare-metal accelerator speed, the software stack is built on three open-source pillars:

• JAX: Google's high-performance Python array library for accelerators, compiled into optimized machine code by the XLA compiler.

• torchax: An interoperability layer that runs PyTorch models directly on Cloud TPUs via JAX, enabling startup teams to bring existing PyTorch components over without an upfront, ground-up rewrite.

• Pallas: The custom kernel programming language for TPUs in JAX — the TPU counterpart to writing custom Triton or CUDA kernels on GPUs.

Real-time performance breakdown: Hitting the clock 

By profiling and optimizing each stage on a single slice in BF16, the engineering team achieved a total per-chunk latency of under the target while — maintaining roughly 7% headroom to absorb variance during live streaming:

Because video denoising accounts for nearly two-thirds of the total compute budget, that stage became the primary focus of our kernel and parallelism optimizations.

Inside the stack: Three optimizations that unlocked real-time streaming

1. Ulysses context parallelism across all TPU chips

A video chunk expands into a massive sequence of spatiotemporal tokens inside the video transformer. To distribute that workload across the TPU chips at batch size 1, the team implemented Ulysses context parallelism.

Instead of splitting model weights, Ulysses splits the token sequence across chips. Immediately before the attention layer, a high-speed all-to-all interconnect exchange regroups the tensors so that each chip holds the entire video sequence, but only for one-quarter of the attention heads. After each chip computes attention over the full sequence for its assigned heads, a second all-to-all exchange restores the sequence-split layout for the feed-forward layers.

2. Custom Pallas splash attention from MaxDiffusion

To accelerate the attention computation itself, the team integrated a custom Pallas splash attention kernel from MaxDiffusion, Google Cloud's open-source reference library for diffusion models on Cloud TPU.

Splash attention brings FlashAttention-style tiling to TPU architecture: rather than materializing the full quadratic attention matrix in High Bandwidth Memory (HBM), it computes attention in compact tiles that remain inside ultra-fast on-chip VMEM. Startup teams can drop the exact kernel used in this port straight into their own pipelines via custom_splash_attention.py.

3. A 5.8x audio speedup by moving from torchax to pure JAX

We first brought all three Orbis models onto Cloud TPU with *torchax*, which runs PyTorch code on TPU through JAX and XLA without a rewrite. For the video and upscaling models, `torchax` paired with Pallas attention kernels met our latency target directly, and both stay in `torchax` in the final pipeline.

Only the audio model needed a rewrite. Moving the audio stage from `torchax` to *native JAX* made it *5.8x faster* and brought the full pipeline under the playback deadline.

Startup Builder Tip: Port strategically.
The takeaway isn't "skip torchax" — it's to use it as a fast bring-up bridge. torchax got our PyTorch models running on TPU in days rather than weeks, giving us the end-to-end telemetry needed to pinpoint true bottlenecks. Measure the full pipeline first, and invest engineering cycles rewriting only the components that dominate your latency budget.

Field notes for AI builders: Cloud TPU lessons

Takeaways from tuning live multimodal inference on Cloud TPU:

- *Ulysses context parallelism for single stream video:* At batch size 1, splitting the token sequence across chips with `all-to-all` interconnect exchanges was faster than both tensor parallelism and ring attention.

- *Native BF16 for continuous streams:* FP8 quantization cuts memory bandwidth pressure, but small quantization errors build up across chunks in a continuous stream. Native BF16 kept visual quality intact within the latency budget.

- *Selective JAX ports:* `torchax` kept the video and upscaling models in PyTorch, so the team only rewrote the audio bottleneck in native JAX.

- *XLA compilation caching:* The first boot spends a few minutes compiling XLA graphs. Caching compiled artifacts brings worker restarts under a minute.

Building real-time generative models on Cloud TPU? Start here

If your startup is pushing the boundaries of live video, world models, or ultra-low-latency multimodal inference, you can build with the same toolkit that powered Visko's port:

• Accelerate diffusion attention out of the box: Grab the open-source splash attention kernel used in this collaboration from MaxDiffusion, and explore the Pallas documentation to write custom TPU kernels.

• Migrate PyTorch fast, then optimize selectively: Bring your existing PyTorch models onto TPU quickly with torchax, pinpoint bottlenecks with the Cloud TPU Profiler, and port latency-critical stages to native JAX.

• Spin up TPUs on demand: Use TPU Flex-start VMs to provision TPU capacity for short development and benchmarking runs without a long-term reservation.

• Scale with Google for Startups: Early-stage AI startups looking for cloud credits, technical mentorship, and access to Google Cloud's AI Hypercomputer stack can apply to the Google for Startups Cloud Program.

^{Special thanks to the Visko.ai team for the close technical partnership, and to the Google Cloud TPU engineers who lead the project including: Yixin Shi, James Huang, and Elisa Tsai.}

As enterprises shift their AI usage from experimental chatbots to autonomous agents, teams are hitting a critical bottleneck: accessing and reasoning over data scattered across multiple clouds, SaaS platforms, and on-premises systems. To turn AI agents into true systems of action, organizations need a foundation that can handle petabyte-scale reasoning without the security risks of data movement. Last week at our Gemini at Work event, we announced how Gemini can help you transform business workflows. To help power your agents with your enterprise data, we will dive deeper into the new Gemini integration with borderless Lakehouse to help you build, secure, and scale agents that work across your data estate.

Access data that lives outside of Google Cloud through Gemini 

The ultimate value of the borderless Lakehouse is realized through the agents you build to transform your organization, with Gemini as the front door. With the new Gemini integration, data that’s either preconfigured or stored in your borderless Lakehouse is now directly integrated into Gemini. Currently in preview, this means you can set up access to new data sources, such as Databricks Unity Catalog or Snowflake Horizon, through Lakehouse catalog federation directly from Gemini. 

Now your business users can use natural language to query records that live across clouds while your AI agents can ground their reasoning and generate code from the same data — all without moving it. By directly contextualizing AI agents across your borderless Lakehouse, you can help accelerate your organization’s AI adoption while simplifying access to critical information.

Borderless Lakehouse advancements

In addition to this new, native integration with Gemini, we’re sharing  that several features we introduced earlier this year to power your data foundation with the borderless Lakehouse are now generally available, including:

• Support for Cross-Cloud Interconnect and cross-cloud caching for AWS and Azure

• BigQuery support for Apache Iceberg V3 specifications

• Read-write interoperability on Apache Iceberg tables for both BigQuery and Managed Service for Apache Spark

• Support for AlloyDB lakehouse federation 

• Bi-directional caching with AWS Glue, Databricks Unity Catalog, and Snowflake Horizon Catalog using Lakehouse runtime federation 

• Integration with Knowledge Catalog

• Zero-copy data integration with Salesforce, SAP and Workday, and now, Confluent Tableflow

• Synchronization of Databricks Delta Lake assets through the Lakehouse console using Dataflow

Let’s explore.

Performance and reasoning at petabyte scale

While agent context windows are typically limited to ingesting hundreds of kilobytes at a time, enterprise workflows sometimes join petabytes of data across clouds. An agent-native data platform that needs to combine data across sources, like product and customer data, to your CRM cloud and warehouse telemetry needs to be able to process data at scale. When you do this over MCP, you are limited to megabyte scale because of context windows and payload size. By integrating at every layer of the data and AI stack, the borderless Lakehouse provides enterprise-grade scalability, enabling agents to reason over petabytes of data with similar performance to it being stored locally. 

At the networking layer, Cross-Cloud Interconnect and cross-cloud caching provide high-throughput, low-latency paths to AWS and Azure, eliminating variable egress fees. When an agent joins Iceberg tables in Amazon Azure Data Lake Storage with data in BigQuery, every uncached byte it scans crosses a cloud boundary, so the network sets the pace. Cross-Cloud Interconnect gives the Lakehouse a dedicated, private connection between Google Cloud and AWS or Azure, at 10 Gbps or 100 Gbps per connection. Lakehouse queries travel over that connection rather than the public internet, so latency stays low and predictable. Cross-cloud caching serves repeat reads from inside Google Cloud, so agents can query data in other clouds without migrating it.

The borderless Lakehouse includes several key optimizations to integrate Iceberg metadata within the query engine. BigQuery now supports Apache Iceberg V3 specifications, including binary deletion vectors and sort orders. These features allow the engine to handle row-level updates and deletes more efficiently by eliminating the need to rewrite entire data files. We also upgraded BigQuery’s execution engine with a new FlatBuffers-based Parquet reader and parallelized I/O to help reduce CPU overhead and serialization bottlenecks and narrow the performance gap between open formats and native storage.

This builds upon recent performance innovations to the query engine designed to support agent workloads, which often come in herds and need to process small queries quickly and cost effectively. With BigQuery fluid scaling and history based optimizations, you can run scalable and efficient agent workloads while maintaining openness with Apache Iceberg. 

Meanwhile, read-write interoperability on Apache Iceberg tables for both BigQuery and Managed Service for Apache Spark makes it easy for all of your data teams to work off the same Iceberg tables in Google Cloud Storage. This vertically optimized stack helps make your agents faster and more cost-effective, leveraging the power of BigQuery and Lightning Engine to activate data without moving it. To expand the analytical depth of the lakehouse, graph support for Iceberg in BigQuery lets you analyze relationship networks, such as supply chains, directly on open table datasets.

Then, for operational data, AlloyDB lakehouse federation lets you query live database records and historical warehouse data together from a single connection without copying any information. This simplifies your system and lowers costs while giving agents instant access to both real-time updates and deep background history with the borderless Lakehouse.

Comprehensive context and data access for your agents

An AI agent is only as smart as the context it can access. To help prevent hallucinations and improve accuracy at the metadata layer, the borderless Lakehouse unifies your enterprise data using the Lakehouse runtime catalog. Based on the Apache Iceberg REST Catalog, it provides bi-directional catalog federation with AWS Glue, Databricks Unity Catalog, and Snowflake Horizon Catalog. The Lakehouse runtime catalog also includes a new standard Hive Metastore-compatible endpoint, allowing engines like Apache Spark, Trino, and Presto to discover metadata without architectural changes. 

The borderless Lakehouse also integrates directly with Knowledge Catalog, which automatically indexes your distributed data assets, to provide the precise business metadata and semantic definitions agents need to reason over your entire enterprise corpus — including unstructured data in Cloud Storage. This gives agents just enough context to be accurate while controlling token costs, helping you develop an intelligent and sustainable agentic strategy. 

The borderless Lakehouse provides zero-copy data integration with your SaaS applications like Salesforce, SAP and Workday and integrates with a larger partner ecosystem, capturing enterprise data in Knowledge Catalog and acting as a single context engine for your agents. Today, we’re expanding the partner ecosystem with new integrations for Confluent Tableflow, now generally available, which syncs to Lakehouse runtime catalog so the Iceberg tables it produces show up directly in BigQuery, with no copying and no extra ETL.

Finally, organizations with Databricks-centric repositories can now easily import, transform and synchronize their Delta Lake assets into Lakehouse through the Lakehouse console using Dataflow.

Modernize with ease

Customer use of Apache Iceberg has more than tripled since the beginning of 2025, and enterprises building with Google Cloud’s borderless Lakehouse are seeing significant impact from it: 

• Etsy unified three petabytes with borderless Lakehouse, joining data in place to accelerate pipeline queries by 60%. 

• Commerzbank transforms raw distributed data into governed, agent-ready access.

• Hightouch integrates directly with borderless Lakehouse so teams can power their marketing directly on an open Iceberg lakehouse and activate that data across the Google Cloud ecosystem.

• Roundel modernized its Target Product Ads technology stack from the ground up with borderless Lakehouse, completely rebuilding its data ingestion, processing, and measurement pipelines. 

If you are ready to get started with the transition to borderless Lakehouse for your organization, it doesn’t need to be difficult. For organizations currently using legacy Dataproc Metastore, we now offer an automated, zero-downtime migration path to the serverless Lakehouse runtime catalog. This preserves your existing schemas and lineage while unlocking the AI-ready features of the borderless Lakehouse, so you can stop managing infrastructure and start activating agents.

Get started today with the Build a borderless Lakehouse codelab or visit our website to learn more.

The open-source Temporal execution platform is rapidly becoming a foundational layer of modern software infrastructure — especially for orchestration in distributed systems and AI agent workloads. However, deploying and scaling workers in Temporal comes with unique challenges. Workers poll for tasks instead of serving requests, so request-driven serverless platforms can be an awkward fit. This often results in running worker fleets on Kubernetes or VMs, which you have to manage and pay for continuously. You build a metrics pipeline and tune autoscaling triggers, and scaling still trails bursts because of metrics lag. Every deployment means juggling upgrades and versions, and you pay for that compute whether or not there's work in the queue.

We built Cloud Run worker pools for exactly this kind of pull-based work: Your instances connect to a queue or stream, take the next item, and process it. Starting today, Temporal Cloud, which is the fully managed version of Temporal’s open-source platform, can scale them for you. Currently in preview, Temporal Serverless Workers on Cloud Run let you deploy your workers as a worker pool, and Temporal sizes the pool to match the work on your task queues. When there's no work, it can scale the pool to zero. There's no cluster or autoscaler to run, and your worker code stays the same.

How Cloud Run worker pools fit with Temporal

A worker pool is Cloud Run's resource for continuous background work. It has no public endpoint, and each instance is allocated CPU for its whole lifetime, which matches how a Temporal worker runs. Temporal manages your Cloud Run worker pools as follows:

1. Your Temporal workers run as Cloud Run worker pool instances in your project and poll Temporal Cloud for tasks.

2. Temporal Cloud tracks your task queues and sets the worker pool's instance count through the Cloud Run Admin API, using the IAM permissions you grant. Temporal never opens a connection to your instances, so you don't need to open ports or add firewall rules for it.

3. You run each build version as its own Cloud Run worker pool. An older build's worker pool can sit at zero instances until a workflow that started on it needs it again.

Your Temporal workers run in your project, under your service account, and on your network. Temporal decides how many of them run.

What you can build with Temporal Serverless Workers

The combination of Cloud Run worker pools with Temporal Serverless Workers opens up a variety of interesting use cases:

Autoscaled agentic workers: AI agents run multi-step chains of LLM reasoning and tool calls. Temporal's durable execution and Cloud Run worker pools work together here. Temporal workers spin up instantly to handle bursts of LLM steps, embeddings or API calls, then scale to zero when the queue empties. Workers get keyless access to Google Cloud AI APIs. You can also attach a GPU for inference, and the workers attached to the GPUs will still scale to zero. Temporal runs each model and tool call as a durable activity; if a worker disappears mid-step, the activity retries on another worker and the workflow continues.

Long-running Workflows: Temporal Workflows can run indefinitely, allowing for building applications that track the state of real-world resources over long periods of time. This pattern is typically known as the “Entity Pattern,” because the Workflow represents the state of some entity. However, these long-running Workflows don’t usually have consistent compute requirements: Some individual steps are short, some are long, and other times, the Workflow is sleeping, waiting for external input, such as from a human or another system. Cloud Run worker pools provide flexible compute to support this pattern: Individual steps can run for as long as is required, and Temporal will automatically scale down the worker pool when there is no work to be done. 

Bursty batch work: Worker pools bill only while instances run, so a worker pool that sits idle between batch runs costs nothing while it's at zero instances. For jobs like billing runs or media transcoding, Temporal scales the pool out as batches arrive and back in once the queue drains. Your maximum instance count caps how many instances you pay for. Tasks beyond that capacity wait in the queue rather than being dropped.

Why Cloud Run worker pools for Temporal workers

Before, running Temporal workers on Cloud Run services took some workarounds:

• an HTTP listener, because services expect containers to listen on a port

• instance-based billing, so workers kept their CPU between requests

• minimum instances, so the service didn't scale to zero

By running Temporal workers on Cloud Run worker pools, you don’t need those workarounds anymore, and you have:

• No endpoint to manage. There's no public URL or load balancer to lock down, and no health-check port to implement.

• Steady CPU at a lower price. Instances don't idle or throttle. Cloud Run doesn't remove instances because they look idle, so Temporal alone decides when the pool shrinks. Worker pools list at almost 40% less per vCPU-hour and GB-hour than instance-billed Cloud Run services.

• Scaling without revision churn. Changing the instance count doesn't create a new revision, so Temporal can resize the pool as often as the load changes.

How Temporal manages scaling and versioning of worker pools

Temporal decides when to add capacity using one main signal: the delay between a task landing in the task queue backlog and a worker picking it up. If that delay grows, the available workers are saturated. Scaling occurs as follows:

• Server-side detection: The Temporal server tracks this backlog-to-worker latency for each task queue.

• Push notification: When the latency crosses its trigger conditions, the server sends a push notification to the worker controller instance.

• Provisioning: The worker controller then adds capacity to the worker pool. New instances take time to start and connect before they can poll for tasks, so capacity lags behind the signal.

Because Temporal keeps spare capacity, the pool grows ahead of demand instead of after tasks pile up. Scale-in is more conservative. Temporal waits for a stabilization period after the last time a task found no available worker (90 seconds by default), and it pauses between successive reductions. Cloud Run’s configurable graceful shutdown extends shutdown from 10 seconds to 10 minutes. That gives SIGTERM-aware workers time to finish or checkpoint. The instance count stays between a minimum and a maximum that you set. If the first task after an idle period can't wait for an instance to start, set the minimum above zero.

Deploying new workflow code without breaking running executions is hard. Temporal workflows are stateful and deterministic, so if you swap their code mid-run, you get non-determinism errors. Serverless Workers on Cloud Run solve this by requiring Temporal Worker Versioning. You don't patch code with conditional branches. Instead, you deploy each Worker Deployment Version as its own worker pool deployment, identified by a deploymentName and a buildId. Using distinct worker pools solves two problems:

1. Workflows that started on a particular Worker Deployment Version continue to run and complete on that version.

2. Each Worker Deployment Version is scaled and drained individually, matching compute spend and utilization to the requirements of your application. As existing workflows complete and drain on a previous Worker Deployment Version, the underlying Worker Pool is eventually scaled down to zero automatically.

You set a versioning behavior for each Workflow type, or set a Worker-level default:

• PINNED: A workflow stays on the Worker version (Build ID) it started on and finishes there. Best for short-running workflows, order processing and multi-step APIs.

• AUTO_UPGRADE: A workflow moves to the new "Current" or "Target" Worker Deployment Version as you roll it out. Best for long-running workflows, such as subscription billing or tasks that run for weeks.

Learn more about configuring worker versioning here.

Get started

Setup takes three steps:

1. Containerize your Temporal Worker and deploy the container as a Cloud Run worker pool, with the Build ID in the pool name. Keep credentials such as your Temporal API key in Secret Manager.

2. Connect Temporal to your worker pool by granting Temporal Cloud permission to manage the worker pool so it can scale it.

3. Create a Worker Deployment Version in Temporal that points at the pool's project, region, and name.

Temporal Serverless Workers on Cloud Run is in preview, so APIs and configuration may change before the stable release. To get started,

• Learn how Serverless Workers on Cloud Run scale and handle versions.

• Deploy your first worker with the setup guide.

• See Cloud Run worker pools for CPU, memory, GPU and networking options.

• Sign up for the configurable graceful shutdown preview.

Small teams, short runway, and constant pressure to ship faster — it’s the reality we see every day among the founders we work with at Google for Startups. When you are building a company from the ground up, every hour spent on operational overhead or infrastructure plumbing is an hour taken away from your core product.

Last week at Gemini at Work 2026, we introduced the Gemini agent: a single, universal agent designed to work both as your personal assistant and as a dedicated member of your team. And this week, we made our new frontier model, Gemini 4 Argon, widely available — trained to maintain context across long, multi-step tasks.

Here is a look at how the Gemini agent works, five practical takeaways for founders, and how startups are already using it to punch above their weight.

What is the Gemini agent?

Instead of prompting separate chatbots for writing, coding, and data analysis, the Gemini agent is a single agent that takes high-level objectives and returns finished work. Three principles define it:

• Unified: A single agent can now handle knowledge work, data analysis, creative asset generation, and writing and running code. For complex jobs, it dynamically spins up temporary, job-specific sub-agents and coordinates their work in parallel or in sequence.

• Omnipresent access: It works across the surfaces you use every day — on the web, command line, desktop, and mobile, as well as natively inside Gmail, Docs, Sheets, Slides, Drive, Chat, and Calendar.

• Persistent: It runs in the cloud with continuous memory across four dimensions (session, semantic, procedural, and episodic). It onboards itself like a new hire — learning your team, tools, and business context so you don't have to re-explain your startup with every prompt, and keeping multi-day tasks moving even after you close your laptop.

For founders wearing five hats at once, Gemini works like a digital chief of staff that's already briefed on your calendar, projects and documents. Ask it to research market trends, build a financial model in Sheets and turn both into a pitch deck, without repeating what the project or goals are at each step. It also works ahead of you. When an investor emails asking for your latest product update as a deck, Gemini spots the request and offers to handle it; one click and it comes back with completed work, ready for sharing or review.

Five takeaways for startups

1. Coworker agents and skills help you scale your output

Beyond personal assistance, you can create persistent coworker agents that work with your entire team. Each agent comes with its own Workspace account, email address, Drive storage, and directory presence. You can define a specific role — such as a launch coordinator, QA triager, or financial analyst — and your teammates can add agents to a Chat space or @mention it in a Google Doc comment to delegate work.

You can also equip agents with reusable skills — or let Gemini write skills for itself as it learns your processes. For example, with operational reporting skills connected to BigQuery and the Knowledge Catalog, anyone on your team can ask a plain-language question to build and save a live report. Once saved, those reports run on demand without incurring additional token costs. Gemini also offers pre-built specializations, starting with legal and financial services.

• What it means for startups: Coworker agents can immediately increase the capacity of your teams, and skills can standardize repeatable processes like investor and operational reporting without burning tokens on every run.

2. Open and third-party model choice

Gemini is the agent, and the model underneath is your choice. As startup architectures mature, sending every task to a single frontier model quickly eats into gross margins. With the Gemini agent, you can dynamically route workloads across Gemini models, third-party models like Anthropic’s Claude, and lightweight open models like Gemma.

• What it means for startups: You get flexibility without lock-in. You can optimize your cost per task — using compact models for high-volume, deterministic jobs and frontier models for complex reasoning — and swap models as your product evolves without rebuilding your workflows.

3. Connected to your services and tools 

An agent is only as useful as the systems it can reach. In addition to Google Workspace, the Gemini agent connects directly to your engineering, data, and business stack, including partner solutions:

• Development and collaboration: Git, Slack, Jira and more 

• Data and local files: BigQuery, Postgres, Snowflake, Databricks, and more alongside files on your desktop

• Custom tools via MCP: Any Model Context Protocol (MCP) server inside or outside your network so your engineers can build and share internal tools.

What it means for startups: Instead of spending engineering cycles maintaining custom connectors, your team can plug the agent straight into your existing codebase, databases, and MCP tools so it can act on live data wherever it lives.

4. Built-in security and governance

For B2B startups, the enterprise security audit is often where deals stall. Under the hood, every agent is assigned a cryptographic Agent Identity for clear audit trails, runs inside an isolated sandbox, and protected with built-in prompt injection defenses. External traffic is routed through Agent Gateway, allowing you to enforce governance policies. And managing coworker agents is as straightforward as managing users today, since they all have a Workspace account.

• What it means for startups: When an enterprise buyer’s CISO asks how your AI workflows handle data privacy, prompt injection, and access control, you have an enterprise-grade architecture ready — helping you clear procurement hurdles.

5. Protect your runway with real-time cost controls

Autonomous agents that run sub-agents for hours or days can quickly run up a bill if a loop goes unchecked. You can now set hard, real-time spend caps directly in the Cloud Billing Console; if a cap is reached, the project's agent automatically pauses so a bug never turns into a billing surprise. Underneath, Gemini runs on Google’s specialized AI infrastructure, including our inference-optimized TPU 8i, which delivers 80% better performance-per-dollar than the prior generation.

• What it means for startups: You and your developers can experiment with long-running, parallel agentic loops with a built-in financial safety net that protects your cash reserves and builds confidence with your backers.

Founders already building

Startups and AI-native innovators are already putting Gemini to work. Immersive storytelling platform Whenere used Dreamlands, an agentic 3D worldbuilding platform built on Gemini platform, to bring Pride and Prejudice to Fortnite — with Gemini generating textures for each new object in about 7 seconds for under 10 cents; one artist created nearly 250 custom assets in a single day and completed a historically accurate estate in under a week. They join application-building platform Lovable and fellow startups and innovators including  OpenEvidence, Recursion, and Replit and more.

Build with Google for Startups

The agentic era gives lean teams the leverage to punch far above their weight — whether you’re deploying coworker agents to cover internal operations or building agentic workflows directly into your product.

While these new Gemini agent capabilities are not publicly available yet, startups can contact our sales team to learn more and discuss how to get ready. In the meantime, you can explore the Google for Startups program to access credits, hands-on technical guidance, mentorship, global founder community programs and more.

Agents represent a fundamental shift in cloud operations. They can act on behalf of users or autonomously, make non-deterministic decisions, and request data across projects at unprecedented speed and scale. Because agents are a distinct category of non-human identity (NHI), they require unique secure-by-default guardrails.

Traditional, manually-operated network perimeters are vulnerable to today’s machine-speed attacks. Overly permissive perimeters, built to avoid blocked connections, increase the risk of unauthorized data theft, but overly strict perimeters stall development under a flood of access denials.

Today, we're excited to announce key enhancements to VPC Service Controls (VPC-SC) that make perimeters easier to create, maintain, and scale. You can now add folders to perimeter membership, so that projects contained in them inherit policy. You can also manage granular exceptions and resolve denials with guided, scoped remediation. 

The result is data protected by default, so you can scale autonomous agents across your enterprise without letting data leave your perimeter. This update is generally available, and at no additional cost.

Perimeters aligned to your architecture

VPC-SC provides a context-aware network security perimeter around your Google Cloud deployment, designed to secure your organization against data theft, even in the event of compromised credentials, identity and access management (IAM) misconfigurations, and insider threats. By extending network-layer protection across service-to-service interactions in Google Cloud's infrastructure, VPC-SC goes beyond traditional firewalls and creates an additional layer of defense that prevents access outside your corporate network — and is now easier for your organization to manage.

Cloud administrators configuring VPC-SC can now include folders in perimeter membership, automatically extending the folder’s access policy to every project and nested subfolder it contains. New workloads can inherit protection by default, with no further modification.

Perimeter membership updates when a project is moved, simplifying lifecycle maintenance. Security coverage follows the resource hierarchy that your organization already governs, and compliance verification requires only confirming that the resource is located in the correct part of the hierarchy. 

Dry-run mode also supports folder membership, helping administrators evaluate the effect of a hierarchy-wide perimeter against live traffic before enforcement.

Folder and organization support in ingress and egress rules

Well-defined ingress and egress rules establish granular, context-aware perimeters without sacrificing operational flexibility across complex multi-project environments. Security administrators can now specify Google Cloud folders and organizations directly within those rules. A single hierarchy reference stands in for the set of projects beneath it, which keeps rules concise, keeps attribute counts well within quota limits, and extends access automatically to projects created later in the hierarchy.

Using Private Service Connect endpoints in ingress and egress rules

Enterprises increasingly use private paths to reach desired endpoints in their deployment instead of exposing the traffic to the internet. Private Service Connect endpoints for Google APIs are now supported as an attribute in ingress and egress rules. Administrators can author a rule that authorizes access specifically over a designated private endpoint, producing a policy that corresponds directly to the connectivity architecture it is intended to govern.

For organizations that have standardized on Private Service Connect, including endpoints propagated across the estate through Network Connectivity Center, this provides both a more precise and a more concise means of expressing source for the request.

Service patterns in VPC accessible services

Service patterns in VPC accessible services provide granular control over Google APIs accessed with private Virtual IP addresses (VIP), permitting interoperability beyond just VPC-SC integrated services. If the workload requires controlled access to validated API endpoints over a private path, administrators can now allow only the specific API URL patterns required. 

Additionally, the capability supports appending org-restriction-headers with egress requests to facilitate access to only authorized domains and resources. Because you don’t need to manage egress-proxy, you can accommodate legitimate operational requirements while maintaining a tightly-bounded private path, instead of choosing between a broad allowance and no access at all.

Guided violation remediation

VPC-SC perimeter policies evolve continuously alongside the workloads they protect, and administrators periodically need to authorize a legitimate access request not yet covered. New Guided Violation Remediation in VPC-SC Violation Analyzer cuts resolution time for blocked requests from days to minutes by generating a policy fix with remediation recommendations, which you can validate in Dry-Run mode first. 

Rather than just diagnosing why the violation occurred for a given denial, Violation Analyzer evaluates the blocked request against the applicable perimeter configuration, and proposes a specific configuration change, narrowly scoped by least-privilege design principles, to authorize the traffic: an ingress rule, an egress rule, an access level, or an update to VPC accessible services. This relieves pressure on administrators to broadly open up a perimeter just to resolve a violation and get an application working quickly. 

Read more about how BlackLine uses Violation Analyzer to streamline investigations.

Get started today

VPC-SC makes it simple and automated to deploy and refine granular network perimeter protection, even for autonomous agents. To get started:

1. Establish your folder baseline: Identify the folders in your resource hierarchy that correspond to the boundaries you intend to enforce, and define perimeters around them so that new workloads inherit policy on creation. 

2. Run risk-free in Dry-Run mode: Observe production traffic safely without blocking active developers.

3. Unblock traffic with Violation Analyzer and Violation Dashboard: Use automated least-privilege remediation recommendations to resolve any  denials that surface. 

4. Enforce with confidence: Secure your full data estate by transitioning to enforcement once your dry-run logs show your intended access with clean, authorized traffic.

To learn more about how VPC-SC can help simplify your data security posture and confidently secure your autonomous agent fleets, check out the VPC-SC documentation.

※この投稿は米国時間 2026 年 10 月 3 日に、Google Cloud blog に投稿されたものの抄訳です。

トラフィックの急増に対応してマイクロサービスを起動する場合でも、新しいソフトウェア リリースをデプロイする場合でも、アプリケーション レプリカをスケールアップする場合でも、Google Kubernetes Engine（GKE）で実行されるアプリケーションの高速で応答性の高いユーザー エクスペリエンスを維持するには、Pod の起動時間が重要です

※この投稿は米国時間 2026 年 10 月 3 日に、Google Cloud blog に投稿されたものの抄訳です。

トラフィックの急増に対応してマイクロサービスを起動する場合でも、新しいソフトウェア リリースをデプロイする場合でも、アプリケーション レプリカをスケールアップする場合でも、Google Kubernetes Engine（GKE）で実行されるアプリケーションの高速で応答性の高いユーザー エクスペリエンスを維持するには、Pod の起動時間が重要です

※この投稿は米国時間 2026 年 10 月 2 日に、Google Cloud blog に投稿されたものの抄訳です。

エンタープライズ AI エージェントが、複数日にわたる複雑なワークフローや長期にわたるタスクを実行するには、永続メモリが必要です。このブログ投稿では、短期バッファメモリに Memorystore for Valkey を、長期永続メモリに AlloyDB AI を使用する 2 層メモリ アーキテクチャによって、重要なデータとエンタープライズ ガードレールを維持しながら、トークン消費量を最大 70% 削減する仕組みを見ていきます。

たとえば、ユーザーの休暇の予約を支援する、パーソナライズされた旅行エージェントを構築するとします。ユーザーは数日間にわたってエージェントと何度もやり取りし、旅行プランのアイデア出しから具体的な購入・予約の検討に至るまで、さまざまな質問をします。真にシームレスなエクスペリエンスを提供するには、このエージェントは、以前のセッションで設定されたフライトの好み（例: 「直行便のみ」）、ホテルの予算、食事の要望（例: 「グルテンフリーのメニューが必要」）を記憶しておく必要があります。さらに重要なのは、会話が観光のおすすめや旅行プランの計画といった細かい内容に深入りしても、これらの重要情報を保持する必要がある点です。エージェントは、追加の質問や訪問先に関する魅力的な詳細情報によって、ユーザーの基本的な要件や決定事項を忘れたり上書きしたりすることがないようにしなければなりません。

ステートフルなワークフローは、LLM のステートレスな性質と本質的に矛盾する

AI エージェントは、マルチターンの会話や長期にわたるワークフローに活用されていますが、大規模言語モデル（LLM）はセッション間でステートレスのままとなっています。ユーザーが数日後にエージェントを再び利用する際、モデルはコンテキスト ウィンドウが空の状態で処理を開始します。アプリケーションがエージェントの長期メモリを使用して過去のコンテキストを再構築するように設計されていない限り、ユーザーは目標とコンテキストを最初から説明し直す必要があるため、ストレスのたまる断片的なユーザー体験になってしまいます。

100 万トークンのコンテキスト ウィンドウが標準となった現在、この問題に対する手っ取り早い解決策としてよく使われるのは「コンテキスト スタッフィング」です。これは、未加工のチャット履歴からツールの実行ログまで、プロンプトに収まるあらゆるものを会話のターンごとに詰め込むというものです。実際、これは AI モデルが使用され始めた初期によく見られたパターンです。しかし、この手っ取り早い解決策では、規模が大きくなるとすぐに問題が発生します。メッセージごとのトークン消費量が倍増し、単純なプロンプトでも応答時間が 30 秒を超えるようになり、モデルでは「lost in the middle（情報の埋もれ）」と呼ばれる精度低下が生じ、大量のプロンプト テキストに埋もれた重要な指示を見落とすようになります。

もう一つの一般的な回避策は、ローリング サマリーを使用することです。これは LLM に古いメッセージを定期的に要約して、短い文章に圧縮するよう指示する手法です。これによりプロンプトのサイズは削減されますが、LLM ベースの要約は本質的に情報の欠落を伴います。数回の圧縮を繰り返すうちに、些細ながらも重要な詳細がバックグラウンド ノイズとして除去されてしまうからです。数ターンも経つと、エージェントはユーザーが最初に設定した制約を、いつの間にか無視するようになります。以前の会話やマルチターン タスクの記憶を保持するには、ユーザー体験の低下や、新たなボトルネックの発生を回避したうえでの、よりスケーラブルなアプローチが必要であることは明らかです。

2 層メモリ アーキテクチャの実装

ガードレールと制約を遵守し、信頼性と費用対効果に優れたエンタープライズ エージェントを構築するには、2 層メモリ アーキテクチャの採用をおすすめします。

• 短期セッション バッファ: トークン数で制限されたスライディング ウィンドウを使用して、アクティブな会話ターンをメモリにキャッシュします。これにより、デバイスをまたいだ複数回のやり取りでもコンテキスト ウィンドウのデータ量を一定に保ち、直近のメッセージを常に最新の状態に保つことができます。この階層では、ターンごとに 1 ミリ秒未満の高速かつ高スループットなルックアップが必要となるため、Memorystore for Valkey がアクティブなセッション状態の維持に最適です。

• 長期永続メモリ: 重要な事実、ユーザーの好み、エピソード的事実をセッションをまたいで保存します。この階層では、トランザクションの整合性、データ ガバナンス、リレーショナル データとベクトルにわたるハイブリッド検索が必要となります。これらは、AlloyDB AI がネイティブに提供する機能です。

長期記憶とアクティブ セッション バッファの両方をリレーショナル データベースに直接保存することも可能ですが、インメモリ キャッシュを備えた 2 層アーキテクチャの方がパフォーマンスとスケーラビリティが向上します。アクティブ セッション バッファは非常に一時的なもので、会話のターンごとに 1 ミリ秒未満の高スループットな更新が必要です。これらの高頻度で発生する書き込みをインメモリ Key-Value キャッシュで処理することで、リレーショナル データベースでの書き込みの増大とテーブルの肥大化を防ぎます。これを行わないと、頻繁な行の削除や負荷の高いバキューム処理が必要になってしまいます。これは、データベースの前段にキャッシュ階層を追加して、アクセスが集中している行への高頻度なルックアップをオフロードするのと同様の仕組みです。この分業により、プライマリ データベースは負荷を抑えて応答性の高い状態を維持し、トランザクションの整合性、複雑なハイブリッド ベクトル検索、長期的な分析クエリの実行など、本来の目的にリソースを集中させることができます。

Memorystore for Valkey の短期キャッシュと AlloyDB AI のネイティブ機能を組み合わせることで、エンティティ抽出、メモリ圧縮、セッションをまたいだメモリ、ハイブリッド検索をデータベース層内で直接実行し、アクティブなプロンプトを軽量かつ高速に維持し、優れた費用対効果を実現できます。

4 種類のメモリについて

エージェントの長期的なステートを効果的に整理するために、エージェントのメモリを、前述の短期ストレージ レイヤと長期ストレージ レイヤにわたる、互いに補完し合う 4 つのタイプにさらに分割します。

短期的な会話コンテキストを構造化された長期的なルールから分離することで、エージェントは、トークン ウィンドウを未加工のインタラクション ログで埋め尽くすことなく、関連するコンテキストをオンデマンドで取得できます。

階層型メモリ アーキテクチャのブループリント

以下の図は、アプリケーション オーケストレーション レイヤ、Memorystore for Valkey の短期バッファ、AlloyDB AI の長期リポジトリを接続する読み取りパスと書き込みパスの概要を示しています。

このシステムは、以下の 2 つの連携した実行パスで動作します。

• 読み取りパス: ユーザーが質問すると、アプリケーションは Memorystore for Valkey からアクティブなスライディング ウィンドウを取得し、ai.generate を使用してデータベース内クエリの正規化を実行します。そして ai.hybrid_search を使用して AlloyDB にクエリを送信し、スコープ設定されたエンティティ ルールと関連するエピソード的事実を取得します。

• 書き込みパス: 回答が生成されると、会話のターンはすぐに Memorystore for Valkey にキャッシュされます。非同期のバックグラウンド キュー ワーカーが、会話のやり取りから構造化エンティティを抽出し、AlloyDB に直接書き込みます。AlloyDB では、トランザクション自動エンベディングによってベクトル表現が即座に計算され、データベースに保存されます。

ビジネスへの影響と費用対効果

開発におけるマルチターンの対話（45 ターン以上、大量のツール実行）を対象としたベンチマーク テストでは、短期キャッシュを長期永続化から分離することで、単純なコンテキスト スタッフィングと比較して、費用削減とパフォーマンス向上を実現しました。

注: 上記の指標は、実際のエンタープライズ AI ペア プログラミング アシスタントが、複数のセッション、プロジェクト、コンテキストの切り替えにわたって開発者とやり取りする様子を再現した、シミュレートされた開発ワークロードに基づく内部ベンチマーク結果を反映したものです。実際の費用削減とレイテンシは、プロンプトの構造、クエリの頻度、データ量によって異なります。

これらの結果は、階層型メモリがエージェントのユニット エコノミクスをどのように変化させるかを示しています。ターンが追加されるたびに費用曲線が上昇するのではなく、プロンプト サイズが一定の範囲内に収まるため、高速の応答時間を維持しながら、継続的な LLM API 費用を削減できます。

AlloyDB AI の核となる技術的メリット

AlloyDB AI は、コア AI 機能をデータベース エンジンに直接組み込むことで、永続的なエージェント メモリの実装に伴う運用上のオーバーヘッドを削減します。その主な特長は以下のとおりです。

• データベース内トランザクション自動エンベディング（ai.initialize_embeddings）: AlloyDB は、Agent Platform（旧称 Vertex AI）とのネイティブ統合を使用して、テキスト列のベクトル エンベディングを自動的に生成し、1 秒あたり最大 3,000 個のエンベディングを処理できます。incremental_refresh_mode => 'transactional' を使用すると、AlloyDB は同じトランザクション内でソースデータが変更されるとエンベディングを最新の状態に保つため、カスタム エンベディング パイプライン、外部スケジューラ、複雑な再試行ロジックが不要になります。

• データベース内生成 AI 関数（ai.generate）: AlloyDB では、Gemini などの基盤モデルを SQL クエリから直接実行できます。これを使用すると、データベース内でクエリを分解できます。つまり、ユーザーの複合的な質問を単一の要素からなるサブクエリに分割し、相対的な時間表現（「前回のセッション」など）を明示的な識別子に解決できます。アプリケーション側で個別にラウンドトリップを行う必要もありません。

• 組み込みの Reciprocal Rank Fusion（ai.hybrid_search）によるネイティブ ハイブリッド検索: AlloyDB は、エンジン内で直接 Reciprocal Rank Fusion（RRF）を実行する組み込み SQL 関数を提供します。ベクトル コサイン類似度（HNSW または ScaNN インデックスに対する <=>）と PostgreSQL 全文検索（BM25、RUM、または GIN を使用）を 1 回のデータベース呼び出しで組み合わせ、セマンティック マッチングと正確なキーワード検索を融合させます。同時に、メタデータ フィルタ プッシュダウン（filter_condition）をサポートすることで、パフォーマンスの向上と確定的なスコープ分離を実現します。

• IAM 認証情報を使用した Agent Platform との直接統合: AlloyDB は、Google Cloud IAM サービス アカウントのロールとデータベース認証を使用して、Google Cloud のプライベート ネットワーク経由で Agent Platform 基盤モデルに直接接続します。これにより、アプリケーション コードで API キーを保存、ローテーション、受け渡す必要がなくなります。

• 運用、ベクトル、ガバナンスの統合エンジン: AlloyDB は、リレーショナル ビジネスデータ、ベクトル エンベディング、全文インデックス、エンタープライズ権限を、ACID 準拠の単一の PostgreSQL データベースに統合します。これにより、個別の運用データベースとベクトル データベース間で発生するデータのドリフトや統合の複雑さを回避できます。

主な実装パターン

以下は、2 層メモリ アーキテクチャの構成に使用されるコア データベース パターンです。実行可能な完全な Python スクリプトと SQL スクリプトについては、対応する AlloyDB エージェント メモリ Codelab をご覧ください。

1. スキーマと自動エンベディングの設定

AlloyDB で、必要な拡張機能をインストールし、構造化メタデータ、全文検索用に生成された tsvector 列、ベクトル エンベディング列を含む agent_entities テーブルを定義します。

次に、ai.initialize_embeddings を使用してエンベディングを生成します。トランザクション モードを使用すると、ソースデータの変更に応じてエンベディングが最新の状態に保たれます。

最後に、HNSW ベクトル インデックスと RUM 全文検索インデックスを作成して、ハイブリッド検索を高速かつ効率的に行えるようにします。

2. ネイティブ ハイブリッド検索による長期記憶のクエリ

読み取りパスで、ai.hybrid_search を使用して関連する長期エンティティを取得します。このネイティブ SQL 関数は、AlloyDB で直接 Reciprocal Rank Fusion（RRF）を実行し、ベクトル類似性検索と全文キーワード検索の結果をシームレスに再ランキングして、単一のデータベース クエリに結合します。

3. AI 関数を使用したデータベース内メモリ圧縮

カスタムのプルーニング スクリプトを記述せずに長期的なストレージの増加を管理するには、AlloyDB で自動抽出および圧縮クエリを直接実行して、保存しておくべき重要な部分を特定します。このパターンでは、SQL 共通テーブル式（CTE）と ai.generate を使用して、古いエピソード的事実の履歴を高密度の要約に統合します。

たとえば、子どもを連れて飛行機を乗り継ぐことの複雑さに関する長いやり取りを、「直行便を希望」という要約にまとめることができます。

4. 2 層メモリ アーキテクチャの ADK エージェントへの接続

コアの 2 層メモリ アーキテクチャを構成したら、対応する Codelab で示されるように、デフォルトの ADK Memory プロバイダを拡張して、この 2 層メモリ アーキテクチャを使用できます（例: ADKTieredMemoryProvider）。長期記憶を ADK エージェントにアタッチするには、次のようにツール（例: longterm_memory_tool）として提供するだけです。

エンタープライズ ガバナンスとマルチテナント セキュリティ

エンタープライズの本番環境でエージェント メモリを実行するには、厳格なセキュリティ境界とアクセス制御が必要です。まず、スコープとマルチテナントの分離を維持する必要があります。agent_entities の user_id、project_id、scope 列にインデックスを付け、PostgreSQL の行レベル セキュリティ（RLS）を適用することで、同じデータベース クラスタ内の部門、チーム、個々のユーザー間でメモリストアを分離できます。パラメータ化されたセキュアビュー（PSV）は、もう一つの確定的なアプリケーション レベルのセキュリティ レイヤを提供し、悪意のあるプロンプトや過度に広範な SQL クエリから保護するのに役立ちます。

さらに、上記の実装パターンに示すように、メモリ ライフサイクルの自動管理を提供する必要があります。スケジュールされた SQL コンパクション クエリと時間ベースのパーティション プルーニングを組み合わせることで、データベースのフットプリントとクエリのレイテンシを長期にわたって予測可能な状態に維持できます。このアプローチの詳細については、対応する Codelab をご覧ください。

まとめと次のステップ

アクティブなコンテキスト ウィンドウを永続ストレージから分離することは、プロダクション レディな AI エージェントを構築するための実用的なアプローチです。1 ミリ秒未満のセッション キャッシュを実現する Memorystore for Valkey と、トランザクション対応の長期ストレージである AlloyDB AI を組み合わせることで、長期にわたるタスクや複数ターンのエージェント エクスペリエンス全体で厳格なビジネスルールを維持しながら、トークン費用を大幅に削減し、応答時間を短縮できます。

利用方法は次のとおりです。

• 対応する AlloyDB エージェント メモリ Codelab の実践的なチュートリアルを最後まで進め、実際に動作する 2 層メモリ アーキテクチャをデプロイします。

• データベース側の ML 機能の詳細については、AlloyDB AI のドキュメントをご覧ください。

• 自動ベクトル エンベディングの生成とハイブリッド ベクトル検索の実行に関するガイドをご覧ください。

- Google、AlloyDB AI シニア エンジニアリング マネージャー、Itai Rosenblatt

- Google、AlloyDB AI / Cloud SQL 担当プロダクト マネージャー、Paul Ramsey

Editor's note: Generative AI startups are moving rapidly beyond static clip generation to build interactive, real-time worlds. In this technical deep dive, we explore how AI startup Visko collaborated with Google Cloud TPU engineers to deploy Orbis — its continuously streaming Live World Model—on Cloud TPUs, going from initial kickoff to production handover in just 30 days.

What makes a live video world model so challenging to serve in production? The bottleneck isn't average throughput — it's beating a relentless real-time clock. Unlike offline video generators where a user submits a prompt and waits in a queue, a live world model streams continuously. Every new chunk of video and synchronized audio must finish rendering before the viewer finishes watching the previous one, or the world freezes.

Today, we're excited to share how Visko, an AI startup pioneering interactive world models, teamed up with Google Cloud to solve this exact challenge. Working side by side, Visko and Google Cloud engineers deployed Visko's end-to-end live inference pipeline — combining video generation, super-resolution upscaling, and synchronized audio—onto a single Cloud TPU slice. Together, the teams brought end-to-end generation down, rendering at 1.07x playback speed.

Here is an inside look at how we engineered the pipeline to hit that milestone in a single month, the architectural optimizations that moved the needle, and practical lessons for other startups building ultra-low-latency generative AI on Google Cloud.

Redefining interactive media with Live World Models

While most generative video platforms focus on rendering fixed, pre-baked clips, Visko is building what it calls Live Models: video world models that generate continuously and respond on the fly.

Visko's flagship model, Orbis, streams live video starting from a single text or image prompt. As the stream plays, viewers can enter new steering prompts in real time, dynamically evolving the scene without restarting or breaking continuity. For instance, in one test session, the livestream opens on a cat napping beside a crackling fireplace; when the viewer steers the scene to introduce a bowl of food, the bowl materializes seamlessly, and the cat stretches, stands up, and meows.

Since launching its public demo on September 1, 2026, Orbis has unlocked new possibilities across interactive entertainment, live commerce, AI companions, immersive education, and robotics simulation. To scale these interactive sessions reliably, Visko partnered with Google Cloud TPU engineers in August 2026 to port the Orbis live pipeline to Google's Tensor Processing Unit (TPU). Within 30 days, the full multimodal pipeline was running in real time on TPUs.

Visko’s ambition is to power a new generation of interactive worlds that people can step into and shape in real time. Streaming continuous video and audio under a strict clock is a fundamentally different system challenge than offline clip generation. Working side by side with Google Cloud's TPU engineers, we brought the full Orbis pipeline to TPUs in just 30 days — giving us the low-latency compute and memory bandwidth required to bring Live Models to creators and developers at scale.

"Visko is bringing an exciting new class of real-time, steerable workloads to the Cloud TPU ecosystem. By co-engineering their live multimodal pipeline on a singleTPU slice using JAX, torchax, and custom Pallas kernels, we proved how quickly a lean startup team can go from PyTorch prototype to render at 1.07x playback speed on Google Cloud's AI Hypercomputer."
– Dave Elliott, Head of Developer Advocacy for AI, Google Cloud 

Live video is a hard-deadline problem

When you serve an offline clip generator, latency is forgiving — the user waits once while the backend processes frames. A live world model has no waiting room.

Every single chunk must be denoised, decoded from latents, upscaled to 1664x936 (at 18 fps), and paired with generated audio faster than realtime playback speed, for as long as the session runs.

Two structural constraints make live world models far more demanding than standard batch inference:

• Batch size of one: Each live stream represents an individual viewer's interactive session. You cannot mask latency by grouping dozens of concurrent requests into a massive batch — the standard playbook for keeping AI accelerators saturated.

• Strict sequential dependency: Every chunk depends directly on the one before it. Because the video model maintains a rolling memory of recent video latents so that chunk N+1 smoothly continues the action from chunk N, you cannot pre-render chunks out of order or parallelize across future timestamps.

For startup infrastructure engineers coming from LLM serving, it helps to think of live world modeling not as a tokens-per-second throughput benchmark, but as an unforgiving per-step latency SLO.

Architecting the multimodal pipeline on a single TPU slice

To keep infrastructure lean and cost-effective as Visko scales, the team designed the live pipeline to chain three distinct generative models on a single Cloud TPU slice running in BF16 precision.

When a viewer submits an initial prompt or a live steering update, all three generative models — the core video diffusion model, the super-resolution upscaler, and the audio generator — share the same chips, taking turns on the hardware for each chunk.

To combine rapid startup iteration with bare-metal accelerator speed, the software stack is built on three open-source pillars:

• JAX: Google's high-performance Python array library for accelerators, compiled into optimized machine code by the XLA compiler.

• torchax: An interoperability layer that runs PyTorch models directly on Cloud TPUs via JAX, enabling startup teams to bring existing PyTorch components over without an upfront, ground-up rewrite.

• Pallas: The custom kernel programming language for TPUs in JAX — the TPU counterpart to writing custom Triton or CUDA kernels on GPUs.

Real-time performance breakdown: Hitting the clock 

By profiling and optimizing each stage on a single slice in BF16, the engineering team achieved a total per-chunk latency of under the target while — maintaining roughly 7% headroom to absorb variance during live streaming:

Because video denoising accounts for nearly two-thirds of the total compute budget, that stage became the primary focus of our kernel and parallelism optimizations.

Inside the stack: Three optimizations that unlocked real-time streaming

1. Ulysses context parallelism across all TPU chips

A video chunk expands into a massive sequence of spatiotemporal tokens inside the video transformer. To distribute that workload across the TPU chips at batch size 1, the team implemented Ulysses context parallelism.

Instead of splitting model weights, Ulysses splits the token sequence across chips. Immediately before the attention layer, a high-speed all-to-all interconnect exchange regroups the tensors so that each chip holds the entire video sequence, but only for one-quarter of the attention heads. After each chip computes attention over the full sequence for its assigned heads, a second all-to-all exchange restores the sequence-split layout for the feed-forward layers.

2. Custom Pallas splash attention from MaxDiffusion

To accelerate the attention computation itself, the team integrated a custom Pallas splash attention kernel from MaxDiffusion, Google Cloud's open-source reference library for diffusion models on Cloud TPU.

Splash attention brings FlashAttention-style tiling to TPU architecture: rather than materializing the full quadratic attention matrix in High Bandwidth Memory (HBM), it computes attention in compact tiles that remain inside ultra-fast on-chip VMEM. Startup teams can drop the exact kernel used in this port straight into their own pipelines via custom_splash_attention.py.

3. A 5.8x audio speedup by moving from torchax to pure JAX

We first brought all three Orbis models onto Cloud TPU with *torchax*, which runs PyTorch code on TPU through JAX and XLA without a rewrite. For the video and upscaling models, `torchax` paired with Pallas attention kernels met our latency target directly, and both stay in `torchax` in the final pipeline.

Only the audio model needed a rewrite. Moving the audio stage from `torchax` to *native JAX* made it *5.8x faster* and brought the full pipeline under the playback deadline.

Startup Builder Tip: Port strategically.
The takeaway isn't "skip torchax" — it's to use it as a fast bring-up bridge. torchax got our PyTorch models running on TPU in days rather than weeks, giving us the end-to-end telemetry needed to pinpoint true bottlenecks. Measure the full pipeline first, and invest engineering cycles rewriting only the components that dominate your latency budget.

Field notes for AI builders: Cloud TPU lessons

Takeaways from tuning live multimodal inference on Cloud TPU:

- *Ulysses context parallelism for single stream video:* At batch size 1, splitting the token sequence across chips with `all-to-all` interconnect exchanges was faster than both tensor parallelism and ring attention.

- *Native BF16 for continuous streams:* FP8 quantization cuts memory bandwidth pressure, but small quantization errors build up across chunks in a continuous stream. Native BF16 kept visual quality intact within the latency budget.

- *Selective JAX ports:* `torchax` kept the video and upscaling models in PyTorch, so the team only rewrote the audio bottleneck in native JAX.

- *XLA compilation caching:* The first boot spends a few minutes compiling XLA graphs. Caching compiled artifacts brings worker restarts under a minute.

Building real-time generative models on Cloud TPU? Start here

If your startup is pushing the boundaries of live video, world models, or ultra-low-latency multimodal inference, you can build with the same toolkit that powered Visko's port:

• Accelerate diffusion attention out of the box: Grab the open-source splash attention kernel used in this collaboration from MaxDiffusion, and explore the Pallas documentation to write custom TPU kernels.

• Migrate PyTorch fast, then optimize selectively: Bring your existing PyTorch models onto TPU quickly with torchax, pinpoint bottlenecks with the Cloud TPU Profiler, and port latency-critical stages to native JAX.

• Spin up TPUs on demand: Use TPU Flex-start VMs to provision TPU capacity for short development and benchmarking runs without a long-term reservation.

• Scale with Google for Startups: Early-stage AI startups looking for cloud credits, technical mentorship, and access to Google Cloud's AI Hypercomputer stack can apply to the Google for Startups Cloud Program.

^{Special thanks to the Visko.ai team for the close technical partnership, and to the Google Cloud TPU engineers who lead the project including: Yixin Shi, James Huang, and Elisa Tsai.}

Small teams, short runway, and constant pressure to ship faster — it’s the reality we see every day among the founders we work with at Google for Startups. When you are building a company from the ground up, every hour spent on operational overhead or infrastructure plumbing is an hour taken away from your core product.

Last week at Gemini at Work 2026, we introduced the Gemini agent: a single, universal agent designed to work both as your personal assistant and as a dedicated member of your team. And this week, we made our new frontier model, Gemini 4 Argon, widely available — trained to maintain context across long, multi-step tasks.

Here is a look at how the Gemini agent works, five practical takeaways for founders, and how startups are already using it to punch above their weight.

What is the Gemini agent?

Instead of prompting separate chatbots for writing, coding, and data analysis, the Gemini agent is a single agent that takes high-level objectives and returns finished work. Three principles define it:

• Unified: A single agent can now handle knowledge work, data analysis, creative asset generation, and writing and running code. For complex jobs, it dynamically spins up temporary, job-specific sub-agents and coordinates their work in parallel or in sequence.

• Omnipresent access: It works across the surfaces you use every day — on the web, command line, desktop, and mobile, as well as natively inside Gmail, Docs, Sheets, Slides, Drive, Chat, and Calendar.

• Persistent: It runs in the cloud with continuous memory across four dimensions (session, semantic, procedural, and episodic). It onboards itself like a new hire — learning your team, tools, and business context so you don't have to re-explain your startup with every prompt, and keeping multi-day tasks moving even after you close your laptop.

For founders wearing five hats at once, Gemini works like a digital chief of staff that's already briefed on your calendar, projects and documents. Ask it to research market trends, build a financial model in Sheets and turn both into a pitch deck, without repeating what the project or goals are at each step. It also works ahead of you. When an investor emails asking for your latest product update as a deck, Gemini spots the request and offers to handle it; one click and it comes back with completed work, ready for sharing or review.

Five takeaways for startups

1. Coworker agents and skills help you scale your output

Beyond personal assistance, you can create persistent coworker agents that work with your entire team. Each agent comes with its own Workspace account, email address, Drive storage, and directory presence. You can define a specific role — such as a launch coordinator, QA triager, or financial analyst — and your teammates can add agents to a Chat space or @mention it in a Google Doc comment to delegate work.

You can also equip agents with reusable skills — or let Gemini write skills for itself as it learns your processes. For example, with operational reporting skills connected to BigQuery and the Knowledge Catalog, anyone on your team can ask a plain-language question to build and save a live report. Once saved, those reports run on demand without incurring additional token costs. Gemini also offers pre-built specializations, starting with legal and financial services.

• What it means for startups: Coworker agents can immediately increase the capacity of your teams, and skills can standardize repeatable processes like investor and operational reporting without burning tokens on every run.

2. Open and third-party model choice

Gemini is the agent, and the model underneath is your choice. As startup architectures mature, sending every task to a single frontier model quickly eats into gross margins. With the Gemini agent, you can dynamically route workloads across Gemini models, third-party models like Anthropic’s Claude, and lightweight open models like Gemma.

• What it means for startups: You get flexibility without lock-in. You can optimize your cost per task — using compact models for high-volume, deterministic jobs and frontier models for complex reasoning — and swap models as your product evolves without rebuilding your workflows.

3. Connected to your services and tools 

An agent is only as useful as the systems it can reach. In addition to Google Workspace, the Gemini agent connects directly to your engineering, data, and business stack, including partner solutions:

• Development and collaboration: Git, Slack, Jira and more 

• Data and local files: BigQuery, Postgres, Snowflake, Databricks, and more alongside files on your desktop

• Custom tools via MCP: Any Model Context Protocol (MCP) server inside or outside your network so your engineers can build and share internal tools.

What it means for startups: Instead of spending engineering cycles maintaining custom connectors, your team can plug the agent straight into your existing codebase, databases, and MCP tools so it can act on live data wherever it lives.

4. Built-in security and governance

For B2B startups, the enterprise security audit is often where deals stall. Under the hood, every agent is assigned a cryptographic Agent Identity for clear audit trails, runs inside an isolated sandbox, and protected with built-in prompt injection defenses. External traffic is routed through Agent Gateway, allowing you to enforce governance policies. And managing coworker agents is as straightforward as managing users today, since they all have a Workspace account.

• What it means for startups: When an enterprise buyer’s CISO asks how your AI workflows handle data privacy, prompt injection, and access control, you have an enterprise-grade architecture ready — helping you clear procurement hurdles.

5. Protect your runway with real-time cost controls

Autonomous agents that run sub-agents for hours or days can quickly run up a bill if a loop goes unchecked. You can now set hard, real-time spend caps directly in the Cloud Billing Console; if a cap is reached, the project's agent automatically pauses so a bug never turns into a billing surprise. Underneath, Gemini runs on Google’s specialized AI infrastructure, including our inference-optimized TPU 8i, which delivers 80% better performance-per-dollar than the prior generation.

• What it means for startups: You and your developers can experiment with long-running, parallel agentic loops with a built-in financial safety net that protects your cash reserves and builds confidence with your backers.

Founders already building

Startups and AI-native innovators are already putting Gemini to work. Immersive storytelling platform Whenere used Dreamlands, an agentic 3D worldbuilding platform built on Gemini platform, to bring Pride and Prejudice to Fortnite — with Gemini generating textures for each new object in about 7 seconds for under 10 cents; one artist created nearly 250 custom assets in a single day and completed a historically accurate estate in under a week. They join application-building platform Lovable and fellow startups and innovators including  OpenEvidence, Recursion, and Replit and more.

Build with Google for Startups

The agentic era gives lean teams the leverage to punch far above their weight — whether you’re deploying coworker agents to cover internal operations or building agentic workflows directly into your product.

While these new Gemini agent capabilities are not publicly available yet, startups can contact our sales team to learn more and discuss how to get ready. In the meantime, you can explore the Google for Startups program to access credits, hands-on technical guidance, mentorship, global founder community programs and more.