Databases

Multiple result sets: How Database Migration Service automates SQL server to PostgreSQL translation

Tue, 04 Aug 2026 16:00:00 +0000

In the Medium blog post, "From MARS to SETOF REFCURSOR: Migrating Multi-Result Stored Procedures to PostgreSQL," we explored the fundamental architectural differences between SQL Server and PostgreSQL regarding multiple result sets. We looked at how SQL Server natively streams multiple tabular streams from a single execution, whereas PostgreSQL requires a more deliberate strategy using explicit cursor manipulation.

If you’re facing a massive database migration with hundreds of these procedures, manually rewriting them is a non-starter. This is where automated tools come in. In this post, we’ll explore in detail how Google Cloud’s Database Migration Service (DMS) approaches this exact challenge, the conversion logic it applies under the hood, and how to actually run and test the generated code.

The Core Conversion Strategy of DMS

There are many reasons to migrate your databases to PostgreSQL, including enterprise performance and availability, a thriving developer and user community, and strong AI capabilities. But tricky queries, like those with multiple result sets, can slow down your migration project.

DMS looks at two specific things: How many result sets does the procedure return? and Does it use a scalar RETURN value?

The decision matrix for the translation looks like this:

Scenario	SQL Server Characteristic	PostgreSQL Target Object	Mechanism
Scenario A	1 Result Set OR a Scalar Return Value only	STORED PROCEDURE	Handled natively via an INOUT refcursor parameter or standard variable tracking.
Scenario B	Multiple Result Sets OR a combination of Result Sets + Scalar Return	FUNCTION	Converted to a RETURNS SETOF refcursor block. The scalar return value is appended as its own separate cursor dataset.

Automating Multi-Result Set Conversions: Inside DMS

SQL Server utilizes a tabular data stream protocol that allows multiple results to be transmitted over a single connection execution path without explicit declarations. PostgreSQL, by contrast, relies on a distinct execution protocol where multiple datasets are managed deterministically via cursors. To bridge this structural difference, DMS automates the translation logic.

Consider a baseline healthcare reporting scenario. We have a master procedure (sp_GetPatientSummary) that orchestrates data retrieval for a patient by conditionally calling two child procedures: one for lab results (sp_GetPatientLabResults) and one for clinical visits (sp_GetPatientDoctorVisits).

Depending on conditional logic and procedural execution paths, a single execution can return up to four distinct result sets plus a status integer indicating whether the patient was found.

SQL

code_block: <ListValue: [StructValue([('code', 'CREATE OR ALTER PROCEDURE sp_GetPatientSummary\r\n @PatientID INT\r\nAS\r\nBEGIN\r\n SET NOCOUNT ON;\r\n DECLARE @LabReturnValue INT;\r\n\r\n IF NOT EXISTS (SELECT 1 FROM Patient WHERE PatientID = @PatientID) \r\n RETURN 0; \r\n\r\n SELECT PatientID, FirstName, LastName, DateOfBirth FROM Patient WHERE PatientID = @PatientID;\r\n\r\n EXEC @LabReturnValue = sp_GetPatientLabResults @PatientID = @PatientID, @QueryType = 1;\r\n\r\n-- Conditional: If child returned 1, call it again for Result Set #3\r\n IF @LabReturnValue = 1\r\n EXEC sp_GetPatientLabResults @PatientID = @PatientID, @QueryType = 2;\r\n\r\n EXEC sp_GetPatientDoctorVisits @PatientID = @PatientID;\r\n \r\n RETURN 1; -- Success scalar status\r\nEND;'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef01930d0>)])]>

The Target: PostgreSQL’s Deterministic Cursors

To align with PostgreSQL's execution model, DMS maps the original T-SQL behavior into a structural PL/pgSQL architecture using SETOF refcursor and explicit cursor management.

1. Simple Tracking: Stored Procedure Translation

For the child procedure sp_getpatientdoctorvisits, which yields exactly one result set, DMS creates a standard PostgreSQL PROCEDURE utilizing an explicit INOUT refcursor parameter to safely pass the pointer back to the caller.

SQL

code_block: <ListValue: [StructValue([('code', "CREATE PROCEDURE dbo.sp_getpatientdoctorvisits(\r\n _patientid INTEGER, \r\n INOUT result_set_refcursor refcursor\r\n)\r\nLANGUAGE plpgsql\r\nAS $$\r\nBEGIN\r\n sp_getpatientdoctorvisits.result_set_refcursor := NULL;\r\n OPEN result_set_refcursor FOR \r\n SELECT visitid, patientid, doctorname, visitdate, visitnotes\r\n FROM dbo.doctorvisits\r\n WHERE doctorvisits.patientid = sp_getpatientdoctorvisits._patientid\r\n AND doctorvisits.visitdate >= localtimestamp + interval '-6 MON';\r\nEND;\r\n$$;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef0193880>)])]>

2. Multi-Set Tracking: Function Translation

For the master routine and the complex lab child routine, an INOUT parameter isn't enough to capture the varying arrays of output. DMS transforms these into PL/pgSQL Functions returning a SETOF refcursor.

Take note of how the translated sp_getpatientlabresults builds its cursors sequentially and, at the very end, dynamically opens a distinct cursor explicitly named "return_value" to pass the scalar integer back to the execution stack:

SQL

code_block: <ListValue: [StructValue([('code', "CREATE OR REPLACE FUNCTION dbo.sp_getpatientlabresults(_patientid INTEGER, _querytype INTEGER)\r\nRETURNS SETOF refcursor\r\nLANGUAGE plpgsql\r\nAS $$\r\nDECLARE rc refcursor; _recordsfound INTEGER := 0;\r\nBEGIN\r\n IF _querytype IN (1, 3) THEN\r\n OPEN rc FOR \r\n SELECT resultid, patientid, testname, testdate, isexception\r\n FROM dbo.labresults\r\n WHERE patientid = _patientid \r\n AND testdate >= localtimestamp - interval '6 months';\r\n RETURN NEXT rc;\r\n\r\n IF EXISTS (\r\n SELECT 1 FROM dbo.labresults \r\n WHERE patientid = _patientid \r\n AND testdate >= localtimestamp - interval '6 months'\r\n ) THEN recordsfound := 1; \r\n END IF;\r\n END IF;\r\n\r\n IF _querytype IN (2, 3) THEN \r\n rc := NULL; -- Reset to generate a new unique cursor name\r\n OPEN rc FOR \r\n SELECT resultid, patientid, testname, testdate, isexception\r\n FROM dbo.labresults\r\n WHERE patientid = _patientid \r\n AND isexception = TRUE \r\n AND testdate >= localtimestamp - interval '2 years';\r\n RETURN NEXT rc;\r\n\r\n IF EXISTS (\r\n SELECT 1 FROM dbo.labresults \r\n WHERE patientid = _patientid \r\n AND isexception = TRUE \r\n AND testdate >= localtimestamp - interval '2 years'\r\n ) THEN \r\n _recordsfound := 1; \r\n END IF;\r\n END IF;\r\n\r\n rc := 'return_value';\r\n OPEN rc FOR SELECT _recordsfound AS return_value;\r\n RETURN NEXT rc;\r\nEND;\r\n$$;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef01937c0>)])]>

Parsing the Output Programmatically

When integrating these migrated routines back into your application data access layer, QA and application engineers need to adapt how they process execution results. Instead of reading standard tabular rows sequentially, the calling application or test harness receives an array of cursor references. To handle this programmatically, your application must fetch the data from each returned portal sequentially. DMS isolates the scalar return value by placing it inside its own dedicated, explicitly named "return_value" cursor dataset at the very end of the execution stack. Preparing your development teams for this structural mapping ensures that your application logic and validation scripts can accurately parse the multi-layered response arrays without disruption.

Executing and Testing the Migrated Code

Testing these migrated objects in PostgreSQL requires working within explicit transaction blocks. Because PostgreSQL cursors are bound to the transaction lifecycle, accessing the data from the memory portals requires encapsulating the execution and the data retrieval commands within a single BEGIN ... COMMIT block.

Here is how you execute and fetch the entire complex dataset for Patient 3 inside PostgreSQL:

SQL

code_block: <ListValue: [StructValue([('code', 'BEGIN;\r\n\r\n-- 1. Exec the function to initialize and stream back the cursors\r\nSELECT * FROM dbo.sp_getpatientsummary(3) AS summary_cursors; \r\n\r\n-- 2. Fetch data from the sequentially generated anonymous portals\r\nFETCH ALL FROM "<unnamed portal 1>"; -- Patient Demographics\r\nFETCH ALL FROM "<unnamed portal 2>"; -- Lab Results (Query Type 1)\r\nFETCH ALL FROM "<unnamed portal 3>"; -- Lab Results (Query Type 2)\r\nFETCH ALL FROM "<unnamed portal 4>"; -- Doctor Visits\r\n\r\n-- 3. Fetch the named scalar return value tracking cursor\r\nFETCH ALL FROM "return_value";\r\n\r\nCOMMIT;'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef0193910>)])]>

DMS Insight: The Mechanics of Result Set Counting

The DMS product employs a sophisticated pre-processing mechanism to understand the expected result set count and the possible existence of a return value. The ultimate goal is to map every SQL Server procedure into one of three distinct categories: no result sets, a single result set, or multiple/dynamic result set counts.

To achieve this accurately, DMS performs a deep structural analysis:

Direct Result Sets: First, the engine scans the procedure's body to count the direct result sets, which are the explicit SELECT statements executed directly within it.
Dynamic Considerations: Special consideration must be given when dealing with looped or conditional SELECT/EXEC statements. Because these constructs inherently mean the number of returned result sets can differ between executions, the count for that procedure is immediately marked as "dynamic".
Building the Call Network: When the engine encounters references to other stored procedures, it does not yet know how many result sets those child procedures expose. To solve this, DMS builds a comprehensive directed graph to model the entire call hierarchy between procedures.
DFS Propagation: Only once this directed network is fully built can the engine run a Depth First Search (DFS) algorithm. This DFS traversal systematically propagates the result set counts—whether fixed integers or dynamic flags—back up the call chain to the top-level procedures.

Consequently, the generated code is accurately modified to support both returning the outer datasets and allowing child routines to be invoked correctly within nested execution stacks. This graph-based approach guarantees accuracy and seamlessly supports highly complex inter-procedural interactions, including both direct and indirect recursions.

Summary

Google Cloud’s Database Migration Service takes the guesswork out of structural transformations by programmatically applying the FUNCTION vs. PROCEDURE decision tree based on result set counts. While it completely preserves your core business and conditional execution logic, it does change how application connection pools and QA engineers interact with execution results.

Understanding this automated architecture ensures you can effectively map out your validation scripts and configure your data access layers for seamless day-two operations in PostgreSQL.

Now that you learned how to convert multiple result sets, it's time to start a PostgreSQL database on Google Cloud. Please let us know how your journey was.

Get started with a Database Migration Service with the Google Cloud $300 free credits. Start building for free.

Introducing Database Operations Agents: The future of autonomous database management

Tue, 04 Aug 2026 16:00:00 +0000

As part of the Agentic Data Cloud launch at Google Cloud Next ‘26, we announced two AI-powered database agents to simplify database management. These include the Database Onboarding Agent for Day 0 operations — setup, configuration, and initial deployment — as well as the Database Observability Agent for Day 1 and 2 operations, including monitoring, troubleshooting, and ongoing maintenance.

These agents are always on, informed by Google’s years of experience, and integrated across Google surfaces such as Chat, CLI, the Google Cloud console, Managed Context Protocol (MCP) servers, and third-party tools — including your preferred integrated development environment (IDE), so you get help where and when you need it.

Traditionally, managing and creating databases has involved a combination of manual architecture planning, custom scripts, and distinct tools. Teams handle database provisioning, schema design, index configuration, and query tuning, alongside performance monitoring—often cycling through repeated testing and optimization cycles as application demands change. Although this method is functional, it demands substantial technical skill and continuous attention throughout the entire database lifecycle. For example, developers often fear making an update that may limit their ability to scale the system later. Similarly, when an application slows down, finding the exact query or resource constraint causing the issue can take hours of manual investigation and troubleshooting.

Intelligent AI-powered agents can simplify database lifecycle management by automating many of these tasks such as recommending the right database type for the workload, detecting anomalies, recommending the right configurations, optimizing queries, and providing actionable insights to improve operational efficiency. By embedding these capabilities directly into workflows where you need them, agents help organizations build, operate, and optimize databases more efficiently while reducing operational overhead.

Let’s take a closer look at these new database agents.

Database Observability Agent: From diagnosis to remediation

The Observability Agent empowers Site Reliability Engineers (SREs), DevOps pros, DBAs and developers to diagnose complex issues and remediate them using simple natural language prompts.

As your operations scale, identifying subtle issues like query hotspots or lock contention becomes an expensive burden. The database observability agent uses Google’s operational expertise and the reasoning capabilities of Gemini to solve these challenges. By automatically connecting telemetry across multiple sources including Database Insights, Cloud Monitoring, Cloud Logging, and Cloud Trace the agent provides a clear root cause analysis in minutes.

Beyond just identifying the "why," the agent suggests recommended actions to fix the issues found, and can execute validated actions with your approval. For example, if it detects a bottleneck, it might suggest you "Enable connection pooling for Cloud SQL instance," providing the rationale and expected impact before you commit to the change. Some capabilities include:

Fleet-level troubleshooting: The Observability Agent is integrated with Database Center so you can use Gemini Chat to ask complex fleet-wide questions like, "Which databases in my fleet consumed the most CPU in the last 7 days?" to receive a summarized analysis across your entire fleet.
In-product investigations: The agent correlates complex telemetry across Database telemetry, Cloud Monitoring, Cloud Logging, Cloud Trace, and multiple other data sources to pinpoint issues like latency spikes or lock contention. (In preview with select customers)
Validated remediations: Instead of just identifying problems, the agent provides crisp recommendations and can execute validated actions with your approval, such as adding indexes for a Cloud SQL instance. (In preview with select customers)
MCP tools: The Observability Agent derives insights with the help of tools such as system metrics, query metrics, fleet inventory, and issues, which are also available as MCP tools via the Database Insights MCP Server and Database Center MCP Server.

Integration that fits your workflow

You can access these Database Observability Agent capabilities directly within your existing database management processes. The agent powers several experiences, including:

Cloud Assist chat: Ask questions in natural language, for example, "What is the CPU utilization trend for my top Cloud SQL instances?" to get a summarized analysis complete with charts. Then, within the Chat window, you can start an investigation for any issues found,and get a root-cause analysis and remediations.

In-product investigations: Use Gemini Cloud Assist to investigate and remediate issues in-context on relevant database pages from the console.

Developer tools: Consume the agent’s capabilities through Antigravity or an IDE of your choice. This is augmented by the rich set of observability MCP tools that Google provides. All of these tools are available on Google Remote MCP servers. Combining them together is like giving developers a virtual DBA to optimize their databases, but all within their IDEs.

Supports multiple managed databases

You can use Observability Agent to get answers to your database queries, to access any database metric instantaneously, or to leverage AI-powered diagnosis to resolve complex problems. The agent covers a broad set of issues across a variety of Google Cloud databases, including:

Cloud SQL: Troubleshoot and optimize your database instance load, query performance or connectivity issues for all Cloud SQL database engines. For Cloud SQL for PostgreSQL, leverage the agent to troubleshoot common database issues.

Similarly, the agent helps you identify issues, find their root cause, and take remediation actions for other supported databases and issue types.

Spanner: Here, the most common troubleshooting scenario involves optimizing read and write latencies. The agent helps you do that in minutes, covering a broad set of scenarios ranging from hotspots to lock contentions.
AlloyDB: Troubleshoot and optimize your database instance load, query performance or replica lag issues.
Bigtable: Diagnose and optimize your read and write latencies, complete with crisp, actionable recommendations.

Database Onboarding Agent

The new Database Onboarding Agent is your active partner during the database selection process. Instead of spending hours reading documentation, you can describe your application requirements to the agent in natural language. The agent understands technical metrics like IOPS, latency limits, and replication lag, so it can provide a sound recommendation. You can access the Database Onboarding Agent’s capabilities directly within the Gemini chat interface. With the Database Onboarding Agent, you get:

Recommends database solutions: Analyzes user requirements regarding workload performance, scale, data type, and reliability to suggest optimal Google Cloud Managed Database services (e.g., Cloud SQL, Spanner, AlloyDB).
Smart recommendations: The agent reflects your requirements back to you, such as recommending AlloyDB for a high availability configuration, helping you have confidence in its selections.
Streamlined configuration: Once you choose a service, the agent generates the required commands. You can then use these commands to provision your database instance, configure the correct features, and deploy it.

Get started

The Database Observability and Onboarding Agent’s capabilities are available for a wide range of services, including AlloyDB, Bigtable, Cloud SQL (PostgreSQL, MySQL, SQL Server), Firestore, Memorystore, and Spanner. These agents are currently available via Gemini Cloud Assist. Explore AI assisted troubleshooting and Gemini Chat for AlloyDB, Cloud SQL, Spanner, and Visit Gemini Cloud Assist page to learn more.

How Target is enhancing retail discovery and cutting database maintenance by 50% with Spanner Graph

Tue, 04 Aug 2026 16:00:00 +0000

In today’s retail environment, shoppers expect highly personalized product discovery experiences and conversational assistance that feels genuine, natural, and genuinely helpful. Today, successful product discovery is about understanding semantic meaning and the rich, connected relationships between products, categories, and guest intent. It is no longer just about keywords and basic browsing.

At Target, this work is handled by our Guest Product Confidence platform team. They are responsible for building the features that establish trust and guide purchasing decisions, such as ratings, reviews, and AI-driven digital shopping assistants. An exciting example of this is our Gift Finder chat agent, which we launched during the 2025 holiday season online and in the Target app to help shoppers discover the perfect items through friendly, conversational dialogue.

To deliver real-time personalization and context-rich semantic responses like these at global scale, we identified a critical architectural need to move away from a fragmented data ecosystem toward a unified data platform. We needed a solution capable of supporting high-throughput transactional workloads, highly connected graph relationships, vector similarity search, and full-text keyword search all at once.

In this post, we’ll explore how we achieved all four with Spanner.

Overcoming fragmented architecture

Previously, Target’s discovery data ecosystem relied on a combination of Elasticsearch clusters for search and inverted indexes, alongside separate NoSQL datastores for our transactional data. While functional, this fragmented architecture presented significant operational and technical challenges.

Disconnected context: Keeping separate search, vector, and transactional databases in perfect sync was a constant challenge. Siloed information led to missing context, disconnected attribute relationships, and inconsistent query results.
High operational overhead: Managing independent clusters, tuning search indexes, and handling complex, custom synchronization and aggregation logic required intensive manual intervention from our engineering teams.
Expansion bottlenecks: Expanding our retail data domains required adding new database collections, maintaining complex joins, and navigating weak transactional guarantees across our discovery and core transactional systems.
Siloed intelligence: We lacked the ability to query graph relationships, vector similarity, and keyword search indexes in a single transaction.

To build the next generation of AI-driven guest experiences, we needed to consolidate on one platform.

Building the enterprise ontology on Spanner Graph

We evaluated multiple specialized technologies, including standalone vector databases and niche graph databases. However, adding more single-purpose databases would have only worsened our operational complexity and data synchronization pipelines.

We ultimately chose Spanner Graph to build our enterprise ontology, which is a "graph-of-graphs" paradigm that allows us to construct a massive, generative AI-powered shopping graph.

By unifying our data, we bring semantic data, graph relationships, vector embeddings, and operational transactions under one roof. This establishes Spanner as our single authoritative source of truth for both transactional state and semantic intelligence.

Our high-level architecture now consists of three core pillars:

1. Enterprise augmentation
This layer captures our enterprise retail catalog, aggregates relevant metadata from multiple backend sources, and utilizes generative AI for agentic data enrichment to dramatically improve the quality and depth of the product data we ingest.

2. Unified graph, vector, and search store
Instead of shifting data across multiple databases, Spanner Graph stores our entity nodes, relationship edges, and vector embeddings in the same database engine. Spanner Graph natively supports multi-hop graph traversals, semantic vector similarity, and full-text keyword queries over our relational tables. Because this multi-model synergy is native, we get strict ACID transactions for absolute correctness across distributed workloads without the need for fragile external sync pipelines.

3. Orchestration and AI layer
This layer powers our conversational guest interfaces, utilizing rich, structured context fed directly from Spanner Graph to ground our LLMs. It extracts highly specific product relationships to power tools like the Gift Finder while governing responsible AI processes and evaluating generated outputs.

A smooth, zero-downtime incremental migration

Transitioning critical search and discovery infrastructure that millions of guests rely on required a cautious, zero-downtime approach. We executed this migration in four structured phases.

Schema and ontology mapping: We defined the specific retail entities, such as products, categories, brands, and guest preferences, and their corresponding relationships within the Spanner Graph schema.
Data integration and parallel replay: We built mutation-based data integrations in a parallel pipeline. This allowed us to continuously replay live transactional updates, apply schema transformations, generate embeddings, and write them directly into Spanner Graph in real-time.
Canary deployment: We gradually shifted live read traffic to the new Spanner Graph-backed platform, validating query performance, semantic accuracy, and database stability under real retail workloads.
Cutover and cleanup: Once performance was thoroughly verified, we fully transitioned all search and discovery traffic to Spanner and deprecated our legacy Elasticsearch stack, entirely removing the maintenance burden of those clusters.

Business impact

By building directly on Spanner Graph, we unlocked measurable technical and business outcomes:

The ultimate GraphRAG foundation: Traditional RAG relies on flat vector similarity, which often misses the structured associations between products, such as matching a toy with its compatible accessories or age-appropriateness. By combining deep graph traversals with semantic vector search in a unified GraphRAG architecture, we grounded our LLMs with highly precise context. This directly improved our recommendation relevancy, enhanced guest satisfaction, and boosted our Net Promoter Score.

Consolidated SQL + GQL interoperability: With Spanner Graph, our developers query structured relational catalog data and connected graph relationships in a single query using standard SQL and GQL (Graph Query Language). This eliminates the need for data duplication, latency, or complex ETL pipelines to bridge these paradigms.

Serverless scalability with zero growth ceiling: Spanner automatically handled massive, unpredictable traffic spikes during peak retail events like Black Friday and Cyber Monday. Spanner's built-in autoscaler dynamically adjusted computing capacity to handle burst traffic during high-intensity, limited-time promotional offers without sacrificing performance.

50% reduction in infrastructure maintenance: By consolidating our transactional NoSQL and search index databases into a single managed Google Cloud service, we eliminated the operational burden of maintaining separate database clusters. Our developers now spend 50% less time on database administration and infrastructure upkeep, allowing us to build and deploy new, customer-facing AI features much faster.

Migrating to Spanner Graph has accelerated our generative AI roadmap, serving as the ultimate proof of what is possible when you build on the right data foundation.

Want to supercharge your AI apps? It starts with databases with the right graph capabilities at virtually unlimited scale. Discover how Spanner Graph can turn data into action for your organization.

Unifying public and private data: Scale knowledge graphs with Data Commons on Spanner

Mon, 03 Aug 2026 16:00:00 +0000

To make informed decisions, businesses often need to connect their internal data with public reference data, to create a knowledge graph that connects real-world things and their relationships. However, bridging data from public and private worlds has traditionally been complex. Today, we are streamlining these connections with the general availability of Data Commons on Spanner Graph and the preview of the new Data Commons Platform to unify your private knowledge with knowledge graphs from public datasets.

The overarching Data Commons project supports Google’s mission to organize the world's information and make it universally accessible and useful. Data Commons unifies fragmented public datasets from over 100 authoritative providers, including the United Nations, World Bank, US Census Bureau, Eurostat, WHO, and NOAA, with over 400 billion data points structured using standardized Schema.org definitions. Data Commons provides data exploration tools, MCP tools, and cloud-based APIs to access and integrate the clean datasets.

Data Commons integrates public information across multiple domains, including agriculture, demographics, economy, environment, and health. This standardized approach unlocks powerful use cases, for instance, letting you analyze national GDP trends, map regional smoke pollution levels, track local health equity, or demographic distributions over time, all using data that has already been preprocessed and normalized for you.

Data Commons knowledge graph dimensions

Dimension	Size	Technical description
Statistical observations	400+ billion	Individual metric data points
Graph edges	2.6+ billion	Relationships
Knowledge graph nodes	1.7+ billion	Standardized entities
Data sources	100+ providers	Authoritative institutions

^{Data Commons makes meaningful quantities of public administrative data available to users on readily consumable cloud-based infrastructure.}

A modern infrastructure powered by Spanner Graph

When we first built Data Commons, our goal was to aggregate massive, disparate public datasets using the tools available at the time. The platform relied on Bigtable as a caching layer, which was an effective strategy for handling large-scale lookups in the absence of native graph database technology.

Today, we have transitioned our architecture to a native graph model with Spanner Graph, which brings the convenience of a SQL-like interface and graph expressiveness to Spanner, with its high availability, horizontal scale-out, multi-region transactional consistency, and native ISO/IEC 39075 Graph Query Language (GQL) support.

By adopting a multi-entity Spanner Graph schema, we represent entities as nodes and their domain links as dynamic graph edges, allowing us to move away from pre-computed cache structures and perform complex relationship queries directly within the database using GQL.

This architecture also simplifies our pipelines by removing the need for complex, pre-computed indices that require costly in-memory rebuilds and multiple snapshots. Spanner Graph enables incremental updates to specific datasets without refreshing the entire database, while stale reads maintain consistent data snapshots during ingestion.

Key benefits by moving to Spanner Graph

Unified storage and incremental updates: By utilizing Spanner Graph’s multi-entity schema, the platform replaces complex caches with a model that supports incremental data imports, allowing for targeted updates to specific datasets.
Dynamic graph traversals via GraphRAG: The system executes multi-hop queries such as navigating hierarchies like continent → country → state → county → city on the fly. This removes reliance on static caches and enables GraphRAG workflows, where the database maps natural language queries directly to structured path-matching traversals.
Consistent data snapshots: Leveraging Spanner TrueTime and stale reads, the platform provides you with a version-consistent snapshot of data, maintaining integrity across distributed nodes following batch ingestion cycles.
Operational analytics at scale: Spanner’s columnar engine efficiently scans massive time-series datasets by reading only the necessary fields, while BigQuery federation that leverages Spanner’s Data Boost technology performs complex aggregations via EXTERNAL_QUERY in an isolated environment, helping isolate production traffic.

Bridging systems with SDMX 3.0 interoperability

To facilitate the use of complex statistical data, Data Commons adopts a lean implementation of Statistical Data and Metadata eXchange (SDMX) technical standard. As an ISO specification, SDMX provides a consistent approach for describing and exchanging statistical data along with descriptive statistical meta-information.

In this Data Commons Platform update we added support for the SDMX technical standard version 3.0, providing out-of-the-box integration with third-party tools like Tableau, Flourish, and Observable for multi-dimensional datasets. This is made possible using the API standard SDMX-JSON and SDMX-CSV 2.0 formats across two high-value endpoints:

The availability API: A programmatic discovery mechanism to identify existing dimensions, variables, and date ranges without reading raw values.
The data API: Retrieves actual observations and metadata, using named parameters to help prevent code from breaking when dimensions are added.

Transforming private instances of Data Commons Platform

For organizations that want to build private instances of the Data Commons Platform, this new modern architecture resolves legacy scaling limits and simplifies data schematization. Developers can instantiate a private instance of the Data Commons Platform leveraging the same scalable technology that powers Google’s Data Commons instance. As a private instance, users retain full control of their own data and have the ability to limit access, while enabling natural language queries to blend results from their private data with Google’s public data that is hosted on the Google Data Commons instance. By federating across our public knowledge graph and a private knowledge graph containing your own data, you can light up exciting new use cases, while maintaining data isolation and ensuring no data duplication.

For instance, a retail enterprise can combine public data such as national GDP trends, regional demographic breakdowns, and employment statistics, with their own enterprise data, including sales histories, store performance metrics, and supply chain logistics. This allows analysts to contrast public macroeconomic indicators against their own company transactions to optimize merchandise distribution and identify untapped markets.

Example of a natural language query combining statistical data from the Directorate General of Commercial Intelligence and Statistics (DGCIS) stored in a Data Commons Platform private instance with World Development Indicators from the World Bank stored in the Google Data Commons public instance.

A user is querying a Data Agent for average annual temperature trends in the country. The agent retrieves information from Data Commons, explaining that while historical data is available, it provides projected temperature changes, climate drivers, and CMIP6 climate model scenarios (SSPs), with options to export the generated report.

A user asks the Data Agent to compare the Worker Population Ratio (WPR) of rural versus urban males in a country. Fetching data from Data Commons, the agent defines WPR—the percentage of workers relative to the total population—and outlines the available demographic variables to analyze and compare both groups.

Get started today

Explore Data Commons: Visit datacommons.org to query global statistical knowledge.
Explore Spanner Graph's use cases and setup guide for your knowledge graphs.
Deploy Data Commons Platform: contact support@datacommons.org to request preview access and to review the developer tools.

AlloyDB adds group authentication to secure enterprise scale and AI agents

Thu, 30 Jul 2026 16:00:00 +0000

Database security traditionally relies on a fragile balance between the granular control developers need and the administrative overhead of managing thousands of individual database passwords. Between managing AI agent access, rotating static credentials, handling employee on-boarding and off-boarding, and auditing access logs, passwords remain an operational tax — and a potential security vulnerability.

At Google Cloud, our goal is to help make database access transparent, secure, and passwordless.

Today, we are taking an important step forward in that journey. We’re announcing Identity and Access Management (IAM) group authentication for AlloyDB, available in preview. This capability brings identity-driven access control to your enterprise workloads.

Cloud SQL customers have already adopted this authentication pattern with great success, and this launch unifies our security stance across both services.

The problem with individual scale

For years, both Cloud SQL and AlloyDB have mapped individual Google Cloud identities directly to database users using native IAM authentication. However, at enterprise scale, managing access on an individual basis can introduce significant complexity.

Without group-based management, scaling to hundreds of instances and thousands of users creates distinct challenges:

On-boarding bottlenecks: Every new team member requires individual database user provisioning.
Off-boarding risks: Ensuring an employee’s access is entirely removed across a distributed database environment can complicate auditing.
Policy drift: Maintaining identical permissions across development, staging, and production systems becomes highly error-prone.

Faced with these challenges, it is tempting to use a single, powerful user or service account to serve a whole application. However, oversimplifying access for such powerful application accounts comes at the cost of risk exposure and loss of granular auditing capabilities.

Securing the future of agentic AI

The scale challenge isn't just about human users anymore. As organizations deploy an increasing number of AI agents, managing database identity and access controls will become more complex.

If an AI agent connects to a database using a generic, shared account, or a broad service account, it risks acting as a confused deputy. When using credentials with overly-powerful permissions instead of carrying through the user’s identity, an agent could access or modify data beyond what the end user requesting the action is authorized to see. Crucially, it can hide individual accountability from audit logs because actions map to a generic service account.

Granular authentication can help mitigate this risk. Agents can pass the end user’s specific identity and authentication scope through to the database layer so that queries are run on behalf of the user, limiting data access to objects that an end user is allowed to.

Passing user group identity through an AI agent to AlloyDB allows the database to authorize access and record precise audit trails.

IAM group authentication simplifies this architecture. Instead of managing micro-permissions for every combination of agent and user, security teams can define up to 200 functional Google Groups (such as financial-agents@company.com or regional-analysts@company.com). Google Cloud’s managed database infrastructure validates the user's group context, helping ensure the database authorizes data access at the database or table level while audit logs capture exactly what data was accessed, modified, and on whose behalf.

Proven value for digital leaders

Enterprises are already improving their operational velocity by adopting centralized identity principles. Bilt, a leading platform rewards program, uses our unified approach to help enhance the security of its high-scale database environments:

"By combining AlloyDB’s group-based IAM with our automated group management and Privileged Access Manager (PAM) entitlements, we've eliminated the risk of shared credentials entirely. Database and role provisioning are now fully templated from day one, allowing our engineers to securely access only the data they need and exactly when they need it," said Kosta Krauth, CTO, Bilt.

A unified blueprint for passwordless access

With this launch, Google Cloud provides a unified approach for access control across both Cloud SQL and AlloyDB. Organizations can now enforce a standardized, defense-in-depth access strategy across their relational database portfolio.

By pairing IAM group authentication with features like VPC Service Controls, Organization Policies, IAM conditions, and Private Service Connect, security teams can help ensure that database access — whether by a human engineer or an autonomous AI agent — is bound to verified corporate identities and secure network perimeters.

Moving toward a Zero Trust database future

Security shouldn't force a trade-off between engineering velocity and compliance. By integrating AlloyDB with Cloud Identity and Workforce Identity Federation, we are removing the friction of database administration while helping you implement a more secure architecture.

You can explore how to set up group-based database roles by trying out the feature today. You can find more recommendations and best practices for security and compliance in our documentation.

Your AI agents are ready. Is your data?

Thu, 23 Jul 2026 16:00:00 +0000

What’s one of the biggest bottlenecks stopping organizations from scaling their AI initiatives? It isn’t the capabilities of today’s models — it’s their access to business context and semantic meaning.

In the agentic era, enterprises need to go beyond simply storing data to activating it with trusted context, moving from passive systems of record to proactive systems of action.

But AI agents operate with nonlinear speed; for example, a single prompt can trigger the agent to independently browse, query, and execute across multiple systems, placing stress on the underlying infrastructure. If the compute, networking, and storage layers aren't optimized for agentic AI, the data platform sitting on top of them will buckle.

It’s no wonder that, according to our State of infrastructure report, 83% of organizations believe they require infrastructure upgrades to support production-grade agentic AI systems.

To solve this problem, we introduced the Agentic Data Cloud at Google Cloud Next 2026; unifying your data, AI models, and operational databases into a single System of Action. To make an Agentic Data Cloud work, it must be AI-native from the chip to the model. The underlying infrastructure must be able to accommodate agentic load.

Google’s Agentic Data Cloud

Let’s explore how the right infrastructure foundation empowers an Agentic Data Cloud to solve the biggest data challenges organizations face today.

Overcoming a lack of context

To be effective, agentic systems require access to context that is often found in fragmented data systems and legacy architectures. This can make it hard for agents to get this context, leading to incomplete, inaccurate results. In fact, our report found that 43% of IT leaders cite “difficulty integrating with legacy APIs and data sources” as their biggest agentic AI infrastructure gap.

But organizations cannot simply move massive datasets and connect them to AI without increasing complexity and cost.

Our Agentic Data Cloud solves this by leveraging a borderless Lakehouse running on open, flexible infrastructure. By accessing powerful native engines like BigQuery and Spanner over open standards (Apache Spark, Apache Iceberg), agents can read, reason over, and activate data across environments as if it were local, bypassing the latency and costs of traditional setups.

Escaping unnecessary manual work

Scaling agents on a patchwork of disconnected systems can create significant bottlenecks. In our research, 81% of leaders called out operational complexity and engineering overhead as top unforeseen expenses when scaling AI, citing the time engineers spend doing manual work to patch together AI agents across disparate systems.

To move from thinking to doing, agents must be able to connect real-time data across both analytical and operational sources. This requires vertical integration. When an Agentic Data Cloud is built on an AI-native infrastructure where the models, data systems, and underlying accelerators are co-designed, there are fewer network hops and tooling is better integrated. This unified system allows an agent to reach an insight and trigger secure transactions without the typical engineering overhead.

Bringing trust and knowledge to the data

It’s not enough for agents to just discover and query data. To take safe, accurate actions, agents also need rich context and business logic. Yet, 36% of leaders cite a lack of specialized, high-throughput vector databases used for AI model grounding, as a key infrastructure gap, hindering their ability to give agents context.

In order to work to their full potential, agents need a foundation which is built to read and write data systems in real-time, including legacy ERPs and third-party CRMs. It also gives them the long-term memory to recall a user’s preference from, say, three weeks ago, while executing a complex task today. And without this real-time automation, agents have to re-process data for every single query.

To provide context for AI, organizations are using Knowledge Catalog to aggregate and enrich data in their data lakes, and enable agentic searches. By extracting meaning from unstructured data and automatically generating semantics, the catalog acts as an active reasoning layer. That catalog in turn, must be backed by high-throughput infrastructure, so that agents can retrieve the right context.

The path forward

To turn AI into a true competitive advantage, it’s time to build a connected, active data ecosystem. Giving your agents seamless access to all of your data is a must to move from pilots to production, and this must be supported by an infrastructure that can handle the demands of the agentic era. The winners in 2026 and beyond won’t necessarily be the ones with the smartest agents. They’ll be the ones who can feed those agents the right knowledge — securely, cost-effectively, and at scale. Is your data ready for the agentic era?

See how leaders are taking an AI-optimized approach to architecture in the State of infrastructure in the agentic AI era report.

Supercharging pgvector: 4x faster HNSW vector search with AlloyDB

Tue, 21 Jul 2026 16:00:00 +0000

AlloyDB is a fully managed, PostgreSQL-compatible database service built for your most demanding enterprise workloads. It combines the best of open source PostgreSQL with Google’s advanced technology, offering massive scalability, high availability, and native AI capabilities. It serves as a performant relational store, a unified backend for vector and full text search, and an analytics engine that is up to 100x faster than standard PostgreSQL.

Vector search is the foundation of modern AI and Retrieval Augmented Generation (RAG) applications. For developers using AlloyDB and other PostgreSQL databases, pgvector is a widely adopted extension for storing, indexing, and querying vector embeddings, and HNSW (Hierarchical Navigable Small World) is a highly efficient graph-based algorithm designed for approximate nearest neighbor search across multi-layered structures. With columnar engine accelerated HNSW in AlloyDB (now in preview), you can achieve up to 4x higher queries per second (QPS) for vector search compared to standard PostgreSQL HNSW.

Enterprise AI applications face a constant trade-off between speed and accuracy. When searching through millions or billions of vectors, maximizing Queries per Second (QPS) without sacrificing search quality (recall) is critical for scaling production workloads. The PostgreSQL pgvector extension offers HNSW as one of the indexes that can speed up Approximate Nearest Neighbor (ANN) searches. Let’s dive deep into how AlloyDB solves the speed vs. accuracy trade-off.

Note: While this post focuses on HNSW performance, it’s worth noting that HNSW is just one part of AlloyDB’s advanced vector toolkit. AlloyDB also features ScaNN—a cutting-edge index backed by over 14 years of Google Research—giving you the flexibility to choose the perfect index for your workload. Additionally, for use cases demanding absolute precision, standard k-nearest neighbor (KNN) search is always available for 100% recall. Check out our Choose a Vector Index Guide to see how they stack up.

aside_block: <ListValue: [StructValue([('title', 'Get started with a 30-day AlloyDB free trial instance'), ('body', <wagtail.rich_text.RichText object at 0x7faef037f5b0>), ('btn_text', 'Start building for free'), ('href', 'http://goo.gle/try_alloydb'), ('image', None)])]>

First, what is the AlloyDB columnar engine?

The AlloyDB columnar engine is a built-in, in-memory cache that automatically stores frequently queried data in a specialized, scan-optimized columnar format. It allows AlloyDB to handle heavy analytical queries up to 100x faster than standard PostgreSQL. Additionally, it accelerates ANN searches by storing the index in memory, using a vectorized memory layout for fast traversals, and bypassing standard PostgreSQL buffer manager overhead.

Performance visualization

To understand the real-world performance characteristics of columnar engine Accelerated HNSW, we plotted standard QPS vs Recall curves for the GloVe 100 Angular dataset by searching more than 1M records with a limit of 100.

Running this benchmark script yields the following visualization:

Note: These measurements were taken on an AlloyDB C4A 16vCPU machine. Due to the inherent randomness in HNSW graph building, results may slightly vary across runs.

The data reveals two transformative benefits:

Massive performance throughput gains: For any given target recall (e.g. 0.95), QPS is increased by approximately 4.2x to 4.9x. This allows you to handle significantly more concurrent vector searches on the same hardware.
Significant recall (accuracy) improvement: Conversely, at a fixed QPS level, columnar engine accelerated HNSW provides a substantial boost in recall. For example, we saw that at ~350 QPS (in the above chart), enabling the columnar engine improves recall from roughly 0.78 to over 0.94 – a 0.163 recall gain. This means your AI applications get much more accurate results without any latency impact.

It is important to note that the baseline (blue line) already represents the index being fully cached in the PostgreSQL shared buffer cache. The performance gains shown here are not the result of moving data from disk to RAM, but rather the result of a more efficient memory architecture.

How it works: Columnar engine Accelerated HNSW

In standard PostgreSQL architectures, index operations utilize the shared buffer cache. Even when data is fully in-memory, the database still incurs significant overhead from the buffer manager, which must handle operations such as page pinning and unpinning, lock acquisition, buffer table lookups, and Least Recently Used (LRU) management.

AlloyDB's columnar engine is a built-in, in-memory cache that stores data in a specialized, scan-optimized format.

With this release, AlloyDB can use columnar engine accelerated HNSW to:

Pin the index: The pgvector HNSW index is pinned (kept persistently in-memory to ensure fast access) directly into the columnar engine’s memory.
Vectorized access: It utilizes a memory layout specifically designed for the high-concurrency, pointer-heavy traversals required by HNSW graphs.
Bypass buffer overhead: By navigating the graph in a specialized memory space, AlloyDB avoids the standard buffer manager bottlenecks. This architectural shift is what enables the dramatic QPS and recall improvements shown above, even when comparing against a fully-cached standard index.

Why it Matters

For enterprise-scale applications, this isn't just about a faster database—it's about cost and quality:

Reduced infrastructure costs: Achieve the same performance with significantly lower compute resources.
Better AI accuracy: Reach higher recall and quality at speeds that were previously only possible for "draft" (high-speed, lower-accuracy results) quality search.
No application changes required: Because this is built into AlloyDB, you get these gains using the same standard pgvector SQL syntax.

Note that the columnar engine does utilize memory, but it is highly compressed and meticulously managed. Because the engine stores vector data in an efficient columnar format, the memory footprint is minimal compared to the massive performance gains—making it a highly favorable trade-off for enterprise workloads.

Quick Start Guide

To try out columnar engine accelerated HNSW in AlloyDB, follow these steps:

1. Enable the columnar engine and index caching

Ensure that both google_columnar_engine.enabled and google_columnar_engine.enable_index_caching flags are set to on for your AlloyDB instance.

2. Add the HNSW Index to columnar engine

Once your HNSW index is created via pgvector, execute the following SQL command to cache it in the columnar engine:

code_block: <ListValue: [StructValue([('code', "SELECT google_columnar_engine_add_index('<hnsw_index_name>');"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef037f910>)])]>

3. Additional Resources

New to AlloyDB? Discover AlloyDB with a 30-day free trial.
Google Colab Notebook: An end-to-end Python script to ingest the GloVe dataset, create indexes, and plot Recall vs QPS curves.
Is HNSW the right vector index choice for your use case? Check our ‘Choose a vector index in AlloyDB AI’ guide.

How to solve PostgreSQL multilingual full-text search limitations with AlloyDB AI

Wed, 15 Jul 2026 16:00:00 +0000

AlloyDB powers enterprise-grade search for some of the largest organizations, providing robust hybrid search capabilities that combine text, vector, and keyword searches into a simple ranked SQL query. And with our recent launch of RUM index support, AlloyDB customers now have even more powerful full-text search capabilities at their fingertips.

However, database developers often face limitations when indexing continuous text in logographical languages like Chinese, Japanese, and Korean, where traditional whitespace-based tokenization fails. Gemini’s multilingual capabilities enable you to intelligently parse text in these languages to implement intelligent word segmentation and stop-word removal, but orchestrating row-wise API calls on massive datasets is slow and fragile. Now you can integrate the world knowledge of Gemini models natively into your database using AlloyDB AI Functions, enabling highly accurate full-text search for logographical languages without the administrative overhead of complex ETL pipelines.

Continuous text and logographical languages

To understand why this native integration is such a significant advancement, we must first examine the underlying mechanics of text search and why traditional indexing methods fail when processing continuous text.

To build an effective full-text search index in PostgreSQL, the engine must parse text into search tokens (lexemes) using the to_tsvector function. By default, standard text search configurations like simple or english assume that words are separated by whitespace. The database engine extracts search terms by splitting the input string at these spaces.

However, Chinese and other logographical languages do not use spaces between words. Words are written continuously, with punctuation serving as the only boundaries. Because of this, standard PostgreSQL parsers fail to extract individual keywords. Instead, they treat entire sentences or long clauses as a single, continuous lexeme.

For example, consider this input string: "你们研究所有十个图书馆" (Your research institute has ten libraries)

Without spaces, passing this to to_tsvector('simple', ...) produces a single, massive lexeme: '你们研究所有十个图书馆'

If you search for "研究所" (research institute) or "图书馆" (library), the query fails to return a match. The keywords are trapped inside the larger string, forcing you to search for the exact, long-form sentence to get a result.

Traditional workarounds and their limits

You might try to resolve this using traditional tools and pipelines, each of which introduces significant operational friction or accuracy limits:

Third-party database extensions: Extensions like zhparser or pg_jieba add Chinese tokenization to PostgreSQL. However, these are often not supported in fully-managed database environments. They also rely on static dictionaries, which frequently fail to parse modern jargon, brand names, or context-dependent terms correctly.
External preprocessing pipelines: Exporting text to an external application (such as a Python microservice running jieba or spaCy) to insert spaces before saving it to the database. This pattern introduces substantial ETL complexity, network latency, and data exposure risks by moving your data out of the database tier.
Inadequacy of rule-based and dictionary-driven segmentation: Traditional tokenizers rely on static dictionaries and hand-coded syntactic rules to split text. They often struggle to resolve semantic ambiguity, where the exact same sequence of characters must be segmented differently depending on the context. To perform accurate segmentation, you need the world knowledge and contextual intelligence of a large language model like Gemini.

In-database pre-processing with Gemini

AlloyDB AI bypasses these workarounds — and their limits — by introducing native, in-database AI Functions like ai.generate(). This allows you to call Gemini directly from SQL, keeping your data and intelligence in one place.

This approach provides three core advantages:

No data movement: All text preprocessing and segmentation happen directly within the database engine. This minimizes network latency and keeps your data protected within your database boundaries.
In-database intelligence: You do not need to build, deploy, or maintain external microservices or orchestration frameworks. The database engine coordinates the model calls natively.
Stored procedure-based batching: By using a PL/pgSQL stored procedure with array aggregation, you can process rows in parallel batches, unpack the results safely using GENERATE_SERIES, and commit each batch immediately. This prevents database memory exhaustion, bypasses row lock contention, and supports stable, performant execution even when handling massive tables.

Implementing semantic word segmentation

To implement this solution, we will create a database table where the raw content, the segmented text, the search vector, and the vector embeddings are stored together.

code_block: <ListValue: [StructValue([('code', "CREATE TABLE documents (\r\n id SERIAL PRIMARY KEY,\r\n title TEXT NOT NULL,\r\n original_content TEXT NOT NULL,\r\n content_segmented TEXT,\r\n search_vector tsvector GENERATED ALWAYS AS (to_tsvector('english', content_segmented)) STORED,\r\n embedding vector(3072) GENERATED ALWAYS AS (embedding('gemini-embedding-001', content_segmented)) STORED\r\n);"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef00e6700>)])]>

By defining search_vector and embedding as generated columns, AlloyDB automatically updates both the full-text search index and the vector embeddings whenever the content_segmented column is updated. This reduces your application-side logic to a single update statement.

Step 1: Document batch segmentation

If you consult the AlloyDB AI documentation, it recommends using cursor-based processing when dealing with large datasets (10,000 to millions of rows) to avoid memory bottlenecks.

However, the documentation’s cursor examples focus on append-only operations — streaming text into a new, empty table using INSERT. Our use case requires an in-place UPDATE on our live documents table. Implementing this with a raw cursor loop in a standard anonymous block (DO $$) introduces three important production hazards: excessive row locking that freezes live applications, a rollback risk if a network blip occurs, and alignment risks where parallel cursors fall out of step.

To mitigate these challenges and accelerate performance, we use a stored procedure configured with high-throughput array-based batching:

code_block: <ListValue: [StructValue([('code', "CREATE OR REPLACE PROCEDURE segment_all_documents(p_batch_size INT DEFAULT 100)\r\nLANGUAGE plpgsql AS $$\r\nDECLARE\r\n v_processed_count INT;\r\nBEGIN\r\n LOOP\r\n -- 1. Grab a single isolated batch, aggregate into arrays, and call Gemini.\r\n -- We explicitly ORDER BY id during aggregation to guarantee the arrays match perfectly.\r\n WITH batch_raw AS (\r\n SELECT id, original_content\r\n FROM documents\r\n WHERE content_segmented IS NULL OR content_segmented = ''\r\n ORDER BY id\r\n LIMIT p_batch_size\r\n ),\r\n batch_processed AS (\r\n SELECT \r\n ARRAY_AGG(id ORDER BY id) AS target_ids,\r\n ai.generate(\r\n prompts => ARRAY_AGG('Perform Chinese word segmentation (分词) on the provided text to prepare it for full-text search indexing.\r\nRules:\r\n- Insert a single space between every atomic, meaningful word.\r\n- Separate all punctuation marks (both full-width and half-width) with spaces.\r\n- Preserve the original structure, line breaks, and non-Chinese characters (e.g., English words, numbers).\r\n- Output ONLY the processed text. Do not include any greetings, explanations, or formatting wrappers like markdown code blocks unless they exist in the original text.\r\nText to process: ' || original_content ORDER BY id),\r\n model_id => 'gemini-2.5-flash-lite'\r\n ) AS ai_outputs\r\n FROM batch_raw\r\n ),\r\n -- 2. Use GENERATE_SERIES to unpack the paired array indexes safely\r\n unpivoted_results AS (\r\n SELECT \r\n b.target_ids[i] AS doc_id,\r\n b.ai_outputs[i] AS segmented_text\r\n FROM batch_processed b,\r\n GENERATE_SERIES(1, COALESCE(ARRAY_LENGTH(b.target_ids, 1), 0)) i\r\n )\r\n -- 3. Execute the batch update for this chunk\r\n UPDATE documents d\r\n SET content_segmented = r.segmented_text\r\n FROM unpivoted_results r\r\n WHERE d.id = r.doc_id;\r\n\r\n -- Check how many rows were updated in this pass\r\n GET DIAGNOSTICS v_processed_count = ROW_COUNT;\r\n\r\n -- If the update affected 0 rows, it means the whole table is finished\r\n EXIT WHEN v_processed_count = 0;\r\n\r\n -- 4. Commit immediately to save progress and release row locks!\r\n COMMIT;\r\n \r\n RAISE NOTICE 'Successfully processed and committed a batch of % rows.', v_processed_count;\r\n END LOOP;\r\nEND $$;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef00e6910>)])]>

To run this preprocessing pipeline across your entire table, simply call the stored procedure:

code_block: <ListValue: [StructValue([('code', 'CALL segment_all_documents(100);'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef00e6ac0>)])]>

This stored procedure approach provides three benefits that directly solve your production challenges:

Parallelized array aggregation (solving the sequential bottleneck): By aggregating the batch into arrays and calling ai.generate(prompts => ...) with the array, AlloyDB batches the model requests and executes them in parallel. This is faster than processing rows one-by-one sequentially.
Safe index-based unpacking (solving cursor desync): Using GENERATE_SERIES to unpack the array indexes maps the model output back to the correct document ID, reducing the risk of parallel streams falling out of step.
Immediate commits and lock release (solving blocking and rollback risks): Committing the transaction at the end of each loop iteration (COMMIT;) is an essential optimization. It immediately saves progress to disk and releases row locks, preventing long-running transactions from freezing your live application and ensuring a network blip won't roll back hours of work.

Once this pipeline runs, our example sentence is stored in content_segmented as: "你们研究所有十个图书馆"

Step 2: Choosing simple vs. english

When defining your tsvector generated column, you must choose the appropriate PostgreSQL text search configuration. This choice depends on your dataset:

When to use simple: If your database contains only Chinese text, the simple configuration is ideal. It converts text to lowercase but does not perform stemming or default stop-word removal. Since the model prompt already handles semantic segmentation and custom stop-word filtering, the simple configuration maps directly to the model's optimized output without further modification.
When to use english: In modern enterprise applications, Chinese documentation and user queries frequently contain embedded English terms (e.g., product codes, brand names, or technical terms). In these bilingual scenarios, the english configuration is superior. The English Porter stemmer leaves non-ASCII Chinese characters completely intact as exact lexemes, while automatically normalizing the English terms (e.g., stemming "running" to "run") and filtering out generic English stop words ("the", "and"). This achieves unified bilingual search capabilities without requiring separate columns or complex routing logic.

Step 3: Query-time preprocessing

Segmenting the document content is only half the battle. To match the indexed data, the incoming search queries must be pre-processed using the same Gemini-based segmentation logic.

We can take this a step further to improve search precision. We can instruct the model to act as an intelligent stop-word filter, stripping away low-value grammatical noise that would otherwise clutter your search results:

Grammatical particles (e.g., 的, 了)
Pronouns (e.g., 你, 我们)
Comparison words (e.g., 比, 最)
Question words (e.g., 怎么, 为什么)

For example, we can process a user query on the fly using a SQL query:

code_block: <ListValue: [StructValue([('code', "SELECT ai.generate(\r\n 'Perform Chinese word segmentation (分词) on the provided search query.\r\n Additionally, remove low-value grammatical noise such as particles (e.g., 的, 了), pronouns (e.g., 你, 我们), comparison words (e.g., 比, 最), and question words (e.g., 怎么, 为什么).\r\n Rules:\r\n - Insert a single space between every remaining meaningful word.\r\n - Output ONLY the processed keywords. Do not include greetings or explanations.\r\n Query to process: 你们研究所的图书馆在哪里？'\r\n);"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef00e6100>)])]>

The model processes this query and returns the keyword string: "研究所图书馆"

Step 4: executing the search with RUM

With your documents segmented and indexed, you can create a RUM index on your generated search_vector column. A RUM index is an index type that stores lexeme positions directly. This helps AlloyDB calculate search relevance and word distance directly within the index, avoiding the slow re-scan operations required by traditional GIN indexes.

code_block: <ListValue: [StructValue([('code', 'CREATE INDEX idx_docs_rum\r\nON documents\r\nUSING rum (search_vector rum_tsvector_ops);'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef00e6a00>)])]>

To run a search, you convert your preprocessed query string ("研究所图书馆") into a search query using plainto_tsquery and execute it against the RUM index. You can use the RUM distance operator (<=>) to sort the results by relevance:

code_block: <ListValue: [StructValue([('code', "SELECT id, title, original_content,\r\n search_vector <=> plainto_tsquery('english', '研究所 & 图书馆') AS distance\r\nFROM documents\r\nWHERE search_vector @@ plainto_tsquery('english', '研究所 & 图书馆')\r\nORDER BY distance ASC;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef00e6e20>)])]>

Because the RUM index calculates the distance score directly, this query executes with high efficiency, returning relevant matches in milliseconds.

Extending to hybrid search

While keyword-based text search is excellent for finding exact matches, it can miss relevant documents that use different terminology. To solve this, you can combine your segmented full-text search with semantic vector search using the multilingual gemini-embedding-001 model. This pattern, known as hybrid search, retrieves results that are both lexically and semantically relevant.

AlloyDB makes it easy to run hybrid search. You can create a ScaNN index (Google's vector index technology) on your embedding column and combine it with your RUM index.

Creating the ScaNN index

To accelerate your vector search, you create a ScaNN index on the embedding column:

code_block: <ListValue: [StructValue([('code', 'CREATE EXTENSION IF NOT EXISTS alloydb_scann;\r\n\r\nCREATE INDEX idx_docs_scann\r\nON documents\r\nUSING scann (embedding cosine);'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef00e6310>)])]>

Running the hybrid search query

To combine the results of your vector and text searches, you can use a SQL query that implements Reciprocal Rank Fusion (RRF). RRF is a rank-based algorithm that merges multiple search result lists into a single, unified list by assigning a score to each document based on its rank in the individual lists.

The following query performs both searches in parallel using Common Table Expressions (CTEs), joins the results using a FULL OUTER JOIN, and calculates the final RRF score:

code_block: <ListValue: [StructValue([('code', "WITH vector_search AS (\r\n SELECT id,\r\n RANK() OVER (ORDER BY embedding <=> ai.embedding('gemini-embedding-001', '研究所图书馆')::vector) AS rank\r\n FROM documents\r\n ORDER BY embedding <=> ai.embedding('gemini-embedding-001', '研究所图书馆')::vector\r\n LIMIT 10\r\n),\r\ntext_search AS (\r\n SELECT id,\r\n RANK() OVER (ORDER BY search_vector <=> plainto_tsquery('english', '研究所 & 图书馆')) AS rank\r\n FROM documents\r\n WHERE search_vector @@ plainto_tsquery('english', '研究所 & 图书馆')\r\n ORDER BY search_vector <=> plainto_tsquery('english', '研究所 & 图书馆')\r\n LIMIT 10\r\n)\r\nSELECT\r\n COALESCE(vector_search.id, text_search.id) AS id,\r\n COALESCE(1.0 / (60 + vector_search.rank), 0.0) + COALESCE(1.0 / (60 + text_search.rank), 0.0) AS rrf_score\r\nFROM vector_search\r\nFULL OUTER JOIN text_search ON vector_search.id = text_search.id\r\nORDER BY rrf_score DESC\r\nLIMIT 5;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef00e6040>)])]>

In this query:

The vector_search CTE uses the ScaNN index to find the top 10 documents that are semantically closest to your query, using the gemini-embedding-001 model.
The text_search CTE uses the RUM index to find the top 10 documents that match your segmented keywords, using the RUM distance operator for ranking.
The final select statement joins these lists and calculates the RRF score using the standard constant of 60. The top 5 results are returned, providing a precise blend of exact keyword matches and semantic matches.

Why AlloyDB AI is ideal for search

By using AlloyDB AI to solve the challenges of multilingual and hybrid search, you build a robust, scalable, and cost-effective foundation for your enterprise AI applications.

Native, in-database intelligence: By running model processing directly within AlloyDB AI, you avoid the cost, latency, and data exposure risks of moving transactional data to external AI services.
Enterprise-grade search performance: Combining ScaNN vector search (built on Google's search technology) and RUM full-text search in a single relational database delivers fast, accurate search outcomes without needing to maintain separate, complex search engines.
High context-aware accuracy: Unlike static, rule-based dictionaries that struggle with modern terminology, Gemini's world knowledge brings deep semantic understanding to word segmentation, providing high search precision.
Operational simplicity: You get robust bilingual and hybrid search capabilities using standard SQL. This means you can build and scale AI applications using the database skills you already have, without learning new APIs or managing complex external pipelines.

What's next

Giving your applications the ability to safely and efficiently interact with transactional data moves us away from fragmented data silos and toward an architecture where AI can reliably access enterprise truth.

Ready to build? Discover AlloyDB with a 30-day free trial, and dive into the Getting started with hybrid search in AlloyDB Codelab to start creating intelligent search experiences in your applications today.

Building the AI-defined vehicle with Android, Google Cloud, and Nexus SDV

Mon, 13 Jul 2026 16:00:00 +0000

The automotive industry is moving from building hardware-centric platforms toward building their own sophisticated Software-Defined Vehicle (SDV) architectures. For OEMs, a vehicle is no longer just a way to go from point A to point B, but an intelligent, connected node within an AI-native ecosystem!

With its partners, Google’s Android and Google Cloud are at the forefront of this transition. Android’s open source Automotive OS (AAOS) SDV implements the AI-defined vehicle while Google Cloud provides scalable infrastructure including a full suite of AI integration tools, leveraging services like Bigtable for automotive and manufacturing telematics at scale. Valtech, a Google Cloud partner, uses Google technologies as part of its Nexus SDV platform, establishing a full end-to-end connected vehicle system that enables truly agentic mobility, offering automotive OEMs a ready-to-use, end-to-end foundation for the next generation of connected vehicles. Let’s take a look at how this all comes together.

The vehicle side: AAOS SDV

As the foundational in-vehicle platform, Google’s open source AAOS SDV platform abstracts core functions into reusable services independent of physical hardware, establishing a modular Service-Oriented Architecture (SOA). By decoupling non-safety domains like climate control, lighting, and diagnostics from Electronic Control Units (ECUs), the AAOS SDV platform introduces dynamic runtime service discovery. With this, the SDV can easily discover what services are running (e.g., the odometer, HVAC, sunroof, motorized seats, electric windows, etc.) and their status.

To accelerate development, engineering teams leverage the Android Cuttlefish emulator to build digital twins in the cloud, simulating high-frequency sensor streams to validate these decoupled services bit-for-bit before physical silicon is ready. Valtech Nexus SDV utilizes this AAOS SDV middleware layer to discover, map, and manage vehicle resources, structuring and streaming high-frequency telemetry data straight into Bigtable. Compare this to the prior state of affairs, where OEMs outsourced system software to a variety of suppliers, each with their own pipelines, protocols, and data stored in separate silos.

Crucially, this model decouples services from the heavy main infotainment stack, so they can run independently, even when the vehicle is off and parked. This allows functions like remote vehicle monitoring to remain active even when the primary infotainment system is powered down, ensuring continuous telemetry access without draining the vehicle’s 12V battery or main EV battery pack.

This tight integration between the AAOS SDV platform and Nexus SDV enables a number of agentic AI and innovative first-party solutions. Unlike traditional sandboxed infotainment tools, multimodal AI agents can utilize the service discovery layer to safely interact with the physical car and process complex, intent-based requests. For example, an AI agent could automatically adjust climate zones, window actuators, or interior lighting based on a conversation with the driver, or in response to climate sensors, as in this clip:

By linking this on-vehicle service layer managed by Nexus SDV with historical fleet telemetry stored in Bigtable, you deliver deeply integrated experiences that unlock new mobility solutions. Now let’s take a quick look at the Cloud side.

The Google Cloud side: AI-native mobility

Beyond SDV, we are rapidly moving toward AI-defined vehicles, or AIDV, where AI is core to a vehicle's operational logic. To be AI-native means being autonomous by design, with AI embedded at every architectural level. With this level of AI, the system can perceive environments, reason through complex scenarios using engines like Google Gemini, and proactively execute actions. For example, a Gemini-powered vehicle doesn't just warn you that you’re low on power; it analyzes your schedule, traffic, and charger availability to suggest an optimized charging stop that pre-conditions the battery for maximum efficiency. This is the level of contextual understanding and proactive automation that characterizes AIDV.

Compare this to legacy architectures, which weren’t designed to capture the volume and variety of data coming from different systems across the vehicle. This can lead to data silos of isolated maintenance and safety information telematics. Moreover, because this data is fragmented, it can be very difficult to get cohesive value from the data across systems. An AI-native approach can help collapse these silos, providing a unified contextual understanding. This solves a primary OEM pain point: the massive complexity of managing high-bandwidth telemetry from multiple sources like SDV telematics.

Bigtable: The data backbone for Automotive Telemetry

Bigtable was purpose-built for the massive ingestion rates and sub-millisecond latency requirements, and serves as the data backbone for petabyte-scale automotive and manufacturing telemetry datasets. In fact, Bigtable is already being used to support business critical automotive telemetry solutions. Its flexible, sparse-row schema allows OEMs to evolve their data models without downtime, accommodating diverse sensor arrays — from high-frequency engine metrics to LiDAR point clouds — within a single, unified table structure. Then, by versioning time-series events in a way that is natively optimized for both massive writes and complex, multi-dimensional analytical lookups, Bigtable helps avoid the data overload typical of legacy systems.

Meanwhile, features like Continuous Materialized Views (CMV) allow for pre-calculating key metrics, such as average battery temperature or fleet-wide torque distributions, directly within the storage layer, minimizing computational overhead. Bigtable’s integration with Agent Development Kit (ADK) further bridges the gap between data and action by giving AI agents access to data. This kit combined with Bigtable’s integrations with frameworks like Apache Spark help monitor the "firehose" of live telemetry data and trigger automated workflows in real time, e.g., logging mission-critical alerts, initiating proactive over-the-air (OTA) software adjustments, or pre-ordering replacement parts, the moment specific degradation patterns are detected.

Bring it all together: Nexus-SDV platform

The Nexus SDV platform is built on Google Cloud and integrated with AAOS SDV, supporting the future of connected vehicles. By providing a standardized data foundation, Nexus empowers automotive OEMs to go beyond building infrastructure from scratch and start focusing on unique brand experiences.

Nexus SDV uses Google components like Gemini Enterprise Agent Platform, Bigtable, and BigQuery. Setting up Nexus SDV is quick, automated and transparent. OEMs can create brand-specific customer experiences in the vehicle, as well as in other customer touch points such as the UI screen, mobile app, or service centers. The connection to the vehicle is accomplished by leveraging the open source Synadia NATS interface. This integration with the vehicle is facilitated through simple Cloud and vehicle SDKs, for service discovery on both sides. Nexus SDV is optimized for AAOS SDV, but can integrate with any vehicle framework.

Security is woven into the Nexus architecture via a "Defense-in-Depth" model. Mutual TLS (mTLS) and Google Cloud Certificate Authority Service (CAS) provide vehicles with a cryptographically secure identity. Network isolation is maintained through Private GKE clusters, while the Secure AI Framework (SAIF) helps ensure data privacy throughout the machine learning lifecycle, protecting sensitive user data and OEM intellectual property.

Together, the quick setup and integration time coupled with a standardized data foundation and built-in state-of-the-art security leads to an immediate and measurable business impact for the car manufacturer.

Let’s put it all together and look at a use case in more detail…

Predictive maintenance

By moving from reactive to predictive maintenance, OEMs can reduce warranty costs, improve customer loyalty, and ensure higher vehicle uptime.

The challenge: Traditional scheduled maintenance is often inefficient, leading to unnecessary service visits or unexpected vehicle breakdowns that incur significant costs for both OEMs and owners. By moving to a proactive, AI-driven approach, Nexus SDV, Bigtable, and ADK transform this experience. The process begins by taking the firehose of vehicle telemetry data —monitoring engine RPM, vibration, fluid levels, brake pressure, and more — ingesting it and storing it directly into Bigtable.

To enable real-time anomaly detection, agentic AI can monitor telemetry streams as they arrive. Bigtable CMVs pre-calculate rolling aggregations such as average engine vibration or sudden fluctuations in battery temperature profiles. AI models consuming these live aggregates can then detect subtle deviations from normal parameters, identifying early signs of engine wear or accelerated battery degradation long before a warning light appears on the dashboard.

Once an anomaly is detected by specialized AI models, the system shifts into the agentic reasoning and action phase. A Gemini-powered engine assesses the severity and context of the data, considering factors like mileage, model, make, service history, and upcoming trips. Based on this intelligent assessment, the system can proactively notify the driver via the AAOS infotainment system, suggests an optimized service appointment at a nearby dealership, or can even trigger an automated parts order to ensure everything is ready upon arrival. The AI model works against false negatives to protect customer sentiment or erosion of confidence, while the solution as a whole ensures higher vehicle uptime, transforming maintenance from a reactive burden into a brand-defining service experience.

Get started

The AI-native Nexus SDV platform with AAOS SDV is available today, providing a sophisticated, end-to-end connected vehicle ecosystem designed to meet the extreme scale and analytical rigors of modern mobility. By adopting this unified, open-source architecture, OEMs can transcend the limitations of legacy infrastructure and redirect their resources toward the development of high-impact, brand-defining features.

Nexus SDV takes the connected vehicle service into the agentic era, where vehicles are no longer merely connected, but serve as intelligent, proactive partners in the driving experience. Give it a try today.

Learn more

If you’d like to learn more about Nexus SDV platform, AAOS SDV and Bigtable contact us today at nexus-sdv@google.com.

AAOS SDV is available in the Android Automotive 26Q2 release. Nexus SDV documentation can be found here.

Go here to learn more about Bigtable as the time-series database for automotive telemetry.

Thinking about your connected vehicle security, check this out, Shift into high gear with agents: Securing the software-defined vehicle.

SOCRadar powers rapid threat detection with AlloyDB and Gemini Enterprise

Wed, 01 Jul 2026 19:00:00 +0000

Editor’s note: SOCRadar is a leading cybersecurity company that provides threat intelligence to businesses worldwide. As the volume of cyber threats continued to grow, SOCRadar needed to modernize its data infrastructure to deliver faster insights to its customers. By migrating from PostgreSQL to AlloyDB, SOCRadar achieved a 20x performance boost, reduced operational overhead, and is now better positioned to innovate and grow.

How SOCRadar supercharges rapid threat detection with AlloyDB

SOCRadar provides external threat intelligence to help organizations across 30+ countries defend against cyberattacks. On the front lines of cybersecurity, timely intelligence is everything and a delay of a few minutes can mean the difference between a blocked exploit and a full-scale breach.

As SOCRadar’s business scaled and cyber threat volumes exploded, their on-premises, self-managed PostgreSQL database hit a wall. The database simply couldn't keep pace with the simultaneous demands of high-velocity data ingestion and heavy, real-time analytical queries. This created a severe data bottleneck, slowing down the delivery of critical insights to customers and pulling engineers away from innovation to focus on constant manual database tuning.

Evaluating database alternatives: The hunt for scalability

The engineering team realized their traditional PostgreSQL environment had reached its absolute performance limits. To scale, SOCRadar needed a high-performance fully managed database that could dramatically slash operational overhead while elegantly handling a complex, hybrid workload.

They evaluated alternatives and selected Google Cloud's AlloyDB for PostgreSQL. Because AlloyDB is fully PostgreSQL-compatible, it offered a low-risk migration path while promising a specialized architecture built to handle both high-volume transactions and real-time analytics simultaneously. To accelerate the transition, SOCRadar partnered with NGC, a Premier Business Partner, who meticulously validated the architecture before executing a precision cutover with minimal downtime.

Taming a "triple-threat" workload

Migrating to AlloyDB transformed how SOCRadar processes massive, diverse cyber telemetry. Today, AlloyDB effortlessly manages what SOCRadar’s engineering team calls a "triple-threat" query environment, maintaining sub-second lookup latency even as processing volumes scale.

To understand the performance leaps, it helps to separate the system’s velocity (handling live data streams) from its depth (analyzing historical data):

High-Velocity Transactional Ingestion (OLTP): The platform constantly ingests real-time telemetry from thousands of disparate, fast-moving sources—including Dark Web forums, botnet logs, and social media feeds. AlloyDB handles these continuous INSERT and UPSERT operations with a 3.2x boost in live ingestion velocity, ensuring that the newest threat indicators are immediately recorded and available for detection.
Real-Time Operational Point-Reads: When a security analyst is actively investigating a live incident, speed is everything. Baseline performance testing under zero-load conditions for random ID lookups on indexed fields (e.g., querying a specific Indicator of Compromise by ID) showed that standard queries requiring 3 to 3.5 seconds were completed in just 1 second on AlloyDB.
Deep Analytical Aggregations (OLAP): When a client requests a complex sectoral report such as correlating the most prevalent attack vectors in the finance sector over an entire year, the database must execute deep scans across vast historical datasets. Leveraging AlloyDB’s built-in In-Memory Columnar Engine, these analytical queries run up to 20x faster than standard PostgreSQL.

More than just speed: Reclaiming 45 TB and 75% of DBA time

While the raw performance gains were massive, the operational and financial impact completely changed how SOCRadar's engineering team works day-to-day.

Thanks to AlloyDB's advanced automation, including intelligent memory management and write-ahead log (WAL) optimization, the need for constant, manual database tuning evaporated. The database administrator's (DBA) workload dropped significantly, requiring a system health check just “about once every two or three days." This freed up 75% of SOCRadar’s DBA resources, allowing them to pivot away from maintenance and focus entirely on core platform innovation.

Financially, AlloyDB’s dynamic storage management solved a massive cost efficiency issue. Unlike traditional database environments that lock you into paying for fixed, provisioned storage even after data is purged, AlloyDB automatically scales storage down to match actual data footprints. By clearing out legacy, unnecessary logs, SOCRadar was able to instantly reclaim over 45 TB of storage, achieving massive, automated cost optimization.

Fighting alert fatigue with integrated Gemini Enterprise Agent Platform

Beyond scaling infrastructure, AlloyDB has allowed SOCRadar to redefine the core architecture of their threat response using artificial intelligence.

Security operations centers (SOCs) globally are plagued by "alert fatigue"—the sheer volume of security alarms makes it easy to miss a critical attack. To solve this, SOCRadar integrated Gemini Enterprise Agent Platform as a core component of their solution architecture, linking it directly to their Alarm Management framework running on AlloyDB.

By running Gemini AI-native filtering directly on their active data workloads, SOCRadar can automatically distinguish between true positives and benign false alarms. The AI categorizes, filters, and routes alerts before they ever reach the end-user. This ensures security analysts are insulated from noise and receive only the most critical, validated, and actionable intelligence.

Expanding capabilities: The future of agentic threat hunting

With a high-performance foundation firmly established, SOCRadar’s dedicated AI team is transitioning from passive analytics to active automation. The company is currently testing Agentic AI workloads, with plans to roll them into production in subsequent phases.

By integrating Real-time Data Agents with Gemini Enterprise and AlloyDB, SOCRadar is transforming with autonomous agents that don't just store data, but actively hunt threats, reason over context, and take action. Their upcoming production roadmap includes:

Natural Language Querying (NLQ): Allowing analysts to conduct rapid threat hunting using conversational language, lowering the technical barrier to querying massive database sets.
Intelligent Semantic Similarity Search: Leveraging native vector embeddings and Gemini Enterprise to allow Data Agents to independently surface hidden patterns across historical logs that traditional keyword searches would miss.
Automated Incident Summarization: Instantly transforming hundreds of lines of complex, deeply technical logs into concise, plain-language executive summaries for security analysts during critical incidents.

By consolidating transactional velocity, historical depth, and built-in AI intelligence into a unified platform, SOCRadar has eliminated its data bottlenecks and built a highly automated, future-proof framework for global cybersecurity defense.

Ready to modernize your database infrastructure? AlloyDB provides a fully managed, PostgreSQL-compatible database with high performance for transactional, analytical, and AI workloads. Learn how you can reduce costs, eliminate management overhead, and build intelligent applications.

AlloyDB AI Functions - now with revolutionary performance boosts and cost savings

Wed, 01 Jul 2026 18:00:00 +0000

AlloyDB is an AI-native database—it isn’t just a passive data store, it intelligently understands and processes your data. With AlloyDB, you get industry-leading vector and hybrid search, near 100% accurate natural language-to-SQL capabilities to build conversational agents, tools to enable you to build with your agentic IDEs of choice, and the ability to bring the intelligence of foundation models like Gemini directly to your data through AI functions.

In this blog post, we discuss the massive breakthroughs in AI function processing alongside a suite of brand-new AI functions.

But first: what exactly are AI functions? They bring Gemini’s world knowledge to your AlloyDB data. Consider the challenge of managing raw user feedback: it’s unstructured, and difficult to parse through. Before this data can be leveraged for search, it may require pre-processing and entity extraction. Rather than maintaining complex custom pipelines for knowledge extraction, you can use Gemini’s generation capabilities directly within AlloyDB to transform raw text into structured, searchable insights. For example, here is how you can use ai.generate to instantly turn raw feedback into clean, structured JSON (see more examples here):

code_block: <ListValue: [StructValue([('code', "SELECT\r\n log_id,\r\n raw_content,\r\n -- Use Gemini 3.0 to reason through the raw user feedback and extract structure\r\n ai.generate(\r\n model_id => 'gemini-3.1-pro-preview',\r\n prompt =>\r\n 'Analyze this raw customer feedback entry. Extract the country, service name, and a 1-sentence summary of the feedback. Return as JSON.'\r\n || raw_content) AS structured_feedback\r\nFROM raw_feedback_logs\r\nWHERE user_type <> 'internal';"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef0210fd0>)])]>

Here is a sample result:

log_id	raw_content	structured_analysis
1001	2025-12-16 08:00:01 [ERROR] Service: OrderSvc \| DbConnectionTimeout: Failed to acquire connection from pool "primary-shard-04" after 5000ms.	{"errorCode": "DbConnectionTimeout", "serviceName": "OrderSvc", "rootCause": "The service failed to acquire a database connection from the primary shard pool within the 5000ms timeout limit."}
1002	2025-12-16 08:05:12 [WARN] Service: IdentityProvider \| 401 Unauthorized: Bearer token validation failed for user_id=9942. Signature mismatch.	{ "error_code": "401", "service_name": "IdentityProvider", "root_cause": "The bearer token validation failed due to a signature mismatch." }
1003	2025-12-16 08:12:45 [CRITICAL] Service: AnalyticsEngine \| OutOfMemoryError: Java heap space. Allocation of 1.2GB array failed. Heap usage 99%.	{ "error_code": "OutOfMemoryError", "service_name": "AnalyticsEngine", "root_cause": "The service exhausted available Java heap memory attempting to allocate a 1.2GB array." }
1004	2025-12-16 08:25:33 [ERROR] Service: WebFrontEnd \| 404 NotFound: Resource /api/v3/users/profile/settings not found. Upstream returned 404.	{ "error_code": "404", "service_name": "WebFrontEnd", "root_cause": "The requested API resource for user profile settings was not found by the upstream service." }
1005	2025-12-16 08:35:50 [WARN] Service: NotificationGateway \| GatewayTimeout: External provider "SendGrid" failed to respond within 30s. Retry scheduled.	{"error_code": "GatewayTimeout", "service_name": "NotificationGateway", "root_cause": "The external provider SendGrid failed to respond within the 30-second timeout limit."}

More functions to summarize and analyze sentiment

Our core AI functions —ai.generate, ai.rank, ai.if, and ai.forecast—are now Generally Available. To learn more about use cases for the first three, refer to this blog post. To explore the forecast function in action, check out this deep dive.

Building on this momentum, we have introduced three brand new functions: ai.summarize, ai.agg_summarize, and ai.analyze_sentiment.

ai.analyze_sentiment: Automatically classifies the emotional tone of text as positive, negative, or neutral.
ai.summarize: Condenses lengthy text into its most essential information while preserving the original tone and nuance.
ai.agg_summarize: An aggregate tool that processes multiple rows within a column to generate a single, unified summary for an entire group (e.g., via a GROUP BY clause).

Here’s an example of how to use ai.agg_summarize to consolidate a product reviews for products on a retail website:

code_block: <ListValue: [StructValue([('code', 'SELECT productname, ai.agg_summarize(review) as reviews_summary\r\nGROUP BY productname;'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef0210a90>)])]>

Here is a sample result of summarized reviews for two gaming console products:

productname

reviews_summary

AlphaCore Console

Users praise the stunning 4K graphics, smooth 120Hz frame rates, and the highly ergonomic controller design.

However, several reviews express frustration over the loud cooling fan noise during extended gaming sessions.

Overall, it is considered a top-tier console despite minor thermal and noise complaints.

NeoCore Console

Customers love the exceptional battery life and vibrant OLED display for handheld gaming on the go.

A significant number of users noted that the UI can feel sluggish and the game library is currently limited.

It represents great value for casual gamers but power users may find the performance lacking.

The power of LLMs on your data: now significantly faster and cheaper

We now have achieved unprecedented performance and cost breakthroughs in AI function processing. Previously, running a foundation model call for every single row in a massive database introduced cost and latency constraints. We have shattered these barriers by introducing two breakthrough capabilities:

Smart Batching for AI Functions: This AI Function Acceleration capability provides intelligent batching of AI function calls for optimal performance and quality. This efficiency is achieved by deduplicating prompt overhead; the LLM's boilerplate instructions are transmitted once per batch rather than repeated across every individual row. A question you may have is - “Why not do this in my own application layer?”. That’s because, AlloyDB intelligently determines the right batch size for optimal results - if you underestimate the batch size, you won’t reap gains for cost and latency, and if you overestimate the batch size, the prompt to the LLM could get bloated and lead to hallucinations, or you could exceed the model's token limits. In addition to calculating the perfect batch size for every request, AlloyDB also handles retries automatically out of the box, ensuring your pipeline stays resilient. We did some testing internally and saw massive gains; for example, an up to 2,400x performance boost (processing 10,000 rows/sec) over traditional row-at-a-time LLM calls. This is currently available for the ai.if and ai.rank functions, with support for additional functions coming in the future.

Let’s look at an example of using Smart Batching / Acceleration with ai.if to solve this use case: Imagine a customer on a gadget retail site searching for a camera that can handle an underwater depth of '60 meters or deeper.' Traditional hybrid search will pull the closest semantic and full-text matches, but it misses the hard constraints of numerical data—meaning it might serve up a camera that works only at 20 meters depth. By using AlloyDB’s ai.if-based intelligent filtering, the database actually understands the nuance of depth and makes the query return products that meet or exceed that 60-meter depth criteria. Notice how, in the example below, you don’t need to specify the batch size - AlloyDB handles all the optimizations under the hood when using ai.if.

code_block: <ListValue: [StructValue([('code', "-- Smart Batching / AI Function Acceleration \r\nSET google_ml_integration.enable_ai_function_acceleration = on;\r\nSELECT productid, productname, category,description\r\nFROM products AS p\r\nWHERE\r\n ai.if(\r\n 'Evaluate if the product description indicates that the product is waterproof at depth 60m or deeper. Description:'\r\n || description);"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef0210d90>)])]>

Here is a sample result on a hypothetical gadgets site. Notice how the expanded descriptions of products really match the criteria of working at a depth of 60 meters:

Optimized AI Functions: For even greater efficiency, we’ve introduced an optimized mode, starting with ai.if. By deploying a small, proxy model that utilizes your embeddings and is trained on your specific LLM outputs, we can process decisions natively within the database. This drastically reduces the need to call the external LLM - and based on some of our internal tests, we saw staggering gains; for example, up to 100,000 rows processed per second (a 23,000x improvement) and costs slashed by 6,000x (down to 1/10th of a cent). For technical insights on this technique, including when it works best and when not, refer to this blog post. AlloyDB does the following when using optimized ai.if:

Trains a proxy model: AlloyDB trains a lightweight proxy model on a sample of your data. This happens in the background when you use the PREPARE statement with ai.if function to train the model for optimized queries.
Executes the query: When you use the EXECUTE statement, AlloyDB uses the trained proxy model to process the query locally.
Falls back to the LLM: If the accuracy of the model is low, or if AlloyDB can't find a model, AlloyDB automatically falls back to using the LLM.

Let’s look at the same example of searching for a camera that can handle an underwater depth of 60 meters or deeper using optimized ai.if. Here we train a proxy model using the PREPARE statement and then EXECUTE the statement thereafter.

code_block: <ListValue: [StructValue([('code', "-- Prepare the Optimized Function / Proxy Model\r\nPREPARE waterproof_camera_60m AS\r\nSELECT productid, productname, category, description\r\nFROM products AS p\r\nWHERE\r\n ai.if(\r\n 'Evaluate if the product description indicates that the product is waterproof at depth 60m or deeper. Description:'\r\n || description,\r\n description_embedding);\r\n\r\n-- Run the Proxy Model\r\nEXECUTE waterproof_camera_60m;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef0210220>)])]>

You see the same products that truly match the criteria of working at a depth of 60 meters - as shown in the screenshot above. Here’s a tabulated version for the first three products, so you can look at the descriptions more closely:

productname	description
Pulsetron Action Camera MZ314	Conquer your next adventure with this camera. Don't let the elements hold you back; dive up to 60 meters deep or withstand rugged trails with its shock-resistant, adventure-ready chassis. Every jump, every turn, every splash is rendered flawlessly smooth with advanced Horizon Lock stabilization, ensuring your footage tells the story with unparalleled fluidity.
Hyperbyte Action Camera LG688	Capture the world in breathtaking detail, even when the action is at its most intense. This camera packs a formidable 1-inch sensor into a remarkably tough, pocket-sized frame. Shoot stunning 5K video and crystal-clear 20MP stills that rival professional equipment. Dive deeper than ever before with robust waterproofing at 60 meters.
Alphasync Action Camera WW897	This formidable, compact camera shrugs off the elements, while the massive 1-inch sensor translates every breathtaking moment into stunning 5K video and crystal-clear 20MP stills. Conquer any environment – from the deepest dive to the highest peak – thanks to its 60 meter waterproofing and revolutionary Horizon Lock, ensuring your footage remains impossibly steady.

See it in action!

Watch how this all comes together in this demo video.

Getting started is easy

Ready to bring unprecedented speed and cost-efficiency to your AI workloads?

New to AlloyDB? Discover AlloyDB with a 30-day free trial.
AI functions quickstart: Enable a few quick prerequisites and start calling functions like ai.if, ai.generate, or ai.analyze_sentiment directly within your SQL queries. Check out these practical examples to begin.
Boost performance and optimize costs: To unlock the biggest performance and cost gains, follow our guide on optimized functions. This is available in preview for ai.if, and will be expanding to more functions soon. For technical insights on this technique, including when it works best and when not, refer to this blog post.
Scale your throughput: Use smart batching to accelerate AI functions (available in preview for ai.if and ai.rank) or array-based functions (generally available for all LLM-based AI functions) to handle bulk prompting smoothly.

Modernizing financial services with deployment freedom and transformational AI with AlloyDB Omni

Tue, 30 Jun 2026 16:00:00 +0000

The financial services industry (FSI) operates under a unique set of non-negotiable requirements: the need for strict regulatory compliance, sub-millisecond transactional speeds, and security that verges on impenetrable. Historically, organizations have met these standards by relying on brittle, proprietary database systems, leaving them with massive technical debt, operational overhead, and vendor lock-in.

At the same time, financial services companies are facing a series of daunting challenges:

The licensing trap and technical debt: Decades of reliance on legacy commercial databases have left institutions with skyrocketing maintenance costs and restrictive licenses that refuse to scale. In fact, a global investment bank might find that over 70% of its IT budget is swallowed up by decades-old COBOL core banking systems and siloed ledger databases—leaving virtually no capital to develop the real-time, AI-driven fraud detection tools their clients are actively demanding.
The tug of war between sovereignty and innovation: Emerging regulations like EMEA’s Digital Operational Resilience Act (DORA) and strict national data residency laws require institutions to maintain ironclad control over where their data lives. This often creates a massive barrier to public cloud adoption for sensitive workloads, effectively siloing a regional payment processor from modern AI tools simply because they cannot legally move transaction data to a public cloud for processing.
The "insights gap" in real-time operations: While agile fintech upstarts launch with flexible, cloud-native architectures, traditional firms struggle to turn vast data reserves into actionable intelligence. Their data is trapped in legacy environments that hit a performance ceiling during peak market volatility, leaving an investment firm struggling to scale its high-frequency trading ledgers when standard PostgreSQL or legacy systems max out.

As the industry enters the era of Agentic AI — where autonomous AI agents handle complex workflows like real-time risk assessment and automated trading — financial services firms must adopt a fundamentally new database strategy.

To overcome these entrenched challenges, they need to shift their strategy, moving away from proprietary databases that lock them in toward a hybrid, open-standards-based paradigm. This allows them to embrace the best of cloud-native innovation , like empowering real-time agentic AI workloads and edge computing , while maintaining control and residency of their own data on-premises.

At Google Cloud, we designed AlloyDB for PostgreSQL to unify operational data, real-time analytics, and generative AI into a single platform, and you can run it anywhere. Further, it specifically addresses the above mentioned FSI challenges directly through three guiding principles:

The licensing trap -> open standards: AlloyDB is 100% PostgreSQL-compatible, allowing institutions to modernize from expensive, legacy proprietary databases to an open platform that minimizes licensing headaches and vendor lock-in.
Sovereignty -> heterogeneous support: With AlloyDB Omni’s flexible deployment model, organizations can keep up with the complex topologies that characterize global banks, allowing mission-critical applications to run in a hybrid cloud, at the edge, or on-prem in air-gapped environments.
The insights gap -> battle-tested scale: By incorporating architectural lessons from Google's billion-user applications, the cloud-managed AlloyDB service delivers superior performance, running over 4x faster for transactional workloads than standard PostgreSQL. Crucially, the downloadable AlloyDB Omni engine brings this exact same high-concurrency scaling power straight to your local hardware—outperforming standard PostgreSQL by over 2x for transactions—while both deployment models accelerate real-time analytical queries by up to 100x.

Institutions are already realizing the benefits of this new approach:

Cynergy Bank: By migrating from on-prem SQL databases to AlloyDB, the bank successfully modernized a key element of its infrastructure. This critical initiative reduced app account loading times to under three seconds and enabled the integration of data and AI, providing a more personal "human touch" to digital banking and financial services.
Apex Fintech: The company leveraged AlloyDB to speed up margin calculations by 50%, enabling them to calculate risk for 100,000 accounts in just one minute while eliminating the need for a separate analytical system.

To ensure financial institutions can leverage these exact same breakthrough database innovations anywhere—without being forced into a public cloud migration—we built AlloyDB Omni to extend our signature kernel performance directly to your owned infrastructure.

AlloyDB Omni: Strong performance and deployment freedom

Whether running mission-critical applications on-premises, at the edge, or across hybrid clouds, financial institutions shouldn't have to choose between deployment flexibility and database performance. AlloyDB Omni bridges this gap by bringing Google’s breakthrough kernel innovations directly to your infrastructure. By design, it delivers enterprise-grade capabilities across three core dimensions:

True portability and modernization in place: Take absolute control over your data residency. Deploy AlloyDB Omni on-premises or at the edge to help comply with strict data sovereignty laws and regulations. This allows you to upgrade your legacy estates right where they live, avoiding the immense operational risk, latency, and vendor concentration risks of a forced public cloud migration.
Operational simplicity on your terms: Manage your databases like any other modern application. AlloyDB Omni is deployable across containerized environments, bare metal, or VMs. By leveraging tools like our Kubernetes Operator to automate routine provisioning, backups, and failovers, your platform teams gain integrated, API-driven control that elevates the database into a first-class citizen of your infrastructure alongside compute and storage. For non-containerized setups, Omni can be downloaded as a standalone RPM and managed with CLI or Ansible automation, and it is fully validated to run on Google Distributed Cloud (GDC) for the most restrictive air-gapped workloads.
Shattering the PostgreSQL performance ceiling: While standard PostgreSQL is highly trusted, high-concurrency financial workloads often hit a scaling wall. AlloyDB Omni breaks through these limits directly on your local hardware:

Superior transactional scalability: Delivers up to 2x faster transaction processing than standard PostgreSQL, ensuring payment processing and high-frequency trading ledgers maintain ultra-low latency even during volatile operational spikes.
Real-time analytics (HTAP): An intelligent, built-in columnar engine accelerates analytical queries by up to 100x. This enables instant, local business intelligence and reporting directly on live transactional data without the latency of moving it to a warehouse.
Secure, local AI transformation: Build fraud detection, risk modeling, or semantic search applications locally. AlloyDB Omni includes integrated AlloyDB AI vector capabilities—featuring a ScaNN index that is up to 10x faster and 4x more memory efficient than standard PostgreSQL's HNSW index. This allows you to scale generative AI apps while keeping sensitive financial data and foundation models strictly within your secured infrastructure boundaries.

Enterprise-grade security and compliance

Security cannot be an afterthought. We built AlloyDB Omni to exceed the rigorous standards of the finance industry, offering a hardened posture out of the box. AlloyDB includes:

Granular access and auditing: AlloyDB Omni integrates with Active Directory for unified identity management and provides detailed audit logging to track every access event — essential for regulatory audits.
Compliance-ready infrastructure: By utilizing features like Transparent Data Encryption (TDE) at rest, AlloyDB Omni is specifically engineered to help you meet your regulatory compliance obligations.

By providing a platform that is secure by design and that can be flexibly deployed in a variety of configurations, AlloyDB Omni enables financial institutions to stop choosing between stability and innovation and start delivering both.

Next steps

To learn more and get started, please visit https://cloud.google.com/alloydb/omni. You can learn more from the AlloyDB Omni documentation.

AlloyDB Omni is covered by the Google Cloud support plan the customer has chosen for their Google Cloud account; more information on support can be found at https://cloud.google.com/support.

Technology partners, system integrators and ISVs play an important role in helping customers modernize and build differentiated applications., We are extending the AlloyDB Cloud Ready program to now include AlloyDB Omni and enable our partner ecosystem to bring the best of what AlloyDB Omni has to offer to their customers. Customers can trust these validated partner products to work well with AlloyDB Omni, and can focus their time on modernizing database workloads and applications that will drive value for their business.

Get started with AlloyDB Omni by downloading and deploying in your preferred location, including on your laptop!

Supercharging the agentic era with Spanner’s multi-model architecture

Mon, 29 Jun 2026 23:00:00 +0000

In the agentic era, the role of the database has fundamentally changed. It is no longer a passive repository; it’s a critical context engine designed to ground generative AI apps, models and power autonomous workflows. To do this effectively, databases must move beyond fragmented architectures and embrace a unified, multi-model foundation, facilitating deep reasoning and transforming static data into a system of action.

Spanner is leading this charge, and as a foundational pillar of Google’s Agentic Data Cloud, the industry is taking notice. In the 2025 Gartner® Critical Capabilities for Operational Cloud Database Management Systems report, Google (Spanner) ranked #1 in the Lightweight Transactions Use Case for the second consecutive year — in our opinion proving it is the most efficient engine for modern microservices and event-driven architectures.

Gartner® Operational Cloud DBMS use cases:

#1 in Lightweight Transactions
4.9 / 5.0 for Transactional Consistency
4.6 / 5.0 for AI/Machine Learning and GenAI

This technical momentum, which also recently earned Spanner the prestigious SIGMOD Systems Award, is matched by undeniable economic value. A recent Forrester Consulting Total Economic Impact™ (TEI) study commissioned by Google Cloud found that an organization (based on composite customer profile from Forrester’s survey) realized a 132% ROI with a fast 9-month payback period, yielding $7.74M in total benefits over three years having deployed Spanner.

Real-world impact

To see the real-world impact of this vision, look no further than MakeMyTrip, India's leading online travel company. Their story perfectly illustrates a massive industry-wide paradigm shift: the move away from managing a complex sprawl of single-purpose, specialized databases towards a unified, multi-model architecture, with Spanner leading this evolution.

Historically, powering MakeMyTrip’s AI-driven travel features required a fragmented patchwork of four separate database engines: MongoDB (document), Neo4j (graph), Elasticsearch (text search), and Qdrant (vector embeddings). Managing this sprawl meant maintaining custom synchronization pipelines just to keep data in lockstep, resulting in high operational overhead and the constant threat of data drift.

By consolidating all four specialized workloads into a single Spanner instance, MakeMyTrip reduced operational complexity by 75%. Because Spanner natively executes unified lexical, keyword, and embedding searches across a single data layer, MakeMyTrip eliminated synchronization friction, improving the response-quality score of their gen AI "Destination Expert" bot by 9%.

This is the true multi-model advantage: Spanner's unique strength is reflected in how you can combine these different modalities in a single SQL statement to get the most accurate search results with strong consistency of the data. You can now eliminate database silos with a single, highly coherent foundation engineered for the agentic era.

The multi-model advantage for the agentic era

True AI autonomy requires deep context. To reason effectively, an AI agent cannot look at data through a single lens; it must simultaneously understand structured history (relational), semantic meaning (vectors), real-world connections (graphs), and textual details (full-text search).

Spanner natively breaks down these multi-model barriers. Instead of forcing you to stitch together disparate engines, Spanner unifies relational, vector, graph, key-value, and full-text search data directly within a single, highly performant database architecture. This architectural integration allows AI models to leverage situational, semantic, and relationship context instantly and concurrently.

Spanner’s fully interoperable multi-model capabilities allow organizations to build intelligent applications without compromise:

Spanner Graph: A unified graph and relational experience built on the ISO-standard GQL. You can model data natively as a graph or as an overlay on top of relational data, which is critical for building knowledge graphs that ground AI agents in real-world facts. Customers like Palo Alto Networks leverage Spanner Graph to power crucial access-control use cases at planet-scale, securing their AI infrastructure without needing a specialized, siloed graph database.
Integrated vector search: A fully integrated semantic search solution offering both K-Nearest Neighbors (KNN) and Approximate Nearest Neighbor (ANN) search, capable of supporting indexes with over 10 billion vectors for fast, low-latency retrieval-augmented generation (RAG).
Relational and key-value: Spanner pioneered the relational scale-out database (Google SQL and PostgreSQL). We've also introduced high-performance key-value capabilities via a Cassandra-native endpoint, allowing for easy lift-and-shift of Cassandra workloads.
Full-text search: Building on Google's decades of search expertise, Spanner provides advanced information retrieval across structured and unstructured data, including an enhance_query option for automatic synonym matching and spell correction. Streaming legal intelligence platform Inspira simplified a 4.5 TB data pipeline into a unified, high-performance single-source of truth. Leveraging Spanner’s native support for FTS and vector search capabilities Inspira achieved high-precision snippets for LLM-based legal analysis with RAG workflow.
Spanner columnar engine: This architectural breakthrough enables analytical queries to run up to 200× faster on live operational data, bridging the gap between OLTP and analytics to provide agents with real-time context without the "ETL tax." AI-powered fraud prevention platform Verisoul uses the columnar engine to run rich analytics on high-velocity transactional writes in one place, eliminating data copies and replication lag to get near-instant answers.

True interoperability means these aren't just isolated features , they are tightly integrated. Instead of writing complex application logic and brittle ETL pipelines to stitch together a graph database, a vector database, and a search engine, developers can query relationships, semantic meaning, and keywords in a single, ACID-compliant SQL statement.

Here’s an example of how a developer can combine relational, graph traversal, full-text search, and vector similarity search in one cohesive query to power an intelligent product recommendation agent:

Spanner Omni: Multi-model capabilities, everywhere

To truly be the unified data foundation for the Agentic Data Cloud, a database cannot be confined by infrastructure borders. That’s why we expanded our vision with Spanner Omni, bringing these multi-model capabilities to any environment without hardware restrictions, just as we did with AlloyDB Omni.

Spanner Omni is a downloadable version of Spanner in a fully containerized deployment model that requires absolutely zero dedicated hardware. It is designed with maximum flexibility in mind, running natively on Kubernetes using the infrastructure you already own. Whether your workloads are running on-prem, at the edge, or across other major public clouds like AWS and Azure, Spanner Omni gives you control and helps ensure you have a consistent, globally distributed data foundation.

This means organizations can leverage Spanner Graph, vector search, full text search, and our columnar engine anywhere, effectively breaking down cloud silos and making these cutting-edge capabilities available without vendor lock-in.

Industry-defining capabilities for core databases

In the 2025 Gartner® Critical Capabilities for Cloud Database Management Systems for Operational Use Cases, for the second consecutive year, Gartner ranked Google (Spanner) #1 in the Lightweight Transactions Use Case. We believe this a testament to its efficiency and low latency for modern, event-driven microservices.

In our opinion, this industry recognition goes far beyond simple market presence, it is validated by deep foundational technical breakthroughs that separate Spanner from legacy architectures. Unlike platforms that bolt disparate, siloed database engines together and label it as "multi-model," or require users to select the modality at the time of database creation with no interoperability between modalities, Spanner’s capabilities are built on a bedrock of Google’s most advanced computer science:

TrueTime and Paxos for global consistency: Spanner’s distributed transactions are governed by TrueTime — a highly available, globally synchronized clock system utilizing GPS and atomic clocks. This enables lock-free distributed reads and strict external consistency globally. Combined with highly optimized Paxos consensus, Spanner delivers synchronous replication with zero data loss (Recovery Point Objective, i.e. RPO=0) and rapid recovery timelines (Recovery Time Objective, i.e. RTO=0) even during total regional failures.
Integrated columnar engine: To eliminate the ETL tax and bridge the gap between OLTP and OLAP, we integrated a breakthrough columnar engine directly into Spanner's distributed storage layer (Colossus). This allows developers to run complex analytical queries to run up to 200x faster directly on live, operational data without impacting transactional performance. And with full separation of storage and compute, users are able to run large analytical queries without impacting the operational workload using Spanner DataBoost, a serverless technology that directly accesses the database storage.
ScaNN-powered vector search: Our native vector search isn't a bolted-on afterthought. It’s powered by Scalable Nearest Neighbors (ScaNN) — the exact same state-of-the-art indexing algorithm that powers Google Search and YouTube. This allows Spanner to execute sub-millisecond similarity searches across 10-billion-plus vector indexes natively alongside relational and graph data.
Dynamic resharding: Under the hood, Spanner's architecture automatically reshards data based on size and load. This transparent load balancing eliminates the dreaded "hotspotting" that plagues legacy NoSQL and distributed SQL systems.

While some industry evaluations often measure the market through a fragmented lens of disconnected database engines, we believe true innovation requires engineering for this level of deep, architectural integrations. For the agentic era, anything other than a natively unified foundation is simply a bottleneck.

A unified vision for the agentic era

We believe that the future of data is unified, open, and inseparable from AI. Spanner’s momentum reflects a market rapidly shifting away from a patchwork of isolated databases towards a singular, intelligent context hub.

To meet this future head-on, we are relentlessly expanding what is possible with a single unified database. This includes breakthrough innovations like our integrated columnar engine for real-time analytics, native vector search powered by Google's world-class ScaNN technology, and built-in AI functions that bring model inference directly to your data. Furthermore, by integrating Spanner Graph integrated with Graph Neural Networks (GNNs) for deep predictive reasoning, and Spanner Omni to extend this unified architecture across hybrid and multi-cloud environments, we are delivering a platform designed for what comes next.

Crucially, Spanner does not exist in isolation; it is a foundational pillar of Google’s broader Agentic Data Cloud. Through seamless, zero-ETL integrations across our Data Cloud Including BigQuery for enterprise-wide analytics and Gemini Enterprise Agent Platform for advanced model orchestration, Spanner breaks down the barriers between operational data and enterprise intelligence.

In the agentic era, AI models require more than just isolated data points; they need a cohesive ecosystem. By natively federating real-time operational context from Spanner with petabyte-scale historical insights from BigQuery, we empower agents to act autonomously, reason deeply, and drive unprecedented business value.

By providing a real-time, trustworthy, and multi-faceted view of data, regardless of where it lives, Spanner empowers organizations to build the next wave of transformative, intelligent applications.

We are incredibly excited about the journey ahead and will continue to pioneer the frontiers of what a true multi-model database can achieve.

Try Spanner for free for 90-days or for as little as $65 USD/month for a production-ready instance that grows with your business without downtime or disruptive re-architecture.

^{Critical Capabilities for Cloud Database Management Systems for Operational Use Cases, By Ramke Ramakrishnan, Masud Miraz, Xingyu Gu, Henry Cook, Aaron Rosenbaum, November 19, 2025.}

^{GARTNER and MAGIC QUADRANT are registered trademarks and service marks of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. All rights reserved.}

^{Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.}

How Atlas scales hundreds of merchant databases with Cloud SQL Enterprise Plus edition

Tue, 16 Jun 2026 16:00:00 +0000

Atlas is building the operating system for restaurants. Online storefronts, point of sale, third-party logistics, food platform integrations, customer loyalty, and AI tools represent everything a restaurant needs to start, run, and grow. We work with brands like SaladStop, Killiney, Haidilao, Raffles Hotel, Lo and Behold Group and the Les Amis Group in Singapore, helping merchants increase basket sizes, grow sales, and reduce operational costs.

Every merchant on Atlas gets their own dedicated Cloud SQL for PostgreSQL database. Restaurants are very different from each other. A single-outlet cafe and a multi-outlet chain should not look the same underneath. Isolated databases give us full data separation, predictable performance even during peak lunch and dinner rushes, and the flexibility to scale, tune, or migrate each merchant independently. As Atlas grows, the number of databases grows with us.

The challenge: Scaling beyond standard

We started on the standard Cloud SQL Enterprise edition. It was a solid foundation, but as we onboarded more merchants and shipped more features, the operational layer required to manage our databases became a bottleneck.

We were managing connection pooling as a separate layer, which meant more services to run, secure, and monitor. When a query caused a CPU spike, we needed to know exactly what happened and which merchant triggered it, but we were spending too much time reconstructing problems from limited signals. With a lean team and no dedicated database engineers, every extra component multiplied the maintenance load.

The shift to Enterprise Plus edition

When we needed to provision new database instances, the Google Cloud team introduced us to Cloud SQL Enterprise Plus edition. We were already asking ourselves how much more operational overhead this was going to add, and what stood out was that Enterprise Plus edition removed whole categories of work we would otherwise have to own.

Managed connection pooling: Now built directly into Cloud SQL, we no longer run pooling as a separate layer. This means fewer moving parts, less to maintain, and a smaller security surface area.
Query insights: This was the most impactful feature for our needs. We can now see exactly which queries are expensive and which merchant is triggering them. It turns performance tuning from guesswork into something concrete and actionable. For a platform running hundreds of databases, this visibility is a "superpower."
Data cache: This keeps read performance consistent even as merchant datasets grow. Since restaurants generate more data every day, the data layer needs to stay fast as that complexity compounds.
Near-zero downtime scaling: We can now scale instances as merchants grow without disrupting service during off-peak hours.

After seeing the results on the new instance, we migrated all our existing databases to Enterprise Plus edition as well.

The impact: Focus on innovation, not plumbing

Atlas today powers thousands of restaurant outlets, processes tens of thousands orders daily using hundreds of managed databases. The biggest change is where engineering time goes. We spend 30% less time on database operations and more time building products. Merchant onboarding got simpler because a new merchant is provisioned in seconds with a ready-to-use managed database. We are much more proactive on performance now, catching and fixing issues before they reach merchants. Day to day, we are not thinking about database plumbing. We are thinking about how to serve merchants better and that has allowed Atlas to grow 200% to 300% year over year.

Looking ahead: An AI-first future

We are investing deeply in AI, both internally and externally. Internally, we have gone all in on agentic engineering through AI-assisted development workflows that let a lean team build, review, and ship code significantly faster. Externally, we are building AI-powered tools that help restaurant operators make better decisions and act on them. We have a lot of experimental ideas on the roadmap, including new product surfaces and new ways to help restaurants grow. The thing that gives us confidence to move fast on all of this is that the foundational layer, Cloud SQL and Google Kubernetes Engine (GKE), is battle-tested and does not get in the way.

Google Cloud handles the infrastructure complexity. Atlas stays focused on building the best tools for restaurants.

Cloud SQL Enterprise Plus gave us a database architecture that is flexible, observable, and easy to scale. We are not thinking about infrastructure anymore, we are thinking about our merchants. As we go deeper on AI and continue growing the platform, Google Cloud gives us the confidence to move fast without worrying about what is underneath.

Ready to scale your database architecture?

Don't let infrastructure bottlenecks slow down your innovation. Whether you are managing tens or hundreds of databases, see how Google Cloud SQL can streamline your operations, enhance observability, and give your engineering team the freedom to focus on what matters most.

Explore Cloud SQL Enterprise Plus edition today
Sign up to try Cloud SQL for free.

What’s new in data agents: Supercharging your AI workflows

Mon, 15 Jun 2026 17:00:00 +0000

The rise of AI agents is fundamentally disrupting applications and analytical systems. Generic AI platforms don't usually have access to the context stored within enterprise databases. This is because traditional data architectures often lack context for agents across the data estate, which can lead to agents being inaccurate. They’re also prone to security gaps due to a lack of granular access controls.

Google’s Agentic Data Cloud is an AI-native system of action that includes both operational and analytical systems. By infusing AI across the entire stack — from custom silicon to frontier Gemini models — we provide a deterministic, template-driven developer framework that allows agents to ground their reasoning in real-time enterprise data with near-100% accuracy, as well as unified governance.

Today, we’re making it easier to develop agents, with a whole host of new data agents and tools: for business analysts within Conversational Analytics; for data scientists, engineers, and database admins with a series of Google-built Data Agents that provide greater automation and intelligence; and finally, for developers, with Data Agent tools that help you better integrate with today’s open agentic ecosystem.

1. Conversational Analytics

To support developers building agents using natural language, we’re announcing expanded support for Conversational Analytics across Data Cloud.

Conversational Analytics in BigQuery integrates a sophisticated AI reasoning engine directly into BigQuery Studio, helping data and business teams go beyond writing manual SQL, leveraging business context to ground answers using multimodal synthesis and deep-dive research. Agentic workflows, in preview for select customers, automate root-cause analysis, and schedule actions — turning enterprise data into proactive, actionable intelligence.

Create agents for faster data insights with Conversational Analytics in BigQuery

Conversational Analytics in Lakehouse, now in preview, extends the Lakehouse unified infrastructure, so users can query distributed data lakes across AWS, Azure, and Google Cloud using natural language. This makes it possible to combine insights across cloud platforms without moving a single byte of data.

Conversational Analytics in AlloyDB, Spanner, and Cloud SQL, now in preview, supports out-of-the-box conversational AI, making data accessible for everyone. AlloyDB, Spanner, and Cloud SQL users can start natural-language conversations with their databases to gain visibility into their real-time operational data and capture analytical insights.

Use Conversational Analytics to get answers from your operational data

Looker Embedded Conversational Analytics, now generally available, allows you to embed agents directly into your custom applications and internal workflows via a low-code iframe implementation, making it easier to ship production-ready, conversational AI within any application. Additionally, with the Conversational Analytics API in Looker, you can create multi-turn conversational workflows that offer AI-powered recommendations, while also verifying and explaining the underlying SQL query. We are also significantly upgrading Looker’s core Conversational Analytics agent, which is already GA, with superior reasoning and semantic grounding, helping to eliminate ambiguity.

Embed agents directly into your applications for conversational AI

2. New data agents

To help data professionals move from reactive data management to proactive intelligence, and business analysts better interact with their dashboards, we’re announcing a new set of data agents that bring automation, intelligence, and natural language capabilities into their daily workflows.

Data Engineering Agent, now generally available, automates the heavy lifting of building and maintaining data pipelines. It transforms natural language requirements into optimized SQL or Python code for BigQuery and Dataflow, while proactively identifying and fixing pipeline breaks. By suggesting schema improvements and partitioning strategies, it ensures your data foundation is scalable, reliable, and performance-tuned without manual trial and error.

Data Science Agent accelerates the path from raw data to production-ready models. It assists data scientists by suggesting relevant features, generating boilerplate notebook code, and automating the technical documentation process.

Database Observability Agent, in preview with select Cloud SQL, AlloyDB, Spanner, and Bigtable customers, proactively monitors database performance and continuously identifies potential issues before they escalate. It then delivers intelligent recommendations and multi-turn remediation workflows for fast, comprehensive troubleshooting and optimization. It provides performance analytics for the entire database fleet, helping you quickly identify performance optimization opportunities across databases.
Database Onboarding Agent, in preview with select customers, takes the guesswork out of database selection and deployment. By evaluating your stated requirements — from simple use case descriptions, to complex enterprise needs — it recommends the best Google Cloud database and guides you through provisioning.
Looker Dashboard Agent, now in preview, enables conversational interaction with data within dashboards. Users can ask natural language questions and receive context-aware answers within the dashboard. This feature also provides AI-generated summaries that highlight key takeaways and insights from the dashboard.
Conversational Analytics in Gemini Enterprise, now in preview for Looker, BigQuery, and Lakehouse, brings governed intelligence built by data practitioners directly to business leaders. It serves as a "front door" to the Google Data Cloud, allowing business users to consume agents built in BigQuery, Looker, or Lakehouse without needing to access technical consoles. By publishing these agents from Google Data to Gemini Enterprise, organizations provide a single, grounded interface for precision data exploration and immediate answers to the business users.

Deep Research Agent, now in preview, uses the Knowledge Catalog to solve high-stakes, multi-layered business problems. It moves beyond simple search to build comprehensive research plans that synthesize intelligence from internal documents, BigQuery tables, and the public web. The result is a detailed report with dynamic visualizations and verifiable citations, that respect enterprise privacy and user permissions all the while.

3. Tools for data agents

Open-source standards for agentic development provide developers building AI applications and custom agents with a unified framework to access data and tools consistently and securely. Today, we are announcing the following tools to help ground your agentic development initiatives:

Data Agent Kit: now in preview, provides a standardized suite of skills and tools directly within preferred developer environments (IDE/CLI), empowering data practitioners to discover, transform, and action data at scale using the prescriptive guidance from the Agentic Data Cloud capabilities.
Managed MCP Servers for Databases, now generally available for AlloyDB, Spanner, Cloud SQL, Bigtable, and Firestore, fully manages the infrastructure required to connect AI models securely to your data, so you don’t have to host, secure, or scale MCP servers yourself. Now, developers can provide their agents with up-to-date context from across our database portfolio, so that your AI models can reason and act upon your most up-to-date enterprise data.
Managed MCP Server for Looker, now in preview, allows any MCP client or agent platform to query Looker's semantic models, extending governed BI insights across third-party applications.

Access Looker semantic models through Managed MCP Server

MCP Toolbox for Databases 1.0, now generally available, has achieved a major stability milestone, giving you the confidence to build production applications. We also overhauled the documentation, making the platform significantly more approachable for both human developers and autonomous agents.
QueryData for Cloud SQL, AlloyDB, and Spanner, now in preview, turns natural language questions into database queries. It’s built natively into these databases, and provides near-100% accuracy for natural language to SQL conversions through metadata, query examples, and evals.
Universal Commerce Protocol (UCP) Analytics powered by BigQuery, now in preview, enables merchants and developers to stream real-time events from UCP directly into BigQuery (see sample). This integration provides out-of-the-box observability for agentic commerce, allowing teams to monitor conversion funnels, track automated checkout performance, and identify system errors. By standardizing these metrics within BigQuery, businesses can bridge the gap between AI-driven transactions and existing business intelligence workflows.

Details on how to access the new agents and tools can be found from each of the documentation links on this page. Data agents are also available through Gemini Enterprise and the Google Cloud console.

Architecting a trusted agentic platform with graph technologies: A Yahoo case study

Mon, 15 Jun 2026 16:00:00 +0000

As enterprises adopt agentic AI, they need to shift from reactive systems of intelligence to proactive systems of action to equip the agents they’re building with the context and performance they need, plus regulator-grade accountability, where every decision is explainable and auditable.

At Google Cloud Next ‘26, we discuss how our Agentic Data Cloud enables a system of action, and Yahoo’s digital media buying platform is a compelling example of this vision. Yahoo partnered with Google Cloud to build its Seller Agent digital media buying platform using Google Data Cloud graph technologies. Seller Agent condenses multi-week manual processes into fully governed, live campaigns that can be executed in just seconds. Ultimately, this agentic platform serves as a powerful blueprint for multiple industries, demonstrating that autonomous systems can operate at remarkable speed while remaining strictly accountable.

"Yahoo's mission is to be a trusted guide through the digital world. In partnership with Google Cloud, we're extending that promise to advertisers: agentic media buying that's fast, transparent, effective, and built to be trusted." - Gabriel DeWitt, Head of Monetization, Yahoo

In this blog, we explore the shift toward agentic AI, examine how Yahoo’s Seller Agent architecture solves for speed and trust in media buying, and show you how to apply this graph-based pattern to build trusted systems of action in your own organization.

Case study: agentic media buying

For years, complex, high-value workflows—like premium digital advertising campaigns—have required weeks of human handoffs, fragmented spreadsheets, and manual analysis. Yahoo recognized that agentic AI could collapse this timeline, allowing agents to plan and execute campaigns in mere seconds. This leap from manual to autonomous execution represents a massive opportunity to reclaim operational efficiency and ensure more of every dollar reaches measurable outcomes.

But simply dropping LLMs into a high-stakes workflow does not solve the problem; an agent attempting to negotiate contracts or ad placements without a deterministic understanding of real-time inventory, pricing rules, and business constraints is prone to hallucinate — potentially resulting in disastrous deals. A trusted agentic platform requires a definitive, real-time source of truth, ensuring it acts on hard facts rather than statistical guesses.

Furthermore, speed and factual grounding are only half the equation. The moment an AI agent starts moving real budgets, it faces scrutiny from regulators who demand instant answers to why specific decisions were made or which policies were applied. Digging through raw system logs after the fact is the wrong control surface for autonomous execution. Real-world systems of action require regulator-grade governance and auditability built directly into the workflow, not bolted on as an afterthought.

The architecture of a trusted system of action

Yahoo's mission has always been to be a trusted guide through the digital world. Agentic media buying extends that promise to advertisers, agencies, publishers, and regulators who entrust Yahoo with their budgets — and expect real accountability. The issue was automating campaign execution in a way that was explainable, governable, and auditable.

To meet this challenge, Yahoo built its Seller Agent as a multi-agent system running on Google Cloud. Buyer requests enter through a planning supervisor agent running on Google Kubernetes Engine (GKE) and orchestrated with Google's Agent Development Kit (ADK). The supervisor decomposes each request into specialized tasks including inventory discovery, audience matching, forecasting, pricing analysis, package recommendation, governance review, and execution. Agents coordinate through the open Agent2Agent (A2A) protocol, while Gemini Enterprise Agent Platform hosts models for embeddings, forecasting, and graph learnings.

But the true breakthrough — what makes autonomous execution both fast and fully transparent — is the platform’s dual-graph foundation. The platform is anchored by two specialized graph systems with an intentional separation of duties: a knowledge graph that’s optimized for acting, and a second context graph for remembering and learning.

"As the industry moves from systems of intelligence to systems of action, the constraint on autonomous AI shifts from model capability to whether a business can trust what an agent does unsupervised. Autonomous systems must record why decisions were made and learn from outcomes. That trust is earned through robust data infrastructure. We built that foundation with Google Data Cloud: a knowledge graph for operational truth in Spanner Graph, a context graph for decision lineage in BigQuery Graph — the blueprint for enterprise-scale agentic platforms." - Swapnil Patel, Senior Director and Head of Monetization Engineering, Yahoo

The knowledge graph: Grounding agents in business reality

Powered by Spanner Graph, Yahoo’s knowledge graph represents its monetization business as a connected operational model, grounding every agent decision in business reality. It models advertising products, placements, audience segments, inventory, contracts, and governance controls as first-class entities and relationships. Crucially, policies live directly within the graph as versioned relationships rather than being buried in application logic. This design allows the system to evaluate products, contractual obligations, consent requirements, and regulatory constraints together in a single, unified graph traversal.

The graph acts as a semantic contract across the agentic platform. During campaign evaluation, an agent can navigate from initial buyer requirements to eligible audiences and governing policies within a single query plan. Gemini Enterprise Agent Platform embeddings enrich these entities with semantic similarity, while graph neural networks contribute inferred relationships. Ultimately, this allows agents to do more than just retrieve available inventory — they understand exactly why it is relevant and help ensure it satisfies all governing constraints.

Yahoo’s knowledge graph ontology, aligned with industry standards like IAB AdCOM

The context graph: creating an auditable memory

Execution at agent-scale is only safe if it is entirely transparent — which is the core function of the context graph. Every time the Seller Agent takes an action, that exact operational span is captured by the BigQuery Agent Analytics plugin. In addition to logging the raw events, the system shapes this evidence into a typed, queryable context graph using BigQuery Agent Analytics SDK utilizing Yahoo's decision-trace ontology, stored in BigQuery Graph.

Consequently, every decision point, candidate package, policy evaluation, specialist-agent delegation, and execution outcome becomes a connected graph of evidence. Because this trace is structured as a typed graph, explaining the agent’s decision making process becomes a simple query. An auditor can instantly trace a decision from the originating campaign brief through every score that’s assigned and policy that’s applied. This transforms autonomous behavior from an opaque process into a fully transparent and continuously improving record of decision-making, helping to ensure absolute accountability.

Yahoo’s context graph ontology

From human to agent scale

For a concrete example of the architecture in action, consider an ad campaign run. What traditionally required weeks of coordination across planning, sales, operations, and compliance can now be completed in seconds through two simultaneous processes.

Acting via the knowledge graph. This pipeline moves the budget, navigating linearly from the buyer's request to a live campaign ground on the knowledge graph. This proceeds in four steps:

Submitting the brief: A buyer agent submits a campaign brief over Ad Context Protocol (AdCP) that describes the desired audience, budget, geography, and business objective.
Knowledge retrieval: The Seller Agent queries the knowledge graph to identify relevant inventory, audiences, contractual availability, historical performance, and governing policies.
Evaluation and scoring: The agent evaluates these factors together to assemble a package of media buying candidates. Forecasting models score the opportunities, while a governance agent independently reviews consent, brand safety, and regulatory constraints.
Approval and execution: The package is either approved automatically under policy thresholds or escalated for human review. Once approved, the media buy is executed and activated.

Auditing and learning via the context graph. While the execution pipeline moves forward, this parallel loop continuously captures the system's reasoning in the context graph, helping to ensure transparency and improve future cycles. This offers the following capabilities:

Continuous capture: Every candidate considered, score assigned, policy applied, and governance decision becomes a connected record in the context graph, linked to the originating campaign session.
Closed-loop learning: As delivery, attribution, and outcome signals arrive, they are joined back to the decisions that produced them, creating the training data that improves future recommendations.
Instant explainability: If an advertiser asks why a particular package was selected or which policies influenced the outcome, the answer is preserved in the context graph and reachable through a single query.

The result is a platform where knowledge, decision-making, governance, measurement, and learning operate together — allowing autonomous media buying to remain explainable, auditable, and continuously improving.

A blueprint for many industries

The era of AI as a mere advisor is ending. Enterprises are demanding systems of action — autonomous agents capable of executing complex, multi-step workflows. But in regulated sectors, the speed that AI brings to the table turns into a liability if you cannot prove how a decision was made. The primary barrier to autonomous execution is no longer intelligence; it is trust.

The architecture that Yahoo and Google Cloud built provides a broadly applicable blueprint with which to solve this. While designed to fix the bottlenecks of digital media buying, the underlying pattern applies to any industry managing high-stakes decisions — from financial trading to supply chain logistics. To operate at agent speed but still maintain human oversight, enterprises must adopt a new architectural baseline that:

Grounds decisions in business reality: Agents cannot rely on probabilistic models alone. They must be grounded by a knowledge graph that deterministically maps your business logic, active contracts, and compliance rules.
Builds an auditable memory: You cannot govern what you cannot trace. Every agentic action must be captured in a context graph, creating an immutable, queryable record of exactly why a decision was made and which alternatives were rejected.
Embraces open interoperability: Trust requires transparency. By building on open protocols and provenance standards, industries can establish a common, auditable language for agentic behavior.

As foundational models become commoditized, enterprises’ competitive advantages are shifting. Long term, your moat will not be the language model you deploy, but the proprietary graph of your business operations and governed history. Likewise, the future of enterprise AI isn’t simply systems that can act, but systems that can explain, govern, and take accountability for those actions.

Get started today

Ready to build your own trusted system of action? Start by exploring Spanner Graph to ground your agentic workflows in business reality. Next, use BigQuery Graph to build an auditable memory that powers closed-loop learning and regulator-grade explainability. You can begin capturing and analyzing these operational traces today using the BigQuery Agent Analytics Plugin and SDK. Finally, review the Ad Context Protocol to understand the open communication standards underpinning Yahoo’s agentic platform.

What’s new with Google Data Cloud

Wed, 10 Jun 2026 16:00:00 +0000

July 6 - July 10

New Lakehouse managed tables now in preview
Lakehouse tables for Apache Iceberg are now in preview and available in the console. By using Google-managed Apache Iceberg tables in Lakehouse, you can eliminate the overhead of maintaining duplicate data pipelines and complex synchronization logic between BigQuery and open-source engines. This unified table format delivers native, multi-engine read and write interoperability, allowing you to run concurrent DML/DDL operations across diverse analytics tools on a single, shared storage layer. Built-in automated table management handles painful background optimization tasks like compaction and partition tuning, freeing up your team to focus on building rather than managing storage maintenance.

June 1 - June 5

Beyond the Query: Powering AI Agents with Bigtable, Firestore & Memorystore
Discover the latest advancements in Google Cloud's NoSQL Database portfolio, including Bigtable, Firestore, and Memorystore. This series is designed for a broad audience: whether you are exploring these databases for the first time or are an existing user looking to leverage the new capabilities announced at Next '26.

Register here to secure your spot!

Cloud Engineer's AI Toolkit Workshops: Solve data-driven challenges with BigQuery, AlloyDB, Gemini and more. Hosted by Google Cloud Labs, this highly technical event is built specifically for Platform Engineers, SREs, and cloud infrastructure teams ready to bridge the gap between AI prototypes and production-grade deployments. Look out for more locations coming soon

Toronto - June 25 (Data Cloud) | RSVP Here
Chicago - June 30 (Data Cloud) | RSVP Here
Start a 10-day Bigtable free trial with a 1 node SSD cluster and up to 500GB of storage capacity. With no credit card required to start, you can easily ingest workloads and manage workloads that require low-latency, high-throughput, and predictable access. Plus, new Google Cloud customers get $300 in free credits on signup.

May 11 - May 15

Managed Service for Apache Airflow has launched a wave of new features, including the general availability of Airflow 3.1, AI-powered agentic troubleshooting, a new managed Airflow MCP Server for custom agent integration, and declarative YAML-based orchestration pipelines—discover all the details in the full blog post.

April 20 - April 24

Google-built ODBC Driver for BigQuery is now available in Preview
We are excited to announce the launch of the new, Google-built ODBC driver for BigQuery. This new open-source driver provides a direct, high-performance connection for applications to BigQuery and is developed entirely in-house by Google. Download a new driver and connect your application to BigQuery.

April 13 - April 17

We announced we are reintroducing Data Studio to play a significant role in the AI era, expanding from data visualizations and reports to host BigQuery conversational agents and data apps built in Colab notebooks.
We announced BigQuery Graph is now available in preview, offering an easy-to-use, highly scalable graph analytics solution, empowering data professionals to model, analyze and visualize massive-scale relationships in an entirely new way.

April 6 - April 10

We introduced Conversational Analytics for Looker Embedded environments, enabling users to add natural language experiences to their own custom data-driven applications, powered by Gemini.
We expanded Looker’s capabilities for faster ad-hoc analysis, with the introduction of self-service Explores, enabling you to bring your own data to Looker’s semantic layer and gain instant access to insights in a governed data environment.

March 23 - March 27

We showed you how you can scale your reads with Cloud SQL autoscaling read pools. This feature allows you to provision multiple read replicas that are accessible via a single read endpoint and to dynamically adjust your read capability based on real-time application needs.
Our customers are leveraging the full power of Conversational Analytics and Looker to drive major business and technical breakthroughs in the AI era. Companies like Telenor, Pet Circle, Fluent Commerce, Lighthouse Intelligence, Wego, and ROLLER are turning data into insights and actions, grounded by Looker’s semantic layer.

March 16 - March 20

We introduced an enhanced Gemini assistant in BigQuery Studio, transforming the agent from a code assistant into a fully context-aware analytics partner.

February 23 - February 27

We introduced managed and remote MCP support for Google Cloud databases, including AlloyDB, Spanner, Cloud SQL, Bigtable and Firestore, to power the next generation of agents. This announcement extends the ability for AI models to plan, build, and solve complex problems, connecting to the database tools our customers leverage daily as the backbone of their work environment.
We outlined how you can build a conversational agent in BigQuery using the Conversational Analytics API to help you build context-aware agents that can understand natural language, query your BigQuery data, and deliver answers in text, tables, and visual charts.

February 16 - February 20

Our customers are leveraging the full power of Looker to drive major business and technical breakthroughs. Companies like Arrive, Audika, Carousell, Framebridge, GumGum, Intel, Overdose Digital, Ocean Network Express, Subskribe and Promevo are leveraging Looker’s newest AI-driven capabilities, including Conversational Analytics, to transform data to insights and actions, and empower their entire organization with a single source of truth, powered by Looker’s semantic layer.

February 2 - February 6

Join us on March 4 for our webinar, Win Your AI Strategy with Cloud SQL Enterprise Plus, to learn how to power your generative AI workloads with 3x higher performance and 99.99% availability. Register today to discover how to build a scalable, enterprise-grade foundation for your most demanding AI applications.

January 26 - January 30

We introduced Conversational Analytics in BigQuery, which allows users to analyze data using natural language. Conversational Analytics in BigQuery is an intelligent agent that generates, executes and visualizes answers grounded in your business context directly in BigQuery Studio, making data insights for data professionals more conversational.
We outlined how data products have become the foundation for AI agents, providing the context needed to make autonomous agents reliable and trusted for real business use, backed by organized business logic and semantic understanding.
We highlighted how you can supercharge data analytics workflows, and outlined Google Cloud’s AI agent offerings for data engineering, data science, and development tools, so you can integrate agentic workflows in your applications, empower your teams and speed discovery.

January 19 - January 23

We have fundamentally reimagined Firestore with pipeline operations for Enterprise edition. Experience a powerful new engine featuring over a hundred new query features, index-less queries, new index types, and observability tooling to improve query performance. Seamlessly migrate using built-in tools and leverage Firestore’s existing differentiated serverless foundation, virtually unlimited scale, and industry-leading SLA. Join a community of 600K developers to craft expressive applications that maximize the benefits of rich queryability, real-time listen queries, robust offline caching, and cutting-edge AI-assistive coding integrations.
Introducing Google Cloud SQL on MSSQLTips: We are highlighting a new technical guide published on MSSQLTips titled "Introducing Google Cloud SQL." This article serves as an essential resource for SQL Server administrators and developers exploring Google Cloud's fully managed database service. It provides a detailed overview of Cloud SQL capabilities, including high availability, security integration, and the seamless transition of on-premises SQL Server workloads to the cloud, making it an ideal resource for those planning their migration strategy.
We are excited to announce the Public Preview of Microsoft Entra ID (formerly Azure Active Directory) integration with Cloud SQL for SQL Server. Designed to tackle the challenge of identity sprawl in multi-cloud environments, this integration allows organizations to govern database access using their existing Microsoft identity infrastructure. Key benefits include centralized identity management, enhanced security features like Multi-Factor Authentication (MFA), and simplified user administration through direct group mapping. This feature is available for SQL Server 2022 and supports both public and private IP configurations.

January 12 - January 16

Google-built JDBC Driver for BigQuery is now available in Preview
We are excited to announce the launch of the new, Google-built JDBC driver for BigQuery. This new open-source driver provides a direct, high-performance connection for Java applications to BigQuery and is developed entirely in-house by Google. Download a new driver and connect your Java application to BigQuery.
Troubleshoot Airflow tasks instantly with Gemini Cloud Assist investigations: Cloud Composer just got smarter. We are excited to announce that Gemini Cloud Assist investigations are now available directly within Cloud Composer 3. Instead of manually sifting through raw logs, you can now simply click "Investigate" on a failed Airflow task. Gemini analyzes logs and task metadata to identify failure patterns—such as resource exhaustion or timeouts—and provides actionable recommendations driven by Gemini Cloud Assist to resolve the issue. This integration shifts the debugging experience from manual toil to automated root cause analysis, significantly reducing the time required to restore your pipelines. Learn more about AI-assisted troubleshooting.

Modernizing Healthcare: How Alcidion achieved greater stability and performance with AlloyDB

Mon, 08 Jun 2026 16:00:00 +0000

In clinical informatics, every second counts. For Alcidion, a global leader in smart health solutions, the mission is simple but critical: use technology to reduce cognitive load for clinicians and present the right information at the right time to save lives.

Whether it’s managing patient flow in an emergency department or ensuring a patient is in the correct ward to avoid adverse outcomes, Alcidion’s flagship platform, Miya Precision, serves as a dynamic intelligent care platform for modern hospitals. To power this mission, the platform recently underwent a major architectural transformation, migrating from a legacy Microsoft SQL Server environment to Google Cloud’s AlloyDB for PostgreSQL.

The challenge: overcoming performance bottlenecks

Operating in an industry where data integrity and uptime are non-negotiable, Alcidion faced several technical and operational hurdles with its previous setup:

Operational overhead: Managing persistent backends for SQL Server required significant manual effort. The team had to manually balance database loads between elastic pools to maintain performance while trying to optimize costs. They also had to constantly manage the gap between allocated and used space to prevent shared pools from being consumed by excessive slack space.
Performance latency: Complex JSON data processing, critical for modern health informatics, was taking up to 30 minutes for certain jobs.
Stability concerns: The team sought a more stable Kubernetes environment and a persistent backend that could scale without constant administrative intervention.

The solution: a smooth migration to AlloyDB

Alcidion used the Database Migration Service (DMS) to move from SQL Server to AlloyDB, achieving a remarkably efficient cutover. The total learning and migration process took under one month, with the core database move completed in only one and a half weeks.

By creating custom synchronization tools and using Google Cloud’s managed services, the team reduced the final transition window to just 15 minutes. Alcidion achieved this by spinning up a new Google Cloud instance synchronized to the active one, with both accessible via unique fully qualified domain names. The new environment remained in read-only mode for customer validation.

During the final cutover, the old instance was set to read-only, synchronization was halted, and external integration links were toggled to the new environment. This streamlined process allowed users to log into the new instance and resume work within minutes, with the primary delay being DNS record updates.

Alcidion chose a fully managed AlloyDB service to eliminate control plane tasks and administrative overhead. This shift allows their engineering team to focus on clinical innovation and product development rather than "managing the container" or the underlying database infrastructure.

Being able to cut over to AlloyDB in about 15 minutes had our users back to work almost immediately. For a system clinicians rely on around the clock, that kind of smooth transition gave Alcidion real confidence.

The results: impact by the numbers

The shift to AlloyDB and Google’s Agentic Data Cloud has delivered immediate, quantifiable improvements for Alcidion and its healthcare customers:

Faster data processing: Data processing that previously relied on SQL Server stored procedures — a process that became increasingly time-consuming as data volumes grew — has been transformed. By migrating to AlloyDB and using BigQuery and Dataflow for processing, Alcidion has seen jobs that once took 30 minutes now complete in just 5 to 60 seconds.
Enhanced stability: The migration has delivered a step-change in reliability. In the previous environment, the team faced monthly disruptions, ranging from failed scheduled maintenance to connectivity issues that required manual intervention. In contrast, AlloyDB and Google Cloud’s compute services have proven exceptionally stable, allowing the team to move away from the "firefighting" mode associated with frequent infrastructure crashes.
Reduced cognitive load: By simplifying their backend and clinical dashboards, Alcidion’s SREs have significantly reduced their administrative burden. This shift has freed the team to focus on high-value innovation, such as refining predictive analytics and generative AI that empower clinicians to make informed clinical decisions faster.

Future vision: AI and beyond

Alcidion isn't stopping at database modernization. The move to AlloyDB is a foundational step for their next phase of growth:

AlloyDB columnar engine: The team is exploring the columnar engine for a second round of query optimization and real-time analytics.
Generative AI apps: Alcidion is actively working with Google to use AlloyDB’s Gemini Enterprise Agent Platform integration to perform concept analysis and pick out critical clinical insights from vast datasets.

By moving to AlloyDB, Alcidion has improved its stability and performance and built a strong foundation to keep delivering smarter, safer care to hospitals worldwide.

Ready to modernize your database? Learn more about how AlloyDB can transform your operational workloads.

Announcing Spanner Graph algorithms: Google-grade intelligence for connected data

Tue, 02 Jun 2026 16:00:00 +0000

At Google Cloud Next, we announced the preview of graph algorithms with Spanner Graph, bringing Google Research’s state-of-the-art graph mining capabilities natively to your database. These graph intelligence capabilities can help you derive valuable insights from graph data faster, cheaper, and at scale.

Enterprises are increasingly leveraging graph technologies to uncover complex relationships in data for use cases such as fraud detection, social network analysis, entity resolution, and healthcare research. Graph algorithms, such as node centrality and community detection, are the computational methods used to analyze these structures, and work by quantifying the patterns and strength of connections between entities. However, running graph algorithms at scale has historically been challenging and resource-intensive, often requiring complex ETL pipelines to dedicated analytic solutions or risking the transactional performance of the graph database.

We designed Spanner Graph algorithms to tackle demanding enterprise workloads without compromising on the performance of your operational database. This architecture provides several distinct advantages:

Tight integration with GQL: Directly invoke algorithms using ISO Graph Query Language (GQL) to run structural analytics across your data. By sequentially weaving algorithms and standard queries together, Spanner Graph minimizes complex data movement to external engines, simplifying your architecture and accelerating time-to-insight.
Near-zero transactional impact and lower TCO: Algorithm execution happens on dedicated compute resources, so as not to impact live production traffic. Spanner automatically provisions resources and securely routes data via Data Boost without having to create a custom ETL pipeline. Pay only for what you use, avoiding expensive licensing and operational overhead of legacy solutions.
Global insights on billion-edge graphs in minutes: Built for scale and speed, our engine can run algorithms on graphs with tens of billions of edges within minutes. Encoding topologies in a dense format that’s optimized for random access enables high-performance structural analytics on massive datasets.

While Google Research has published several research papers, held workshops, and released open-source projects based on its graph mining tools (e.g., for multi-core clustering), this is the first time that they are widely available to Google Cloud customers. Let’s take a deeper look at graph algorithms, and how you can use them with Spanner Graph.

Algorithms: Deeper insights for connected data

When we first launched Spanner Graph, our goal was to reimagine graph data management with a native graph database experience within Spanner, Google’s highly scalable, distributed database. Spanner Graph unifies relational and graph models, allowing developers to query connected data using the ISO GQL, while also interoperating with Spanner's existing tabular, search, and vector capabilities. This allows you to build intelligent applications without creating complex data pipelines, duplicating data, or increasing security and governance risk.

Building on this foundation, Spanner Graph algorithms help you to extract even deeper insights from your connected data. Graph algorithms analyze the relationships and connections within data, revealing hidden patterns and insights that might be missed with traditional analytical methods. With this launch, you can analyze connectedness to, for example, detect fraud rings, conduct clustering for entity resolution, identify points of failure in complex networks, or recommend products based on the preferences of connected users.

We use graphs extensively at Google. In fact, many popular algorithms like PageRank, the foundational technology that powers Google Search, were invented here. With native algorithm support in Spanner Graph, we are bringing some of Google’s leading graph intelligence capabilities directly to Google Cloud customers, with a set of essential graph algorithms that help you easily uncover the hidden structures within your data:

Centrality: Pinpoint the most influential and central nodes within your network using betweenness centrality, closeness centrality, and PageRank.
Community detection: Automatically group highly connected entities to uncover hidden segments with label propagation, correlation clustering, modularity clustering, weakly connected components, and clique aggregator.
Similarity and path finding: Find optimal routes using set-to-set shortest paths, or measure node similarities using Jaccard, cosine, common neighbors, and total neighbors.

An integrated developer experience

You can invoke graph algorithms directly using GQL on the entire graph, subgraphs, or a select set of nodes and edges. Spanner offers an integrated workflow: results from graph algorithm runs can be written directly back to Spanner Graph. This lets you invoke algorithms and standard queries sequentially, using the output of one operation as input to the next. Additionally, you can also store results in Cloud Storage buckets.

Example: Uncovering the ringleader of a fraudulent network

Consider a scenario where you are analyzing financial transactions to combat money laundering. Fraudsters usually manipulate a set of “mule” accounts (intermediary accounts for money laundering) that interact with one another to collectively commit fraud. To capture the teamwork between detected and hidden mule accounts, anti-fraud experts usually resort to link analysis and community detection graph algorithms. Here’s how you can use algorithms and queries together in Spanner Graph to catch them.

Step 1: Identify communities of accounts (algorithm)
First, we apply a modularity clustering algorithm to cluster accounts into communities. We then write the resulting community_id directly back to the Account in Spanner Graph.

code_block: <ListValue: [StructValue([('code', "-- Runs community detection and update results to the graph\r\nEXPORT DATA OPTIONS(\r\n format ='CLOUD_SPANNER',\r\n table = 'Account',\r\n write_mode = 'update_ignore_all'\r\n) AS\r\nGRAPH FinGraph\r\nCALL ModularityClustering(\r\n node_labels => ['Account'],\r\n edge_labels => ['Transfer']\r\n)\r\nYIELD node, cluster\r\nRETURN node.id, cluster AS community_id;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef1abed00>)])]>

Step 2: Pinpoint the suspicious community (query)
Now that every account belongs to a community, we can use a GQL query to perform analytical queries on each community to uncover anomalous behaviors. For example, we can check the total number of known fraud accounts within each community.

code_block: <ListValue: [StructValue([('code', '-- Finds the community with the highest concentration of flagged fraud\r\nGRAPH FinGraph\r\nMATCH (a:Account)\r\nWHERE a.community_id IS NOT NULL\r\n AND a.fraud_flag = TRUE\r\nRETURN a.community_id AS community_id, COUNT(*) AS fraud_count\r\nORDER BY fraud_count DESC;'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef1abe310>)])]>

Step 3: Calculate influence to find the "ringleader" (algorithm on a subgraph)
Let's assume the query above reveals that Community 2 has seen a massive spike in fraudulent activity. In this step, we filter the graph to isolate only the accounts in that specific community and run the PageRank algorithm to find the central ringleader within that exact group.

code_block: <ListValue: [StructValue([('code', "EXPORT DATA OPTIONS(\r\n format = 'CLOUD_SPANNER',\r\n table = 'Account',\r\n write_mode = 'update_ignore_all' \r\n) AS\r\n-- Specifies a suspicious subgraph\r\nGRAPH FinGraph\r\nMATCH (n:Account {community_id: 2})\r\nRETURN n\r\nFULL UNION ALL\r\nMATCH -[e:Transfer]->\r\nRETURN e\r\nNEXT\r\n-- Runs PageRank \r\nCALL PER() PageRank(max_iterations => 20) \r\nYIELD node, score\r\nRETURN node.id, score AS pagerank_score;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef1abe040>)])]>

Step 4: Investigate the target (query)
Now that the accounts in Community 2 have a pagerank_score, we can write a query that isolates the most central account and that immediately traces where that specific ringleader moved their funds recently.

code_block: <ListValue: [StructValue([('code', "-- Finds the top scorer (ringleader) and trace their money\r\nGRAPH FinGraph\r\nMATCH (ringleader:Account {community_id: 2})\r\nORDER BY ringleader.pagerank_score DESC\r\nLIMIT 1\r\nWITH ringleader\r\nMATCH (ringleader)-[e:Transfer]->{1, 5}(receiver:Account)\r\nWHERE e.ts > '2025-12-01'\r\nRETURN ringleader.id AS ringleader_id, receiver.id AS receiver_id, e.amount, e.ts;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef1abe790>)])]>

By allowing you to weave high-performance algorithms with standard GQL queries, Spanner Graph eliminates the need to move data back and forth between operational databases and external analytics engines. This unified approach dramatically simplifies your data architecture and accelerates your time to insight.

Trusted by industry leaders

Customers like DaVita, Yahoo!, SoundCloud, and WPP are already leveraging Spanner Graph algorithms to solve some of their most complex data challenges.

"Leveraging Spanner Graph for our Patient 360 initiative has allowed us to consolidate complex healthcare data into a single, unified view. The addition of native graph algorithms like community detection and centrality is a major step forward, enabling us to uncover deep insights within our patient networks faster and at scale. These fully managed capabilities allow our team to focus on driving innovation in patient care without the operational burden of managing complex data pipelines." - Sam Ghosh, Chief Enterprise Architect at DaVita Kidney Care

"Operating at global scale across Yahoo’s iconic consumer properties requires us to unify billions of user profiles into a single, real-time view. With Spanner Graph, we’ve modeled our Unified User Profile (UUP) as a graph, bringing together previously distributed systems into a centralized source of truth. The addition of fully managed graph algorithms on Spanner further accelerates our ability to deliver personalization at scale. By leveraging algorithms such as community detection and PageRank, we can drive deeper audience segmentation and power more relevant, engaging user experiences across our platform." - Chris James, Director of Engineering, Yahoo

"With 500+ million tracks from 40+ million artists across 190+ countries, SoundCloud is where emerging artists find their sound, hidden gems are discovered, and music culture is shaped in real time. We have been running graph algorithms in batch mode for years, with processes often taking multiple hours on custom clusters to analyze our massive, multi-billion-edge music graph. The launch of Spanner Graph algorithms is a true game-changer: It not only provides the massive scalability we need, but also allows us to move away from complex custom Python workflows to a fully managed service. Most importantly, it unlocks the ability to run graph algorithms on our most up-to-date data for use cases like identifying creator hubs and improving recommendations, without requiring complex ETL pipelines or impacting the low-latency transactional workloads running on Spanner today." - Sergey Chekanskiy, VP of Engineering - Data Foundation, SoundCloud

“We've been eager to leverage advanced graph algorithms for Open Intelligence, our foundational intelligence layer that securely connects trillions of live data points from clients, partners and WPP in a privacy-first way and that is now integrated and powers WPP’s agentic marketing platform, WPP Open. In order to have instant, exploratory access to complex relationships across billions of entities – driving planning, modelling, and experimentation — we need native support for deep graph traversal, structural pattern recognition, and advanced algorithms. Algorithm support on Spanner Graph provides the performance and scalability to tackle our most challenging graph analytics problems without operational overhead or expensive licensing." - Rob Marshall, Head of Strategy, Data & Intelligence, WPP

Build more intelligent applications

Now with native support for algorithms in Spanner Graph you can move beyond basic relationship traversals and run deep structural analytics directly on your freshest transaction data. By applying these classic graph algorithms at scale, you can unlock new capabilities for your enterprise applications:

Proactive fraud detection and anti-money laundering: Expose coordinated fraud rings by automatically grouping connected mule accounts with Community Detection (like modularity clustering), then apply centrality (like PageRank) to pinpoint the ringleader who controls the illegal fund flow.
Customer 360 and entity resolution: Unify fragmented, cross-channel data into a single canonical profile using similarity functions like Jaccard and community detection like label propagation. These profiles can be further enriched for downstream ML training by generating topological features, such as PageRank, for each node.
Autonomous network operations and digital twins: Model your IT or telecom infrastructure as a digital twin, using similarity and path finding (like set-to-set shortest path) to proactively identify critical vulnerabilities and predict cascading failures.
Hyper-personalized product recommendations: Move beyond basic purchase histories by analyzing broader user behaviors. Use similarity algorithms (like common neighbors) to find overlapping preferences between entities, and centrality (like personalized PageRank) to surface the most relevant recommendations for those peer groups.
Resilient supply chain and logistics: Protect your supply chain from hidden bottlenecks using centrality (like betweenness centrality) to pinpoint over-relied-upon distribution hubs, and path finding to instantly calculate efficient alternative routes during disruptions.
Cybersecurity threat hunting and blast-radius analysis: Accelerate threat hunting by applying community detection (like correlation clustering) to isolate anomalous machine communications, and path finding to trace the attacker's exact lateral movement and blast radius.
Predictive customer churn analysis: Stop contagious customer churn by mapping out tight-knit subscriber groups with community detection, then apply centrality to identify and target core influencers with retention promotions before the churn spreads.

Get started today

Spanner Graph algorithms are supported with the Enterprise and Enterprise+ editions of Spanner. To learn more, view the documentation or try out this codelab. You can also watch this video for a summary of graph algorithm support with Spanner Graph.

Modeling a digital twin of a food supply chain using BigQuery Graph

Mon, 01 Jun 2026 16:00:00 +0000

The example of a growing restaurant

Imagine you are running a restaurant chain. You just can't physically feel and touch things to know how your business operates. You need tools and a digital replica of your business to sense the health of the business for you.

The friction of growth

Growth creates a unique kind of friction that spreadsheets simply weren't built to solve:

The bullwhip effect: Small downstream demand shifts swell into upstream inventory tidal waves.
SOP drift: Tiny departures from standard prep work eventually erode the entire brand vibe.
The food safety blast radius: One contaminated ingredient creates a messy, complex map of risk across the network.
Maverick spend: The "million-dollar leak" caused by local managers purchasing ingredients off-contract.

The digital twin

Digital models empower us to ask more insightful questions about the world, but they also force a critical choice in how we structure data. While traditional relational tables have been the standard, we must ask: are they still the right tool for everything? Given that our world is inherently interconnected, perhaps shifting to graph-based models is the natural evolution for capturing reality.

When managing thousands of assets, complex supply chains, or global logistics networks, traditional relational databases require massive, resource-intensive SQL joins to trace dependencies. This architecture creates a latency gap between physical events and operational awareness.

Modeling with BigQuery Graph

BigQuery Graph allows you to build a digital twin of your entire supply chain within your existing data platform. By turning your physical world—items, recipes, and locations—into a searchable map of nodes and edges, you gain a new level of clarity.

1. Defining the Semantic Layer

Instead of moving data to a new database, you create a Graph View over your existing tables. This tells BigQuery exactly how your tables relate to one another.

Query Language:

code_block: <ListValue: [StructValue([('code', '# Build the Graph Nodes & Edges\r\nCREATE or REPLACE PROPERTY GRAPH `restaurant.bombod`\r\nNODE TABLES (\r\n `restaurant.item` label item properties all columns,\r\n `restaurant.location` label location properties all columns,\r\n `restaurant.itemlocation` label itemlocation properties all columns\r\n)\r\nEDGE TABLES (\r\n `restaurant.bom`\r\n KEY(bomKey)\r\n SOURCE KEY (childItemLocation) REFERENCES `restaurant.itemlocation`(itemLocationKey)\r\n DESTINATION KEY (parentItemLocation) REFERENCES `restaurant.itemlocation`(itemLocationKey)\r\n LABEL consists_of properties all columns\r\n);'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef026cd30>)])]>

Image of a fictitious restaurant supply chain modeled using BigQuery Graph

Precision in practice

How does this change daily operations? It moves the business from panic to precision.

Surgical recalls: If a supplier reports a Listeria breakout, you walk the graph forward to find exactly which menu items in which specific restaurants are affected.
Weather risk analysis: When a hurricane threatens a distribution center, you don't see a list of stores; you see the blast radius. You identify the locations critically dependent on that hub and reroute supplies.

2. Executing the search

Graph Queries are a new tool for modelers and data scientists to query their data - it simplifies complex multi-domain data concepts and simplifies querying and makes data analysis a simpler more natural representation of problem articulation. For example: If I want to know which all locations handle chicken I could run a graph query as shown below:

To investigate a specific complaint or risk, you run a search on the model using graph query language.

Graph Query Language

code_block: <ListValue: [StructValue([('code', "# Navigate to the source of a specific ingredient issue\r\nGraph restaurant.bombod\r\nMATCH (a:itemlocation)-[c:consists_of]->(b:itemlocation) \r\nWHERE b.itemKey LIKE '%Chicken%'\r\nRETURN to_json([to_json(a),to_json(c),to_json(b)]) as result"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7faef026cf40>)])]>

Source of a foul odor - modeled as a graph

Building for the future

To get the most out of your digital twin, follow these guiding principles:

Focus on structure: Use graphs for relationships and dependencies; keep daily sales totals in relational tables.
Clean your keys: Spend time on data engineering; a graph is only as strong as its connections.
Capture edge properties: Store metadata like lead times or shipping costs directly on the edges to increase the model's utility.

Conclusion

The restaurant industry has outgrown the relational way of treating business data only as a list. By building inter-domain relationships as a digital twin with BigQuery Graph, you move from reactive problem solving to proactive modeling. It’s time to stop managing your network with a list and start seeing the connections in seconds.

Get started today

Check out the tutorial here
Visit the BigQuery documentation: find overview and quickstart guide.
Share your feedback: join our community, and get your questions answered via bq-graph-preview-support@google.com.
Related blog: Introducing BigQuery Graph

Databases

Multiple result sets: How Database Migration Service automates SQL server to PostgreSQL translation

The Core Conversion Strategy of DMS

Automating Multi-Result Set Conversions: Inside DMS

The Target: PostgreSQL’s Deterministic Cursors

1. Simple Tracking: Stored Procedure Translation

2. Multi-Set Tracking: Function Translation

Parsing the Output Programmatically

Executing and Testing the Migrated Code

DMS Insight: The Mechanics of Result Set Counting

Summary

Introducing Database Operations Agents: The future of autonomous database management

Database Observability Agent: From diagnosis to remediation

Database Onboarding Agent

Get started

How Target is enhancing retail discovery and cutting database maintenance by 50% with Spanner Graph

Overcoming fragmented architecture

Building the enterprise ontology on Spanner Graph

A smooth, zero-downtime incremental migration

Business impact

Unifying public and private data: Scale knowledge graphs with Data Commons on Spanner

Data Commons knowledge graph dimensions

A modern infrastructure powered by Spanner Graph

Bridging systems with SDMX 3.0 interoperability

Transforming private instances of Data Commons Platform

Get started today

AlloyDB adds group authentication to secure enterprise scale and AI agents

The problem with individual scale

Securing the future of agentic AI

Proven value for digital leaders

A unified blueprint for passwordless access

Moving toward a Zero Trust database future

Your AI agents are ready. Is your data?

Overcoming a lack of context

Escaping unnecessary manual work

Bringing trust and knowledge to the data

The path forward

Report: 83% of organizations need to upgrade their infrastructure to support agentic AI

Supercharging pgvector: 4x faster HNSW vector search with AlloyDB

First, what is the AlloyDB columnar engine?

Performance visualization

How it works: Columnar engine Accelerated HNSW

Why it Matters

Quick Start Guide

How to solve PostgreSQL multilingual full-text search limitations with AlloyDB AI

Continuous text and logographical languages

In-database pre-processing with Gemini

Implementing semantic word segmentation

Extending to hybrid search

Why AlloyDB AI is ideal for search

What's next

Building the AI-defined vehicle with Android, Google Cloud, and Nexus SDV

The vehicle side: AAOS SDV

The Google Cloud side: AI-native mobility

Bigtable: The data backbone for Automotive Telemetry

Bring it all together: Nexus-SDV platform

Predictive maintenance

Get started

Learn more

SOCRadar powers rapid threat detection with AlloyDB and Gemini Enterprise

How SOCRadar supercharges rapid threat detection with AlloyDB

Evaluating database alternatives: The hunt for scalability

Taming a "triple-threat" workload

More than just speed: Reclaiming 45 TB and 75% of DBA time

Fighting alert fatigue with integrated Gemini Enterprise Agent Platform

Expanding capabilities: The future of agentic threat hunting

AlloyDB AI Functions - now with revolutionary performance boosts and cost savings

More functions to summarize and analyze sentiment

The power of LLMs on your data: now significantly faster and cheaper

See it in action!

Getting started is easy

Modernizing financial services with deployment freedom and transformational AI with AlloyDB Omni

AlloyDB Omni: Strong performance and deployment freedom

Enterprise-grade security and compliance

Next steps

Supercharging the agentic era with Spanner’s multi-model architecture

Real-world impact

The multi-model advantage for the agentic era

Spanner Omni: Multi-model capabilities, everywhere

Industry-defining capabilities for core databases