What's new for Data Analytics in the AI driven, autonomous and agentic era

Today's fast-paced environment demands more than just data access; it requires a real-time data activation flywheel. A new reality is emerging where AI, infused directly into the data landscape, works hand-in-hand with intelligent agents. These agents act as catalysts, unlocking insights for everyone, and enabling the autonomous, real-time action that's critical for success. Google’s Data & AI Cloud is built to power this flywheel, bringing AI to data for continuous, real-time data activation — a focus that’s attracting 5x more organizations to BigQuery than the two leading cloud companies that exclusively offer data warehouse and data science platforms.

Today, Google Cloud announcing several new innovations with our autonomous data to AI platform powered by BigQuery, alongside our unified, trusted, and conversational BI platform with Looker:

1. Specialized agents for every user: New assistive and agentic experiences, grounded in your trusted data and available in BigQuery and Looker, are set to simplify and accelerate the work of data engineers, data scientists, analysts and business users.
2. Accelerating data science and advanced analytics: Google Cloud is enhancing data science workflows in BigQuery with new AI-assisted notebooks and unlocking new insights with our BigQuery AI Query Engine, alongside seamless integration with real-time and open-source technologies.
3. Autonomous data foundation: New autonomous capabilities in BigQuery capture, manage, and orchestrate all data types, including native support for unstructured data handling and open data formats like Iceberg.

Let’s take a deeper look at each of these developments.

1. Specialized agents for every user

Google Cloud believe AI should be accessible to everyone. Google Cloud has made AI driven assistive experiences broadly available in BigQuery and Looker, and now Google Cloud has expanded to specialized agents that best meet the needs for all data roles, including:

• Data engineering agent capabilities, embedded in BigQuery pipelines (GA), deliver support to build data pipelines, perform data preparation (GA) like transformation and enrichment of data, maintain data quality with anomaly detection (preview), and automate metadata generation. Traditionally data engineers spend countless hours cleaning, transforming, and validating data — these agents replace tedious and time consuming tasks and enable trusted data, boosting productivity of your data teams.

• Data science agent (GA), embedded within Google’s Colab notebook, enables every stage of model development. It automates feature engineering, provides intelligent model selection, enables scalable training, and faster iteration. This agent allows data science teams to focus on building advanced data science workflows, instead of wrestling with data and infrastructure.

• Looker conversational analytics (preview) empowers every user to interact with data using natural language. Expanded capabilities developed in partnership with DeepMind, not only conducts advanced analysis but also explains its thinking transparently, empowering all users to understand the agent’s behavior and seamlessly resolve ambiguities. In addition, Looker’s semantic layer improves accuracy by as much as two thirds. As users reference business terms like ‘revenue’ or ‘segments,’ the agent knows exactly what you mean and can calculate metrics in real-time, ensuring it delivers accurate, relevant, and trusted results. Additionally, we are launching a conversational analytics API (preview) for developers to build and embed conversational analytics into applications and workflow.

Interact using natural language with Looker Conversational Analytics Agent

To power intelligence across assistive and agentic experiences in the BigQuery autonomous data to AI platform, Google Cloud also launching BigQuery knowledge engine (preview). It leverages the power of Gemini to analyze schema relationships, table descriptions, and query histories to generate metadata on the fly, model data relationships, and recommend business glossary terms. This knowledge engine is the foundation of AI powered experiences, including AI-powered data insights and semantic search (GA) across BigQuery, grounding AI and agents in business context.

Today all of Google Gemini powered assistive and agentic experiences in BigQuery and Looker are available to all customers within the existing tiers of our pricing models — no add-ons!

2. Accelerating data science and advanced analytics

Google Cloud BigQuery autonomous data to AI platform is fundamentally changing how data scientists and analysts work, enabling new AI driven data science experiences and new engines to process complex data and support advanced analyses in real-time.

First, we’re supercharging the BigQuery notebook experience with AI. Google Cloud introducing intelligent SQL cells that understand your data's context and provide smart suggestions as you write code and enable you to join data sources directly within your notebook. Google Cloud also adding native exploratory analysis and visualization capabilities, making it easy to explore data, as well as adding features to enable easier collaboration with colleagues. Data scientists can also schedule analyses to run and refresh insights periodically. And, to share insights more broadly across the organization, Google Cloud introducing the ability to build interactive data apps – dynamic, user-friendly interfaces powered by your notebook.

AI Powered Data Science Experience

Building on this enhanced notebook environment, Google Cloud also announcing BigQuery AI query engine to support advanced, AI-driven analytics. This engine enables data scientists to move beyond simply retrieving structured data to seamlessly processing both structured and unstructured data together with added real-world context. The BigQuery AI query engine co-processes traditional SQL alongside Gemini to inject runtime access to real-world knowledge, linguistic understanding, and reasoning abilities. A data scientist can now ask questions like: "Which products in our inventory are primarily manufactured in countries with emerging economies?" The foundation model inherently knows which countries are considered emerging economies. Analysts can ask: “Which products are included in these social media images?” and Google Cloud new engine processes the unstructured images and matches them to your product catalog. This engine supports a broad range of use cases, including building richer features for models, performing nuanced segmentation, and uncovering insights previously out of reach.

Furthermore, Google Cloud empower users with the best of the open-source ecosystem, enhanced for the cloud. Google Cloud for Apache Kafka (GA) facilitates real-time data pipelines for event sourcing, model scoring, messaging and real-time analytics, powering serverless execution of Apache Spark workloads within BigQuery (preview). Customer use of Google Cloud serverless Spark capability has nearly doubled in the past year, and Google Cloud has enhanced this engine to provide 2.7x faster processing than the prior year.

BigQuery allows data scientists to leverage the tools they need on Google’s serverless and scalable architecture, whether it's SQL, Spark, or the semantic power of foundation models, enabling faster innovation without traditional infrastructure challenges.

We see SQL and Spark as two complementary ways of accessing and transforming data. Spark is especially useful to us in use cases that require complex business logic, which although niche, are extremely business-critical. Having an unified platform for SQL, Spark and AI, with the development experience in notebooks will considerably simplify these critical use cases.” - Andrés Sopeña Pérez, Head of Content Engineering, Trivago

3. An autonomous data foundation across the entire data lifecycle 

Underpinning our specialized agents and advanced analytics engines is an autonomous data foundation designed for the complexity of modern data. Google Cloud fundamentally changing the landscape by making unstructured data a first-class citizen within BigQuery. This is achieved with new capabilities in the platform including autonomous and invisible governance, orchestration for diverse data workloads, and a commitment to flexibility via open formats, ensuring your data is always ready for any data science or AI challenge. And we do all of this while minimizing operational overhead and delivering optimal price performance.

The biggest untapped opportunity for many organizations lies in the potential of their unstructured data. While structured data has pathways to analysis, unique insights embedded in images, audio, video, and text are often hard to extract and underutilized, and typically reside in siloed systems. BigQuery directly tackles this challenge by making unstructured data a first-class citizen with multimodal tables (preview), allowing you to bring rich, complex data types alongside structured data for unified storage and querying. To effectively manage this comprehensive data estate, Google Cloud enhanced BigQuery governance (preview) provides a single, unified view for data stewards and professionals to handle discovery, classification, curation, quality, usage, and sharing, including automated cataloging (GA) and metadata generation (experimental). Moreover, to ensure timely insights from all your data streams, BigQuery continuous queries (GA) enable instant analysis and action on streaming data using SQL, regardless of its original format.

Google Cloud advanced support of multimodal data, both structured and unstructured, is driving adoption - customer use of Google’s AI models in BigQuery for multimodal analysis has grown by 16x year over year. Google Cloud integrated approach to data and AI is also cost effective, together BigQuery and Vertex AI are between 8-16x more cost efficient when compared to other independent data warehouse and AI platforms.

Google Cloud commitment to an open ecosystem remains paramount. BigQuery tables for Apache Iceberg (preview), delivers the flexibility of an open data lakehouse alongside the performance and integrated tooling of BigQuery, allowing you to connect your Iceberg data to SQL, Spark, AI and third party engines in an open and interoperable manner. This offering provides adaptive and autonomous table management, delivers high-performance streaming, auto-AI generated insights, near infinite serverless scale, and advanced governance. Through integration with Cloud storage, Google Cloud managed service provides centralized fine-grained access control management and fail-safe capabilities.

Finally, the autonomous data to AI platform is self-optimizing. It scales resources, manages workloads, and helps ensure their cost-effectiveness with advanced workload management capabilities (GA). Furthermore, Google Cloud simplified purchasing with the new BigQuery spend commit (GA), unifying spend across our BigQuery platform, providing flexibility to move spend across data processing engines, streaming, governance, and more.