Introducing agentic video understanding with Gemini
2026-09-02 · Google DeepMind
Google launches agentic video understanding for Gemini
Google has launched agentic video understanding across its latest models: Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite. The new capability improves accuracy while dramatically reducing token usage and costs for video analysis.
Similar to agentic vision, which combines code execution with Gemini's native image understanding, agentic video understanding uses Gemini's native video tools to improve performance and unlock new video processing capabilities, such as sub-second moment retrieval, more accurate anomaly detection, precise counting and more. The feature is available today for video uploads and YouTube videos via the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform.
Benchmarks
Current "static" processing ingests video at a fixed frames-per-second rate (default 1 FPS, adjustable via API). Agentic video understanding instead pairs the model's core reasoning with native video tools to dynamically search, scan and inspect target video segments across visual frames, audio and transcripts.
Across standard video analysis benchmarks, Gemini models with agentic video understanding achieve:
- Up to 88% reduction in token consumption
- Up to 66% reduction in analysis costs
- Up to 7% improvement in accuracy
These efficiency gains are especially pronounced on long-form video — from 10-minute how-to guides to 90-minute lectures and multi-hour recordings — where static processing forces developers to choose between high token costs or techniques that drop critical details.
While the gains span all three supported models, Google says Gemini 3.7 Flash with agentic understanding offers the best overall quality and the best combination of quality and cost efficiency, placing it at the accuracy-to-cost pareto frontier among tested models for video understanding.
How it works
Rather than ingesting media streams at a fixed frame rate, agentic video understanding enables Gemini to take an active, goal-directed role in determining what to watch, at what speed, and through which modality — frames, audio or transcript — fetching only the moments and signals needed.
Developers could previously do this manually, but with agentic video understanding, Gemini accomplishes it through an agentic loop, invoking an internal tool to load the relevant part of the video file and significantly reducing development overhead.
Capabilities and use cases
The feature transforms how developers can process long-form video content across demanding applications:
- Sub-second moment retrieval: Pinpoint split-second state changes and tight cut boundaries that are easily missed at 1 FPS, enabling precise automated video editing.
- Long-form needle-in-a-haystack search: Answer complex queries across multi-hour videos without consuming millions of tokens.
- Anomaly detection: Resample interesting time windows at higher FPS to inspect rapid motion and subtle visual artifacts.
- Counting actions and objects: Accurately track repeated physical movements and distinct objects over time.
Google's demonstrations highlight several results. On LongVideoBench, a long-form video understanding benchmark, Gemini 3.7 Flash with agentic video understanding shows large token reductions alongside accuracy improvements. For fast action analysis, the model can accurately count fast-paced movements by scanning and rewatching the video at different frame rates as needed. In needle-in-a-haystack search, Gemini 3.7 answers complex questions about video content while consuming significantly fewer tokens than static analysis. Google adds that many early access partners saw strong performance while testing the feature.
Getting started
Agentic video understanding is available via the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform, launching across Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite. It uses standard Gemini API token pricing with no additional feature fee.
To enable it, simply set processing to "agentic" in the API configuration:
from google import genai
client = genai.Client()
interaction = client.interactions.create(
model="gemini-3.7-flash",
input=[
{
"type": "video",
"uri": "https://youtu.be/7Z5Vy9JBANs",
"processing": "agentic"
},
{
"type": "text",
"text": "What are the 3 most important announcements in this keynote?",
},
],
)
print(interaction.output_text)
Developers can read Google's developer guide for more insight into the feature and how to get started.