Google Research | Techlist.io

googleMar 6, 20263 min readCurated summary

Where wild things roam: Identifying wildlife with SpeciesNet

SpeciesNet is an open-source AI tool that identifies wildlife in camera-trap images, making large-scale monitoring faster and more practical. Trained on more than 65 million labeled images, it can classify nearly 2,500 animal categories and process tens of thousands of images per day. Its adoption by researchers, governments, and conservation groups is expanding wildlife research and enabling more responsive conservation efforts. ## A New Era for Wildlife Monitoring - Motion-triggered camera traps generate enormous volumes of images, often far beyond what human teams can classify manually. - Automated identification helps researchers: - Track population health and changes. - Study migration and climate-related movement. - Estimate population sizes. - Detect rare or endangered species. - SpeciesNet uses deep learning to identify animals in camera-trap photos, accelerating analysis and improving wildlife-management decisions. - The tool is part of Google Earth AI, a collection of geospatial AI tools intended to support environmental and conservation work. ## SpeciesNet’s Training and Performance - SpeciesNet classifies 2,498 categories of mammals, birds, reptiles, and other animals. - It works with MegaDetector, another open-source model that identifies which images and pixels contain animals. - The system provides: - Species names. - Confidence scores. - Multiple identifications when several animals appear in one image. - Processing capacity is approximately: - 30,000 images per day on a standard laptop. - 250,000 or more images per day on a low-end gaming GPU. - SpeciesNet was trained on more than 65 million images from Wildlife Insights and public repositories. - On held-out camera-trap projects, it: - Detected animals in 99.4% of relevant images. - Reached species-level classification 83% of the time. - Produced correct species-level predictions in 94.5% of those cases. - Human-verified labels from Wildlife Insights can be reused as additional training data, creating a feedback loop for improving the model. ## Conservation Projects Using SpeciesNet - **Snapshot Serengeti:** Researchers can analyze roughly 11 million images collected since 2010 in just days, rather than relying exclusively on citizen scientists. Field processing also allows cameras to be redeployed based on recent sightings. - **Wildlife Observatory of Australia:** The organization trained a regional version of SpeciesNet to recognize Australian species missing from the original label set, including musky rat-kangaroos and orange-footed scrubfowl. - **Idaho Department of Fish and Game:** SpeciesNet serves as a first-pass classifier for images of deer, elk, black bears, coyotes, and other wildlife, speeding up human verification. - **Public and private platforms:** Tools including Animl and AddaxAI have integrated SpeciesNet, while companies such as Okala use it alongside Google’s Perch audio model to monitor biodiversity in Africa. - The model has also supported studies of pumas and ocelots in Colombia, cassowaries in Australia, and lions and elephants in Tanzania. SpeciesNet demonstrates how open-source AI can turn massive camera-trap datasets into usable scientific evidence. Its strongest role is as a scalable first-pass system combined with human review, while regional adaptations can extend its usefulness to local and threatened species.

Read original

Deep Learning Data Analysis Google Cloud Speciesnet+3

googleMar 4, 20263 min readCurated summary

Teaching LLMs to reason like Bayesians

LLMs often struggle to update their beliefs as new evidence arrives, relying instead on simplistic heuristics. Google Research tested whether training models to imitate an optimal Bayesian assistant could improve this capability. The results show that Bayesian teaching substantially improves recommendation accuracy, adaptation across interactions, and generalization to other tasks—more effectively than training on always-correct answers. ## Testing Bayesian Reasoning in LLMs - Researchers created a five-round flight recommendation task involving three options with different: - Departure times - Flight durations - Number of stops - Costs - Simulated users had hidden preferences, such as strong, weak, or no preference for high or low values of each feature. - After every recommendation, the user revealed the correct choice, giving the assistant new evidence. - The benchmark compared: - Off-the-shelf LLMs - Human participants - An optimal Bayesian assistant - The Bayesian assistant maintained a probability distribution over possible user preferences and updated it using Bayes’ rule. - Most LLMs performed substantially worse and often stopped improving after the first interaction, showing limited ability to incorporate information over time. - Humans improved more than most LLMs but still failed to match the Bayesian assistant. ## Bayesian Teaching Framework - Bayesian reasoning requires an agent to: - Start with a prior belief about the world - Incorporate new evidence - Produce a posterior belief - Use that posterior as the prior for future reasoning - For LLMs, the “world state” includes facts, relationships, concepts, and inferred user preferences. - Researchers used supervised fine-tuning on many simulated user interactions to teach models this update process. ## Oracle Teaching vs. Bayesian Teaching - **Oracle teaching** trained models on interactions with an assistant that knew the user’s preferences perfectly and always selected the correct option. - **Bayesian teaching** trained models to imitate an assistant that estimated preferences probabilistically and sometimes made mistakes, especially during early uncertain rounds. - The researchers argued that Bayesian examples better preserve uncertainty and demonstrate how beliefs should change as evidence accumulates. - This approach resembles knowledge distillation: the LLM learns to reproduce the predictions of a more principled teacher rather than memorizing only correct outcomes. ## Results and Generalization - Both fine-tuning strategies improved performance compared with the original LLMs. - Bayesian teaching consistently outperformed oracle teaching. - Models trained on Bayesian predictions more often agreed with the optimal Bayesian assistant. - Improvements extended beyond the original flight recommendation task, suggesting the models learned a broader approximation of probabilistic reasoning rather than merely memorizing task-specific patterns. - The findings indicate that LLMs can acquire reasoning strategies from examples and apply them in new domains. The practical implication is that training models on the behavior of an optimal probabilistic reasoner may be more effective than supplying only correct answers. For agents that must learn user preferences or update beliefs over time, examples that explicitly preserve uncertainty and demonstrate evidence-based belief revision could produce more reliable behavior.

Read original

Large Language Models Supervised Fine Tuning Bayesian Inference Probabilistic Reasoning+3

googleFeb 16, 20263 min readCurated summary

Teaching AI to read a map

MapTrace addresses a major weakness in multimodal language models: recognizing objects on maps is easier for them than understanding connectivity, obstacles, and valid routes. The authors propose a synthetic-data pipeline that generates maps, identifies walkable areas, constructs navigation graphs, and verifies computed paths with AI critics. They report releasing 2 million map question-answer pairs and show that fine-tuning on a much smaller subset improves route tracing on unseen real-world maps. ## The Challenge: Weak Spatial Grounding - MLLMs may recognize locations and objects in an image but still draw routes through walls, buildings, enclosures, or shops. - Effective navigation requires understanding: - Which regions are traversable - How paths connect - That routes are ordered sequences of connected points - The geometric and topological relationships between map features - Existing image-text training rarely teaches this “spatial grammar.” - Manual pixel-level route annotation would be expensive and difficult to scale. - Many useful maps of malls, museums, and theme parks are proprietary, limiting access to real-world training data. ## A Scalable Synthetic-Data Pipeline MapTrace uses generative AI to create diverse maps and automatically produce valid route annotations. ### Generating Diverse Maps - An LLM creates detailed prompts for environments such as: - Zoos with interconnected habitats - Shopping malls with food courts - Fantasy theme parks with themed areas - A text-to-image model renders the prompts as map images. - This approach provides control over map diversity and complexity. ### Identifying Walkable Areas with a Mask Critic - Pixels are clustered by color to produce candidate masks representing possible walkways. - An MLLM reviews each mask alongside the original map. - The “Mask Critic” rejects masks that do not represent realistic, connected traversable regions. - Accepted areas may include sidewalks, crosswalks, and pedestrian paths. ### Converting Maps into Navigation Graphs - The selected traversable mask is converted into a pixel-based graph. - Walkway intersections become nodes, while connected stretches become edges. - This graph captures the map’s connectivity and enables computational route planning. ### Generating and Validating Routes - Thousands of random start and end points are sampled for each map. - Dijkstra’s algorithm computes the shortest path between each pair. - A “Path Critic” checks the overlaid route to ensure it: - Stays within traversable regions - Avoids obstacles - Follows a logical human route - Routes approved by the critic become training examples. ## Dataset and Evaluation - The pipeline generated 2 million annotated map question-answer pairs. - The authors note that generated maps sometimes contain incorrect text, but the study focuses primarily on path fidelity. - They fine-tuned models including Gemma 3 27B and Gemini 2.5 Flash on 23,000 generated paths. - Performance was evaluated on MapBench, which contains unseen real-world maps. - Route accuracy was measured using normalized dynamic time warping (NDTW), which compares predicted and reference coordinate sequences while accounting for differences in sampling and travel speed. - Lower NDTW scores indicate closer agreement with the reference route. ## Conclusion The work suggests that targeted synthetic training data can teach MLLMs map-based spatial reasoning that is largely missing from general pretraining. The released dataset and pipeline provide a foundation for improving visual navigation, while better image-generation models should reduce remaining typography and rendering artifacts.

Read original

Gemini Synthetic Data Generation Mllm Spatial Reasoning+3

googleFeb 10, 20263 min readCurated summary

Scheduling in a changing world: Maximizing throughput with time-varying capacity

The post presents scheduling algorithms for non-preemptive jobs when cloud capacity changes over time because of failures, maintenance, power limits, or higher-priority workloads. The goal is to maximize completed job value while respecting release times, deadlines, processing durations, and fluctuating parallel capacity. The research establishes the first constant-factor guarantees for several offline and online variants, including a 1/11 competitive ratio for a demanding common-deadline model. ## Scheduling with Time-Varying Capacity - A capacity profile specifies how many jobs can run simultaneously at each point in time. - Each job has: - A release time - A hard deadline - A processing duration - A weight or profit - Jobs must run continuously once started in the non-preemptive setting. - If capacity drops during execution, an interrupted job loses its progress. - The objective is to select and schedule jobs maximizing total completed weight. - The study considers: - **Offline scheduling**, where future jobs and capacity changes are known. - **Online scheduling**, where jobs arrive dynamically and decisions cannot be reversed. ## Offline Scheduling Results - The optimal problem is NP-hard, so the work focuses on approximation guarantees. - For unit-profit jobs, an earliest-finish-time Greedy algorithm achieves a **1/2-approximation**. - It completes at least half as many jobs as an optimal schedule. - This matches the classic guarantee for single-capacity scheduling. - For jobs with different weights, a primal-dual algorithm achieves a **1/4-approximation**. ## Why Online Non-Preemptive Scheduling Is Difficult - Online schedulers must commit without knowing future jobs. - Starting a long job can block many shorter jobs that arrive later. - Because each completed job may have equal value regardless of duration, one poor decision can sharply reduce throughput. - Consequently, standard non-preemptive online algorithms have competitive ratios approaching zero. ## Interruption with Restarts - An active job may be interrupted, but its completed work is discarded and the job can be retried later. - A modified earliest-finish-time Greedy algorithm achieves a **1/2 competitive ratio**. - This means it can guarantee at least half the throughput of an optimal schedule with complete knowledge of future arrivals. ## Interruption Without Restarts - If an interrupted job is permanently discarded, online scheduling becomes substantially harder. - In general, every online algorithm can be forced into decisions that prevent it from completing much future work. - The competitive ratio again approaches zero. - The authors therefore study a practical special case in which all jobs share a common deadline. ## A Common-Deadline Algorithm For a unit-capacity system, the algorithm maintains a tentative schedule of jobs in disjoint time intervals. When a new job arrives, it applies the first suitable action: 1. Place the job in an empty interval. 2. Replace a scheduled future job if the new job is significantly shorter. 3. Interrupt the current job if the new job is shorter than its remaining processing time. 4. Discard the new job. - The approach balances immediate execution against preserving capacity for shorter future jobs. - A generalized version works with arbitrary capacity profiles. - The resulting algorithm achieves the first constant competitive guarantee for this setting: **1/11**. The results suggest that schedulers for volatile cloud environments need controlled interruption and carefully designed replacement policies. Allowing restarts offers strong guarantees, while stricter interruption rules require additional structure—such as a shared deadline—to achieve predictable performance.

Read original

Cloud Computing Resource Management Scheduling Algorithms Approximation Algorithms+3

googleFeb 10, 20263 min readCurated summary

Beyond one-on-one: Authoring, simulating, and testing dynamic human-AI group conversations

DialogLab is an open-source research prototype for designing, simulating, and evaluating dynamic human–AI group conversations. It addresses the tension between rigid scripts and unpredictable generative dialogue by combining structured conversational phases with real-time improvisation. Its evaluation with 14 participants suggests that human-guided simulation offers the strongest balance of realism, engagement, and control. ## A Framework for Multi-Party Conversations - DialogLab separates a conversation’s social structure from its progression over time. - **Group dynamics** define: - Groups, such as a conference or social event - Parties, such as presenters and audiences - Elements, including human or AI participants and shared content - **Conversation-flow dynamics** define: - Snippets, or distinct phases such as opening, debate, and consensus - Participants and turn sequences within each snippet - Interaction styles, including collaborative or argumentative modes - Rules for interruptions and backchanneling - This separation makes complex conversation designs modular and easier to revise. ## The Author–Test–Verify Workflow ### Authoring with Visual Tools - Designers use a drag-and-drop canvas to arrange avatars and shared content. - Inspector panels configure personas, roles, interaction patterns, and snippet behavior. - Automatically generated prompts can be customized for specific narrative or conversational goals. ### Human-in-the-Loop Simulation - A live preview displays the evolving transcript. - In human-control mode, an audit panel suggests possible AI responses. - Designers can edit, accept, or reject suggestions, retaining control over the agents’ contributions. - The system supports both structured interactions and more improvisational conversations. ### Verification and Analytics - A verification dashboard provides post-hoc analysis of the conversation. - Visualizations show turn-taking distributions and sentiment flows. - These tools help creators diagnose interaction patterns without manually reviewing entire transcripts. ## Prototype Evaluation - Fourteen participants from game design, education, and social science research evaluated DialogLab. - They designed an academic social event and tested AI group discussions under three conditions: - **Human control:** Users prompted agents to shift topics, introduce perspectives, ask probing questions, or generate emotional responses. - **Autonomous:** Agents participated proactively according to predefined random or sequential orders. - **Reactive:** A simulated human agent responded only when directly addressed. - Human control was rated significantly more engaging and was generally considered more effective and realistic. - Participants also described the interface as intuitive, flexible, and enjoyable. - Users valued the combination of automated prompt generation, detailed customization, and support for different moderation strategies. DialogLab demonstrates that effective multi-party conversational design benefits from combining explicit structure with controlled improvisation. For developers and researchers building group-based human–AI experiences, a visual authoring workflow paired with human-guided simulation and analytics can provide a practical foundation for rapid iteration and more realistic interactions.

Read original

Large Language Models Generative AI Conversational Ai Multi Party Dialogue+3

googleFeb 9, 20263 min readCurated summary

How AI trained on birds is surfacing underwater mysteries

Perch 2.0, Google DeepMind’s bioacoustics foundation model, was trained mainly on birds and terrestrial animals yet performs strongly on underwater audio. The study shows that its learned audio embeddings can support accurate whale, dolphin, reef-sound, and killer-whale classification with only a few labeled examples. This suggests that large, broadly trained bioacoustics models can transfer across environments and accelerate marine research without requiring extensive underwater training data. ## Underwater Mysteries and Bioacoustics - Ocean recordings reveal animal behavior, species distributions, and unexplained sounds. - The “biotwang,” recently attributed by NOAA to Bryde’s whales, illustrates how new calls and species identifications continue to emerge. - Google has previously developed models for humpback whales and multi-species whale detection. - Perch 2.0 extends this work despite having no underwater audio in its training data. ## Transfer Learning for Custom Classifiers - Researchers can use an existing model directly when its labels match their data. - For new sounds or datasets, transfer learning avoids training a deep neural network from scratch. - Perch 2.0 converts audio windows into compact numerical embeddings. - A logistic regression classifier is then trained on those embeddings using labeled examples. - This requires far less computation, experimentation, and training data than full neural-network training. ## Evaluation on Marine Datasets - The researchers tested Perch 2.0 with few-shot linear probes using 4, 8, 16, or 32 examples per class. - Performance was measured using ROC-AUC, where values closer to 1 indicate better class separation. - Evaluation datasets included: - **NOAA PIPAN:** Baleen-whale recordings, including minke, humpback, sei, blue, fin, and Bryde’s whales. - **ReefSet:** Reef biological sounds, fish, dolphins, anthropogenic noise, and waves. - **DCLDE:** Killer whales, humpbacks, abiotic sounds, unknown sounds, and killer-whale ecotypes. - More examples generally improved results. - ReefSet performance was already high with four examples per class for most models. - Perch 2.0 was consistently among the best-performing models across datasets and sample sizes. ## Comparisons with Other Models - Perch 2.0 was compared with Perch 1.0, SurfPerch, and Google’s multi-species whale model. - It also outperformed AVES-bird and AVES-bio on most underwater tasks. - The results show that strong underwater transfer is not limited to models trained on marine audio. ## Why Bird-Based Training Transfers to Whales - The authors suggest that large models trained on extensive datasets can generalize effectively to unfamiliar downstream tasks. - Shared acoustic patterns across animal vocalizations may allow representations learned from birds and other terrestrial species to remain useful underwater. - The findings challenge the assumption that a model must be trained directly on underwater recordings to perform well on marine classification tasks. ## Practical Tools for Researchers - Google provides a paper and a Google Colab tutorial. - The tutorial demonstrates an end-to-end workflow for building a whale-vocalization classifier. - It uses NOAA’s NCEI Passive Acoustic Data Archive and Google Cloud. - Researchers can create agile, task-specific models with relatively small labeled datasets. Perch 2.0 demonstrates that broad bioacoustic pretraining can substantially reduce the effort required to study marine sounds. Researchers can begin with general-purpose embeddings and adapt them to new whale species, calls, or underwater sound categories using only modest labeled data.

Read original

Machine Learning Foundation Models Embeddings Few Shot Learning+4

googleFeb 5, 20263 min readCurated summary

How AI tools can redefine universal design to increase accessibility

Google Research proposes Natively Adaptive Interfaces (NAI), a framework that uses multimodal and agentic AI to make interfaces adapt to individual users rather than forcing everyone into a fixed design. Developed through co-design with disability communities, NAI aims to reduce the accessibility gap by embedding assistive capabilities directly into products. Early prototypes suggest that personalized, context-aware interfaces can improve experiences for disabled users while also benefiting the broader population. ## Community-led co-design - Google follows the principle “Nothing About Us, Without Us,” involving people with disabilities as co-designers from the beginning. - Partnerships include RIT/NTID, The Arc of the United States, RNID, and Team Gleason. - These collaborations focus on real-world barriers and recognize the expertise of disability communities. - The approach also aims to create employment and economic opportunities for people who help shape the technology. ## Moving from reactive accessibility to adaptive interfaces - Google identifies an “accessibility gap” between the release of new features and the development of compatible assistive tools. - NAI addresses this by making accessibility native to the interface instead of adding it afterward. - Static navigation is replaced with dynamic, agent-driven modules that can interpret context and adjust the experience. ## Multi-system agents - An Orchestrator maintains shared context and delegates tasks to specialized sub-agents. - A Summarization Agent breaks down complex documents and assigns subtasks to expert agents. - A Settings Agent dynamically adjusts interface elements such as text size. - This structure lets users accomplish tasks without navigating complicated menus or searching for the right control. ## Multimodal interaction - Gemini-based prototypes combine voice, vision, and text rather than limiting accessibility to text-to-speech. - Live video can be converted into interactive audio descriptions. - Users can ask follow-up questions about specific visual details as events unfold. - Conversational interaction provides situational awareness and may reduce cognitive load. ## Proven prototypes - **StreetReaderAI** - Supports blind and low-vision users navigating physical spaces. - Combines an AI Describer that analyzes visual and geographic information with an AI Chat system for questions. - Maintains context so users can ask about previously encountered locations, such as the position of a bus stop. - **Multimodal Agent Video Player (MAVP)** - Makes audio description interactive rather than static. - Users can change the level of detail or ask questions during playback. - Uses an offline “dense index” of visual descriptions and retrieval-augmented generation (RAG) for fast responses. - **Grammar Laboratory** - Developed by RIT/NTID with Google.org support for American Sign Language and English learners. - Provides grammar instruction through ASL videos, English captions, spoken narration, and written transcripts. - Uses adaptive AI to customize lessons according to each student’s language preferences and interactions. ## The curb-cut effect - Accessibility features designed for people with significant constraints can benefit many other users. - Voice interfaces created for blind users may help sighted people who are multitasking. - AI synthesis and learning tools designed for people with learning disabilities can also support users who want information presented more clearly or flexibly. - NAI therefore treats accessibility as a source of better universal design, not as a specialized add-on. NAI’s central recommendation is to build accessibility into interfaces from the start, using multimodal AI, persistent context, and community-led design. The most effective systems will adapt to users while remaining accountable to the people whose needs they are intended to serve.

Read original

AI Agents Generative AI Accessibility Gemini+4

googleFeb 4, 20263 min readCurated summary

Sequential Attention: Making AI models leaner and faster without sacrificing accuracy

Sequential Attention is a greedy subset-selection method designed to make large machine-learning models smaller and faster without materially reducing accuracy. It selects features, layers, blocks, or weights one at a time using attention scores that are recalculated after each choice, allowing the model to account for nonlinear interactions and redundancy. By integrating selection into a single training process, it aims to retain the quality of traditional greedy methods while avoiding their prohibitive computational cost. ## The Subset-Selection Challenge - Feature selection removes irrelevant or redundant inputs, but finding the optimal subset is NP-hard. - Deep neural networks make selection harder because: - A feature that seems unimportant alone may be essential in combination with others. - Features that appear valuable individually may become redundant when selected together. - The same problem applies beyond input features: - Selecting embedding dimensions or chunks. - Pruning entries or blocks from weight matrices. - Choosing layers or other model components. ## How Sequential Attention Works - The method builds a subset step by step rather than weighting all candidates at once. - At each stage: - Previously selected candidates provide context. - Attention scores estimate the importance of every remaining candidate. - The highest-scoring candidate is added permanently. - The model recalculates scores to reflect the candidate’s marginal contribution. - This adaptive process can identify high-order nonlinear interactions that simpler filter methods may miss. - It uses softmax-based attention scores for ranking, but applies them sequentially instead of in a single pass. - Although greedy selection can be expensive when each candidate requires model retraining or evaluation, Sequential Attention performs selection within one training process, greatly reducing overhead. ## Main Benefits - **Efficiency and accuracy:** Candidates can be evaluated in parallel once attention scores are available, while sequential updates preserve adaptive selection. - **Interpretability:** Attention scores provide a view into which inputs or components the model considered important. - **Scalability:** The approach is intended for large candidate sets and modern deep-learning architectures. - **Reduced redundancy:** Recalculating scores after each selection helps prevent the model from repeatedly choosing overlapping or unnecessary components. ## Feature Selection - Traditional greedy feature selection repeatedly retrains or reevaluates a model for every possible feature at every step. - Sequential Attention replaces these expensive marginal-gain calculations with the model’s internal attention weights. - The algorithm: - Scores all unselected features. - Adds the feature with the highest score. - Reruns the model and updates the scores for the remaining features. - The method reportedly achieved state-of-the-art or competitive results across proteomics, image, and activity-recognition benchmarks. - Its one-pass implementation makes greedy-style selection substantially faster. - For linear regression, Sequential Attention is mathematically equivalent to Orthogonal Matching Pursuit (OMP), an established method with theoretical reliability and performance guarantees. ## Block Sparsification - Neural-network pruning removes unnecessary weights to reduce model size and improve deployment efficiency. - Block sparsification removes groups of parameters rather than individual weights, making the resulting sparsity more compatible with hardware acceleration. - Earlier approaches generally fell into two categories: - **Differentiable pruning**, which learns continuous importance proxies. - **Combinatorial optimization**, which searches directly for sparse structures. - The referenced work, “SequentialAttention++ for Block Sparsification,” aims to combine these differentiable and combinatorial approaches into a unified pruning framework. Sequential Attention is best understood as an adaptive, attention-based alternative to costly repeated subset searches. It is particularly promising when model components interact nonlinearly and when hardware-friendly sparsity or feature reduction is needed at scale.

Read original

Deep Learning Model Optimization Attention Mechanism Subset Selection+4

googleFeb 3, 20263 min readCurated summary

Collaborating on a nationwide randomized study of AI in real-world virtual care

Google and Included Health plan to launch, pending IRB approval, a nationwide randomized study of conversational AI in real-world virtual care. Unlike prior simulated or small feasibility studies, it will prospectively evaluate AI with consented patients across varied conditions and locations, comparing it with standard clinical practice. The goal is to generate rigorous evidence about safety, usefulness, limitations, and impact on patients and clinicians. ## Moving from Simulation to Real-World Evaluation - Earlier research demonstrated clinician-level capabilities in simulated consultations and retrospective analyses. - A feasibility study with Beth Israel Deaconess Medical Center began testing conversational AI in clinical workflows, using measures such as safety-supervisor interruptions. - The new study will advance beyond feasibility through: - A randomized controlled design - Nationwide recruitment - Consented participants - Real patients, clinical concerns, and virtual-care workflows - Controlled comparison with standard practice ## A Phased Approach to Medical AI Research - Google argues that medical AI should be evaluated with evidence standards similar to other medical interventions. - Each research phase adds information about: - Patient and clinician experiences - Safety - Usefulness - The AI system’s capabilities and limitations - Results from each stage are intended to guide safer, more responsible development and deployment. ## Foundational Research Behind the Study ### Diagnostic and Management Reasoning - The AMIE system was developed to handle medical interviews and clinical reasoning. - Studies with patient actors and synthetic cases found that AMIE could match or exceed primary care physicians in simulated diagnostic accuracy and conversation quality. - Later work expanded the system to: - Longitudinal disease management - Clinical-guideline and patient-history reasoning - Investigation and treatment planning - Interpretation of multimodal evidence ### Personalized Health Insights - Research on the Personal Health Agent examined how AI could interpret personal health data, including sleep and activity information from wearables. - Its multi-agent architecture combined the roles of: - Data scientist - Medical domain expert - Health coach - This work informed Fitbit Labs tools such as Symptom Checker and Medical Records Navigator and Plan for Care. ### Navigating Health Information - Google’s “wayfinding” AI research explored how conversational agents can help people find and understand health information. - The system uses proactive guidance, goal recognition, and tailored conversations to make health information searches more practical and useful. ## Practical Conclusion The partnership with Included Health represents a transition from demonstrating what medical AI can do in controlled environments to measuring how it performs at scale in actual care. A nationwide randomized trial could provide the evidence needed to determine whether conversational AI can safely improve virtual care and expand access to medical expertise.

Read original

Conversational Ai Healthcare Ai Diagnostic Reasoning Multimodal Models+3

googleJan 28, 20263 min readCurated summary

Towards a science of scaling agent systems: When and why agent systems work

AI agent systems do not improve simply by adding more agents. Google Research’s evaluation of 180 configurations found that coordination helps substantially on parallelizable tasks but can severely hurt sequential workflows and tool-heavy tasks. The study proposes measurable design principles and a predictive model that selected the best architecture for 87% of unseen tasks. ## Defining Agentic Tasks The study distinguishes agentic tasks from static benchmarks by requiring: - Sustained, multi-step interaction with an external environment. - Iterative information gathering under partial observability. - Adaptive strategy changes based on environmental feedback. Researchers tested five architectures across Finance-Agent, BrowseComp-Plus, PlanCraft, and Workbench: - **Single-agent:** One agent handles reasoning and actions sequentially. - **Independent:** Agents work in parallel without communication and combine results at the end. - **Centralized:** An orchestrator delegates work and synthesizes outputs. - **Decentralized:** Agents communicate directly in a peer-to-peer network. - **Hybrid:** Hierarchical oversight is combined with peer coordination. ## Coordination Must Match the Task - Multi-agent systems produced very different results across GPT, Gemini, and Claude models. - On parallelizable financial reasoning tasks, centralized coordination improved performance by **80.9%** over a single agent. - Parallel agents can independently analyze areas such as revenue, costs, and market comparisons before combining their findings. - On sequential planning tasks, every multi-agent architecture performed worse, with declines of **39–70%**. - Communication and synchronization overhead can fragment reasoning and consume the available cognitive budget. ## The Tool-Coordination Trade-off - As tasks require more tools, coordinating multiple agents becomes increasingly expensive. - Tool-heavy systems, such as coding agents with access to 16 or more tools, face a disproportionate coordination “tax.” - Adding agents is therefore especially risky when actions must be tightly ordered or frequently synchronized. ## Architecture and Reliability - Architecture affects not only performance but also how errors spread. - Independent agents amplified errors by up to **17.2×**, because no mechanism checked their intermediate results. - Centralized systems limited error amplification to **4.4×**. - An orchestrator acts as a validation bottleneck, detecting and containing mistakes before they propagate. ## Predicting the Best Architecture - The researchers built a predictive model using properties such as task decomposability and tool count. - The model achieved an **R² of 0.513**. - It correctly predicted the optimal coordination strategy for **87% of unseen task configurations**. - These results point toward systematic, task-driven agent design rather than relying on the assumption that more agents are always better. For practical deployments, choose architecture based on the task: use coordinated parallel agents for decomposable work, simpler sequential systems for tightly ordered reasoning, and centralized oversight when reliability and error containment are priorities.

Read original

Large Language Models AI Agents Multi Agent Systems Llm Evaluation+3

googleJan 27, 20263 min readCurated summary

ATLAS: Practical scaling laws for multilingual models

ATLAS introduces practical scaling laws for training multilingual language models, addressing the lack of public guidance for non-English systems. Based on 774 runs covering 400+ languages and models from 10M to 8B parameters, it predicts how to combine languages, data, and model capacity efficiently. The study finds strong cross-lingual transfer, a manageable multilingual capacity tax, and clear trade-offs between fine-tuning and pretraining from scratch. ## Adaptive Scaling for Multilingual Mixtures - ATLAS extends traditional scaling laws with: - A cross-lingual transfer matrix identifying helpful language combinations. - Rules for scaling model size and data as supported languages increase. - Guidance on whether to pretrain from scratch or fine-tune a multilingual checkpoint. - It separates training data into: - The target language. - Similar “transfer languages,” such as Spanish, Portuguese, and Italian for Catalan. - All other languages. - This allows ATLAS to estimate which languages help or hinder a target language. ## Evaluation Across Languages and Model Sizes - Experiments used MADLAD-400, spanning more than 750 monolingual, bilingual, and multilingual runs. - ATLAS outperformed earlier scaling laws when predicting performance for new: - Model sizes. - Data volumes. - Language mixtures. - Optimal scaling patterns were broadly similar across English, French, Russian, Chinese, Hindi, and Swahili. - Multilingual vocabularies and data impose a compute-efficiency tax, particularly for English. - Low-resource languages eventually encounter data repetition, causing their scaling curves to bend upward. ## Cross-Lingual Transfer - The transfer matrix measures how training on one language affects another. - Examples of strong transfer include: - Norwegian benefiting from Swedish and German. - Malay benefiting from Indonesian. - Arabic benefiting from Hebrew. - English, French, and Spanish are broadly useful training languages, partly because of their large, diverse, and high-quality web corpora. - Shared writing systems and language families are the strongest predictors of positive transfer, with statistical significance of p < .001. - Transfer is asymmetric: language A may help language B more than B helps A. - The results replace informal language-selection assumptions with empirical data. ## Scaling the Number of Supported Languages - ATLAS formalizes the “curse of multilinguality,” in which adding languages can reduce performance because model capacity is limited. - Adding languages creates a modest capacity cost but also substantial positive transfer. - To support twice as many languages, the study recommends approximately: - 1.18× larger model size. - 1.66× more total training data. - Although each language receives less data individually, cross-lingual synergies offset much of the degradation. ## Pretraining Versus Fine-Tuning - Fine-tuning a strong multilingual “Unimax” checkpoint generally delivers the best early performance for the least additional compute. - Pretraining from scratch can eventually produce better results when substantially more tokens are affordable. - For 2B-parameter models, the crossover typically occurs between roughly 144B and 283B tokens, depending on the language. - The supplied article ends while discussing how ATLAS further models this crossover point. ## Practical Recommendation Use ATLAS to select language mixtures based on measured transfer rather than intuition. Fine-tune an existing multilingual checkpoint under tight compute budgets, but consider training from scratch when enough data and compute are available to pass the language-specific crossover point.

Read original

Large Language Models Fine Tuning Scaling Laws Pre Training+4

googleJan 23, 20263 min readCurated summary

Introducing GIST: The Next Stage in Smart Sampling | Google Research

GIST is a data-subset selection algorithm designed to balance diversity and utility when training on massive datasets. It converts the difficult diversity–utility optimization problem into a series of thresholded graph problems and uses a bicriteria greedy strategy to find a high-quality subset efficiently. The algorithm guarantees at least half the value of the optimal solution, while the authors prove that improving beyond a 0.56 approximation is NP-hard. ## Why Smart Sampling Is Difficult - Large ML systems need to process datasets that are increasingly expensive to store, analyze, and train on. - Subset selection aims to choose a smaller but representative set of examples. - **Diversity** prevents redundant selections by maximizing the minimum distance between selected points, typically in embedding space. - **Utility** measures how much relevant or unique information the subset provides, modeled using monotone submodular functions. - Optimizing both objectives simultaneously is NP-hard: - A diversity-only method may select irrelevant examples. - A utility-only method may select many similar examples from one highly relevant cluster. ## How GIST Works ### Diversity Thresholding - GIST fixes a candidate minimum distance rather than optimizing the distance directly. - It builds a graph in which two data points are connected when they are closer than the chosen threshold. - Connected points are considered too similar to coexist in the selected subset. - Selecting points that are not connected enforces the desired spacing between examples. ### Utility-Constrained Independent Sets - For each threshold, GIST seeks a high-utility independent set: a group of points with no edges between them. - This corresponds to selecting valuable examples without choosing mutually conflicting or overly similar points. - Because maximum independent set is NP-complete and lacks practical general-purpose approximation algorithms, GIST uses a specially designed bicriteria greedy method. - The algorithm repeatedly selects high-scoring points and excludes nearby candidates, effectively creating “no-go zones” around selected data. ### Searching Across Thresholds - GIST evaluates all relevant distance thresholds derived from the dataset. - It greedily constructs a candidate subset for each threshold. - It returns the best candidate found across these runs. - If the optimal solution achieves minimum distance \(d\), GIST obtains comparable utility while guaranteeing a minimum distance of roughly \(d/2\). ## Theoretical Guarantees - GIST is presented as the first algorithm with a strong provable guarantee for this diversity–utility tradeoff. - Its output has at least half the value of the absolute optimum. - The authors also prove that finding a solution worth more than 0.56 of the optimum is NP-hard. - These results provide a mathematical guarantee that GIST is not merely producing empirically good subsets, but making a bounded tradeoff between informativeness and coverage. ## Practical Evaluation - GIST was evaluated against several common subset-selection approaches in ML applications. - Comparisons included: - **Random**, a simple baseline that often provides reasonable diversity. - **Margin**, which selects examples the model is uncertain about but does not explicitly promote diversity. - **k-center**, which minimizes representation gaps by keeping all data points close to a selected representative. - **Submod**, which combines utility with an older formulation of diversity. - The experiments, including image-classification benchmarks, reportedly show that GIST outperforms state-of-the-art alternatives while retaining formal guarantees. GIST is therefore a practical choice when subset selection must preserve both broad data coverage and task relevance. Its main advantage is combining competitive real-world performance with a clear approximation guarantee, rather than relying solely on heuristic results.

Read original

Machine Learning Approximation Algorithms Subset Selection Data Sampling+4

googleJan 22, 20263 min readCurated summary

Small models, big results: Achieving superior intent extraction through decomposition

Small multimodal models can outperform much larger models at extracting user intent from UI interaction trajectories when the task is decomposed. Google’s approach first summarizes each screen and interaction, then derives an overall intent from those summaries. This enables accurate, faster, and more privacy-preserving on-device understanding without sending sensitive UI data to servers. ## Why On-Device Intent Understanding Matters - Understanding what users are doing across mobile and web interfaces can help agents anticipate useful next actions. - Large multimodal models perform well but often require server-side processing, introducing latency, cost, and privacy risks. - The goal is to make intent understanding practical with smaller models running directly on devices. ## Two-Stage Intent Extraction ### Screen and Interaction Summaries For each interaction, a small multimodal model examines a sliding window of three screens: the previous, current, and next screens. It generates information about: - Salient context on the current screen. - Actions the user just performed. - A speculation about what the user is trying to accomplish. This converts raw screenshots and actions into structured, manageable event summaries. ### Intent Extraction from Summaries A fine-tuned small model then processes the sequence of summaries and produces a single concise intent statement. The authors improve this stage through: - **Fine-tuning:** Training on examples of high-quality intent statements helps the model retain relevant details and discard noise. - **Label preparation:** Training intents are stripped of details absent from the summaries, reducing hallucinated information. - **Removing speculation:** Speculative fields help create richer individual summaries but are excluded from the second stage because they can confuse intent extraction. ## Evaluation with Atomic Facts - The authors use the Bi-Fact evaluation method to compare predicted intents with reference intents. - Each intent is split into indivisible “atomic facts,” such as “a one-way flight” or the separate origin and destination in a flight request. - The method measures: - **Recall:** How many reference facts were captured. - **Precision:** How many predicted facts are supported by the reference. - **F1:** The balance between precision and recall. - Tracking facts through both stages also reveals where details are lost or hallucinated. ## Results - The decomposed method outperformed chain-of-thought prompting and end-to-end fine-tuning. - Improvements held across both mobile and web interaction trajectories. - Results were consistent across Gemini and Qwen2 base models. - Gemini 1.5 Flash 8B achieved results comparable to Gemini 1.5 Pro while offering substantially lower cost and faster processing. - On mobile data, the small-model approach approached the performance of the larger Gemini Pro model. The study suggests that decomposing intent understanding into local summarization followed by sequence-level extraction is an effective path toward accurate, private, and efficient on-device assistants. As mobile hardware and small models improve, this technique could support a broad range of assistive features.

Read original

Gemini Natural Language Processing Multimodal AI Fine Tuning+4

googleJan 14, 2026Original article

Unlocking health insights: Estimating advanced walking metrics with smartwatches (opens in new tab)

Google researchers have validated that smartwatches are a highly reliable and accurate platform for estimating complex spatio-temporal gait metrics, rivaling the performance of smartphone-based methods. By utilizing a multi-head deep learning model, the study demonstrates that wrist-worn devices can provide continuous, lab-grade health insights into a user's walking speed, step length, and balance without requiring the specific pocket placement or specialized laboratory equipment previously necessary for such data. ## Multi-Head Deep Learning for Wrist-Based Sensors * The researchers developed a temporal convolutional network (TCN) architecture designed to process raw inertial measurement unit (IMU) data, specifically 3-axis accelerometer and gyroscope signals sampled at 50 Hz. * Unlike traditional models that only track temporal events and are prone to integration drift, this multi-head approach directly estimates both unilateral and bilateral metrics simultaneously. * The model architecture extracts embeddings from the IMU signals and concatenates them with user height (a demographic scalar input) to improve the precision of spatial predictions. * The system estimates a comprehensive suite of metrics, including gait speed, double support time (the proportion of time both feet are on the ground), step length, swing time, and stance time. ## Large-Scale Validation and Study Protocol * To ensure rigorous results, the study involved a diverse cohort of 246 participants across two international sites, generating approximately 70,000 walking segments. * Ground truth measurements were captured using a professional-grade Zeno Gait Walkway system to provide high-precision reference data for comparison. * The study protocol included various walking conditions to test the model's versatility: a self-paced six-minute walk test (6MWT), fast-paced walking, and induced physical asymmetry created by wearing hinged knee braces at specific angles. * Researchers employed a five-fold cross-validation strategy, ensuring that all data from a single participant remained within a single split to prevent data leakage and ensure the model generalizes to new users. ## Clinical Validity and Comparative Performance * Smartwatch estimates demonstrated strong validity and excellent reliability, with Pearson correlation coefficients (r) and intraclass correlation coefficients (ICC) exceeding 0.80 for most metrics. * Performance comparisons showed non-significant differences in Mean Absolute Percentage Error (MAPE) between the Pixel Watch and Pixel phone, establishing the smartwatch as a viable alternative to smartphone-based tracking. * While double support time showed slightly lower but acceptable reliability (ICC 0.56–0.60), other metrics like step length and gait speed proved highly consistent across different walking speeds and styles. * The model’s success suggests that smartwatches can effectively bridge the gap in gait analysis, providing a more practical and consistent platform for continuous health tracking than handheld devices. This research establishes smartwatches as a powerful tool for longitudinal health monitoring, enabling the detection of neurological or musculoskeletal changes through passive, continuous gait analysis in everyday environments.

AI Deep Learning Wearables Health Tech+5

googleJan 12, 2026Original article

Dynamic surface codes open new avenues for quantum error correction (opens in new tab)

Google Research has demonstrated the operation of dynamic surface codes for quantum error correction, marking a significant shift from traditional static circuit architectures. By alternating between different circuit constructions and re-tiling "detecting regions" in each cycle, these dynamic circuits offer greater flexibility to avoid hardware defects and suppress correlated errors. Experimental results on the Willow processor show that these methods can match the performance of static codes while significantly simplifying the physical design and fabrication of quantum chips. ## Error Triangulation via Dynamic Detecting Regions Quantum error correction (QEC) functions by localizing physical errors within specific "detecting regions" over multiple cycles to prevent them from affecting logical information. While standard surface codes use a static, square tiling for these regions, dynamic codes periodically change the tiling pattern. * Dynamic circuits allow the system to "deform" the detecting regions in spacetime, providing multiple perspectives to triangulate errors. * This approach enables the use of different gate types and connectivity layouts that are not possible with fixed, repetitive cycles. * The flexibility of dynamic re-tiling allows the system to sidestep common superconducting qubit issues such as "dropouts" (failed qubits or couplers) and leakage out of the computational subspace. ## Quantum Error Correction on Hexagonal Lattices Traditional square lattices require each physical qubit to connect to four neighbors, which creates significant overhead in wiring and coupler density. Dynamic circuits enable the use of a hexagonal lattice, where each qubit only requires three couplers. * The hexagonal code alternates between two distinct cycle types, utilizing one of the three couplers twice per cycle to maintain error detection capabilities. * Testing on the Willow processor showed that scaling the hexagonal code from distance 3 to 5 improved the logical error rate by a factor of 2.15, matching the performance of standard static circuits. * Reducing coupler density simplifies the optimization of qubit and gate frequencies, leading to a 15% improvement in simulated error suppression compared to four-coupler designs. ## Walking Circuits to Mitigate Leakage Superconducting qubits are prone to "leakage," where a qubit exits its intended computational states (0 and 1) into a higher energy state (2). In static circuits, repeated measurements on the same physical qubits can cause these leakage errors to accumulate and spread. * "Walking" circuits solve this by shifting the roles of data and measurement qubits across the lattice in each cycle. * By constantly moving the location where errors are measured, the circuit effectively "flushes out" leakage and other correlated errors before they can damage logical information. * Experiments confirmed that walking circuits achieve error suppression equivalent to static circuits while offering a more robust defense against long-term error correlations. ## Flexibility with iSWAP Entangling Gates Most superconducting quantum processors are optimized for Controlled-Z (CZ) gates, but dynamic circuits prove that QEC can be effectively implemented using alternative gates like iSWAP. * The research team demonstrated a dynamic surface code that utilizes iSWAP gates, which are native to many quantum hardware architectures. * This flexibility ensures that QEC is not tethered to a specific gate set, allowing hardware designers to choose entangling operations that offer the highest physical fidelity for their specific device. The move toward dynamic surface codes suggests a future where quantum processors are more resilient to manufacturing imperfections. By adopting hexagonal layouts and walking circuits, developers can reduce hardware complexity and mitigate physical noise, providing a more scalable path toward fault-tolerant quantum computing.

Quantum Computing Superconducting Qubits Quantum Error Correction Logical Qubits+3

Filter by tag