google·Jun 12, 2026·3 min readCurated summary

Research into how AI can help users understand skin conditions

Google Research examined how AI tools can help non-experts understand skin conditions and decide what to do next. In a large study, AI substantially improved people’s ability to identify possible conditions, but it did not reliably improve the accuracy of recommended next steps. The research therefore argues that dermatology AI should be designed around human decision-making, safety, and clear guidance—not diagnosis alone. ## Why Dermatology AI Needs Human-Centered Research - More than half of adults use the Internet for health information, and about one-third use AI. - People often lack the medical vocabulary needed to search effectively—for example, searching for “red dots on legs” instead of “palpable purpura.” - Google Research has developed dermatology AI models, validated their generalization, and released datasets such as SCIN. - Earlier research found that online tools can improve condition recognition without necessarily helping people choose appropriate next steps. - The researchers emphasize studying how people interpret and act on AI-generated information. ## Large-Scale Evaluation of an AI Information Tool - A JAMA Dermatology study involved 2,345 participants reviewing de-identified skin-condition cases with images and structured medical histories. - Participants were assigned to one of three groups: - **Standard-search control:** Used familiar text-based search tools. - **AI group:** Used a prototype showing 3–7 AI-predicted conditions, textbook images, and information about symptoms and treatments. - **“Wizard of Oz” control:** Used the same interface, but with dermatologist-provided differential diagnoses presented as if generated by AI. - The AI interface increased participants’ willingness to name a condition: - More than 62% attempted a diagnosis with AI. - Only 41% did so using standard search. - Accuracy also improved: - AI users correctly identified a matching condition about 23% of the time. - Standard-search users achieved 8%. - The “perfect-prediction” interface reached 36%, showing that even accurate candidate lists did not make users nearly perfect. - AI users reported greater confidence, satisfaction, and satisfaction with the time spent searching. ## Identifying a Condition Does Not Guarantee Safe Action - The prototype intentionally avoided prescribing actions or making individualized diagnoses. - Treatment information was dermatologist-written and based on the condition name, rather than the severity or details of the specific case. - Choosing the right next step—such as home care, routine care, or urgent evaluation—remained difficult. - Next-step accuracy improved only slightly in the “Wizard of Oz” group, from 60% in the standard-search control to 63.5%. - The standard AI group showed no statistically significant improvement. - AI users were slightly more likely than control participants to recommend a less urgent action than dermatologists would: 30% versus 27%. - These findings show that identifying possible conditions is insufficient without stronger safety-oriented guidance. ## Studying Real Users and Diverse Communities - The researchers also conducted a qualitative study, published at ACM CHI, to examine how people use AI for their own active skin concerns. - The project partnered with Stanford’s Healthcare AI Applied Research Team and the Santa Clara Family Health Plan. - The community included many Medi-Cal users who rely on a healthcare safety net. - Researchers aimed to gather richer feedback than survey-based studies provide by observing real-world use. - Because participants spoke four primary languages, the application was translated into those languages, with multilingual volunteers or staff available to support communication. AI can make dermatology information easier to find and improve recognition of possible conditions, but it should not be treated as a substitute for professional judgment. Future tools should focus equally on urgency assessment, personalized context, uncertainty, and clear recommendations for when to seek medical care.

Read original(opens in new tab)

AIMachine LearningHealthcare AiDifferential Diagnosis+3

google·Jun 10, 2026·3 min readCurated summary

New framework for auditing machine unlearning

Machine unlearning must be auditable, but conventional two-sample tests can be expensive, insensitive to localized changes, and prone to false positives. Google Research introduces Regularized f-Divergence Kernel Tests, which compare an unlearned model relative to both a safely retrained model and the original compromised model. The framework adaptively detects meaningful privacy or unlearning failures while controlling false positives and improving sensitivity as more samples become available. ## Why Standard Auditing Methods Fall Short - Auditors often lack access to model internals or the original training data, so they must assess unlearning by querying models and comparing outputs. - Standard two-sample tests require many samples to distinguish genuine violations from random variation in large models. - Maximum mean discrepancy (MMD) is effective for broad distribution shifts but can miss: - Rare, highly localized outliers - Non-smooth differences - Changes that require careful kernel-bandwidth tuning - Existing methods often require manual selection of statistics and hyperparameters. - Comparing an unlearned model only with a retrained reference can produce false positives because independently trained models may differ due to factors such as batch size. - Perfect retrain equivalence is generally impossible for local unlearning methods, since they retain traces of the original training process. ## Regularized f-Divergence Kernel Tests - The framework uses a relative-distance test to determine whether an unlearned model is closer to: - A safely retrained model that never saw the data - The original model that still contains the data - It supports multiple f-divergences for different types of distribution changes: - **Chi-squared and KL divergence:** Useful for smooth or localized differences, including outliers. - **Hockey-stick divergence:** Designed for privacy and unlearning thresholds, ignoring differences below a defined safety budget while flagging meaningful breaches. - Kernel regularization makes high-dimensional divergence estimation computationally tractable. - The adaptive procedure selects suitable divergences and hyperparameters automatically. - It avoids the need for sample splitting. - The authors theoretically show that false-positive rates are controlled for any sample size, while false-negative risk approaches zero as the number of samples increases. ## Experimental Evaluation - The framework was tested on: - Synthetic perturbed-uniform distributions - The Expo1D outlier-detection task from high-energy physics - High-energy physics provides a stringent benchmark because detecting rare particles requires identifying extremely subtle distributional differences. - For differential-privacy auditing, the researchers compared outputs generated from datasets differing by one record: - A private mechanism should produce statistically indistinguishable samples. - A non-private mechanism should reveal a detectable difference. - For machine-unlearning evaluation, the researchers used a three-sample relative test rather than simply comparing an unlearned model with a gold-standard retrained model. - The evaluation included methods such as Selective Synaptic Dampening, pruning, and random-label techniques. ## Practical Recommendation Auditors should use relative, divergence-based testing rather than relying on a single comparison with a retrained reference model. Regularized f-divergence kernel tests offer a more flexible way to detect localized privacy leaks and unlearning failures while reducing false alarms and costly manual tuning.

Read original(opens in new tab)

Differential PrivacyMachine UnlearningTwo Sample TestingF Divergence+3

google·Jun 5, 2026·3 min readCurated summary

Unlocking dependable responses with Gemini Enterprise Agent Platform’s Agentic RAG

Google’s Agentic RAG framework extends traditional retrieval-augmented generation to handle complex, multi-source enterprise questions. Its multi-agent system plans searches, rewrites queries, routes them across data sources, and iteratively retrieves missing information instead of stopping after one pass. A Sufficient Context Agent verifies that the evidence supports every part of the request, improving factual accuracy by up to 34% on evaluated datasets. ## Why Standard RAG Falls Short - Conventional RAG typically performs one retrieval step before generating an answer. - Enterprise information is often distributed across separate data sources and requires multi-hop reasoning. - For example, a project document may contain a server ID, while the server’s specifications exist in another database. - Without a second search, the system may produce an incomplete answer or incorrectly conclude that the information is unavailable. ## Multi-Agent Planning, Rewriting, and Routing The framework divides research into specialized roles: - **Orchestrator:** Determines whether the request requires multiple steps and delegates tasks. - **Planner Agent:** Maps the information needed and identifies which sources to search. - **Query Rewriter:** Converts a broad question into targeted search queries. - **Search Fanout Agent:** Sends those queries to multiple retrieval systems. - **Synthesis Agent:** Combines the gathered evidence into the final response. This architecture is designed to coordinate complex searches rather than treat retrieval as a single matching operation. ## Iterative Retrieval with Sufficient Context The central innovation is persistence: the system detects when its evidence is incomplete and continues searching. - The **Sufficient Context Agent** reviews: - Retrieved text snippets - An intermediate draft answer - The original user request - It identifies which requested elements are supported and which are missing. - Instead of merely reporting insufficient information, it produces specific feedback describing the gap and suggesting what to search for next. - This feedback drives another query-rewriting and retrieval cycle. - Retrieval stops only when the system determines that the available context is sufficient for a grounded answer. ## Example: Patient Discharge Information For a request involving medications, dietary restrictions, and allergic reactions, the workflow proceeds as follows: - The Root Agent delegates the task to Pharmacy, Nutrition, and Clinical Notes sub-agents. - The initial search finds medication and diet information but no obvious allergy records. - The Sufficient Context Agent flags the missing allergy information. - The Query Rewriter generates focused searches such as “rashes” or “adverse events.” - A deeper search finds the missing evidence. - The system performs a final context check before producing the doctor’s summary. ## Evaluation Results - The framework was evaluated on the FramesQA benchmark, which contains multi-hop questions. - It reportedly improved factuality accuracy by up to 34% compared with standard approaches. - Google also tested it on proprietary internal datasets and observed better grounding and reasoning accuracy across several domain-specific tasks. - The approach is hosted through Gemini Enterprise Agent Platform’s Cross-Corpus Retrieval capability. Agentic RAG is most useful when answers depend on several documents, databases, or reasoning steps. For enterprise deployments, iterative retrieval and explicit context verification can reduce incomplete answers and unsupported guesses, though they add orchestration and retrieval overhead.

Read original(opens in new tab)

Large Language ModelsRAGMulti Agent SystemsAgentic Rag+1

google·Jun 4, 2026·3 min readCurated summary

Towards passive heart health monitoring via smartphone camera

PHRM is a Google Research system that passively estimates heart rate and daily resting heart rate using short facial videos captured by a smartphone’s front-facing camera after face unlock. It applies remote photoplethysmography and efficient deep learning, achieving wearable-comparable accuracy in laboratory and real-world settings. A central contribution is its deliberate validation across skin tones, including substantial representation of participants with dark skin. ## Passive Heart Monitoring with a Smartphone - PHRM records approximately eight seconds of facial video during everyday phone use. - It estimates heart rate from changes in how light reflects from the skin as blood pulses through facial tissue. - The system uses: - Temporal shift convolutional neural networks for efficient on-device prediction. - Confidence scores to filter unreliable measurements. - Kalman filtering and daily aggregation to estimate resting heart rate. - Unlike earlier smartphone approaches requiring a finger over the camera, PHRM works passively in the background. ## Designing for Skin-Tone Inclusivity - Existing remote PPG research often relied on small, controlled studies and underrepresented people with dark skin. - Google trained PHRM on more than 350,000 video clips from nearly 700 participants across laboratory and real-world conditions. - Dataset targets based on the Monk Skin Tone scale ensured: - At least 25% representation of light and medium skin tones. - At least 33% representation of dark skin tones. - The researchers required heart-rate error differences between skin-tone groups to remain below five percentage points. ## Laboratory Validation - The model was trained using synchronized facial video and ECG recordings from 365 diverse participants. - On a separate 104-person test set, PHRM achieved mean absolute percentage error below 10% for every skin-tone group after confidence filtering. - It outperformed 15 leading published remote-PPG models and was the only evaluated model to meet the under-10% target across all groups. ## Real-World Smartphone Study - In an eight-day free-living study, 231 participants used their own phones normally while wearing an ECG chest strap and Fitbit Charge 6. - The research app captured an average of 231 facial clips per participant per day after face unlock. - Participants reviewed and explicitly authorized uploads each day; recordings were sent to secure, encrypted servers. - On a held-out group of 101 participants: - Overall MAPE was 6.09% after confidence gating. - MAPE was 5.04% for light skin, 5.12% for medium skin, and 7.84% for dark skin. - The average underestimation was only 0.64 beats per minute. - The model again outperformed the 15 comparison systems. ## Resting Heart Rate and Research Access - By combining reliable heart-rate estimates throughout the day, PHRM produced daily resting-heart-rate estimates with mean absolute error below five beats per minute compared with a wearable tracker. - Google released the largest and most diverse publicly available smartphone-video dataset for remote PPG research, along with a pre-trained “PHRM-mini” model. - Access is available to qualified researchers. PHRM demonstrates that smartphones could broaden access to passive cardiovascular monitoring without requiring dedicated wearable hardware. Before clinical use, the system would still require continued validation, careful privacy protections, and assessment across additional devices and populations.

Read original(opens in new tab)

Deep LearningComputer VisionConvolutional Neural NetworksPhotoplethysmography+3

google·Jun 3, 2026·3 min readCurated summary

The next chapter in flood resilience: Open sourcing Google’s hydrology framework

Google Research has open-sourced the hydrology framework behind its Flood Hub river forecasts. The Python/PyTorch package lets researchers and national forecasting agencies train AI models with global and local data while retaining control over their information. Google argues that open access, local expertise, and interoperable tools can make advanced flood warnings more accurate, affordable, and widely deployable. ## The Open-Source Hydrology Framework - The framework is available on GitHub under an Apache 2.0 license. - It provides model architectures, training pipelines, documentation, and tutorials. - Users can train models with climate, soil, topography, land-cover, and weather data. - Historical river observations come from the open Caravan dataset, which agencies can extend with local measurements. - The package is built with PyTorch and is intended for both researchers and operational forecasters. ## Model Versions and Improvements - The release includes: - The original model used in Google’s 2024 benchmarking study. - An upgraded v2 model currently used for real-time global forecasts in Flood Hub. - The v2 model uses a multi-input ME-LSTM architecture. - Separate networks embed different meteorological products before combining them in an LSTM. - Inputs include GraphCast, ECMWF forecasts, NASA IMERG satellite rainfall estimates, and NOAA CPC precipitation data. - Benchmarking showed the newer model extends the reliable forecast horizon by: - Six days in gauged river basins. - One day in ungauged basins. ## Local Data and Operational Forecasting - Agencies can fine-tune models for specific watersheds using local observations and expert knowledge. - The approach supports the integration of Indigenous and Local Knowledge, which the World Meteorological Organization says is still rarely incorporated systematically. - Models are designed to be relatively inexpensive and easier to train than traditional conceptual hydrological systems. - Local organizations can preserve control over their data while adapting the models to regional conditions. ## Partnership with the Czech Hydrometeorological Institute - Google worked with CHMI to validate the model against locally calibrated traditional forecasting models. - CHMI created an adapter connecting the framework to Delft-FEWS, a widely used operational forecasting platform. - This integration demonstrates how machine-learning forecasts can fit into existing workflows used by government agencies, NGOs, and private organizations. - The partnership provides a practical model for other national hydrological services. ## Broader Flood-Resilience Goals - Open-source distribution could help resource-constrained regions access advanced forecasting without expensive infrastructure. - The framework is intended to support capacity building for early-warning systems worldwide. - Google presents the release as a way to let the global hydrology community reproduce, improve, and localize its research. National hydrological agencies and researchers should evaluate the open-source framework using their own watershed data, integrate it with existing forecasting systems, and validate its predictions against established local models before operational deployment.

Read original(opens in new tab)

Machine LearningPythonPytorchFlood Forecasting+3

google·May 28, 2026·3 min readCurated summary

A New Era of Innovation: Google Research at I/O 2026

Google’s I/O 2026 research announcements present AI as an “agentic” amplifier of human ingenuity, particularly in science and healthcare. New systems such as Gemini for Science, ERA, Co-Scientist, and Gemini Deep Think are designed to generate hypotheses, write and optimize code, evaluate evidence, and solve difficult research problems. Google also highlighted health-focused AI that supports users before, during, and after medical visits, while emphasizing collaboration, validation, and responsible deployment. ## AI-Driven Scientific Discovery - **Gemini for Science** is a suite of experimental tools built from Google Research and developed with Google Cloud, Google DeepMind, and Google Labs. - **Empirical Research Assistance (ERA)** acts as a code-optimizing research engine: - Proposes concepts and writes software. - Evaluates results against a defined scoring system. - Uses tree search to test thousands of code variants. - Has supported work in neuroscience, cosmology, respiratory-illness forecasting, and California runoff prediction. - **Co-Scientist** is a Gemini-based multi-agent collaborator that generates, evaluates, and refines hypotheses. - Researchers have applied it to antimicrobial resistance, plant immunity, and liver fibrosis. - **Computational Discovery**, combining ERA and AlphaEvolve, runs thousands of code variations in parallel to test scientific models and hypotheses more quickly. - **Hypothesis Generation** uses a multi-agent “idea tournament” to debate and rank research ideas, with clickable citations supporting claims. - **Literature Insights**, powered by NotebookLM, helps researchers synthesize large bodies of scientific literature. - **Science Skills** can automate specialist workflows such as structural bioinformatics and genomic analysis on agentic coding platforms. ## AI for Peer Review and Advanced Reasoning - Google is piloting the **Paper Assistant Tool (PAT)** for scientific peer review. - PAT has experimentally reviewed more than 10,000 papers for conferences including ICML, STOC, and NeurIPS. - Its feedback has helped authors identify theoretical gaps and design additional experiments. - **Gemini Deep Think** has been used with mathematicians, physicists, and computer scientists to address open problems involving network deadlocks, optimization, machine-learning behavior, auction theory, and cosmic-string singularities. ## Advancing Health with AI - Google’s health research focuses on supporting people throughout the full healthcare journey, from understanding symptoms and preparing for appointments to interpreting medical records. - Research contributions underpin the **Google Health app** and **Google Health Coach**, with the app beginning rollout to existing Fitbit users. - **Symptom AI** investigates how conversational AI can reason about information relevant to a person’s symptoms. - A Fitbit-based study included 13,917 participants. - In blind comparisons, clinicians preferred Symptom AI’s differential diagnoses roughly twice as often as those produced by other clinicians. - The **Plan for Care** pilot involved 1,779 participants preparing for doctor visits. - Compared with baseline systems, 15% more users felt prepared. - 13% more users felt confident they could make effective use of their appointment. - Google is also studying personal health large language models and the use of personal health record data to improve health guidance. Google’s announcements point toward research systems that actively experiment, collaborate, and reason rather than merely retrieve information. Their practical value will depend on continued scientific validation, clinician involvement, privacy protections, and careful expansion from experimental tools into real-world use.

Read original(opens in new tab)

Machine LearningGenerative AIGeminiMulti Agent Systems+2

google·May 27, 2026·3 min readCurated summary

Private analytics via zero-trust aggregation

Google presents a private analytics architecture combining one-shot cryptographic secure aggregation with trusted execution environments (TEEs). The design follows a zero-trust model: cryptography prevents individual data from being reconstructed, while TEE attestation verifies that approved, publicly auditable code is running. It aims to provide useful population-level insights without exposing raw user data, even if hardware protections are compromised. ## Why Private Analytics Matters - On-device AI protects user information, but developers still need to measure performance across millions of devices. - Aggregated analytics can reveal: - Model drift, such as difficulty handling new language or slang. - Hidden biases across regions, environments, or demographic conditions. - Real-world error rates and user reactions to model outputs. - Google already uses federated analytics in products including Pixel Recorder and Gboard. ## Limitations of Existing Protections - **TEEs** isolate sensitive computation from compromised operating systems and hypervisors. - Hardware attestation produces a cryptographic fingerprint proving which firmware and software are running. - However, TEEs remain vulnerable to newly discovered side-channel attacks. - Traditional cryptographic aggregation offers mathematical privacy guarantees, but many protocols require devices to stay online through several interactive rounds. - Extended multiround participation limits practical deployment at large scale. ## Combining Encryption and Isolation - The new system allows devices to submit data in a single message. - This removes the need for devices to remain connected for multiple protocol rounds. - Data is encrypted before leaving the device and is never reconstructed in server memory, including inside a TEE. - Unencrypted information is processed off-device only after it has been aggregated and anonymized. - TEE attestation provides verifiable evidence that the intended secure aggregation implementation is running correctly. - The cryptographic and hardware layers provide defense in depth: failure of one layer does not automatically expose individual data. ## One-Shot Lattice-Based Aggregation - The protocol uses lattice-based cryptography. - Ciphertexts can be combined so that their underlying data—and encryption keys—are aggregated together. - A resulting decryption key can reveal only the aggregate, not individual contributions. - Small client committees hold decryption hints and help unlock the aggregated result. - Differential privacy noise is added to further protect the aggregate. - Decryption authority is distributed across multiple parties, preventing any single party from accessing encrypted data. ## Application to Android SafetyCore - SafetyCore provides privacy-preserving, on-device safety features for Android 9 and later. - Private analytics can help measure which threats are detected and identify areas where detection needs improvement. - Google is working with the Android SafetyCore team to apply this system while preserving the confidentiality of individual users’ data. ## Practical Takeaway The approach pairs the scalability of one-shot cryptographic aggregation with the verifiability of TEEs. It is designed for private, large-scale analytics where neither individual data nor trust in a single hardware or software component is required.

Read original(opens in new tab)

Zero TrustDifferential PrivacyOn Device AiFederated Analytics+3

google·May 19, 2026·2 min readCurated summary

Empirical Research Assistance (ERA): From Nature publication to catalyzing Computational Discovery

Empirical Research Assistance (ERA) is a Google AI system designed to help scientists develop expert-level computational models. Using Gemini, it searches literature, generates and evaluates code, and explores thousands of possible solutions through tree search. A Nature paper reports strong performance across scientific benchmarks, while new applications suggest ERA can accelerate research in health, climate, energy, and economics. ## How ERA Supports Scientific Coding - ERA starts with a scientific problem and a success metric. - It searches relevant research, combines methods, writes code, and iteratively tests and improves solutions. - Its tree-search process evaluates thousands of alternatives to optimize the resulting model. - Benchmarks in genomics, public health, satellite imagery, neuroscience, time-series forecasting, and mathematics showed expert-level performance. ## Applications to Open Scientific Problems - **Epidemiological forecasting** - Predicted U.S. hospital admissions up to four weeks ahead for flu, COVID-19, and RSV. - Forecasts ranked at or near the top of CDC leaderboards. - The techniques can potentially be adapted to other countries and diseases. - **California water-supply forecasting** - Produced seasonal runoff predictions for snow-fed river basins. - Delivered more accurate early forecasts than California’s official Bulletin 120 outlook. - Improved predictions could support water management and agriculture. - **Atmospheric carbon dioxide monitoring** - Combined geostationary weather-satellite data with other inputs to estimate CO₂ concentrations every 10 minutes across broad areas. - Captured urban emissions, plant-driven daytime absorption, and other atmospheric cycles. - Provides higher spatial and temporal coverage than measurements from satellites such as Orbiting Carbon Observatory-2. - **Solar-energy design** - Combined ERA with Google Antigravity to optimize three-dimensional solar-panel geometries. - Identified a 500-triangle volumetric fan design that could capture scattered radiation without backward shading. - **Retail forecasting** - Used economic indicators, Google Trends, historical patterns, and consumer sentiment. - Matched or exceeded commercial consensus forecasts and the Chicago Fed’s monthly retail forecast. ## Computational Discovery - Google is gradually opening access to Computational Discovery through a trusted tester program in Google Labs. - The system combines ERA with AlphaEvolve to support computational scientific investigation. - It complements other Gemini for Science experiments: - **Hypothesis Generation**, built with AI Co-Scientist, supports developing scientific hypotheses. - **Literature Insights** supports research and literature analysis. ERA’s demonstrated value lies in automating the labor-intensive cycle of designing, testing, and refining scientific software. Its expanding applications indicate that AI-assisted computational research could broaden access to advanced modeling while helping experts investigate complex scientific problems more quickly.

Read original(opens in new tab)

AIMachine LearningGeminiTime Series Forecasting+4

google·May 1, 2026·3 min readCurated summary

Catalyzing scientific impact through global partnerships and open resources

Google Research argues that scientific breakthroughs have the greatest impact when their software, datasets, and methods are openly shared and responsibly maintained through global partnerships. Its open-science efforts span genomics, neuroscience, climate, biodiversity, and healthcare, reaching more than 250,000 researchers and developers. The post concludes that collaboration and open resources can turn individual discoveries into tools for broader scientific progress and real-world benefits. ## Partnerships Across the Scientific Ecosystem - Google Research works with organizations including UCSC’s Genomics Institute, Janelia Research Campus, ISTA, CSIRO, AIIMS, and the Centre for Population Genomics. - It supports major international initiatives such as: - The Human Pangenome Research Consortium - The Earth BioGenome Project - The NIH BRAIN Initiative - Google is also developing communities of practice for scientific developers, beginning in India, Korea, Japan, and Australia. ## Open-Source Tools and Datasets - **Genomics** - DeepVariant, DeepConsensus, and DeepPolisher support DNA analysis from sequencing through genome assembly. - These tools have helped process exomes and whole genomes from 2.5 million people. - **Neuroscience** - Flood-filling networks, Neuroglancer, and TensorStore enable analysis and visualization of petascale brain reconstructions. - The public H01 dataset contains 1.4 petabytes of human brain tissue data and has been accessed more than 200,000 times. - MICrONS provides a large wiring and functional map of the mouse visual cortex. - **Earth and Atmospheric Science** - Open Buildings contains 1.8 billion building detections across 58 million square kilometers. - Caravan supports large-scale hydrology and flood forecasting in 150 countries, covering roughly 2 billion people. - Groundsource includes 2.6 million historical urban flood events from more than 150 countries. - NeuralGCM is a differentiable hybrid atmospheric model, while FireBench supports wildfire research with high-resolution synthetic data. - **Biodiversity** - SpeciesNet classifies 2,498 animal categories in wildlife-camera images. - **Healthcare** - HAI-DEF provides open-weight medical foundation models, including MedGemma, with more than 4.8 million downloads. - Open Health Stack offers secure, offline-capable tools based on modern healthcare standards. - OHS-powered applications have reached more than 65 million people across over 10 countries. ## Scientific and Humanitarian Impact - **Genomics** - Work with UCSC improved pangenome references and reduced genetic-variant identification errors by 50%. - The research contributes to more representative genomic resources through the Human Pangenome Research Consortium. - **Weather and Agriculture** - The University of Chicago’s Human-Centered Weather Forecasts Initiative used NeuralGCM and ECMWF systems to predict India’s monsoon onset up to a month ahead. - Forecasts, including an unusual dry spell, were delivered by SMS to 38 million Indian farmers to support planting decisions. - **Disaster Response** - UNHCR and other organizations use Open Buildings to improve survey sampling for displaced populations. - The dataset also supports research into building vulnerability to sea-level rise in the Global South. - Sunbird AI uses the data to assess energy needs in urban and rural communities. - **Neuroscience and Medicine** - Johns Hopkins researchers used the H01 brain dataset to identify a possible new form of neuronal communication, suggesting that current models of brain organization may be incomplete. - The finding could have implications for understanding conditions such as Alzheimer’s disease. - Google also partnered with Stanford Medicine and UCSC to accelerate genome analysis in urgent cases of suspected genetic disease. ## Practical Conclusion The post presents open-source scientific infrastructure, accessible datasets, and cross-border partnerships as essential to accelerating discovery. Researchers and institutions can maximize impact by sharing reproducible tools, maintaining resources collaboratively, and applying them to urgent global challenges.

Read original(opens in new tab)

Database DesignDeep LearningGenomicsHealthcare Ai+4

google·Apr 29, 2026·2 min readCurated summary

Four ways Google Research scientists have been using Empirical Research Assistance

Empirical Research Assistance (ERA) is being used by Google researchers to tackle practical scientific problems rather than only benchmark exercises. Early applications span public-health forecasting, cosmology, and climate monitoring, showing that AI can improve prediction, solve difficult mathematical problems, and extract new value from existing data. The results suggest ERA could make advanced computational research more accessible while producing interpretable, scientifically grounded models. ## Public Health Forecasting - Google expanded ERA-based hospitalization forecasts from COVID-19 to influenza and RSV. - The team submits weekly forecasts for every U.S. state, covering horizons of up to four weeks. - Google forecasts have performed at or near the top of public CDC flu and COVID-19 leaderboards, with similarly strong internal results for RSV. - Forecast accuracy is evaluated using the Weighted Interval Score on log-transformed hospitalization data. - This approach could broaden access to epidemiological modeling and support forecasting for more diseases and regions. ## Cosmology: Cosmic Strings - Cosmic strings are theoretical spacetime defects that may emit gravitational radiation. - Calculating their radiation spectrum is difficult because the governing equations contain singularities. - Earlier work found only a partial solution for a square loop with an angle of 90 degrees. - By combining ERA with Gemini Deep Think, researchers derived six general solutions and a concise formula for the asymptotic limit. - The result demonstrates how AI systems can help explore advanced mathematical techniques and address previously unsolved cosmological problems. ## Climate Monitoring with Weather Satellites - Existing CO₂ satellites provide highly precise but infrequent and geographically limited measurements. - Geostationary satellites such as GOES East scan large areas every 10 minutes, but were not designed to measure CO₂. - Researchers used ERA to create a physics-guided neural network that combines: - 16 GOES East wavelength bands - Lower-troposphere meteorology - Solar angles - Time of year - Trained using sparse OCO-2 and OCO-3 observations, the model estimated column-averaged CO₂ continuously across the satellite’s coverage area. - Comparisons with independent satellite and ground-based observations showed that it captured real CO₂ variation. - The work illustrates how AI can repurpose existing instruments and improve the value of expensive scientific datasets. ERA’s early applications indicate that AI-assisted empirical software can support accurate forecasting, novel mathematical discovery, and higher-resolution environmental monitoring. Its greatest potential may lie in combining domain expertise with existing data and infrastructure to solve problems that would otherwise require substantial time and specialized resources.

Read original(opens in new tab)

AINeural NetworksGeospatial AnalysisData Collection+4

google·Apr 22, 2026·2 min readCurated summary

It's all about the angle: Your photos, re-composed

Google’s new Auto frame feature in Google Photos re-composes photos by changing the apparent camera viewpoint after capture. It combines 3D scene reconstruction with generative AI to adjust framing, reduce wide-angle distortion, and generate background areas that were outside the original frame. The result is a more natural perspective while preserving the visible content and subjects’ identities. ## Why Traditional Editing Falls Short - Cropping and zooming cannot change the original camera perspective or parallax. - They also cannot reveal areas hidden outside the captured frame. - This is especially limiting for portraits, selfies, and “almost perfect” photos with poor framing or lens distortion. ## Two-Stage 3D-Aware Editing - **3D scene and camera estimation** - A machine-learning model creates a 3D point map from the 2D image using monocular depth estimation. - Each pixel is associated with an estimated 3D surface point. - The system estimates the original focal length and reconstructs human faces and bodies carefully to reduce identity-distorting artifacts. - Semantic analysis detects subjects’ face positions and 3D orientations to determine better camera parameters. - **Generative completion and retouching** - Classical 3D rendering produces an image from the modified camera position, orientation, and focal length. - Moving the virtual camera exposes “holes” where the original image contained no information. - A latent diffusion model fills these missing regions and corrects the rendered result. - Classifier guidance with regional scaling helps preserve original content while allowing generation in newly revealed areas. ## Automatic Framing and Distortion Correction - The system automatically selects camera parameters for improved portrait composition. - It can adjust both camera pose and intrinsic properties such as focal length. - For wide-angle selfies, it detects perspective distortion that enlarges features close to the lens. - By effectively moving the virtual camera backward and adjusting the optics, it produces more natural facial proportions. ## Availability in Google Photos - The technology is now integrated into the Auto frame feature. - It applies to eligible photos containing people. - Users receive the re-composed image as a second Auto frame candidate. - The enhancement requires only a single action and automatically changes the viewpoint while retaining the original scene. The feature demonstrates how combining geometric 3D reconstruction with controlled generative inpainting can perform edits that conventional cropping and retouching cannot. For portraits and selfies, Auto frame offers a practical way to improve composition and perspective after the moment has passed.

Read original(opens in new tab)

Machine LearningGenerative AI3d RenderingImage Editing+4

google·Apr 21, 2026·2 min readCurated summary

ReasoningBank: Enabling agents to learn from experience

ReasoningBank is an agent-memory framework designed to help deployed agents learn continuously from both successful and failed task attempts. Rather than storing exhaustive action histories or only successful workflows, it distills reusable reasoning strategies, decision rationales, and preventative lessons. Evaluations on WebArena and SWE-Bench-Verified show higher success rates and fewer execution steps, especially when combined with memory-aware test-time scaling. ## Distilling Generalizable Reasoning - Each memory contains: - A concise title - A brief description - Detailed reasoning steps, rationales, or operational insights - The agent retrieves relevant memories before acting. - After completing a task, an LLM judge evaluates the trajectory and identifies useful success or failure signals. - The agent converts those signals into new memories and appends them to the ReasoningBank. - Failure analysis is central: mistakes become counterfactual guidance and strategic guardrails, such as verifying the current page before repeatedly clicking “Load More.” ## Memory-Aware Test-Time Scaling - Memory-aware test-time scaling (MaTTS) connects inference-time exploration with long-term memory. - **Parallel scaling:** Multiple trajectories are generated and compared, allowing the agent to distinguish robust strategies from flawed reasoning. - **Sequential scaling:** The agent progressively refines a single trajectory, preserving useful intermediate insights from trial and error. - This creates a feedback loop: better memories guide exploration, while richer exploration produces better memories. ## Benchmark Results and Strategic Maturity - Against memory-free ReAct agents using Gemini-2.5-Flash: - Success rates improved by 8.3% on WebArena. - Success rates improved by 4.6% on SWE-Bench-Verified. - SWE-Bench-Verified tasks required nearly three fewer execution steps on average. - Adding MaTTS with parallel scaling factor **k=5** produced further gains: - A 3% success-rate increase over ReasoningBank alone on WebArena. - 0.4 fewer steps per task. - Over repeated tasks, simple procedural checklists evolved into more sophisticated memories containing compositional and preventative logic. ReasoningBank suggests that effective agent scaling requires more than additional inference compute or stored trajectories. Agents should systematically learn from both outcomes and mistakes, using structured reasoning memories to become more capable and efficient after deployment.

Read original(opens in new tab)

Database DesignLarge Language ModelsPostgreSQLContrastive Learning+4

google·Apr 16, 2026·3 min readCurated summary

AI-generated synthetic neurons speed up brain mapping

Google Research developed MoGen, an AI model that generates realistic synthetic neuron geometries to improve brain-mapping systems. Adding these synthetic examples to PATHFINDER reduced neuron reconstruction errors by 4.4%, primarily by preventing incorrect merges. At the scale of a complete mouse brain, the improvement could eliminate an estimated 157 person-years of manual proofreading. ## Connectomics and the Reconstruction Challenge - Connectomics creates brain wiring maps by imaging thin tissue slices, aligning them, and reconstructing 3D neurons. - AI assists with segmentation and reconstruction, but human experts must still correct errors. - Larger brains pose major scaling challenges: the fruit fly map contains about 166,000 neurons, while a mouse brain is roughly 1,000 times larger. ## Why Neuron Shape Matters - Neurons have complex structures, including long axons, branching dendrites, dendritic spines, and synapses. - PATHFINDER identifies neurite segments and combines them into complete neurons. - Poor or ambiguous microscopy data can cause: - **Split errors:** connected neurites are separated. - **Merge errors:** unrelated neurites are incorrectly joined. - Correcting these mistakes manually is one of the most time-consuming parts of brain mapping. ## MoGen’s Synthetic Neurons - MoGen uses point-cloud flow matching to transform random 3D point clouds into realistic neuronal shapes. - It was trained on surface samples from 1,795 human-verified mouse axons. - Experts could not reliably distinguish MoGen-generated neurite fragments from real ones. - The synthetic data reproduced features such as bending, twisting, thickening, and branching. ## Results with PATHFINDER - Training PATHFINDER with 10% MoGen-generated data reduced reconstruction errors by 4.4%. - The largest gains came from reducing merge errors. - Millions of synthetic neuron shapes were added to the training pipeline. - Although the percentage improvement is modest, it could save the equivalent of 157 years of expert proofreading for a full mouse-brain map. ## Future Applications - MoGen could be tuned to generate neuron types with specific lengths, branching patterns, or spatial ranges. - Future versions may focus on geometries that are particularly likely to cause reconstruction errors. - Google has also trained species-specific models for zebra finches and fruit flies. - The team is exploring synthetic electron-microscopy images to improve earlier stages of reconstruction. - MoGen and its species-specific models have been released as open source. Synthetic neuron generation is a practical way to expand training data without requiring additional manual annotation. Combined with targeted generation and synthetic microscopy, it could help make large-scale projects such as complete mouse-brain mapping more feasible.

Read original(opens in new tab)

AISynthetic DataNeural NetworksConnectomics+4

google·Apr 16, 2026·4 min readCurated summary

Designing synthetic datasets for the real world: Mechanism design and reasoning from first principles

Synthetic data generation should be treated as dataset-level mechanism design rather than one-sample-at-a-time prompting. Google’s Simula framework uses reasoning models to control coverage, diversity, complexity, and quality independently, enabling seedless, reproducible datasets for scarce or privacy-sensitive domains. Experiments across five domains show that thoughtfully designed data can outperform larger datasets, but the best configuration depends on the target task and model. ## Why Real-World Data Is Insufficient - Specialized AI applications often lack accessible data because domains are uncommon, expensive to label, or privacy-sensitive. - Manually creating datasets is costly, slow, and error-prone. - Real-world datasets are static, limiting rapid iteration and making it difficult to proactively generate safety edge cases. - Synthetic-first workflows can make data programmable, versioned, reproducible, and inspectable. ## Limitations of Existing Synthetic Data Methods - Many approaches depend on manual prompts, evolutionary algorithms, or large amounts of seed data. - These dependencies reduce scalability and explainability. - Generation parameters are often entangled, making it difficult to independently control diversity, difficulty, and correctness. - Most methods optimize individual samples instead of designing the dataset’s overall distribution. ## Simula’s Reasoning-First Design - Simula constructs datasets from first principles using reasoning models rather than opaque generation processes. - The framework is seedless and agentic, allowing improvements as the underlying models become better at reasoning. - It separates generation into independently controllable axes. ### Global Diversification - Reasoning models map a domain into deep, hierarchical taxonomies that serve as sampling scaffolds. - A recursive propose-and-refine process generates candidate categories, then evaluates, merges, and filters them with a critic model. - These taxonomies help datasets cover long-tail concepts instead of concentrating on common examples. - The approach was demonstrated with structures such as a Cyber Threat Intelligence taxonomy. ### Local Diversification - Taxonomy nodes are converted into “meta-prompts” representing scenarios. - Multiple distinct instantiations are generated for each scenario. - This reduces mode collapse—for example, representing SQL injection through varied contexts rather than repeating nearly identical questions. ### Complexification - A configurable portion of scenarios is made more elaborate or difficult. - Complexity can therefore be adjusted without changing the dataset’s semantic coverage. - The appropriate difficulty level depends on the capabilities of the model that will consume the data. ### Quality Checks - A dual-critic loop independently assesses whether outputs and answers are correct. - Independent verification helps reduce sycophancy and improves label reliability. - This enables quality control without requiring human review of every example. ## Reasoning-Based Evaluation - Conventional metrics such as embedding cosine distance offer only broad signals and limited practical guidance. - Simula introduces: - **Taxonomic Coverage**, which measures how thoroughly the conceptual space is represented. - **Calibrated Complexity Scoring**, which uses LLM-based batch comparisons and chess-style Elo ratings to estimate the difficulty of individual examples. - These metrics aim to evaluate diversity and difficulty in ways that better reflect downstream usefulness. ## Results Across Domains - Simula was evaluated using Gemini 2.5 Flash as a teacher and Gemma 3 4B as a student. - The experiments covered cybersecurity, legal reasoning, grade-school mathematics, and multilingual academic knowledge, with datasets reaching 512,000 examples per domain. - The full combination of global coverage, local diversity, and quality critique consistently outperformed simpler baselines. - High complexity improved math reasoning accuracy by 10% in GSM8k but harmed legal reasoning performance, where the teacher model was weaker. - Simula often achieved stronger downstream results with fewer examples, suggesting that data properties matter as much as volume. - The findings show that there is no universally optimal synthetic-data recipe; effective dataset design must be tailored to the domain and target model. Synthetic data is most effective when treated as an engineered system rather than a source of additional volume. Practitioners should separately tune coverage, variation, difficulty, and verification, then validate those choices against the downstream task.

Read original(opens in new tab)

PostgreSQLSynthetic DataData GenerationMechanism Design+4

google·Apr 13, 2026·3 min readCurated summary

Towards developing future-ready skills with generative AI

Vantage is a Google Research experiment that uses generative AI to assess durable “future-ready” skills such as critical thinking, collaboration, conflict resolution, and creativity. It places students in realistic conversations with AI teammates, dynamically introduces challenges, and evaluates performance against educational rubrics. A study with New York University found that AI-generated scores agreed with human expert ratings at a comparable level to agreement between human raters. ## Why Future-Ready Skills Are Difficult to Measure - Skills such as collaboration, creative thinking, and conflict resolution are increasingly important as technology changes work and education. - Traditional tests are too rigid to capture how people think, communicate, and respond in realistic situations. - Human-based assessments can be resource-intensive, difficult to standardize, and dependent on whether challenging situations arise naturally. - Vantage aims to make these skills measurable, scalable, and useful for guiding instruction and student growth. ## AI-Simulated Team Assessments - Students participate in open-ended tasks, such as preparing a debate or pitching a creative idea, alongside AI avatars. - An “Executive LLM” uses an assessment rubric to manage the conversation and introduce targeted challenges, such as disagreement or conflict. - This adaptive process is designed to elicit enough evidence to assess a particular skill while keeping the interaction natural. - An “AI Evaluator” reviews the conversation transcript using the same rubric. - Students receive a visual skill map and qualitative feedback describing their demonstrated strengths and areas for improvement. ## Validation with New York University - Google Research partnered with NYU to align Vantage’s tasks and scoring criteria with established educational rubrics. - The joint study involved 188 U.S. participants aged 18–25 and focused on conflict resolution and project management. - Researchers tested whether the Executive LLM could steer conversations toward specific skills. - Steered conversations produced significantly more skill-relevant information than conversations involving independent, uncoordinated AI avatars. - The AI Evaluator’s scores showed agreement with human expert ratings comparable to the agreement between two human raters. - The results suggest that LLM-based assessment can provide a scalable alternative for evaluating complex interpersonal skills. ## Additional Research - Google also collaborated with OpenMic to study creativity and English language arts. - The collaboration analyzed work from 180 students completing creative multimedia assignments, including character interviews and literature-related media articles. - These studies tested whether the evaluation approach could extend beyond collaboration-focused tasks. Vantage is available in English through Google Labs as a research experiment. Its approach could help educators provide more consistent practice, evidence-based feedback, and scalable assessment for skills that conventional tests struggle to capture.

Read original(opens in new tab)

Generative AIProject ManagementConflict ResolutionExecutive Llm+4