google·Aug 12, 2026·3 min readCurated summary

Empty shelves or lost keys? Recall is the bottleneck for parametric factuality

Frontier LLMs often fail at factual questions not because they lack the information, but because they cannot reliably retrieve information already encoded in their parameters. The authors’ knowledge-profiling framework finds that models such as Gemini 3 and GPT-5 encode 95–98% of tested facts, yet still fail to directly recall 26–34% of them. This suggests that improving factuality increasingly depends on better knowledge utilization, not simply larger models or more training data. ## Knowledge Profiling - The framework evaluates facts rather than isolated question-answer pairs. - Each fact is assigned one of five profiles: - Encoding failure - Recall failure - Direct recall - Recall with thinking - Inference without encoding - It distinguishes three behaviors: - **Encoding:** reproducing a fact in a pre-training-like context. - **Knowledge:** answering semantically equivalent direct and reverse questions. - **Recall:** retrieving an encoded fact without external cues. - “Thinking” can help models recover facts through intermediate reasoning, multi-hop inference, or educated guesses. ## The WikiProfile Benchmark - WikiProfile contains 2,150 facts extracted from Wikipedia. - Each fact is tested with 10 tasks: - Two encoding tasks - Four knowledge questions - Four multiple-choice recognition questions - Questions include different phrasings and relational directions, such as asking either what entity *B* is associated with or what entity *A* is associated with. - An automated pipeline using Gemini-2.5-Pro generated, refined, searched, and filtered questions, followed by manual validation. - The study evaluated 13 LLMs, with and without thinking. - Eight responses were sampled for every model, fact, and task, producing roughly 4.5 million responses graded by LLM-based autoraters. ## Recall, Not Encoding, Is the Main Bottleneck - Frontier models show near-saturated factual encoding: - Gemini 3 and GPT-5 encode approximately 95–98% of the benchmark’s facts. - They nevertheless fail to directly recall 26–34% of those facts. - Thinking reduces, but does not eliminate, the failures; 11–12% remain inaccessible. - Scaling improves encoding more strongly than recall. - In the Gemma 3 family, larger models have substantially fewer encoding failures, while recall failures remain significant and account for a larger share of residual errors. - The results indicate a shift from knowledge acquisition toward knowledge accessibility and utilization. ## Why Recall Fails - Recall appears sensitive to the conditions under which a fact was learned. - Retrieval becomes more difficult when a question differs from the fact’s training-time context, wording, or ordering. - Rare, long-tail facts illustrate this distinction: - Their encoding rates are relatively close to those of popular facts. - Their recall rates are substantially worse. - This suggests that many rare facts are present in model parameters but are difficult to access, rather than completely absent. The practical implication is that future factuality improvements should target retrieval, post-training, prompting, and inference-time reasoning—not only model scale or broader data coverage.

Read original(opens in new tab)

Large Language ModelsBenchmarkingChain Of ThoughtPre Training+3

google·Aug 11, 2026·3 min readCurated summary

Advancing AMIE towards expert-level audio-visual clinical consultations

AMIE (Video) is Google’s real-time audiovisual medical AI system, designed to overcome the limitations of text-only clinical conversations. Built on Gemini and Project Astra, it observes visual and auditory cues, guides patients through virtual examinations, and performs diagnostic reasoning during live consultations. In a randomized study involving 300 simulated consultations, the system was evaluated against text-only AMIE and board-certified primary care physicians. ## Why Audio-Visual Consultation Matters - Traditional text-based systems lose important clinical information, including: - Gait and visible physical symptoms - Breathing patterns and signs of distress - Vocal and auditory cues - Patient responses during physical examination maneuvers - Requiring patients to describe symptoms in writing can reduce diagnostic accuracy, particularly for people with limited digital or health literacy. - Audiovisual interaction may also improve trust, communication, and access to medical expertise. ## AMIE’s Broader Development - Earlier versions of AMIE demonstrated expert-level performance in: - Text-based diagnostic dialogue - Differential diagnosis support - Disease treatment and longitudinal management - Specialist evaluations in oncology, cardiology, and ophthalmology - Reasoning over medical images and clinical documents - Google has also explored physician oversight and real-world clinical feasibility studies. ## Asynchronous Multi-Agent Architecture AMIE (Video) divides the consultation among three agents operating in parallel: - **Talker agent** - Maintains natural, low-latency spoken conversation. - Incorporates information and recommendations from the other agents. - **Planner agent** - Performs deeper clinical reasoning in the background. - Updates differential diagnoses and management plans. - Identifies missing information and reprioritizes clinical objectives. - **Perception agent** - Continuously analyzes audio and video. - Detects non-verbal findings such as visible distress, physical signs, and auditory abnormalities. - Interprets observations in the context of the conversation. This separation allows AMIE to reason deeply without creating long conversational pauses. Automated tests indicated that the agents contributed to improvements in history-taking, clinical reasoning, treatment recommendations, communication quality, and response latency. ## Automated Evaluation Framework - Google created a taxonomy of audiovisual clinical competencies based on medical literature. - The taxonomy covered: - Non-verbal visual cues - Auditory signals - Physical examination maneuvers - The evaluation suite included: - Single-turn tests targeting specific perception and reasoning abilities - Multi-turn simulated consultations assessing complete conversational performance - Simulations injected visual findings as textual descriptions, such as a patient holding handwriting samples up to the camera. - These tests helped identify capabilities and failure modes before human evaluation. ## Randomized Video Study - The study used a synchronous video consultation interface and an Objective Structured Clinical Examination format. - It included: - 100 clinical scenarios - Five body systems: cardiopulmonary, abdominal, HEENT, neurological/psychiatric, and musculoskeletal - 15 trained patient actors - 300 standardized consultations - Three study arms were compared: - **AMIE (Video):** Real-time audiovisual consultations - **AMIE (Text):** Text-only AMIE used to isolate the value of audiovisual capabilities - **PCP (Video):** Board-certified primary care physicians using the same video interface - An independent panel of 20 experienced primary care physicians assessed the consultations using established clinical rubrics. AMIE (Video) represents a move from text-based medical dialogue toward interactive, multimodal consultations. Its multi-agent design and audiovisual perception are intended to preserve conversational responsiveness while supporting richer clinical reasoning, though the reported findings come from simulated consultations and require further validation in real-world clinical care.

Read original(opens in new tab)

GeminiMultimodal AITelehealthMulti Agent Architecture+3

google·Jul 30, 2026·3 min readCurated summary

Science One Framework: A verifiable autonomous research framework via Chain-of-Evidence

The Science One Framework addresses a central weakness in autonomous AI research: polished papers can contain fabricated citations, unreproducible results, or methods that do not match the underlying code. Its Chain-of-Evidence (CoE) framework requires every claim to be linked to concrete evidence and introduces CoE Audit to test those links automatically. In evaluations, Science One produced fully verifiable papers while matching or exceeding baseline research agents and human performance on several benchmarks. ## Chain-of-Evidence for Verifiable Research - CoE defines trustworthy research artifacts through two requirements: - **Completeness:** Every claim has a recorded evidence chain. - **Correctness:** The evidence genuinely supports the claim. - Claims may include: - Bibliographic references - Reported scores - Method descriptions - Conclusions - Evidence can include peer-reviewed papers, experiment logs, executed code, or result tables. - Hallucinated citations, unreproducible scores, and discrepancies between described and implemented methods are treated as broken evidence chains. ## The Science One Framework The framework builds evidence into the research process instead of attempting to verify a paper after it has been written. - **Problem Investigator** - Uses the Semantic Scholar API to construct a citation graph. - Reads up to 100 full-text PDFs per topic. - Produces a structured research brief. - Restricts final-paper references to sources retrieved through the grounded API, avoiding citations generated from model memory. - **Discovery Engine** - Explores ideas through parallel explore-exploit branches. - Each isolated cycle includes a Solver agent and a task-specific evaluator. - High-performing solutions are iteratively refined. - Raw evaluator outputs are preserved in strict, read-only records. - **Paper Writer and Claim Verifier** - Creates a structured inventory of factual claims. - Attaches inline evidence tags linking claims to workspace artifacts. - Checks each claim against its declared source. - Rewrites unsupported claims conservatively rather than allowing them to exceed the evidence. ## CoE Audit Integrity Checks CoE Audit is an automated, post-hoc forensic review of a paper, solution, code, and references. - **Score verification:** Re-runs the submitted code independently and compares the result with the paper’s reported score. - **Specification violation:** Checks whether the code solves the intended task without exploiting the evaluator or accessing ground-truth answers. - **Reference verification:** Validates every bibliography entry against academic APIs. - **Method-code alignment:** Compares the paper’s method description with the actual implementation using LLM-based judges. ## Evaluation Results - The audit evaluated 75 papers across five systems-optimization tasks: Prism, Cloudcast, EPLB, LLM-SQL, and transaction scheduling. - Science One led the evaluated systems on all four integrity checks. - It had: - Zero phantom references - Perfect score verification - The strongest method-code alignment - Baseline systems hallucinated up to 21% of references and sometimes described advanced algorithms that were implemented as simple deterministic heuristics. - Strong verification did not reduce performance: - Science One matched or exceeded human experts on all five ADRS tasks. - It achieved the best overall result on Cloudcast and EPLB. - On additional MLE-Bench and Parameter Golf evaluations, the framework also demonstrated competitive performance, including two Gold Medals across five difficult Kaggle competitions. The main recommendation is to design autonomous research systems around evidence generation from the beginning. Grounded retrieval, immutable experiment records, claim-level verification, and independent auditing can substantially improve reliability without necessarily sacrificing research performance.

Read original(opens in new tab)

Large Language ModelsMachine LearningAutonomous AgentsSemantic Scholar Api+3

google·Jul 22, 2026·3 min readCurated summary

Towards a quantum computer that learns from its errors

Quantum computers require constant recalibration because analog control signals drift during computation. Google Quantum AI combined reinforcement learning (RL) with quantum error correction (QEC), allowing a system to learn from detected errors and adjust thousands of control parameters while computation continues. Tests on the Willow processor showed improved logical stability, suggesting this approach could support much longer quantum computations. ## The Challenge of Quantum Errors - Quantum systems are highly sensitive to drift in signal frequencies, amplitudes, and phases. - Conventional recalibration requires stopping the entire computation, limiting algorithms that may need to run for days or months. - QEC uses many physical qubits to form logical qubits and converts analog noise into binary error-detection events. - These events indicate that an error occurred within a spacetime region but do not identify its precise cause or location. - Decoders such as AlphaQubit and Tesseract infer corrections, but they do not explain whether errors arose from environmental decoherence or preventable calibration drift. ## Moving Beyond Physics-Based Calibration - Traditional calibration depends on manually designed physical models. - Such models can reach performance limits when hardware behavior involves complex, poorly understood interactions. - Google argues that quantum control may benefit from the same shift toward data-driven learning seen in computer vision, robotics, and protein-folding research. - As quantum hardware improves, remaining errors increasingly reflect subtle phenomena that are difficult to model analytically. ## Using Error Detection as a Learning Signal - An RL agent experiments with control strategies and improves based on the resulting error data. - QEC detection events serve two purposes: - Decoders use them to infer logical corrections. - The RL system uses them to identify drift and refine control parameters. - This enables continuous calibration without interrupting the quantum computation. - The approach can steer thousands of analog control parameters dynamically. ## Results on the Willow Processor - Researchers deliberately introduced control-parameter drift into Google’s Willow superconducting processor. - RL steering improved the logical stability of the error-correcting code by 3.5 times. - After expert, human-guided calibration, RL fine-tuning reduced the logical error rate by an additional 20%. - Combined improvements produced fewer than one logical error per 1,000 surface-code correction cycles and fewer than one per 100 color-code cycles. - The processor therefore operated as a more reliable quantum memory for longer periods. ## Scaling to Larger Systems - Simulations included hundreds of qubits and tens of thousands of control parameters. - The RL agent reduced initially high physical error rates by learning better control settings. - QEC suppressed the logical error rate exponentially as the number of physical qubits increased. - The simulations indicated that the number of RL training iterations needed to reduce physical errors did not depend on system size, supporting potential scalability. The results suggest that future quantum computers could use QEC not only to correct errors but also to learn their causes and continuously adapt to hardware drift. RL-based calibration could reduce dependence on manual tuning and help make long-running, fault-tolerant quantum computation practical.

Read original(opens in new tab)

Machine LearningQuantum ComputingReinforcement LearningSuperconducting Qubits+4

google·Jul 22, 2026·3 min readCurated summary

SymptomAI: Towards a conversational AI agent for everyday symptom assessment

SymptomAI explores whether conversational AI can conduct realistic symptom interviews and generate useful differential diagnoses outside curated medical vignettes. In a randomized national study of 13,917 participants, SymptomAI agents often performed as well as or better than clinician-generated differentials according to expert reviewers, particularly when they actively asked follow-up questions. The study also found that diagnoses associated with infectious illnesses corresponded with shifts in participants’ Fitbit biosignals, suggesting potential for large-scale health research. ## Moving Beyond Curated Medical Cases - Existing language-model evaluations often use detailed, synthetic, or highly structured patient vignettes. - Real patients may provide incomplete information, have varying medical literacy, or describe symptoms unpredictably during conversation. - SymptomAI was designed to test end-to-end symptom assessment in a more natural setting, while making clear that its outputs were research results rather than clinical diagnoses. ## National-Scale Study Design - 13,917 consenting participants were randomly assigned to one of five Gemini Flash 2.0 SymptomAI agents. - Participants described their symptoms, answered follow-up questions, received a differential diagnosis (DDx), and were given next-step recommendations. - Two weeks later, participants reported diagnoses received from healthcare providers. - Three board-certified clinicians reviewed the conversations, created their own differentials, and blindly ranked SymptomAI’s DDx against clinician-generated alternatives. ## SymptomAI Compared Favorably with Clinicians - Clinical reviewers preferred SymptomAI’s differential diagnosis over those from other clinicians in more than 50% of cases. - SymptomAI’s DDx was more likely to be ranked as the highest-quality option. - Using top-five accuracy—whether the eventual provider diagnosis appeared among five proposed diagnoses—reviewers found SymptomAI’s differentials accurate more often than the comparison clinician differentials. ## Follow-Up Questions Improved Accuracy - The study tested five interview strategies: - Dynamic Live and Dynamic Final agents could ask unrestricted follow-up questions. - Fixed Canonical and Flexible Canonical agents used standardized medical history questions. - The Base condition represented a user-led interaction with an unprompted language model. - Every agent-driven strategy significantly outperformed the Base condition. - The findings indicate that actively eliciting additional information is more effective than relying solely on what users initially choose to disclose. ## Strongest Results in Uncertain Cases - SymptomAI’s advantage over clinician baselines was greatest when clinicians expressed low confidence in their own differentials. - This suggests conversational AI may be especially useful as a second opinion or support tool in ambiguous cases, though the study does not establish that it can replace professional diagnosis. ## Connecting Diagnoses with Wearable Data - The researchers used SymptomAI’s diagnostic outputs as potential reference labels for analyzing population-scale physiological data. - Participants provided up to 30 days of Fitbit biometric data before their SymptomAI interaction. - Acute respiratory infection cases showed noticeable biosignal changes in the days leading up to symptom reporting. - These shifts appeared consistent with symptom onset and possible immune responses, although the provided text ends before presenting the full analysis. SymptomAI’s results support building conversational systems that ask structured follow-up questions and assist with differential diagnosis. Any practical deployment should retain clinician oversight, communicate uncertainty clearly, and treat AI-generated assessments as decision support rather than confirmed medical diagnoses.

Read original(opens in new tab)

GeminiConversational AiDifferential DiagnosisLanguage Models+3

google·Jul 15, 2026·2 min readCurated summary

Towards demystifying the creativity of diffusion models

Diffusion models generate novel data because neural networks learn a smoothed approximation of the score function rather than perfectly memorizing it. This smoothing prevents denoising trajectories from collapsing directly onto training examples, allowing samples to interpolate between them. In high-dimensional data spaces, it helps recover the underlying data manifold while preserving realism and novelty. ## How Diffusion Models Denoise - Training corrupts real samples with noise, then teaches a model to reverse the corruption step by step. - The score function acts like a force field, directing noisy points toward meaningful data. - A perfectly learned score function would pull every generated sample onto one of the training examples, resulting in memorization. - In practice, neural networks learn an approximation of this function and therefore can generalize beyond the training set. ## Score Smoothing Creates Interpolation - Neural network regularization, including weight decay, makes sharp changes in the ideal score function difficult to represent. - In a one-dimensional example with training points at -1 and +1: - The perfect score sharply switches direction at zero. - Generated points eventually collapse onto either -1 or +1. - A smoothed score creates a gentler transition near zero. - Points in this transition region move more slowly and can settle between the training points. - This interpolation produces novel but plausible samples. - Smoothing can arise from explicit regularization or implicit regularization caused by gradient-based optimization. ## Recovering the Hidden Data Manifold - Real images occupy a small, structured manifold within a much larger high-dimensional pixel space. - Generating new images requires recovering this manifold from finite training data. - Score smoothing behaves differently depending on direction: - Along the manifold, it slows movement toward individual training examples. - Toward the manifold, the score is already relatively smooth, so smoothing has little effect. - This directional behavior prevents samples from becoming blurry in empty regions while reducing memorization along the manifold. - The result is a balance between fidelity and creativity: generated outputs remain realistic while differing from the training examples. ## Conclusion The paper argues that diffusion-model creativity is a predictable mathematical consequence of score smoothing. Neural networks’ regularized, approximate learning allows denoising trajectories to interpolate across the data manifold instead of merely retrieving memorized samples.

Read original(opens in new tab)

Generative AIDiffusion ModelsNeural NetworksScore Function+4

google·Jul 9, 2026·3 min readCurated summary

SensorFM: Towards a general intelligence and interface for wearable health data

SensorFM is a large foundation model designed to turn wearable-device signals into a reusable representation of human physiology. Trained on more than one trillion minutes of multimodal data from five million people, it performs well across 35 health-related prediction tasks while requiring relatively few labeled examples. The authors argue that scaling both data and model size enables a general-purpose interface for wearable health data rather than isolated, task-specific models. ## The challenge of wearable health data - Wearables continuously capture heart rate, movement, temperature, blood oxygen, sleep, and related signals. - Interpreting these measurements is difficult because: - Baseline physiology and lifestyle vary substantially between individuals. - Reliable labels such as diagnoses, lab results, and validated questionnaires are costly and difficult to collect. - Traditional models usually target one health outcome at a time and generalize poorly. ## Training on more than a trillion minutes - SensorFM was trained on de-identified data from five million consenting participants collected between September 2024 and September 2025. - The dataset includes: - More than 100 countries and all 50 U.S. states. - Over 20 Fitbit and Pixel Watch models. - More than two billion sensor-hours of data. - The model processes 34 minute-level features from: - PPG - Accelerometry - Electrodermal activity - Skin temperature - Altimetry - These signals represent heart rate, heart-rate variability, blood oxygen, sleep, movement, skin conductance, and temperature across full days. ## Learning from incomplete sensor recordings - SensorFM uses self-supervised masked reconstruction rather than relying on medical labels. - Its Adaptive and Inherited Masking framework treats naturally missing data as part of the learning problem. - This avoids: - Imputing gaps, which can introduce bias. - Discarding incomplete windows, which wastes real-world data. - The resulting representation is explicitly aware of missingness and can learn from fragmented wearable recordings. ## Scaling data and model capacity - Experiments varied training data from roughly two million to two billion sensor-hours and model size from 100,000 to 100 million parameters. - Larger models trained on more data consistently improved both reconstruction and downstream health prediction. - The largest model: - Reduced reconstruction loss by 31% compared with the smallest version. - Improved classification performance by an average of 9% in AUC. - Improved regression performance by 21% in Pearson correlation. - Won on 33 of 35 downstream tasks. - Scaling data and model size together produced near-linear gains with no observed saturation. ## One representation across many health domains - SensorFM was tested on 35 tasks from three prospective studies involving 13,985 participants. - The tasks covered: - Cardiovascular health - Metabolic risk - Mental health - Sleep - Demographics - Lifestyle - With the encoder frozen and only a lightweight linear head trained, SensorFM embeddings outperformed engineered-feature supervised baselines on 34 of 35 tasks. - Larger models appeared to learn physiologically relevant demographic and individual differences without being explicitly given demographic inputs. - The model showed particular value for difficult-to-measure conditions such as depression and anxiety. - It also reached strong performance with relatively small quantities of labeled data, addressing a major constraint in healthcare modeling. ## Automated adaptation through an agentic “classroom” - The authors introduce a collaborative system of LLM agents intended to automate the creation of prediction heads. - This approach aims to reduce the manual work traditionally required for: - Feature engineering - Architecture selection - Hyperparameter tuning - The supplied article excerpt ends while introducing this system, so its detailed workflow and results are not described here. SensorFM demonstrates that large-scale, missingness-aware pre-training can produce a broadly useful representation of wearable physiology. Its strongest practical promise is label-efficient adaptation across many health applications, potentially providing a foundation for personalized health agents and more general wearable-data interfaces.

Read original(opens in new tab)

Healthcare AiSelf Supervised LearningFoundation ModelWearable Health+2

google·Jul 7, 2026·2 min readCurated summary

The power of collaboration: How we can reduce traffic congestion

Network-aware navigation can reduce citywide congestion by rerouting a small share of vehicles away from heavily overloaded roads. In a six-month experiment across 10 U.S. cities, altering routes for fewer than 2% of trips increased speeds, lowered fuel consumption, and reduced emissions across the broader road network. The results suggest that navigation apps can evolve from optimizing individual trips to coordinating traffic for system-wide benefit. ## Experiment Design - Google Maps routing was modified to favor alternative routes with similar travel times and road characteristics. - Approximately 100 historically congested road segments were selected in each city. - The study used a citywide switchback design, alternating between standard and modified routing on consecutive days. - Unlike experiments that randomly alter individual trips, the intervention was applied systematically across each city. - Fewer than 2% of observed trips received changed recommendations. ## Measurable Traffic Improvements - Targeted congested segments experienced a median speed increase of about 2%. - Fuel consumption rates on targeted segments fell by approximately 0.5% to 1%. - Across all affected segments—including roads receiving diverted traffic—median speeds increased by about 0.35%. - During morning and afternoon peaks, speeds improved by roughly 0.5%. - The estimated impact could save thousands of tons of CO2e emissions per city each year. ## Dispersing Traffic More Efficiently - The intervention shifted vehicles away from major bottlenecks and distributed them across a larger number of peripheral roads. - Alternative roads absorbed additional traffic without suffering comparable congestion because the volume increase was spread out. - In Atlanta, for example, traffic was diverted from a central highway to a more distributed network around the city. - Both navigation users and non-users benefited from reduced congestion on shared roads. ## Analytical Approach - Researchers used hierarchical Bayesian outcome modeling. - The model estimated effects at both citywide and hourly local levels. - Information was shared across cities and time periods, helping produce more reliable estimates for individual locations and time windows. - Improvements in speeds and emissions were statistically significant across the network. The study demonstrates that even limited, strategically coordinated rerouting can produce broad public benefits. Navigation platforms, connected vehicles, and smart-city systems could build on this approach to support dynamic traffic-signal control and real-time network optimization.

Read original(opens in new tab)

Network Aware RoutingTraffic OptimizationNavigation AppsSmart Cities+2

google·Jun 30, 2026·3 min readCurated summary

Introducing TabFM: A zero-shot foundation model for tabular data

TabFM is a foundation model for tabular classification and regression that uses in-context learning instead of dataset-specific training. It processes an entire table in one forward pass, eliminating much of the usual hyperparameter tuning and feature engineering. The authors report that it performs competitively with or better than tuned tree-based models, with planned integration into BigQuery. ## The Challenge of Traditional Tabular ML - Tabular prediction supports applications such as churn prediction and fraud detection. - Algorithms including XGBoost, AdaBoost, and random forests remain widely used. - Building reliable models typically requires: - Hyperparameter optimization - Cross-validation - Domain-specific feature engineering - Repeated model training for each dataset ## Tabular Prediction Through In-Context Learning - TabFM treats tabular prediction as an in-context learning problem. - Users provide historical training rows and target rows together as a single input. - The model learns relationships between rows and columns during inference without updating model weights. - This enables zero-shot predictions on previously unseen datasets. ## Hybrid Architecture TabFM combines ideas from TabPFN and TabICL to address the two-dimensional, order-independent nature of tables. - **Alternating row and column attention** - Attends across both features and examples. - Learns feature interactions and dependencies directly from the table. - Reduces the need for manually engineered features. - **Row compression** - Converts the contextualized information for each row into a dense vector. - **In-context Transformer** - Operates on compressed row representations rather than the full table grid. - Reduces computation and supports larger datasets efficiently. ## Training with Synthetic Data - Large, diverse real-world tables are difficult to obtain because they often contain proprietary or sensitive information. - TabFM is trained entirely on hundreds of millions of synthetic datasets. - Structural causal models generate tables using varied random functions and relationships. - The goal is to reproduce the broad range of distributions and feature interactions found in real-world data. ## Benchmark Results and Model Variants - Evaluation used TabArena, an Elo-based benchmark covering: - 38 classification datasets - 13 regression datasets - Dataset sizes from 700 to 150,000 samples - **TabFM** - Works out of the box. - Produces predictions in one forward pass without tuning or cross-validation. - **TabFM-Ensemble** - Adds cross features and SVD-derived features. - Uses a 32-model ensemble with non-negative least squares to optimize weights. - Applies Platt scaling for classification calibration. - The authors state that TabFM consistently outperforms heavily tuned conventional supervised algorithms. ## Availability and BigQuery Integration - TabFM is available through Google’s Hugging Face and GitHub repositories. - Google plans to integrate it into BigQuery. - Users will be able to run regression and classification through an `AI.PREDICT` SQL command without specialized machine-learning expertise. TabFM’s main promise is to make high-quality tabular prediction as simple as supplying a dataset and requesting predictions. Its zero-shot approach could reduce the engineering burden of traditional workflows, though practical adoption will depend on benchmark reproducibility, inference costs, and performance across specific production datasets.

Read original(opens in new tab)

TransformerSynthetic DataHugging FaceXgboost+4

google·Jun 30, 2026·3 min readCurated summary

Expanding our Heat Resilience data to 50+ global cities

Google Research has expanded its building-level rooftop reflectivity dataset from 14 pilot cities to more than 50 cities across nine countries. By combining Sentinel-2 satellite data with 30-centimeter commercial imagery and machine learning, the project identifies roofs where reflective “cool roof” interventions could reduce urban heat. The data is publicly available through a new Google Earth Engine app for planners and researchers. ## Why Rooftop Reflectivity Matters - Extreme heat contributes to approximately 500,000 deaths annually. - Urban heat islands make cities warm faster than the global average. - Dark roofs, pavements, and limited vegetation increase heat absorption. - Reflective roofs reduce the solar energy absorbed by buildings and can lower local surface temperatures. - Earlier pilot data from 14 cities supported cool-roof ordinances and broader climate adaptation plans. ## Building-Level Albedo Mapping - Sentinel-2 provides global albedo data but at 10-meter resolution, which cannot reliably distinguish individual rooftops. - Google Research fused Sentinel-2’s spectral and global coverage with 30-centimeter Airbus Pléiades Neo imagery. - Machine learning and radiometric calibration reconstruct detailed reflectance profiles at the urban-pixel level. - Validation against airborne hyperspectral measurements in Boulder, Colorado, produced an RMSE of 0.04. - The resulting maps allow planners to prioritize large, low-reflectivity buildings for cool-roof retrofits. - The modeling suggests targeted interventions could reduce extreme urban heat by up to 0.5°C globally. ## Heat Resilience Earth Engine App - Displays building-level albedo using rooftop centroids to highlight low-reflectivity surfaces. - Provides baseline analyses and supports monitoring changes over time. - Allows users to download high-resolution data for local studies and policy development. - Offers a nested view that moves from census-tract summaries to individual buildings. ## Expanded Global Coverage - The dataset now covers more than 50 cities in nine countries. - Newly included urban areas span Europe, Brazil, and the United States. - Examples include London, Athens, Barcelona, Rio de Janeiro, São Paulo, Los Angeles, Austin, and New York City. - The open dataset is intended to help municipalities accelerate reflective-surface programs. ## Access and Collaboration - The interactive app and datasets are publicly available through Google’s Heat Resilience site. - The methodology is described in the Nature Communications paper “Estimating high-resolution albedo for urban applications.” - The work was developed by Google Research in collaboration with the World Resources Institute. Cities can use the app to identify the buildings and neighborhoods where cool-roof investments are likely to have the greatest heat-reduction benefits.

Read original(opens in new tab)

Machine LearningGeospatial DataSatellite ImageryEarth Engine+3

google·Jun 26, 2026·3 min readCurated summary

Accelerating Gemini Nano models on Pixel with frozen Multi-Token Prediction

Google introduces a way to add Multi-Token Prediction (MTP) to already-deployed, frozen Gemini Nano models, accelerating on-device generation without a separate drafting model. The approach attaches a lightweight Transformer head to the existing model, reuses its hidden states and KV cache, and preserves identical final outputs through verification. On Pixel 9 and 10 devices, it delivers faster generation, lower memory use, and reduced energy consumption for features such as Notification Summaries and Proofread. ## The Mobile Inference Bottleneck - Autoregressive models generate one token at a time, creating latency and underusing mobile hardware. - Phones face strict RAM and energy constraints that make conventional acceleration techniques difficult. - A standalone speculative-decoding drafter consumes additional memory and must independently process the prompt. ## A “Late Exit” MTP Strategy - Speculative decoding uses: - A small drafter to propose several tokens. - The large model to verify those tokens in parallel. - MTP replaces the separate drafter with a lightweight Transformer head attached near the end of the main model. - The head uses the backbone’s high-dimensional activations to predict future tokens, benefiting from semantic context already computed by the larger model. ## Retrofitting a Frozen Backbone - Google freezes the fully trained Gemini Nano v3 weights and trains only the attached MTP head. - This avoids retraining or fine-tuning the production foundation model. - Incorrect draft tokens are discarded during verification, so the final output remains bit-for-bit identical to the original model. - The method therefore improves efficiency without changing the model’s capabilities or safety alignment. ## Zero-Copy Memory Architecture - The MTP head cross-attends directly to the backbone’s existing KV cache instead of maintaining a duplicate cache. - This eliminates separate prompt-prefill work for the drafter. - It also removes redundant embedding tables, attention variants, and application-specific tuning parameters. - Compared with a standalone drafter, the design saves up to 130 MB per instance. ## Accuracy and Speed Improvements - Access to the backbone’s richer internal representations makes MTP predictions more accurate than those of similarly sized standalone drafters. - Instruction-following tasks such as summarization and rewriting show especially strong gains. - For predictable formats such as smart replies, token acceptance improved by up to 55%. - Pixel 9 experiments showed speedups of 50% or more depending on the task. ## Production Impact - The updated inference stack coordinates drafting and verification on Pixel 9 and 10 devices. - In workloads including AI Notification Summaries and Proofread, MTP predicts nearly two additional tokens per inference pass on average. - Fewer verification cycles reduce processor wake-ups, improving latency and battery efficiency. MTP provides a practical way to accelerate existing on-device models without maintaining a separate drafter for every application. Reusing the frozen backbone’s computations and memory state makes it particularly well suited to mobile deployments where RAM, energy, and backward compatibility are critical.

Read original(opens in new tab)

KubernetesTransformerEdge ComputingSpeculative Decoding+3

google·Jun 25, 2026·3 min readCurated summary

Optimizing cloud economics with linear elastic caching

Linear elastic caching treats cache memory as a variable cost rather than a fixed allocation. It dynamically adjusts how long pages remain in memory by balancing ongoing memory expense against the cost of fetching evicted data again, using the ski rental problem as its theoretical foundation. Experiments in Spanner and public cache traces show meaningful cost reductions with only modest increases in misses. ## Fixed-Size Cache Limitations - Traditional caches allocate a fixed amount of RAM and use policies such as LRU when space runs out. - Undersizing the cache causes excessive disk or storage access and poor performance. - Oversizing it wastes money during periods of low demand; some serverless providers charge up to $3 per day for 1 GiB of memory. - Fixed sizing therefore creates a “Goldilocks” problem as workloads fluctuate. ## Ski Rental Model for Cache Eviction - Each cached page presents two choices: - **Rent:** Keep it in RAM and continuously pay for its memory footprint. - **Buy the miss:** Evict it and risk a latency and I/O penalty if it is requested again. - A ski rental algorithm assigns each page a time-to-live (TTL). - If the page is not accessed before its TTL expires, it is evicted. - If the cache becomes physically full, a conventional policy such as LRU handles capacity pressure. - The researchers prove that eviction policy and rental duration can be optimized separately, simplifying implementation. - Unlike worst-case break-even or randomized ski rental strategies, lightweight machine learning can exploit predictable workload patterns. ## Lightweight TTL Prediction - In Spanner, each page receives a TTL based on: - Page size - Cost of a cache miss - Type of database operation - Observed access behavior - A shallow decision tree was chosen because Spanner processes billions of requests per second. - The model can be translated into a few lines of interpretable C++ code. - Its cost-aware decisions allow extra misses mainly for data that is inexpensive to retrieve. ## Spanner Production Results - Compared with a standard fixed-size cache: - Memory usage fell by **15.5%**. - Cache misses increased by only **5.5%**. - Total cost of ownership fell by approximately **5%**. - The additional misses increased actual I/O costs by only **0.5%**, because they were concentrated on cheap-to-fetch data. - The policy was deployed on production Spanner servers and evaluated over several months. ## Public Trace Evaluation - The approach was tested on public industry cache traces using GDSF as the fixed-size baseline. - GDSF generalizes LRU to account for pages with different sizes. - Researchers evaluated four elastic-cache variants using: - Break-even or randomized ski rental policies - Learned or non-learned TTL selection - Because public traces lacked application-level features, learning used the first half of each trace to calculate the best TTL for individual pages. - Caches were warmed with one day of requests before performance measurement began. ## Overall Results - Elastic caching consistently produced lower total cost across diverse workloads. - Its advantage increased as memory became more expensive relative to cache misses. - At comparable cache sizes, elastic policies also achieved substantially lower miss rates than fixed-size approaches. Linear elastic caching is most useful when memory costs vary significantly or workloads are bursty and predictable. Dynamically assigning page TTLs offers a practical way to reduce memory spending while limiting performance impact, especially when the system can estimate the cost of each miss.

Read original(opens in new tab)

Machine LearningCachingC Plus PlusDatabase Systems+4

google·Jun 24, 2026·3 min readCurated summary

Thinking to recall: How reasoning unlocks parametric knowledge in LLMs

Reasoning can help LLMs recall simple facts even when no genuine multi-step deduction is required. Experiments with Gemini-2.5 and Qwen3 show that reasoning improves access to facts stored in model weights through two mechanisms: extra reasoning tokens provide computational capacity, while related factual statements prime retrieval. However, natural reasoning remains more effective than empty computation, and self-generated intermediate facts can introduce hallucination risks. ## Measuring the Knowledge Boundary - The researchers use **pass@k** to determine whether a correct answer appears among multiple generated attempts, rather than evaluating only the top answer. - They compare reasoning-enabled and reasoning-disabled versions of Gemini-2.5 Flash, Gemini-2.5 Pro, and Qwen3-32B. - Tests use the closed-book **SimpleQA Verified** and **EntityQuestions** datasets, which mainly contain simple, single-hop factual questions. - Reasoning-enabled models recover answers that are nearly unreachable when reasoning is disabled, showing that the benefit is not limited to solving complex problems. ## The Computational Buffer - Generating additional reasoning tokens gives the model more forward passes and therefore more opportunities to update its internal state. - To isolate this effect, the researchers replace the model’s natural reasoning with repeated meaningless text such as “Let me think.” - This dummy reasoning substantially improves factual recall compared with having reasoning completely turned off. - The effect has limits: - Longer dummy traces eventually produce diminishing returns. - Dummy reasoning never performs as well as natural reasoning. - These results indicate that extra computation helps, but the semantic content of the reasoning trace also contributes. ## Factual Priming - Natural reasoning traces often contain related facts rather than logical deductions. - This resembles **spreading activation** in human memory, where recalling one concept makes related concepts easier to access. - The researchers call this mechanism **factual priming**: the model generates nearby facts that create a contextual bridge to the target fact. - When the researchers extract only concrete facts from reasoning traces—removing filler, search plans, and the target answer itself—those facts recover most of reasoning’s benefit. - For example, when asked for Nepal’s 10th king, the model may recall the first nine kings. Listing those related facts primes retrieval of the requested answer. ## The Hallucination Trap - Generative self-retrieval depends on facts produced by the model during reasoning. - Because those intermediate facts may be hallucinated, factual priming can potentially reinforce incorrect information. - The excerpt introduces this risk but does not provide the researchers’ full evaluation or mitigation findings. The practical conclusion is that reasoning traces can function both as a computational workspace and as a semantic memory primer. For factual recall, systems should preserve useful intermediate retrieval while monitoring or verifying generated facts, since the same mechanism that unlocks obscure knowledge can also amplify errors.

Read original(opens in new tab)

Large Language ModelsChain Of ThoughtParametric MemoryLatent Computation+3

google·Jun 16, 2026·3 min readCurated summary

From pixels to planning: Earth AI for nature restoration

Google Research developed a high-resolution AI system that converts satellite imagery into vector data identifying small ecological features such as hedgerows, stone walls, and copses. These features can store carbon and support biodiversity without taking agricultural land out of production, but they are often missed by conventional forest inventories. The resulting open dataset aims to make fine-scale nature restoration measurable and actionable across the UK. ## The case for fine-scale restoration - Forests support carbon sequestration, water filtration, and biodiversity, but expanding forests can compete with food production. - Conservation efforts can also create “leakage,” shifting environmental damage elsewhere. - Hedgerows, shelterbelts, and other woody features offer ecological benefits while remaining integrated into farmland. - Google’s earlier **Farmscapes 2020** raster map detected these features across England, but pixel data was insufficient for planning, restoration, and carbon accounting. ## Challenges in mapping the countryside - Agricultural features often overlap spatially, such as hedgerows growing alongside or over stone walls. - Processing data in S2-cell tiles can split features at tile boundaries. - A generic “woody” classification does not reveal whether a feature is a forest, wildlife corridor, or isolated copse. - England’s more than 130,000 km² of high-resolution imagery created major computational demands for conventional raster-to-vector processing. ## Deep learning and vectorization - The team fine-tuned Remote Sensing Foundations’ Vision Transformer, pretrained on more than 300 million global satellite images. - Only about 247 km² of annotated British landscape data was available, so the pretrained model helped transfer broad visual knowledge to local conditions. - A dual-layer labeling system combined submeter imagery with 1-meter LiDAR to distinguish: - Ground-level boundaries such as fields and water - Above-ground features such as trees and walls - Geometry-merging algorithms reconnected features split across S2-cell borders. ## Classifying ecological features - The system used the **Polsby–Popper compactness score** to classify detected shapes by geometry. - Woodlands were defined as contiguous canopies at least 30 meters in diameter. - Woody patches included small copses and individual trees. - Linear woody features, including hedgerows and corridors, were identified using a compactness score below 0.5. - This classification turns raw detections into an inventory tied more closely to ecological function. ## Scaling the analysis - Google Earth Engine processed thousands of S2 cells in parallel. - This made it possible to generate vector geometries for millions of features across England without overwhelming traditional systems. - The resulting dataset is intended for landowners, conservationists, scientists, and policymakers. ## Future applications - The researchers plan to apply the method to silvopasture and agrisilviculture. - High-precision monitoring could help detect conservation “leakage” beyond project boundaries. - Open access to the data could support restoration while protecting agricultural productivity and food security. The dataset provides a practical foundation for planning and measuring nature recovery on working lands, demonstrating how AI can translate overlooked landscape details into conservation decisions.

Read original(opens in new tab)

Deep LearningComputer VisionRemote SensingGeospatial Data+4

google·Jun 12, 2026·3 min readCurated summary

A low-carbon computing platform from your retired phones

Retired smartphones can become low-carbon cloud infrastructure by reusing their still-capable motherboards instead of manufacturing new servers. Researchers at UC San Diego, with Google’s support, are developing clusters of stripped-down Pixel phones managed by Kubernetes. Their planned 2,000-phone datacenter aims to provide affordable computing for education and research while reducing hardware-related emissions. ## The Carbon Case for Reusing Smartphones - Computing emissions come from: - **Operational carbon**, produced by electricity consumed during use. - **Embodied carbon**, produced during hardware manufacturing and raw-material extraction. - Reusing phones primarily addresses embodied carbon by extending the life of components that remain functional. - Since people typically replace phones every four years, many retired devices still contain capable processors, accelerators, memory, and storage. ## Smartphone Performance and Limitations - Modern smartphone performance cores can match or exceed the per-core performance of some data-center servers. - Smartphones have significant limitations compared with servers: - Fewer, heterogeneous processor cores. - Only 8–12 GB of memory. - Less capacity for large, multithreaded workloads. - The platform therefore targets workloads that fit on a phone or can be distributed across multiple devices. ## Converting Phones into Datacenter Hardware - Unmodified phones are unsuitable for datacenters because they include unnecessary and potentially hazardous components such as: - Displays and cameras. - Batteries not designed for sustained datacenter operation. - Consumer-oriented chassis and peripherals. - Researchers remove everything except the motherboard, which accounts for roughly 50% of a phone’s embodied carbon. - Android’s mobile userspace is replaced with a general-purpose Linux distribution. - This removes mobile-specific restrictions such as Android’s “low memory killer” and enables broader server-style programmability. - Kubernetes manages containerized applications across clusters of approximately 25–50 phones, equivalent to roughly one conventional server. ## Applications for Education and Research - Many university workloads—including Jupyter notebooks, grading systems, and research applications—require modest resources that a single smartphone can provide. - Early tests showed that a 20-phone cluster could handle peak grading demand for a class of more than 75 students while achieving latency below a typical AWS backend. - The planned 2,000-phone cluster could support around 100 comparable classes simultaneously. - The deployment would provide approximately 50 server-equivalents at substantially lower cost. ## Testing Computing at Scale - The project will evaluate whether consumer smartphone hardware can operate reliably under sustained datacenter workloads. - It will also serve as a large-scale testbed for distributed smartphone computing. - The system is expected to launch at UC San Diego in fall 2026. Repurposing retired phones offers a practical way to reduce demand for newly manufactured computing hardware, especially for lightweight academic and cloud workloads. The approach is most promising when applications can tolerate distributed resources and the reliability challenges of consumer-grade components.

Read original(opens in new tab)

KubernetesAndroidDistributed SystemsCloud Computing+4