Google Research / deep-learning

Unlocking health insights: Estimating advanced walking metrics with smartwatches (opens in new tab)

Google researchers have validated that smartwatches are a highly reliable and accurate platform for estimating complex spatio-temporal gait metrics, rivaling the performance of smartphone-based methods. By utilizing a multi-head deep learning model, the study demonstrates that wrist-worn devices can provide continuous, lab-grade health insights into a user's walking speed, step length, and balance without requiring the specific pocket placement or specialized laboratory equipment previously necessary for such data. ## Multi-Head Deep Learning for Wrist-Based Sensors * The researchers developed a temporal convolutional network (TCN) architecture designed to process raw inertial measurement unit (IMU) data, specifically 3-axis accelerometer and gyroscope signals sampled at 50 Hz. * Unlike traditional models that only track temporal events and are prone to integration drift, this multi-head approach directly estimates both unilateral and bilateral metrics simultaneously. * The model architecture extracts embeddings from the IMU signals and concatenates them with user height (a demographic scalar input) to improve the precision of spatial predictions. * The system estimates a comprehensive suite of metrics, including gait speed, double support time (the proportion of time both feet are on the ground), step length, swing time, and stance time. ## Large-Scale Validation and Study Protocol * To ensure rigorous results, the study involved a diverse cohort of 246 participants across two international sites, generating approximately 70,000 walking segments. * Ground truth measurements were captured using a professional-grade Zeno Gait Walkway system to provide high-precision reference data for comparison. * The study protocol included various walking conditions to test the model's versatility: a self-paced six-minute walk test (6MWT), fast-paced walking, and induced physical asymmetry created by wearing hinged knee braces at specific angles. * Researchers employed a five-fold cross-validation strategy, ensuring that all data from a single participant remained within a single split to prevent data leakage and ensure the model generalizes to new users. ## Clinical Validity and Comparative Performance * Smartwatch estimates demonstrated strong validity and excellent reliability, with Pearson correlation coefficients (r) and intraclass correlation coefficients (ICC) exceeding 0.80 for most metrics. * Performance comparisons showed non-significant differences in Mean Absolute Percentage Error (MAPE) between the Pixel Watch and Pixel phone, establishing the smartwatch as a viable alternative to smartphone-based tracking. * While double support time showed slightly lower but acceptable reliability (ICC 0.56–0.60), other metrics like step length and gait speed proved highly consistent across different walking speeds and styles. * The model’s success suggests that smartwatches can effectively bridge the gap in gait analysis, providing a more practical and consistent platform for continuous health tracking than handheld devices. This research establishes smartwatches as a powerful tool for longitudinal health monitoring, enabling the detection of neurological or musculoskeletal changes through passive, continuous gait analysis in everyday environments.

Forecasting the future of forests with AI: From counting losses to predicting risk (opens in new tab)

Research from Google DeepMind and Google Research introduces ForestCast, a deep learning-based framework designed to transition forest management from retrospective loss monitoring to proactive risk forecasting. By utilizing vision transformers and pure satellite data, the team has developed a scalable method to predict future deforestation that matches or exceeds the accuracy of traditional models dependent on inconsistent manual inputs. This approach provides a repeatable, future-proof benchmark for protecting biodiversity and mitigating climate change on a global scale. ### Limitations of Traditional Forecasting * Existing state-of-the-art models rely on specialized geospatial maps, such as infrastructure development, road networks, and regional economic indicators. * These traditional inputs are often "patchy" and inconsistent across different countries, requiring manual assembly that is difficult to replicate globally. * Manual data sources are not future-proof; they tend to go out of date quickly with no guarantee of regular updates, unlike continuous satellite streams. ### A Scalable Pure-Satellite Architecture * The ForestCast model adopts a "pure satellite" approach, using only raw inputs from Landsat and Sentinel-2 satellites. * The architecture is built on vision transformers (ViTs) that process an entire tile of pixels in a single pass to capture critical spatial context and landscape-level trends. * The model incorporates a satellite-derived "change history" layer, which identifies previously deforested pixels and the specific year the loss occurred. * By avoiding socio-political or infrastructure maps, the method can be applied consistently to any region on Earth, allowing for meaningful cross-regional comparisons. ### Key Findings and Benchmark Release * Research indicates that "change history" is the most information-dense input; a model trained on this data alone performs almost as well as those using raw multi-spectral data. * The model successfully predicts tile-to-tile variation in deforestation amounts and identifies the specific pixels most likely to be cleared next. * Google has released the training and evaluation data as a public benchmark dataset, focusing initially on Southeast Asia to allow the machine learning community to verify and improve upon the results. The release of ForestCast provides a template for scaling predictive modeling to Latin America, Africa, and boreal latitudes. Conservationists and policymakers should utilize these forecasting tools to move beyond counting historical losses and instead direct resources toward "frontline" areas where the model identifies imminent risk of habitat conversion.

Using AI to identify genetic variants in tumors with DeepSomatic (opens in new tab)

DeepSomatic is an AI-powered tool developed by Google Research to identify cancer-related mutations by analyzing a tumor's genetic sequence with higher accuracy than current methods. By leveraging convolutional neural networks (CNNs), the model distinguishes between inherited genetic traits and acquired somatic variants that drive cancer progression. This flexible tool supports multiple sequencing platforms and sample types, offering a critical resource for clinicians and researchers aiming to personalize cancer treatment through precision medicine. ## Challenges in Somatic Variant Detection * Somatic variants are genetic mutations acquired after birth through environmental exposure or DNA replication errors, making them distinct from the germline variants found in every cell of a person's body. * Detecting these mutations is technically difficult because tumor samples are often heterogeneous, containing a diverse set of variants at varying frequencies. * Sequencing technologies often introduce small errors that can be difficult to distinguish from actual somatic mutations, especially when the mutation is only present in a small fraction of the sampled cells. ## CNN-Based Variant Calling Architecture * DeepSomatic employs a method pioneered by DeepVariant, which involves transforming raw genetic sequencing data into a set of multi-channel images. * These images represent various data points, including alignment along the chromosome, the quality of the sequence output, and other technical variables. * The convolutional neural network processes these images to differentiate between three categories: the human reference genome, non-cancerous germline variants, and the somatic mutations driving tumor growth. * By analyzing tumor and non-cancerous cells side-by-side, the model effectively filters out sequencing artifacts that might otherwise be misidentified as mutations. ## System Versatility and Application * The model is designed to function in multiple modes, including "tumor-normal" (comparing a biopsy to a healthy sample) and "tumor-only" mode, which is vital for blood cancers like leukemia where isolating healthy cells is difficult. * DeepSomatic is platform-agnostic, meaning it can process data from all major sequencing technologies and adapt to different types of sample processing. * The tool has demonstrated the ability to generalize its learning to various cancer types, even those not specifically included in its initial training sets. ## Open-Source Contributions to Precision Medicine * Google has made the DeepSomatic tool and the CASTLE dataset—a high-quality training and evaluation set—openly available to the global research community. * This initiative is part of a broader effort to use AI for early detection and advanced research in various cancers, including breast, lung, and gynecological cancers. * The release aims to accelerate the development of personalized treatment plans by providing a more reliable way to identify the specific genetic drivers of an individual's disease. By providing a more accurate and adaptable method for variant calling, DeepSomatic helps researchers pinpoint the specific drivers of a patient's cancer. This tool represents a significant advancement in deep learning for genomics, potentially shortening the path from biopsy to targeted therapeutic intervention.

Smarter nucleic acid design with NucleoBench and AdaBeam (opens in new tab)

Google Research and Move37 Labs have introduced NucleoBench, a comprehensive open-source benchmark for nucleic acid design, alongside AdaBeam, a high-performing new optimization algorithm. While AI models have become highly proficient at predicting the biological properties of DNA and RNA, generating optimal sequences within massive search spaces—such as the $2 \times 10^{120}$ possible variations for a 5' UTR—remains a significant hurdle. By standardizing evaluation across 16 distinct biological tasks, this research identifies AdaBeam as a superior method that scales effectively to the large-scale models required for modern drug discovery. ## Standardizing the Optimization Pipeline The process of computational nucleic acid design typically follows a five-step workflow: data collection, training a predictive model, generating candidate sequences (the design step), wet-lab validation, and iterative retraining. NucleoBench focuses specifically on the design step, which has historically lacked standardized evaluation. * Most existing benchmarks rely on decades-old methods like simulated annealing or vanilla genetic algorithms. * Traditional algorithms often treat predictive models as "black boxes," failing to leverage internal model data to guide the search. * The vastness of genomic search spaces makes brute-force optimization impossible, necessitating more intelligent, model-aware generation strategies. ## The NucleoBench Framework NucleoBench is the first large-scale benchmark designed to compare gradient-free and gradient-based design algorithms under identical conditions. The framework encompasses over 400,000 experiments to ensure statistical rigor across diverse biological challenges. * **Algorithm Categories**: It compares gradient-free methods (like directed evolution), which are simple but ignore model internals, against gradient-based methods (like FastSeqProp), which use the model’s internal "direction of steepest improvement" to find better sequences. * **Task Diversity**: The 16 tasks include controlling gene expression in specific cell types (liver or neuronal), maximizing transcription factor binding, and improving chromatin accessibility. * **Scale**: The benchmark includes long-range DNA sequence challenges using large-scale models like Enformer, which are computationally demanding but critical for understanding complex genomic interactions. ## AdaBeam’s Hybrid Optimization Performance Drawing on insights from the NucleoBench evaluation, the researchers developed AdaBeam, a hybrid algorithm that combines the strengths of various optimization strategies. * **Success Rate**: AdaBeam outperformed existing algorithms on 11 of the 16 tasks in the benchmark. * **Efficiency and Scaling**: Unlike many gradient-based methods that struggle with computational overhead, AdaBeam demonstrates superior scaling properties as sequences become longer and predictive models grow in complexity. * **Methodology**: It functions as a hybrid approach, using sophisticated search techniques to navigate the sequence space more effectively than "vanilla" algorithms developed before the era of deep learning. The researchers have made AdaBeam and the NucleoBench repository freely available to the scientific community. By providing a standardized environment for testing, they aim to accelerate the development of next-generation treatments, including more stable mRNA vaccines and precise CRISPR gene therapies.

Highly accurate genome polishing with DeepPolisher: Enhancing the foundation of genomic research (opens in new tab)

DeepPolisher is a deep learning-based genome assembly tool designed to correct base-level errors with high precision, significantly enhancing the accuracy of genomic research. By leveraging a Transformer architecture to analyze sequencing data, the tool reduces total assembly errors by 50% and insertion or deletion (indel) errors by 70%. This advancement is critical for creating near-perfect reference genomes, such as the Human Pangenome Reference, which are essential for identifying disease-causing variants and understanding human evolution. ## Limitations of Current Sequencing Technologies * Genome assembly relies on reading nucleotides (A, T, G, and C), but the microscopic scale of these base pairs makes accurate, large-scale sequencing difficult. * Short-read sequencing methods provide high signal strength but are limited to a few hundred nucleotides because identical DNA clusters eventually desynchronize, blending signals together. * Long-read technologies can sequence tens of thousands of nucleotides but initially suffered from high error rates (~10%); while tools like DeepConsensus have reduced this to 0.1%, further refinement is necessary for high-fidelity reference genomes. * Even a 0.1% error rate results in millions of inaccuracies across the 3-billion-nucleotide human genome, which can cause researchers to miss critical genetic markers or misidentify proteins. ## DeepPolisher Architecture and Training * DeepPolisher is an open-source pipeline adapted from the DeepConsensus model, utilizing a Transformer-based neural network. * The model was trained using a human cell line from the Personal Genomes Project that is estimated to be 99.99999% accurate, providing a "ground truth" for identifying and correcting errors. * The system takes sequenced bases, their associated quality scores, and the orientation of the DNA strands to learn complex error patterns that traditional methods might miss. * By combining sequence reads from multiple DNA molecules of the same individual, the tool iteratively "polishes" the assembly to reach the accuracy required for reference-grade data. ## Impact on Genomic Accuracy and Gene Discovery * The tool’s ability to reduce indel errors by 70% is particularly significant, as these specific errors often interfere with the identification of protein-coding genes. * DeepPolisher has already been integrated into major research efforts, including the enhancement of the Human Pangenome Reference, providing a more robust foundation for clinical diagnostics. * Improved assembly accuracy allows for better mapping of regions where the genome is highly repetitive, which were previously difficult to sequence and assemble confidently. For researchers and bioinformaticians, DeepPolisher represents a vital step in moving from "draft" genomes to high-fidelity references. Adopting this tool in assembly pipelines can drastically improve the reliability of variant calling and gene annotation, especially in complex clinical and evolutionary studies.

Insulin resistance prediction from wearables and routine blood biomarkers (opens in new tab)

Researchers at Google have developed a novel machine learning approach to predict insulin resistance (IR) by integrating wearable device data with routine blood biomarkers. This method aims to provide a scalable, less invasive alternative to traditional "gold standard" tests like the euglycemic insulin clamp or specialized HOMA-IR assessments. The study demonstrates that combining digital biomarkers with common laboratory results can effectively identify individuals at risk for type 2 diabetes, particularly within high-risk populations. ## Barriers to Early Diabetes Screening * Insulin resistance is a primary precursor to approximately 70% of type 2 diabetes cases, yet it often remains undetected until the disease has progressed. * Current diagnostic standards are frequently omitted from routine check-ups due to high costs, invasiveness, and the requirement for specific insulin blood tests that are not standard practice. * Early detection is vital because insulin resistance is often reversible through lifestyle modifications, making accessible screening tools a high priority for preventative medicine. ## The WEAR-ME Multimodal Dataset * The research utilized the "WEAR-ME" study, which collected data from 1,165 remote participants across the U.S. via the Google Health Studies app. * Digital biomarkers were gathered from Fitbit and Google Pixel Watch devices, tracking metrics such as resting heart rate, step counts, and sleep patterns. * Clinical data was provided through a partnership with Quest Diagnostics, focusing on routine blood biomarkers like fasting glucose and lipid panels, supplemented by participant surveys on diet, fitness, and demographics. ## Predictive Modeling and Performance * Deep neural network models were trained to estimate HOMA-IR scores by analyzing different combinations of the collected data streams. * While models using only wearables and demographics achieved an area under the receiver operating characteristic curve (auROC) of 0.70, adding fasting glucose data boosted the auROC to 0.78. * The most comprehensive models, which combined wearables, demographics, and full routine blood panels, achieved the highest accuracy across the study population. * Performance was notably strong in high-risk sub-groups, specifically individuals with obesity or sedentary lifestyles. ## AI-Driven Interpretation and Literacy * To assist with data translation, the researchers developed a prototype "Insulin Resistance Literacy and Understanding Agent" built on the Gemini family of large language models. * The agent is designed to help users interpret their IR risk predictions and provide personalized, research-backed educational content. * This AI integration aims to facilitate better communication between the data results and actionable health strategies, though it is currently intended for informational and research purposes. By utilizing ubiquitous wearable technology and existing clinical infrastructure, this approach offers a path toward proactive metabolic health monitoring. Integrating these models into consumer or clinical platforms could lower the barrier to early diabetes intervention and enable more personalized preventative care.

Measuring heart rate with consumer ultra-wideband radar (opens in new tab)

Google Research has demonstrated that ultra-wideband (UWB) radar technology, which is already integrated into many modern smartphones for tasks like precise location and vehicle unlocking, can be repurposed for contactless heart rate monitoring. By employing a transfer learning approach, researchers successfully applied models trained on large datasets from Frequency Modulated Continuous Wave (FMCW) radar to the newer UWB systems. This development suggests that everyday consumer electronics could soon provide accurate vital sign measurements without the need for additional specialized sensors or physical contact. ## Leveraging Existing Consumer Hardware While Google previously used Soli radar (FMCW) for sleep sensing in the Nest Hub, UWB technology represents a more widely available hardware platform in the mobile market. * UWB is currently used primarily for non-radar applications like digital car keys and item tracking (e.g., Apple AirTags). * The technology is increasingly standard in high-end mobile phones, providing a ready-made infrastructure for health sensing. * Utilizing existing UWB chips eliminates the need for manufacturers to add dedicated medical sensors to devices. ## Overcoming Signal Interference in Vital Sensing The primary challenge in radar-based heart rate monitoring is that the micro-movements of the chest wall caused by a heartbeat are significantly smaller than movements caused by breathing or general body shifts. * The system utilizes three-dimensional spatial resolution to create a "measurement zone" focused specifically on the user's torso. * High temporal resolution, sampling at speeds up to 200Hz, allows the radar to capture the rapid, subtle pulses of a heartbeat. * By isolating reflections from the chest area, the radar can ignore stationary background objects and external movements that would otherwise corrupt the data. ## Cross-Radar Transfer Learning Because the researchers possessed extensive datasets for FMCW radar but very limited data for UWB, they developed a method to transfer learned features between different radar types despite their different physical principles. * FMCW radar transmits continuous sinusoidal waves, whereas UWB radar transmits extremely short pulses (picoseconds to nanoseconds). * The study used a large 980-hour FMCW dataset to "teach" the model the characteristics of human vitals. * This pre-trained knowledge was then applied to a smaller 37.3-hour UWB dataset, proving that heart rate features are consistent enough across hardware types for effective transfer learning. ## A Novel Spatio-Temporal Deep Learning Model The researchers designed a custom neural network architecture to process the complex multidimensional data generated by radar sensors. * The framework uses a 2D ResNet to analyze the input data across two axes: time and spatial measurements. * Following the initial analysis, the model uses average pooling to collapse the spatial dimension, focusing purely on the temporal signal. * A 1D ResNet then identifies long-range periodic patterns to estimate the heart rate. * The model achieved a mean absolute error (MAE) of 0.85 beats per minute (bpm), which is a 50% reduction in error compared to previous state-of-the-art methods. This research indicates that high-precision health monitoring can be integrated into the mobile devices users already carry. By transforming smartphones into passive health sensors, UWB technology could allow for continuous heart rate tracking during routine activities, such as sitting at a desk or holding a phone in one's lap.

Unlocking rich genetic insights through multimodal AI with M-REGLE (opens in new tab)

Google Research has introduced M-REGLE, a multimodal AI framework designed to analyze diverse health data streams simultaneously to uncover the genetic underpinnings of complex diseases. By jointly modeling complementary signals—such as electrocardiograms (ECG) and photoplethysmograms (PPG)—the method captures shared biological information and reduces noise more effectively than unimodal approaches. This integrated analysis significantly enhances the discovery of genetic associations and improves the prediction of cardiovascular conditions like atrial fibrillation. ## Technical Architecture and Workflow M-REGLE utilizes a multi-step process to transform raw physiological waveforms into actionable genetic insights: * **Multimodal Integration:** Instead of processing data types in isolation, the model combines multiple inputs, such as the 12 leads of an ECG or paired ECG and PPG data, to capture overlapping signals. * **Latent Representation Learning:** The system employs a convolutional variational autoencoder (CVAE) to compress these high-dimensional waveforms into a low-dimensional "signature" or latent factors. * **Statistical Refinement:** Principal component analysis (PCA) is applied to the CVAE-generated signatures to ensure the learned factors are independent and uncorrelated. * **Genetic Mapping:** These independent factors are analyzed via genome-wide association studies (GWAS) to identify significant correlations between physiological signatures and specific genetic variations. ## Improved Data Reconstruction and Genetic Sensitivity The transition from unimodal (U-REGLE) to multimodal modeling has led to substantial gains in both data accuracy and biological discovery: * **Error Reduction:** M-REGLE achieved a 72.5% reduction in reconstruction error for 12-lead ECGs compared to analyzing each lead separately, indicating a much higher fidelity in capturing essential waveform characteristics. * **Increased Discovery Power:** In a study involving over 40,000 participants from the UK Biobank, the multimodal approach identified 3,251 significant genetic loci associated with 12-lead ECGs, a notable increase over the 2,215 loci found by unimodal methods. * **Novel Findings:** The model identified specific genetic links, such as the *RBM20* locus, which were previously missed by standard clinical measurements but are known to be critical for heart muscle function. ## Interpretability and Disease Prediction Beyond identifying associations, M-REGLE offers generative capabilities that help clinicians understand the relationship between latent data and physical health: * **Waveform Synthesis:** By altering specific coordinates within the learned embeddings, researchers can observe how individual latent factors correspond to physical changes in a patient's ECG T-wave or PPG peaks. * **Clinical Utility:** The model identified specific embeddings (positions 4, 6, and 10) that distinguish patients with atrial fibrillation (AFib) from those without. * **Predictive Performance:** M-REGLE’s embeddings outperformed traditional clinical polygenic risk scores (PRS) in predicting AFib, demonstrating the value of incorporating raw waveform data into risk assessments. ## Practical Applications Researchers and clinicians can leverage M-REGLE to extract richer insights from existing biobank data and wearable device outputs. By integrating multiple modalities into a single analytical pipeline, the framework provides a more comprehensive view of organ system health, facilitating the identification of therapeutic targets and more accurate disease screening protocols.