Convolutional Neural Networks

googleJun 4, 20263 min readCurated summary

Towards passive heart health monitoring via smartphone camera

PHRM is a Google Research system that passively estimates heart rate and daily resting heart rate using short facial videos captured by a smartphone’s front-facing camera after face unlock. It applies remote photoplethysmography and efficient deep learning, achieving wearable-comparable accuracy in laboratory and real-world settings. A central contribution is its deliberate validation across skin tones, including substantial representation of participants with dark skin. ## Passive Heart Monitoring with a Smartphone - PHRM records approximately eight seconds of facial video during everyday phone use. - It estimates heart rate from changes in how light reflects from the skin as blood pulses through facial tissue. - The system uses: - Temporal shift convolutional neural networks for efficient on-device prediction. - Confidence scores to filter unreliable measurements. - Kalman filtering and daily aggregation to estimate resting heart rate. - Unlike earlier smartphone approaches requiring a finger over the camera, PHRM works passively in the background. ## Designing for Skin-Tone Inclusivity - Existing remote PPG research often relied on small, controlled studies and underrepresented people with dark skin. - Google trained PHRM on more than 350,000 video clips from nearly 700 participants across laboratory and real-world conditions. - Dataset targets based on the Monk Skin Tone scale ensured: - At least 25% representation of light and medium skin tones. - At least 33% representation of dark skin tones. - The researchers required heart-rate error differences between skin-tone groups to remain below five percentage points. ## Laboratory Validation - The model was trained using synchronized facial video and ECG recordings from 365 diverse participants. - On a separate 104-person test set, PHRM achieved mean absolute percentage error below 10% for every skin-tone group after confidence filtering. - It outperformed 15 leading published remote-PPG models and was the only evaluated model to meet the under-10% target across all groups. ## Real-World Smartphone Study - In an eight-day free-living study, 231 participants used their own phones normally while wearing an ECG chest strap and Fitbit Charge 6. - The research app captured an average of 231 facial clips per participant per day after face unlock. - Participants reviewed and explicitly authorized uploads each day; recordings were sent to secure, encrypted servers. - On a held-out group of 101 participants: - Overall MAPE was 6.09% after confidence gating. - MAPE was 5.04% for light skin, 5.12% for medium skin, and 7.84% for dark skin. - The average underestimation was only 0.64 beats per minute. - The model again outperformed the 15 comparison systems. ## Resting Heart Rate and Research Access - By combining reliable heart-rate estimates throughout the day, PHRM produced daily resting-heart-rate estimates with mean absolute error below five beats per minute compared with a wearable tracker. - Google released the largest and most diverse publicly available smartphone-video dataset for remote PPG research, along with a pre-trained “PHRM-mini” model. - Access is available to qualified researchers. PHRM demonstrates that smartphones could broaden access to passive cardiovascular monitoring without requiring dedicated wearable hardware. Before clinical use, the system would still require continued validation, careful privacy protections, and assessment across additional devices and populations.

Read original

Convolutional Neural Networks Deep Learning Computer Vision Photoplethysmography+3

googleOct 19, 2025Original article

Teaching Gemini to spot exploding stars with just a few examples (opens in new tab)

Researchers have demonstrated that Google’s Gemini model can classify cosmic events with 93% accuracy, rivaling specialized machine learning models while providing human-readable explanations. By utilizing few-shot learning with only 15 examples per survey, the model addresses the "black box" limitation of traditional convolutional neural networks used in astronomy. This approach enables scientists to efficiently process the millions of alerts generated by modern telescopes while maintaining a transparent and interactive reasoning process. ## Bottlenecks in Modern Transient Astronomy * Telescopes like the Vera C. Rubin Observatory are expected to generate up to 10 million alerts per night, making manual verification impossible. * The vast majority of these alerts are "bogus" signals caused by satellite trails, cosmic rays, or instrumental artifacts rather than real supernovae. * Existing specialized models often provide binary "real" or "bogus" labels without context, forcing astronomers to either blindly trust the output or spend hours on manual verification. ## Multimodal Few-Shot Learning for Classification * The research utilized few-shot learning, providing Gemini with only 15 annotated examples for three major surveys: Pan-STARRS, MeerLICHT, and ATLAS. * Input data consisted of image triplets—a "new" alert image, a "reference" image of the same sky patch, and a "difference" image—each 100x100 pixels in size. * The model successfully generalized across different telescopes with varying pixel scales, ranging from 0.25" per pixel for Pan-STARRS to 1.8" per pixel for ATLAS. * Beyond simple labels, Gemini generates a textual description of observed features and an interest score to help astronomers prioritize follow-up observations. ## Expert Validation and Self-Assessment * A panel of 12 professional astronomers evaluated the model using a 0–5 coherence rubric, confirming that Gemini’s logic aligned with expert reasoning. * The study found that Gemini can effectively assess its own uncertainty; low self-assigned "coherence scores" were strong indicators of likely classification errors. * This ability to flag its own potential mistakes allows the model to act as a reliable partner, alerting scientists when a specific case requires human intervention. The transition from "black box" classifiers to interpretable AI assistants allows the astronomical community to scale with the data flood of next-generation telescopes. By combining high-accuracy classification with transparent reasoning, researchers can maintain scientific rigor while processing millions of cosmic events in real time.

Convolutional Neural Networks AI Generative AI Gemini+4

googleOct 15, 2025Original article

Using AI to identify genetic variants in tumors with DeepSomatic (opens in new tab)

DeepSomatic is an AI-powered tool developed by Google Research to identify cancer-related mutations by analyzing a tumor's genetic sequence with higher accuracy than current methods. By leveraging convolutional neural networks (CNNs), the model distinguishes between inherited genetic traits and acquired somatic variants that drive cancer progression. This flexible tool supports multiple sequencing platforms and sample types, offering a critical resource for clinicians and researchers aiming to personalize cancer treatment through precision medicine. ## Challenges in Somatic Variant Detection * Somatic variants are genetic mutations acquired after birth through environmental exposure or DNA replication errors, making them distinct from the germline variants found in every cell of a person's body. * Detecting these mutations is technically difficult because tumor samples are often heterogeneous, containing a diverse set of variants at varying frequencies. * Sequencing technologies often introduce small errors that can be difficult to distinguish from actual somatic mutations, especially when the mutation is only present in a small fraction of the sampled cells. ## CNN-Based Variant Calling Architecture * DeepSomatic employs a method pioneered by DeepVariant, which involves transforming raw genetic sequencing data into a set of multi-channel images. * These images represent various data points, including alignment along the chromosome, the quality of the sequence output, and other technical variables. * The convolutional neural network processes these images to differentiate between three categories: the human reference genome, non-cancerous germline variants, and the somatic mutations driving tumor growth. * By analyzing tumor and non-cancerous cells side-by-side, the model effectively filters out sequencing artifacts that might otherwise be misidentified as mutations. ## System Versatility and Application * The model is designed to function in multiple modes, including "tumor-normal" (comparing a biopsy to a healthy sample) and "tumor-only" mode, which is vital for blood cancers like leukemia where isolating healthy cells is difficult. * DeepSomatic is platform-agnostic, meaning it can process data from all major sequencing technologies and adapt to different types of sample processing. * The tool has demonstrated the ability to generalize its learning to various cancer types, even those not specifically included in its initial training sets. ## Open-Source Contributions to Precision Medicine * Google has made the DeepSomatic tool and the CASTLE dataset—a high-quality training and evaluation set—openly available to the global research community. * This initiative is part of a broader effort to use AI for early detection and advanced research in various cancers, including breast, lung, and gynecological cancers. * The release aims to accelerate the development of personalized treatment plans by providing a more reliable way to identify the specific genetic drivers of an individual's disease. By providing a more accurate and adaptable method for variant calling, DeepSomatic helps researchers pinpoint the specific drivers of a patient's cancer. This tool represents a significant advancement in deep learning for genomics, potentially shortening the path from biopsy to targeted therapeutic intervention.

Convolutional Neural Networks AI Machine Learning Deep Learning+5