google·Feb 3, 2026·3 min readCurated summary

Collaborating on a nationwide randomized study of AI in real-world virtual care

Google and Included Health plan to launch, pending IRB approval, a nationwide randomized study of conversational AI in real-world virtual care. Unlike prior simulated or small feasibility studies, it will prospectively evaluate AI with consented patients across varied conditions and locations, comparing it with standard clinical practice. The goal is to generate rigorous evidence about safety, usefulness, limitations, and impact on patients and clinicians. ## Moving from Simulation to Real-World Evaluation - Earlier research demonstrated clinician-level capabilities in simulated consultations and retrospective analyses. - A feasibility study with Beth Israel Deaconess Medical Center began testing conversational AI in clinical workflows, using measures such as safety-supervisor interruptions. - The new study will advance beyond feasibility through: - A randomized controlled design - Nationwide recruitment - Consented participants - Real patients, clinical concerns, and virtual-care workflows - Controlled comparison with standard practice ## A Phased Approach to Medical AI Research - Google argues that medical AI should be evaluated with evidence standards similar to other medical interventions. - Each research phase adds information about: - Patient and clinician experiences - Safety - Usefulness - The AI system’s capabilities and limitations - Results from each stage are intended to guide safer, more responsible development and deployment. ## Foundational Research Behind the Study ### Diagnostic and Management Reasoning - The AMIE system was developed to handle medical interviews and clinical reasoning. - Studies with patient actors and synthetic cases found that AMIE could match or exceed primary care physicians in simulated diagnostic accuracy and conversation quality. - Later work expanded the system to: - Longitudinal disease management - Clinical-guideline and patient-history reasoning - Investigation and treatment planning - Interpretation of multimodal evidence ### Personalized Health Insights - Research on the Personal Health Agent examined how AI could interpret personal health data, including sleep and activity information from wearables. - Its multi-agent architecture combined the roles of: - Data scientist - Medical domain expert - Health coach - This work informed Fitbit Labs tools such as Symptom Checker and Medical Records Navigator and Plan for Care. ### Navigating Health Information - Google’s “wayfinding” AI research explored how conversational agents can help people find and understand health information. - The system uses proactive guidance, goal recognition, and tailored conversations to make health information searches more practical and useful. ## Practical Conclusion The partnership with Included Health represents a transition from demonstrating what medical AI can do in controlled environments to measuring how it performs at scale in actual care. A nationwide randomized trial could provide the evidence needed to determine whether conversational AI can safely improve virtual care and expand access to medical expertise.

Read original(opens in new tab)

Diagnostic ReasoningConversational AiHealthcare AiMultimodal Models+3

google·Apr 30, 2025Original article

AMIE gains vision: A research AI agent for multimodal diagnostic dialogue (opens in new tab)

Google Research and DeepMind have introduced multimodal AMIE, an advanced research AI agent designed to conduct diagnostic medical dialogues that integrate text, images, and clinical documents. By building on Gemini 2.0 Flash and a novel state-aware reasoning framework, the system can intelligently request and interpret visual data such as skin photos or ECGs to refine its diagnostic hypotheses. This evolution moves AI diagnostic tools closer to real-world clinical practice, where visual evidence is often essential for accurate patient assessment and management. ### Enhancing AMIE with Multimodal Perception To move beyond text-only limitations, researchers integrated vision capabilities that allow the agent to process complex medical information during a conversation. * The system uses Gemini 2.0 Flash as its core component to interpret diverse data types, including dermatology images and laboratory reports. * By incorporating multimodal perception, the agent can resolve diagnostic ambiguities that cannot be addressed through verbal descriptions alone. * Preliminary testing with Gemini 2.5 Flash suggests that further scaling the underlying model continues to improve the agent's reasoning and diagnostic accuracy. ### Emulating Clinical Workflows via State-Aware Reasoning A key technical contribution is the state-aware phase transition framework, which helps the AI mimic the structured yet flexible approach used by experienced clinicians. * The framework orchestrates the conversation through three distinct phases: History Taking, Diagnosis & Management, and Follow-up. * The agent maintains a dynamic internal state that tracks known information about the patient and identifies specific "knowledge gaps." * When the system detects uncertainty, it strategically requests multimodal artifacts—such as a photo of a rash or an image of a lab result—to update its differential diagnosis. * Transitions between conversation phases are only triggered once the system assesses that the objectives of the current phase have been sufficiently met. ### Evaluation through Simulated OSCEs To validate the agent’s performance, the researchers developed a robust simulation environment to facilitate rapid iteration and standardized testing. * The system was tested using patient scenarios grounded in real-world datasets, including the SCIN dataset for dermatology and PTB-XL for ECG measurements. * Evaluation was conducted using a modified version of Objective Structured Clinical Examinations (OSCEs), the global standard for assessing medical students and professionals. * In comparative studies, AMIE's performance was measured against primary care physicians (PCPs) to ensure its behavior, accuracy, and tone aligned with clinical standards. This research demonstrates that multimodal AI agents can effectively navigate the complexities of a medical consultation by combining linguistic empathy with the technical ability to interpret visual clinical evidence. As these systems continue to evolve, they offer a promising path toward high-quality, accessible diagnostic assistance that mirrors the multimodal nature of human medicine.

Diagnostic ReasoningAILarge Language ModelsAI Agents+5