Lstm | Techlist.io

googleJun 3, 20263 min readCurated summary

The next chapter in flood resilience: Open sourcing Google’s hydrology framework

Google Research has open-sourced the hydrology framework behind its Flood Hub river forecasts. The Python/PyTorch package lets researchers and national forecasting agencies train AI models with global and local data while retaining control over their information. Google argues that open access, local expertise, and interoperable tools can make advanced flood warnings more accurate, affordable, and widely deployable. ## The Open-Source Hydrology Framework - The framework is available on GitHub under an Apache 2.0 license. - It provides model architectures, training pipelines, documentation, and tutorials. - Users can train models with climate, soil, topography, land-cover, and weather data. - Historical river observations come from the open Caravan dataset, which agencies can extend with local measurements. - The package is built with PyTorch and is intended for both researchers and operational forecasters. ## Model Versions and Improvements - The release includes: - The original model used in Google’s 2024 benchmarking study. - An upgraded v2 model currently used for real-time global forecasts in Flood Hub. - The v2 model uses a multi-input ME-LSTM architecture. - Separate networks embed different meteorological products before combining them in an LSTM. - Inputs include GraphCast, ECMWF forecasts, NASA IMERG satellite rainfall estimates, and NOAA CPC precipitation data. - Benchmarking showed the newer model extends the reliable forecast horizon by: - Six days in gauged river basins. - One day in ungauged basins. ## Local Data and Operational Forecasting - Agencies can fine-tune models for specific watersheds using local observations and expert knowledge. - The approach supports the integration of Indigenous and Local Knowledge, which the World Meteorological Organization says is still rarely incorporated systematically. - Models are designed to be relatively inexpensive and easier to train than traditional conceptual hydrological systems. - Local organizations can preserve control over their data while adapting the models to regional conditions. ## Partnership with the Czech Hydrometeorological Institute - Google worked with CHMI to validate the model against locally calibrated traditional forecasting models. - CHMI created an adapter connecting the framework to Delft-FEWS, a widely used operational forecasting platform. - This integration demonstrates how machine-learning forecasts can fit into existing workflows used by government agencies, NGOs, and private organizations. - The partnership provides a practical model for other national hydrological services. ## Broader Flood-Resilience Goals - Open-source distribution could help resource-constrained regions access advanced forecasting without expensive infrastructure. - The framework is intended to support capacity building for early-warning systems worldwide. - Google presents the release as a way to let the global hydrology community reproduce, improve, and localize its research. National hydrological agencies and researchers should evaluate the open-source framework using their own watershed data, integrate it with existing forecasting systems, and validate its predictions against established local models before operational deployment.

Read original

Lstm Machine Learning Python Pytorch+3

googleMar 12, 20263 min readCurated summary

Protecting cities with AI-driven flash flood forecasting

Google Research is expanding Flood Hub with urban flash flood forecasts that can provide up to 24 hours’ warning. The system addresses the lack of historical flood observations by using Gemini to extract verified events from public news reports, creating the Groundsource dataset for model training. Its global, lower-resolution approach aims to extend useful warnings to regions that lack expensive sensors and forecasting infrastructure, particularly in the Global South. ## The Need for Earlier Flash Flood Warnings - Flash floods cause roughly 85% of flood-related deaths worldwide and kill more than 5,000 people annually. - They often develop within six hours of intense rainfall, making rapid warnings essential. - Even 12 hours of warning can reduce flood damage by about 60%. - Early warning coverage remains highly unequal: fewer than half of developing countries have access to multi-hazard warning systems. - Flood Hub previously focused mainly on slower-moving riverine floods, covering more than 2 billion people across 150 countries. ## The Data Problem: “Invisible” Floods - River flood models can rely on stream gauges that record water levels and flow. - Flash floods may occur far from gauges, especially in cities where rainfall, impermeable surfaces, drainage, and terrain interact unpredictably. - Building detailed physical simulations globally would be computationally expensive. - Historical, precisely located flash flood records are also scarce, preventing conventional supervised machine learning. - Google’s Groundsource method uses Gemini to analyze public news reports, verify flood locations and times, and assemble a historical flash flood dataset. ## Scaling from Local Systems to Global Coverage - Local flash flood systems can be highly accurate using rain sensors, radar, water-level monitors, and flow measurements. - These systems are expensive to deploy and require location-specific calibration and engineering expertise. - Broader systems such as WMO’s FFGS, ERIC, and the U.S. NWS warning system depend on high-resolution maps, radar forecasts, and skilled hydrologists. - Those resources are often unavailable in the Global South. - Google’s model instead uses globally available products, including NASA IMERG, NOAA CPC, ECMWF’s IFS HRES forecasts, and Google DeepMind’s medium-range weather model. - Forecasts currently operate at a 20-by-20-kilometer resolution, constrained by the resolution of global data sources. ## The Urban Flash Flood Model - The model estimates whether a flash flood is likely in a given area during the next 24 hours. - It uses a recurrent neural network with a long short-term memory (LSTM) component to process meteorological time series. - Inputs also include static geographic and human-environment factors: - Urbanization density - Topography - Soil absorption rates - The initial rollout targets urban regions, where news coverage is denser and most of the world’s population lives. - It currently predicts impacts in areas with population densities above 100 people per square kilometer. ## Evaluation and Reported Performance - Precision was measured against the Groundsource dataset, but raw precision likely understates actual performance because some genuine floods are never reported. - A manual review of 100 alerts per continent found that many apparent false positives were confirmed flood events. - Recall was also evaluated against major floods recorded by the Global Disaster Awareness and Coordination System (GDACS). - Results indicate comparable precision and recall in regions such as South America and Southeast Asia and in wealthier countries with better instrumentation. The approach demonstrates how AI and unstructured public information can help provide scalable flash flood warnings where conventional monitoring infrastructure is limited. Its current urban focus and 20-kilometer resolution make it a broad early-warning tool rather than a replacement for highly localized sensor networks.

Read original

Lstm AI Machine Learning Data Analytics+4