google·Jun 29, 2025Original article

How we created HOV-specific ETAs in Google Maps (opens in new tab)

Google Maps has enhanced its routing capabilities by introducing HOV-specific ETAs, addressing the significant speed differences between carpool and general lanes. This was achieved through a novel unsupervised learning approach that classifies historical trips into HOV or non-HOV categories without initial manual labels. The resulting system enables more precise travel predictions, helping users optimize their commutes and supporting the shift toward sustainable travel modes. ### Segment-Level Speed Distribution * The model analyzes trip segments within short, 15-minute time windows to identify patterns in aggregated, anonymized traffic data. * During peak traffic hours, researchers often observe a bimodal speed distribution where HOV lanes maintain significantly higher average speeds compared to general lanes. * The classification system distinguishes between "Scenario A," where the speed gap is dramatic (e.g., 65 mph vs. 25 mph), and "Scenario B," where HOV lanes are only marginally faster, ensuring accurate modeling even when benefits are minimal. * Individual trip points, including speed and observation time, are processed collectively to determine if a specific segment of a journey occurred in a restricted lane. ### Incorporating Lateral Distance and Soft Clustering * To refine accuracy beyond simple speed metrics, the model incorporates the estimated lateral distance of a vehicle from the center of the road. * While GPS data is inherently noisy, this spatial information helps identify lane-specific behaviors by mapping trip points to the known physical location of HOV lanes (e.g., the far-left lanes). * The system employs soft clustering techniques, calculating the probability of a point belonging to a specific cluster rather than using hard binary assignments, which better manages borderline data points. * Temporal clustering via a weighted median approach is used to prioritize more recent traffic observations, ensuring the model accounts for the most current road conditions and availability constraints. By integrating these segment-level classifications into full-trip analyses, Google Maps can train its ETA prediction models on high-fidelity, lane-specific data. This implementation provides users with a more realistic view of their travel options, encouraging the use of high-occupancy lanes to reduce individual travel time, urban congestion, and overall emissions.

Unsupervised LearningAIMachine LearningTraffic Analysis+5

datadog·Jun 3, 2025·3 min readCurated summary

Detecting faulty deployments: Our journey from unlabeled data to supervised learning

Deployments are a major source of software incidents, making rapid detection of faulty releases essential. Datadog developed Automatic Faulty Deployment Detection to identify releases associated with significant, deployment-related increases in error rates, despite having no reliable labeled dataset. Their solution evolved into an iterative, unsupervised ensemble of statistical checks designed to balance precision, recall, and the diverse behavior of customer applications. ## Challenges in Detecting Faulty Deployments - No universal ground truth exists because teams define “faulty” differently depending on their applications. - Faulty deployments are rare, creating severe class imbalance: - Random manual labeling would produce few useful examples. - Even a low false-positive rate could result in poor precision. - Applications have widely varying traffic and error patterns: - Seasonal applications naturally experience periodic changes. - Low-traffic services need longer observation periods. - Frequent deployments can make it difficult to identify which release caused an incident. ## Defining a Faulty Deployment Datadog focused on deployments that caused a significant and sustained increase in error rate. The definition relied on three attributes: - **Impact** - The total number of errors must be meaningfully higher than the baseline. - The increase must be significantly worse than in previous versions. - **Temporal correlation** - The error increase should align with the introduction of the new version. - **Persistence** - The elevated error rate must continue over time rather than reflecting temporary deployment noise. ## Building an Iterative Detection Framework - The initial system applied simple statistical rules to the first 60 minutes after each deployment. - Manual annotation was used to estimate precision, but this required substantial effort and did not reveal recall. - Datadog created an iterative framework composed of checks for different deployment requirements. - Checks included: - Comparing error rates before and after deployment. - Comparing a release with previous versions. - Accounting for periodic traffic and errors. - Handling sparse traffic patterns. - The checks were combined into a unanimous-voting ensemble: a deployment was flagged only when every check classified it as faulty. - The process began with a high-recall model, then: - Manually reviewed predicted faults. - Analyzed false positives. - Added new checks and adjusted thresholds to improve precision and recall. - Incident data and version rollbacks provided additional signals for finding faulty deployments the model had missed. ## Balancing Detection Speed and Recall - The model used the first hour after deployment to gather enough data to determine whether increased errors were persistent. - Increasing the observation period can improve confidence but delays detection. - The framework became progressively more sophisticated, adapting to: - Periodic error and traffic patterns. - Sparse traffic. - Multiple concurrent application versions. The practical recommendation is to begin with simple, high-recall statistical rules, then iteratively improve them through targeted manual review, false-positive analysis, and additional operational signals such as incidents and rollbacks. This approach can support other anomaly-detection problems where labels are scarce, failures are rare, and application behavior varies significantly.

Read original(opens in new tab)

Unsupervised LearningDatabase DesignApplication Performance MonitoringIncident Management+4