Improving trust with Datadog Log Management
Datadog handles hundreds of thousands of emails daily and uses Amazon SES for critical messages such as password resets. Because SES and CloudWatch did not provide sufficiently accessible, support-friendly event data, Datadog built a serverless pipeline that forwards SES events to Datadog Log Management. This provides low-maintenance delivery infrastructure, searchable email metrics, and monitoring for failures. ## Exporting Amazon SES Events - SES configuration sets define which email events to capture: - Send - Reject - Bounce - Complaint - Delivery - Open - Click - Events are published to an Amazon SNS topic. - SNS invokes an AWS Lambda function for every event. - The Lambda forwards the event to Datadog Logs using the Datadog API key. - Terraform provisions the SNS topic, SES configuration set, event destination, IAM role, and Lambda function. - The example uses Python 2.7 and stores the API key as a Lambda environment variable; production systems should encrypt the key. - This architecture avoids maintaining a custom email service while preserving visibility into email processing. ## Making SES Events Searchable in Datadog - SES events arrive in Datadog as JSON. - Datadog’s existing AWS integration pipeline processes the logs automatically. - Important fields can be converted into facets directly from a log entry. - Datadog uses fields such as the email event type and subject to quickly search for specific password reset activity. ## Monitoring and Operational Benefits - Support teams can verify whether a recipient received or interacted with a password reset email. - The entire delivery and logging pipeline is serverless and requires minimal maintenance. - Monitors can be configured on the logs to alert normal escalation channels when any part of the pipeline fails. - The solution combines the reliability of Amazon SES with Datadog’s observability and search capabilities. Overall, routing SES events through SNS and Lambda into Datadog Log Management is a practical way to create a trusted, observable password-reset email system without operating a separate mail infrastructure.
Read original(opens in new tab)