Visa’s Digital Commerce Authentication Program (DCAP) rewards US businesses for sharing richer transaction data with issuers during authentication, offering a five-basis-point net interchange reduction on qualifying transactions. However, businesses must balance savings against integration complexity, latency, fraud risk, and authorization rates. Stripe addresses this with transaction-level decisions about when to use Data Only 3DS.
## DCAP’s Opportunity and Complexity
- DCAP is designed to reduce card-not-present fraud and improve authorization rates.
- Businesses can share data such as:
- Device ID
- Billing address
- IP address
- Customer email
- Participation requires sending the required cardholder data through frictionless authentication.
- Issuers may interpret newer data signals differently, creating uncertainty around performance and latency.
- Businesses must evaluate whether interchange savings improve overall transaction economics without reducing conversions.
## Stripe’s Transaction-Level Optimization
- Stripe worked with Visa on readiness testing before rolling out DCAP.
- Stripe Authorization Boost determines which transactions should use Data Only 3DS.
- Data Only 3DS sends additional risk information through the card network to issuers without adding unnecessary customer friction.
- Instead of static rules, Authorization Boost evaluates each transaction based on:
- Potential cost savings
- Conversion impact
- Fraud risk
- Authorization performance
## Results and Eligibility
- Since April 18, Stripe has helped businesses generate $18.4 million in annualized network cost savings.
- Collecting and passing the required data increased DCAP-eligible transactions eightfold.
- Stripe continues working with Visa to expand eligibility.
## How to Participate
- Businesses using Authorization Boost and collecting the required data already receive the optimizations automatically.
- Businesses using standalone 3DS can participate by setting `flow_preference[type]` to `data_share` and ensuring all required fields are populated.
Businesses should adopt DCAP with transaction-level optimization rather than applying it indiscriminately, maximizing interchange savings while protecting authorization rates and customer conversion.
Spotify’s data assistant, Vedder, relies less on model size than on carefully curated domain context. With more than 70,000 datasets, schemas alone cannot capture business definitions, data quality issues, or preferred query patterns. Spotify’s solution is a cluster-based context layer owned by domain experts, making AI-generated SQL more reliable, transparent, and maintainable.
## Why Schemas Alone Are Not Enough
- Spotify has petabytes of data across more than 70,000 datasets, making it impossible to provide an LLM with the entire warehouse.
- Even large context windows cannot represent all available schemas effectively.
- Schema types and column names omit critical meaning, such as:
- Which values represent test or legacy data
- What “active user” means in a particular domain
- Which tables or columns are authoritative
- Without this context, an AI assistant may confidently choose the wrong dataset.
## Spotify’s Data Agent
- Users ask questions in natural language, and the agent:
- Selects the relevant context
- Generates SQL
- Executes it against the warehouse
- Returns the answer, query, and sources
- It uses a ReAct loop to reason, call tools, inspect results, and revise its approach.
- Users can see how an answer was produced rather than receiving an opaque result.
- The assistant is available through:
- Slack
- An MCP server for IDEs and AI tools
- A dedicated web interface
- Since August 2025, it has supported more than 2,100 users, 13,000 conversations, and 60,000 messages across 177 domain clusters.
## The Cluster Model
Spotify organizes data domains into “clusters,” each owned by a named team of experts. A cluster contains:
- **Datasets**
- Relevant warehouse tables with schemas and profiling
- Column cardinality, common values, and partition information
- Details that help the model construct accurate filters and queries
- **Pairs**
- Expert-approved natural-language questions paired with SQL
- Examples of both query patterns and domain semantics
- **Docs**
- Business terminology and definitions
- Known data pitfalls
- Guidance about which columns to use or avoid
Clusters can represent organizations, initiatives, or specialized areas of interest. Domain experts decide what belongs in each cluster and which examples best represent correct practice.
## Why Human Curation Matters
- Spotify considered automatically generating training pairs from historical query logs.
- That approach produced unreliable results because query history contains:
- Exploratory analysis
- Debugging queries
- One-off investigations
- Incorrect table choices
- Technically valid but misleading patterns
- Cluster curators accepted only 12.5% of the proposed question-SQL pairs.
- Experts therefore determine what is canonical and trustworthy, while the model uses that curated knowledge to answer more users.
- The goal is not to replace data specialists, but to scale their judgment and expertise.
## Keeping Context Current
- Data models and business logic change continuously.
- Cluster health scores monitor signals such as:
- Underlying data quality
- Whether curated SQL still works after schema changes
- Coverage of users’ real questions
- Reproducibility of generated SQL
- Renamed columns or deprecated tables can immediately reduce the validity of existing examples.
- Cluster owners use health dashboards and recommended actions to prioritize maintenance.
## Learning from Every Conversation
- Vedder records conversations, queries, answers, generated SQL, and user feedback.
- Cluster owners use this information to identify missing documentation, weak examples, and emerging needs.
- Each approved example or clarified definition improves future answers.
- The system treats context as an ongoing product that requires ownership and maintenance, not a one-time upload of metadata.
Spotify’s approach suggests that trustworthy enterprise AI depends on a maintained context layer: curated datasets, expert-approved examples, clear documentation, and continuous feedback. The model supplies reasoning and automation, but domain experts remain responsible for defining what the data means.
Google Research has open-sourced the hydrology framework behind its Flood Hub river forecasts. The Python/PyTorch package lets researchers and national forecasting agencies train AI models with global and local data while retaining control over their information. Google argues that open access, local expertise, and interoperable tools can make advanced flood warnings more accurate, affordable, and widely deployable.
## The Open-Source Hydrology Framework
- The framework is available on GitHub under an Apache 2.0 license.
- It provides model architectures, training pipelines, documentation, and tutorials.
- Users can train models with climate, soil, topography, land-cover, and weather data.
- Historical river observations come from the open Caravan dataset, which agencies can extend with local measurements.
- The package is built with PyTorch and is intended for both researchers and operational forecasters.
## Model Versions and Improvements
- The release includes:
- The original model used in Google’s 2024 benchmarking study.
- An upgraded v2 model currently used for real-time global forecasts in Flood Hub.
- The v2 model uses a multi-input ME-LSTM architecture.
- Separate networks embed different meteorological products before combining them in an LSTM.
- Inputs include GraphCast, ECMWF forecasts, NASA IMERG satellite rainfall estimates, and NOAA CPC precipitation data.
- Benchmarking showed the newer model extends the reliable forecast horizon by:
- Six days in gauged river basins.
- One day in ungauged basins.
## Local Data and Operational Forecasting
- Agencies can fine-tune models for specific watersheds using local observations and expert knowledge.
- The approach supports the integration of Indigenous and Local Knowledge, which the World Meteorological Organization says is still rarely incorporated systematically.
- Models are designed to be relatively inexpensive and easier to train than traditional conceptual hydrological systems.
- Local organizations can preserve control over their data while adapting the models to regional conditions.
## Partnership with the Czech Hydrometeorological Institute
- Google worked with CHMI to validate the model against locally calibrated traditional forecasting models.
- CHMI created an adapter connecting the framework to Delft-FEWS, a widely used operational forecasting platform.
- This integration demonstrates how machine-learning forecasts can fit into existing workflows used by government agencies, NGOs, and private organizations.
- The partnership provides a practical model for other national hydrological services.
## Broader Flood-Resilience Goals
- Open-source distribution could help resource-constrained regions access advanced forecasting without expensive infrastructure.
- The framework is intended to support capacity building for early-warning systems worldwide.
- Google presents the release as a way to let the global hydrology community reproduce, improve, and localize its research.
National hydrological agencies and researchers should evaluate the open-source framework using their own watershed data, integrate it with existing forecasting systems, and validate its predictions against established local models before operational deployment.
BewAIre evolved from a pull-request malware detector into a system for scanning dependency packages and upstream registries. Its core improvement is a two-stage pipeline: a cheap LLM filter handles routine changes, while a more capable agent investigates suspicious cases using external tools and repository context. This approach raised accuracy from 97.4% to 99.86%, eliminated false positives in a 690-diff sample, and reduced latency and cost through early exits.
## Expanding Beyond Pull Requests
- Software supply-chain attacks increasingly compromise trusted dependencies such as axios, LiteLLM, and Mistral.
- BewAIre initially focused on detecting malicious pull requests, identifying security testing, bug-bounty activity, and real attacks such as the Hackerbot campaign.
- The team aimed to apply the same LLM-based detection to complete packages and package registries without sacrificing accuracy, latency, or predictable cost.
## Limits of Single-Pass LLM Evaluation
- BewAIre began as a basic “LLM-as-judge” system that analyzed diffs through an inference API.
- More capable reasoning models improved detection but increased costs.
- Large diffs, especially dependency upgrades, challenged context-window limits.
- Two changes addressed these limitations:
- A filter-then-review escalation path.
- Tool-enabled investigation allowing models to gather additional evidence.
## Two-Stage Filtering and Investigation
- The filter phase:
- Runs on every change using a fast, inexpensive model.
- Uses straightforward prompts and diff chunking for large changes.
- Produces a binary suspicious/benign decision.
- Ends processing immediately when a change appears benign.
- The investigation phase:
- Runs only when the filter raises a concern.
- Uses a stronger reasoning model in an agentic loop.
- Can inspect commits, files, contributor histories, dependency metadata, and commit ranges through GitHub APIs.
- Checks for reverted commits, typosquatting, suspicious contributor behavior, and dependency risks using sources such as osv.dev and Datadog SCA.
## Detecting Obfuscated Attacks
- In the Hackerbot Claw example, the system identified a malicious filename containing shell command substitution.
- A base64-encoded payload decoded to a `curl ... | bash` command that downloaded and executed remote code.
- The investigation agent added useful context:
- The contributor account was newly created, had no profile information, and had no followers.
- The pull request had no reviews or approvals.
- `${IFS}` obfuscation was used to evade security filters.
- Combining code analysis with repository and author context made the final assessment more precise.
## Combining LLMs with Static Checks
- The filter model could mistakenly treat Datadog-like typosquatting domains as legitimate without access to investigative tools.
- BewAIre added preprocessing that extracts domains and compares them against a static list of known typosquatting variants.
- This hybrid design improves reliability while avoiding the cost and nondeterminism of performing every check through a powerful LLM.
## Measured Results
- Accuracy improved from 97.4% to 99.86% across 690 representative test diffs.
- False positives fell from 17 to zero.
- Most benign changes exit during the inexpensive filter stage.
- Suspicious changes still receive deeper analysis, preserving broad coverage while controlling latency and cost.
The practical recommendation is to combine inexpensive broad screening with selective, tool-driven investigation. Static security checks should complement LLM reasoning, especially for predictable threats such as domain typosquatting.
OpenAI GPT-5.5, GPT-5.4, and Codex are now generally available through Amazon Bedrock. GPT-5.5 targets the most demanding coding, reasoning, and agentic workloads, while GPT-5.4 emphasizes price-performance. Customers can access the models through the Responses API and use Codex across CLI, desktop, IDE, and Xcode integrations, with regional processing and token-based pricing.
## Model Access Through Amazon Bedrock
- Models are served through Bedrock’s next-generation inference engine and the OpenAI Responses API.
- GPT-5.5 is positioned for the hardest workloads; GPT-5.4 offers a balance of capability and cost.
- Processing remains within the selected Bedrock Region, supporting data residency requirements.
- Pricing is based on token usage, with no seat licenses or per-developer commitments.
## Calling GPT Models Programmatically
- The OpenAI Python SDK can be configured with Bedrock’s OpenAI-compatible endpoint:
- Install with `pip install -U openai`.
- Set `OPENAI_BASE_URL`, `OPENAI_API_KEY`, and `BEDROCK_OPENAI_MODEL_ID`.
- Applications can call `client.responses.create()` with:
- Developer and user messages
- Configurable reasoning effort
- Output verbosity controls
- The same endpoint can be called directly with `curl`.
- The Responses API supports multi-turn state, hosted and function tools, tool orchestration, and background or long-running work.
## Using Codex with Bedrock
- Codex is available through the Codex CLI, desktop app, VS Code, JetBrains, and Xcode integrations.
- It supports:
- A Bedrock API key via `AWS_BEARER_TOKEN_BEDROCK`
- The AWS SDK credential chain as a fallback
- Configure the model and Region in `~/.codex/config.toml`, for example:
- Model: `openai.gpt-5.5`
- Provider: `amazon-bedrock`
- Region: `us-east-2`
- Other supported model IDs include `openai.gpt-5.4`, `openai.gpt-oss-120b`, and `openai.gpt-oss-20b`.
- Desktop and VS Code users can place environment variables in `~/.codex/.env`.
- Applications must be restarted after configuration changes.
## Latency and Scaling Considerations
- Actual latency depends on reasoning effort, response length, tool calls, background execution, Region, quotas, throttling, prompt size, and cache hits.
- AWS recommends starting GPT-5.5 with medium reasoning effort.
- GPT-5.4 should use an explicitly chosen effort level rather than relying on its default of `none`.
- Bedrock’s inference engine is designed to provision capacity dynamically.
- During demand spikes, requests may be queued instead of rejected.
## Regional Availability
- GPT-5.5 is initially available in the US East (Ohio) Region.
- GPT-5.4 is available in US East (Ohio) and US West (Oregon).
- Additional Regions may be added over time.
Teams needing advanced coding and reasoning capabilities can now use OpenAI models and Codex through Bedrock while retaining AWS authentication, regional processing, and usage-based pricing. Evaluate reasoning effort, latency, quotas, and regional availability before moving workloads into production.
Cloudflare traced a dramatic increase in core-server boot times—from minutes to nearly four hours—to a firmware update that caused UEFI to probe network boot interfaces sequentially. Each failed IPv4 HTTPS or iPXE attempt waited roughly five minutes before the firmware reached the working IPv6 interface. By explicitly configuring the correct boot interface and adapting their automation to firmware limitations, Cloudflare reduced later boot times to under a minute.
## Network Boot in Cloudflare’s Core
- Core servers use automated network booting because they are bare-metal machines distributed across many environments.
- The fleet relies primarily on:
- PXE, often through the open-source iPXE firmware.
- Native UEFI HTTPS boot on hardware that supports it.
- iPXE enables scripted provisioning and can retrieve operating-system images over HTTP or HTTPS.
## The Linear Search That Caused Four-Hour Boots
- After a firmware update, affected machines completed POST and hardware initialization normally but stalled before reaching the operating system.
- UEFI tried network interfaces in sequence:
- IPv4 HTTPS boot
- IPv4 iPXE
- Repeated attempts
- IPv6 HTTPS boot, which eventually succeeded
- Each failed attempt consumed about five minutes waiting for a timeout.
- Multiple attempts added roughly 20 minutes to every boot.
- Firmware upgrades required several reboots, causing the delays to compound to nearly four hours per server.
## Declaring the Correct Boot Interface
- Cloudflare eliminated the linear search by configuring the preferred network boot interface order in advance.
- The configuration was moved earlier into the PXE pre-boot stage and tailored to each hardware type and use case.
- This removed repeated interface probing during firmware-upgrade reboots and reduced total upgrade time by about an hour.
- Subsequent boots fell from approximately 20 minutes to less than one minute.
## Handling Firmware Limitations
- Older UEFI versions did not support boot-order configuration.
- Firmware upgrades could reset the boot-order settings.
- Cloudflare added state validation to its automation:
- Check the configuration after changes or upgrades.
- Reapply the desired settings if they were lost.
- Reboot when necessary.
- The first boot could take slightly longer, but future boots became substantially faster.
## Working Around Lazy-Loaded Firmware Settings
- The network boot configuration was represented by an `EFI_IFR_REF3` structure that firmware loaded only when accessed through a graphical BIOS callback.
- Because the structure was not instantiated during programmatic scans, automation could not see or modify the network boot priorities.
- Cloudflare worked with hardware vendors to enable tokens in the fixed “Boot Order Module.”
- This forced the firmware to expose the network boot interface during startup without requiring manual BIOS interaction.
## Practical Conclusion
The incident demonstrates that boot delays can come from firmware behavior rather than hardware or operating-system failures. Explicitly declaring boot interfaces, validating configuration persistence, and coordinating with vendors to expose hidden firmware settings turned repeated timeout waits into predictable, automated boot cycles.
The provided Microsoft Engineering blog URL no longer contains the article “Scaling AI for Silicon.” Instead, it returns a 404 error page, so the post’s argument, technical details, and conclusion cannot be determined from the supplied content.
## Missing Article
- The requested URL points to `devblogs.microsoft.com/engineering-at-microsoft/scaling-ai-for-silicon/`.
- Microsoft reports: “Oops, 404 Error! That page can’t be found.”
- No article text, author, publication date, or technical discussion is included.
## Available Resources
- Links are provided to Microsoft Docs, Visual Studio, Microsoft Learn, Developer Community, and the Dev Blogs FAQ.
- The page also lists Microsoft’s other technology blogs, including Azure, .NET, AI, engineering, and Windows-related blogs.
- A link to the Engineering@Microsoft blog homepage is available, but the requested post itself is not.
The article cannot be accurately summarized without its original text or an accessible replacement URL.
AWS’s latest roundup highlights a rapid shift toward AI-augmented software development, illustrated by workshops where 17 teams delivered nearly 20 use cases in two days. The centerpiece is Anthropic Claude Opus 4.8, now available through Amazon Bedrock and the Claude Platform on AWS, alongside tools that improve resilience, agentic search, migration analysis, database management, and desktop streaming. Together, these launches show AWS moving toward real-time, AI-assisted development and operations.
## AI-Driven Development and Claude Opus 4.8
- AI-DLC workshops are helping teams deliver software use cases at unprecedented speed, especially when combined with Claude Code on Amazon Bedrock.
- Development roles are increasingly consolidating into smaller, AI-augmented teams.
- AWS account teams are shifting from producing advisory documents to building solutions collaboratively with customers.
- Claude Opus 4.8 is Anthropic’s most capable generally available model and is designed for:
- Agentic coding and extended autonomous tasks
- Long-document analysis and knowledge work
- Deeper reasoning and error recovery
- Maintaining context across lengthy coding sessions
- On Amazon Bedrock, it includes AWS features such as Guardrails, Knowledge Bases, and data-residency controls.
- The Claude Platform on AWS provides Anthropic’s native APIs with unified AWS billing.
## AWS Resilience Hub
- The next-generation Resilience Hub provides a unified way to define resilience standards, assess applications, and demonstrate compliance.
- Modular policies cover:
- Service-level objectives
- Multi-Availability Zone and multi-Region disaster recovery
- Data recovery
- Generative AI assessments align with AWS Well-Architected and Resilience Analysis Framework guidance.
- DNS query-log analysis automatically discovers application dependencies.
- AWS Organizations integration enables centralized, organization-wide resilience management.
## OpenSearch Serverless for Agentic AI
- Amazon OpenSearch Serverless is now positioned as a managed search and vector engine for agent-based applications.
- It scales from zero to thousands of requests per second and is approximately 20 times faster than the previous generation.
- AWS claims up to 60% cost savings compared with peak-provisioned clusters.
- New capabilities include GPU acceleration and dedicated `SEARCH` and `VECTORSEARCH` collection types.
- OpenSearch Agent Skills integrate with Vercel, Kiro, Claude Code, and Cursor.
## AWS Transform Assessment and Modernization Tools
- AWS Transform can build migration business cases and estimate total cost of ownership.
- It accepts data from RVTools, CMDBs, AWS discovery tools, and third-party discovery platforms.
- Users can model what-if scenarios involving regions, utilization, and service mappings for services such as EC2, FSx, S3, SQL Server on EC2, and virtual desktops.
- Agentic Readiness Analysis and Modernization Analysis scan repositories in roughly 5–30 minutes.
- Findings include severity levels, file-level evidence, and AWS-specific remediation recommendations.
## Aurora MySQL with Kiro Powers
- Aurora MySQL now integrates with Kiro Powers, which provide curated MCP servers, steering files, and hooks.
- Developers can use natural language for both:
- Data-plane tasks such as queries and schema changes
- Control-plane tasks such as cluster administration
- The integration offers guidance for Serverless scaling, RDS-to-Aurora migrations, and replication.
- Kiro generates the relevant API calls, SQL, and configuration for developer review before execution.
- Installation is available through the Kiro IDE or website.
## Windows Desktop Streaming with WorkSpaces Applications
- Amazon WorkSpaces Applications now supports bring-your-own-license Windows Desktop OS deployments.
- Customers can stream complete Windows desktops and applications from dedicated AWS hardware.
- BYOL removes operating-system fees, leaving compute and streaming infrastructure charges.
- Eligible Microsoft 365 Apps for enterprise are supported.
- Users receive a consistent experience across local and streamed desktops.
## Additional AWS News
- AWS introduced its May 2026 cohort of AWS Heroes.
- A new Vercel and AWS Databases integration allows provisioning Aurora PostgreSQL, DynamoDB, and Aurora DSQL through Vercel or v0.
- The H0 hackathon offers $160,000 in prizes for applications built with this stack.
- AWS GovCloud customers now receive 24/7 support from US-based, US-citizen engineers automatically, without opting in.
AWS’s direction is clear: combine capable AI agents with managed infrastructure, developer tools, and operational automation. Teams evaluating these services should focus on practical agentic use cases while retaining review and governance for generated code, infrastructure changes, and database operations.
Netflix’s Graph Abstraction is designed for OLTP graph workloads requiring millions of operations per second and millisecond-level latency, rather than open-ended analytical exploration. Built on existing Netflix abstractions, it supports real-time and optional historical graph views while handling nearly 10 million operations per second across 650 TB of data. Its core design emphasizes strong schemas, efficient traversal planning, low-latency caching, and controlled trade-offs such as eventual consistency and bounded query depth.
## OLTP Graph Use Cases
- Netflix distinguishes between:
- **OLAP workloads**, which prioritize large-scale exploration using RDF/SPARQL, property graphs, Gremlin, openCypher, or SQL.
- **OLTP workloads**, which require extremely high throughput, low latency, and global availability.
- OLTP queries may restrict traversal starting points, depth, or complexity to meet performance goals.
- Key applications include:
- **Real-Time Distributed Graph**, modeling dynamic relationships and interactions across Netflix.
- **Social Graph**, supporting social connections in Netflix Gaming.
- **Service Topology**, enabling real-time and historical analysis of internal services during incidents.
## Architecture and Netflix Data Abstractions
- The Graph Abstraction builds on existing platform components rather than implementing storage and caching independently.
- **Key-Value (KV) Abstraction** provides the latest state of nodes and edges and serves as the real-time index.
- **TimeSeries (TS) Abstraction** can be added for historical graph views.
- **EVCache** delivers low-millisecond latency, with additional specialized caching layers under experimentation.
- The **Data Gateway Control Plane** manages:
- Graph schemas
- Dataset provisioning and deletion
- KV and TS configuration
## Property Graph Model
- Graphs contain typed nodes and edges, each with associated properties.
- Properties are strongly typed to support:
- Efficient filtering
- Consistent data exports
- Validation during writes
- Edges may be:
- **Unidirectional**, representing one-way relationships
- **Bidirectional**, representing relationships traversable in both directions
## Namespaces and Provisioning
- Data is isolated into logical units called **namespaces**.
- Each namespace maps to a physical storage layer and may use dedicated or shared hardware.
- Provisioning automation selects an appropriate hardware configuration based on:
- Required throughput
- Latency targets
- Dataset size
- Workload criticality
## Graph Schema and Query Optimization
- Every namespace has an explicit schema defining:
- Node and edge types
- Valid properties and their types
- Allowed relationships
- Edge directions
- Schemas are represented through edge mappings, such as an `account owns profile` relationship or a bidirectional `profile linked_to device` relationship.
- Property definitions can specify types such as `TIMESTAMP` and `STRING`.
- Servers load schemas into an in-memory metadata graph, enabling:
- Rejection of invalid nodes, edges, and properties
- Faster traversal-path planning
- Deduplication of bidirectional edge traversals
- Removal of impossible paths and incompatible filters
- Servers periodically poll the Control Plane so schema changes are reflected without requiring manual updates.
- Planned improvements include:
- Using edge cardinality to reduce query fanout
- Generating type-safe data-access layers
- Making the Gremlin-like API schema-aware
## Real-Time Indexing with Key-Value Storage
- KV stores the real-time representation of all graph nodes and edges.
- Each namespace corresponds to a table, partitioned into records by unique IDs.
- Records contain multiple sorted key-value items, effectively forming a map of sorted maps.
- Writes to the same ID and key are idempotent, allowing safe retries and request hedging.
- KV uses timestamp-based tokens to enforce **Last-Write-Wins (LWW)** semantics.
- The post begins discussing the two-tier partitioning strategy for node storage, but the provided content ends before that design is explained.
Netflix’s approach demonstrates that high-throughput graph serving depends on specialized constraints and platform integration rather than unrestricted graph querying. Strong schemas, bounded traversals, KV-based indexing, automated provisioning, and low-latency caching together provide a practical foundation for production-scale OLTP graph workloads.
Netflix built Service Topology to give engineers a real-time, unified view of dependencies across its thousands of microservices. Traditional metrics, logs, and traces provide isolated signals but do not reveal the broader service relationships needed to diagnose failures or assess blast radius. The system combines multiple dependency sources into a living map that supports fast, context-rich troubleshooting.
## The Observability Problem
- Netflix’s distributed architecture involves thousands of services and complex chains of calls for actions such as playback, authentication, recommendations, and optimization.
- During incidents, engineers need to determine:
- Which services depend on one another
- What the potential blast radius is
- Whether a failure originates locally or upstream
- Existing observability tools show symptoms, logs, or individual request paths, but not the complete steady-state topology.
- Manually combining information from different tools is slow and error-prone, especially during urgent incidents.
## Why Real-Time Service Mapping Matters
- Frequent deployments and changing traffic patterns make static architecture diagrams quickly obsolete.
- Netflix’s Live programming and advertising-supported plans increase the need for rapid diagnosis and operational awareness.
- Engineers repeatedly asked about dependencies, failures, maintenance impact, unknown metrics, and recent call-path changes.
- These recurring questions demonstrated the need for accurate, near-real-time dependency information.
## Lessons from Earlier Approaches
- Netflix evaluated vendor platforms, graph databases, and internal prototypes before developing Service Topology.
- Key lessons included:
- Dependency data must update in near real time.
- Storage and query systems must operate at Netflix’s scale.
- The solution should integrate with existing observability workflows.
- Incorrect or incomplete topology data can mislead engineers during incidents.
- No single data source captures every aspect of service relationships.
## Requirements for a Living Map
Service Topology was designed to provide:
- Real-time updates as services deploy and dependencies change
- Sub-second queries for traversing service call graphs
- Both network-level and application-level views
- Context such as health, availability tiers, ownership, and business domains
- A visual interface for engineers and programmatic APIs for automation, resilience systems, and blast-radius analysis
## Combining Multiple Sources of Truth
Netflix separates dependency information into physically distinct graphs so each layer can evolve and be queried independently. When a unified view is requested, the system traverses the layers in parallel and merges the results to maintain fast response times.
### eBPF Network Flows
- eBPF captures network activity at the kernel level, recording which services communicate over the network.
- This provides broad coverage, including services that lack application instrumentation.
- It supports both cluster-level and application-level topology.
- Its limitation is that network traffic alone does not provide application-specific context, such as the APIs or endpoints involved.
Netflix’s approach is to combine complementary perspectives rather than rely on a single imperfect dependency source, producing a more complete and actionable service map.
Dr. Brian Harfe addressed generative AI in student writing by redesigning an essay assignment instead of relying on surveillance or AI detectors. In a 310-essay study, students began with AI-generated drafts and revised them into essays reflecting their own views, with word-level provenance tracked through Grammarly Authorship. The results suggest that assignment design can encourage meaningful engagement and provide more reliable insight into AI use than surveys or detection scores.
## Limits of Surveys and AI Detectors
- Surveys are influenced by students’ perceptions of acceptable behavior, fear of penalties, and difficulty recalling how much AI assistance they used.
- AI detectors provide probabilistic judgments rather than proof.
- Most detectors assess an entire document and cannot identify which passages were AI-generated or how the text developed.
- These methods measure the final product, not the writing process.
## An Assignment Built Around AI
- In the University of Florida course “Can We Design Better Humans? Should We?”, students had to start with a fully AI-generated essay.
- They then revised it to express their own views on human cloning and genetic engineering.
- Students could keep, modify, or discard as much of the AI draft as they wanted.
- Because AI use was explicitly permitted, the assignment removed the incentive to conceal it.
- Grammarly Authorship tracked whether each word was typed by the student, copied from AI, or drawn from another source.
- Students submitted authorship reports, allowing Harfe to replay the evolution from AI draft to final essay.
## Findings from 310 Essays
- The study included students from seven colleges and more than 100 majors.
- Students who wrote more original text generally spent more time completing the assignment, linking time-on-task with deeper revision.
- STEM students produced more human-generated text than non-STEM students, although both groups were similarly likely to agree with the AI draft.
- Higher-performing students revised AI-generated material more extensively across all disciplines.
- Students retained approximately 76% of the AI draft on average.
- The roughly 5% who disagreed with the AI’s position revised substantially more, adding more of their own writing.
- Only two students submitted the AI draft without edits, despite being explicitly allowed to do so for full credit.
## Implications for Education
- Harfe’s exact assignment may not apply to every course, but its underlying principle is broadly useful: incorporate AI into learning activities rather than treating it solely as a threat.
- Provenance tools provide a record of writing activity instead of an uncertain verdict about authorship.
- The findings challenge the assumption that students will automatically surrender their thinking to AI when given permission to use it.
- Students’ willingness to revise appears connected to academic engagement and performance.
- Reflective assignments can help students evaluate AI’s strengths, weaknesses, and appropriate future uses.
Instructors and institutions should focus less on detecting AI after the fact and more on designing assignments that require students to evaluate, revise, and take responsibility for AI-assisted work.
LY Corporation’s Orchestration Development Workshop demonstrated how Slack MCP can turn scattered Slack conversations into structured operational knowledge. Using AI, employees can summarize incidents, generate reports, create FAQs, and publish results to Confluence with minimal setup. The workshop’s central conclusion was that hands-on practice, reusable skills, and human review are essential for turning new AI capabilities into practical workplace tools.
## The Information and Adoption Challenge
- Slack contains valuable real-time information from incident response, customer inquiries, and project discussions.
- Much of this information remains unstructured because employees lack time to document it.
- Documentation quality varies by author, causing useful knowledge to become difficult to find.
- Although Slack MCP became available internally in March 2026, adoption barriers remained:
- Limited time to explore new tools
- Concerns about complex configuration
- Slow internal distribution of technical knowledge
## Introducing Slack MCP Through Hands-On Practice
- Slack MCP is an internally developed MCP server connected to company authentication.
- Employees can access internal Slack data without issuing personal tokens or configuring OAuth.
- The workshop began with a simple exercise:
- Launch a coding tool such as Claude Code
- Ask the AI to post “Hello” in a designated Slack channel
- Confirm that the message was actually posted
- This immediate success helped participants understand MCP’s practical capabilities.
## Combining Slack MCP with Other MCP Servers
Slack MCP supports several core operations:
- Reading messages and threads
- Posting messages and performing actions
- Looking up channels and members
- Searching Slack content
Combined with other MCP servers, it can support broader workflows:
- Slack plus Confluence MCP: Generate and publish project reports or FAQs
- Slack plus Jira MCP: Create work tickets from discussions
- Slack conversations can be transformed into structured documents rather than remaining isolated in chat history.
## Automatically Creating FAQs from Slack Inquiries
The first major exercise converted repeated support discussions into reusable knowledge.
- Slack inquiry threads were collected and converted into FAQ-formatted Markdown.
- Existing Confluence content was checked to identify duplicates.
- New FAQs were published as child pages under an existing Confluence knowledge base.
- The output was formatted as a table containing:
- Symptoms
- Causes
- Solutions
The workflow was packaged into reusable skills such as:
- `slack-to-faq`: Searches recent inquiry threads and generates new FAQ files
- `faq-to-confluence`: Converts and publishes the FAQs to Confluence
This demonstrated how MCP can automate the entire path from conversation search to knowledge-base publication.
## Supporting Incident Response
The second exercise focused on reducing the time needed to understand and document incidents.
### Rapid Situation Summaries
Participants could ask the AI to summarize an outage in natural language. MCP searched relevant Slack threads and organized the information into:
- Current resolution status
- Customer impact
- Actions being taken by team members
- A chronological timeline
This helps managers or newly joining responders understand the situation quickly without reading every thread.
### Automated Incident Reports
After resolution, the AI generated reports in a specified format, including:
- Incident and detection times
- Duration
- Root cause
- Affected users and features
- Whether data was lost
- Remediation steps
The `slack-incident-status` and `slack-incident-report` skills separated real-time status checking from post-incident documentation.
## Practical Guidelines and Safeguards
- Clean and constrain source data before processing it:
- Compare results with existing Confluence FAQs
- Filter messages using reactions or other markers
- Limit searches to relevant channels and threads
- Do not publish AI-generated documents without review.
- Check for personal information and confirm that the output accurately reflects the source conversations.
- Include links or references to the original Slack threads.
- Specify the desired output structure, such as a three-column table for symptoms, causes, and solutions.
- Convert successful prompts into reusable skills so teams can avoid rewriting complex instructions and maintain consistent output quality.
## Lessons from the Workshop
- **Timing matters:** Holding the workshop soon after Slack MCP became available captured user interest and accelerated experimentation.
- **Practice is more effective than explanation:** Starting with a simple Slack post and progressing to FAQs and incident reports made the benefits immediately tangible.
- **Real work makes training relevant:** Inquiry handling and incident response were chosen because they are common, time-consuming tasks.
- **Reusable skills improve adoption:** Prompt patterns were tested manually, refined, saved as skills, and shared with participants for continued workplace use.
The recommended approach is to introduce new AI tools through timely, task-focused workshops, then refine successful workflows into shared skills. MCP can greatly reduce the effort of operational documentation, but human validation remains necessary before generated knowledge is published.
AWS introduces the next generation of AWS Resilience Hub to help organizations manage application resilience consistently across large portfolios. The update adds a new business-oriented application model, modular resilience policies, dependency discovery, generative AI failure-mode analysis, and organization-wide reporting. Its goal is to help SREs and development teams define expectations, identify weaknesses, implement fixes, and demonstrate resilience through testing.
## Modular Resilience Policies
- Policies are built from composable requirements rather than a single fixed policy type.
- Requirements can include:
- Availability service-level objectives (SLOs)
- Multi-AZ and multi-Region disaster recovery
- Recovery time objectives (RTOs)
- Recovery point objectives (RPOs)
- Backup data recovery requirements
- Example policies can define 99.95% availability, a 15-minute RTO, and a 5-minute RPO for financial applications.
## Business-Oriented Application Modeling
- A **system** represents a business application.
- **User journeys** describe critical end-user paths and business outcomes.
- **Services** represent deployable units such as microservices, AWS resources, code, and observability components.
- Resilience Hub automatically maps resource relationships into a topology showing data flow, containment, and permissions.
## Dependency Discovery
- Resilience Hub analyzes VPC DNS query logs to identify dependencies that may not be documented.
- It can discover:
- AWS services
- Internal endpoints
- Third-party endpoints
- Unexpected cross-Region calls
- Critical external dependencies
- Dependency discovery can be enabled or disabled for individual services.
## Generative AI Failure-Mode Assessments
- Assessments analyze services against:
- Configured resilience policies
- AWS Well-Architected best practices
- The AWS Resilience Analysis Framework
- Findings explain the failure mode, its architectural impact, recommended remediation, and related policy requirement.
- Users can add or modify assertions to guide the AI agents and improve assessment accuracy.
- Findings can be marked as resolved after remediation or irrelevant when they do not apply.
## Getting Started and Enterprise Management
- Users must configure an invoker IAM role granting read-only access to AWS resources.
- Cross-account roles or AWS Organizations service-linked roles can support multi-account environments.
- AWS Organizations integration allows a delegated administrator to assess resilience across the enterprise without signing into individual accounts.
- A typical workflow is:
- Create a resilience policy.
- Create a system and service.
- Associate resources through tags, CloudFormation, Terraform, or Amazon EKS.
- Enable dependency discovery.
- Run a failure-mode assessment.
- Review and implement recommendations.
## Migration, Availability, and Pricing
- Migration APIs convert older Resilience Hub policies and applications into the new model, including mapping multiple related applications into one system with multiple services.
- The new version is generally available in supported AWS commercial Regions.
- Pricing uses a service-based model and includes two failure-mode assessments per service per month, with automated dependency assessment available optionally.
- AWS offers a free trial.
The updated Resilience Hub is most useful for organizations that need consistent resilience standards and evidence across many AWS accounts and applications. Teams should begin by defining reusable policies, modeling critical services and dependencies, and using the AI assessments to prioritize remediation.
