techlist.io
KREN

Techlist.io - Korean Tech Blog Curator

gitlab3 min readCurated summary

Automate deployment processes with GitLab Duo Agent Platform

GitLab Duo Agent Platform can automate the complex, repetitive work of onboarding a microservice into an established GitOps workflow. By analyzing an application’s repositories and configuration, a custom agent can generate manifests, update pipelines, configure image automation, and follow organization-specific conventions. The approach combines AI-driven speed with GitLab-managed versioning, governance, and enterprise security. ## TanukiBank’s GitOps Use Case - The fictional TanukiBank application needs a new `intra-account-transfers` microservice for its Quick Transfer feature. - Its deployment architecture includes: - Individual service projects with container registries and build pipelines. - **Tanuki Bank - Delivery**, which stores deployment manifests and delivery pipelines. - **Flux Config**, which contains Flux manifests for Kubernetes. - Flux Image Automation watches service registries and updates corresponding delivery manifests. - A delivery pipeline then builds and signs the image, while Flux CD synchronizes it to the Kubernetes cluster. - Adding a service manually requires coordinated changes across all these components. ## Generating the Custom Agent’s System Prompt - GitLab Duo Agentic Chat examines the TanukiBank group, subgroups, source files, Dockerfiles, manifests, configuration, and dependencies. - It generates a detailed system prompt describing: - The existing GitOps workflow. - Required operating rules. - Reporting instructions. - Recommended tools. - The prompt is specific to the workflow at the time it is generated. - If the application’s GitOps process changes, the prompt should be regenerated. ## Creating and Configuring the Agent - A new `application-agents` project manages custom agents, their administrators, and where they can run. - A managed agent named **TanukiBank Microservice Onboarder** is created with: - A description. - The generated system prompt. - Tools recommended by GitLab Duo. - The agent is enabled in both **Tanuki Bank - Delivery** and **Flux Config**. - Its presence in each project’s Agentic Chat agent selector confirms that it is available. ## Creating the Microservice - A new `services/intra-account-transfers` project is created. - GitLab Duo’s **Developer** foundational flow implements the service from an issue specification. - The flow: - Reads the requirements. - Writes the implementation. - Creates a branch and merge request. - Links the merge request to the issue. - After local verification with `curl`, the merge request is merged and the project pipeline publishes container images. - At this stage, the service exists, but the GitOps system has not been updated: - `manifests/dev` has no service manifests. - The delivery pipeline does not reference the service. - `Flux Config` lacks an `image-update-automation.yaml` entry. ## Using the Custom Onboarding Agent - The custom agent is enabled in the new service project. - From **Tanuki Bank - Delivery**, the user selects **TanukiBank Microservice Onboarder** in Agentic Chat and provides the service name and hostname. - The agent begins onboarding by: - Finding and reading the service’s Dockerfile. - Determining the application port. - Generating the required Kubernetes manifests. - Updating the relevant delivery pipelines. - This automates the coordinated repository changes normally required for a new microservice. ## Practical Takeaway A custom GitLab Duo agent is most valuable when it is grounded in an organization’s real repositories and deployment conventions. Generate its prompt from the current system, keep the agent centrally governed, and regenerate the prompt whenever the GitOps workflow changes.

Read original(opens in new tab)
KubernetesCI/CDDocker & ContainersMicroservices+3
gitlab2 min readCurated summary

A leaked personal access token shouldn't expose every project its owner can reach. Fine-grained PATs scope each token’s permissions to the job.

Fine-grained personal access tokens (PATs) reduce credential exposure by limiting each token to only the projects, groups, resources, and actions required for a specific task. GitLab’s beta lets users replace broad `api` or `read_api` tokens with narrowly scoped permissions, reducing the impact of leaks. The feature is not yet recommended for production because coverage is still incomplete. ## Why Narrow PAT Privileges - Broad, user-scoped tokens can access every project the user can reach. - A leaked token might expose source code, pipelines, container images, or CI/CD variables across many projects. - Fine-grained tokens limit both access and potential remediation to the affected project or resource. - They complement lifetime limits and automatic revocation. ## How Fine-Grained Tokens Work - Scope access by location: - Personal projects - All projects and groups where the user is a member - Specifically selected projects and groups - Assign independent Create, Read, Update, and Delete permissions. - Supported resources include Issues, Merge Requests, Pipelines, Repositories, and Container Registry. - Example: a container-publishing pipeline can receive Create and Read access only to one project’s registry. ## Auditing and Beta Coverage - The token management table displays scopes and per-resource permissions for all tokens. - This makes over-privileged credentials easier to identify during reviews. - Fine-grained PATs currently support about 75% of REST API endpoints. - GitLab plans to add remaining REST endpoints and expand GraphQL support. - Existing traditional PATs continue working alongside fine-grained tokens during the beta. ## Getting Started - Go to **User Settings → Personal Access Tokens**. - Select **Fine-grained token** when generating a token. - Choose the permitted projects or groups and assign resource permissions. - GitLab recommends avoiding fine-grained PATs in production until general availability. Teams should begin evaluating fine-grained tokens for automation and adopt one token per job, with the smallest practical scope. Feedback during the beta will help shape broader endpoint coverage and future improvements.

Read original(opens in new tab)
GitlabGraphQLRest ApiApi Security+3
github3 min readCurated summary

Validating agentic behavior when “correct” isn’t deterministic

Agentic systems such as GitHub Copilot cloud agent can complete tasks through multiple valid action sequences, making traditional deterministic tests unreliable. Timing changes, loading screens, and UI differences often produce false negatives even when the agent achieves the correct result. The post proposes an independent “Trust Layer” that validates essential outcomes and convergent behavior rather than rigid step-by-step execution paths. ## Challenges of Agent-Driven Validation - An agent may adapt to network delays or changing UI conditions and still complete its task successfully. - Conventional CI tests can fail when execution no longer matches a recorded script or expected assertion timing. - This creates a trust gap: - **False negatives:** successful tasks are reported as failures. - **Fragile infrastructure:** rendering, timing, and environment noise affect test results. - **Compliance trap:** valid behavioral variation is mistaken for regression. - Agent correctness should focus on reliably achieving essential outcomes, not reproducing an identical sequence of actions. ## Why Traditional Testing Breaks Down - **Assertion-based tests** require manually specifying every expected check and often omit valid alternative paths. - **Record-and-replay tools** are highly sensitive to timing and rendering differences. - **Visual regression tests** compare screenshots without understanding semantic meaning or the broader workflow. - **ML-based oracles** need large training datasets and generally provide little explanation for their decisions. - All four approaches assume correctness means following a stable sequence of observable states, which does not fit autonomous agents. ## Essential, Optional, and Convergent Behavior The proposed approach distinguishes between behavior that determines success and behavior that merely reflects environmental variation: - **Essential states:** Required milestones, such as reaching a VS Code “Search Results” screen. - **Optional variations:** Incidental states, including loading spinners or decorative UI changes. - **Convergent paths:** Different action sequences—such as using a keyboard shortcut or a menu—that eventually reach the same result. - A loading screen may appear in one run and not another, but the appearance of search results is what establishes success. ## Dominator Analysis The post connects this model to **dominator relationships** from compiler theory: - In a control-flow graph, node A dominates node B when every path to B must pass through A. - Applying dominator analysis to agent execution traces can identify: - Mandatory states - Optional states - Points where different execution paths converge - This produces a minimal and explainable definition of correctness instead of relying on every recorded step. ## Graph-Based Execution Modeling - Agent behavior should be represented as a graph rather than a linear script. - Graphs capture branching paths, optional states, and convergence points. - This structure provides a foundation for lightweight, explainable validation in GitHub Actions and other CI environments. A reliable validation system for agents should test whether essential outcomes occurred and whether critical invariants held, while ignoring harmless differences in timing, rendering, and execution order. This outcome-oriented Trust Layer can reduce false failures and make agentic workflows more dependable in production CI pipelines.

Read original(opens in new tab)
AI AgentsCI/CDGithub CopilotGitHub Actions+3
cloudflare3 min readCurated summary

When DNSSEC goes wrong: how we responded to the .de TLD outage

On May 5, 2026, DENIC published invalid DNSSEC signatures for the `.de` zone, causing validating resolvers to return `SERVFAIL` for affected domains. Cloudflare’s 1.1.1.1 mitigated much of the impact by serving expired cached records and temporarily bypassing DNSSEC validation for `.de`. The incident demonstrates both the value of DNSSEC and the operational risks of a failure at a top-level domain. ## How DNSSEC Protects DNS - DNSSEC adds cryptographic authentication to DNS records through `RRSIG` signatures. - It protects integrity and authenticity, but does not encrypt DNS traffic. - DNSSEC creates a chain of trust from the root zone to child domains: - The root delegates trust to `.de`. - `.de` delegates trust to domains such as `example.de`. - A failure anywhere in the chain causes validation to fail below that point. - Zones generally use: - A Zone Signing Key (ZSK) for signing records. - A Key Signing Key (KSK) for signing the ZSK. - Errors during key rotation can produce signatures that resolvers cannot validate, forcing them to return `SERVFAIL`. ## Impact of the `.de` Outage - Around 19:30 UTC, DENIC began publishing invalid DNSSEC signatures for `.de`. - Any DNSSEC-validating resolver, including Cloudflare’s 1.1.1.1, had to reject the responses. - The failure spread across domains under `.de`, potentially affecting millions of German websites and services. - `SERVFAIL` responses increased gradually as cached records expired and resolvers requested fresh, invalidly signed data. - Query volume also rose because clients commonly retry failed DNS queries multiple times. ## How “Serve Stale” Reduced the Damage - Recursive resolvers normally serve cached records only until their TTL expires. - Cloudflare’s 1.1.1.1 implements RFC 8767, allowing it to serve expired records when authoritative resolution fails. - Cached `.de` records from before the incident continued resolving successfully after their TTLs expired. - This kept the overall `NOERROR` rate relatively stable, even though fresh lookups increasingly failed. - Without stale serving, successful responses would have declined steadily from the start of the outage. ## Temporarily Bypassing DNSSEC with an NTA - RFC 7646 defines Negative Trust Anchors (NTAs), which allow resolvers to treat a broken signed zone as temporarily unsigned. - NTAs are intended for situations such as a TLD operator publishing invalid signatures. - Bypassing validation can restore availability because the failure originates in the parent zone, not necessarily in individual domains. - The tradeoff is reduced security: while the exception is active, `.de` domains are exposed to DNS spoofing or other attacks that DNSSEC would normally prevent. ## Cloudflare’s Mitigation - Cloudflare’s Big Pineapple resolver did not yet have a native NTA implementation. - Instead, Cloudflare used an existing override mechanism to mark `.de` as an insecure zone. - This caused `.de` queries to be resolved without DNSSEC validation, functionally providing the same result as an NTA. - Combined with stale serving, this reduced the outage’s effect while DENIC worked to correct the zone. Cloudflare’s response illustrates a practical incident strategy: preserve cached answers where possible, then use a narrowly scoped DNSSEC exception when a parent zone is demonstrably broken. Such overrides should be temporary and carefully monitored because they restore availability at the cost of DNSSEC protection.

Read original(opens in new tab)
DnsDnssecDomain Name SystemCloudflare 1.1.1.1+4
aws3 min readCurated summary

The AWS MCP Server is now generally available | Amazon Web Services

The AWS MCP Server is now generally available as a managed way for AI agents to access AWS securely through IAM-authenticated tools. It combines live AWS documentation, access to more than 15,000 API operations, and sandboxed scripting so agents can produce more current, efficient, and production-ready results. The post concludes that this solves major limitations of model-only AWS assistance without granting agents unrestricted credentials. ## Why AI Agents Struggle with AWS - Models may lack knowledge of recently launched services such as Amazon S3 Vectors, Aurora DSQL, and Bedrock AgentCore. - Agents often default to the AWS CLI instead of AWS CDK or CloudFormation. - Generated IAM policies are frequently broader than necessary. - The resulting infrastructure may work in demos but fail production standards. ## Core AWS MCP Server Tools - `call_aws` can execute more than 15,000 AWS API operations using the user’s existing IAM credentials. - `search_documentation` and `read_documentation` retrieve current AWS documentation and best practices at query time. - The compact tool set reduces model context usage and is intended to support newly launched APIs within days. ## General Availability Improvements - IAM context keys allow fine-grained access control through standard IAM policies without requiring a separate server permission. - Documentation retrieval no longer requires authentication. - Reduced token consumption improves complex, multi-step workflows. - The `run_script` tool executes short Python scripts in a server-side sandbox. - The sandbox inherits IAM permissions. - It has no network access or access to the user’s local filesystem and shell. - It can combine multiple API calls, filter results, and calculate outputs in one round trip. ## Skills and AWS Best Practices - Skills replace Agent SOPs with curated guidance for common AWS tasks. - AWS service teams contribute and maintain the Skills. - They help agents avoid mistakes, use validated patterns, reduce hallucinations, and consume fewer tokens. - Keeping the tool list small makes agent behavior more predictable. ## Enterprise Security and Observability - IAM policies and Service Control Policies can separate human permissions from agent permissions. - For example, a user may perform write operations while the MCP server is restricted to read-only access. - CloudWatch metrics under the `AWS-MCP` namespace distinguish agent activity from direct human calls. - AWS CloudTrail records all API calls for auditing and compliance. ## Demonstration with Claude Code - Without the MCP Server, Claude Opus 4.6 suggested several valid ways to store embeddings on S3 but missed Amazon S3 Vectors because the service launched after its training cutoff. - With the MCP Server, Claude Code searched current AWS documentation and correctly identified S3 Vectors. - Claude Code can connect through the open-source `mcp-proxy-for-aws`, which bridges local IAM credentials and MCP’s OAuth 2.1 requirement. - The server works with Claude Code, Kiro, Cursor, Codex, and other MCP-compatible clients. ## Availability and Cost - The service is available in US East (N. Virginia) and Europe (Frankfurt). - It can make API calls across AWS Regions. - There is no additional charge for the MCP Server; users pay for AWS resources and applicable data transfer. The AWS MCP Server is a practical foundation for giving agents current AWS knowledge and controlled operational access. Teams should pair it with narrowly scoped IAM policies, read-only defaults where possible, and CloudWatch or CloudTrail monitoring.

Read original(opens in new tab)
Model Context ProtocolPythonAWSClaude Code+4
gitlab3 min readCurated summary

Claude Code and GitLab: Three workflows that ship

Claude Code accelerates coding, but writing code is only one part of shipping software. The post argues that GitLab complements Claude Code by handling CI/CD, security scanning, code review, approvals, and auditability. It presents three workflows: fixing bugs locally, enriching Claude with GitLab context through MCP, and using a Claude-powered external agent to resolve merge request feedback. ## The Gap Between Coding and Shipping - Agentic tools can quickly understand unfamiliar code, propose fixes, and scaffold features. - Faster code generation can leave teams with: - Larger bug backlogs - More pipeline failures - Accumulating security vulnerabilities - Bottlenecks in review and approval - GitLab manages the downstream software lifecycle after Claude Code produces a change. ## Prerequisites and Project Setup - Claude Code must be installed and configured. - A GitLab project containing bug reports and feature proposals is required, such as the Tanuki IoT Platform. - Optional workflows require: - The GitLab MCP Server - GitLab Duo Agent Platform with external agents - The example project uses CMake, Make, and GCC or Clang for C++ builds. - Developers clone the repository, launch `claude`, and can ask it to explain the project before making changes. ## Workflow One: Fix a C++ Bug and Ship It Through GitLab - The Arduino IoT Collector crashes when `/dev/ttyACM0` is unavailable. - The failure can be reproduced by building and running the application with CMake: - `cmake -S . -B build` - `cmake --build build` - `./build/arduino_iot_collector` - Claude Code examines `sensors/arduino-iot-collector/src/main.cpp` and identifies an uncaught `std::runtime_error`. - The recommended behavior is to log a clear configuration error and continue running instead of terminating. - After the fix, Claude Code can create a branch, commit the changes, and push them, or the developer can run the Git commands manually. - Opening a merge request triggers: - Build and test pipelines - Security scanning - GitLab Duo Code Review Flow - Checks against project style guides and custom review instructions ## Workflow Two: Add GitLab Context with MCP - Local repository files may not contain the full history behind a bug. - GitLab issues, debugging discussions, previous merge requests, and related fixes provide valuable software development lifecycle context. - The GitLab MCP Server connects Claude Code to this information. - It can be added over HTTP with a command such as: `claude mcp add --transport http GitLab https://gitlab.example.com/api/v4/mcp` - In a new Claude Code session, `/mcp` starts OAuth authentication through the browser. - Developers can verify the integration by asking Claude which GitLab MCP tools and server version are available. - MCP uses the developer’s existing GitLab identity: - It does not grant elevated permissions. - Claude can access only projects, issues, merge requests, and other data already visible to that user. ## Workflow Three: Resolve Review Feedback with an External Agent - The third workflow uses a Claude-powered external agent in GitLab Duo Agent Platform. - Rather than requiring a developer to manually interpret review comments, the agent can address code review feedback directly in the merge request. - This extends Claude Code’s implementation abilities into GitLab’s review and delivery workflow. ## Overall Recommendation Use Claude Code for investigation and implementation, then use GitLab to provide the context, automated validation, security checks, review, and approval process needed to ship those changes safely.

Read original(opens in new tab)
GitlabCode ReviewClaude CodeC Plus Plus+4
aws2 min readCurated summary

Modernize your workflows: Amazon WorkSpaces now gives AI agents their own desktop (preview) | Amazon Web Services

Amazon WorkSpaces now lets AI agents operate desktop and legacy applications directly, eliminating the need to build APIs or modernize existing software. Agents use managed virtual desktops with IAM authentication, security controls, and auditability through CloudTrail and CloudWatch. The feature is in public preview and supports agent frameworks through the Model Context Protocol (MCP). ## The Challenge of Legacy Applications - Many enterprises depend on applications without modern APIs: - 75% of organizations reportedly run legacy applications. - 71% of Fortune 500 companies rely on mainframe-based processes with limited programmatic access. - Organizations can either delay AI adoption or undertake costly, risky modernization projects. ## AI Agents in Secure WorkSpaces - AI agents operate desktop applications inside managed WorkSpaces environments. - Agents authenticate with AWS Identity and Access Management (IAM). - Existing security and compliance controls remain in place because agents do not run on local machines. - AWS CloudTrail and Amazon CloudWatch provide audit trails. - WorkSpaces supports MCP, making it compatible with frameworks such as LangChain, CrewAI, and Strands Agents. ## Configuring Agent Access - Administrators create a WorkSpaces Applications stack and enable the **Add AI Agents** option. - Agent capabilities can include: - **Computer input:** Clicking, typing, and scrolling. - **Computer vision:** Capturing screenshots so the agent can interpret the interface. - **Screenshot storage:** Saving session images for auditing and debugging. - Administrators define screen resolution and image format. The example uses 1280×720 resolution and PNG images. - Agents connect through a managed MCP endpoint using IAM credentials. ## Automating Unmodified Desktop Workflows - A Strands Agent SDK and Amazon Bedrock example completes a prescription refill by: - Looking up a patient record. - Searching for medication. - Placing the order. - Confirming the refill. - The pharmacy application requires no API, code changes, migration, or awareness that an agent is controlling it. ## Availability - The feature is in public preview at no additional cost. - It is available in selected AWS Regions across the United States, Canada, Europe, and Asia. - Developers can begin with AWS’s GitHub repository or the Amazon WorkSpaces product page. Organizations can use WorkSpaces as a governed execution environment for AI agents, allowing them to automate legacy desktop workflows while postponing or avoiding extensive application modernization.

Read original(opens in new tab)
AI AgentsModel Context ProtocolAmazon BedrockAmazon Cloudwatch+4
kakao3 min readCurated summary

Bringing a Voice AI Model to Production: The Journey of Optimizing Kanana-O Serving

Kanana-O is a multimodal model that understands text, images, and audio, then responds with text and speech. Deploying it for real-time voice conversations required solving problems that do not arise during model training, including low first-response latency, concurrent users, streaming across multiple models, and uneven GPU memory demands. Kakao built the specialized Kanana-Omni Server, achieving 1.6× the throughput of vLLM-Omni at 64 concurrent users. ## Kanana-O’s Three-Stage Architecture - **Thinker** processes multimodal inputs and generates text. - **Talker** converts Thinker’s text embeddings into sequential speech tokens. - **VoiceBox** combines speech tokens into audible audio waveforms. - In production, these components must operate concurrently rather than sequentially to deliver audio within hundreds of milliseconds. ## Why a Specialized Serving Server Was Needed - Thinker passes hidden-state embeddings directly to Talker rather than ordinary token IDs. - These high-dimensional tensors must be transferred continuously, making serialization or CPU copies too expensive. - Talker produces speech tokens step by step, while VoiceBox waits for enough tokens to form larger audio chunks. - Talker also combines speaker embeddings, Thinker outputs, and its own accumulated audio embeddings, creating an input structure unlike standard autoregressive decoding. - These constraints made a custom server more suitable than general-purpose frameworks. ## Zero-Copy Data Transfer - The server preallocates shared-memory blocks during startup. - Thinker writes tensors into an available block, while Talker receives only metadata such as the block identifier and byte size. - This avoids repeated allocation, copying, and serialization. - For GPU tensors on the same node, CUDA IPC transfers data directly between GPU processes, avoiding Device→Host→Device movement. ## Cascaded Streaming Pipeline - Thinker, Talker, and VoiceBox run as overlapping asynchronous stages. - Thinker can send its first output chunk while Talker processes earlier chunks and VoiceBox synthesizes audio from still earlier ones. - Talker buffers speech tokens until VoiceBox has enough data to create an audio chunk. - This pipelining significantly reduces the time before the user hears the first response. ## Process Isolation and Fault Containment - Thinker and Talker each run their own vLLM engine in separate processes. - This avoids conflicts between CUDA contexts, model memory, KV caches, and schedulers. - Processes are started with `spawn` rather than `fork`, preventing inherited CUDA state from causing corruption. - If one component fails, such as Thinker running out of memory, the other components and the API server can continue operating and be restarted independently. ## Continuous Batching with vLLM - Manually batching requests is difficult because multimodal inputs and accumulated Talker embeddings vary in size. - The server submits requests rapidly and delegates batch construction to vLLM’s continuous-batching scheduler. - Each request runs as an independent asynchronous generation task. - vLLM combines requests internally during forward passes, while request IDs ensure each task receives only its own streamed output. - This improves GPU utilization without requiring custom synchronization and padding logic. ## Single FastAPI Worker and Asynchronous Execution - Multiple Uvicorn workers would load separate copies of the vLLM engines, multiplying GPU memory usage and model-loading costs. - Therefore, the server uses `workers=1`. - Since a blocking operation would otherwise stall every connected user, the entire request path—from the API endpoint through final audio generation—is designed around `async`/`await`. - Keeping the pipeline non-blocking allows one worker to accept and progress many concurrent requests. Kakao’s main recommendation is to design serving infrastructure around the model’s actual dataflow rather than forcing it into a generic framework. For complex multimodal pipelines, zero-copy transfers, asynchronous cascaded streaming, process isolation, and engine-level continuous batching can be more important than simply scaling API workers.

Read original(opens in new tab)
Multimodal AIVllmContinuous BatchingFastapi+3
slack3 min readCurated summary

From SSH to REST: A Security-Driven Modernization of Slack’s EMR Data Pipelines

Slack had more than 700 SSH-based operators running critical EMR workloads, creating security risks, operational failures, and barriers to infrastructure modernization. The company replaced these connections with REST-based job submission across eight data regions without downtime. YARN Distributed Shell was the key enabler for migrating arbitrary command-line jobs that lacked dedicated REST APIs. ## How Slack’s SSH Architecture Developed - Airflow originally connected directly to EMR master nodes using `SSHOperator`. - Over time, teams created more than 700 SSH-based jobs for: - Spark and MapReduce workloads - AWS CLI commands - Custom Python scripts - Data-transfer operations such as `hadoop distcp` - The approach was simple but tightly coupled orchestration workers to production clusters. ## Security and Operational Costs of SSH - Direct SSH access expanded the attack surface. - SSH keys had to be distributed and rotated across orchestration workers. - Auditing required correlating activity across multiple systems. - Permissions became complicated, often involving custom security groups and configurations. - Jobs ran on EMR master nodes, causing resource contention. - Restarted Kubernetes pods could break SSH connections. - Long-running processes could become orphaned “zombie” jobs. - Connection failures made job success or failure difficult to determine. - SSH dependencies blocked Spark-on-Kubernetes, EMR on EKS, AWS child-account migration, and better observability. - Slack’s search-indexing pipeline was especially sensitive because it processed terabytes of data daily and supported search for millions of users. ## REST-Based Job Submission - SSH creates a stateful connection whose failure can leave job status ambiguous. - REST APIs provide a durable, server-managed lifecycle: - `POST` submits a job and returns an ID. - `GET` retrieves its status. - `DELETE` cancels it cleanly. - Clients can crash or restart without terminating the underlying job. - Existing systems such as YARN, Trino, and Snowflake use this model. - YARN provides REST submission for Hadoop, Spark, Hive, and MapReduce workloads, but not arbitrary shell commands. ## YARN Distributed Shell - Spark and Hive already had REST-compatible options through Livy and HiveServer2. - The difficult cases were MapReduce and more than 300 CLI-based jobs. - Slack considered custom wrapper services, Ansible or Salt, and creating a new YARN job type. - These alternatives added complexity, security work, or long-term maintenance. - YARN Distributed Shell—implemented through `ApplicationMaster`—could execute arbitrary scripts inside YARN containers. - It used existing YARN APIs and authentication mechanisms, avoiding a custom security layer. ## The Distributed Shell Workflow - Upload a command script to S3, such as an `aws s3 sync` operation. - Submit a YARN application specifying: - The Distributed Shell application master - The S3 script location - Script metadata such as length and timestamp - YARN then: - Allocates a resource-managed container - Downloads and executes the script - Enforces memory and vCore limits - Provides isolation, retries, cancellation, and centralized logging By using REST submission and YARN Distributed Shell, Slack could remove SSH from its EMR data pipelines while preserving support for both standard data-processing jobs and arbitrary command-line workloads.

Read original(opens in new tab)
KubernetesRest ApiApache SparkApache Airflow+3
gitlab3 min readCurated summary

8 Agentic AI patterns reshaping team collaboration

A synthesis of 17 agentic AI platforms identifies eight patterns that help teams work faster, work smarter, and maintain control. The strongest platforms do more than provide capable individual agents: they reduce coordination overhead, embed agents in existing workflows, and provide governance across the software lifecycle. The post argues that integrated team collaboration and managed deployment will distinguish leading AI platforms. ## Eight Collaboration Patterns ### Proactive Status Updates - Agents generate progress summaries from live task data. - They identify blockers, risks, and slipping deadlines before escalation. - Automated updates reduce status meetings and manual check-ins. ### Intelligent Work Routing - Agents assign work based on skills, capacity, and project context. - Continuous workload balancing replaces periodic planning adjustments. - Transparent routing logic lets humans review and correct assignments. ### Team Communication Support - Agents summarize chats, threads, and meetings. - Decisions and conversation history remain available to new participants. - Async summaries reduce repeated explanations and unnecessary meetings. ### Role-Specific Agents in Chat - Specialist agents operate inside tools such as Slack. - They can handle onboarding, IT, sales, and other role-based tasks. - Simple interactions, such as an emoji reaction, can create tracked work. ### Shared Conversational Context - Agents retain awareness of participants, threads, and files. - Teams benefit from knowledge gathered through another member’s prompt. - Shared context prevents duplicated prompting and helps new members continue work immediately. ### Role-Based Access Control - Agents inherit permissions from their assigned identities and roles. - Access controls can apply at the field level, preventing unauthorized reading or actions. - Detailed action logs provide an auditable record for compliance. ### Governed Environments - Agents move through development, testing, and production using managed pipelines. - Sandboxes isolate early development and prevent conflicts. - Controlled promotion prevents untested agents or disruptive updates from reaching production. ### Collaborative Agent Development - Multiple team members can co-own, edit, debug, and maintain agents. - Tiered permissions support shared ownership without removing accountability. - Standardized protocols help agents created by different contributors work together. ## Lessons from the Competitive Landscape - Agents are increasingly embedded in existing communication and work tools rather than isolated portals. - Governance becomes essential as organizations scale agent usage. - Agent development is evolving into a collaborative discipline requiring shared ownership, versioning, and auditing. - The biggest opportunity is reducing the “coordination tax” of meetings, check-ins, and repeated explanations. - Few platforms provide an end-to-end governance experience combining environment grouping, shared catalogs, and managed promotion pipelines. ## Implications for GitLab GitLab’s integrated DevSecOps lifecycle gives it a structural advantage because software delivery workflows, context, and controls already exist in one platform. GitLab Duo Agent Platform is positioned to embed agents directly into those workflows, allowing teams to orchestrate work while agents execute across the software development lifecycle. Teams evaluating agentic AI should prioritize not only agent capability, but also shared context, transparent automation, permissions, deployment governance, and collaborative maintenance.

Read original(opens in new tab)
AI AgentsGitlabDevSecOpsGovernance+3
aws4 min readCurated summary

AWS Weekly Roundup: What’s Next with AWS 2026, Amazon Quick, OpenAI partnership, and more (May 4, 2026) | Amazon Web Services

The AWS weekly roundup highlights a major shift toward agentic AI across Amazon’s products and its partnership with OpenAI. The biggest announcements include expanded Amazon Quick capabilities, four specialized Amazon Connect solutions, and OpenAI models and Codex becoming available through Amazon Bedrock. AWS also introduced new EC2 instances, agent optimization tools, Ruby 4.0 support for Lambda, and a transition plan from Amazon Q Developer to Kiro. ## What’s Next with AWS 2026 - AWS and OpenAI executives presented new ways businesses are using AI agents to automate operations. - The announcements centered on Amazon Quick, Amazon Connect, and deeper integration with OpenAI through Amazon Bedrock. ## Amazon Quick Expands Beyond Chat - A new desktop app, currently in preview, connects Quick to local files, calendars, and communications without requiring a browser. - Users can sign up with a personal email or Google, Apple, GitHub, or Amazon credentials; an AWS account is not required. - Quick can generate: - Documents - Presentations - Infographics - Images - New integrations include Google Workspace, Zoom, Airtable, Dropbox, and Microsoft Teams. - The preview “Build custom apps with Quick” feature lets users create intelligent applications, dashboards, and web pages using natural-language instructions. ## Amazon Connect Becomes Four Agentic AI Products - **Amazon Connect Decisions** applies Amazon’s operational expertise and supply-chain tools to help organizations move from reactive crisis management to proactive planning. - **Amazon Connect Talent** provides AI-led interviews, science-backed assessments, and consistent candidate evaluations for large-scale hiring. - **Amazon Connect Customer**, the renamed customer-service product, supports personalized voice, chat, and digital experiences, with conversational AI that can be configured in weeks. - **Amazon Connect Health** supports patient verification, appointment management, patient insights, ambient documentation, and medical coding. ## OpenAI Partnership Expands Through Amazon Bedrock - OpenAI models, including GPT-5.5 and GPT-5.4, are coming to Bedrock in limited preview. - Customers can use existing Bedrock APIs with AWS security, governance, and cost controls, without managing new infrastructure. - **Codex on Amazon Bedrock** brings OpenAI’s coding agent into AWS environments: - Authentication uses AWS credentials. - Inference runs through Bedrock. - Usage can count toward AWS cloud commitments. - Initial access includes the Codex CLI, desktop app, and Visual Studio Code extension. - **Bedrock Managed Agents powered by OpenAI** combines OpenAI models with AWS infrastructure and the OpenAI harness for long-running, production-oriented agent workflows. ## New EC2 Instance Families - **M8in and M8ib** instances are generally available, offering up to 43% higher performance than M6in and M6ib. - M8in provides up to 600 Gbps of network bandwidth. - M8ib provides up to 300 Gbps of EBS bandwidth. - **R8in and R8ib** target memory-intensive workloads such as commercial databases, data lakes, and SAP HANA. - **C8ine and M8ine** provide up to 2.5 times higher packet performance per vCPU and up to twice the internet-gateway throughput of their predecessors. - These network-optimized instances are designed for virtual firewalls, load balancers, security appliances, and 5G user-plane workloads. ## AgentCore and Lambda Updates - Bedrock AgentCore Optimization, in preview, adds: - Production-trace analysis - Recommendations for system prompts and tool descriptions - Batch evaluations - A/B testing against live traffic - Recommendations require human approval before deployment. - AWS Lambda now supports Ruby 4.0 as a managed runtime and container base image. - Ruby 4.0 support includes advanced logging features such as structured JSON logs, configurable log levels, and custom CloudWatch log groups. ## Amazon Q Developer Moves Toward Kiro - Amazon Q Developer IDE plugins and paid subscriptions will reach end of support on April 30, 2027. - New signups will be blocked beginning May 15, 2026. - Existing subscriptions can continue adding users until then. - Opus 4.6 will leave Q Developer Pro on May 29, 2026, while newer coding models such as Opus 4.7 will be exclusive to Kiro. - Q Developer experiences in the AWS Console, documentation, mobile app, Slack, and Microsoft Teams are unaffected. AWS’s direction is increasingly centered on managed AI agents integrated into everyday business workflows. Organizations adopting these services should evaluate the new Bedrock, Quick, and Connect capabilities while also planning migration from Q Developer to Kiro before the announced support deadlines.

Read original(opens in new tab)
AI AgentsAWSAmazon BedrockAmazon Ec2+3
netflix3 min readCurated summary

Democratizing Machine Learning at Netflix: Building the Model Lifecycle Graph

Netflix’s growing use of machine learning across personalization, Studio, payments, advertising, and other domains has created a fragmented ecosystem of tools and metadata. The Metadata Service (MDS) addresses this problem by building a Model Lifecycle Graph that connects models, features, pipelines, experiments, datasets, and ownership information. Its goal is to make ML assets discoverable, understandable, and reusable across organizational boundaries. ## A Fragmented Machine Learning Landscape - Netflix ML has expanded from personalization into areas such as: - Studio production and post-production - Fraud detection and payment optimization - Advertising and real-time targeting - Each domain uses different technologies, metrics, and organizational structures. - Valuable assets often remain isolated in specialized systems. - For example, Studio-generated content embeddings could support: - Contextual ad matching - Episodic merchandising - Recommendations based on tone, topic, or mood - Practitioners struggle to answer basic questions because relevant information is split across: - Model registries - Pipeline orchestrators - Experimentation platforms - Feature stores - Dataset systems - This fragmentation makes discovery, lineage tracking, impact analysis, and ownership difficult. ## The Challenge of Connecting ML Infrastructure - MDS must unify metadata from many independent systems, including: - Pipeline execution and transformation data - Model versions, artifacts, deployments, and staleness - A/B test configurations - Feature definitions and usage - Dataset creation and discovery - User, team, and organization information - These systems use different identifiers, formats, and conceptual models. - The core challenge is transforming heterogeneous metadata into a common entity model and connected graph—not merely creating a consolidated user interface. ## The Model Lifecycle Graph - Netflix’s Metadata Service indexes ML-related assets and materializes relationships between them. - It supports real-time metadata ingestion and cross-domain questions such as: - Which experiments use a particular model? - Which models depend on a feature? - What data sources feed a model? - Who owns each part of the workflow? - The graph is intended to make every ML asset discoverable and reusable regardless of its originating team or business domain. ## Core Concepts and Vocabulary - **Component:** Any uniquely addressable object identified by an AIP URI, such as: - `aip://model/registry/ranking-v5` - `aip://user/identity/alice` - `aip://pipeline/orchestrator/weekly-training` - **Entity:** A component enriched with properties such as name, description, creation date, and ownership. - **Entity type:** A group of entities sharing the same data shape and required properties. - **Domain:** An abstract interface for a category of ML assets, such as Models or Pipelines. - **Provider:** A concrete backend implementation of a domain, such as Netflix’s internal model registry. - Separating domains from providers allows multiple systems to implement the same interface without changing how consumers interact with MDS. - URI-based addressing gives services a consistent way to reference assets and resolve them to connected metadata. ## From Events to a Queryable Graph - MDS receives metadata events through Kafka and AWS SNS/SQS. - Source systems emit lightweight events containing an event type and resource identifier. - For example, a model registry might emit a `model_instance_created` event with the new instance’s ID. - This keeps event producers simple while allowing MDS to enrich events, construct entities, and infer relationships such as connections between models and A/B tests. The Model Lifecycle Graph provides Netflix with a common layer for connecting previously isolated ML systems. By standardizing identifiers, entities, domains, and providers, MDS can support cross-domain discovery, lineage, impact analysis, and collaboration at scale.

Read original(opens in new tab)
Machine LearningApache KafkaFeature StoreGraph Database+3
gitlab3 min readCurated summary

Atlassian will train on your data: Opt out with GitLab

Atlassian plans to use customer metadata and in-app content from Jira, Confluence, and other cloud products to train AI services beginning August 17, 2026. Collection will be enabled by default, with mandatory metadata collection for Free, Standard, and Premium customers; only Enterprise customers can opt out. The post argues this weakens data governance, particularly for regulated organizations, while presenting GitLab’s no-collection, no-training approach as a stronger privacy model. ## What Atlassian’s Policy Change Covers - Atlassian will collect: - Metadata such as story points, sprint dates, SLA values, and signals from Teamwork Graph and connected apps. - In-app content including Confluence pages, Jira issue titles, descriptions, and comments. - Atlassian says data will be de-identified and aggregated before training. - Data may be retained for up to seven years. - After opting out, in-app data is reportedly removed within 30 days and models retrained within 90 days. - Customers using customer-managed encryption keys, Government Cloud, Isolated Cloud, or HIPAA-related configurations are excluded. - The change reverses Atlassian’s previous position that customer data would not be used to train or improve AI services. ## Problems with Opt-Out-by-Default Governance - Customers must notice the policy change, assess its legal and security impact, and act within the available timeframe. - Free, Standard, and Premium customers cannot disable metadata collection. - Enterprise is the only opt-out route, requiring at least 801 users and custom pricing. - “De-identified” metadata can still reveal team performance, project structure, delivery cadence, and competitive operational intelligence. - The policy turns data protection into a purchasing decision rather than a default customer right. ## Why Atlassian Customers Face Greater Exposure - Jira and Confluence often contain: - Project plans and sprint data - Security tickets and incident postmortems - Internal documentation - Bug, release, and portfolio management information - Organizations using Bitbucket and Bamboo may also expose source-code metadata and CI/CD configuration signals. - Teamwork Graph connectors can extend the data scope to tools such as Slack, Figma, Google Drive, Salesforce, and ServiceNow. - Customers migrating from Data Center or Server editions to Atlassian Cloud must now evaluate not only cloud migration, but also the possibility of default AI training. ## Compliance and Regulatory Implications - Financial institutions may need to reassess vendor controls under frameworks such as SR 11-7 and DORA. - Public-sector organizations must consider NIST 800-53 and FISMA requirements around sensitive-data flows. - Healthcare organizations need to evaluate potential HIPAA implications. - EU AI Act obligations may create additional concerns because European expectations often favor opt-in consent. - Existing vendor-risk, model-risk, and data-processing assessments should be updated before August 17, 2026. ## GitLab’s Contrasting Approach - GitLab is presented as opposing opt-out-by-default collection. - Its stated principles are: - No collection of customer data - No AI training on customer data - The same privacy commitment regardless of subscription tier - This approach avoids making stronger data protection dependent on Enterprise pricing and simplifies compliance reviews. Organizations should inventory the data and integrations connected to Atlassian, review contractual and regulatory obligations, and determine whether they can opt out or need to reconsider their platform strategy.

Read original(opens in new tab)
GitlabCloud ComputingJiraData Privacy+3
figma3 min readCurated summary

PGKeeper: Building the Bouncer We Needed for Postgres | Figma Blog

Figma built PGKeeper to replace PgBouncer as its PostgreSQL connection and load-management layer. Growing traffic, sharding, and stricter reliability requirements exposed PgBouncer’s limits in scalability, prioritization, backpressure, connection protection, and extensibility. PGKeeper is a custom Go service positioned between Figma’s DBProxy routing layer and PostgreSQL, designed to protect databases from overload and connection churn. ## Figma’s Database Architecture - PostgreSQL powers Figma’s OLTP workloads. - Figma scales through horizontal and vertical sharding across multiple database instances. - DBProxy hides sharding complexity from application code by: - Parsing and analyzing queries. - Selecting the appropriate PostgreSQL instances. - Rewriting requests into queries for the selected targets. - A dedicated set of connection-pooler replicas serves each PostgreSQL machine, creating an n-to-one relationship between poolers and databases. ## Why PgBouncer Was No Longer Enough - **Limited scalability** - PgBouncer’s single-threaded architecture created a vertical scaling ceiling. - Adding replicas helped, but uneven load distribution caused performance degradation. - **Insufficient load management** - PgBouncer could not prioritize critical traffic over lower-priority or misbehaving requests. - It lacked effective backpressure and advanced load-shedding algorithms such as Controlled Delay (CoDel). - CoDel sheds work based on how long requests wait, rather than simply counting queued requests. - **Unsafe connection behavior** - PostgreSQL connections are expensive resources. - Rapid connection creation and churn could destabilize database nodes. - Recovery after overload could trigger another surge of connections, creating cascading failures and prolonged overload. - **Limited extensibility and control** - Figma needed deep observability, feature-flagged rollouts, admission control, and fair resource sharing. - Even maintaining small PgBouncer patches proved costly. - Extending PgBouncer substantially would create an ongoing maintenance burden. ## Why Connection Pooling Could Not Live in DBProxy - Figma generally limits each PostgreSQL instance to roughly 100 pooled connections. - Hundreds of stateless DBProxy replicas sit in front of those databases. - Giving every DBProxy replica its own pool would either exceed database connection limits or require complex coordination. - Centralizing pooling in a separate service provided a better fit for the mismatch between many routers and a small fixed connection budget. ## Why Figma Built PGKeeper - PGCat addressed PgBouncer’s single-threaded scalability problem, but customizing it would require deep changes to its core execution paths. - Those changes would likely require Figma to maintain a long-term fork. - Figma therefore created PGKeeper as a Go-based service tailored to its infrastructure and operational requirements. - Its role is to act like a goalkeeper: protecting PostgreSQL from harmful traffic and protecting connections from uncontrolled churn. PGKeeper was chosen because Figma needed more than a basic connection pooler: it needed a scalable, observable, controllable layer capable of prioritizing traffic and preventing database overload.

Read original(opens in new tab)
PostgreSQLGoDatabase ShardingLoad Balancing+3
discord3 min readCurated summary

Discord Patch Notes: May 4, 2026

Discord’s May 4, 2026 patch focuses on reliability, performance, usability, and server administration. Major improvements include nearly 5% faster Android video startup, reorganized desktop settings, faster Soundboard access, and self-updating Linux support. The release also fixes numerous bugs across account profiles, search, payments, moderation, notifications, and mobile platforms. ## Performance and Platform Improvements - Android video startup improved by 4.89%, bringing average feed startup time below 600 milliseconds. - Soundboard data now loads when users join a voice channel rather than when they first open the Soundboard, reducing initial access time. - Linux now uses Discord’s Rust-based automatic updater, eliminating the need for manual update installation. - Linux installation now supports `.rpm` and `.pkg.tar.zst` packages. ## Server Administration and Moderation - Discord fixed several issues affecting server administrators and moderators. - Fixes covered permissions, user states, and interactions with account-safety systems. - Discord encouraged administrators to report remaining problems through its community bug megathread. ## Desktop Settings and Organization - Desktop settings were consolidated into three pages: - Appearance - Accessibility - Developer - The former Appearance, Accessibility, Chat, Streamer Mode, and Advanced sections were reorganized. - Several settings received clearer wording and improved layouts. - Fixed problems involving theme synchronization, profile editing, unsaved-change warnings, currency labels, and server administration prompts. ## General User Interface Fixes - “GODLIKE!!” and “BEYOND GODLIKE!!” copy-username messages now have opaque backgrounds. - The Quick Switcher can now accept invite links, join the associated server, and navigate to it. - Fixed display problems involving long nicknames, search filters, avatar controls, profile links, status indicators, and Nitro badges. - French users can now search for “sondage” without the search term being incorrectly split. - Android search filters now correctly display their active blue state. - Fixed several modal, button-spacing, and navigation issues across desktop and mobile. ## Profiles, Avatars, and Customization - Per-server avatar links now copy correctly instead of copying the main profile avatar link. - Clearing per-server pronouns no longer repopulates them from the main profile; the main pronouns appear only as a placeholder. - iOS no longer switches unexpectedly to a per-server profile after copying a username. - Recent-avatar delete controls now appear correctly. - Custom Status editing opens above the full profile instead of replacing it. - Fixed an issue where clicking outside profile editing discarded changes without warning. ## Notifications, Search, and Media - Desktop Inbox no longer crashes when many notifications are cleared rapidly. - Mobile search tabs for Media, Pins, Files, and Links no longer spam errors or retry repeatedly after a connection loss. - Expired public image links in the Inbox preview now behave more reliably. - Fixed an issue where long mobile search-result nicknames obscured timestamps. ## Payments and Server Boosting - Server Boost marketing audio now stops when users enter the purchase flow. - Pressing Escape during payment no longer closes the underlying Server Boost page instead of the payment window. - Nitro trial recipient checkboxes now select friends correctly. - Fixed overlapping controls in Profile Settings and spacing issues on the domain-connection page. - GBP subscription settings now show the currency’s full name. Discord’s changes combine small interface corrections with measurable performance and platform improvements. Users should receive the fixes progressively, since deployment may still be rolling out across platforms.

Read original(opens in new tab)
DiscordAndroidiOSRust+3