Aws Ec2 | Techlist.io

awsAug 6, 20263 min readCurated summary

Runtime instances: persistent compute for production AI agents on Amazon Bedrock AgentCore | Amazon Web Services

Amazon Bedrock AgentCore Runtime Instances provides persistent, managed compute for production AI agents that need more than short-lived invocations. It supports multi-day workflows, shared state, GPU acceleration, multi-agent collaboration, and direct OS access while AWS manages the underlying EC2 infrastructure. Runtime Instances complements AgentCore’s lightweight microVMs, enabling teams to combine fast-scaling orchestration with persistent worker environments. ## Why Persistent Compute Matters - Production agents often run for hours or days and must preserve state across workflow steps. - Complex systems may require: - Collaboration between multiple agents - Shared files and context - GPU acceleration - Direct operating-system access - Continuous execution across multiple days - Previously, teams had to provision EC2 instances, configure networking, manage sessions and scaling, and build monitoring themselves. ## What Runtime Instances Provides - AWS-managed EC2 infrastructure for hosting multiple agents in one runtime. - Shared sessions that persist for up to 14 days. - Separate dependencies and artifacts for each deployed agent. - GPU-capable infrastructure for compute-intensive workloads. - Session stop and restart capabilities to reduce idle costs. - Support for zip packages and container images. - Compatibility with frameworks such as CrewAI, LangGraph, LlamaIndex, and Strands. - Integration with existing AgentCore APIs, identity controls, and observability. - Persistent knowledge storage through Amazon EBS and AgentCore Memory. ## Combining MicroVMs and Runtime Instances - Runtime microVMs remain useful for lightweight orchestrator agents that need rapid scaling. - Runtime Instances are better suited to persistent, resource-intensive workers. - An orchestrator can: - Route tasks to specialized agents - Make API calls - Aggregate results - Instance-based workers can handle tasks such as code compilation, security scanning, or GUI automation while retaining local state. ## Shared-Filesystem Agent Example The demonstration uses two Strands Agents applications: - A code writer: - Generates Python code from a natural-language task. - Saves the result as `code.py` in a session-specific shared directory. - A code reviewer: - Reads the writer’s file from the same filesystem. - Reviews it for bugs, style issues, and suggestions. - Both applications use: - An `@app.entrypoint` decorator - A selected Bedrock model - The session ID to identify shared storage - Because both agents share the host filesystem, they exchange artifacts without API calls or explicit data transfer. ## Deployment Workflow ### Create a Capacity Provider - Select the operating system, allowed EC2 instance types, VPC, subnets, and security groups. - The example uses: - Linux 64-bit ARM - `c7g.2xlarge` - 8 vCPUs and 16 GiB of memory - A default `gp3` volume - AgentCore creates or assigns the required infrastructure role and instance profile. - Once active, most capacity provider settings cannot be changed, so configuration should be verified beforehand. ### Create a Runtime and Deploy an Agent - Create a runtime using the **Instances** compute type. - Associate it with the capacity provider. - Upload the agent package to Amazon S3. - Specify the language runtime, such as Python 3.13, and the entry-point file containing `@app.entrypoint`. - Deployment can be performed through the AWS Management Console, AgentCore CLI, AWS CLI, or infrastructure as code. Runtime Instances are a strong fit for agents with long-running, stateful, collaborative, or compute-heavy workloads. Use them alongside microVMs when a system needs both elastic orchestration and persistent worker infrastructure, while relying on EBS or AgentCore Memory for state that must outlive individual sessions.

Read original

Aws Ec2 Python Multi Agent Systems Amazon Bedrock Agentcore+2

awsApr 7, 20263 min readCurated summary

Launching S3 Files, making S3 buckets accessible as file systems | Amazon Web Services

Amazon S3 Files makes general-purpose S3 buckets accessible through a native NFS-based file system. It combines S3’s durability, cost, and broad service integration with interactive file operations, shared access, and low-latency performance. The post concludes that this reduces the need to choose between object storage and traditional file systems for many AWS workloads. ## Bridging Object Storage and File Systems - S3 Files presents S3 objects as files and directories. - Applications can use standard NFS v4.1+ operations, including creating, reading, updating, and deleting files. - Changes made through the file system are synchronized back to S3 as new objects or object versions. - Changes made directly in S3 generally appear in the file system within seconds, though synchronization can sometimes take longer. - Multiple compute resources can mount the same file system and share data without duplicating it. ## Performance and Data Access - S3 Files uses Amazon EFS underneath and provides approximately 1 ms latency for active data. - Frequently accessed metadata and file contents are placed on high-performance storage. - Large sequential reads can be served directly from S3 to maximize throughput. - Byte-range reads transfer only the requested portion of a file, reducing data movement and cost. - Intelligent prefetching anticipates access patterns. - Administrators can choose whether to cache complete files or metadata only. - NFS close-to-open consistency supports concurrent, interactive workloads such as ML pipelines and collaborative AI agents. ## Supported AWS Compute Services S3 Files can expose buckets to: - Amazon EC2 instances - Amazon ECS and EKS containers - AWS Fargate workloads - AWS Lambda functions This allows production applications, machine-learning systems, and agentic AI tools to access shared S3 data using ordinary file-system interfaces. ## Creating and Mounting an S3 File System The demonstration uses an EC2 instance and a general-purpose S3 bucket: - Create an S3 file system from the S3 console, AWS CLI, or infrastructure-as-code tools. - Configure or discover a mount target inside the relevant VPC. - Mount the file system on EC2 with commands such as: ```bash sudo mkdir /home/ec2-user/s3files sudo mount -t s3files fs-...:/ /home/ec2-user/s3files ``` - Files created in the mounted directory become visible in the S3 bucket after synchronization. - Standard commands such as `ls`, `echo`, and AWS CLI operations can verify that file contents are consistent between the mount and S3. ## Security, Permissions, and Monitoring - IAM identity and resource policies control access at both the file-system and object levels. - Data is encrypted in transit with TLS 1.3. - Data at rest uses SSE-S3 or customer-managed AWS KMS keys. - POSIX permissions rely on user IDs and group IDs stored as object metadata. - CloudWatch provides performance and update metrics. - CloudTrail records management events. - EC2 instances should use the latest `amazon-efs-utils` package, which is included in AWS-provided AMIs. S3 Files is best suited to workloads requiring shared, interactive file access while retaining data in S3. Teams should still evaluate access patterns and latency requirements, but the service offers a practical way to use familiar file operations without giving up S3’s centralized, durable storage model.

Read original

Aws Ec2 Amazon S3 Aws Cli Aws Eks+4

slackOct 23, 20253 min readCurated summary

Advancing Our Chef Infrastructure: Safety Without Disruption

Slack chose to improve its existing Chef and EC2 infrastructure rather than migrate to Policyfiles, avoiding disruptive cookbook and role changes. The central strategy is to divide production into six Availability Zone–based Chef environments, limiting deployment blast radius while preserving existing workflows. A canary environment and staggered release train provide earlier detection of configuration problems and safer fleet-wide rollouts. ## Why Slack Avoided Policyfiles - Policyfiles could have improved long-term safety by replacing roles and environments. - Migrating dozens of teams and their cookbooks would have required substantial effort. - Slack concluded that the short-term disruption and migration risk outweighed the benefits. - Instead, the team extended its existing EC2 framework without requiring cookbook or role changes. ## Splitting Production Chef Environments - Previously, all production nodes used one shared Chef environment. - Cron-triggered Chef runs were staggered across Availability Zones to prevent simultaneous fleet-wide changes. - This reduced the impact of bad changes on existing nodes, but newly provisioned instances immediately consumed the latest version from the shared environment. - During large scale-out events, a broken configuration could therefore spread rapidly to many new nodes. - Slack split production into six environments: `prod-1` through `prod-6`. - Service teams still launch instances as `prod`; internally, nodes are assigned to a numbered environment based on their Availability Zone. - Updates to one environment now affect only the nodes mapped to that environment. ## Extending Poptart Bootstrap - Slack’s base AMIs include `Poptart Bootstrap`, which runs through `cloud-init` during instance startup. - It creates the node’s Chef object, configures DNS, and posts success or failure notifications to Slack. - Slack extended it to inspect the node’s AZ ID and select the appropriate numbered production environment. - This automatically distributes new nodes across isolated Chef environments without requiring service teams to change their provisioning process. ## Canary Deployments and the Release Train - Cookbook changes are promoted: - To sandbox at the top of the hour - To development environments through a Kubernetes cron job - To production beginning at 30 minutes past the hour - `prod-1` acts as the canary production environment. - It receives the latest changes hourly when new cookbook artifacts exist. - This tests changes in real production conditions soon after they are created. - `prod-2` through `prod-6` follow a release train. - A version advances gradually through the production environments. - The next rollout begins only after the previous version has reached `prod-6`. - This sequencing limits the number of affected nodes and makes regressions easier to identify. ## Why `prod-1` Updates Frequently - If the canary waited until a version had passed through every production environment, it would test artifacts containing larger batches of accumulated changes. - Updating `prod-1` frequently keeps the feedback loop close to the originating change. - The remaining production environments provide progressively broader validation after the canary stage. - For example, a new artifact can move from sandbox and dev to `prod-1`, then advance through `prod-2` to `prod-6` while newer artifacts continue entering the canary path. Slack’s approach preserves its existing Chef ecosystem while adding isolation, automated environment assignment, and staged promotion. The result is a safer deployment process that reduces blast radius and catches production issues earlier without forcing widespread application changes.

Read original

Aws Ec2 Kubernetes Chef Canary Deployment+3

datadogMay 26, 20233 min readCurated summary

Not just another network latency issue: How we unraveled a series of hidden bottlenecks

Repeated high-startup-latency pages in Datadog’s usage estimation service were caused by several independent bottlenecks rather than application changes. The investigation eventually identified four issues: CPU-throttled Envoy sidecars, a Linux kernel bug affecting ENA transmit queues, insufficient EC2 network bandwidth, and requests routed to terminating cache pods. Fixing each layer progressively reduced remote-cache p99 latency from roughly one second to its normal level of about 100 ms. ## Service Architecture and the Original Symptoms - The service consists of router, counter, and aggregator applications. - At startup, `counter` loads data from a remote cache into a local cache. - While the local cache is populating, request processing is slower and backlog grows. - Normal p99 remote-cache latency was approximately 100 ms, but it exceeded one second during deployments. - Scaling the remote cache did not help, indicating that the cache itself was not underprovisioned. ## CPU-Throttled Envoy Sidecars - Requests to the remote cache passed through an Envoy sidecar that batched queries into packets. - When `counter` restarted, Envoy reached its two-core CPU limit and was throttled. - Delayed request and response processing caused retries, TCP retransmits, and increased remote-cache latency. - Increasing Envoy’s CPU allocation eliminated the issue in staging and reduced production latency, but did not fully resolve rollout spikes. ## Linux Kernel and ENA Transmit-Queue Bug - Investigation of system and network metrics revealed a Linux kernel bug affecting AWS Elastic Network Adapter traffic. - The kernel mapped all outbound traffic to the first transmit queue instead of distributing it across eight queues. - This limited throughput and caused retransmits during high-traffic periods such as deployments. - A hotfix distributed traffic across all eight queues. - The change removed non-rollout latency spikes but rollout latency still fluctuated between 200 and 600 ms. ## EC2 Network Bandwidth Limits - ENA metrics showed that instances exceeded AWS inbound and outbound bandwidth allowances. - AWS dropped packets at the hypervisor when those limits were exceeded, causing retransmissions and slower cache requests. - Migrating to network-optimized EC2 instance types with higher bandwidth allowances largely restored p99 latency to around 100 ms. - Occasional one-second spikes continued despite the improvement. ## Requests Sent to Terminating Cache Pods - Remaining spikes correlated with remote-cache pods that were shutting down. - Clients continued sending requests to terminating pods, leading to one-second timeouts and retries. - The cache’s graceful-shutdown behavior did not adequately wait for Envoy clients’ in-flight requests. - The team added a `preStop` hook that sets an `XXX_MAINTENANCE_MODE` key to notify clients before termination and began coordinating shutdown with outstanding requests. The incident demonstrates the importance of tracing latency across the entire request path, from application startup through proxies, kernel networking, hardware interfaces, cloud bandwidth limits, and pod lifecycle behavior. Layered system metrics and component-level investigation were necessary to eliminate alert fatigue and restore reliable deployment behavior.

Read original

Aws Ec2 Database Design Linux Kernel Envoy+4