techlist.io
KR EN

vector-db

6 posts

aws

Amazon OpenSearch Service improves vector database performance and cost with GPU acceleration and auto-optimization | AWS News Blog (opens in new tab)

Amazon OpenSearch Service has introduced serverless GPU acceleration and auto-optimization features designed to enhance the performance and cost-efficiency of large-scale vector databases. These updates allow users to build vector indexes up to ten times faster at a quarter of the traditional indexing cost, enabling the creation of billion-scale databases in under an hour. By automating complex tuning processes, OpenSearch Service simplifies the deployment of generative AI and high-speed search applications. ### GPU Acceleration for Rapid Indexing The new serverless GPU acceleration streamlines the creation of vector data structures by offloading intensive workloads to specialized hardware. * **Performance Gains:** Indexing speed is increased by 10x compared to non-GPU configurations, significantly reducing the time-to-market for data-heavy applications. * **Cost Efficiency:** Indexing costs are reduced to approximately 25% of standard costs, and users only pay for active processing through OpenSearch Compute Units (OCU) rather than idle instance time. * **Serverless Management:** There is no need to provision or manage GPU instances manually; OpenSearch Service automatically detects acceleration opportunities and isolates workloads within the user's Amazon VPC. * **Operational Scope:** Acceleration is automatically applied to both initial indexing and subsequent force-merge operations. ### Automated Vector Index Optimization Auto-optimization removes the requirement for deep vector expertise by automatically balancing competing performance metrics. * **Simplified Tuning:** The system replaces manual index tuning—which can traditionally take weeks—with automated configurations. * **Resource Balancing:** The tool finds the optimal trade-off between search latency, search quality (recall rates), and memory requirements. * **Improved Accuracy:** Users can achieve higher recall rates and better cost savings compared to using default, unoptimized index configurations. ### Configuration and Integration These features can be integrated into new or existing OpenSearch Service domains and Serverless collections through the AWS Console or CLI. * **CLI Activation:** Users can enable acceleration on existing domains using the `update-domain-config` command with the `--aiml-options` flag set to enable `ServerlessVectorAcceleration`. * **Index Settings:** To leverage GPU processing, users must create a vector index with specific settings, notably setting `index.knn.remote_index_build.enabled` to `true`. * **Supported Workloads:** The service supports standard OpenSearch operations, including the Bulk API for adding vector data and text embeddings. For organizations managing large-scale vector workloads for RAG (Retrieval-Augmented Generation) or semantic search, enabling GPU acceleration is a highly recommended step to reduce operational overhead. Developers should transition existing indexes to include the `remote_index_build` setting to take immediate advantage of the improved speed and reduced OCU pricing.

vector-dbaigen-aiaws+4
aws

Amazon S3 Vectors now generally available with increased scale and performance | AWS News Blog (opens in new tab)

Amazon S3 Vectors has reached general availability, establishing the first cloud object storage service with native support for storing and querying vector data. This serverless solution allows organizations to reduce total ownership costs by up to 90% compared to specialized vector database solutions while providing the performance required for production-grade AI applications. By integrating vector capabilities directly into S3, AWS enables a simplified architecture for retrieval-augmented generation (RAG), semantic search, and multi-agent workflows. ### Massive Scale and Index Consolidation The move to general availability introduces a significant increase in data capacity, allowing users to manage massive datasets without complex infrastructure workarounds. * **Increased Index Limits:** Each index can now store and search across up to 2 billion vectors, representing a 40x increase from the 50 million limit during the preview phase. * **Bucket Capacity:** A single vector bucket can now scale to house up to 20 trillion vectors. * **Simplified Architecture:** The increased scale per index removes the need for developers to shard data across multiple indexes or implement custom query federation logic. ### Performance and Latency Optimizations The service has been tuned to meet the low-latency requirements of interactive applications like conversational AI and real-time inference. * **Query Response Times:** Frequent queries now achieve latencies of approximately 100ms or less, while infrequent queries consistently return results in under one second. * **Enhanced Retrieval:** Users can now retrieve up to 100 search results per query (increased from 30), providing broader context for RAG applications. * **Write Throughput:** The system supports up to 1,000 PUT transactions per second for streaming single-vector updates, ensuring new data is immediately searchable. ### Serverless Efficiency and Ecosystem Integration S3 Vectors functions as a fully serverless offering, eliminating the need to provision or manage underlying instances while paying only for active storage and queries. * **Amazon Bedrock Integration:** It is now generally available as a vector storage engine for Bedrock Knowledge Bases, facilitating the building of RAG applications. * **OpenSearch Support:** Integration with Amazon OpenSearch allows users to utilize S3 Vectors for storage while leveraging OpenSearch for advanced analytics and search features. * **Expanded Footprint:** The service is now available in 14 AWS Regions, up from five during the preview period. With its massive scale and 90% cost reduction, S3 Vectors is a primary candidate for organizations looking to move AI prototypes into production. Developers should consider migrating high-volume vector workloads to S3 Vectors to benefit from the serverless operational model and the native integration with the broader AWS AI stack.

vector-dbairagamazon-s3+4
line

Milvus: Building a (opens in new tab)

LINE VOOM transitioned its recommendation system from a batch-based offline process to a real-time infrastructure to solve critical content freshness issues. By adopting Milvus, an open-source vector database, the team enabled the immediate indexing and searching of new video content as soon as it is uploaded. This implementation ensures that time-sensitive posts are recommended to users without the previous 24-hour delay, significantly enhancing user engagement. ### Limitations of the Legacy Recommendation System * The original system relied on daily offline batch processing for embedding generation and similarity searches. * New content, such as holiday greetings or trending sports clips, suffered from a "lack of immediacy," often taking up to a full day to appear in user feeds. * To improve user experience, the team needed to shift from offline candidate pools to an online system capable of real-time Approximate Nearest Neighbor (ANN) searches. ### Selecting Milvus as the Vector Database * The team evaluated Milvus and Qdrant based on performance, open-source status, and on-premise compatibility. * Milvus was selected due to its superior performance, handling 2,406 requests per second compared to Qdrant's 326, with lower query latency (1ms vs 4ms). * Key architectural advantages of Milvus included the separation of storage and computing, support for both stream and batch inserts, and a diverse range of supported in-memory index types. ### Reliability Verification via Chaos Testing * Given the complexity of Milvus clusters, the team performed chaos testing by intentionally injecting failures like pod kills and scaling events. * Tests revealed critical vulnerabilities: killing the `Querycoord` led to collection release and search failure, while losing the `Etcd` quorum caused total metadata loss. * These findings highlighted the need for robust high-availability (HA) configurations to prevent service interruptions during component failures. ### High Availability (HA) Implementation Strategies * **Collection-Level HA:** To prevent search failures during coordinator issues, the team implemented a dual-writing system where embeddings are recorded in two separate collections simultaneously. * **Alias Switching:** Client applications use an "alias" to reference collections; if the primary collection becomes unavailable, the system instantly switches the alias to the backup collection to minimize downtime. * **Coordinator-Level HA:** To eliminate single points of failure, coordinators (such as `Indexcoord`) were configured in an Active-Standby mode, ensuring a backup is always ready to take over management tasks. To successfully deploy a large-scale real-time recommendation engine, it is critical to select a vector database that decouples storage from compute and to implement multi-layered high-availability strategies, such as dual-collection writing and active-standby coordinators, to ensure production stability.

vector-dbhigh-availabilityembeddingmilvus+3
line

Implementing a RAG-Based Bot to (opens in new tab)

To address the operational burden of handling repetitive user inquiries for the AWX automation platform, LY Corporation developed a support bot utilizing Retrieval-Augmented Generation (RAG). By combining internal documentation with historical Slack thread data, the system provides automated, context-aware answers that significantly reduce manual SRE intervention. This approach enhances service reliability by ensuring users receive immediate assistance while allowing engineers to focus on high-priority development tasks. ### Technical Infrastructure and Stack * **Slack Integration**: The bot is built using the **Bolt for Python** framework to handle real-time interactions within the company’s communication channels. * **LLM Orchestration**: **LangChain** is used to manage the RAG pipeline; the developers suggest transitioning to LangGraph for teams requiring more complex multi-agent workflows. * **Embedding Model**: The **paraphrase-multilingual-mpnet-base-v2** (SBERT) model was selected to support multi-language inquiries from LY Corporation’s global workforce. * **Vector Database**: **OpenSearch** serves as the vector store, chosen for its availability as an internal PaaS and its efficiency in handling high-dimensional data. * **Large Language Model**: The system utilizes **OpenAI (ChatGPT) Enterprise**, which ensures business data privacy by preventing the model from training on internal inputs. ### Enhancing LLM Accuracy through RAG and Vector Search * **Overcoming LLM Limits**: Traditional LLMs suffer from "hallucinations," lack of up-to-date info, and opaque sourcing; RAG fixes this by providing the model with specific, trusted context during the prompt phase. * **Embedding and Vectorization**: Textual data from wikis and chats are converted into high-dimensional vectors, where semantically similar phrases (e.g., "Buy" and "Purchase") are stored in close proximity. * **k-NN Retrieval**: When a user asks a question, the bot uses **k-Nearest Neighbors (k-NN)** algorithms to retrieve the top *k* most relevant snippets of information from the vector database. * **Contextual Generation**: Rather than relying on its internal training data, the LLM generates a response based specifically on the retrieved snippets, leading to higher accuracy and domain-specific relevance. ### AWX Support Bot Workflow and Data Sources * **Multi-Source Indexing**: The bot references two main data streams: the official internal AWX guide wiki and historical Slack inquiry threads where previous solutions were discussed. * **Automated First Response**: The workflow begins when a user submits a query via a Slack workflow; the bot immediately processes the request and provides an initial AI-generated answer. * **Human-in-the-Loop Validation**: After receiving an answer, users can click "Issue Resolved" to close the ticket or "Call AWX Admin" if the AI's response was insufficient. * **Efficiency Gains**: This tiered approach filters out "RTFM" (Read The F***ing Manual) style questions, ensuring that human administrators only spend time on unique or complex technical issues. Implementing a RAG-based support bot is a highly effective strategy for SRE teams looking to scale their internal support without increasing headcount. For the best results, organizations should focus on maintaining clean internal documentation and selecting embedding models that reflect the linguistic diversity of their specific workforce.

vector-dbairagpython+5