VLOps:Event-driven MLOps & Omni-Evaluator (opens in new tab)
Naver’s VLOps framework introduces an event-driven approach to MLOps, designed to overcome the rigidity of traditional pipeline-based systems like Kubeflow. By shifting from a monolithic pipeline structure to a system governed by autonomous sensors and typed messages, Naver has achieved a highly decoupled and scalable environment for multimodal AI development. This architecture allows for seamless functional expansion and cross-cloud compatibility, ultimately simplifying the transition from model training to large-scale evaluation and deployment. ### Event-Driven MLOps Architecture * Operations such as training, evaluation, and deployment are defined as "Typed Messages," which serve as the primary units of communication within the system. * An "Event Sensor" acts as the core logic hub, autonomously detecting these messages and triggering the corresponding tasks without requiring a predefined, end-to-end pipeline. * The system eliminates the need for complex version management of entire pipelines, as new features can be integrated simply by adding new message types. * This approach ensures loose coupling between evaluation and deployment systems, facilitating easier maintenance and infrastructure flexibility. ### Omni-Evaluator and Unified Benchmarking * The Omni-Evaluator serves as a centralized platform that integrates various evaluation engines and benchmarks into a single workflow. * It supports real-time monitoring of model performance, allowing researchers to track progress during the training and validation phases. * The system is designed specifically to handle the complexities of Multimodal LLMs, providing a standardized environment for diverse testing scenarios. * User-driven triggers are supported, enabling developers to initiate specific evaluation cycles manually when necessary. ### VLOps Dashboard and User Experience * The VLOps Dashboard acts as a central hub where users can manage the entire ML lifecycle without needing deep knowledge of the underlying orchestration logic. * Users can trigger complex pipelines simply by issuing a message, abstracting the technical difficulties of cloud infrastructure. * The dashboard provides a visual interface for monitoring events, message flows, and evaluation results, improving overall transparency for data scientists and researchers. For organizations managing large-scale multimodal models, moving toward an event-driven architecture is highly recommended. This model reduces the overhead of maintaining rigid pipelines and allows engineering teams to focus on model quality rather than infrastructure orchestration.
