open-source

15 posts

discord

Osprey: Open Sourcing our Rule Engine (opens in new tab)

Osprey is an open-source safety rules engine designed to help platforms address emerging threats without the need to build custom security infrastructure from scratch. Developed through a collaboration between industry experts and the internet.dev team, it provides a standardized framework for investigating real-time activities and deploying dynamic safety measures. By using this engine, companies can significantly reduce engineering overhead while maintaining a proactive stance against platform abuse. **The Osprey Safety Rules Engine** * Functions as an open-source toolset aimed at replacing the fragmented, "reinvent the wheel" approach many companies take toward platform safety. * Enables security and moderation teams to monitor live activity streams to identify patterns of misuse as they occur. * Allows for the rapid creation and implementation of dynamic rules, ensuring that safety protocols can evolve alongside shifting threat landscapes. **Streamlining Engineering and Investigation** * Reduces the technical debt typically associated with building and maintaining internal safety tools by providing a pre-built, scalable engine. * Facilitates deep-dive investigations into platform events, allowing teams to react to threats with minimal intervention from core engineering staff. * Promotes a collaborative approach to safety by making the underlying technology accessible to the broader developer community. For organizations looking to strengthen their security posture, Osprey offers a practical alternative to bespoke safety systems. Teams should consider integrating the engine to automate their threat responses and leverage its real-time investigation capabilities to protect their users more efficiently.

gitlab

What’s new in Git 2.53.0? (opens in new tab)

Git 2.53.0 introduces significant performance and maintenance improvements, specifically targeting large repositories and complex history rewriting workflows. Key updates include compatibility between geometric repacking and partial clones, as well as more granular control over commit signatures during imports. These enhancements collectively move Git toward more efficient repository management and better data integrity for modern development environments. ## Geometric Repacking Support with Promisor Remotes * Git utilizes repacking to consolidate loose objects into packfiles, with the "geometric" strategy maintaining a size-based progression to minimize the computational overhead found in "all-into-one" repacks. * Previously, geometric repacking was incompatible with partial clones because it could not correctly identify or manage "promisor" packfiles, which contain the metadata for objects expected to be backfilled from a remote. * The 2.53.0 release enables geometric repacking to process promisor packfiles separately, preserving the promisor marker and preventing the tool from crashing when used within a partial clone repository. * This fix removes a major blocker for making the geometric strategy the default repacking method for all Git repositories. ## Preserving Valid Signatures in git-fast-import(1) * The `git-fast-import` tool, a backend for high-volume data ingestion and history rewriting, previously lacked the nuance to handle commit signatures during partial repository edits. * A new `strip-if-invalid` mode has been added to the `--signed-commits` option to solve the "all-or-nothing" problem where users had to choose between keeping broken signatures or stripping valid ones. * This feature allows Git to automatically detect which signatures remain valid after a rewrite and only strip those that no longer match their modified commits. * This provides a foundation for tools like `git-filter-repo` to preserve the chain of trust for unchanged commits during migration or cleaning operations. ## Expanded Data in git-repo-structure * The `structure` subcommand of `git-repo`, intended as a native alternative to the `git-sizer` utility, now provides deeper insights into repository scaling. * The command now reports the total inflated size and actual disk size of all reachable objects, categorized by type: commits, trees, blobs, and tags. * These metrics are essential for administrators managing massive repositories, as they help identify which object types are driving disk consumption and impacting performance. These updates reflect Git’s continued focus on scalability and developer experience, particularly for organizations managing massive codebases. Users of partial clones and repository migration tools should consider upgrading to 2.53.0 to leverage the improved repacking logic and more sophisticated signature handling.

kakao

Smarter and More (opens in new tab)

Kakao has released Kanana-2, a high-performance open-source language model specifically engineered to power Agentic AI by enhancing tool-calling and instruction-following capabilities. Surpassing its predecessors and rivaling global frontier models like Qwen3, Kanana-2 offers a versatile suite of variants designed for practical, high-efficiency application in complex service environments. ### Optimized Model Lineup: Base, Instruct, and Thinking * **Kanana-2-30b-a3b-base:** Provided as a foundational model with pre-training weights, allowing researchers to fine-tune the model using their own datasets. * **Kanana-2-30b-a3b-instruct:** A version optimized through post-training to maximize the model's ability to follow complex user instructions accurately. * **Kanana-2-30b-a3b-thinking:** Kakao’s first reasoning-specialized model, designed for tasks requiring high-level logical thinking, such as mathematics and coding. ### Strengthening Agentic AI Capabilities * **Tool Calling:** Multi-turn tool-calling performance has improved more than threefold compared to Kanana-1.5, significantly enhancing its utility with the Model Context Protocol (MCP). * **Instruction Following:** The model's ability to understand and execute multi-step, complex user requirements has been refined to ensure reliable task completion. * **Reasoning-Tool Integration:** Unlike many reasoning models that lose instruction-following quality during deep thought, the "Thinking" variant maintains high performance in both logical deduction and tool use. ### High-Efficiency Architecture for Scale * **MLA (Multi-head Latent Attention):** Compresses memory usage to handle long contexts more efficiently, reducing the resources needed for extensive data processing. * **MoE (Mixture of Experts):** Activates only the necessary parameters during inference, maintaining high performance while drastically reducing computational costs and response times. * **Improved Tokenization:** A newly trained tokenizer has improved Korean language token efficiency by 30%, enabling faster throughput and lower latency in high-traffic environments like KakaoTalk. ### Expanded Multilingual Support * **Broad Linguistic Reach:** The model has expanded its support from just Korean and English to include six languages: Korean, English, Japanese, Chinese, Thai, and Vietnamese. By open-sourcing Kanana-2, Kakao provides a robust foundation for developers seeking to build responsive, tool-integrated AI services. Its focus on practical efficiency and advanced reasoning makes it an ideal choice for implementing agentic workflows in real-world applications where speed and accuracy are critical.

naver

Naver TV (opens in new tab)

This technical session from NAVER ENGINEERING DAY 2025 details the transition from traditional open-source exporters to a Telegraf-based architecture for collecting custom system metrics. By evaluating various monitoring tools through rigorous benchmarking, the developers demonstrate how Telegraf provides a more flexible and high-performance framework for infrastructure observability. The presentation concludes that adopting Telegraf streamlines the metric collection pipeline and offers superior scalability for complex, large-scale service environments. ### Context and Motivation for Open-Source Exporters * The project originated from the need to overcome the limitations of standard open-source exporters that lacked support for specific internal business logic. * Engineers sought a unified way to collect diverse data points without managing dozens of fragmented, single-purpose agents. * The primary goal was to find a solution that could handle high-frequency data ingestion while maintaining low resource overhead on production servers. ### Benchmark Testing for Metric Collection * A comparative analysis was conducted between several open-source monitoring agents to determine their efficiency under load. * Testing focused on critical performance indicators, including CPU and memory footprint during peak metric throughput. * The results highlighted Telegraf's stability and consistent performance compared to other exporter-based alternatives, leading to its selection as the primary collection tool. ### Telegraf Architecture and Customization * Telegraf operates as a plugin-driven agent, utilizing four distinct categories: Input, Processor, Aggregator, and Output plugins. * The development team shared their experience writing custom exporters by leveraging Telegraf’s modular Go-based framework. * This approach allowed for the seamless transformation of raw data into various formats (such as Prometheus or InfluxDB) using a single, unified configuration. ### Operational Gains and Technical Options * Post-implementation, the system saw a significant reduction in operational complexity by consolidating various metric streams into a single agent. * Specific Telegraf options were utilized to fine-tune the collection interval and batch size, optimizing the balance between data granularity and network load. * The migration improved the reliability of metric delivery through built-in retry mechanisms and internal buffers that prevent data loss during transient network failures. For teams currently managing a sprawling array of open-source exporters, migrating to a Telegraf-based architecture is recommended to centralize metric collection. The plugin-based system not only reduces the maintenance burden but also provides the necessary extensibility to support specialized custom metrics as service requirements evolve.

discord

ROOST Announces “Coop” and “Osprey”: Free, Open-Source Trust and Safety Infrastructure for the AI Era (opens in new tab)

ROOST, a non-profit dedicated to digital safety, has launched two open-source tools, Coop and Osprey, to provide enterprise-grade content moderation and threat investigation capabilities to organizations of all sizes. By open-sourcing technology previously developed by industry leaders like Discord and Cove, ROOST aims to democratize access to the infrastructure required to detect, triage, and respond to online harms. This initiative shifts Trust and Safety from a proprietary competitive advantage to a shared public resource, enabling platforms to prioritize user protection without the burden of expensive enterprise software. ### Content Review and Compliance with Coop Built on technology acquired from Cove and utilized by platforms like Notion, Coop focuses on the human-in-the-loop aspect of content moderation. * The platform provides robust tools for content review, allowing teams to route specific cases to subject-matter experts for deeper analysis. * It includes built-in integration with the National Center for Missing & Exploited Children’s (NCMEC) API, automating the mandatory reporting process for child sexual abuse material (CSAM). * The interface is designed to surface relevant context and metadata, ensuring moderators can make informed decisions and take immediate action against policy violations. ### Incident Response and Investigation with Osprey Osprey is a lightweight investigation tool originally developed by Discord to manage large-scale safety incidents and platform-wide threats. * It serves as a foundation for incident response, helping safety teams understand platform trends and investigate coordinated threats like phishing or harassment campaigns. * The tool is designed to be user-friendly and accessible for grassroots communities while remaining powerful enough for established platforms. * Early adopters, including the decentralized social network Bluesky, are implementing Osprey to demonstrate that effective safety infrastructure can be scalable and resource-efficient. ### A Collaborative Model for Safety Infrastructure The launch of these tools represents a strategic shift toward a collaborative "public-interest" model for digital defense. * ROOST acquired the intellectual property of Cove and received the donation of Osprey from Discord to ensure these tools remain available as a public good. * The initiative is backed by philanthropic funding and legal support from Perkins Coie, removing the financial barriers that often prevent smaller platforms from implementing high-level safety measures. * Major industry players like Notion and Bluesky are championing the move, signaling an industry-wide push to share safety innovations rather than silo them. Platforms and developers should prepare to integrate these tools into their safety stacks as they become publicly available in the coming months. By adopting open-source infrastructure for routine tasks like NCMEC reporting and incident triage, organizations can focus their internal resources on platform-specific innovations while maintaining a high standard of digital safety.

datadog

Robust statistical distances for machine learning | Datadog (opens in new tab)

Datadog has introduced Toto, a new open-weights foundation model specifically designed for time-series forecasting and anomaly detection within observability contexts. While general-purpose time-series models often struggle with the unique volatility and high-frequency patterns of IT telemetry, Toto is pre-trained on a massive dataset of 500 billion observations to provide superior zero-shot performance. This release, accompanied by the BOOM benchmark, addresses the critical need for specialized AI tools capable of handling the complexity of modern cloud infrastructure. ### Toto Model Architecture and Training * Toto utilizes a decoder-only transformer architecture, adapting large language model (LLM) principles to the domain of continuous numerical data. * The model employs a "patching" mechanism, which groups multiple time-series data points into single tokens to improve computational efficiency and allow the model to capture longer historical dependencies. * It incorporates Rotary Positional Embeddings (RoPE) to better handle sequences of varying lengths and maintain temporal relationships across different frequencies. * Training was conducted on a curated dataset of 500 billion anonymized data points from real-world observability metrics, including CPU usage, memory consumption, and network traffic. ### Specialized Observability Features * Unlike existing models like TimesFM or Chronos, which are trained on diverse but general datasets like weather or retail trends, Toto is optimized for the specific "spikiness" and abrupt level shifts common in IT environments. * The model supports zero-shot forecasting, allowing users to generate predictions for new metrics immediately without the need for expensive or time-consuming fine-tuning. * Toto is designed to handle varying sampling rates, from one-second intervals to hourly aggregations, making it versatile across different infrastructure layers. * The open-weights release on Hugging Face allows researchers and engineers to integrate the model into their own AIOps workflows or private cloud environments. ### The BOOM Evaluation Framework * Datadog released the Benchmarking Observability Models (BOOM) framework to provide a standardized method for evaluating time-series models on infrastructure-specific tasks. * BOOM focuses on metrics that represent real-world operational challenges, such as seasonal traffic patterns and sudden system failures. * Comparative testing shows that Toto consistently outperforms general-purpose models in Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) when applied to observability datasets. * The benchmark provides a transparent way for the industry to measure progress in time-series foundation models, moving beyond generic datasets that do not reflect the realities of microservices and distributed systems. Organizations looking to automate capacity planning, optimize cloud spend, or implement intelligent alerting should consider adopting Toto for their time-series analysis. By utilizing the open-weights model alongside the BOOM benchmark, teams can achieve high-accuracy forecasting and objective performance validation without the overhead of building specialized models from scratch.