clickhouse

5 posts

gitlab

GitLab Patch Release: 19.1.2, 19.0.4, 18.11.7 | GitLab Docs (opens in new tab)

GitLab released patch versions 19.1.2, 19.0.4, and 18.11.7 on July 8, 2026, addressing multiple security vulnerabilities and bugs in CE and EE. GitLab strongly recommends that all affected self-managed installations upgrade immediately; GitLab.com is already patched, while GitLab Dedicated customers need no action. The release includes fixes ranging from cross-site scripting and HTML injection to authorization and credential-access issues. ## Release Scope and Upgrade Guidance - Applies to GitLab Community Edition and Enterprise Edition. - Patch releases address security vulnerabilities and important bugs. - Scheduled patch releases occur twice monthly, on the second and fourth Wednesdays. - Vulnerability details are generally made public 90 days after the fixing release. - All self-managed deployments—including Omnibus, source, and Helm installations—should upgrade to the latest supported patch version. ## High-Severity Browser Script Injection Fixes - **CVE-2026-6896 — CVSS 8.7** - Affected the vulnerability evidence table renderer in GitLab EE. - An authenticated developer could potentially execute scripts in another user’s browser through unsanitized input. - **CVE-2026-13320 — CVSS 7.3** - Affected wiki markup rendering in GitLab CE and EE. - Improper sanitization could allow an authenticated user to execute scripts in another user’s browser. ## Authorization and Information Disclosure Fixes - **CVE-2026-11827 — CVSS 4.9** - Affected repository mirroring in GitLab EE. - Maintainers could potentially access credentials stored for another user because of inadequate authorization checks. - **CVE-2026-8472 — CVSS 4.3** - Affected work items in GitLab EE. - Users with minimal access could read metadata from work items in private projects. - **CVE-2026-7492 — CVSS 4.3** - Affected commit discussion display in GitLab CE and EE. - Unauthenticated users could determine whether a private project existed through cross-project references. ## Repository and Configuration Security Fixes - **CVE-2025-12506 — CVSS 3.5** - Addressed ambiguous Git tag or branch references. - A repository could display content in the web interface that differed from the content available for download. - **CVE-2026-13151 — CVSS 2.7** - Fixed incorrect authorization in GitLab EE group-level settings. - Some authenticated users could modify settings beyond their intended permissions. - **CVE-2026-6352 — CVSS 2.7** - Fixed authorization flaws in compliance violation management. - Auditor-level users could modify compliance records through certain GraphQL operations. ## Bug Fixes in GitLab 19.1.2 - Set and backfilled `organization_id` for OAuth applications before constraint validation. - Upgraded Go to version 1.25.11. - Fixed multi-architecture tags on the legacy container registry path. - Improved external agent flows by using commit author and committer identities. - Fixed ClickHouse 23.x compatibility for `ci_finished_builds`. - Added cursor pagination and checkpoint limits to Duo workflow event retrieval. - Reverted a problematic merge request and removed an obsolete active-user cron schedule. - Fixed approval-rule regressions affecting Developer-authored merge requests. - Resolved a memory leak on the commits page caused by eager description loading. - Updated the builder image revision to `5.60.1`. ## Additional Fixes in GitLab 19.0.4 - Backported the OAuth `organization_id` fixes. - Added Skopeo registry authentication through `CI_JOB_TOKEN`. - The release also contains further backported fixes, though the provided release notes are truncated before listing them. Self-managed GitLab administrators should upgrade to 19.1.2, 19.0.4, or 18.11.7 according to their supported release line, prioritizing the update because of the two high-severity script-injection vulnerabilities.

stripe

Stripe Projects adds new agent integrations, more providers, and custom developer controls (opens in new tab)

Agent traffic now exceeds human internet traffic, driven largely by agents independently building software and integrating APIs. Stripe reports that agents account for nearly 40% of its documentation traffic and 70% of API-resource requests through the Stripe CLI. Stripe Projects is expanding to help agents handle the infrastructure, credentials, services, and operational controls surrounding software development. ## Agent Integrations - Stripe Projects is available as a skill in Hermes, an open-source AI agent from Nous Research. - Hermes can retain context across sessions, allowing it to collaborate on complex projects over days or weeks. - Factory Droids and Warp have integrated the Projects CLI into their coding workflows. ## Expanded Provider Support - Projects now supports 49 providers, adding 16 new integrations. - New providers include: - Metronome for usage-based billing - Wix for storefronts - ClickHouse for LLM observability - Agents can provision applications, billing, storefront, monitoring, and other services without manually navigating provider dashboards. ## Controls for Safe Agent Provisioning Stripe is adding guardrails similar to those used for agent-driven purchases: - **Unified cost visibility:** Developers can view current and historical spending across providers for each project. - **Per-provider spending limits:** Teams can set different caps for services such as AI models, hosting, and databases. - **Named environments:** Isolated credentials can be created for development, staging, production, or custom environments. Agents default to development, reducing the risk of affecting production. - **Platform delegation:** Platforms can provision services for users using scoped credentials and white-labeling, keeping developers inside the platform’s environment. ## Future Direction Stripe plans to extend Projects across the full lifecycle of agent-built software, including operations and security. Planned additions include stronger security primitives for autonomous agents and a data layer that lets providers meter and bill for software created by agents. Stripe’s broader recommendation is to use Projects as an agent-accessible way to provision infrastructure—for example, asking an agent to add a Prisma database.

gitlab

Introducing GitLab Orbit (opens in new tab)

GitLab Orbit is a public-beta, queryable graph that connects code with merge requests, pipelines, deployments, vulnerabilities, incidents, and ownership. GitLab argues that this unified context makes AI agents faster, more accurate, and capable of answering cross-system questions that traditional file search or RAG cannot. In testing, Orbit improved code-review accuracy while reducing agent runtime, token use, and hallucinations. ## The Problem with Code-Only Agents - Agents often struggle to understand the systems surrounding code: - Related files and dependencies - Tests and pipelines - Deployments and environments - Vulnerabilities and ownership - Work items and merge requests - In large or multi-repository codebases, agents can waste tokens exploring irrelevant paths, miss dependencies, or run out of context. - This can produce code that appears correct but is later reverted or requires substantial human correction. ## Results from Compare the Market - Compare the Market tested four context-retrieval approaches across 79 real merge requests. - An Orbit-grounded AI reviewer: - Placed accurate inline comments about 70% of the time. - Outperformed RAG, which achieved roughly 58%. - Produced slightly better summaries of key changes: 68% versus 66%. - RAG performed worse than the other tested approaches, including using no additional context. ## Faster and More Efficient Coding Agents - Claude Code and other agents can connect to Orbit through the Model Context Protocol (MCP). - Instead of crawling a repository to infer relationships, an agent can query the graph directly for: - Where code is located - What depends on it - Which tests and pipelines cover it - GitLab reports up to: - 11× faster execution - 4.5× lower token usage - 45× fewer hallucinations ## Cross-System Engineering Workflows Orbit enables agents on the GitLab Duo Agent Platform to investigate relationships beyond source code. - **Pipeline failure triage** - Agents can identify merge requests associated with a failing job. - They can find other projects and in-flight changes likely to encounter the same failure. - This can help teams resolve shared incidents centrally rather than repeating investigations. - **Vulnerability blast-radius analysis** - Queries can trace vulnerable components through services, pipelines, environments, and owning teams. - Security teams can produce assigned remediation plans shortly after a CVE is discovered. - **Engineering metrics** - Teams can query relationships between cycle time, pipeline failure rates, and deployment frequency without waiting for custom dashboards or SQL analysis. - **Migration planning** - Orbit identifies dependent services, jobs, environments, and owners. - This reduces the risk of discovering hidden dependencies late in a migration. ## Architecture and Access - Orbit ingests lifecycle data through change-data capture into ClickHouse. - It parses code in 12 languages, including Ruby, Java, Python, TypeScript, Rust, Go, C#, C++, and PHP. - GitLab reports indexing more than: - 40,000 projects - 500 million nodes - 2 billion edges - The graph can be queried through: - A Cypher-like query language - MCP - REST - The GitLab CLI - An event-driven engine keeps the graph current as changes occur. - Indexing runs separately from GitLab, so query traffic does not burden the GitLab instance. - Authorization follows existing GitLab permissions, limiting agents to data their users can access. - Queries pass through validation, planning, optimization, and security stages before reaching the database. ## Engineer-Facing Data Explorer - The Data Explorer provides direct access to the same graph without an AI agent. - Engineers can use it to: - Investigate incidents - Trace dependencies across services - Diagnose recurring CI failures - GitLab positions it as a way to answer open-ended system questions in seconds rather than reconstructing the answer manually across multiple tools. GitLab Orbit is best suited to organizations where code, CI/CD, security, and ownership data are spread across large repositories or many projects. Its main recommendation is to use one permission-aware graph as shared context for both AI agents and engineers, rather than relying on repository search or disconnected tool calls.

cloudflare

How we built Cloudflare's data platform and an AI agent on top of it (opens in new tab)

Cloudflare built Town Lake to unify data scattered across production databases, analytics systems, streams, and object storage behind one governed SQL interface. The platform combines Trino, Iceberg on R2, DataHub, and custom access-control and PII-detection services to make data fresher, more discoverable, and safer to use. Skipper extends Town Lake with a natural-language AI interface intended to provide fast, accurate, and auditable answers without requiring users to write SQL. ## The Data Sprawl Problem - Cloudflare processes over a billion events per second across a network spanning more than 330 cities and 120 countries. - Relevant data was distributed across: - Postgres - ClickHouse - BigQuery - Kafka - Google Cloud Storage and R2 - Numerous pipelines and production databases - Users needed separate credentials, query languages, retention expectations, and system knowledge for each source. - Sampled analytics data worked for dashboards but was unsuitable for billing, usage calculations, and security investigations. - External vendors created cost and dependency concerns. - Important data was difficult to discover because table locations, schemas, joins, and customer-ID mappings depended on tribal knowledge. - Data infrastructure had historically been treated as a back-office service rather than core company infrastructure. ## Goals for the New Platform Cloudflare wanted a single place where authorized employees could answer questions about customers, traffic, billing, security events, and support activity. - Support both: - Fresh, accurate, unsampled data for billing and investigations - Fast, downsampled data for dashboards and exploration - Provide built-in governance: - Automatic PII detection - Sensitive tables locked down by default - Auditable access - Time-limited permission grants - Build the system using Cloudflare’s own products, including R2, Workers, Access, and Workflows. - Eventually let employees ask questions in plain English rather than requiring SQL knowledge. - That natural-language interface became Skipper. ## Town Lake’s Lakehouse Architecture Town Lake is a lakehouse: a query engine combines data from object storage and operational systems while a metadata layer makes the data behave like a unified database. - **Trino** serves as the query engine. - A single query can join Postgres, ClickHouse, and Iceberg tables stored on R2. - Trino pushes filters into source systems and combines results without requiring intermediate materialization. - **R2 Data Catalog and Apache Iceberg** store warm and cold data. - Iceberg provides schema evolution, time travel, partition evolution, and compaction. - Data can be rolled from per-minute to hourly and eventually daily granularity as it ages. - Older data becomes cheaper to store while remaining queryable. - Parquet files on R2 cost less than retaining equivalent data in an OLAP database. - **DataHub** provides the metadata catalog. - It stores table and column descriptions, owners, lineage, and glossary terms. - Users can discover what a table contains, which teams maintain it, and how it relates to upstream and downstream data. ## Access Control and Privacy - **Lifeguard** manages access policies. - Rules are stored in D1. - User and group memberships are retrieved dynamically from Cloudflare’s internal access-management system. - Lifeguard produces JSON policies that Trino reads over HTTP. - It also supplies access information to Skipper and the Gateway, allowing users to be blocked before queries execute. - **Skimmer** continuously scans tables for PII. - It samples rows from columns across the data platform. - Workers AI classifies whether columns contain personally identifiable information. Cloudflare’s overall approach is to combine unified querying, durable low-cost storage, rich metadata, and policy enforcement so data can be broadly useful without sacrificing accuracy or governance.

cloudflare

Our billing pipeline was suddenly slow. The culprit was a hidden bottleneck in ClickHouse (opens in new tab)

Cloudflare’s migration to per-namespace retention in ClickHouse unexpectedly caused billing queries to slow dramatically, even though I/O, memory usage, rows scanned, and parts read appeared normal. The hidden bottleneck was query-planning lock contention: as the new `(namespace, day)` partitioning scheme multiplied the number of data parts, queries spent much of their time waiting for a mutex protecting the table’s active-part list. Investigation with flame graphs exposed the issue, leading Cloudflare to develop ClickHouse fixes. ## A Petabyte-Scale ClickHouse Platform - Cloudflare stores more than 100 PB across dozens of ClickHouse clusters. - Its Ready-Analytics system lets hundreds of internal teams share a massive table using: - A `namespace` to identify each dataset - A standard schema - A primary key of `(namespace, indexID, timestamp)` - By December 2024, the system contained more than 2 PiB and ingested millions of rows per second. ## The Limits of a Global Retention Policy - The table was partitioned by day, and a retention job dropped partitions older than 31 days. - This prevented teams from applying different retention periods: - Some needed years of data. - Others needed only a few days. - Teams requiring custom retention had to use more complicated, conventional table setups. ## Moving to Per-Namespace Partitions - Cloudflare considered: - Creating a separate table for every namespace. - Changing the partition key from `(day)` to `(namespace, day)`. - They chose the second option because it preserved the existing retention workflow while enabling namespace-level deletion. - The team expected more total parts but assumed query performance would remain stable because queries already filtered by namespace. - Migration began in January 2025 using ClickHouse’s `Merge` table feature. ## Billing Queries Begin to Slow - By late March 2025, billing aggregation jobs were approaching their daily deadlines. - Standard performance indicators looked healthy: - I/O and memory were normal. - Queries scanned no more rows or parts than before. - Query latency correlated strongly with the growing total number of parts in the cluster, revealing that merely having more parts could hurt performance. ## Finding the Hidden Lock Bottleneck - Cloudflare used ClickHouse’s `trace_log` to generate flame graphs for leaf `SELECT` queries. - CPU traces showed that roughly 45% of sampled CPU time was spent in `filterPartsByPartition`, which filters parts during query planning. - Reordering pruning heuristics produced only a 5% improvement. - “Real” traces, which include waiting and inactive threads, exposed the real issue: - More than half of query time was spent waiting on a mutex protecting the table’s active-part list. - Every query-planning thread had to contend for the `MergeTreeData` lock. - The migration increased the number of parts enough to make this previously unnoticed planning bottleneck dominant. The main lesson is that ClickHouse performance can degrade during query planning even when execution metrics look normal. When partitioning changes substantially increase part counts, teams should monitor planning time and lock contention—not just data scanned, I/O, or memory—and use real-time flame graphs to identify waits hidden by CPU-only profiling.