dropboxJul 20, 2026

How our universal content processing platform Riviera evolved for AI and beyond (opens in new tab)

Riviera evolved from Dropbox’s preview-generation service into a shared content-processing platform used by products including Search, Replay, Sign, and Dash. Its core insight was to compose reusable transformations rather than build separate pipelines for every file type and output. As AI increased demand for consistent document extraction and preparation, Dropbox expanded Riviera’s capabilities and began offering them through APIs and Model Context Protocol tools. ## The Preview Problem - Dropbox supports more than 300 file formats, each requiring outputs such as: - Thumbnails - Full previews - Extracted text - Streaming manifests - Metadata - Building a separate service for every format and output would duplicate logic, dependencies, and operational work. - Configurations and package versions could drift across services, making the system harder to maintain and scale. ## Reusable Transformations as the Foundation - Riviera treats previews as sequences of smaller, reusable transformations. - For example, a PowerPoint preview can be produced by: - Converting the presentation to PDF - Rendering each PDF page as an image - The same PDF-to-image transformation can support PDFs and other workflows requiring page images. - This approach enables new formats and products to reuse existing capabilities instead of starting from scratch. ## Separating Coordination from Execution - Riviera uses a central coordinator to: - Collect and validate requests - Compose transformation workflows - Cache responses - Dispatch jobs to backend workers - Each worker handles a specific transformation, creating a clear unit for maintenance and scaling. - The platform now includes more than 100 capabilities and performs hundreds of thousands of transformations per second. - New formats and transformations can generally be added as plugins without changing the core system. ## From Internal Service to Shared Platform - Other Dropbox teams quickly adopted Riviera when they discovered overlapping content-processing needs. - Machine learning teams reused preview thumbnails for image normalization, avoiding duplicate generation. - Search used Riviera to prepare documents for indexing, while Sign, DocSend, and Replay reused existing transformations. - Dropbox eventually opened the plugin model to product teams, allowing them to add capabilities while the Riviera team maintained the platform’s core architecture. - Replay particularly benefited from Riviera’s complex video transcoding and manipulation capabilities, accelerating product development from months to weeks. ## Supporting AI Workloads - Dash introduced greater demand for reliable document preparation before AI processing. - AI systems require content to be transformed into consistent, machine-readable representations, including: - Extracted text - Data from scanned pages - File metadata - Normalized versions of hundreds of file types - These are fundamentally content-transformation challenges rather than AI-model challenges. - Because Riviera already supported many formats and transformations, Dash could build on existing infrastructure instead of creating a separate document-processing system. ## Broader Availability - Dropbox is making Riviera’s capabilities available to external developers and design partners. - Access is provided through APIs and Model Context Protocol tools. - The platform is intended for applications such as content management, document automation, search indexing, and AI document processing. Riviera’s evolution demonstrates the value of a shared transformation platform: reusable workers reduce duplication, centralized coordination improves reliability, and each new capability benefits multiple products. For teams building content-heavy or AI-powered applications, using standardized transformation infrastructure can be more efficient than maintaining format-specific pipelines independently.

aimodel-context-protocoldistributed-systemsplugin-architecture+3

dropboxJun 25, 2026

How we used DSPy to turn AI evaluations into better responses in Dash chat (opens in new tab)

Dropbox uses DSPy to turn AI evaluations into improvements for its Dash chat agent. The process first calibrates LLM judges against human-labeled conversations, then uses those judges to optimize the agent’s system prompt. This feedback loop reduced incomplete answers and token usage while maintaining answer quality. ## The Complexity of Evaluating AI Agents - Agent quality depends on more than the final response: - Understanding user intent - Selecting relevant context - Choosing and using tools - Synthesizing information across documents, messages, and meetings - Handling ambiguity and follow-up turns - Producing grounded, complete answers - Evaluations therefore inspect the full interaction trajectory, not just the output. - Separate evaluations for intent understanding, tool use, context selection, grounding, adaptation, and task completion help identify the source of failures. - Reliable judges were necessary before evaluation results could safely guide agent improvements. ## Calibrating LLM Judges with Human Labels - Dropbox sampled internal chats containing final answers and agent trace logs. - Human reviewers scored five dimensions: - User-intent following - Semantic relevance - Tool calling - Instruction following - Context selection - Reviewers followed a structured process: - Determine whether the agent understood the request. - Check whether it selected appropriate context. - Inspect searches, retrievals, and other tool actions. - Verify that final claims were supported by evidence. - Score relevance, grounding, completeness, and instruction adherence. - Many metrics used a 1–5 scale. - Reviewers also added: - Reasoning notes explaining their scores - Failure codes for issues such as stale evidence, missing context, unsupported claims, incomplete coverage, and poor personalization - These richer annotations helped improve judge prompts while also supporting debugging, error analysis, roadmap planning, and prioritization. ## Using DSPy to Improve Evaluation - DSPy was used to make LLM judges align more closely with human evaluations. - Judges were required to follow a retrospective workflow: - Infer the user’s intent - Inspect the conversation and agent trace - Review supporting evidence - Assess context selection and tool use - Produce scores, failure codes, and reasoning notes - GEPA and MIPROv2, optimization algorithms within DSPy, automatically proposed and tested prompt changes against human-labeled examples. - Optimization supported several scenarios: - Rewriting judge instructions entirely - Adapting a judge to another underlying model - Targeting specific failure modes while preserving the existing evaluation behavior The overall approach creates a scalable improvement loop: human labels calibrate the judges, calibrated judges provide consistent evaluation signals, and those signals guide improvements to the chat agent itself.

llmagentic-aidspyprompt-optimization+4

dropboxJun 12, 2026

How Dropbox uses MCP and Dash to close the design-to-code security gap (opens in new tab)

Dropbox found a significant gap between security design reviews and implementation. Only 12% of implementing pull requests linked back to their original threat models, and the median delay between review and code submission was about five weeks. To close this gap, Dropbox built a system using Dash, Model Context Protocol (MCP), and foundational models to automatically retrieve relevant security requirements and compare them with code during review. ## The Design-to-Code Gap - Threat models document risks, attack scenarios, and agreed-upon mitigations during security review. - These documents often remain in wikis or documentation systems, while implementation happens later through pull requests. - At Dropbox: - Only 12% of implementing PRs linked to their original design review. - Among 79 verified pairs, 54% of PRs were opened more than a month after the review. - The median delay was approximately five weeks, with some delays exceeding 11 months. - Only 29% of PRs were opened within two weeks of the security review. - About 15% of design reviews were filed retroactively, suggesting that some security-sensitive work was not identified early enough. ## Why Existing Tools Fall Short - Static analysis can detect whether certain security patterns or controls exist. - It generally cannot determine whether those controls satisfy the specific requirements agreed upon during design review. - Manual linking between PRs and threat models depends on engineers remembering additional workflow steps. - Reminder bots may improve compliance temporarily, but adherence tends to decline. - The core problem is not a lack of security documentation; it is the difficulty of making that existing context available during implementation. ## Dash and MCP as a Context Bridge - Dash already indexes Dropbox content and connected applications, including threat models and engineering documentation. - Dropbox used Dash’s MCP server to let an AI security agent search and read this content. - MCP provides a common interface for bringing multiple context sources into one agent session. - When a PR is opened, the agent retrieves relevant threat models and supporting documents. - A foundational model then compares the documented requirements with the proposed code. - For example, it can identify whether code implementing an endpoint enforces authentication required by the corresponding threat model. - Unlike traditional static analysis, this approach evaluates code against documented security intent, not just known code patterns. ## Integrating Security into Code Review - Dropbox surfaced the system directly within the existing code-review workflow rather than creating a separate security process. - This allows developers and reviewers to receive relevant security context where implementation decisions are already evaluated. - The same mechanism can potentially identify missing reviews when code appears security-sensitive but has no corresponding threat model. Dropbox’s approach shows how retrieval and AI reasoning can reconnect design decisions with implementation. Organizations can apply the same pattern beyond security—for example, to compliance requirements, privacy reviews, accessibility standards, or other design-to-code checks.

llmmodel-context-protocolcode-reviewstatic-analysis+2

dropboxMay 28, 2026

Beyond code generation: rethinking engineering productivity in the age of AI agents (opens in new tab)

AI coding agents have increased Dropbox’s code production, but they have also exposed bottlenecks in review, testing, release coordination, and operations. The central argument is that engineering productivity must be measured by end-to-end product velocity and customer impact—not code or pull-request volume alone. Dropbox is responding with agent platforms, stronger workflow infrastructure, broader quality metrics, and training that helps engineers adapt to new responsibilities. ## From Copilots to Agents - Copilots assist with explanations, snippets, and questions within existing workflows. - Agents can take scoped tasks, inspect repositories, edit files, run tests, fix failures, and return changes for human review. - Engineers remain responsible for intent, architecture, quality, and release decisions. - Increased parallel work and code output create new pressure on: - Code review systems - CI and testing infrastructure - Validation workflows - Release processes - Production operations - More code and pull requests do not necessarily create more customer value. ## Nova as Dropbox’s Agent Platform - Nova runs AI coding agents in controlled environments with relevant codebase context and internal engineering practices. - Its value comes from the surrounding platform—safe execution, workflow integration, guardrails, and human review—not only from the underlying model. - Nova generates approximately one in twelve Dropbox pull requests. - It supports both feature work and maintenance tasks, including: - Migrations - Flaky-test remediation - Bug investigation - Dependency updates - Other high-effort engineering work - The intended workflow is structured: define the task, let the agent work within constraints, validate the result, and require human approval before production. ## Measuring Product Velocity and Impact - Pull-request throughput was useful when implementation speed was the main constraint, but it is insufficient as AI increases output. - Dropbox evaluates whether the broader engineering system can absorb additional work efficiently. - Its four-stage measurement model tracks: - **Fuel:** Whether engineers use AI tools - **Adoption:** How teams change their workflows - **Output:** Whether AI contributes to production work - **Impact:** Whether products reach customers faster and create greater value - Quality and trust metrics include review turnaround time, first-run test pass rate, defect ratio, and rework rate. - Productivity improvements must not come at the expense of reliability or customer trust. ## Evolving Engineering Workflows - Engineers increasingly focus on defining intent, framing problems, reviewing changes, and making architectural and quality decisions. - Adoption requires more than tools; Dropbox uses hands-on learning, hackathons, bootcamps, workflow examples, and peer-led training. - Teams should adopt agents at different speeds based on risk, context, and readiness. - The goal is not to automate every workflow, but to make agentic development safe, useful, measurable, and repeatable. ## Broader Lessons - AI does not remove bottlenecks; it shifts them downstream. - Organizations must invest in validation, orchestration, governance, workflow integration, and measurement alongside code generation. - Competitive advantage will come less from access to common foundation models and more from the systems built around them: internal context, safeguards, quality controls, and integrated engineering workflows. Dropbox’s experience suggests that companies should treat AI agents as a change to the entire software delivery system, not merely as faster coding tools. The strongest results will come from improving the full path from idea to validated production impact.

ai-agentcode-reviewcontinuous-integrationworkflow-automation+3

dropboxMay 21, 2026

Introducing Nova, our internal platform for coding agents (opens in new tab)

Nova is Dropbox’s internal cloud platform for running coding agents across the software development lifecycle. Rather than building separate tools for coding, CI debugging, migrations, and operational tasks, Dropbox created a shared platform that supports interactive sessions and autonomous workflows within its monorepo and infrastructure. The platform grounds agent changes in real builds and tests, making AI assistance more reliable and easier to integrate into engineering workflows. ## The Case for a Shared Platform - Engineering work includes repetitive but important tasks such as: - Debugging CI failures - Updating dependencies - Improving test coverage - Fixing flaky tests - Managing migrations and operational work - Different tasks require different interaction models: - Interactive chat for developer-driven work - Asynchronous workflows for long-running remediation and automation - Dropbox’s environment has specialized requirements: - A large monorepo - Bazel for builds and tests - Caching and remote execution - On-premises infrastructure - Dropbox-specific validation workflows - Off-the-shelf coding agents were designed primarily for local development and did not naturally fit this environment. ## How Nova Runs Coding Sessions - Each session runs in an isolated environment using a specific snapshot of the codebase. - Callers provide: - The repository commit - A task description - Optional validation commands - Iteration limits and branch settings - Nova can run builds and tests after an agent proposes a change. - If validation fails, the results are sent back to the agent so it can continue troubleshooting. - This creates a feedback loop of: - Propose a change - Validate it in the real environment - Correct failures - Repeat as needed - Nova supports multiple coding agents behind a common interface. - Engineers can access it through: - A web interface - A command-line client - An API - Internal scripts and services - The platform also provides prompt evaluation, observability, feedback collection, skills, plugins, and MCP integrations for accessing systems such as logs and monitoring tools. ## Deterministic Code Publication - Nova keeps code publication outside the agent. - Each session is limited to a single branch. - This makes active work and publication status predictable. - It avoids the complexity of agents creating and managing multiple branches. - The deterministic model simplifies automation such as: - Running tests - Rebasing onto the main branch - Tracking which changes belong to each session ## Engineering Workflows Using Nova ### Developer-Driven Sessions - Engineers use Nova’s web interface for quick fixes and prototypes without disrupting local work. - Validation commands can use Bazel selectivity tools to target the relevant compile and test dependencies. - Slack discussions can be carried into Nova sessions, preserving context and reducing manual setup. ### Flaky Test Remediation - Dropbox built Deflaker, a durable workflow connected to Athena, its flaky-test detection system. - Deflaker gathers examples of a test passing and failing. - It sends the associated logs to Nova. - The agent analyzes the evidence, identifies a likely cause, and proposes a fix. - This demonstrates how Nova can combine investigation, context gathering, and code changes in a longer-running automated process. ## Practical Takeaway Dropbox’s experience suggests that coding agents are most useful when embedded in existing engineering systems rather than treated as isolated code-generation tools. A shared platform like Nova can support many workflows while preserving consistent execution, validation, context, and observability.

observabilityslackmcpcontinuous-integration+4

dropboxApr 2, 2026

Improving storage efficiency in Magic Pocket, our immutable blob store (opens in new tab)

Magic Pocket’s immutable design protects data integrity but makes storage efficiency dependent on continuous reclamation. A new Live Coder service reduced write amplification while unintentionally creating severely under-filled volumes, driving fragmentation and storage overhead sharply upward. Dropbox responded by rethinking compaction, since its existing steady-state strategy was too slow to recover space from the resulting long tail of sparse volumes. ## The Cost of Immutability - Magic Pocket stores user files as immutable blobs distributed across its storage fleet. - Updates and deletions never modify data in place; obsolete blobs remain until compaction. - Garbage collection identifies unreferenced blobs, while compaction physically moves live blobs into new volumes and retires old ones. - Because closed volumes cannot be reopened, deleted data creates unused space unless it is actively consolidated. - Durability also increases storage requirements: - Replication stores multiple complete copies. - Erasure coding splits data into fragments and adds parity, providing fault tolerance with less overhead. - Fragmentation determines how efficiently that redundant capacity is used: - A volume with 50% live data effectively doubles required storage. - A volume with 10% live data uses roughly ten times the necessary space. ## The Live Coder Incident - A new on-the-fly erasure-coding service created severely under-filled volumes as it rolled out to new regions. - In the worst cases, less than 5% of a volume’s capacity contained live data. - Since volumes have fixed allocations, many mostly empty volumes consumed nearly as much raw capacity as full volumes. - Dropbox detected rising effective replication-factor signals, indicating more raw storage was being used per live byte. - The existing compaction system continued reclaiming space but was not designed for a long tail of extremely sparse volumes. - The incident demonstrated that compaction must adapt when the distribution of live data changes substantially. ## Steady-State L1 Compaction - Dropbox’s baseline strategy, L1, treats compaction as a packing problem. - It selects: - A highly filled host volume with available space. - Donor volumes whose live data fits into that space. - Live blobs from the donors are written into a new volume, eventually leaving the donors empty and removable. - L1 is simple, fast, and limits placement risk and metadata changes. - However, each run can read tens of GiB while typically producing only one densely packed volume. - Fewer than one complete volume is reclaimed on average because only donor volumes are fully drained. - This works well when volumes are already near full, but performs poorly when storage overhead is concentrated in many severely under-filled volumes.

database-designgarbage-collectionerasure-codingmagic-pocket+4

dropboxMar 25, 2026

Reducing our monorepo size to improve developer velocity (opens in new tab)

Dropbox’s server monorepo grew to 87GB, making full clones take over an hour and threatening GitHub’s 100GB limit. The root cause was inefficient Git delta compression of internationalization files, not unusually large source files. By changing how the repository was repacked, Dropbox reduced it to about 20GB and cut clone times to under 15 minutes. ## Repository Size and Developer Velocity - The monorepo contains backend services and libraries used across Dropbox. - AI feature development often requires coordinated changes across ranking, retrieval, evaluation, and UI systems. - A full clone exceeded one hour at 87GB, slowing onboarding and affecting CI jobs that start from fresh clones. - Internal synchronization systems also processed more data, increasing timeout and reliability risks. - The repository grew by roughly 20–60MB per day, with occasional increases above 150MB. - At that rate, Dropbox expected to hit GitHub Enterprise Cloud’s 100GB hard limit within months. ## How Git Compression Caused the Growth - Git normally reduces storage by representing similar file versions as deltas rather than complete copies. - Its default file-matching heuristic considers only the final 16 characters of a path. - Dropbox’s i18n files used paths such as: - `i18n/metaserver/[language]/LC_MESSAGES/[filename].po` - Because the language component appears early in the path, Git often compared files from different languages instead of related versions of the same language. - Translation updates consequently produced oversized deltas and disproportionately large pack files. ## Testing `--path-walk` - Dropbox tested Git’s experimental `--path-walk` option during a local repack. - The option considers the full directory structure when selecting delta candidates. - A local repack reduced the repository from the low-80GB range to the low-20GB range, confirming that packing—not data volume—was the main issue. - GitHub could not use this approach because it conflicted with server-side optimizations such as bitmaps and delta islands. ## Why Server-Side Repacking Was Necessary - Local optimization cannot permanently change the packs GitHub generates for clones and fetches. - GitHub dynamically constructs transfer packs based on what each client needs. - Dropbox’s mirror experiment showed that an aggressive repack could reduce the repository from 84GB to 20GB: - `git repack -adf --depth=250 --window=250` - The repack took approximately nine hours. - Dropbox worked with GitHub Support to apply a compatible server-side solution. - Larger `window` and `depth` values make Git search more thoroughly for compression opportunities, trading increased repack time for smaller storage and transfer sizes. ## Results - Repository size fell from 87GB to approximately 20GB—a 77% reduction. - Clone time dropped from more than an hour to under 15 minutes. - The work reduced pressure on GitHub’s repository size limit and improved the performance of developer and CI workflows. Dropbox’s experience shows that monorepo growth can result from repository layout interacting poorly with Git’s compression heuristics. When large repositories exhibit abnormal growth, teams should inspect pack-file behavior and consider server-side repacking rather than focusing only on removing large files.

database-designgithubgitcontinuous-integration+4

dropboxMar 17, 2026

How we optimized Dash's relevance judge with DSPy (opens in new tab)

Dropbox Dash needed a relevance judge that could score query–document pairs accurately, cheaply, and reliably at scale. Its original judge used OpenAI’s o3, but the cost made it impractical for large-scale labeling, while its prompt performed poorly when moved to the cheaper gpt-oss-120b model. Dropbox used DSPy’s GEPA optimizer to turn prompt tuning into a measurable feedback loop, improving alignment with human judgments while preserving production-ready output formatting. ## Measuring Agreement with Human Reviewers - The judge rates each query–document pair on a 1–5 relevance scale: - **5** means a perfect match. - **1** means no meaningful connection to the query or user intent. - Human annotators provide both: - A relevance score. - A short explanation for their judgment. - Dropbox evaluates the model with normalized mean squared error (NMSE): - It measures the squared difference between model and human ratings. - Scores are normalized to a 0–100 scale. - **0** represents perfect agreement; higher values indicate worse performance. - Invalid JSON or incorrectly structured responses are treated as fully incorrect because they cannot be consumed reliably by downstream systems. - The optimization objective is therefore twofold: - Minimize disagreement with human ratings. - Ensure consistently parseable, production-ready outputs. ## Moving from o3 to a Lower-Cost Model - The original judge used OpenAI’s o3 because it delivered strong agreement with human ratings. - Running o3 across orders of magnitude more query–document pairs was too expensive. - Dropbox selected **gpt-oss-120b**, an open-weight model offering a better cost-performance balance. - The carefully tuned o3 prompt did not transfer directly: - Relevance quality declined under the NMSE metric. - Manual prompt rewriting would have required extensive iteration and regression testing. ## DSPy and GEPA-Based Prompt Optimization - Dropbox defined the optimization problem using: - A fixed relevance-rating task. - Human-annotated examples. - NMSE as the evaluation metric. - DSPy’s **GEPA optimizer** iteratively improves prompts for a specific target model. - Instead of relying only on an aggregate score, GEPA analyzes individual disagreements and generates structured feedback. - Feedback combines: - The difference and direction between predicted and human ratings. - The human annotator’s explanation. - The model’s reasoning. - DSPy then uses a reflection loop: - Evaluate the current prompt. - Identify recurring failure modes. - Revise the prompt with generalizable rules. - Repeat the process against the human-alignment metric. - This approach can address systematic errors such as: - Overvaluing keyword overlap. - Undervaluing document recency. - Misinterpreting user intent. - The feedback explicitly discourages overfitting to individual examples and preserves core task constraints, including the 1–5 rating range. Dropbox’s experience suggests that relevance judges should be optimized systematically rather than tuned manually. Defining a clear human-alignment metric, including structural validity, allows DSPy to adapt prompts across models while reducing cost and limiting regressions.

database-designjsonopenaidspy+3

dropboxFeb 26, 2026

Using LLMs to amplify human labeling and improve Dash search relevance (opens in new tab)

Dropbox Dash improves AI answers through retrieval-augmented generation (RAG): enterprise search retrieves relevant company documents, and an LLM uses a small subset of them to generate grounded responses. Because ranking determines which documents reach the LLM, search relevance depends heavily on high-quality query–document labels. Dash combines a small set of human judgments with large-scale LLM-generated labels to produce training data efficiently while retaining human oversight. ## How Dash search ranking works - Dash uses a trained ranking model, such as XGBoost, rather than manually configured rules. - The model learns from query–document pairs labeled on a 1–5 relevance scale: - **5:** Closely matches the user’s intent. - **1:** Not useful enough to display. - Relevance depends on the query, user context, and timing; it is not an intrinsic property of a document. - Ranking quality is especially important because enterprises may have millions or billions of indexed documents, while only a small selection can be sent to the answer-generating LLM. ## Sources of relevance labels - Labels can come from: - User behavior, such as clicks or skipped results. - Human evaluators assigning relevance scores. - LLMs directly judging query–document relevance. - Behavioral signals are useful but often sparse, biased by existing rankings, and unevenly distributed, so they work best as a supplement. - Human evaluators can provide comprehensive judgments across result sets, but labeling is expensive, difficult to scale, and vulnerable to inconsistency. - Humans also cannot directly review sensitive or proprietary customer data in this process, and different content types—such as Slack messages, Jira tickets, and Salesforce records—require different contextual expertise. ## LLM-assisted relevance evaluation - LLMs can evaluate far larger candidate sets at lower cost and with greater consistency than human annotators. - They can operate across languages and analyze customer content within established compliance boundaries. - Their judgments still depend on the model’s quality and the clarity of the evaluation prompt. - LLM-generated labels therefore require calibration and validation before being used for model training. ## Combining human review with LLM scale - Dropbox first creates a relatively small, high-quality dataset using human evaluators and limited, non-sensitive internal data. - These human labels are used to tune LLM prompts and model parameters. - Once the LLM meets quality thresholds, it generates hundreds of thousands or millions of relevance labels. - This approach multiplies human labeling effort by roughly 100 times, enabling broader and more representative training data. - LLMs are used offline rather than directly at query time because production-time use would introduce excessive latency and context-window limitations. - The LLM acts as a teacher for smaller, faster ranking models that can serve searches at scale. ## Evaluation as the foundation - Dash follows an iterative process: measure performance, change the model or instructions, and measure again. - The article compares this to chess engines, where the quality of the evaluation function determines which possible moves are preserved or discarded. - The same principle applies to ranking: poor relevance judgments can cause useful search-result patterns to be eliminated, while accurate judgments guide the model toward better rankings. Dash’s approach uses humans for quality control and contextual grounding, then uses LLMs to expand that expertise into large-scale training data. This hybrid strategy offers a practical way to improve enterprise search relevance without exposing customer data to human reviewers or imposing LLM latency on every search.

llmmachine-learningragxgboost+3

dropboxFeb 12, 2026

How low-bit inference enables efficient AI (opens in new tab)

Low-bit inference reduces the memory, compute, and energy required to serve modern AI models by representing values with fewer bits. Quantization can substantially increase GPU throughput, but its benefits depend on model accuracy, hardware support, and whether workloads prioritize latency or throughput. The article presents low-bit inference as a production trade-off rather than a universally optimal technique. ## The Rising Cost of Modern Models - Models are growing rapidly, increasing demand for: - Memory capacity - Compute power - Energy - Low-latency serving infrastructure - Dropbox uses attention-based models for Dash and other capabilities involving: - Text, image, video, and audio understanding - Search and summarization - Reasoning over large collections of content - Production deployment requires balancing model capability with hardware utilization, cost, and responsiveness. ## Where Inference Compute Is Spent - Most computation comes from repeated matrix multiplications in two areas: - **Linear layers**, including attention projections, MLP layers, and final output layers. - **Attention mechanisms**, which calculate relationships between input tokens and become increasingly expensive with longer contexts. - GPUs accelerate these operations using specialized hardware: - NVIDIA Tensor Cores - AMD Matrix Cores - These cores execute matrix multiply-accumulate operations much faster than general-purpose CUDA cores. ## How Lower Precision Improves Efficiency - Quantization reduces the number of bits used to represent model values. - Converting values from 16-bit to 8-bit or 4-bit formats: - Reduces memory usage - Lowers memory-transfer costs - Can increase matrix-operation throughput - Reduces energy consumption - GPU throughput generally improves as precision decreases; halving precision can approximately double the number of operations performed per second in suitable workloads. - Eight-bit quantization maps values into 256 discrete levels. Formats below 8 bits typically require **bitpacking**, combining multiple values into types such as `uint8` or `int32` because 4-bit values are not normally stored as native hardware types. - Newer hardware, such as Blackwell GPUs with FP4 support, can provide major energy savings compared with higher-precision systems like the H100. ## Limits of Extremely Low-Bit Formats - Binary and ternary quantization restricts weights to two or three possible values, offering greater theoretical savings. - These formats are not well matched to today’s GPUs because they cannot fully use Tensor or Matrix Cores. - Specialized accelerators could make them more practical, but adoption remains limited by: - Weak ecosystem support - Hardware availability - Concerns about model quality - Practical gains therefore depend not only on bit width, but also on how well the format is supported by existing hardware and software. ## Quantization Formats and Deployment Trade-offs - Quantization is a family of techniques with different choices for: - Numerical representation - Scaling - Execution strategy - These choices affect: - Model accuracy - Inference speed - Memory consumption - Hardware utilization - Different workloads have different priorities: - Latency-sensitive applications need fast individual requests. - Throughput-oriented workloads prioritize processing large volumes efficiently. - Depending on the workload, inference may be limited by software overhead, memory bandwidth, or specialized GPU compute units. ## Pre-MXFP and MXFP Approaches - The article divides modern low-bit formats into two broad groups following the introduction of **MXFP microscaling**: - **Pre-MXFP formats** rely on software-managed scaling and explicit dequantization. - **MXFP formats** move scaling and related operations into Tensor Core hardware. - MXFP aims to standardize low-bit data types while making them more directly usable by modern GPUs. - The choice between these approaches depends on the hardware generation and the specific performance requirements of each production workload. Low-bit inference is most effective when quantization formats, model quality, and hardware capabilities are considered together. Teams should select formats based on the actual bottleneck—memory, bandwidth, latency, or compute—rather than assuming that the fewest possible bits will always deliver the best result.

llmgpu-accelerationmodel-optimizationquantization+3

dropboxFeb 11, 2026

Insights from our executive roundtable on AI and engineering productivity (opens in new tab)

Dropbox argues that AI improves engineering productivity only when tied to measurable business outcomes rather than adopted for its own sake. The company has expanded AI use across the software development lifecycle, while recognizing trade-offs involving quality, maintenance, and organizational change. Its executive roundtable concluded that leadership, formal AI competency, and stronger outcome measurement will be central to realizing AI’s potential. ## Dropbox’s AI Adoption Strategy - Dropbox made AI adoption a company-wide priority with leadership sponsorship, enabling teams to experiment more easily and reducing delays in approving new tools. - Engineers use AI across code review, documentation, debugging, testing, and other stages of development. - Because Dropbox operates a large, multilingual monorepo, it combines commercial tools such as Claude Code and Cursor with internally built systems. - One internal tool detects failed pull-request builds and uses Dropbox’s AI platform to suggest fixes. - Most developers now use at least one AI tool. - Dropbox tracks monthly pull-request throughput per engineer and has observed higher output among developers who use AI coding tools more actively. - The company also monitors engineer sentiment, reporting increased positive sentiment and reduced negative sentiment as adoption improves. ## Focus of the Executive Roundtable Leaders from multiple companies discussed engineering productivity and AI in rotating peer groups organized around three themes: - **Measuring impact** - Identifying ways to measure AI-driven productivity gains. - Connecting engineering improvements to broader business results. - **Leadership alignment** - Establishing how executives should communicate AI deployment progress. - Determining the appropriate pace and scope of adoption. - **The human element** - Recruiting, evaluating, and developing AI-capable employees. - Applying lessons from developer productivity to help non-engineering teams work more effectively. ## Lessons About AI and Productivity - **Balance is essential:** Faster development must not come at the expense of software quality or increased long-term maintenance costs. - **Leadership sets standards:** Technical managers play a key role in defining responsible and effective AI usage norms. - **AI skills should be formalized:** Including AI competency in career frameworks demonstrates that it is a lasting strategic capability rather than a temporary trend. - **Extra capacity needs direction:** Dropbox is currently using productivity gains to address technical debt, complete migrations, and improve reliability. ## Priorities for 2026 Dropbox’s main unresolved challenge is linking engineering productivity metrics to tangible business outcomes. Its next phase will focus on mapping AI-driven gains to specific results, extending operational discipline beyond engineering, and improving end-to-end product velocity.

aiclaude-codemonoreposoftware-development-lifecycle+4

dropboxJan 28, 2026

Engineering VP Josh Clemm on how we use knowledge graphs, MCP, and DSPy in Dash (opens in new tab)

Dropbox VP Josh Clemm argues that useful workplace AI requires a unified context engine capable of securely understanding and retrieving information across many SaaS applications. Dropbox Dash combines custom connectors, multimodal content processing, knowledge graphs, hybrid search, and personalized access-control-aware ranking to make that possible. Clemm favors indexed retrieval over purely federated approaches because preprocessing enables richer context, faster search, and company-wide access, though it requires substantial engineering and infrastructure. ## Building Dash’s Context Engine - Custom connectors crawl third-party applications while handling: - Rate limits - API differences - Application-specific permissions and ACLs - Incoming content is normalized, often into Markdown, and enriched with: - Titles and metadata - Extracted links - Embeddings - Other key information - Different media types require different processing: - Documents can be text-extracted and indexed. - Images may require CLIP or multimodal models. - PDFs combine text, figures, and other elements. - Audio is transcribed. - Videos may require scene-by-scene multimodal analysis when dialogue is insufficient. - Dash models relationships between content as a knowledge graph: - Meetings can connect to documents, participants, transcripts, and previous notes. - Cross-application relationships provide richer context for search and agents. - Data is stored in secure systems using: - BM25 lexical search - Dense-vector storage - Hybrid retrieval - Multiple ranking stages personalize results and enforce user-specific permissions. ## Indexed Retrieval Versus Federated Retrieval - Federated retrieval queries external systems at runtime. - Its advantages include: - Fast initial implementation - Minimal storage requirements - Relatively fresh data - Easy addition of MCP servers and connectors - Its drawbacks include: - Inconsistent API speed, quality, and ranking - Limited access to company-wide content - Expensive post-processing and reranking - Large token usage when agents reason over returned results - Indexed retrieval preprocesses content during ingestion. - Its advantages include: - Access to shared company connectors - Enriched datasets created offline - Faster queries - More opportunities for recall and ranking experiments - Its costs include: - Significant custom connector development - Freshness challenges - High hosting costs - Difficult storage and architecture choices involving vector search, BM25, hybrid retrieval, or graph RAG. ## Making MCP Practical at Scale - MCP can simplify tool integration, but tool definitions consume substantial context-window space. - Large tool descriptions and retrieval results contribute to context rot and reduce agent effectiveness. - Dash aims to limit context usage to roughly 100,000 tokens. - MCP-based agents can also be slow: simple queries may take up to 45 seconds, while direct index retrieval returns results within seconds. - Dropbox’s approach is to wrap its index in a consolidated “super tool,” reducing the need to expose many separate tools to the agent. ## Broader AI Engineering Practices - The talk also covers Dropbox’s use of: - LLMs as evaluators or judges - Prompt optimization with DSPy - Tool calling and MCP design - These techniques complement the underlying context engine rather than replacing the indexing, enrichment, graph modeling, and permission systems required for reliable workplace AI. A practical takeaway is that organizations building AI over proprietary data should treat retrieval as a full data-platform problem. Start with robust connectors and permissions, enrich content before retrieval, model relationships across sources, and use MCP selectively where indexed retrieval can provide faster and more controlled results.

mcpragmultimodal-aidspy+4

dropboxDec 18, 2025

Inside the feature store powering real-time AI in Dropbox Dash (opens in new tab)

Dropbox Dash’s ranking system depends on a hybrid feature store that can combine real-time user behavior with large-scale historical data. Because Dropbox operates across on-premises and cloud environments, and because each query can trigger thousands of feature lookups, off-the-shelf systems could not meet its latency, scale, and integration requirements. The resulting architecture uses Feast for orchestration, Spark for computation, Dynovault for low-latency storage, and a custom Go serving layer, achieving roughly 25–35 ms p95 latency while keeping features fresh. ## Goals and Requirements - Dash ranks documents, images, and conversations using behavioral, contextual, and real-time signals. - A single query can fan out into thousands of feature lookups across many candidate files. - The feature store needed to: - Support sub-100 ms search latency. - Reflect user actions within seconds or minutes. - Bridge Dropbox’s on-premises services and Spark-based cloud infrastructure. - Handle both streaming-style updates and batch computations. - Let engineers develop features without managing serving and orchestration details. ## Choosing a Hybrid Architecture - Dropbox evaluated Feast, Hopsworks, Featureform, Feathr, Databricks, and Tecton. - Feast was selected because: - It separates feature definitions from infrastructure concerns. - Engineers can focus on PySpark transformations. - Its modular adapter system supports existing Dropbox infrastructure. - Feast’s DynamoDB adapter enabled integration with Dynovault, Dropbox’s DynamoDB-compatible storage system. - The architecture combines: - Feast for orchestration and serving APIs. - Spark jobs for feature computation and ingestion. - Cloud storage for offline indexing and data management. - Dynovault for online, low-latency lookups. - A custom Go service replacing Feast’s Python online serving path. - Dynovault is colocated with inference workloads and provides approximately 20 ms client-side latency. - Monitoring covers job failures, feature freshness, and data lineage. ## Replacing Python with Go for Low Latency - The initial Feast-based Python service struggled under heavy concurrency. - CPU-bound JSON parsing and Python’s Global Interpreter Lock became bottlenecks. - A multi-process design helped temporarily but introduced coordination overhead. - The serving layer was rewritten in Go using: - Lightweight goroutines. - Shared memory. - Faster JSON parsing. - The Go service now handles thousands of requests per second. - It adds only about 5–10 ms beyond Dynovault latency and achieves roughly 25–35 ms p95 latency. ## Keeping Features Fresh - Fresh signals are essential for ranking quality; actions such as opening a document should influence subsequent searches quickly. - Fully real-time computation is impractical for features requiring large joins, aggregations, and historical context. - Dropbox therefore built a three-part ingestion strategy. - Batch ingestion handles complex, high-volume transformations using a medallion architecture. - Intelligent change detection updates only modified records rather than rewriting every feature. - This reduced online-store writes from hundreds of millions to fewer than one million per run and significantly shortened update time. ## Practical Takeaway The system demonstrates that a feature store does not need to be entirely off-the-shelf or entirely real-time. Combining a modular framework with custom serving, colocated storage, batch optimization, and freshness monitoring allowed Dropbox to meet demanding latency and scale requirements while keeping feature development manageable.

machine-learninggofeature-storedynamodb+4

dropboxNov 26, 2025

Building the future: highlights from Dropbox’s 2025 summer intern class (opens in new tab)

Dropbox’s 2025 intern program brought together 43 interns from 27 universities for 12 weeks of mentorship, technical work, and community-building. The 28 engineering interns contributed to systems spanning AI, search, storage, data infrastructure, and developer tools, with many projects supporting Dropbox Dash. Their work demonstrates how targeted refactoring, automation, and intelligent infrastructure can improve reliability, reduce costs, and expand product capabilities. ## A Diverse, Mentorship-Focused Internship Program - Interns received more than 6,000 hours of one-on-one mentorship. - The cohort included students from institutions in the United States, Canada, Poland, and Ireland. - Programming included Virtual First events, employee resource group activities, and an in-person Emerging Talent Summit. - Projects were aligned with Dropbox’s production systems and company goals rather than being isolated experiments. ## Infrastructure and Reliability Improvements - **Filesystem Data:** Rhea Rai redesigned Dropbox’s file history tracking system, emphasizing strongly tested code and simpler metadata infrastructure while reducing operational costs. - **Storage Core:** Albert Joon Sung reduced Magic Pocket write latency during disk restarts by adding storage-health caching and routing writes away from degraded volumes. - **Metrics:** Yonatan Ginsburg developed adaptive anomaly detection for Vortex2, accounting for changing patterns and seasonality to reduce alert fatigue and improve incident response. - **Analytics Platform:** Sanjith Udupa built recommendations for optimizing Databricks queries and ETL pipelines, and documented a plan to migrate a 500 TB mobile-events dataset to liquid clustering. ## AI, Search, and Dropbox Dash - **ML Platform:** Ben Juntilla created AI Sentinel, which gives engineers real-time visibility into machine-learning deployment health and improves confidence in model releases. - **Connector Platform:** Eddie Ormseth built tools that provide access to fresher Dash persistence data and additional third-party metadata without requiring connector teams to reprocess data. - **Retrieval Platform:** Rishi Peddakama expanded unified search to more than 20 languages by integrating language detection into indexing and retrieval. - **Find & Discover:** Francesca Venditti created in-context document previews for Dash, including PDF viewing and links to AI-powered follow-up chat. - **Conversational AI:** Alan Zhu developed a modular web-automation agent and connected it to Dropbox APIs for actions such as searching for and uploading files. ## Developer Automation - Ahmed Ibrahim built an AI-assisted code migration tool on Dropbox’s internal migration platform. - Developers can run migrations on selected folders, configure them through a CLI or automated workflow, and receive pull requests automatically when jobs succeed. - The tool enabled two major migrations during the internship and illustrates how automation can reduce repetitive engineering work. ## Broader Impact The interns’ projects improved system performance, operational visibility, multilingual accessibility, data freshness, and developer productivity. Together, they supported Dropbox’s move toward AI-first products while reinforcing the company’s emphasis on trustworthy, maintainable, and efficient engineering. For students interested in production-scale software, AI, and infrastructure, Dropbox presents its internship program and open roles as opportunities to contribute directly to products such as Dropbox Dash.

aimachine-learningdistributed-systemsanomaly-detection+4

dropboxNov 17, 2025

How Dash uses context engineering for smarter AI (opens in new tab)

Dash evolved from a traditional RAG search system into an agentic AI that can interpret information, plan tasks, and act on users’ behalf. Dropbox’s experience shows that better agent performance comes not from adding more tools and data, but from carefully engineering context: limiting choices, filtering for relevance, and delegating complex work to specialized agents. The central conclusion is that precise, timely context improves reasoning speed, accuracy, and efficiency. ## From Search to Agentic AI - Dash initially combined semantic and keyword search to retrieve documents and generate concise answers. - Users began asking it to interpret, summarize, and act on retrieved information. - This required Dash to plan and execute multi-step tasks rather than simply search and summarize. - The resulting challenge was determining which information and tools the model actually needed at each stage. ## The Cost of Too Many Tools - Every tool adds descriptions and parameters to the model’s context window. - More tools expand the model’s decision space, potentially causing slower or less reliable choices. - Tool definitions also consume tokens, increasing cost and reducing room for reasoning. - Longer-running tasks suffered from “context rot,” where accumulated tool-call information degraded accuracy. - Model Context Protocol (MCP) standardizes tool descriptions, but does not eliminate the problem of excessive context. ## Limiting Tool Definitions - Dash found that exposing retrieval tools from many services—such as Confluence, Google Docs, and Jira—created confusion. - Instead of requiring the model to choose among numerous APIs, Dash consolidated retrieval into one purpose-built tool backed by its universal search index. - A single retrieval interface: - Simplifies planning - Reduces tool-selection errors - Keeps the context window focused - Provides consistent access across connected services - The same principle shaped Dash’s MCP server, which exposes retrieval through one lean tool to applications such as Claude, Cursor, and Goose. ## Filtering Context for Relevance - Retrieved information is not automatically useful for the task at hand. - Dash combines data from multiple sources in a unified index and uses a knowledge graph to connect people, activity, and content. - These relationships help rank results according to the query and the user’s context. - By filtering results before presenting them to the model, Dash ensures that each piece of supplied context is relevant. - Precomputing the index and graph allows runtime retrieval to remain fast and focused. ## Using Specialized Agents for Complex Tasks - Some tools require substantial instructions and examples to use correctly. - Dash Search became complex because query construction involves: - Understanding user intent - Mapping intent to index fields - Rewriting queries for semantic matching - Handling typos, synonyms, and implicit context - Adding these instructions directly to the main planning agent consumed context that could otherwise support broader reasoning. - Dash therefore moved search into a specialized agent: - The main agent decides when searching is necessary. - The search agent independently constructs the query using its dedicated prompt. - This division lets the main agent focus on the overall task while the specialist handles search details. Dash’s approach recommends treating context as a limited engineering resource. Use a small number of well-designed tools, pre-filter information for relevance, and delegate technically demanding subtasks to specialized agents rather than overwhelming one general-purpose model.

model-context-protocolagentic-airagcontext-engineering+3