Distributed Systems | Techlist.io

figmaJul 20, 20202 min readCurated summary

A deep dive on deep search | Figma Blog

Figma’s deep search lets users find files by searching text inside them rather than relying on file names or metadata. Building it required extending infrastructure originally created for Design System Analytics to process `.fig` files stored in Amazon S3. Because analyzing large file trees is expensive, Figma accepted briefly stale results and processed deduplicated changes hourly. ## Deep Search in a Browser-Based Product - Figma’s web-based architecture provides detailed access to files and usage data. - This enables features such as: - Component usage analytics - File-view frequency - Inspection of file structure - Searching content inside files - Deep search builds on the browser’s collaboration and discoverability advantages. ## Reusing Design System Analytics Infrastructure - Design System Analytics already opened recently edited files, retrieved them from storage, and traversed their contents. - Analytics extracted shared-library usage information. - Deep search applies the same general workflow to extract text from Figma files. - The existing file-analyzer worker platform provided support for computationally intensive, periodic processing. ## Regular Search vs. Deep Search - Regular search indexes database metadata, including: - File name - Creator - Folder ID - Team ID - Its pipeline: - Database changes are streamed into a messaging system. - Search indexers retrieve current records. - The metadata is indexed in Elasticsearch. - Deep search cannot rely on database metadata because the actual file contents are stored as `.fig` documents in Amazon S3. - A `.fig` file is represented as a tree of nodes, such as frames, rectangles, vectors, ellipses, and text objects, each with its own properties. ## Managing the Cost of File Analysis - Retrieving and traversing a complete Figma file is significantly more expensive than reading database records. - Files may contain thousands of nodes, and users can trigger saves approximately every 30 seconds. - Re-indexing every save would produce substantial duplicated computation. - Figma therefore: - Deduplicates file changes over one-hour windows. - Sends changed files to file-analyzer workers. - Allows deep-search results to be temporarily stale. - This tradeoff reduces server workload while maintaining useful search functionality. Deep search demonstrates how content-aware features require different infrastructure from conventional metadata search. Periodic, deduplicated processing offers a practical balance between timely results and the high computational cost of analyzing complete design files.

Read original

Distributed Systems Amazon S3 Data Pipeline Elasticsearch+2

figmaNov 20, 20193 min readCurated summary

Figma’s infrastructure: What goes into powering a web-based design tool | Figma Blog

Figma’s infrastructure team is focused on making a web-based design tool feel as fast and reliable as a desktop application. Its cloud-based, collaborative model creates demanding networking, storage, and scaling challenges, especially as organizations grow. The company is therefore evolving from a simple but operationally intensive backend toward a more mature, horizontally scalable architecture. ## Figma’s Collaborative Design Model - Figma stores designs in the cloud, giving each file a unique URL and making it a shared source of truth. - Users can collaborate, comment, prototype, and support developer handoff without exporting files or managing separate versions. - Its multiplayer functionality allows multiple people to view and edit files simultaneously. - Files may contain complex shapes and large images, creating significant data-transfer demands. - Users accustomed to desktop design tools expect fast interaction despite Figma running in a browser. ## Limits of the Original Infrastructure - Figma’s early backend favored simple strategies that worked well for smaller teams. - The application initially loaded all shared components available to a user when opening a file. - This approach became increasingly inefficient as organizations accumulated nearly 10,000 shared design elements. - Growth among large customers and the broader user base exposed the limits of the original design. ## Reducing Interaction Latency - Figma still preloads more data than users immediately need, increasing backend load and slowing startup. - Fixing this requires more than backend optimization: the client-server interaction model must be redesigned. - The client should request information incrementally, only when it is needed. - This change also requires coordination with product teams because existing user experiences depend on preloaded data. ## Building a Horizontally Scalable Database - At the time of the article, Figma relied on a single powerful AWS database instance. - The simple architecture reflected the company’s preference for the KISS principle and had supported substantial growth. - As usage increased, the database approached its capacity limits. - Figma planned to replace it with a database layer capable of scaling horizontally, a difficult transition because nearly every system depends on the database. ## Improving International Performance - More than 80% of Figma’s weekly active users were located outside the United States. - Their latency was affected by the round-trip distance to Figma’s Oregon datacenter. - Figma planned to move selected infrastructure components closer to users around the world. - The initial step was to deploy strategically placed remote proxies globally. Figma’s broader infrastructure strategy is to preserve the simplicity that helped it move quickly while introducing selective caching, better data loading, geographic distribution, and horizontal scaling where growth demands it.

Read original

Distributed Systems AWS Real Time Collaboration Performance Optimization+2

figmaApr 11, 20173 min readCurated summary

An alternative approach to rate limiting | Figma Blog

Figma built a Redis-backed rate limiter to protect its web application from excessive traffic and spam. The system needed to work across multiple servers, add minimal latency, remove stale data efficiently, remain accurate, and use little memory. Common algorithms each met some of these goals, but introduced trade-offs involving atomicity, burst behavior, or memory consumption. ## Requirements and Redis - Rate limits cap requests from a user or IP within a time period. - Figma needed shared state because its application ran on multiple machines. - Redis was preferred over PostgreSQL because it provides: - Faster in-memory reads and writes - Built-in expiration for stale tracking data - Efficient storage for rate-limit state ## Token Bucket - Stores each user’s last-request timestamp and remaining token count in a Redis hash. - Tokens refill over time; a request is rejected when no tokens remain. - It is memory-efficient and conceptually elegant. - Its read-then-write operations are not atomic: - Two servers can read the same remaining token count. - Both may accept a request even though only one token was available. - Redis locks could prevent this race but would slow concurrent requests and add complexity. - Lua scripting could make the operations atomic, but Figma avoided introducing that complexity. ## Fixed Window Counters - Stores a request count for each user and fixed time interval, such as one Redis key per minute. - Each request atomically increments its interval’s counter. - Keys expire after the interval, preventing stale data from accumulating. - The approach is simple, memory-efficient, and avoids the token bucket’s race condition. - Its major flaw is boundary bursts: - With a five-request-per-minute limit, a user could send five requests at the end of one minute and five more immediately afterward. - This allows up to twice the intended traffic over a short sliding period. ## Sliding Window Logs - A sliding window log records the timestamps of individual requests. - Older timestamps can be removed as the window advances, and the remaining entries provide an accurate count. - This avoids the boundary problem of fixed windows. - The trade-off is memory usage: users making many requests require many timestamps to be stored. ## The Central Trade-off - Token buckets use little memory but require careful handling of distributed atomicity. - Fixed counters are atomic and efficient but can permit bursts at window boundaries. - Sliding logs are accurate but consume more memory. - Figma’s rate limiter was designed around balancing these competing concerns rather than choosing the theoretically simplest algorithm. The practical lesson is to select a rate-limiting strategy based on the required accuracy, concurrency model, storage system, and memory budget. Redis is a strong fit for distributed rate limiting, but the algorithm must account for both race conditions and burst behavior.

Read original

Distributed Systems PostgreSQL Redis Rate Limiting+2

figmaMar 5, 20173 min readCurated summary

Realtime Editing of Ordered Sequences | Figma Blog

Figma needed a way for multiple users to edit ordered object sequences simultaneously while ensuring every client eventually reached the same state. Although Operational Transformation (OT) could solve the problem, Figma chose fractional indexing because it is simpler, supports efficient reordering, and was sufficient for design documents. The trade-offs—such as possible interleaving and growing index lengths—were acceptable in Figma’s use case. ## The Realtime Ordering Problem - Figma documents contain ordered children inside groups, components, and other compound objects. - Users can insert, delete, or reorder objects while edits are applied locally and propagated asynchronously. - Because clients may receive operations in different orders, the system must guarantee eventual consistency: every client must end up with the same document. ## Operational Transformation - OT transforms concurrent operations so they preserve the intended result regardless of application order. - For example, an insertion before a deletion may require adjusting the deletion’s index so it still removes the intended characters. - OT offers: - Strong performance and low memory usage for very large sequences. - Linearized concurrent insertions rather than interleaved content. - However: - It is difficult to understand and implement correctly. - Reordering is typically represented as a delete followed by an insert. - Supporting more operation types increases implementation complexity substantially because operations must be transformed against one another. - Figma considered OT excessive because its sequences were not enormous, interleaving was acceptable, and reordering was especially common. ## Fractional Indexing - Each object receives a numeric position, and children are ordered by sorting these positions. - To insert between two objects, Figma assigns the new object the average of their positions. - Positions are arbitrary-precision fractions between 0 and 1, stored as strings to preserve precision. - Figma uses a compact base-95 representation, omitting the leading `0.` and using the full ASCII range. - Reordering requires changing only one position value. ## Trade-offs and Conflict Handling - Fractional indexes are easy to understand and implement, but: - Index strings can grow after many insertions. - Concurrent insertions may interleave. - Averaging fails if two neighboring objects have identical indexes. - Index growth is not a practical concern for Figma because document sizes and user-driven reorder operations are limited. - Interleaving is generally acceptable for design objects, which often do not overlap; users can manually correct unusual ordering. - If two clients insert between the same objects, the server assigns a unique position to prevent duplicate indexes. Figma’s experience suggests that a simpler, stable algorithm can be more valuable than a theoretically stronger one. Fractional indexing made collaborative ordering easier to maintain and extend while meeting the practical needs of a design tool.

Read original

Distributed Systems Operational Transformation Eventual Consistency Fractional Indexing+3