The Infrastructure Behind AI Search in Figma | Figma Blog

Figma’s AI search lets users find designs and components through text, screenshots, or layer selections. It relies on multimodal embeddings, vector nearest-neighbor search, and large-scale indexing of frames and components. The main infrastructure challenge was generating and indexing billions of embeddings efficiently while controlling costs.

AI-Powered Search Flows

• Search for designs indexes frames across users’ files, including unlabeled frames buried in complex files.
• Users can search designs:
  • Lexically with a text description
  • Visually with a screenshot
  • By selecting similar Figma layers
• Search for components enhances the Assets panel with semantic matching.
  • A component representing 😀 can be found with terms such as “smiley,” “happy,” “face,” or “grin.”
  • Designers no longer need to manually add every possible keyword to component descriptions.
  • Components can also be found using visual queries.

Multimodal Embeddings

• An embedding model converts text or images into numerical vectors that represent their meaning.
• Figma uses the open-source CLIP model, which places text and images in the same embedding space.
  • The embedding for the word “cat” should be numerically close to an embedding generated from an image of a cat.
• Figma’s models were not trained on private customer files or data.
  • Fine-tuning used interface images from public, free Community files.
• Search works by:
  • Generating embeddings for indexed content
  • Creating an embedding for the user’s query
  • Finding indexed vectors that are nearest to the query vector
• Figma tested embeddings based on textual representations such as JSON, but image-based embeddings produced better results and supported the same workflow as screenshot search.
• Layer selections are converted into screenshots before being passed to the embedding model.

Populating the Vector Search Index

• Each searchable item requires:
  • A thumbnail or rendered screenshot
  • An embedding
  • Metadata stored in the search index
• Figma uses DynamoDB for metadata and embeddings because the workload primarily requires high-throughput key-value reads and writes.
• Identifying searchable frames is difficult because unpublished frames are not directly enumerable.
• Figma runs a headless, server-side version of its C++ editor in asynchronous jobs to discover frames within files.
• These jobs use server-side sandboxing techniques to safely run the editor.

Figma’s approach combines CLIP-based multimodal representations, server-side rendering, asynchronous processing, and scalable vector storage to make visual and semantic search practical across large design systems.