gesture-recognition

1 posts

figma

What Does the Future of Software Look Like? | Figma Blog (opens in new tab)

AI may reshape software around more human, contextual interactions rather than fixed menus and mechanical commands. The post argues that future interfaces could understand intent through voice, gesture, emotion, and situation, adapting their behavior to each person. Instead of forcing users to adapt to increasingly powerful systems, software could meet users where they are while encouraging focus, presence, and healthier technology habits. ## Ephemeral Tools - Controls appear only when users select an object and indicate what they want to do. - Contextual options replace persistent menus, panels, and modes. - A video editor, for example, might show timing, pacing, alternate cuts, and sound options around a selected clip. - This lets creators focus on decisions and intent rather than remembering how software is organized. ## Magic Marker - Users interact through a combination of voice, cursor movement, gestures, sound effects, and body language. - Someone could circle an object, drag it into position, and verbally request a change. - AI would interpret these signals together, making it feel more like collaborating with a teammate. - This reduces the need for precise prompt engineering or complex document references. ## Adaptive Presence - Intelligent systems adjust their communication style and level of assistance based on user behavior. - They might offer structured guidance when someone is confused, step back when help is unnecessary, or switch between text, voice, and visuals. - Software could change pacing, simplify language, and divide information into smaller steps. - This approach is especially valuable in healthcare and education, where differences in user readiness can have serious consequences. ## Empathetic Flows - Interfaces could infer emotional states from typing speed, stylus pressure, speech patterns, facial expressions, and repeated revisions. - A food app might reduce choices when someone appears overwhelmed. - A creative tool could become quiet when the user is concentrating, while a hotel app might stop promoting upgrades when the guest seems tired. - Rather than requiring users to explicitly state what they need, systems would respond to behavioral signals. ## Situational Cues - Sound, motion, pacing, progress indicators, and visual transitions can help users understand where they are in an experience. - Earlier digital products used cues such as dial-up sounds, progress bars, and “You’ve got mail” announcements to provide orientation. - Future interfaces should counteract the overstimulation caused by attention-driven notifications. - Persistent progress indicators, transition sounds, and consistent visual language could help users regulate their attention and nervous systems. ## Spatial Tuning - Users could control software through bodily movement instead of conventional tapping and clicking. - Examples include shaping music with hand movements, navigating augmented reality by changing body orientation, or adjusting design elements through gestures. - These interactions demand attention and presence, making them harder to rush or automate. - Technology becomes an experience that intentionally slows users down rather than continually rewarding speed. ## Mash-Ups - Future systems could combine any two inputs—files, objects, sounds, locations, or physical gestures—to create something new. - The system would synthesize the combined inputs while blending their structure, tone, and meaning. - Possible examples include merging a playlist with a city map or combining digital objects through touch or gestures. The overall recommendation is to design AI-powered software around human intent, context, emotion, and physical presence. The most successful future interfaces may be those that make technology feel less like a collection of controls and more like an adaptable, considerate collaborator.