How AI Has Changed the Product Design Process
Toss redesigned its customer-service chatbot after finding that 60% of users abandoned the existing menu-based experience. Instead of forcing customers to identify the correct category, the team used AI to interpret natural-language requests and connect users directly to solutions. The project’s key insight was to prototype and validate experiences early, then improve the system through reusable rules rather than endlessly editing individual scenarios. ## Replacing Menu Navigation with Natural Language - The customer center serves roughly 600,000 visitors monthly, including about 170,000 chat users. - Users often know their problem—such as a failed payment or mistaken transfer—but not how Toss categorizes it. - The team therefore aimed to build a chatbot that understood intent and guided users directly to the appropriate resolution. ## Generating Scenario Drafts from Real 상담 Data - AI created initial scenarios for the 20 most common customer-service issues. - Personal information was removed and data was anonymized or pseudonymized before use. - Instead of relying primarily on policy documents, the team trained the AI on real consultation data. - This revealed: - The language customers actually use - The questions agents ask to narrow down causes - The order in which problems are diagnosed and resolved - Ways to explain complex policies in understandable language - As a result, the chatbot behaved more like a human 상담사 than a system that simply listed policies. ## Validating Many Situations with a Scenario Hub - A single request can hide dozens of possible conditions. For example, membership cancellation may depend on whether: - The monthly fee was already paid - Benefits were used - Cancellation was already scheduled - The team built a scenario hub where these conditions could be saved and selected. - Each selection immediately opened a prototype conversation with the chosen conditions applied. - This enabled rapid testing after scenario changes, new branches, or rule updates. - The prototype became an experimentation environment rather than a static mockup, allowing the team to judge realistic conversations instead of relying on assumptions. ## Replacing Individual Fixes with General Rules - Testing revealed recurring problems, including: - Asking for information the chatbot already knew - Repeating the same explanation - Guessing when it lacked reliable information - Rather than fixing each scenario separately, the team created general rules such as: - Resolve the customer’s problem before giving explanations - Admit uncertainty instead of guessing - Escalate to a human agent only under defined conditions - Clearly limit the AI’s authority - Improving one scenario helped only that scenario, while improving a rule improved many scenarios at once. ## Designing the Experience Before the System - The team first envisioned the desired customer experience, then worked backward to determine the required: - Data structures - APIs - Operational tools - System architecture - This reversed the conventional process of designing infrastructure first and fitting the user experience onto it. - Within approximately three weeks, the team completed analysis, experience design, scenario generation, prototyping, testing, and refinement. - AI changed not only productivity but also the order of product design: designers could validate more possibilities before committing to implementation. ## Practical Lessons - Build an imperfect prototype early instead of waiting for a complete design. - Study real user data before relying solely on guides or documentation. - When similar problems recur, identify a shared rule instead of repeatedly patching individual cases. The broader recommendation is to use AI as a fast experimentation tool. Its greatest value is not simply producing artifacts, but making it cheaper and faster to test which experiences and design principles actually work.
Read original(opens in new tab)