Beyond one-on-one: Authoring, simulating, and testing dynamic human-AI group conversations
DialogLab is an open-source research prototype for designing, simulating, and evaluating dynamic human–AI group conversations. It addresses the tension between rigid scripts and unpredictable generative dialogue by combining structured conversational phases with real-time improvisation. Its evaluation with 14 participants suggests that human-guided simulation offers the strongest balance of realism, engagement, and control. ## A Framework for Multi-Party Conversations - DialogLab separates a conversation’s social structure from its progression over time. - **Group dynamics** define: - Groups, such as a conference or social event - Parties, such as presenters and audiences - Elements, including human or AI participants and shared content - **Conversation-flow dynamics** define: - Snippets, or distinct phases such as opening, debate, and consensus - Participants and turn sequences within each snippet - Interaction styles, including collaborative or argumentative modes - Rules for interruptions and backchanneling - This separation makes complex conversation designs modular and easier to revise. ## The Author–Test–Verify Workflow ### Authoring with Visual Tools - Designers use a drag-and-drop canvas to arrange avatars and shared content. - Inspector panels configure personas, roles, interaction patterns, and snippet behavior. - Automatically generated prompts can be customized for specific narrative or conversational goals. ### Human-in-the-Loop Simulation - A live preview displays the evolving transcript. - In human-control mode, an audit panel suggests possible AI responses. - Designers can edit, accept, or reject suggestions, retaining control over the agents’ contributions. - The system supports both structured interactions and more improvisational conversations. ### Verification and Analytics - A verification dashboard provides post-hoc analysis of the conversation. - Visualizations show turn-taking distributions and sentiment flows. - These tools help creators diagnose interaction patterns without manually reviewing entire transcripts. ## Prototype Evaluation - Fourteen participants from game design, education, and social science research evaluated DialogLab. - They designed an academic social event and tested AI group discussions under three conditions: - **Human control:** Users prompted agents to shift topics, introduce perspectives, ask probing questions, or generate emotional responses. - **Autonomous:** Agents participated proactively according to predefined random or sequential orders. - **Reactive:** A simulated human agent responded only when directly addressed. - Human control was rated significantly more engaging and was generally considered more effective and realistic. - Participants also described the interface as intuitive, flexible, and enjoyable. - Users valued the combination of automated prompt generation, detailed customization, and support for different moderation strategies. DialogLab demonstrates that effective multi-party conversational design benefits from combining explicit structure with controlled improvisation. For developers and researchers building group-based human–AI experiences, a visual authoring workflow paired with human-guided simulation and analytics can provide a practical foundation for rapid iteration and more realistic interactions.
Read original(opens in new tab)