Beyond one-on-one: Authoring, simulating, and testing dynamic human-AI group conversations

DialogLab is an open-source research prototype for designing, simulating, and evaluating dynamic human–AI group conversations. It addresses the tension between rigid scripts and unpredictable generative dialogue by combining structured conversational phases with real-time improvisation. Its evaluation with 14 participants suggests that human-guided simulation offers the strongest balance of realism, engagement, and control.

A Framework for Multi-Party Conversations

• DialogLab separates a conversation’s social structure from its progression over time.
• Group dynamics define:
  • Groups, such as a conference or social event
  • Parties, such as presenters and audiences
  • Elements, including human or AI participants and shared content
• Conversation-flow dynamics define:
  • Snippets, or distinct phases such as opening, debate, and consensus
  • Participants and turn sequences within each snippet
  • Interaction styles, including collaborative or argumentative modes
  • Rules for interruptions and backchanneling
• This separation makes complex conversation designs modular and easier to revise.

The Author–Test–Verify Workflow

Authoring with Visual Tools

• Designers use a drag-and-drop canvas to arrange avatars and shared content.
• Inspector panels configure personas, roles, interaction patterns, and snippet behavior.
• Automatically generated prompts can be customized for specific narrative or conversational goals.

Human-in-the-Loop Simulation

• A live preview displays the evolving transcript.
• In human-control mode, an audit panel suggests possible AI responses.
• Designers can edit, accept, or reject suggestions, retaining control over the agents’ contributions.
• The system supports both structured interactions and more improvisational conversations.

Verification and Analytics

• A verification dashboard provides post-hoc analysis of the conversation.
• Visualizations show turn-taking distributions and sentiment flows.
• These tools help creators diagnose interaction patterns without manually reviewing entire transcripts.

Prototype Evaluation

• Fourteen participants from game design, education, and social science research evaluated DialogLab.
• They designed an academic social event and tested AI group discussions under three conditions:
  • Human control: Users prompted agents to shift topics, introduce perspectives, ask probing questions, or generate emotional responses.
  • Autonomous: Agents participated proactively according to predefined random or sequential orders.
  • Reactive: A simulated human agent responded only when directly addressed.
• Human control was rated significantly more engaging and was generally considered more effective and realistic.
• Participants also described the interface as intuitive, flexible, and enjoyable.
• Users valued the combination of automated prompt generation, detailed customization, and support for different moderation strategies.

DialogLab demonstrates that effective multi-party conversational design benefits from combining explicit structure with controlled improvisation. For developers and researchers building group-based human–AI experiences, a visual authoring workflow paired with human-guided simulation and analytics can provide a practical foundation for rapid iteration and more realistic interactions.