PlotThread: Creating Expressive Storyline Visualizations using Reinforcement Learning

Storyline visualizations may be used to present fictions, meeting content or software evolutions. However, it is difficult to design such visualizations and current software has only limited design options and layout flexibility.

A recent study suggests using reinforcement learning to create a tool which facilitates the easy creation of storyline visualizations. The agent is trained to learn how designers typically make storyline layout clear. Instead of pure automation, this tool seamlessly integrates the work of computational agents and people on a shared problem.

Image credit: Tan Tang et al. / arXiv:2009.00249

During the interviews with design experts, it was stated that the tool based on artificial intelligence balances the aesthetic goals and the narrative constraint more successfully than systems based only on the optimization.  Moreover, a researcher on visual analytics noticed that storylines created with the novel tool can arouse the emotion of viewers.

Storyline visualizations are an effective means to present the evolution of plots and reveal the scenic interactions among characters. However, the design of storyline visualizations is a difficult task as users need to balance between aesthetic goals and narrative constraints. Despite that the optimization-based methods have been improved significantly in terms of producing aesthetic and legible layouts, the existing (semi-) automatic methods are still limited regarding 1) efficient exploration of the storyline design space and 2) flexible customization of storyline layouts. In this work, we propose a reinforcement learning framework to train an AI agent that assists users in exploring the design space efficiently and generating well-optimized storylines. Based on the framework, we introduce PlotThread, an authoring tool that integrates a set of flexible interactions to support easy customization of storyline visualizations. To seamlessly integrate the AI agent into the authoring process, we employ a mixed-initiative approach where both the agent and designers work on the same canvas to boost the collaborative design of storylines. We evaluate the reinforcement learning model through qualitative and quantitative experiments and demonstrate the usage of PlotThread using a collection of use cases.