Improbotics: Bringing machine intelligence into improvised theatre

**Improbotics: bringing machine intelligence into improvised theatre
This picture was taken on 19 August 2018 during a performance of "HumanMachine: Artificial Intelligence Improvisation" at the Edmonton Fringe Festival, and features, from left to right: Kory Mathewson, Nikki Hulowski, Riley Dawson and Piotr Mirowski (operating the AI that generates Nikki's lines). Credit: Alessia Pannese

An unconventional company called HumanMachine has recently devised a unique theatre production called Improbotics, which involves both human performers and machines. This is the latest of a series of projects and initiatives that merge the world of theatre with machine learning and robotics.

As part of a study published on AAAI AIIDE last year, Piotr Mirowski and Kory Mathewson, the two researchers behind the project, developed a deep-learning based artificial "improviser," training it on movie subtitles. This peculiar machine can generate plausible, context-relevant, lines of dialogue that are suitable for theatre.

"In our previous research, we incorporated an artificial intelligence-based chatbot and robot as a (rather difficult) stage partner for improvisational (improv) comedy," Piotr Mirowski, one of the researchers who carried out the study told TechXplore. "We combined humans and machines on the theatrical stage, and the robot was an embodiment for the chatbot."

This experiment highlighted two key limitations of performing improvisational comedy with a robot partner. First, the robot often took a while to generate its lines, resulting in a lack of proper timing, which is the essence of comedy. Second, they also observed difficulties in emotionally interpreting the text.

In their new study, therefore, the researchers replaced the robot with a human performer, who receives the lines via earphones from the chatbot but adds his/her own interpretation to them. Meanwhile, other performers on stage improvise freely and interact with the AI-prompted actor. The researchers set this performance up as a Turing test, or imitation game, taking place in a theatre in which the audience and performers need to guess who is human and who is being prompted by a machine.

"The main objective of this work is to explore how seamless or challenging collaborating with a machine-assisted stage partner is for live performance," Mirowski said. "For the audience's enjoyment, we made the improv show into a guessing game. We hide the identity of the AI-prompted performer and we also add a decoy: another improviser who is controlled but who actually receives lines from a human, backstage. Other improvisers are also wearing similar prompting devices to hide their identity from the audience. At the end of the performance, the audience gets to guess who was who."

The chatbot developed by the researchers is powered by a neural network, which resembles most state-of-the-art algorithms currently used for automated speech recognition and language translation. Trained on movie subtitles from, this chatbot processes an input line of dialogue and outputs potential responses.

"The output responses are generated word by word using a language model, which assigns probability to possible next words," Kory Mathewson, the other researcher who carried out the project, told TechXplore. "So at each step, it samples the next word (like rolling a 50,000 sided weighted die, where 50,000 is the size of the chatbot's vocabulary) over and over until it rolls an END to the sentence. The probabilities, or weighting, for the sides of the word dice are learned from the subtitles of over 100,000 movies."

Initial evaluations of this unusual theatre production found that greater rehearsal increased proficiency and made it easier to control events that happened during the performance. The researchers also observed that the interface and mechanisms used to perform the show resulted in a lower consistency with real-world experiences.

"There are several interesting findings from the study," Mirowski said. "The first is that they system is initially very challenging to improvise with, even though the human is adding their interpretation. The audience survey results also indicated that they believed that the performers had more autonomy than what the performers believed."

Carrying out a performance that combines human- and machine-generated lines also allowed Mirowski and Mathewson to compare the quality of the resulting dialogues. Overall, they found that human-generated lines were shorter, more positive, and used less difficult words than the lines produced by the artificial improviser. Human performers also made more grammar and spelling mistakes.

"We were surprised at how our human subjects were able to successfully pretend to be controlled by an ," Mathewson said. "They were able to modify their speech to mimic the expected speech from the AI-system."

The researchers are now exploring ways to improve the quality of the artificial improviser's lines, by incorporating feedback from human performers. This could be achieved using machine learning and natural language processing (NLP) algorithms.

"The performers who curate the suggestions from the chatbot need to make quick decisions during the performance," Mirowski explained. "The better the set of suggestions they are selecting from, the less time they will need to spend curating and the smoother our show will be."

Improbotics is a fascinating example of how machine learning could be employed not only in industrial or scientific fields, but also to create innovative art and entertainment. While several aspects of this performance still need to be perfected, the believe that AI could add an interesting layer to theatre improvisation.

"We are also hoping to extend the research by focusing on underlying narrative arcs, plot direction, and structure generation," Mathewson said. "By incorporating additional details about the higher level components of the show, we believe the human-machine collaboration can be more entertaining for the audience, and more enjoyable for the human improvisers."

More information: Improbotics: Exploring the imitation game using machine intelligence in improvised theatre. arXiv:1809.01807 [cs.AI].

Improvised theatre alongside artificial intelligence. … E17/paper/view/15825

© 2018 Tech Xplore

Citation: Improbotics: Bringing machine intelligence into improvised theatre (2018, September 20) retrieved 19 April 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Using multi-task learning for low-latency speech translation


Feedback to editors