Creativity Games

alt text

Creativity games are a set of collaborative game based activities that help promote creativity in gameplay. These games are designed for children from 5-10 to be played collaboratively with a robot. Our goal is to use child-robot interaction to enhance creative thinking during play, by scaffolding creativity and demonstrating artificial creativity.

Droodle Game

The Droodle Creativity Game is inspired from the Droodle Creativity Task developed by Kahn et al., to measure creativity. Participants are presented with a Droodle, a simple abstract drawing that “comes into focus” (in a surprising way) with the addition of a clever title, and they are tasked with generating Droodle titles. The child and the robot take turns to come up with Droodle titles. The robot responds to children’s titles, such as ‘I find that funny’, or ‘Oh is that it?’. In addition to these verbal interactions, the robot also shows multiple expressions of thought and curiosity, and expresses surprise and excitement upon coming up with creative titles.

alt text

JiboDraw (Pictionary)

JiboDraw is a collaborative drawing application, where children can play a drawing-guessing game with Jibo, and learn to draw letters, numbers, and drawings with Jibo. JiboDraw has two modes - Learn and Play. The learn mode involves Jibo using drawing guides to help children learn how to draw. The play mode involves the children drawing objects that Jibo can guess.

alt text

MagicDraw (Predictive Drawing):

MagicDraw is a generative drawing game where the player is given a starting drawing prompt (example, a circle), and the other player draws on top of it to convert it to a new object. The robot and the child take turns to provide a starting prompt, while the other player draws an image on top of it.

alt text

Storytelling and Reading

alt text

Our robots like to tell and read stories with children. When Tega and Jibo are listening to children’s stories, they can understand if the child learned a new word or a new sentence. When they are telling stories, they are curious to know if the child is engaged and having fun, so they will closely monitor the child’s emotions and attention. Tega and Jibo also asks questions to the child from time to time, to help children think and learn about what the story means.

Our robots can decide what storybook would help each child learn and practice new words and sentences. Every child learns and engages differently, so we want our robots to understand that, and help each child learn at their level.

Here is the storybook on a tablet. Our robots don’t have hands to touch the screen like we do, but they can use their brain power to tell tablets what to do, like flipping pages! The secret is that the robot and the tablet can talk to each other over Wifi.

alt text

Let’s look at the storybook again. A child can click on the words and pictures in the storybook and it will tell them what the word is. Sometimes, Jibo and Tega might ask, “what are the names of those goats?” and can tell them “Billy Goats Gruff” or touch the words on the screen and the robot understands. A child can also ask the robots, “Hey Jibo, how do you read this word?” and click on the word, and Jibo will help them read it, “Let me try, I think that word is grass!”.

alt text

Here is a video of Tega and children telling stories to each other. These robots get smarter and smarter every time they tell and listen to stories!

Selected Publications:

Hae Won Park, Ishaan Grover, Samuel Spaulding, Louis Gomez, and Cynthia Breazeal. “A Model-free Affective Reinforcement Learning Approach to Personalization of a Social Robot Companion for Early Literacy Education.” In Association for the Advancement of Artificial Intelligence (AAAI), 2019. [Download]

Hae Won Park, Mirko Gelsomini, Jin Joo Lee, and Cynthia Breazeal. “Telling stories to robots: The effect of backchanneling on a child’s storytelling.”, In ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2017. [Download]

Word Quest: An interactive robot game for vocabulary learning

alt text

The WordQuest game is being designed to help develop a child’s vocabulary. It is a collaborative game, with a robot, similar to the classic game “I Spy”. In it a child and a robot take turns. The tablet may show a picture of an object from the scene and ask the player to say what it is, or it may say a word (and shows the written word) and then they take turns identifying the picture of that word on the touchscreen tablet.

A child and robot can also explore the scene by looking around it zooming in and out, and clicking on the various objects, and the tablet will tell them what the object is (and provide the written word for that object). The current version of the game has two scenes, “outdoor” and “indoor”, that in total contain 50 animated, clickable objects for children to learn. Each game quest (e.g., “can you find objects that are in crimson?”) contains one target vocabulary word (e.g., “crimson”) for the child to learn

The game is designed to help the child learn its meaning by taking turns with the robot and finding the objects in the scene that represent the target word’s meaning. When four correct objects are collected either by the child or the robot, the quest is completed.

Additional Technical information: The game is implemented in the Unity 3D game engine, which supports flexible updates and changes of learning curriculum. During the game play, the robot may switch its role between “learner” and “tutor” roles to best promote a child’s learning, based on its assessment on their learning progress. The computational model used to guide the robot’s action is built using Reinforcement Learning.

alt text

Selected Publications:

Chen, H., Park, H.W., Dong, X., Breazeal, C. (in review). Active Role Adaptation for Social Robot Learning Companion. In proceedings of 2019 IEEE International Conference on Robotics and Automation (ICRA 2019)

Rhyme Racer: An interactive robot game for practicing phonological awareness

alt text

The Rhyme Racer game is designed to help children develop rhyming, phonological awareness, and pronunciation skills. It is a fast-paced, fun, and competitive game with a robot, in which children race the robot to identify and say a word that rhymes with the ‘target’ word in each round. Each round, the game uses the information from previous rounds to learn about a child’s individual rhyming and pronunciation skills and adjusts the content of the next round of words to maximize the child’s learning.

Additional Technical information: It is difficult to program robots and games for interactive social behavior during educational co-play. Moreover, an effective program must balance multiple objectives across different time scales, such as teaching new words, acquiring information about the student to model their knowledge, and keeping the interaction fun and engaging. This project aims to address these challenges for educational co-play by developing learning algorithms, developing a computational framework formulating educational co-play as a planning problem, and using children’s responses to model student knowledge.

Selected Publications:

A Social Robot System for Modeling Children’s Word Pronunciation AAMAS 2018

Learning Behavior Policies for Interactive Educational Play Models, Robotics Science and Systems: Workshop on Algorithms, and HRI