Mei Si
Assistant Professor

Cognitive Science Department
Games and Simulation Arts and Sciences (GSAS) Program 
Rensselaer Polytechnic Institute

Carnegie 311
110 8th Street
Troy, NY 12180






I joined RPI at 2010 as an assistant professor in the cognitive science department. I am also part of the Games and Simulation Arts and Sciences (GSAS) Program here. Before coming to RPI, I completed my Ph.D at the computer science department and the Institute for Creative Technologies at the University of Southern California. My adviser was Stacy Marsella.

My main research interest is artificial intelligence and its application in virtual and mixed realities for games, simulations and health interventions. I am interested in using AI technologies to make these environments more accurate, impressive and effective. The specific topics of my research interests are listed below.


Research Interests


Interactive Narratives / Serious Games


Embodied Conversational Agents (ECA)


Emotion Detection and Modeling


Multi-Agent Systems


Human-Computer Interaction

  Game AI

Embodied Cognition

Cognitive Robotics


Recent Conference Organizations


General Chair for the 6th Workshop on Intelligent Narrative Technologies (INT6), 2013


General Chair for the 4th International Conference on Interactive Digital Storytelling (ICIDS), 2011


General Chair for the 6th GAMEON-NA conference, 2011


Current Research Projects

Thespian: A Decision-Theoretic Framework for Authoring Interactive Narratives

Interactive narrative allows people to participate actively in a dynamically unfolding story, by playing a character or by exerting directorial control. The support of user interactivity allows the designer to tailor the experience for different users, which makes interactive narratives particularly suitable for creating user-centered pedagogical and entertainment effects. On the other hand, the support of user interactivity brings tremendous difficulty to the design process, because it results in a huge amount of paths through the story.

Thespian is a multi-agent framework for authoring and simulating interactive narratives. I explored how agent-based techniques and a restricted form of machine learning could be exploited to address the design challenge. In Thespian, each character in the story is modeled as a decision-theoretic goal-based agent, and a director agent is used for coordinating the agents for reaching plot design goals. Thespian provides automated means for configuring and testing virtual characters, and thus supports fast development of interactive narratives in the face of open-ended user interaction. Thespian has been successfully used to build more than thirty interactive narratives in different domains.


Interactive Storytelling for Foreign Language Learning

Storytelling is an interactive process. Though the audience cannot affect how the story develops, a good storyteller observes the listener's responses and adjusts the emphasis of his/her storytelling accordingly. This project is aimed at creating an automated AI agent for interactive storytelling. (video)


Argument Interactive Narratives with Cognitive Robots

This project aims at making the interaction with virtual characters more natural and more intuitive by combing cognitive robots with digital avatars. Primitive movements of the robots can be used to help express emotions, illustrate locations, and even suggest the social relationships among characters. We expect such embodied robot/characters to have great potential over traditional displays for engaging the user, especially children. (video)


Presence in Virtual/Augmented Environment

Giving user the experience of presence – feeling “being there” -- is a basic design goal for most virtual environments. The experience of “presence” is affected by many factors. This projects investigates how task and interaction design affect the user's experience of social and physical presence. (video)

A Virtual Space for Children to Meet and Practice Chinese

This project aims at creating an online space for children to meet virtually and practice Mandarin Chinese. The project will provide a platform for children to talk with each other and communicate using gestures and body languages. The users will solve puzzles collaboratively and improve their language skills along the ways. This project will investigate how the use of media affects children's communication behaviors and how media can be designed to allow children feeling co-existence with other remote users. (video)

New! You can download a trial version of our game from here.

Emotion Contagion and Emergent Evacuation

Newark Liberty Airport evacuation

Emotions can be passed from one person to another. This project creates computational models of emotion contagion and simulates emergent evacuation scenarios using such models.


Emotion Detection

This project explores physiological and vision based techniques for detecting people's emotions.

Previous Projects @ USC

Virtual Sex: Real Risk Reduction for MSM


This is a NIMH funded project for AIDS prevention. In this project, tools are being developed for naïve and non-technical authors to use Thespian and the Unity game engine for designing interactive narratives. The final system is estimated to be used by more than two thousands users in clinical trials.

Little Red Riding Hood Story


A twisted Little Red Riding Hood story is molded for evaluating various components in Thespian. The user assumes the role of the wolf. 

"Everybody knows the story of the Little Red Riding Hood, or at least they think they do. It is about how a sweet little girl encountered an old murderous wolf on a nice Sunday morning when she went to her granny's house to bring the old lady some wine and cake. Today, I present to you a different story. A story that is told by the wolf, or one of its decedents since the wolf should have already been dead for many years..."

The Tactical Language Training Project


The objective of this project is to develop tools to support individualized language learning, and apply them to the acquisition of tactical languages: subsets of linguistic, gestural, and cultural knowledge and skills necessary to accomplish specific missions.

Modeling Belief and Attitude Change in a Coherence-Network


The focus of this research is to model how social groups form and modify their opinion. The basic idea behind the system is that people tend to form mental coherence when they make decisions and/or interpret things happening around them. This system is built as an extension of Thagard's theory on explanational coherence (ECHO).

RAP Teams: Heterogeneous Robot-Agent-Person Teams
  The goal of this research is to provide an infrastructure to support effective coordination of robots, agents and people (RAPs). RAP team combines each team member's unique capabilities and promises to improve safety, efficiency and reliability in their tasks. On the other hand, their heterogeneity imposes challenges on coordination, because each team member has different social abilities and ability to coordinate with others.


Teaching and Student Advisement

Fall, 2010: COMM 4780 Interactive Narrative
Spring, 2011: COGS 4965 Game AI
Fall, 2011: COGS 4960/COGS 6969 Intelligent Virtual Agent
Spring, 2012: COGS 4965 Game AI
Fall, 2012: COGS 4960/COGS 6969 Intelligent Virtual Agent;
  COGS 4967/COGS 6961/CSCI 4972 Affective Computing
Spring, 2013: COGS 4965 Game AI COGS4962
COGS 6963 Learning and Advanced Game AI

Current Students:

Michael Garber-Barron, Ph.D. student in Cognitive Science

Link to Social Interaction Lab (TBA)