Introduction

The creation of displays or environments which passively observe and react to people is an exciting challenge for computer vision. Faces and bodies are central to human communication and yet machines have been largely blind to their presence in real-time, unconstrained environments.

To date, research in computer vision for person tracking has largely focused on exploiting a single visual processing technique to locate and track features of a user in front of the display. These systems have often been non-robust to real-world conditions and fail in complicated, unpredictable visual environments and/or where no prior information about the user population is available. The integration of the three cues in our system is unique, but for related work see [8] which approaches the tracking task using color and motion.

 

Figure:   Vision processing module overview.

We have created a visual person tracking system which achieves robust real-time performance through the integration of multiple visual processing modalities-stereo, color, and pattern. In the following section we describe each of these component modules as well as the framework within which they are integrated. We will then describe an initial application of our tracking system, an interactive video mirror. Finally we will show the results of our system as deployed with naive users, and analyze both the qualitative success of the application and the quantitative performance of the tracking algorithm.