Facial Gesture Analysis in an Interactive Environment

Tuesday, April 29, 2008 - 1:13pm

Gérard Medioni
Computer Science Department, University of Southern California

DATE: MONDAY, May 5, 2008
TIME: 3:30 – 4:30 p.m.
PLACE: Computer Science Conference Room, Harold Frank Hall Rm. 1132

Facial gesture analysis is an important problem in computer vision. Facial gestures carry critical information in nonverbal communication. The difficulty of automatic facial gesture recognition lies in the complexity of face motions. These motions can be categorized into two classes: the global, rigid head motion, and local, nonrigid facial deformations. These two components are coupled in an observed facial motion.

We present our recent research of this topic, which includes tracking and modeling these two motions for gesture understanding. It can be divided into three parts: 3D head pose estimation, modeling and tracking nonrigid facial deformations, and expression recognition. We have developed a novel hybrid 3D head tracking algorithm to differentiate these two motions. The hybrid tracker integrates both intensity and feature correspondence for robust real-time head pose estimation. Based on this tracker, we can classify expression by learning a graphical model of nonrigid face and applying Bayesian estimation.

More recently, nonrigid motions are analyzed in 3D by manifold learning techniques. We decompose nonrigid facial deformations on a basis of 1D manifolds. Each 1D manifold is learned offline from sequences of labeled basic expressions, such as smile, surprise, etc. Any expression is then a linear combination of values along these axes, with the coefficient representing the level of activation. Manifold learning is accomplished using N-D Tensor Voting. The output of our system is a rich representation of the face, including the 3D pose, 3D shape, expression label with probability, and the activation level.

Professor Gérard Medioni received the Diplôme d’Ingenieur from ENST, Paris in 1977, an M.S. and Ph.D. from the University of Southern California in 1980 and 1983 respectively. He has been at USC since then, and is currently Professor of Computer Science and Electrical Engineering, former Chairman of the Computer Science Department, co-director of the Institute for Robotics and Intelligent Systems (IRIS), and co-director of the USC Games Institute. Professor Medioni has made significant contributions to the field of computer vision. His research covers a broad spectrum of the field, such as edge detection, stereo and motion analysis, shape inference and description, and system integration. He has published 3 books, over 50 journal papers and 150 conference articles, and is the recipient of 8 international patents. Prof. Medioni is associate editor of the Image and Vision Computing Journal, associate editor of the Pattern Recognition and Image Analysis Journal, and associate editor of the International Journal of Image and Video Processing. He is a Fellow of IAPR, a Fellow of the IEEE, and a Fellow of AAAI.