Welcome to the Multimodal Signal Processing (MSP) Laboratory

In this website, you will find an overview of the exciting activities that are happening at Multimodal Signal Processing (MSP) laboratory. It also introduces the faculty and students involved in the lab.

Spotlight

This study presents a novel formulation in affective computing. We aim to retrieve sentences with similar emotional content as an anchor sentence. [pdf]

We use conditional sequential GAN to synthesize expressive lip motion driven by speech. This is a powerful model where lip motions are constrained by speech. [pdf]

TBD [pdf]

TBD [pdf]

TBD [pdf]

TBD [pdf]

TBD [pdf]

This study aims to identify emotionally salient segments from speech during longer interactions. More details in our paper. [pdf]

TBD [pdf]

We explore individual evaluations before estimating consensus labels to augment the training set with synthetic samples. Simple but effective. [pdf]

TBD [pdf]

TBD [pdf]

TBD [pdf]

TBD [pdf]

TBD [pdf]

TBD [pdf]

TBD [pdf]

Studies on speech emotion recognition have often focused on accuracy of the predictions. We argue that systems should also estimate their certainty. [pdf]

TBD [pdf]

It is not enough to use an image-based facial expression recognition system to detect emotion from videos, even if it has human level performance. [pdf]

TBD [pdf]

❮ ❯

About the MSP laboratory

The MSP laboratory is dedicated to advance technology in the area human-centered multimodal signal processing. We are looking at theoretical problems with practical applications. Our goal is to develop methods, algorithms and models to recognize and synthesize human verbal and non-verbal communication behaviors to improve human machine interaction.

Our current research includes:

• Affective computing
• Speech, video and multimodal processing
• Multimodal human-machine interfaces
• Analysis and modeling of verbal and non-verbal interaction
• Human interaction analysis and modeling
• Multimodal speaker identification
• Meeting analysis and intelligent meeting spaces
• Machine learning methods for multimodal processing

The MSP lab was established by Prof. Carlos Busso in August 2009. He is also the director of the group.

The MSP lab is part of the Erik Jonsson School of Engineering and Computer Science at The University of Texas at Dallas .