Language and Information Technologies Lab

Multimodal Deception Detection

We explore a new generation of computational tools for joint modeling of physiological and linguistic signals of human behavior. Our research work includes the construction of multimodal resources for the deception detection task, including novel physio-linguistic datasets as well as the development of multimodal approaches that integrate physiological features (e.g., heart rate, respiration rate, galvanic skin response, skin temperature), and linguistic features.

People

Current and previous collaborators

Mihai Burzo

Principal Investigator

Dr. Burzo is an expert in physiological sensors and their use for affective computing, with expertise in thermal imaging and other sensors for modeling and sensing human behavior.

Rada Mihalcea

Co-Principal Investigator

Dr. Mihalcea's research is focused mainly on computational linguistics, including approaches for analysis of lexical semantics, graph-based algorithms for NLP, and multilingual sentiment and subjectivity analysis

Mohammed Abouelenien

Postdoctoral fellow

Dr. Abouelenien's research focuses on multimodal analysis of human behavior. He has previously worked on machine learning aspects of computer vision, with extensive recent research on multimodal data collection and learning.

Verónica Pérez-Rosas

Postdoctoral fellow

Dr. Pérez-Rosas research focuses on developing multimodal strategies and computational methods to analyze, recognize, and predict human affective responses during social interactions.

Projects

Project 1

Deception Detection from Text

This project aims to identify and model linguistic patterns associated to deceptive and truthful text by extracting linguistic features such as, n-grams, shallow and deep syntax features, readability features, and semantic features. .

Project 2

Multimodal Deception Detection

In this project we perform the deception detection task by integrating features from different data streams such as linguistic, visual, physiological, and thermal modalities. This project also targets the construction of multimodal resources for this task.

Project 3

Real-Life Deception Detection

In this project we approach the identification of deceivers in real-life scenarios such as one-on-one interviews and court trials. We use video recordings taken in natural settings and apply multimodal approaches for deception detection, which have shown to provide valuable visual and linguistic patterns to discriminate between liars and truth tellers.

Events

Our work on real-life deception detection was in the news

Lie-detecting algorithm spots fibbing faces better than humans at New Scientist
Lie-detecting Software uses Machine Learning to Achieve 75 Percent Accuracy at Scientific Computing
Lie-detecting software uses real court case data at Michigan News
How to spot a liar: it’s not what you think at the Telegraph
U-M researchers work to determine when people are lying at Detroit Free Press

Invited talk at PETRA 2016

Dr. Rada Mihalcea and Dr. Mihai Burzo will give the keynote talk " Deception Detection using Trial Data " at the 9th PErvasive Technologies Related to Assistive Environments (PETRA) conference. The conference will held on the Island of Corfu, Greece at the Corfu Holiday Palace Hotel from June 29 to July 1, 2016.

Invited talk at Computational Approaches to Deception Detection

Dr. Rada Mihalcea gave the keynote talk " Deception Detection using Trial Data " at the NAACL 2016 Workshop on Computational Approaches to Deception Detection. The workshop was held in San Diego, California, June 17, 2016 at the Sheraton San Diego Hotel & Marina.

1st Workshop on Multimodal Deception Detection

The workshop was held on November 2015 jointly with the ACM 17th International Conference on Multimodal Interaction, ICMI 2015, in Seattle, WA. MWDD focuses on multimodal and interdisciplinary approaches to deception detection, as well as approaches that utilized a single modality with clear potential for integration with additional modalities. More information can be found here . The workshop proceedings can be found here

Publications

  • Mohamed Abouelenien, Rada Mihalcea, Mihai Burzo, Analyzing Thermal and Visual Clues of Deception for a Non-Contact Deception Detection Approach, in Proceedings of the 9th ACM Conference on PErvasive Technologies Related to Assistive Environments (PETRA 2016), Corfu, Greece, July 2016.[pdf]

  • M. Abouelenien, M. Burzo, R. Mihalcea, "Human Acute Stress Detection via Integration ofPhysiological Signals and Thermal Imaging", in Proceedings of the 9th ACM Conference on PErvasive Technologies Related to Assistive Environments, PETRA 2016, Corfu, Greece, July 2016. [PDF]

  • Veronica Perez-Rosas, Mohamed Abouelenien, Rada Mihalcea, Mihai Burzo, Deception Detection using Real-life Trial Data, in Proceedings of the ACM International Conference on Multimodal Interaction (ICMI 2015), Seattle, November 2015. [pdf]

  • Mohamed Abouelenien, Rada Mihalcea, Mihai Burzo, Trimodal Analysis of Deceptive Behavior, Proceedings of the ACM Workshop on Multimodal Deception Detection, Seattle, WA, November 2015.[pdf]

  • Veronica Perez-Rosas, Mohamed Abouelenien, Rada Mihalcea, Yao Xiao, CJ Linton and Mihai Burzo, Verbal and Nonverbal Clues for Real-life Deception Detection, in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), Lisbon, Portugal, September 2015.[pdf]

  • Veronica Perez-Rosas and Rada Mihalcea, Experiments in Open Domain Deception Detection, in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), Lisbon, Portugal, September 2015.[pdf]

  • Mohamed Abouelenien, Mihai Burzo, Rada Mihalcea, Cascaded Multimodal Analysis of Alertness Related Features for Drivers Safety Applications, in Proceedings of the 8th ACM Conference on PErvasive Technologies Related to Assistive Environments (PETRA 2015), Corfu, Greece, July 2015. [pdf]

  • Veronica Perez-Rosas, Exploration of visual, acoustic, and physiological modalities to complement linguistic representations for sentiment analysis, Ph.D. thesis, December 2014.

  • Veronica Perez-Rosas, Cristian Bologa, Mihai Burzo, Rada Mihalcea, Deception Detection Within and Across Cultures, in ``Text mining,'', Springer, 2014.[pdf]

  • Veronica Perez-Rosas and Rada Mihalcea, Gender Differences in Deceivers Writing Style, in Proceedings of the Mexican International Conference on Artificial Intelligence (MICAI 2014), Tuxtla Gutierrez, Mexico, November 2014.[pdf]

  • Mohamed Abouelenien, Veronica Perez-Rosas, Rada Mihalcea, Mihai Burzo, Deception Detection Using a Multimodal Approach, in Proceedings of the ACM International Conference on Multimodal Interaction (ICMI 2014), Istanbul, Turkey, November 2014.[pdf]

  • Mihai Burzo, Mohamed Abouelenien, Veronica Perez-Rosas, Cakra Wicaksono, Yong Tao, Rada Mihalcea, Using Infrared Thermography and Biosensors to Detect Thermal Discomfort in a Building's Inhabitants, in Proceedings of the ASME International Mechanical Engineering Congress and Exposition (IMECE 2014), Montreal, Canada, November 2014. [pdf]

  • Veronica Perez-Rosas and Rada Mihalcea, Cross-Cultural Deception Detection, Proceedings of the Association for Computational Linguistics (ACL 2014), Baltimore, MD, June 2014.[pdf]

  • Veronica Perez-Rosas, Rada Mihalcea, Alexis Narvaez and Mihai Burzo, A Multimodal Dataset for Deception Detection, in Proceedings of the Conference on Language Resources and Evaluations (LREC 2014), Reykjavik, Iceland, May 2014.[pdf]

  • Mihai Burzo, Cakra Wicaksono, Mohamed Abouelenien, Veronica Perez-Rosas, Rada Mihalcea, Yong Tao, Multimodal Sensing of Thermal Discomfort for Adaptive Energy Saving in Buildings, in Proceedings of the Net Zero Symposium, Florida, March 2014. [pdf]

  • Veronica-Perez Rosas, Rada Mihalcea, Mihai Burzo, Automatic Detection of Deceit in Verbal Communication, in Proceedings of the International Conference on Multimodal Interaction (ICMI 2013), Sydney, Australia, December 2013. [pdf]

  • Rada Mihalcea and Mihai Burzo, Towards Multimodal Deception Detection -- Step 1: Building a Collection of Deceptive Videos, in Proceedings of the ACM International Conference on Multimodal Interaction, Santa Monica, October 2012.

Downloads