An Affective Framework for Multimodal Sentiment Analysis to Navigate Emotional Terrains

Abstract

Affective computing, an interdisciplinary research field, unites experts from beginning of artificial intelligence, NLP, and cognitive sciences. The rise of E-content on platforms like YouTube and social media has spurred a shift in affective computing from traditional unimodal to complex multimodal methodologies. To fill the gap in literature discussions, we present a contemporary analysis focuses on multimodal affect analysis, incorporating audio, visual, and text information. Our study critically examines state-of-the-art methods. The new framework aims to establish a foundation for refined understanding. Specifically, we recommend an Interactive outline of framework which is based on Soft Mapping approach for multimodal emotion and sentiment analysis. Evaluated on MOSEI, MOSI and MELD datasets, our model demonstrates accuracy improvements. This innovative framework offers a fresh perspective on addressing the challenge of data communication in multimodal emotion/sentiment analysis, providing insights into emotional terrains within multimedia content.