Deep Neural Networks for Personalized Sentiment Analysis with Information Decay

Doctoral thesis (2019)

People have different lexical choices when expressing their opinions. Sentiment analysis, as a way to automatically detect and categorize people’s opinions in text, needs to reflect this diversity. In ... [more ▼]

People have different lexical choices when expressing their opinions. Sentiment analysis, as a way to automatically detect and categorize people’s opinions in text, needs to reflect this diversity. In this research, I look beyond the traditional population-level sentiment modeling and leverage socio-psychological theories to incorporate the concept of personalized modeling. In particular, a hierarchical neural network is constructed, which takes related information from a person’s past expressions to provide a better understanding of the sentiment from the expresser’s perspective. Such personalized models can suffer from the data sparsity issue, therefore they are difficult to develop. In this work, this issue is addressed by introducing the user information at the input such that the individuality from each user can be captured without building a model for each user and the network is trained in one process. The evolution of a person’s sentiment over time is another aspect to investigate in personalization. It can be suggested that recent incidents or opinions may have more effect on the person’s current sentiment than the older ones, and the relativeness between the targets of the incidents or opinions plays a role on the effect. Moreover, psychological studies have argued that individual variation exists in how frequently people change their sentiments. In order to study these phenomena in sentiment analysis, an attention mechanism which is reshaped with the Hawkes process is applied on top of a recurrent network for a user-specific design. Furthermore, the modified attention mechanism delivers a functionality in addition to the conventional neural networks, which offers flexibility in modeling information decay for temporal sequences with various time intervals. The developed model targets data from social platforms and Twitter is used as an example. After experimenting with manually and automatically labeled datasets, it can be found that the input formulation for representing the concerned information and the network design are the two major impact factors of the performance. With the proposed model, positive results have been observed which confirm the effectiveness of including user-specific information. The results reciprocally support the psychological theories through the real-world actions observed. The research carried out in this dissertation demonstrates a comprehensive study of the significance of considering individuality in sentiment analysis, which opens up new perspectives for future research in the area and brings opportunities for various applications. [less ▲]

Looking into the Past: Evaluating the Effect of Time Gaps in a Personalized Sentiment Model

in ACM/SIGAPP Symposium On Applied Computing, Limassol 8-12 April 2019 (2019, April)

This paper concerns personalized sentiment analysis, which aims at improving the prediction of the sentiment expressed in a piece of text by considering individualities. Mostly, this is done by relating ... [more ▼]

This paper concerns personalized sentiment analysis, which aims at improving the prediction of the sentiment expressed in a piece of text by considering individualities. Mostly, this is done by relating to a person’s past expressions (or opinions), however the time gaps between the messages are not considered in the existing works. We argue that the opinion at a specific time point is affected more by recent opinions that contain related content than the earlier or unrelated ones, thus a sentiment model ought to include such information in the analysis. By using a recurrent neural network with an attention layer as a basic model, we introduce three cases to integrate time gaps in the model. Evaluated on Twitter data with frequent users, we have found that the performance is improved the most by including the time information in the Hawkes process, and it is also more effective to add the time information in the attention layer than at the input. [less ▲]

Personalized Sentiment Analysis and a Framework with Attention-Based Hawkes Process Model

in Agents and Artificial Intelligence (2019)

People use different words when expressing their opinions. Sentiment analysis as a way to automatically detect and categorize people’s opinions in text, needs to reflect this diversity and individuality ... [more ▼]

People use different words when expressing their opinions. Sentiment analysis as a way to automatically detect and categorize people’s opinions in text, needs to reflect this diversity and individuality. One possible approach to analyze such traits is to take a person’s past opinions into consideration. In practice, such a model can suffer from the data sparsity issue, thus it is difficult to develop. In this article, we take texts from social platforms and propose a preliminary model for evaluating the effectiveness of including user information from the past, and offer a solution for the data sparsity. Furthermore, we present a finer-designed, enhanced model that focuses on frequent users and offers to capture the decay of past opinions using various gaps between the creation time of the text. An attention-based Hawkes process on top of a recurrent neural network is applied for this purpose, and the performance of the model is evaluated with Twitter data. With the proposed framework, positive results are shown which opens up new perspectives for future research. [less ▲]

A Bilingual Study for Personalized Sentiment Model PERSEUS

Scientific Conference (2018, September 10)

This paper investigates the significance of analyzing language preferences in personalized sentiment analysis. Motivated by the considerable amount of text generated by multilingual speakers on social ... [more ▼]

This paper investigates the significance of analyzing language preferences in personalized sentiment analysis. Motivated by the considerable amount of text generated by multilingual speakers on social platforms, we focus on constructing a single model that is able to analyze sentiments in a multilingual environment. In particular, Twitter texts are used in this research where the choice of language can be switched at a message-, sentence-, word- or topic-level. To represent and analyze the text, we extract concepts and main topics from the text and apply a recurrent neural network with attention mechanism in order to learn the relation between the lexical choices and the opinions of each sentiment holder. The personalized sentiment model PERSEUS is applied as the central structure of this research. Moreover, a language index is added to each concept to enable multilingual analysis, which provides a solution for analyzing code-switching in the text as well. In this work, English and German are chosen for a pilot study, and an artificial corpus is created to evaluate the situation with multilingual speakers. [less ▲]

Embedding of the Personalized Sentiment Engine PERSEUS in an Artificial Companion

in International Conference on Companion Technology, Ulm 11-13 September 2017 (2017, September)

The term Artificial Companion has originally been introduced by Y. Wilks [1] as “...an intelligent and helpful cognitive agent, which appears to know its owner and their habits, chats to them and diverts ... [more ▼]

The term Artificial Companion has originally been introduced by Y. Wilks [1] as “...an intelligent and helpful cognitive agent, which appears to know its owner and their habits, chats to them and diverts them, assists them with simple tasks. . . ”. To serve the users’ interests by considering a personal knowledge is, furthermore, demanded. The following position paper takes this request as motivation for the embedding of the PERSEUS system, which is a personalized sentiment framework based on a Deep Learning approach. We discuss how such an embedding with a group of users should be realized and why the utilization of PERSEUS is beneficial. [less ▲]