Sentiment Analysis: Effects and underlying Mechanisms – Part I

Sentiment Analysis (SA) is often referred to as Emotion AI. It makes use of natural language processing, text analysis, computational linguistics, and biometrics for extracting and evaluating emotional biases. This first of two posts on the topic introduces the process and its increasing use in the economy and in politics.

Sentiment Analysis – Mining Emotions

Introduction

Let’s start with analyzing what has happened over the past 10 years. The internet has generated huge amounts of data. This data is readily available in several formats including text, sound, and pictures.
The textual data can be gathered everywhere: from feedbacks users have given on products, tweeted statements, social media statuses and comments, news articles, emails, SMSs, chat rooms, information on web pages, video channels and many many more.
On Twitter for example, the number of Tweets per minute has increased by 58% for example since 2013 to more than 474,000 Tweets per minute in 2019!

by Rashed Sabra

Sentiment Analysis debuzzed by L-One Systems R. Sabra
Rashed is currently working on a project developing a platform for a human-robot interface (HRI). In his spare time, he reads books and enjoys spending time with the family.

What is NLP?

Skipping the buzzwords: when the input data is mostly available as natural human language, such as free text, the procedure is defined as Natural Language Processing (NLP).

Sentiment Analysis is a Field of NLP

One of the most important fields of NLP is Sentiment Analysis (SA), the process of mining meaningful patterns from text data. These include interpretation and classifying emotions (positive, negative, and neutral) within text data using text analysis techniques.

From a business point of view, by automatically analyzing customer feedback from survey responses to social media conversations, brands can “listen”, meaning automatically gather their customers’ comments, and adjust product features accordingly.

»With the massive usage of digital platforms for expressing opinions nowadays, Sentiment Analysis allows us a deep insight into the collective behavior of society or on an issue, be it a product performance or even a candidate in an election.«

Rashed Sabra, C++ Developer at L-One

Sentiment data can help us understand the current and historical context of an issue. With automated analytics, we can grasp what the wider audience thinks about an event, product or concept faster than ever before and take appropriate action. We can quickly understand what the wider audience is thinking about an event, product and concept and take the actions accordingly.
The information that is stored in the text allows computing an indicator such as negative, neutral, or positive. This indicator can serve as a signal for decision-makers.
Governments for instance have used Sentiment Analysis results during their election campaigns.

Sentiment Analysis showing emotional language diagram on screen @L-One_Systems
Contextual mining of texts can identify and interpret subjective information in source material helping a business to understand the social sentiment of their brand, product or service

Reasons for caring about Sentiment Analysis

In a commercial context, SA can provide online advice and recommendations for both the customers and merchants. On the one hand, user preferences that the data reveals can be used to help e-commerce platforms analyzing their products and services. On the other hand, for the virtual nature of online shopping, it is not easy to understand a commodity comprehensively and objectively, and whether the consumer is willing to learn about other consumers’ comments or opinions.
The massive demand for political information can be regarded as another important factor driving SA applications. In the analysis of the conversation on Twitter before the European Parliament elections, researchers have collected more than 1.2 million tweets in three languages (English, German, and French) during a two-week period (May 1 to May 14, 2014).

Sentiment Analysis bias twitter posts @ L-One Systems
For the Twitter conversation in English, 39% of the assertions on Twitter are negative toward the European Union (EU), compared to 30% with a neutral and 31% positive bias

Public security, sociopolitical events such as the Arab Spring and the London Riots vividly demonstrate the importance of sentiment analysis to public security.

To sum up, SA is important not only for traditional consumers and companies conducting surveys to gather opinions about corresponding products or services, but also it plays an important role on national security and public opinion analysis.

»How we look at sentiment analysis projects at L-One Systems? We find the right research setup to generate those solutions that our clients need to solve their individual problems.«

Sarah Holschneider, Head of NLP at L-One

In part one of this article, we’ll go into more details about what sentiment analysis is, how it works, and talk a little about its challenges.

1. Types of Sentiment Analysis
2. How Does Sentiment Analysis Work?
3. Challenges

Let’s get started!

Types of Sentiment Analysis

SA can occur in various forms, from models that focus on polarity → positive, negative, neutral to those that detect feelings and emotions → angry, happy, sad, … , or even models that identify intentions like → interested vs. not interested.

Here are some of the most popular types of sentiment analysis:

Fine-grained Sentiment Analysis

If polarity precision is important to your business, you might consider expanding your polarity categories to include:

Sentiment Analysis @L-One Systems: Bar Chart for fine grained SA
Fine-grained sentiment analysis can provide more precise results on an aggregate level than classifying sentiments broadly into positive, neutral or negative, hereby improving the identification of prevailing emotion categories

This is usually referred to as fine-grained sentiment analysis, and could be used to interpret 5-star ratings in a review, for example:

• Very Positive = 5 stars
• Very Negative = 1 star

A fine-grained analysis can provide more precise results to an automated system that prioritizes addressing customer complaints. In addition, dual-polarity sentences such as “The location was truly disgusting … but the people there were glorious.” can confuse binary sentiment classifiers, leading to incorrect class predictions.

Emotion detection

As the name suggests, emotion detection aims at detecting emotions, like happiness, frustration, anger, sadness, and so on. Many emotion detection systems use lexicons, lists of words and the emotions they convey or complex machine learning algorithms.

I simply love it! or This movie scares me to death!are good examples for emotional statements.

One of the downsides of using lexicons is that people express emotions in different ways. Some words that typically express anger, like bad or kill: “Your product is so bad!, Your customer support is killing me!”, might also express happiness : “This is bad ass support or You are killing it!”.

Aspect-based Sentiment Analysis

Usually, when analyzing sentiments of texts, let’s say product reviews, you’ll want to know which particular aspects or features people are mentioning in a positive, neutral, or negative way. That’s where aspect-based sentiment analysis can help. An aspect is an attribute or a component of an entity.
For example, the food was great but the service was awful.
The features: [Food, Services] for the entity: restaurant.

This problem involves subproblems such as identifying relevant entities, extracting their features or aspects, and determining whether an opinion expressed on each aspect or feature is positive, negative, or neutral. Basically, aspect-based analysis needs to find explicit aspect expressions, which usually are nouns and noun phrases from the text of a given domain. These nouns and other word classes can be identified by a part-of-speech tagger (POS).

In part II of this article, we will look at the NLP methods and algorithms on which SA is based in more detail.

Rashed is currently finishing a master degree in Big Data Systems and researches in the area of Natural Language Processing. He holds a Bachelor degree of Artificial Intelligence. In 2020 he joined the L-One Team and since worked on a project developing a platform for a human-robot interface (HRI). In his spare time, he reads books and enjoys watching his favorite football team (FC Bayern Munich).