Sentiment Analysis – Mining Emotions
Introduction
Let’s start with analyzing what has happened over the past 10 years. The internet has generated huge amounts of data. This data is readily available in several formats including text, sound, and pictures.
The textual data can be gathered everywhere: from feedbacks users have given on products, tweeted statements, social media statuses and comments, news articles, emails, SMSs, chat rooms, information on web pages, video channels and many many more.
On Twitter for example, the number of Tweets per minute has increased by 58% for example since 2013 to more than 474,000 Tweets per minute in 2019!
by Rashed Sabra
What is NLP?
Skipping the buzzwords: when the input data is mostly available as natural human language, such as free text, the procedure is defined as Natural Language Processing (NLP).
Sentiment Analysis is a Field of NLP
One of the most important fields of NLP is Sentiment Analysis (SA), the process of mining meaningful patterns from text data. These include interpretation and classifying emotions (positive, negative, and neutral) within text data using text analysis techniques.
From a business point of view, by automatically analyzing customer feedback from survey responses to social media conversations, brands can “listen”, meaning automatically gather their customers’ comments, and adjust product features accordingly.
Rashed Sabra, C++ Developer at L-One»With the massive usage of digital platforms for expressing opinions nowadays, Sentiment Analysis allows us a deep insight into the collective behavior of society or on an issue, be it a product performance or even a candidate in an election.«
Sentiment data can help us understand the current and historical context of an issue. With automated analytics, we can grasp what the wider audience thinks about an event, product or concept faster than ever before and take appropriate action. We can quickly understand what the wider audience is thinking about an event, product and concept and take the actions accordingly.
The information that is stored in the text allows computing an indicator such as negative, neutral, or positive. This indicator can serve as a signal for decision-makers.
Governments for instance have used Sentiment Analysis results during their election campaigns.
Reasons for caring about Sentiment Analysis
In a commercial context, SA can provide online advice and recommendations for both the customers and merchants. On the one hand, user preferences that the data reveals can be used to help e-commerce platforms analyzing their products and services. On the other hand, for the virtual nature of online shopping, it is not easy to understand a commodity comprehensively and objectively, and whether the consumer is willing to learn about other consumers’ comments or opinions.
The massive demand for political information can be regarded as another important factor driving SA applications. In the analysis of the conversation on Twitter before the European Parliament elections, researchers have collected more than 1.2 million tweets in three languages (English, German, and French) during a two-week period (May 1 to May 14, 2014).
Public security, sociopolitical events such as the Arab Spring and the London Riots vividly demonstrate the importance of sentiment analysis to public security.
To sum up, SA is important not only for traditional consumers and companies conducting surveys to gather opinions about corresponding products or services, but also it plays an important role on national security and public opinion analysis.
Sarah Holschneider, Head of NLP at L-One»How we look at sentiment analysis projects at L-One Systems? We find the right research setup to generate those solutions that our clients need to solve their individual problems.«
In part one of this article, we’ll go into more details about what sentiment analysis is, how it works, and talk a little about its challenges.
1. Types of Sentiment Analysis
2. How Does Sentiment Analysis Work?
3. Challenges
Let’s get started!
Types of Sentiment Analysis
SA can occur in various forms, from models that focus on polarity → positive, negative, neutral to those that detect feelings and emotions → angry, happy, sad, … , or even models that identify intentions like → interested vs. not interested.
Here are some of the most popular types of sentiment analysis:
Fine-grained Sentiment Analysis
If polarity precision is important to your business, you might consider expanding your polarity categories to include:
This is usually referred to as fine-grained sentiment analysis, and could be used to interpret 5-star ratings in a review, for example:
• Very Positive = 5 stars
• Very Negative = 1 star
A fine-grained analysis can provide more precise results to an automated system that prioritizes addressing customer complaints. In addition, dual-polarity sentences such as “The location was truly disgusting … but the people there were glorious.” can confuse binary sentiment classifiers, leading to incorrect class predictions.
Emotion detection
As the name suggests, emotion detection aims at detecting emotions, like happiness, frustration, anger, sadness, and so on. Many emotion detection systems use lexicons, lists of words and the emotions they convey or complex machine learning algorithms.
“I simply love it!” or “This movie scares me to death!” are good examples for emotional statements.
One of the downsides of using lexicons is that people express emotions in different ways. Some words that typically express anger, like bad or kill: “Your product is so bad!”, “Your customer support is killing me!”, might also express happiness : “This is bad ass support” or “You are killing it!”.
Aspect-based Sentiment Analysis
Usually, when analyzing sentiments of texts, let’s say product reviews, you’ll want to know which particular aspects or features people are mentioning in a positive, neutral, or negative way. That’s where aspect-based sentiment analysis can help. An aspect is an attribute or a component of an entity.
For example, the food was great but the service was awful.
The features: [Food, Services] for the entity: restaurant.
This problem involves subproblems such as identifying relevant entities, extracting their features or aspects, and determining whether an opinion expressed on each aspect or feature is positive, negative, or neutral. Basically, aspect-based analysis needs to find explicit aspect expressions, which usually are nouns and noun phrases from the text of a given domain. These nouns and other word classes can be identified by a part-of-speech tagger (POS).
In part II of this article, we will look at the NLP methods and algorithms on which SA is based in more detail.
Rashed is currently finishing a master degree in Big Data Systems and researches in the area of Natural Language Processing. He holds a Bachelor degree of Artificial Intelligence. In 2020 he joined the L-One Team and since worked on a project developing a platform for a human-robot interface (HRI). In his spare time, he reads books and enjoys watching his favorite football team (FC Bayern Munich).
Sources & Links
- Yue, L., Chen, W., Li, X., Zuo, W., Yin, M., A survey of sentiment analysis in social media algorithms, Knowledge and Information Systems, 2018.
- MonkeyLearn, Sentiment Analysis: A Definitive Guide
- P. Rao, Fine-grained Sentiment Analysis in Python (Part 1), 2019