People’s desire to engage with businesses and the overall brand perception depend heavily on public opinion. According to a survey by Podium, 93 percent of consumers say that online reviews influence their buying decisions. Users may not give you a chance once they’ve read a few bad reviews. They won’t research whether feedback is fake or not. They'll choose another option.
In this context, organizations that constantly monitor their reputation can timely address issues and improve operations based on feedback. Sentiment analysis allows for effectively measuring people’s attitude toward an organization in the information age.
What is sentiment analysis
Sentiment analysis is a form of text research that uses a mix of statistics, natural language processing (NLP), and machine learning to identify and extract subjective information — for instance, a reviewer’s feelings, thoughts, judgments, or assessments about a particular topic, event, or a company and its activities.
You can learn more about reputation management in general from our separate post.
This analysis type is also known as opinion mining (with a focus on extraction) or affective rating. Some specialists prefer the terms sentiment classification and extraction. Regardless of the name, the goal is the same: to know a user or audience opinion on a target object by analyzing a vast amount of text from various sources.
Sentiment by polarity. Source: KDnuggets
Basically, sentiment analysis distinguishes three types of emotions — negative, neutral, and positive. It can be applied to a separate sentence or its part as well as being used for document classification, where the term document covers a broad range of textual items like emails, reviews, comments, articles, and more.
Types of sentiment analysis
Sentiment analysis comes in many forms — depending on the tasks and objectives you pursue. But in practice, several types are often combined to solve complex real-life problems.
Subjectivity classification
Subjectivity classification divides fragments of text into objective and subjective or opinionated. An objective sentence contains facts and neutral information: Three strangers are reunited by astonishing coincidence after being born identical triplets, separated at birth, and adopted by three different families. In turn, a subjective sentence expresses someone’s attitude, feelings, judgment, belief, and more: This apartment is wonderful. I enjoy every minute I spend in here.
Since subjectivity classification filters out neutral statements, it often serves as the first step of polarity classification.
Polarity classification
Opinionated pieces of text can be further divided into negative and positive, using polarity classification. This technique works for large-scale studies of positive and negative trends in text data like product reviews, social media posts, or customer feedback.
Advanced models go further than just binary classification, identifying the sentiment intensity. In this case, text pieces are categorized into more than two groups — for example, extremely positive, positive, neutral, negative, and extremely negative. The multiclass approach allows not only for solving subjectivity and polarity tasks simultaneously. It also provides more precise results when processing sentences with comparative expressions (better, most disgusting, etc.) and modifiers (too, so, completely, and so on.)
Yet, even multiclass polarity classification has a lot of limitations. It doesn’t detect the customer’s attitude to different aspects or characteristics of your services — which is critical for making improvements and successful product development. That’s where an aspect-based sentiment analysis may help you out.
Aspect-based sentiment analysis
Aspect-based or feature-based sentiment analysis is a multistep process aiming at detecting and extracting sentiments toward a specific component of a product or service. For example, a single hotel review can contain both positive and negative statements — since the same guest may highly appreciate a view from the window and convenient location but complain of a dirty bathroom and expensive drinks in the bar.
To capture all this critical information, an aspect-based mechanism first identifies features discussed in the comments or reviews. Then, polarity classification is applied to text fragments that mention those aspects. Finally, the results are aggregated and scored by aspect to understand the trending attitude toward a certain feature.
Read how we scored hotel amenities based on guest reviews to get an idea of how such an aspect-based mechanism can be built in practice.
Fine-grained sentiment analysis
There is no unified definition for fine-grained sentiment analysis — the meaning varies from study to study. The term is often used as a synonym for aspect-based sentiment analysis, multiclass polarity classification, or subsentence-level sentiment analysis — as opposed to coarse-grained opinion mining performed at document and sentence levels.
In its most complete iteration, fine-grained analysis is conducted to define both the sentiment target (topic) and its intensity at a phrase or clause level. It solves challenges the simpler methods can’t address — for example, processing comparative expressions (e.g., Samsung is way better than iPhone) and dual-polarity comments and reviews (e.g., The soup was truly bad but the way they served it was stunning.)
How to conduct sentiment analysis
Sentiment analysis allows you to look at your operations from a customer point of view. But how do you extract that knowledge from user-generated data? These are common steps to create a custom opinion-mining model by the forces of an in-house or external data science team.
Data collection. First, you need to gather relevant brand reviews and mentions in one dataset. You can collect feedback from your own website or partner with resources that own such data.
Read our article on data collection for machine learning to dive deeper into the topic.
Sentiment annotation. The second step is to assign sentiment tags (positive, neutral, negative, etc.) to words and phrases. Attribute-based and fine-grained types of sentiment analysis will require more labels — and more textual data — to produce accurate results. Keep in mind that sentiment labeling is considered reliable if it’s made by more than one annotator.
Read our articles on data labeling in machine learning and how to organize data labeling to learn more about this process.
Text cleansing. Reviews and comments typically contain a lot of irrelevant and excessive information that can negatively affect a model's precision. So, before feeding the dataset to an algorithm, you must get rid of noises, stop words (articles, pronouns, etc.), and variations of the same words, transforming them into canonical form.
To learn more, read our article on preparing your dataset for machine learning or watch our dedicated video explainer.
Note that all the above-mentioned steps are conducted by freelancers or trainees rather than by experienced data scientists. Moreover, to save time and money, you can take advantage of public datasets for machine learning annotated for sentiment analysis tasks. Some examples are Trip Advisor Hotel Reviews, Sentiment140, and Stanford Sentiment Treebank.
The next two steps require the engagement of experienced data scientists.
Word embedding. To make text data understandable for ML models, you must translate words and phrases into vectors. This process is called word embedding.
Model training and testing. Finally, your data science team proceeds to train an ML model and evaluate its results. For this, the prepared dataset is split into training and testing parts. Once the model achieves satisfactory predictions, it can be used for sentiment detection and classification in new, unlabeled reviews.
Sentiment analysis tools and APIs
Not all companies can afford to build custom ML models for sentiment analysis. Fortunately, there are various off-the-shelf tools that collect feedback from numerous sources, alert on mentions in real time, analyze text, and visualize results. Some of these platforms expose APIs so you can integrate them with your existing system and get access to sentiment analysis instruments directly from your working environment.
Below, we’ll briefly review a few popular solutions available in the market.
InMoment experience improvement platform employs Lexalytics, a world-leading NLP engine, to sort through incoming feedback and determine consumer attitudes to your products. It helps you pinpoint issues and resolve them promptly, thus improving customer experience.
Unlike other sentiment analysis tools, InMoment can define not only how customers feel about your brand or offerings but also what makes them feel a certain way. To make results even more precise, text analytics algorithms can be customized for your business needs based on data collected from your channels.
Clarabridge is a customer experience management platform that pulls and analyzes text from chats, survey platforms, blogs, forums, social media channels, and review sites. Users can also gain insights from emails, employee and agent notes, call recordings, and Interactive Voice Response (IVR) surveys automatically converted into text.
The platform uses over 150 industry-specific NLP models to automatically detect sentiments (positive or negative), their intensity, and even a particular emotion (like frustration or anger.)
Azure AI Language provides three options to access sentiment analysis functionality. You can use a web-based platform, Language Studio, integrate your software with the REST API, or deploy the available Docker container on-premises. In any case, you’ll be able to conduct polarity classification and aspect-based sentiment analysis, taking advantage of powerful prebuilt models. Yet, the Azure solution isn’t meant to collect feedback — you have to do it yourself.
Sentiment analysis with Azure AI Language. Source: Microsoft
Cloud Natural Language API by Google supports sentiment analysis for 16 languages. Once it’s integrated with your software, you can make a request to process your text file or a document kept in Google Cloud Storage. The API will return information about the overall sentiment of the document and the overall strength of emotion within the given text. The response also contains information about sentiments and their intensity at the sentence level.
IBM Watson NLP allows you to add your dataset and train a model for sentiment detection at a document and sentence level and aspect-based sentiment analysis. This solution requires advanced expertise in data science, though it’s less time- and resource-consuming than building a sentiment analysis model from scratch.
Where can you use sentiment analysis
Sentiment analysis allows businesses to harness tremendous amounts of free data to understand customer attitudes toward their brand, improve products and services, and maintain their reputation. It’s a powerful tool for workforce analytics as well.
Brand monitoring
If the Internet was a mountain river, then analyzing user-generated content on social media and other platforms is like fishing during the trout-spawning season. People enjoy sharing their points of view regarding the latest news, local and global events, and their experience as customers. Twitter and Facebook are favorite places for daily comment wars and spirited (to put it mildly!) conversations. News about celebrities, entrepreneurs, and global companies draws thousands of people within a couple of hours after being published on Reddit. Media giants like Time, The Economist, and CNBC, as well as millions of blogs, forums, and review platforms, flourish with content on various topics.
Why not use these data sources to monitor what people think and say about your organization and why they perceive you this way? Sentiment analysis of brand mentions allows you to keep current with your credibility within the industry, identify emerging reputational crises, and respond to them quickly. You can compare this month’s results and those from the previous quarter, for instance, and find out how your brand image has changed during this time.
It’s not only important to know social opinions about your organization but also to define who is talking about you, whether the industry influences your brand, and in what context. What’s more exciting sentiment analysis software does all of the above in real time and across all channels.
Competitive research
There is one thing for sure you and your competitors have in common – a target audience. You can track and research how society evaluates competitors just as you analyze their attitude toward your business. What do customers value most about other industry players? Is there anything competitors lack or do wrong? Which channels do clients use to engage with other companies? Take advantage of this knowledge to improve your communication, marketing strategies, and overall service.
Competitive analysis that involves sentiment analysis will help you understand your weaknesses and strengths and maybe find ways to stand out.
Flame detection and customer service prioritization
Hospitality brands, financial institutions, retailers, transportation companies, and other businesses use sentiment classification to optimize customer care department work. With text analysis platforms like IBM Watson Natural Language Understanding or MonkeyLearn, users can automate the classification of incoming customer support messages by polarity, topic, aspect, and priority. Then, queries are sent to dedicated teams and specialists.
Since it’s better to put out a spark before it turns into a flame, new messages from the least happy and most angry customers are processed first. Satalytics, for example, groups feedback by device, customer journey stage, and new or repeat customers.
Product analysis
Every entrepreneur dies to see fans standing in lines waiting for stores to open so they can run inside, grab that new product, and become one of the first proud owners in the world. How do you bring the desired product to market? The only approach is to ask people what they want.
Successful companies build a minimum viable product (MVP), gather early feedback, and continuously improve features even after the product launch.
Sentiment analysis allows for learning about product advantages and drawbacks. For example, a student from Oklahoma State University analyzed Amazon reviews about two Samsung phone models (Galaxy S6 & Galaxy S7) and two devices from Apple (iPhone 6 & iPhone 7) to find out why customers preferred one brand over another. He found out that users whose priorities are a reliable battery and a good screen choose Samsung phones. And customers who are more interested in design and cameras buy iPhones.
Filtering comments by topic and sentiment, you can also determine which features are necessary and which must be eliminated. Armed with sentiment analysis results, a product development team will know exactly how to deliver an innovation that customers would buy and enjoy.
Market research and insights into industry trends
As we said before, social media sites and forums are sources of information on any topic. People discuss news and products and write about their values, dreams, everyday needs, and events. And they do this voluntarily 24/7.
With sentiment analysis, marketers can track and study consumer behavior patterns in real time to predict future trends and help management make informed decisions.
Workforce analytics and employee engagement monitoring
Some organizations go beyond using sentiment analysis for market research or customer experience evaluation, applying it internally for HR-related processes. These companies measure employee satisfaction and detect factors that discourage team members and eventually reduce their performance. Specialists automate the analysis of employee surveys with sentiment analysis software, which allows them to address problems and concerns faster. Human resource managers can detect and track the general tone of responses, group results by departments and keywords, and check whether employee sentiment has changed over time or not.