When asked “what is sentiment analysis?“, the default answer is often “detecting and understanding how the audience is reacting to a brand, either positively or negatively“.

The truth is, that only describes a fraction of what sentiment analysis involves and its potential. While some marketers prefer leaving the analysis to dedicated tools, the science behind sentiment analysis is nothing short from fascinating – the various levels of analysis, the detail and the intricacy that make this analysis more accurate when performed by another human rather than a machine.

Nowadays, sentiment analysis is an integral part of social listening, although it can also be performed on its own. The usual approach to this is with the following guideline:

“If a piece of content has more positive keywords than negative keyword, it’s positive content; if it has more negative keywords than positive keywords, it’s negative content.”

While this is not completely incorrect, there is a lot more to that. Here I’m going to explain at a surface level how this analysis actually works, the implications of automating sentiment analysis, and what the future holds for sentiment analysis.

What is Sentiment Analysis?

Sentiment analysis is more than just a feature in a social analytics tool – it’s a field of study. This is a field that is still being studied, albeit not at great lengths due to the intricacy of this analysis, in the same way that some aspects of linguistics are still up to debate or not fully understood.

When you perform sentiment analysis on some content, you’re basically looking for the opinions in content and picking the sentiment within those opinions.

An opinion is an expression (a binomial to be precise) that consists of two key components:

• A target (which we shall call “topic”, as referred to by most social Analytics tools);
• A sentiment on the target/topic.

So, in “I love this company“, “this company” is the topic, and the sentiment (as expressed by the verb “love“) is positive.

Semantics and Sentiment

While a few tools out there rely on scoring sentiment words based on their “polarity” (i.e. their position on a scale between positive and negative), accurate sentiment analysis doesn’t look at only keywords or individual words alone, because of semantics.

Semantics is the study of the relation of words, phrases, signs and their denotation (i.e. what signs stand for). Because of semantics, we can’t assume that a sentence that contains a positive sentiment word (e.g. “useful”) is also positive, or that a sentence that contains a negative sentiment word (e.g. “useless”) is also negative. In fact, other words can invert the sentiment of a word, such as:

• sentiment shifters (e.g. “I find this tool less useful than yours”);
• connectives (e.g. “This tool is everything but useful”);
• modals (e.g. “In theory, this tool should be useful”).

Besides individual words, expressions and phrases can be used to show sentiment. Take for instance “I’m over the moon“. If taken word for word, ‘over‘ and ‘moon‘ are not sentiment words, as they don’t express any positivity or negativity. Hence, we can conclude that sentiment cannot be accurately presumed from word-by-word analysis, but rather on a level that allows for semantic interpretation.

Automated Sentiment Analysis in Social Listening

When looking for a Social Listening tool (e.g. Brandwatch, Synthesio, Salesforce Radian 6, Crimson Hexagon, Attensity), it’s now the norm to look for automated sentiment analysis as an added feature. The lack or low accuracy of this feature can definitely be a deal breaker. However, whether you choose to rely on a tool to look for sentiment, or you feel brave enough to do so manually (depending on the volume of content), it’s important to understand what goes on behind the scenes and the building blocks that make sentiment analysis works.

For example: imagine someone, Joe Smith, writing on with the following comment on December 16th 2013:

I’ve just registered for a trial with SimplyMeasured. I’m loving it so far. The reports are incredible. Their blog is so informative, too. However, my colleague thinks it’s expensive. – date: December 16, 2013.

We can break this down into sentences:

(1) I’ve just registered for a trial with SimplyMeasured. (2) I’m loving it so far. (3) The reports are incredible. (4) Their blog is so informative, too. (5) However, my colleague thinks it’s expensive.

While this comment is generally positive, it can be broken down into sentences. This comment has a number of opinions around SimplyMeasured, both positive and negative.

• Sentence 1 is neutral, as it doesn’t offer any sentiment.
• Sentence 2 expresses a positive opinion about the tool as a whole.
• Sentence 3 expresses a positive opinion about the tool’s reports.
• Sentence 4 expresses a positive opinion about the company’s blog.
• Sentence 5 expresses a negative opinion about the tool’s pricing, this time from the eyes of the writer’s colleague.

From this we can deduce the following:

• Content can have opinions from more than one person, which we shall call “opinion holder“. The opinion holder for sentences (2), (3), and (4) is John Smith, while the opinion holder for sentence (5) is John Smith’s colleague.
• Unlike sentence (1), we can’t take sentences (2), (3), (4) and (5) out of context, as they wouldn’t make much sense when read their own. Taking sentence (3) as an example, we can see that the opinion is around one aspect of SimplyMeasured – “reports“. An in depth review would also look at various attributes of this subtopic: the reports’ reliability, design, layout, performance etc. So, topics can be broken down into aspects, as the graph below shows.

• Longer reviews can also have a longer list of opinions, as commonly found on forums and review sites, whether it’s a hotel review on Tripadvisor or a list of gaming reviews. That’s when we start seeing a nested relationship between the main topic of the review and its various aspects (subtopics and attributes included), a bit like this….

Lastly, opinions may change over time, depending on a lot of factors, both subjective (e.g. user perception) or objective (e.g. change in pricing). Hence, it’s imperative that not only sentiment but also time is taken into account during sentiment analysis.

Now we have the 5 main factors to look at when doing sentiment analysis in social listening:

• topics: what are the main areas of discussion?
• aspects (subtopics and attributes): what about those topics is being talked about?
• sentiment: what is the sentiment of the content and the opinions contained?
• holder: whose opinion is being discussed? Are there multiple in the same content? If so, how do they differ, if at all?
• time: when was this content posted?

Levels of Analysis

Let’s refer back to Joe Smith’s review on :

I’ve just registered for a trial with SimplyMeasured. I’m loving it so far. The reports are incredible. Their blog is so informative, too. However, my colleague thinks it’s expensive. – date: December 13, 2013.

While you can approach sentiment analysis in a few ways, you’ll ultimately find three methods:

• Document level: analysing the overall sentiment expressed in the text. This works best on the assumption that the whole comment only discusses one topic.
  • Overall, this comment is positive.
• Sentence level: examining the sentiment expressed in sentences.
  • Out of the 5 sentences, the first one doesn’t express any sentiment, as it only states a fact (“I’ve just registered for a trial with SimplyMeasured“). The remaining sentences express sentiment: the second, third, and fourth sentences are positive, while the fifth one is negative.
• Entity and aspect level: this granular analysis takes into consideration each opinion expressed in the content. The opinions on the reports, blog, and the trial are positive, while the opinion on the price is negative.

Deeper Than Sentiment

Closely related to sentiment, we have two other factors that can shape social sentiment: subjectivity and emotion.

Subjectivity

An objective sentence presents factual information, while a subjective sentence expresses personal feelings, views or beliefs.

Although similar, subjective sentences are not the same as opinionated sentences (i.e. sentences with sentiment):

• A subjective sentence may not express any sentiment.
  • “I think I have the latest version of the browser” is a subjective sentence, although it doesn’t express any sentiment.
• An objective sentence may imply sentiment.
  • “I opened the browser after updating it and it kept crashing”. While this sentence is stating a fact, it shows an implicit opinion about the topic (the browser crashing).

Emotion

While sometimes used interchangeably, emotion and sentiment aren’t the same, especially in sentiment analysis. Sentiment is influenced by emotion, a subjective feeling or thought. The strength of a sentiment is usually linked to the intensity of emotions felt by the opinion holder.

It’s helpful to make a distinction between rational and emotional evaluations:

• Rational evaluation: evaluations from rational reasoning and tangible beliefs (e.g. “The reports from this tool are very useful”).
• Emotional evaluation: evaluations from non-tangible and emotional impulses (e.g. “These reports are the best ever!”).

After making this distinction, we could build a sentiment scoring system, taking into consideration the following sentiment ratings:

• (+2) emotional positive
• (+1) rational positive
• (0) neutral
• (-1) rational negative
• (-2) emotional negative

Sentiment

How many sentiments are there? Mainly two, ultimately four.

The main two are positive and negative, as polar opposites. However, it’s important to understand that sentiment is not a black and white subject. This is why neutral sentiment is equally as important. Some sentences express no sentiment at all: that isn’t indicative of negativity, nor is it of positivity – it’s just the absence of either sentiments.

At the same time, especially when looking at content on a document or sentence level, we can have conflicting opinions or multiple opinions with different sentiments. For instance, “I love frozen yogurt, although I don’t like regular yogurt” is not generally positive or generally negative, and as there are sentiments expressed we cannot conclude that it’s neutral. However, when broken down into opinions, we can see a positive sentiment in the first clause (“I love frozen yogurt…“), and negative sentiment in the second clause (“…although I don’t like regular yogurt“). When we have the same number of positive and negative opinions in a piece of text, we have ambivalent sentiment.

A few industry tools also include two variations of sentiment: positive-neutral (between positive and neutral) and negative-neutral (between negative and neutral). These two are helpful when it comes to showing different grades of sentiment, mainly in terms of strength. For instance, “I kinda like this new product” and “I absolutely adore this new product!!!” are both positive, although in different strengths.

Unfortunately, when it comes to automated sentiment analysis, the higher the number of sentiments we choose to recognise, the higher the chance of errors in sentiment assigning, hence decreasing accuracy.

Issues With Sentiment

One thing I’ve learnt studying Linguistics at university is that language is complex. In fact, it would be too naive to oversimplify language thinking that its underlying sentiment can always be accurately examined by a machine or an algorithm.

There are four main factors that currently stop us from relying blindly on tools for sentiment analysis:

1. Context: a positive or negative sentiment word can have the opposite connotation depending on context (e.g. “my internet provider does a great job when it comes to stealing money from me”. )
2. Sentiment Ambiguity: a sentence with a positive or negative word doesn’t necessarily express any sentiment. (e.g. “can you recommend a good tool I could use?” doesn’t express any sentiment, although it uses the positive sentiment word “good“). Likewise, sentences without sentiment words can express sentiment too. (e.g. “This browser uses a lot of memory” doesn’t contain any sentiment words, although it’s clearly negative at a document level.)
3. Sarcasm: a positive or negative sentiment word can switch sentiment if there is sarcasm in the sentence (e.g. “Sure, I’m happy for my browser to crash right in the middle of my coursework”).
4. Language: a word can change sentiment and meaning depending on the language used. This is often seen in slang, dialects, and language variations. An example is the word “sick“, which can change meaning based on context, tone and language, although clear to the target audience.

The State of Sentiment Analysis Today

What I’ve covered in this article may seem like a lot, but it’s only a small percentage of everything sentiment analysis-related – I haven’t discussed comparative opinions, sentiment analysis in content beyond text (how can you determine sentiment in a Vine video or an Instagram picture?), sentiment analysis in social interactions (can you classify a like and a comment as two positive interactions? Can you classify an unfollow as a negative interaction?), the role of punctuation in sentiment (also known as the infamous difference between “I’m fine!!!” and “I’m fine.”)…

There is just so much information on this topic that perhaps I should write a paper on it. (No, seriously.)

This brings us to two final questions: can tools accurately analyse sentiment, and what does the future hold for sentiment analysis?

Can Tools Accurately Analyse Sentiment?

Tools cannot interpret factors like sarcasm (where the apparent sentiment is picked up by a tool, while the implied opposite sentiment isn’t picked up), and social semantics, where some expressions could be interpreted in the same language depending on where it’s being said (e.g. British English versus American English).

Having said that, while human perception of sentiment is more mature and accurate than a tool’s perception, it still isn’t 100% accurate.

Let’s take this forum review for instance:

“I’m leaving Company A for Company B because of their latest exclusive offer, which I can’t miss! Company A’s customer service is much better though, so I’m definitely going to return.“

While a tool might return this as a neutral mention because of the number of positive and negative opinions, we may or may not disagree with the tool’s judgment based on perception . That review can be perceived differently by Company A and Company B, depending on what aspects they’re looking at. If Company B is looking for feedback on their latest offer, this mention is definitely positive, although it does have a negative mention of customer service. Hence, while we can have a universal sentiment, sentiment can also change based on who’s doing the analysis.

Ultimately, a tool’s accuracy is merely the percentage of times that human judgment agrees with the tool’s judgment. This degree of agreement among humans is also known as human concordance. There have been various studies run by various people and companies (including before it merged into ), and they concluded that the rate of human concordance is between 70% and 79%.

Taking that into consideration, we can safely say that a good accuracy for sentiment analysis tools is 70%. The problem is, a tool that is accurate less than 70% of the time isn’t accurate, and a ‘perfect’ tool that is accurate 100% of the time will draw data that we disagree with roughly 30% of the time.

What Does the Future Hold for Sentiment Analysis?

Due to its large undiscovered territory, there are so many questions about sentiment analysis that remain unanswered.

For instance, can social sentiment analysis help us predict events? We’ve already seen a few attempts at this, as it happened with the election of Barack Obama against Mitt Romney. A more recent example comes from the UK, with an attempt from Tweview to predict the winner of the 2013 X Factor using sentiment, volume, and a lot of other social factors using the ever useful API from Brandwatch). While they predicted as the winner of 2013 X Factor, he ended up second, with winning. (You can read more about Tweview’s experiment here. )

Could we perhaps adapt social predictions from social sentiment for other scenarios, such as a viral campaign, or a PR disaster, or a business restructure or an app update? Perhaps.

Lastly, will we ever have a sentiment Analytics tool that is 100% accurate?

The answer is no, and that is a good thing. Here’s why: a tool that tries to get 100% accuracy would be flawed, as we would naturally disagree with it 30% of the time due to clashes with human concordance (as we’ve already seen).

The ideal tool that a marketer needs to cover automated sentiment analysis needs to provide these 4 cardinal features:

• Content type filter: not all online content is the same – sentiment on forums works differently from other other social platforms, just as sentiment in microblogging sites (e.g. Twitter, Tumblr) works differently from other online sources. Due to that, an ideal tool needs to be able to filter search results by platform, so you can filter those results by content type and, ultimately, sentiment.
• Analysis Levels: while most industry tools measure sentiment at an article level, an ideal tool supports all three levels of sentiment analysis we’ve discussed: article level, sentence level and topic level.
• Human accuracy: instead of a tool trying to be 100% accurate, it needs to be in line with our human concordance, understanding factors that can impact human language, context, and various linguistics aspects (e.g. semantics). The ideal tool has strong competence in those areas, and it constantly evolves based with human feedback (e.g. through corrections, human overrides and regular updates).
• Sentiment override: to tackle the remaining 30% that clashes with our human concordance, the ideal tool needs to allow users to override automated sentiment, change topics and remove irrelevant results.

Ultimately, regardless of what tool you choose (if any), social sentiment has very high potential. While it’s easy to treat it as a “soft metric”, it’s highly useful when used in context – yes you’ve received a positive review, but what does it mean for your brand? What are the underlying opinions behind that specific content? In the right hands, sentiment can be the key to various social analyses, predictions, and ultimately a solid insight into your social performance.