How to use modern language models for enhanced sentiment analysis

Sentiment analysis promised to unlock the secrets of customer feedback, but outdated methods have left us with shallow insights and oversimplified charts. With the rise of advanced language models, it’s time to break free from these limitations and revolutionize how we understand customer sentiment — uncovering the context, emotion and intent that drive opinions.

From Italian flags to wiggly lines: The story of sentiment analysis

Sentiment analysis has been around for a long time. Most companies use some version to analyze large volumes of text — whether from social media posts, survey responses or website comments. 

Typically, each snippet of text is categorized as “‘”Positive,” “Negative” or “Neutral.” Results are often displayed using a horizontal chart with green, white and red stripes. They represent the proportions of positive, neutral and negative comments, leading to the classic “Italian flag” sentiment display. Chances are, your organization has used similar displays before. They may be interesting, but they often provide limited value.

Another common approach is charting the percentages of positive and negative sentiment over time, which results in two wiggly lines that, while visually engaging, offer little actionable insight. Early on, some companies even attempted to create a “sentiment NPS” by subtracting the percentage of negative comments from positive ones — yet another wiggly line and yet another measure of limited use.

Despite these shortcomings, sentiment analysis hasn’t been entirely ineffective. Its real potential lies in going beyond simple positive and negative percentages, which are often too volatile to be meaningful. 

Sentiment categories can serve as a starting point for deeper analysis. For example:

• What topics are people with positive sentiments discussing? 
• How do these differ from topics mentioned negatively? 

Visualization techniques like butterfly charts, which contrast topics mentioned positively on one side and negatively on the other, can highlight actionable insights to improve the customer experience.

This likely feels familiar. For over 15 years, the promise of extracting valuable marketing insights from open-ended comments has been enticing, yet its practical value has remained limited.

The good news is that recent advances in language models make it possible to revisit sentiment analysis in new and exciting ways. With these modern tools, we can uncover what drives customer perceptions of our products and services.

The limitations of traditional sentiment analysis

Before exploring the new capabilities offered by modern language models, it’s essential to understand the limitations of traditional sentiment analysis. Without this understanding, we risk repeating the same problems with new technology. Three primary issues limit the effectiveness of conventional sentiment analysis:

Lack of context

Traditional sentiment analysis typically works with text snippets, often failing to consider the context needed to interpret meaning accurately. For example, take the comment, “You guys are unbelievable.” 

Without context, the sentiment could be positive or negative. The sentiment is likely negative if the person rated their satisfaction with your service as 1/5. If they rated you 5/5, it’s probably positive. Context is critical, yet traditional sentiment analysis rarely incorporates it.

Ambiguity in the neutral category

One reason for the volatility of positive and negative sentiment percentages is the neutral category, which often combines two very different groups: 

• Genuinely neutral comments.
• Comments where sentiment cannot be categorized. 

These groups should be treated separately, but traditional systems fail to distinguish between them. Many providers avoid addressing this issue due to the complexity involved, leaving this ambiguity unresolved.

Oversimplifying human expressions

Categorizing text as either positive or negative oversimplifies the complexities of human language. A single comment can serve multiple purposes or express conflicting emotions, making a binary approach insufficient. This limitation forces frequent reliance on the Neutral category, which further diminishes the accuracy and value of the analysis.

Understanding these limitations is essential to building better systems that avoid the pitfalls of traditional sentiment analysis.

Dig deeper: How to augment market research and glean customer insights with AI

How modern language models are changing the game

Modern language models can revolutionize sentiment analysis, but only if we avoid replicating the flaws of traditional methods. Creating a sentiment analysis system using a modern large language model (LLM) is surprisingly easy, yet it still encounters the same limitations.

For example, consider the following prompt:

• “Please categorize the following statement as Positive, Negative or Neutral. Give me only the single word category: ‘You guys are unbelievable.’”

When using Gemini 1.5, the response is:

Although an LLM powers this sentiment analysis system, its output is as limited as traditional approaches. It’s simply easier to implement.

To truly harness the potential, we must acknowledge the limitations of traditional sentiment analysis and strive for more advanced solutions. Here are some ways current models can address these challenges:

• Include context: Sentiment analysis should incorporate contextual information, such as satisfaction ratings, prior interactions or other relevant data, rather than analyzing isolated text snippets.
• Adopt a more sophisticated categorization scheme: Move beyond basic positive, negative and neutral labels to include categories like frustration, admiration, gratitude or sarcasm — tailored to what matters most for improving customer experience.
• Focus on purpose, not just emotion: Analyze the intent or mood behind statements, whether descriptive, sarcastic, uncertain or informational. Understanding these nuances can lead to deeper insights.

A new wave of sentiment analysis is emerging, one that goes beyond the outdated “Italian flag” visualizations. By using modern tools effectively, we can ensure our sentiment analysis delivers actionable, business-relevant insights.

Dig deeper: From sentiment to empathy: understanding how customers feel