Science Explained: Sentiment in Luminoso

What is Sentiment?

Sentiment is a Luminoso Daylight^Ⓡ feature that examines your documents and reports the presence of sentiment – positive, negative, or neutral – separately for each concept within those documents.

Where other sentiment solutions only analyze at the document-level, Luminoso analyzes at the concept-level so that you don’t miss any nuance in your customers’ feedback. 

Sentiment works in all 15 languages that Luminoso supports. 

Using Sentiment in Luminoso Daylight

Sentiment model: Deep learning

To assemble the various views of sentiment in a project, Luminoso analyzes concepts and their contexts with a multilingual deep learning model. Unlike lexicon-based models, which only look at each word in isolation, deep learning models pay attention at numerous levels to create a complex representation of the concept and its context. These complex representations are then passed to our supervised sentiment classifier, which predicts a sentiment label for each concept-document pair. The model was trained on reviews of products and services and will work the best on such datasets.

Sentiment distribution: Concept-level analysis

Sentiment distribution is the share of positive, negative, and neutral sentiment about concepts in a dataset, described using three representative percentage values. Expressed as a combination, this provides a unique view into the mix of feelings around a particular word or phrase. Understanding a concept’s sentiment distribution is extremely valuable when analyzing datasets that contain no ratings or have a statistically insignificant number of rating responses.

Sentiment distribution is calculated by determining each concept’s sentiment label in all the documents it appears in, adding together the number of positive, negative, and neutral labels, and then calculating the percentage of each sentiment type over the total number of concept labels in the dataset.

Consider the phrase work-life balance in a sample group of HR survey documents:

“world wide reputation, industry respect, competent people all around, different opportunities, strong technical people, good senior management, good work life balance”

“They have fantastic, brilliant colleagues who are very supportive and fun to work with! Poor work-life balance.”

“Flexibility of schedule is good. Educational opportunities are good during times of less expense restrictions. Work/life balance is a joke as the expectations are to keep doing more with fewer resources. Total compensation compared to others is terrible.”

“This company can provide an environment supportive of work life balance especially in relation to those family events which occur from time to time. Extremely poor compensation. HR's sole focus is on reducing $ per head and this is reflected in the treatment of employees with respect to salary, pay increases and bonus payments.”

When uploaded, the application determines a sentiment label for each concept within those documents. Based on the subset of documents in which “work-life balance” appears, there are 2 negative, 2 positive, and 0 neutral labels of that concept. Therefore, the sentiment distribution for this concept is 50% negative, 50% positive, and 0% neutral. 

Sentiment suggestions

In the Sentiment feature, Luminoso displays a list of up to 50 concepts most significantly correlated with positive and negative sentiment in each project. This list of concepts contains words and phrases that are associated with strong sentiment, and not actual sentiment words, like “great”, “amazing”, or “awful”, which are inherently descriptors and don’t provide insights.

The number of suggested sentiment concepts may differ from project to project and from filter to filter since only concepts with statistically significant association with positive or negative sentiment are returned.

Frequently asked questions

Why is neutral sentiment’s percentage not displayed by default?

As Sentiment is designed to show concepts with the most positive and negative sentiment, neutral results are less interesting by comparison. Neutral sentiment can be viewed in the application by hovering over a concept or viewing a concept’s sentiment distribution in the results export. 

You can also view a neutral example of that concept by clicking on the relevant concept. 

What are the current limitations of Sentiment?

Wishes and hypothetical situations – Luminoso Sentiment cannot detect such subtlety, where only positive words are used to set a contrast with the not-so-positive reality. For example, consider the following document:

“If only the company offered a generous tuition reimbursement or a student loan assistance benefit. It would be sooo great to have assistance on student loan repayment. This would increase the level of talent and attract amazing candidates.”

Concepts such as tuition reimbursement and candidates are assigned positive sentiment, even though it’s clear to a human reader that the company lacks these benefits. Sentiment classification is currently unable to differentiate this tone from a direct answer.

Sarcasm. Sentiment cannot detect sarcasm, where vocal tone is needed to detect the actual meaning behind what is said. For example, consider this document:

“Way to go, another excellent choice made by our management team.”

Social nuances. Sentiment has no awareness of social nuances or norms. Consider this mobile gaming review document:

“Men seem to think this game is a dating site.”

Black box. As is the case with many deep learning models, it’s difficult to diagnose why the model is making a specific decision. 

Can I tune the sentiment model? 

No. Luminoso’s Concept-Level Sentiment model works at an extremely high quality compared to industry standards. In English, Luminoso’s model outperformed or nearly matched the gold-standard benchmarks on industry standard datasets, and achieved a similar level of quality in all 15 supported languages. 

Additionally, the model considers the context in which a term is used to determine its sentiment, helping ensure accurate, nuanced results in datasets across industries.

And the best part? Customers don’t need to provide any data, code, or tuning to make Luminoso’s sentiment analysis work. 

Are there industry-specific sentiment models available?

Luminoso’s sentiment model is broadly trained to produce high-quality results across any industry, including those with lots of specialized language, like pharmaceuticals or banking. Luminoso’s training and testing process determined that the model works at a high level of accuracy for any dataset. The model’s use of word context helps with accurate sentiment detection across industries. 

How do I use Concept-Level Sentiment? 

To take advantage of the new concept-level sentiment analysis, either create a new Daylight project, create a project from a subset of documents in an existing project, or upload new documents to an existing project.

Can I still see Document-Level Sentiment? 

Starting on July 25, 2020 for cloud users and in the next on-site release for on-site users, Luminoso offers only concept-level sentiment analysis. This model performs better on every level than the current lexicon-based document-level model. 

For the future, stay tuned – Luminoso is conducting active research to re-introduce document-level results using a deep learning approach.