On-site support changes

• Dropped support: KVM

• Dropped Support: VirtualBox

• Deprecation notice: QuickLearn 1.0

• API client: New features available

Science 

• Science version 2.2.1: Improved handling of an English idiom with negation 

New features

• Concept-Level Sentiment

• API: Concept-Level Sentiment

Enhancements

• Project management: Update to build emails

Fixes

• Settings: When logged in with SSO, a password is not required to create an API token 

• Concept details: Matching documents section now displays up to 50 matches

• Daylight: Reordered feature list

• Highlights: Copying a project now displays correct messages

• Search: If a search query includes only stopwords, now no results are returned

• Search: Search results now highlight exact matches rather than a conceptual match collocation 

• API: Long-lived tokens no longer disappear when signing in with a mixed-case username

• Project management: Projects are no longer inaccessible if a saved concept includes only stopwords

Known issues

• Science: Upgrading science does not rebuild project with Concept-Level Sentiment analysis if no GPU is available

Dropped support: KVM

With this release, KVM is no longer available as an option for hosting your Luminoso on-site instance. 

Dropped support: VirtualBox

With this release, VirtualBox is no longer available as an option for hosting your Luminoso on-site instance. 

Deprecation notice: QuickLearn 1.0

In fall 2019, Luminoso released a 2.0 update to our core science, QuickLearn. We are now preparing to drop support for QuickLearn 1.0. If you access a project that uses QuickLearn 1.0 science, a warning message notifies you that support for QuickLearn 1.0 projects is deprecated, and that you should rebuild to get access to newer science.

Open

Rebuild any project with out-of-date science using the Upgrade science button on the Project details card in the Highlights feature. 

Science version 2.2.1: Improved handling of an English idiom with negation 

With this release, we update Daylight’s science to better handle an idiomatic English-language construction. Previously, when Daylight encountered a sentence with the word “could” followed by a comparative adjective, like “I couldn’t be happier!” it the resulting concept would be parsed as “not|en happy|en.” 

English speakers understand that this phrase means it’s not possible to be more satisfied and is a positive construction, but Daylight interpreted it as a negative construction. 

Now, when Daylight encounters a construction like “I couldn’t be happier!” it categorizes “could not” as a stopword, which is not included in analysis, and correctly parses the words as “happy|en.” 

This change only affects English-language projects. 

New Feature: Concept-Level Sentiment

We’re excited to introduce Concept-Level Sentiment analysis to Luminoso Daylight. Concept-Level Sentiment uses deep learning to intelligently detect sentiment for all concepts in every Daylight document. 

Previously, Daylight detected sentiment in every document as a whole, but did not provide insight for each concept’s sentiment. With this major step forward, Daylight includes sentiment technology that is on par with the industry gold standard, all as part of the app – still with no coding or ontologies required. Head to Science Explained: Sentiment for more information on Concept-Level Sentiment in Daylight 

Use Concept-Level Sentiment to: 

• Catch buried positive and negative sentiment items that other tools miss

• View positive, negative, and neutral sentiment examples for each concept

• Export from the Sentiment feature and see more sentiment examples for each concept

Upgrading projects to Concept-Level Sentiment 

Daylight on-site users may access to Concept-Level Sentiment with Luminoso on-site version 3.0, available 2020-08-31. After installing v3.0, access Concept-Level Sentiment in projects by: 

• Clicking the new Upgrade science button on the Project details card on the Highlights page. 

• Creating a new Daylight project or adding new documents to an existing project. 

Concept-Level Sentiment is calculated after the core Daylight build. Since the time it takes Sentiment to build depends on dataset size, it may take several hours to complete large datasets. While Sentiment is building, you can’t access the Sentiment feature, though you can explore all other Daylight features. 

To upgrade an existing Daylight project to use Concept-Level Sentiment

1. Open the project you wish to rebuild with Concept-Level Sentiment. The project opens to the Highlights feature.

2. Check to see:

   1.  if the project uses Document-Level Sentiment. If it does, you see a banner at the top of the page asking you to upgrade your project. 

   2. If your on-site installation has a GPU available. If not, a message appears in the Sentiment feature telling you to attach a GPU to your Daylight on-site image.

3. Click Upgrade science on the Project details card on the Highlights page. A confirmation window appears, notifying you that your project will be rebuilt with upgraded science.

4. Click Upgrade. Your project begins to upgrade and a bar displays the progress.
   Note: Because Sentiment in Daylight is calculated after the core build is done, you may have to wait to access the Sentiment feature after the rest of the build.

To use the Concept-Level Sentiment Feature

1. Open a fully built project that uses Concept-Level Sentiment. The Highlights feature opens.

2. Take a look at the What concepts do people feel strongly about? card to view initial concept sentiment suggestions.
   Note: Occasionally, Daylight is unable to identify any Sentiment suggestions in a project. If this happens, this card and the Sentiment feature display an error message, and you can still view sentiment for selected concepts. 

3. Open Sentiment by clicking Sentiment feature on the What concepts do people feel strongly about? card in Highlights or the Sentiment icon. The Sentiment feature opens. By default, the visualization displays sentiment suggestions, which are the top 100 strongest sentiment concepts organized by highest prevalence. 

4. Use the arrows at the top of the concept list to sort concepts alphabetically, by number of exact matches, percent negative, or percent positive. 

5. Change the concepts displayed in the concept list using the options in Configure sentiment visualization. 

6. Click on an interesting concept to view examples of how the concept is used in contexts with negative, positive, and neutral sentiment.

7. Note: If Daylight can't confidently determine that a concept was used in a positive, negative, or neutral context, it does not display a corresponding example.

To conduct deeper analysis in spreadsheet exports

1. Apply filters as desired. 

2. Select a concept and click Download more matches to explore more matching documents. A spreadsheet downloads with documents based on:

   • The specific concept(s) you have selected

   • The filters you have selected.

   • The type of concept you have selected – Top concepts, Sentiment suggestions, or Saved concepts

3. Open the spreadsheet. It opens automatically to the Sentiment distribution tab. 

4. Use the Sentiment distribution tab to: 

   • Browse sentiment information for the total product

   • Skim sentiment information using color-coded concepts

   • View sentiment information for all concepts included in your filter

5. Use the Sentiment documents tab to view all documents that were included in your filter, and their associated sentiment information.  

Other major Sentiment-related updates to Daylight

Some Concept-Level Sentiment changes occur outside the Sentiment feature: 

• Project management – Now, the Sentiment feature builds after the rest of the Daylight project. Optionally, receive emails that notify you when Sentiment is done calculating.  

• Highlights card – The What concepts do people feel strongly about? card now displays information about concepts, not documents, that communicate strong sentiment. 

• Update to build emails – Now, build emails reflect the build status for your overall project and for the sentiment feature. 

• Sentiment export – Now, if a search is applied at the time of download, the Sentiment export contains two tabs. The second tab contains Sentiment Documents,

• Sentiment export – Advanced concepts are now included in the sentiment export. THe export represents the same document in multiple rows so that each concept-document pair displays correct sentiment. 

• Sentiment export – When you click a concept in the Sentiment feature and click Download more matches, the XLSX no longer shows summary rows for positive, negative and neutral matches, since these rows only displayed Document-Level Sentiment information.

• Download Documents – The Download Documents export no longer displays sentiment information, since sentiment is now calculated on a concept-document pair basis, and not document- level basis. 

• Concept details pane – Now, there is no option to select Sentiment documents in the Documents section of the Concept details pane, since Daylight no longer calculates sentiment on the document level.

API: Concept-Level Sentiment 

API users will also experience changes as part of Concept-Level Sentiment:

• Sentiment information in the API is not available until sentiment finishes calculating 

• Now, the get_concept_sentiment endpoint returns sentiment_counts and sentiment_shares, rather than only has always returned sentiment_share which is the % of positive negative neutral docs -- a concept “purchase” would turn up the % of positive documents, etc

• To update, use the rebuild endpoint in the API to rebuild an existing project without adding documents 

• Scripts which rely on knowing when the Sentiment build is done are going to have to be updated, as we will now be calculating sentiment after the regular build is complete

Additionally the newest version of the API client, version 2.0.1, now includes a wait_for_sentiment_build() method you can use if Concept-Level Sentiment is enabled in your Daylight Cloud Environment. Since the new Concept-Level Sentiment science is calculated after the core project build, you can use this new command in the same way the existing wait_for_build() method waits for the core project build to finish, wait_for_sentiment_build() waits for the Concept-Level Sentiment build to finish. Since Concept-Level Sentiment always finishes after the core project build, this is useful for scripts that use the API to build a project and then get sentiment data from it.

Project management: Update to build emails

Now, when you build a project in Daylight and indicate that you would like to receive a confirmation email, the subject is updated from:

Luminoso Daylight Job Succeeded ([Project Name])

to

Explore your Luminoso Daylight project: [Project Name]

Additionally, the email’s body now reads:

The Luminoso Daylight project [Project Name], begun at [time/date], is ready to explore.

To view the project, go to:

http://url-for-the-project

If you have any questions, send an email to support@luminoso.com.

Settings: When logged in with SSO, a password is not required to create an API token 

Before, Daylight prompted users who were logged in to Daylight with their organizations’ SSO for a password when the user used the Settings page to create a long-lived API token. 

Now, users who are connected to SSO don’t need to enter a password to create a long-lived API token.

Concept details: Matching documents section now displays up to 50 matches

Now, when you select exact or conceptual matches in the Matching documents section of the Concept details pane, Daylight displays up to 50 matching documents of that type, if available. Previously, Daylight would only display matching documents from a total combined set of 50 matches, meaning that fewer matches were available if you selected only exact or conceptual.

Daylight: Reordered feature list

Now, the feature icons in the sidebar in Daylight are also reorganized to the following order: 

• Highlights

• Volume

• Galaxy

• Drivers

• Sentiment

This change helps to guide users more intuitively through the application. Each Daylight feature pairs well with another feature for various stages of analysis. When initially discovering concepts, some users may prefer the visual Galaxy view, while others may prefer to look at the metric-oriented Volume feature. Some users might prefer to explore the spirit of a dataset through Sentiment, and others might consult quantitative Drivers metrics.

Along with this update, we also reorganize the cards on the Highlights feature. Now, the cards are ordered as follows: 

• Which concepts are most prevalent? — This card displays a snapshot of the Volume feature. 

• What are the largest clusters of conversation? —This card displays a snapshot of the clusters you view in the Galaxy. 

• What issues are affecting [score field]?  —This card displays suggested drivers based on score fields included in your metadata.  

• What do people feel strongly about? — This card displays concepts Daylight detected as displaying strong sentiment

API Client: New features available 

With this release, there are two new features available in the API client:

• Now, when you create an instance of the API client, you can specify a timeout option. Before, if you made a call to the server with the API client that never returned, the script would wait forever. Now, if you set a timeout, it raises an exception if it has not received a response within the amount of seconds for which you specified the timeout.

• If Concept-Level Sentiment is enabled in your Daylight Cloud environment, you can use a new wait_for_sentiment_build() method. Since new Concept-Level Sentiment science is calculated after the core project build, you can use this new command in the same way the existing wait_for_build() method waits for the core project build to finish, wait_for_sentiment_build() waits for the Concept-Level Sentiment build to finish. Since Concept-Level Sentiment always finishes after the core project build, this is useful for scripts that use the API to build a project and then get sentiment data from it.
  Note: If you use wait_for_sentiment_build() method in the updated API client before Concept-Level Sentiment is enabled for you, you will receive an error message.

Sentiment Concept List: Now displays more than 50 top concepts 

Previously, when you selected Top concepts in Configure sentiment visualization and selected a number other than 50, the Sentiment Concept List would not be affected. Now, the Concept List accurately updates to reflect the concepts you selected. 

Highlights: Copying a project now displays correct messages 

Before, when you applied a filter in any analytic feature to narrow the set of documents, and then copied the project through the Highlights feature, a modal would appear and notify that you were only copying that narrow set of documents, not copying all documents as you would expect. This error was cosmetic, as the project copied with all documents as expected.

Now, the modal that appears when you apply a filter and then copy a project through Highlights notifies that you’re copying all your documents and gives you confirmation when the copy is complete.

Search: If a search query includes only stopwords, now no results are returned

When projects are created, Daylight ignores some words. These are usually stopwords, meaning that they do not hold significance for linguistic analysis. Previously, if you searched for a word that did not qualify as a concept in Daylight, the search results would display all documents. Now, the search returns no results. 

Search: Search results now highlight exact matches rather than a conceptual match collocation 

Previously, the search results for a single concept in Daylight, like “cherry,” might highlight a conceptual match, like “tart cherries,” in the search results. Now,  a concept like “cherry” is appropriately highlighted as an exact match, and “tart cherries” is displayed as a conceptual match. 

API: Long-lived tokens no longer disappear when signing in with mixed-case username

Previously, a user who signed in with a mixed-case username (not all lowercase) experienced long-lived API tokens disappearing only a few hours after they were created. Because the capitalization between the user’s ID in the database and in the API tokens didn’t match, the tokens were deleted. Now, token lookups are case-insensitive. 

Project Management: Projects are no longer inaccessible if a saved concept includes only stopwords 

Previously, if a saved concept includes only stopwords (which might occur if a user entered words that Daylight did not recognize as a concept into the search bar and then saved them) a project was not accessible. Now, the project can be explored as usual.

Science: Upgrading science does not rebuild project with Concept-Level Sentiment analysis if no GPU is available

Some users may see a message in your Daylight project that notifies:

“This project's science calculates sentiment on the document level. Use the Upgrade science button on the Project details card in Highlights to access Luminoso's latest science, which includes Concept-Level Sentiment.”

Open

However, if there is no GPU available to the machine that hosts Luminoso Daylight, the project’s science will rebuild when users press the Upgrade science button, but Daylight will not be able to conduct Concept-Level Sentiment analysis. 

This message displays for all users, regardless of GPU configuration. Use the v3.0 Installation and Update guide for instructions on how to add a GPU to your on-site installation.