New features

• Introducing a new single sidebar 

• Collaborative concept lists

  • Active concepts and shared concept lists

Enhancements

• Drivers: Save concepts using + icon 

• Project management: Removed bottom left downloads icon

• Project management: Select or deselect all suggested concepts

• Highlights: Updated card order

• Science: Data updates may affect prevalence in Daylight

API updates

• API: Added result_count field to the Get documents endpoint

• API: v5 endpoints now available for profile, user, organizations, and workspaces functionality

• API: Use a password to retrieve a new API token through the API 

• API: Shared concepts in branched and copied projects use the same concept ID

• API: New endpoints for shared concept lists 

• API: v5 deprecation notice for Saved concept endpoints

• API: v4 deprecation notice for Tokens endpoints

• API: v4 deprecation notice for Users endpoints

Fixes

• Sentiment: Concepts that contain both positive and negative sentiment no longer appear twice in concept lists 

New features

Introducing a new single sidebar 

In Daylight, analysis is a journey. Different users may prefer different features, but wherever you start, one thing is clear: Daylight analysis tools are most effective when they work together.  

That’s why we redesigned the UI to consolidate all project management and configuration tools into a single, left-hand sidebar. This change helps increase parity across features, enhances concept management, and provides more space for visualizations.

Open

The new sidebar contains six tabs with icons. Each of these tabs provide functionality similar to what you could find before in the UI. Hover over an icon to see the tab’s full name. Additionally, we moved the search bar to the top of the visualization space.

Skim the high-level overview of each tab below, or scroll farther for more in-depth details:

• Search details – View exact, conceptual, and excluded match  information for concepts by clicking a concept name in the feature or entering it in the search bar.
  Note: You must use the search bar to view details for multiple concepts at the same time, unless you have saved a previous advanced concept search.

• Active concepts – Use the Active concepts tab to view and manage your active concepts and to create and edit shared concept lists. Now, you can also bulk select and bulk delete concepts from this tab.  

• Configure visualization – Use visualization options to select, if available, what concepts to visualize, whether to use unique to this filter, whether to include conceptual matches in match numbers, and other visualization criteria, depending on the feature.
  Note: This tab is not available in the Galaxy. 

• Filter documents – Use your project’s metadata to filter the concepts that you see in visualizations. Click Expand all to open all available filters. 

• Documents – Explore relevant documents in depth. If you’re viewing documents for a specific concept, you can use the Exact and Conceptual icons to view only that type of document. Click Download to view all matching documents in a spreadsheet. Click create project to create a project that includes only documents that match your concept and filter configurations.

• Export & link – Export different kinds of data from your project, or send a link the feature you’re viewing. 

Open

Search details

To see information in the Search details tab, either enter a search in the search bar or click a concept to select it. Then, the tab displays total, exact, and conceptual match information for the active search.

If you use a minus sign to remove a concept from your search results, like “coffee, -tea,” you also see an Excluded matches section.

Active concepts

Use the Active concepts tab to view and manage suggested or active concepts, and access shared concept lists. In a new project, click Add to active concepts to begin analysis with concepts that Luminoso’s AI suggested, or Dismiss to clear them. To begin using only some of your suggested concepts, clear the check boxes to dismiss the concepts you don’t want and click Keep selected when you’re done.

View all suggested concepts again by clicking the three dots at the top of the Active concepts pane. Daylight identifies these concepts as potentially relevant based on their prevalence and relationship to other concepts in your project. To edit an active concept, select it and click the pencil icon in the search bar.

Additionally, use the Active concepts tab to create shared concept lists based on your active concepts, or start a new active concept list using a copy of a shared concept list. Read more on active concepts use in the Active concepts release note section.  

Filter documents

Use the Filter documents tab to select filters that change what data displays in the feature view. When you select a filter, it remains consistent across features. This is also true if you share a direct link to the page you’re viewing with another Daylight user.

View the number of total documents in your filter in the Current filter section. Use the reset button to clear all filters you’ve applied.

Configure visualization

Use the Configure visualization tab to select concepts to view in visualizations, count matching documents, and select feature configuration options. This tab is not available in the Galaxy. The options you find in this tab vary slightly by feature.

Additionally, you can access the Unique to this filter concept selection feature in the Configure visualization tab.

Documents

In the Documents tab, view documents that match any concept and filter selections you have applied. When a concept is selected, the tab displays only documents that contain matches for that concept. When filters are applied, the tab displays only documents that match the filters.

Click Download to download a spreadsheet that contains all documents that match your concept and filter selections, or click create project to build a new Daylight project based on your concept and filter selections.

Export & share

Use the Export & share tab to manage your project’s exports.

Click Copy link to copy a direct link to the project feature (Volume, Sentiment, Drivers, or Galaxy) that you are viewing to your clipboard. You can share this link with other Daylight users or use it to access the same view later.

In the Export current view section, click Export to download an Excel spreadsheet that contains the data for the feature and configuration that you’re viewing. This spreadsheet reflects any filter information that you have applied.
Note: This option is not available in the Galaxy.

In the Export raw data as spreadsheet section, click Export to download one of the options:

• Current documents – Downloads a spreadsheet that contains all documents included in your search and filter configuration. After clicking Export, choose Total matches to download both exact and conceptual matches, or Exact matches only to download only exact matches for your search. Read our extended documentation on the Downloads tab for detailed information on the selections that affect your download.

• Concept-Filter association scores – Downloads a spreadsheet that includes the association scores between your active concepts and metadata filters.

• Concept-Concept association scores – Downloads a spreadsheet that contains the association scores that each active concept has with every other active concept. 

Search bar moved to the top of visualization space

As part of our series of single sidebar changes, the search bar is located above feature visualizations in Volume, Galaxy, Drivers, and Sentiment.  This change helps make your Daylight experience more universal as you move from feature to feature. 

Now, when you search for a concept or select it from a feature visualization, the concept appears in the search bar. If the concept you select is active, the edit active concept icon appears next to the search bar.

If the concept is not in your active concept list, an Add active concept icon appears. Use the Search details and Active concepts tab to view relevant search information or what concepts are currently active.

Collaborative concept lists

Luminoso is excited to announce Collaborative Concept Lists, the most recent set of usability improvements to Daylight. This change helps improve how users save and organize concepts in Daylight. It combines two changes –  active concepts and shared concept lists – that work together to improve concept management and exploration. 

Before this release: 

• You could save only one set of concepts per project, for all users.

• When one user changed the saved concept list, it updated for all users.

• You could use saved concepts to populate visualizations.

• You interacted with saved concepts in the Select concepts tab

After this release:

• Your existing saved concepts (if any) are converted to a shared concept list named with the date that the list was converted on. 

• Anywhere that Daylight previously referred to “saved concepts,” you’ll now see “active concepts.

• Active concepts are applied for a single user.

• You can save your active concepts as a shared concept list to come back to or share with other users.

• Any user can save as many shared concept lists as they want. 

• Shared concept lists are accessible by any user who has access to the project.

• You interact with active concepts in the Active concepts tab, in the same location as the Select concepts tab.

Active concepts and shared concept lists

Using active concepts is straightforward and is very similar to how you used saved concepts before. Add any concept to your active concept list by selecting a concept from a feature or the search bar, or by opening and editing an existing shared concept list. Then, save and share any group of active concepts you’d like to keep. 

Active concepts: 

• Allow users to experiment with concepts in a private scratch space. 

• Encourage exploration of Daylight without fear of losing concepts or making permanent changes.

• Let view-only users load shared concept lists and manipulate their own active concepts.

• Are persistent after logging out and returning to a project.  

Shared concept lists:

• Provide better organization of concepts with Shared Concept lists. 

• Allow exploration of a single project using different lenses, such as service or emotion.

• Enhance in-project collaboration for projects with multiple users. 

• Separate interesting dataset subsets without needing to create multiple Daylight projects. 

Concepts remain in your Active concepts list until you delete them or load a different shared concept list. You can add the same concept to multiple shared concept lists. For instance, if the concept “smell” was relevant to two users’ lists, both users could add that concept to their lists. Concepts with the same name that are in multiple lists don’t update universally, since the API assigns every concept in a shared list a unique concept ID number.

Changes that you make in your active concept space are not visible to other users until you save them to a shared concept list. When you select Active concepts in the Configure visualization tab, your current active concepts populate the information in Daylight’s analysis features, like saved concepts did previously.

Active concepts are persistent across features. Additionally, whatever active concepts you had loaded when you log out of Daylight are now present when you log back in. 

Once you have one or more shared concept lists set up, you can copy one into your active concepts list to use its concepts in visualizations or to make edits to its contents.

You can create as many shared concept lists as you need in a project. Naming shared concept lists is flexible to meet your analysis and team needs. 

To create a shared concept list with active concepts 

Use Shared Concept Lists to create a better organization system for your concepts in Daylight and to improve your collaboration with other Daylight users. It’s easy to start using Active Concepts:

1. Click Add to active concepts in the Active concepts tab to start a new set of active concepts.

2. Add concepts to your Active Concept List until you have a list you want to save and share to other users. These Active Concepts aren’t visible to other users yet, but will stay in your Active Concept List until you delete them or load a different list.
   Note: An Active Concept List with unshared changes displays Unshared changes in italics in the concept list row. 

To share and save your active concepts

1. Open the Share active concepts dropdown, enter a name for your new list, and click Share. The list is saved to the project, and other users can now also work with it.
   The section that previously read Unshared changes now displays the name of your new shared concept list.

2. Save a concept to as many concept lists as you want to. When you save a concept to a Shared Concept List, Daylight generates a new concept ID for it, so you can update that concept separately within the Shared Concept List.  

To import a shared concept list

Move a copy of the contents of your shared list into your Active concepts list. 

1. Click Import shared concept list and select the name of the shared concept list you want to open from the menu. 

2. Click Open. A message appears informing you that importing a list will overwrite your active concepts.

To edit a shared concept list

1. Open the Shared Concept List you want to edit.

2. Open the dropdown under the concept list’s name in the top panel. 

3. Use this section to delete, rename, or update the list with any changes you’ve made to the concepts in it.

4. After opening a list into your Active concepts list and editing it, click Update “Concept_List_Name” with active concepts to save an updated version of the list. 

Enhancements

Save concepts using + icon in Drivers

Now, when you click the + icon to the left of a concept’s name in the Drivers concept list, the concept is added to your active concept list.  

This change increases feature parity, since this ability was already available in Volume and Sentiment. 

Removed bottom left downloads icon

With this release, we remove the downloads icon from the bottom left corner of the app.

We also remove the Match counts download entirely, since the same information is available through opening Volume, selecting the Export & share tab and clicking Export in the Export current selections as spreadsheet section.  

Now, the Concept-Filter and Concept-Concept association scores downloads that you previously accessed through the bottom left download icon are also available in the Export & share tab in the left sidebar.

Select or deselect all suggested concepts

With this release, we make it possible to select, or deselect, the entire contents of the Select concepts list by checking or unchecking the Select all box.

Highlights: Updated card order

With this release, we update the ordering of the cards on the Highlights feature. Now, feature summary cards are at the top of the page. This change helps orient you to the insights in your project first.  We also removed the What concepts have I saved? card to reflect the new Active Concepts feature, since users no longer save individual concepts. 

Science: Data updates may affect prevalence in Daylight

The data Daylight uses to determine prevalence have been expanded and updated, and may affect the prevalence rankings you see in Daylight. Prevalence is Luminoso’s term for how frequently a word appears in a specific dataset versus how frequently the word appears in a language as a whole.

As a result of the data expansion, prevalence numbers in Daylight now more accurately reflect a word’s use in a given language. 

Since this is a change to Luminoso’s science, it will only affect projects that are created after the change went into effect, or projects that are rebuilt or updated after that date. 

API updates

Added result_count field to the Get documents endpoint

With this release, we added a result_count field to the Get documents endpoint. This new field displays the number of documents that match any filter and search you applied when you called the Get documents endpoint. 

Note: If a search is applied, only documents that contain exact or conceptual matches for that search are included in results.

For more on the Get documents endpoint, read its section in the API documentation. 

v5 endpoints now available for profile, user, organizations, and workspaces functionality

With this release, all endpoints related to the profile, user, workspaces, and organizations are available to users in the v5 API. Previously, this functionality was available in the v4 API. Now, you can do all of your user, workspace, and organization management through the v5 API. We recommend that you access this functionality through the v5 API as we transition towards deprecating the entire v4 API. 

The new endpoints are listed below. Click the link in each section to read more about each endpoint and its functionality in the API documentation: 

Profile

• Get user profile (GET /api/v5/profile/)

• Modify user profile (PUT /api/v5/profile/)

• Change password (PUT /api/v5/profile/password/)

Users

• Delete users (POST /api/v5/users/delete/)

Organizations

• Get organization info (GET /api/v5/organizations/<organization_id>/)

• Get usage for an organization (GET /api/v5/organizations/<organization_id>/usage/)

• Update organization details (PUT /api/v5/organizations/<organization_id>/)

• Edit users’ roles in organization (PUT /api/v5/organizations/<organization_id>/users/)

Workspaces

• Create workspace (POST /api/v5/workspaces/)

• Get workspace info (GET /api/v5/workspaces/<workspace_id>/)

• Delete workspace (DELETE /api/v5/workspaces/<workspace_id>/)

• Update workspace details (PUT /api/v5/workspaces/<workspace_id>/)

• Remove users from a workspace (POST /api/v5/workspaces/<workspace_id>/users/remove/)

• Edit users’ roles in workspace (PUT /api/v5/workspaces/<workspace_id>/users/)

API: Use a password to retrieve a new API token through the API client

With the newly-released version 2.1.0 of the Luminoso API Client, you can obtain and save a long-lived token directly through the client. Previously, you could only do this through the Settings page in the UI.

With this release, we introduce a new command-line tool, lumi-save-token, with the same behavior as the existing interactive client method LuminosoClient.save_token(). Using either method, you can specify and save a token that you obtained through the UI.  If instead you do not specify a token to save, you will be prompted for your username and password, and the API client will obtain and save a long-lived token for you. The token is persistent, and you don’t need to update it.  

API: Shared concepts in branched and copied projects use the same concept ID

With this release, if you branch or copy a project in Daylight, shared concept lists, and all concepts in them, retain the concept ID that they had in the original project. 

API: New endpoints for shared concept lists 

With this release,  we add new API endpoints to match the shared concept list functionality in the UI. Read the details in the Shared concept lists section of the API documentation.

The new endpoints are: 

• List shared concept lists

• Get shared concept list details

• Create shared concept list

• Edit shared concept list details

• Delete shared concept list

• Add concepts to a list

• Delete concepts from a list

• Reorder concepts in a list

• Modify selected concepts within a list

You can use these new endpoints to create, edit, delete, and retrieve shared concept lists. This new set of endpoints replicates the functionality that the Saved concept endpoints provided, except that concepts are returned as an array of concepts in the "concepts" field instead of a simple array. 

You can use these endpoints before a project is built, since they don’t interact with a project’s science. 

Finally, each concept in a shared list has a unique concept ID.  Even if a concept’s name, texts, and color are identical to another concept in another list, the two concepts will have unique IDs. 

API: v5 deprecations for Saved concept endpoints

With this release, we announce the deprecation of all Saved concept endpoints and “saved” as a type selector in all Concept endpoints. This deprecation is targeted for Luminoso on-site release v4.0 in August 2021. Read more in the Deprecations section of the API documentation. 

The endpoints being deprecated are: 

• Get saved concepts 

• Create saved concepts 

• Modify saved concept order 

• Delete saved concepts 

• Update saved concepts

These endpoints will continue to function until their sunset date in June 2021, but we recommend beginning to transition your use to the new shared concept endpoints. 

Though Luminoso strives to maintain backward compatibility at all times, the changes to this endpoint as part of shared concept lists, included in this Luminoso on-site v3.1 release, may create a possible breaking change if your organization uses the API and UI simultaneously.

Saved concept endpoints no longer point to the concept list previously displayed in the Saved concepts tab of the UI. Now, they point to a specific shared concept list named “Saved Concepts” (capitalization-insensitive) instead.  See the Shared concept lists section of the API documentation for details.

API: v4 deprecation notice for Tokens endpoints

With this release, we begin the deprecation of the following v4 API endpoints:

• List tokens (GET /api/v4/user/tokens/)

• Delete token (DELETE /api/v4/user/tokens/<token>/)

• Create token (POST /api/v4/user/tokens/)

• Get token (GET /api/v4/user/tokens/<token>/)

These endpoints continue to be available through this Luminoso on-site release (v3.1), and will be removed with the v3.2 on-site release, targeted for February 2021. For all token management needs, we recommend using the password method described in the previous section or the Settings page in the UI. 

API: v4 deprecation notice for Users endpoints

With this release, we are announcing the deprecation of the following endpoints with this on-site release (v3.1). 

These endpoints have equivalents in the v5 API:

• Change password (PUT /api/v4/user/password/)

• Modify user (self) (PUT /api/v4/user/profile/)

• Modify account info (PUT /api/v4/accounts/<account>/)

These endpoints are not replicated in the v5 API:

• Get user (GET /api/v4/accounts/<account>/users/<path:email>/profile/)

• Modify user (PUT /api/v4/accounts/<account>/users/<path:email>/profile/)

After conducting an internal audit, Luminoso saw that these endpoints were unused in the cloud for the past 180 days. Another deprecation notice will be provided before the deprecation.

Fixes 

Concepts with positive and negative sentiment no longer appear twice in concept list 

Previously, if Luminoso determined with high confidence that a concept had both strong positive sentiment and strong negative sentiment, it would appear twice in the list of sentiment suggestions in both the API and UI. 

Now, concepts that fit this description are only included once, which is more intuitive for users.