Updated product support and service levels policy
Advanced Concept Search
API: Advanced Concept Search
Science: Consistency in Concept Detection
API: Documentation clarification for “limit” in concept clusters
On May 11, 2020, we updated our product support and service levels policy. The policy describes Luminoso Technologies’ support levels and priorities, response times, and service levels.
Now, the updated policy also includes information specific to Luminoso on-site users. Read Luminoso’s support policies here.
Advanced Concept Search makes searching for and combining concepts in Daylight more intuitive. Unlike compound concepts, which averaged together concepts’ vectors, advanced concepts preserve all vector relationships and matches. This helps users to better expect, understand, and verify advanced concept results. Advanced Concept Search:
Provides instant visual feedback on how each match type results from your search
Offers more intuitive conceptual matches, with up to 20 conceptual matches for each concept you include in a search
Can optionally exclude concepts that aren’t relevant to your search
Allows you to save an advanced search as an advanced concept (replaces old compound concepts)
On May 23, 2020, Advanced Concept Search is automatically accessible to all Daylight users.
If you became a Daylight client after May 23, 2020, Advanced Concept Search is always enabled for you, and you do not have access to this toggle.
With this release, any search you enter in the Selected concept search bar in the Concept details pane can use advanced concept search commands. For instance:
Use a comma “,” to separate concepts as an “or” statement. If you enter “coffee, tea, latte,” Daylight interprets the search as “coffee OR tea OR latte.” Your search results include any documents that contain exact or conceptual matches for any of the three concepts.
Use a minus sign “-” to exclude results from a search. You can’t search for only an excluded concept. If you enter “coffee, -tea,” Daylight interprets the search as “coffee NOT tea.” Your search results include any exact or conceptual matches for “coffee,” but exclude any documents in this set with an exact match for “tea.”
Advanced concept search affects sections of Daylight differently:
Highlights — If you save an advanced concept search as an advanced concept, you can now delete it through the What concepts have I saved? card.
Volume — You must save an advanced concept search to view it in the Volume feature. View it by selecting Saved concepts in the Configure volume visualization pane. Generally, advanced concepts will display higher volume than compound concepts would have.
Sentiment — You must save an advanced concept search to view it in the Sentiment feature. View it by selecting Saved concepts in the Configure sentiment visualization pane.
Drivers — You can view the result of an advanced concept search in the Drivers feature, or save it as an advanced concept for later reference. Advanced concepts generally result in more matches, so your advanced concept is likely to be farther to the right on the X-axis. Advanced concepts generally are closer to the mean of the average scores of each included concept on the Y-axis. Drivers may also be more precise, since advanced concepts find more closely related matches for each included concept.
Galaxy — The Galaxy’s heat map now updates based on however many concepts you include in a search. Unlike before, you cannot orient the Galaxy’s X- or Y-axes based on new advanced concepts.
Concept details pane — Use this section to search and save concepts. View information that you previously found in the Match counts and Top related concepts in detailed separate sections for exact, conceptual, and excluded matches.
It’s easy to use Advanced Concept Search. A search process might look something like this:
There are a few actions you can take with advanced concept search. Read more detailed instructions in the Advanced Concept Search quick reference article.
Toggle Advanced Concept Search on or off:
Open the Settings page from the dropdown carat next to Projects in the top ribbon. Use the toggle at the bottom of the User profile tab to switch Advanced Concept Search on or off. Click Save changes when you’re done with your selection.
If you don’t see this option, Advanced Concept Search is your only method of searching.
Search a single concept in Daylight:
Enter a concept like “coffee” into the search bar. View conceptual and exact matches for this concept. These results are the same as before, with or without Advanced Concept Search.
To include multiple concepts in a search:
Enter a concept like “coffee” into the search bar followed by a comma “,”. Add another concept, like “tea.” View full match results for each of the concepts.
Exclude a concept from a search:
Add another comma after “coffee, tea” and enter a minus sign “-” followed by a concept, like “-latte.” Press enter. Now, search results include all match results for “coffee” or “tea” that don’t include “latte.”
Save a search as an advanced concept:
In the Concept details pane, click +Save concept. The search saves as an advanced concept and appears above the search bar. Update the concept’s label, since it originally saves with the search terms as its label.
Edit an advanced concept:
Search for the advanced concept’s label in the Concept details pane or select it from Saved concepts in the Galaxy. The advanced concept appears in the Concept details panel. Click the edit icon next to the concept label. The Edit saved concept flyout opens. Use the options in the flyout to edit the advanced concept.
New Advanced Concept Search functionality is also available through the Luminoso Daylight API. For more information on how to use Advanced Concept Search in the API, read the Concepts section of the API documentation.
As part of this release, we also deprecate the API “vector” field in favor of “vectors.” Read more about this change in the Deprecations section of the API documentation.
With this release, we introduce a change to our science that helps Luminoso consistently understand and display any words in a project that look the same but have different meanings, such as “I left the room,” compared to “my left hand.”
In most languages, it’s common for certain forms of different words to be identical when written out. This meant that previously, you might have seen concepts in Daylight that looked the same but that Daylight had identified as having different meanings. Daylight kept these concepts separate in all visualizations, but when users clicked on the different versions, they could only access the most prevalent version of the word.
For instance, if you saw two concepts for “left” in the Galaxy and clicked on one of them, Daylight would show you the more prevalent sense of the word (“I left the room”). If you clicked on the second, it would begin to display the less common sense (“my left hand”), but then shift quickly back to the first sense (“I left the room”). Because of this behavior, separating the words didn’t help users understand them in their projects.
Now, for English language data, our updated science groups identical word forms together, regardless of meaning, and prioritizes the sense that most commonly occurs in the project in searches and visualizations. In the Daylight app, when you search for or select the word “left,” all instances of “left” appear together under the most commonly used form of the word, and you see all documents containing either version of the word.
Since non-English languages often have many inflected forms (for example, based on the gender of a noun or the declension of a verb), it’s easy for two words with many forms to accidentally overlap some written forms. For these languages, identical word forms that have different meanings are now all grouped under the most common sense. This may result in various forms of the same word being split up in Daylight.
For instance, the Italian verb “leggere” (“to read”) takes many forms, which include "legge" (identical to the singular form of the noun "law") and "letto" ( identical to singular form of the noun "bed"). In an Italian project, all forms of “legge” would be grouped together, and all forms of “letto” would be grouped together, even though both groups might include forms of the verb “leggere.”
We updated the API documentation to clarify how the optional field “limit” works in the context of concept clustering. Find the update and more information about this in the Concept Selectors section of the API documentation.