API Integration Scripts

The following are a set of scripts developed by the Luminoso Sales Engineering team that can enhance your Daylight use. If you’re interested in using any of the scripts listed below, contact your Luminoso CSM to receive the installation files.

All Luminoso sales engineering and example scripts come packaged in a wheel file. A wheel file is a Python archive format used to distribute python libraries, code and scripts. You must install the file in your Python environment, which you can do with a simple command. Then, you can run the scripts and tools that extend the Daylight UI functionality with the API.

Scripts are considered demonstration tools and not supported by the product team. The source code for all the tools and scripts are available from your Luminoso CSM, and you can use them directly and informationally.

You can use and modify any of the scripts at your own risk, since Luminoso does not support modifications. The tools are licensed under the MIT open source license.

There are two files in the distribution. The wheel file (se_code-0.2-py3-none-any.whl) and the source file (se_code-0.2.tar.gz).

Python installation

To use the se_code scripts you will first need Python. If you don’t have Python installed already, we recommend installing Anaconda, a specific Python distribution.

Simply run the installer to install Anaconda:

Installing Jupyter using Anaconda

Python Virtual environments

We recommend that you create a Python virtual environment, which is a self contained execution environment. You can have many environments for different purposes and easily switch between them. Alternatively, you can install the wheel file into the base Python environment without creating a separate virtual environment. However, Luminoso recommends virtual environments, since they're easy to clean up and modify.

To install the Luminos se_code file:

Open the folder where you want to install it.
Note: The path to the folder should not contain spaces, as this can cause problems for virtual environments on some systems.
Run:
python -m venv venv
You only have to run python -m venv venv once. You can always run source ./venv/bin/activate in the folder directory where you created the environment to “activate” the script again.
Run:
source ./venv/bin/activate

If you ever want to clean up or remove the virtual environment, simply remove the “venv” folder that was created with the python -m command listed above.

Installing the wheel file

Run:

pip install se_code-0.2-py3-none-any.whl

This command installs all the dependency libraries as well as the luminoso se_code library into the current Python environment. Now you’re ready to run the file.

Saving your Luminoso token

After updating to the wheel file, you'll need to save a Luminoso token that allows you to securely access the Daylight service. Use the following command:

After you enter the command, you'll be prompted to enter your username and password. Then, a new token is generated and saved in ~/.luminoso/tokens.json.
Note: You don’t need to know anything about that file, but if you ever wonder, that’s where they are stored. You can edit it manually if you want to, but it should not cause problems.

Running

The installation includes shortcuts to each of the scripts included in the se_code wheel file. The following is a list of tools that are included.

To get more information on any of these commands, run it from the command line with the option --help to output all usage information on that script.

For example:

$ lumi-bi-tool-export --help
usage: lumi-bi-tool-export [-h] [-c CONCEPT_COUNT] [-e ENCODING]
[-l CONCEPT_LIST_NAMES] [-sktl SKT_LIMIT] [-docs]
[-terms] [-theme] [-dterm] [-dsubset] [-skt]
[-drive] [-tdrive] [--driver_subset]
[--driver_subset_fields DRIVER_SUBSET_FIELDS]
[-sentiment] [--sdot] [--sdot_end SDOT_END]
[--sdot_iterations SDOT_ITERATIONS]
[--sdot_range SDOT_RANGE]
[--sdot_date_field SDOT_DATE_FIELD]
project_url

Export data to Business Intelligence compatible CSV files.

positional arguments:
project_url The URL of the Daylight project to export from

optional arguments:
-h, --help show this help message and exit
-c CONCEPT_COUNT, --concept_count CONCEPT_COUNT
The number of top concepts to pull from the project
……

Command	Description

Command	Description
lumi-add-concept-relations-to-project	Takes the concept relation information (tagging) and creates a new project with what doc-downloader uses to create a new project.
lumi-capitalize-saved-concepts	Capitalizes all the concepts in all the shared concept lists.
lumi-copy-shared-concepts	Copies shared concepts from one project to another.
lumi-create-train-test-split	Splits a Compass voting classifier file into a training and test set.
lumi-doc-downloader	Downloads documents from a project. This script can also tag each document with the shared concept lists with which it is associated, much like guided-tagging, but using shared concept lists.
lumi-format-multiple-text-fields	Takes a CSV file with multiple text fields and combines them into a single text field with a new metadata column identifying which column the text field was previously associated.
lumi-ignore-terms	Uses the /terms/manage endpoint which requires a project rebuild after setting the ignore terms. The rebuild is called at the end of this script. This ignores the terms scientifically.
lumi-list-metadata	Simply lists all metadata fields and how many unique values each field contains.
lumi-load-shared-concepts	Uploads a file of shared concepts to a daylight project. This script takes either a .csv or a .json file as input. There are two sample files included: load_shared_concepts_example.csv and load_shared_concepts_example.json.
lumi-onsite-invite-user	Invites uses to an onsite image. This script can be used to automate user creation with SAML integration on onsite installation environments.
lumi-onsite-list-users	Lists all the users on an onsite image. This script can be used to automate user creation with SAML integration on onsite installation environments.
lumi-onsite-usage	Downloads usage data for an onsite image. This script can be used to automate user creation with SAML integration on onsite installation environments.
lumi-project-migration	Copies all projects under a specific workspace to a separate server/workspace.
lumi-score-drivers	Downloads score drivers for a project as well as generate output for score drivers over time. It is also used by the bi-tools-export.py script.
lumi-subset-key-terms	This was the pre-cursor to “unique to this filter”. It is still relevant as it will download all tables of “unique to this filter” values. It is also used by the bi-tools-export.py script.
lumi-bi-tools-export	Business Intelligence Export. Creates CSV files for use with business intelligence tools like Tableau, Microstrategy and PowerBI to name a few. It includes many different tables and covers most of the Daylight tables and builds some extras as well.

Please direct any questions you have to your CSM.