Formatting your data

Now that you have an account on Daylight and access to a Workspace, the next step is to upload some text data to analyze. You can refer to the Using different data sources with Daylight page for some pointers on different types of text data to consider. You will also want to include some metadata to provide context for the text data, such as demographic information, dates, scores, etc. Metadata can be used to filter the data to analyze a subset of the data set, and numerical data can be used in Drivers analysis, which you will learn about later in this document.

Key vocabulary

Before we get started, here are some terms that we will be using throughout this section of the document.

A verbatim is the conversational text component of the sample you have collected.
A document is a row of your source data, including the conversational text and any associated metadata.
Metadata is structured data that creates context for text responses. Metadata may include demographics, dates, scores, or product details.
A CSV file, or comma-separated value file, is a plain-text file format that is used to organize data. CSV files exclude styling information that is included in an Excel XLS or XLSX file formats. You can export a CSV file from most spreadsheet editors.

Data Fields

The following table describes the types of data that can be included in the data to be uploaded. You will need to designate the data type for each column of the uploaded data during the uploading process. There are two ways to do this:

In your dataset file, make sure each header has the data type, then an underscore, then the column name. E.g. score_Rating, string_Location
As you upload your dataset file, you can change individual data types in Daylight.

Data type	Examples
Text (Required)	Column header: text or text_[FieldName]
The natural language verbatims for Daylight to analyze There must be one and only one Text column per file Each piece of Text may not exceed 500,000 characters in length	I loved the free coffee and the room was very clean, but it smelled strongly of cigarette smoke. I booked this room last-minute when my travel plans changed. The price was ok considering it was last-minute but it was way out of the way. We come to this hotel every year, and we appreciate the consistently top-notch experience!
Title	Column header: title or title_[FieldName]
A way to refer to each document The Title isn’t analyzed as part of the verbatim, but can help organize documents Only one Title column is permitted per data file Titles are optional	Recent stay Hotel visit We’ll definitely be back!
String	Column header: string_[FieldName]
Words that help categorize the documents Can only filter fields in Daylight with up to 10,000 values Include as many string columns as needed	Example: string_MemberLevel None Business Loyalty
Number	Column header: number_[FieldName]
Numeric data Can optionally use in the Driver feature Include as many number columns as needed	Example: number_Age 18 41 57
Score	Column header: score_[FieldName]
Score or rating data Scores are numbers you want to be higher Recommended for using the Drivers function Include as many score columns as needed	Example: score_OverallExperience 7 4.5 10
Date	Column header: date_[FieldName]
Date or times Accepts ISO 8601 strings, Unix timestamps, or US-style formats Daylight assumes that all dates are in a UTC timezone unless you include an ISO 8601 date with a specific timezone Helps you filter your project, especially if you upload data more than once Include as many date columns as needed	Example: date_CheckoutDate ISO 8601 formatted dates: 2018-04-10 2018-04-10T13:45 2018-04-10T13:45:00Z US-style dates: 04/10/2018 04/10/2018 13:45:15 4/10/18 1:45 PM

Sometimes, a metadata field can have multiple values within a single document. For example, a survey may ask the respondent “which of these products have you tried?”. In such a case, the respondent may select more than one product. There are two ways that you can format the data in such cases:

Have one column for this metadata field and enter all of the values separated with the “|” (pipe) character. In the example above, the column header can be “Products Used” and the value in a given cell could be “ProductA | ProductB | ProductC”.
Have multiple columns with the same column header name with a single value in each cell. In the example above, you would have as many columns as you need with the header name “Products Used” and populate each cell with a single Product. Some of the cells can be left blank.

In both cases, a single metadata field will be created with multiple values.

Supported languages and multilingual datasets

Daylight is capable of performing analysis natively in 15 languages. For best results with a multilingual dataset, split your data into one language per upload file. Each language will be uploaded and analyzed as its own Project.

Save as a CSV file

To upload your data to Daylight, save the file in CSV format, and make sure that the file extension is .csv. Daylight will also accept similarly formatted files, such as a tab separated value (TSV) file. In this case, make sure that the file extension is .tsv.