Managed Datasets

Managed Datasets provide scalable, secure storage and fast analytics in Qrvey, using a high-performance, serverless OpenSearch cluster. They can combine multiple data sources. For multi-source datasets, see Dataset Design.

Create a Managed Dataset

1. Go to Data > Datasets.
2. Click Create New Dataset > New Managed Dataset.
3. Select a data source (existing connection, dataset, or shared structure). To add a new connection, click Create New Connection.
4. Choose your data source and click Next.
   • For databases, pick a table/view or enter a custom SQL query.
   • To preview data, hover over a column and click View Source Data.
5. (Optional) For long queries (10+ min), select This is a long-running query.
6. Click Save to open the Design page.
7. Configure the dataset as desired.
   For details, please see Dataset Design.
8. The dataset saves in draft mode with a default name, which you can edit.
9. Finally, click Load Dataset. A modal will open.
   • Select estimated dataset size and click Load. Progress displays at the top-right.
   • You can continue working while loading.
   • You may adjust estimate later in Advanced Settings.
   • After loading, the Summary view appears.
   • Check the Activity Log for events or errors.

After this, you will want to configure Data Synchronization to keep your data up to date.

View Source Data

Displays the first 100 rows from the data source.

Modify the Connection

1. Click the three-dot menu on the data source pill.
2. Choose:
   • Edit Connection Settings — Change connection properties.
   • Edit Query — Change table/view or SQL query (for databases/MongoDB).
   • Join to Another Data Source — Add another source. See Data Joins.
   • Union with Another Data Source — Append data. See Data Unions.
   • Switch for Another Data Source — Use a different connection.
   • Rename — Change data source name.
   • Delete — Remove a joined data source.

Note: Removing/renaming columns or changing data types may break existing visualizations.

Automatically Add New Columns

Enable Qrvey to add new columns found during data reloads (disabled by default):

1. Scroll to the page bottom.
2. Select Automatically add new columns discovered during data loads.

Select Dataset Columns

Use checkboxes to include/exclude columns:

• Dataset (left): Include in dataset. Required columns for sync/join should not be removed.
• Visualizations (right): Show/hide in visualizations (e.g., hide ID columns).

Note: Columns with Geolocation cannot be removed.

Change Column Name

Click the column name box to edit. No data reload required. Hover over the field for details (friendly name, original name, data type).

Tip: Use the Search box to find columns quickly.

Change Data Type

Qrvey auto-detects data types, but you can change them:

1. Click the Column Type field and select a new type.
   • Requires data reload if already loaded.

Change Input Format

If the type of data in a column is unclear to Qrvey, you can adjust as needed. Requires data reload.

Change Visualization Format

Change how dates/numbers display in visualizations (no reload needed). Useful for formatting currencies, dates, etc.

Custom date format syntax:

Syntax	Type
DD	day
HH	hour
mm	minute
ss	second
SSS	milliseconds
YYYY	year
MM	month

Select Unique Identifier Columns

For Data Synchronization in Append and Update mode, set a unique identifier:

1. Click the column's three-dot menu.
2. Go to Unique ID > On.
   • Requires data reload.

Define a Column Security Name

For Record Level Security (RLS), assign a security name:

1. Click the column's three-dot menu > Enable Record Level Security.
2. Enter a security name and confirm.

Note: RLS must be configured in Admin Center. See Record Level Security. Assigning a security name does not require data reload.

Automatic Column Links

Configure columns to display as links in visualizations (e.g., map links). See Column Links.

Configure Geolocation Groups

See Geolocation Groups for displaying addresses on maps.

Transform Column Data

Apply Transformations (e.g., uppercase, parse JSON) to modify data upon loading.