Table of Contents

Connecting MIStudio to Databricks

This page explains how to connect MIStudio to a Databricks SQL Warehouse using JDBC.

The connection can be used in the MIStudio Logic Editor to read data from Databricks, execute SQL statements, write database data, and send database results to other application components.

Overview

A typical MIStudio-to-Databricks connection requires:

After the connection is configured, database components such as DatabaseRawLookup and DatabaseRawWrite can execute SQL against Databricks.

Step 1: Get the Databricks Connection Details

Sign in to your Databricks workspace.

Open the SQL Warehouse that MIStudio should use.

In Databricks:

Keep these values available while configuring MIStudio.

Typical connection information includes:

Server Hostname YOUR-SERVER-HOSTNAME
Port 443
HTTP Path /sql/1.0/warehouses/YOUR-WAREHOUSE-ID
Copy the Server Hostname and HTTP Path from the Databricks Connection Details page for your own SQL Warehouse.

Step 2: Download the Databricks JDBC Driver

Download the Databricks JDBC Driver from Maven Central:

Databricks JDBC Driver on Maven Central

Databricks publishes the JDBC driver under:

com.databricks:databricks-jdbc

Select the desired driver version and download the JAR file.

The downloaded file follows this naming pattern:

databricks-jdbc-<version>.jar

For example:

databricks-jdbc-3.4.2.jar

The JDBC driver class used by MIStudio is:

com.databricks.client.jdbc.Driver
Use the current Databricks JDBC Driver rather than the legacy Simba JDBC Driver when setting up a new connection.

Step 3: Add the JDBC Driver to the MIStudio Project

The Databricks JDBC driver can be added directly to the MIStudio project.

In the MIStudio project tree:

The driver file follows this naming pattern:

databricks-jdbc-<version>.jar
Restart MIStudio after adding the JDBC driver so the new driver is loaded before configuring or testing the Databricks connection.

Step 4: Choose an Authentication Method

Databricks supports multiple JDBC authentication methods.

For a simple connection test, a Databricks Personal Access Token (PAT) can be used.

For PAT authentication:

Setting Value
User Name `token`
Password Your Databricks Personal Access Token
AuthMech `3`
Do not place your Personal Access Token in wiki pages, screenshots, source control, issue reports, or shared files.

Treat the token as a password.

A Personal Access Token can be useful for testing and development. Production environments may require a different authentication method based on your organization's Databricks security requirements.

Step 5: Add DatabricksConnectionManager

Open the MIStudio project and application that will use Databricks.

Open the MIStudio Logic Editor and add a:

DatabricksConnectionManager

Configure the connection manager using the connection information from your Databricks SQL Warehouse.

Enter the appropriate values for:

For the JDBC Driver Class, use:

com.databricks.client.jdbc.Driver

The Server Hostname should be entered without the protocol.

For example, enter:

YOUR-SERVER-HOSTNAME

Do not enter:

https://YOUR-SERVER-HOSTNAME

Copy the HTTP Path from the Databricks SQL Warehouse Connection Details.

The HTTP Path should include the leading slash, for example:

/sql/1.0/warehouses/YOUR-WAREHOUSE-ID

Step 6: Configure the Connection Properties

For PAT-based authentication, enter the following connection properties.

Enter each property on its own line:

AuthMech=3
transportMode=http
ssl=1

The MIStudio User Name field should contain:

token

The Password field should contain the Databricks Personal Access Token.

Connection properties may differ when using another Databricks authentication method.

Step 7: Enable Simulating Mode

MIStudio must be running the application logic for database components to execute.

Before testing the Databricks connection:

The connection manager reference will normally point to the DatabricksConnectionManager in the application.

For example:

/Main/DatabricksConnectionManager
If the database components appear to be configured correctly but nothing happens, confirm that MIStudio is in Simulating mode.

Step 8: Test the Connection

Start with a simple query before attempting a larger database operation.

In DatabaseRawLookup, enter:

SELECT 1

Trigger the DatabaseRawLookup while MIStudio is in Simulating mode.

If the connection is successful, the query should return:

1

You can also check the Databricks SQL Warehouse Query History to confirm that the query was received and executed.

Step 9: Check the Catalog and Schema

After confirming the connection, determine which Databricks catalog and schema MIStudio is using.

Use DatabaseRawLookup with:

SELECT current_catalog()

Then check the schema:

SELECT current_schema()

These values identify the current location used by unqualified SQL queries.

Databricks objects can also be referenced using fully qualified names:

catalog.schema.table

For example:

my_catalog.my_schema.my_table

Using fully qualified table names can help prevent queries from running against the wrong catalog or schema.

Reading Data with DatabaseRawLookup

Use DatabaseRawLookup for SQL statements that return data.

Common examples include:

Example:

SELECT *
FROM my_catalog.my_schema.my_table

Writing Data with DatabaseRawWrite

Use DatabaseRawWrite for SQL statements that create or modify database content.

Common examples include:

For example:

CREATE TABLE IF NOT EXISTS my_catalog.my_schema.mistudio_test (
    id INT,
    message STRING,
    created_at TIMESTAMP
)

Testing a Write

After creating a test table, insert a value using DatabaseRawWrite.

For example:

INSERT INTO my_catalog.my_schema.mistudio_test
    (id, message, created_at)
VALUES
    (1, 'Hello from MIStudio', CURRENT_TIMESTAMP())

Then use DatabaseRawLookup to read the value back:

SELECT message
FROM my_catalog.my_schema.mistudio_test
WHERE id = 1

A successful readback confirms that MIStudio can:

Read vs. Write Components

Use the MIStudio database component that matches the SQL operation.

SQL Operation MIStudio Component
`SELECT` DatabaseRawLookup
Metadata query DatabaseRawLookup
Catalog or schema query DatabaseRawLookup
`CREATE TABLE` DatabaseRawWrite
`INSERT` DatabaseRawWrite
`UPDATE` DatabaseRawWrite
`DELETE` DatabaseRawWrite

Verify Queries in Databricks

Databricks SQL Warehouse Query History can be useful when testing the connection.

Query History can help confirm that:

If a query appears to execute in MIStudio but the expected result is not visible, check Databricks Query History as part of troubleshooting.

Troubleshooting

Nothing Happens When the Query Is Triggered

Confirm that MIStudio is in Simulating mode.

Also confirm that the database component is configured to use the intended DatabricksConnectionManager.

JDBC Driver Is Not Found

Confirm that:

com.databricks.client.jdbc.Driver

Connection Fails

Confirm:

For PAT authentication, confirm that the User Name is:

token

and that the Password field contains the user's Databricks Personal Access Token.

SELECT Works but CREATE or INSERT Does Not

Confirm that the correct MIStudio database component is being used.

Use:

Table Cannot Be Found

Confirm the current catalog and schema:

SELECT current_catalog()
SELECT current_schema()

If necessary, use the complete table name:

catalog.schema.table

Query Does Not Appear in Databricks

Check that:

Then check the Databricks SQL Warehouse Query History.

Security Notes

Database credentials and Databricks access tokens should be handled securely.

Do not include credentials in:

Use the authentication method required by your organization for production systems.

Connection Workflow Summary

Databricks SQL Warehouse
          |
          | Server Hostname
          | Port
          | HTTP Path
          | Authentication
          v
Databricks JDBC Driver
          |
          v
MIStudio Project Drivers
          |
          v
DatabricksConnectionManager
          |
          +--> DatabaseRawLookup
          |        |
          |        +--> SELECT / Read Data
          |
          +--> DatabaseRawWrite
                   |
                   +--> CREATE / INSERT / UPDATE / DELETE