How to Create a Databricks Connector in Databasin

This guide explains how to create a private Databricks connector within the Databasin application with your Personal Access Token (PAT). It covers the required information and how to collect it from your Databricks workspace.

Prerequisites

• Access to databasin.wustl.edu using your institutional credentials (WashU Key).
• Access to a project within Databasin.
• Access to a Databricks workspace.

Steps

1. Sign into Databasin

1. Go to databasin.wustl.edu.
2. Log in using your WashU Key.
3. Confirm you have access to a project.
   • If not visible, contact the platform engineering team.

2. Navigate to the Connectors Section

1. Navigate to a project
2. In the left sidebar, click Connectors.
3. Click the Create Connector button in the top right.
4. From the list of connector types, select Databricks.

3. Fill in the Create Connector Form

You need to gather specific information from Databricks before completing the form:

3.1 Gather Required Information from Databricks

3.2 Generate a Personal Access Token (PAT)

1. In Databricks, click on your profile menu > Settings.
2. Go to Developer > Access Tokens.
3. Click Manage > Generate New Token.
4. Provide a name (e.g., "Databasin Connector") and select an expiration (90 days maximum).
5. Copy the token immediately — it will not be shown again.
6. Paste the token into the Token field in the Databasin connector form.

Important: If the token expires, you must generate a new one and update your connector.

3.3 Complete the Connector Form

• Name: Choose a meaningful name (e.g., Personal Databricks Connector).
• Hostname: Paste the copied Databricks hostname.
• Port: Enter 443.
• Catalog Name: Enter the catalog name.
• Database Name: Enter the database/schema name.
• HTTP Path: Paste the full HTTP path from your warehouse.
• Authentication: Use your Personal Access Token.

Note: Leave optional fields like "External Location" blank unless explicitly needed for large external datasets.

4. Set Connector Privacy

• Important: Toggle on "Private" to ensure only you can see and use your connector.
• If you do not set the connector as private, all users within your project will have access to your credentials.

5. Test and Save the Connector

1. Click Test Connection.
   • If successful, you will see a confirmation notification.
   • If not, check your token (ensure it's active) and other details.
2. Click Submit to save the connector.

Additional Notes

• Expiration: Remember to update your connector token every 90 days.
• Warehouse Access: SQL warehouses do not restrict data access. Permissions are enforced at the catalog and schema level.
• Connector Visibility: Only users within your Databasin project can view non-private connectors.

Troubleshooting