$customHeader
Skip to end of banner
Go to start of banner

Configuring Databricks Accounts

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

In this article


Articles in this section

Overview

You must create Databricks accounts to connect to Databricks Snaps in your Pipelines with the source or target CDWs (databases)This account enables you to write-and-transform data in the target databases hosted in the following cloud locations. The JDBC URL you define for your target database indicates the respective cloud location where the database is hosted. You can configure your Databricks accounts in SnapLogic using either the Designer or the Manager.

Target Database

Supported Cloud Location

Cloud Location in JDBC URL

Databricks Lakehouse Platform (DLP)

AWS

jdbc:spark://<your_instance_code>.cloud.databricks.com

Microsoft Azure

jdbc:spark://<your_instance_code>.azuredatabricks.net

Different settings are required to access your target database depending on your target database type. Choose your database type from the following list to configure your Snap account.

  • Azure Synapse

  • BigQuery

  • Databricks Lakehouse Platform (DLP)

  • Redshift

  • Snowflake

Snap-Account Compatibility

Snaps in the Databricks Snap Pack work with different accounts and protocols per the following table.

Configuring Databricks Accounts Using SnapLogic Designer

Drag a Databricks Snap to the Canvas and click the Snap to open its settings. Click the Account tab. You can now either use an existing account or create a new one.

Selecting an existing account

SnapLogic organizes and displays all accounts to which you have access, sorting them by account type and location. To select an existing account:

  1. In the Account tab, click the List (blue star)  icon to view the accounts to which you have access, and select the account that you want to use. 

  2. Click the Save (blue star) icon.

Creating an account

  1. In the Account tab, click Add Account below the Account Reference field.

  2. Select the Location in which you want to create the account, select the Account Type, and click ContinueThe Add Account dialog window associated with the account type appears.

  3. Enter the required account details. For detailed guidance on how to provide the information required for each account type, see the following articles:

  4. Click Validate to verify the account, if the account type supports validation.

  5. Click Apply to complete configuring the Databricks account.

Enter additional informationont this account in the Notes field of the Info tab. This will help you and other users understand the purpose of the account, especially if there are multiple accounts of the same type.

Configuring Databricks Accounts Using SnapLogic Manager

You can use Manager to create accounts without associating them immediately with Pipelines.

Accounts in SnapLogic are associated with projects. You can use accounts created in other projects only if you have at least Read access to them.

  1. In the left pane, browse to the project in which you want to create the account and click  Create > Account Databricks, followed by the appropriate account type. The Create Account dialog associated with the selected account type appears.

  2. Repeat the steps numbered 3 through 5 in the Creating an account section.

Avoid updating account credentials while Pipelines using that account are executing. Doing so may lead to unexpected results, including your account getting locked.

Account Settings

Supported JDBC JAR Versions

You can configure your Databricks Account to automatically use an appropriate JDBC JAR file for connecting to your target database and performing the load and transform operations. 

Supported CDW

Certified JDBC JAR File

Azure Synapse

mssql-jdbc-10.2.0.jre8.jar

BigQuery

SimbaJDBCDriverforGoogleBigQuery42_1.2.22.1026.zip

DLP

databricks-jdbc-2.6.25-1.jar

Redshift

redshift-jdbc42-2.1.0.7.jar

Snowflake

snowflake-jdbc-3.13.18.jar

Using Alternate JDBC JAR File Versions

We recommend that you let the Snaps use the listed JAR file versions. However, you may use a different JAR file version of your choice.

Snap Pack History

 Click here to expand...

Release

Snap Pack Version

Date

Type

Updates

May 2024

437patches26400

Latest

Fixed an invalid session handle issue with the Databricks Snap Pack that intermittently triggered an error message when the Snaps failed to connect with Databricks to execute the SQL statement.

May 2024

main26341

Stable

Updated the Delete Condition (Truncates a Table if empty) field in the Databricks - Delete Snap to Delete condition (deletes all records from a table if left blank) to indicate that all entries will be deleted from the table when this field is blank, but no truncate operation is performed.

February 2024

main25112

Stable

Updated and certified against the current SnapLogic Platform release.

November 2023

main23721

Stable

Updated and certified against the current SnapLogic Platform release.

August 2023

main22460

Stable

Updated and certified against the current SnapLogic Platform release.

May 2023

433patches21630

Latest

Enhanced the performance of the Databricks - Insert Snap to improve the amount of time it takes for validation.

May 2023

main21015

Stable

Upgraded with the latest SnapLogic Platform release.

February 2023

main19844

Stable

Upgraded with the latest SnapLogic Platform release.

November 2022

main18944

Stable

The Databricks - Insert Snap now creates the target table only from the table metadata of the second input view when the following conditions are met:

  • The Create table if not present checkbox is selected.

  • The target table does not exist.

  • The table metadata is provided in the second input view.

September 2022

430patches18305

Latest

The following fields are added to each Databricks Snap as part of this enhancement:

  • Number of Retries: The number of attempts the Snap should make to perform the selected operation when the Snap account connection fails or times out.

  • Retry Interval (seconds): The time interval in seconds between two consecutive retry attempts.

September 2022

430patches17796

Latest

The Manage Queued Queries property in the Databricks Snap Pack enables you to decide whether a given Snap should continue or cancel executing the queued Databricks SQL queries.

August 2022

main17386

Stable

Upgraded with the latest SnapLogic Platform release.

4.29.2.0

42920rc17045

Latest

A new Snap Pack for Databricks Lakehouse Platform (Databricks or DLP) introduces the following Snaps:


  • No labels