Databricks Snap Pack

In this article

Overview

Databricks is a data analytics platform optimized for the Microsoft Azure cloud services platform. It can also be run on Amazon AWS cloud and Google Cloud Platform. Databricks offers three environments for developing data-intensive applications: Databricks SQL, Databricks Data Science & Engineering, and Databricks Machine Learning. Learn more: Databricks Documentation.

Articles in this section

This Snap Pack focuses on the Databricks Data Science & Engineering environment which is also referred to as Databricks Lakehouse Platform (DLP) or Databricks. The Databricks Snap Pack contains the following Snaps:

Limitations

  • With the basic authentication type for Databricks Lakehouse Platform (DLP) reaching its end of life on July 10, 2024, SnapLogic Databricks pipelines designed to use this authentication type to connect to DLP instances would cease to succeed. We recommend that you reconfigure the corresponding Snap accounts to use the Personal access tokens (PAT) authentication type.

Prerequisites

Create and configure an application corresponding to your Databricks instance in the AWS, Azure Portal, or Google Cloud Platform before using these Snaps. All the Snaps' accounts require information pertaining to this application for authentication purposes.

Supported Versions

This Snap Pack is tested against JDBC JAR version: databricks-jdbc-2.6.25-1.jar.

Using Alternate JDBC JAR File Versions

We recommend that you let the Snaps use this JAR file version. However, you may choose to use a different JAR file version.

Snap Pack History

Release

Snap Pack Version

Date

Type

Updates

November 2024

main29029

Nov 13, 2024

Stable

Updated and certified against the current SnapLogic Platform release.

August 2024

main27765

Aug 21, 2024

Stable

Upgraded the org.json.json library from v20090211 to v20240303, which is fully backward compatible.

May 2024

437patches27246

Aug 8, 2024

Latest

Added Databricks - Run Job. This Snap executes a job, checks its status in Databricks, and, based on the job's status, completes or fails the pipeline.

May 2024

437patches26400

May 15, 2024

Latest

Fixed an invalid session handle issue with the Databricks Snap Pack that intermittently triggered an error message when the Snaps failed to connect with Databricks to execute the SQL statement.

May 2024

main26341

May 8, 2024

Stable

Updated the Delete Condition (Truncates a Table if empty) field in the Databricks - Delete Snap to Delete condition (deletes all records from a table if left blank) to indicate that all entries will be deleted from the table when this field is blank, but no truncate operation is performed.

February 2024

main25112

Feb 14, 2024

Stable

Updated and certified against the current SnapLogic Platform release.

November 2023

main23721

Nov 8, 2023

Stable

Updated and certified against the current SnapLogic Platform release.

August 2023

main22460

Aug 16, 2023

Stable

Updated and certified against the current SnapLogic Platform release.

May 2023

433patches21630

Jun 28, 2023

Latest

Enhanced the performance of the Databricks - Insert Snap to improve the amount of time it takes for validation.

May 2023

main21015

May 10, 2023

Stable

Upgraded with the latest SnapLogic Platform release.

February 2023

main19844

Feb 9, 2023

Stable

Upgraded with the latest SnapLogic Platform release.

November 2022

main18944

Nov 10, 2022

Stable

The Databricks - Insert Snap now creates the target table only from the table metadata of the second input view when the following conditions are met:

  • The Create table if not present checkbox is selected.

  • The target table does not exist.

  • The table metadata is provided in the second input view.

September 2022

430patches18305

Sep 29, 2022

Latest

The following fields are added to each Databricks Snap as part of this enhancement:

  • Number of Retries: The number of attempts the Snap should make to perform the selected operation when the Snap account connection fails or times out.

  • Retry Interval (seconds): The time interval in seconds between two consecutive retry attempts.

September 2022

430patches17796

Sep 28, 2022

Latest

The Manage Queued Queries property in the Databricks Snap Pack enables you to decide whether a given Snap should continue or cancel executing the queued Databricks SQL queries.

August 2022

main17386

Aug 11, 2022

Stable

Upgraded with the latest SnapLogic Platform release.

4.29.2.0

42920rc17045

Jul 15, 2022

Latest

A new Snap Pack for Databricks Lakehouse Platform (Databricks or DLP) introduces the following Snaps: