Databricks Snap Pack
In this article
Overview
Databricks is a data analytics platform optimized for the Microsoft Azure cloud services platform. It can also be run on Amazon AWS cloud and Google Cloud Platform. Databricks offers three environments for developing data-intensive applications: Databricks SQL, Databricks Data Science & Engineering, and Databricks Machine Learning. Learn more: Databricks Documentation.
Articles in this section
This Snap Pack focuses on the Databricks Data Science & Engineering environment which is also referred to as Databricks Lakehouse Platform (DLP) or Databricks. The Databricks Snap Pack contains the following Snaps:
Databricks - Select: Retrieves information from the target Databricks table.
Databricks - Insert: Inserts new rows of data in the target Databricks table.
Databricks - Delete: Deletes data from a target Databricks table.
Databricks - Bulk Load: Loads millions of rows of data in the target table through a single load operation.
Databricks - Unload: Unloads data from a target Databricks table through a single unload operation.
Databricks - Merge Into: Updates millions of existing rows and inserts new rows in a target Databricks table through a single operation.
Databricks - Execute: Runs multiple SQL statements on the target Databricks instance.
Limitations
With the basic authentication type for Databricks Lakehouse Platform (DLP) reaching its end of life on July 10, 2024, SnapLogic Databricks pipelines designed to use this authentication type to connect to DLP instances would cease to succeed. We recommend that you reconfigure the corresponding Snap accounts to use the Personal access tokens (PAT) authentication type.
Prerequisites
Create and configure an application corresponding to your Databricks instance in the AWS, Azure Portal, or Google Cloud Platform before using these Snaps. All the Snaps' accounts require information pertaining to this application for authentication purposes.
Supported Versions
This Snap Pack is tested against JDBC JAR version: databricks-jdbc-2.6.25-1.jar.
Using Alternate JDBC JAR File Versions
We recommend that you let the Snaps use this JAR file version. However, you may choose to use a different JAR file version.
Snap Pack History
Release | Snap Pack Version | Date | Type | Updates |
---|---|---|---|---|
November 2024 | main29029 | Nov 13, 2024 | Stable | Updated and certified against the current SnapLogic Platform release. |
August 2024 | main27765 | Aug 21, 2024 | Stable | Upgraded the |
May 2024 | 437patches27246 | Aug 8, 2024 | Latest | Added Databricks - Run Job. This Snap executes a job, checks its status in Databricks, and, based on the job's status, completes or fails the pipeline. |
May 2024 | 437patches26400 | May 15, 2024 | Latest | Fixed an invalid session handle issue with the Databricks Snap Pack that intermittently triggered an error message when the Snaps failed to connect with Databricks to execute the SQL statement. |
May 2024 | main26341 | May 8, 2024 | Stable | Updated the Delete Condition (Truncates a Table if empty) field in the Databricks - Delete Snap to Delete condition (deletes all records from a table if left blank) to indicate that all entries will be deleted from the table when this field is blank, but no truncate operation is performed. |
February 2024 | main25112 | Feb 14, 2024 | Stable | Updated and certified against the current SnapLogic Platform release. |
November 2023 | main23721 | Nov 8, 2023 | Stable | Updated and certified against the current SnapLogic Platform release. |
August 2023 | main22460 | Aug 16, 2023 | Stable | Updated and certified against the current SnapLogic Platform release. |
May 2023 | 433patches21630 | Jun 28, 2023 | Latest | Enhanced the performance of the Databricks - Insert Snap to improve the amount of time it takes for validation. |
May 2023 | main21015 | May 10, 2023 | Stable | Upgraded with the latest SnapLogic Platform release. |
February 2023 | main19844 | Feb 9, 2023 | Stable | Upgraded with the latest SnapLogic Platform release. |
November 2022 | main18944 | Nov 10, 2022 | Stable | The Databricks - Insert Snap now creates the target table only from the table metadata of the second input view when the following conditions are met:
|
September 2022 | 430patches18305 | Sep 29, 2022 | Latest |
The following fields are added to each Databricks Snap as part of this enhancement:
|
September 2022 | 430patches17796 | Sep 28, 2022 | Latest | The Manage Queued Queries property in the Databricks Snap Pack enables you to decide whether a given Snap should continue or cancel executing the queued Databricks SQL queries. |
August 2022 | main17386 | Aug 11, 2022 | Stable | Upgraded with the latest SnapLogic Platform release. |
4.29.2.0 | 42920rc17045 | Jul 15, 2022 | Latest | A new Snap Pack for Databricks Lakehouse Platform (Databricks or DLP) introduces the following Snaps:
|