Label*

None.

Required. Unique user-provided label for the account.

Default Value: N/A

Example: ELT DLP Azure Account AD ON

Account Properties*

Use this field set to configure the information required to establish a JDBC connection with the account.

This field set consists of the following fields:

• Database Type

• Download JDBC Driver Automatically

• JDBC JAR(s) and/or ZIP(s)

• JDBC Driver Class

• JDBC URL

• Username

• Password

• Default Database Name

Database Type*

None.

Select the target data warehouse into which the queries must be loaded, that is Databricks Lakehouse Platform. This activates the following fields:

• Use Token Based Authentication

• Username

• Password

• Default Database Name

• DBFS Folder path (source for loading Databricks table)

Default Value: N/A

Example: Databricks Lakehouse Platform

Download JDBC Driver Automatically

None.

Select this checkbox to allow the Snap account to download the certified JDBC Driver for DLP. The following fields are disabled when this checkbox is selected.

• JDBC JAR(s) and/or ZIP(s) : JDBC Driver

• JDBC driver class

To use a JDBC Driver of your choice, clear this checkbox, upload (to SLDB), and choose the required JAR files in the JDBC JAR(s) and/or ZIP(s): JDBC Driver field. 

Default Value: Not Selected

Example: Selected

JDBC JAR(s) and/or ZIP(s): JDBC Driver

Required when the Download JDBC Driver Automatically checkbox is not selected.

Upload the JDBC driver and other JAR files that you want to use into SLDB. Click 

Default Value: N/A

Example: SimbaSparkJDBC42-2.6.21.1021.jar

JDBC driver class*

Required when the Download JDBC Driver Automatically checkbox is not selected.

Specify the driver class to use for your application. We recommend that you use com.simba.spark.jdbc.Driver for DLP, as other classes and methods may change due to future enhancements.

Default Value: N/A

Example: com.simba.spark.jdbc.Driver

JDBC URL*

None.

Enter the JDBC driver connection string that you want to use in the syntax provided below, for connecting to your DLP instance. See Microsoft's JDBC and ODBC drivers and configuration parameters for more information.

• jdbc:spark://dbc-ede87531-a2ce.cloud.databricks.com:443/default;transportMode=http;ssl=1;httpPath=
  sql/protocolv1/o/6968995337014351/0521-394181-guess934;AuthMech=3;UID=token;PWD=<personal-access-token> 

Alternatively, you can make use of the Username, Password, and Database Name fields, along with the Advanced Properties > URL Properties field set to provide the parameters required for building your JDBC URL. See Passing your JDBC URL for more information.

Default Value: N/A

Example: jdbc:spark://adb-2409532680880038.18.azuredatabricks.net:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/2409532680880038/0326-212833-drier754;AuthMech=3;

Use Token Based Authentication

Database Type is Databricks Lakehouse Platform

Select this checkbox to use token-based authentication for connecting to the target database (DLP) instance. Activates the Token field.

Default value: Selected

Example: Not selected

Token*

When Use Token Based Authentication checkbox is selected.

Enter the token value for accessing the target database/folder path.

Default value: N/A

Example: <Encrypted>

Default Database Name*

None.

Enter the name of the database to use by default. This database is used if you do not specify one in the ELT Select, ELT Insert-Select, or ELT Merge Into Snaps. 

Default value: N/A

Example: EMPLOYEEDB

DBFS Folder path (source for loading Databricks table)

Database Type is Databricks Lakehouse Platform

Required for ELT Load Snap.

Specify the fully qualified path to a target folder in your DBFS instance. It should begin with / which denotes the DBFS Root folder.

Default value: N/A

Example: /my_DBFS/mounted0408_folder

Optimize Write

Database Type is Databricks Lakehouse Platform.

Select this checkbox to dynamically optimize the Apache Spark partition sizes based on the actual data. If selected, DLP attempts to write 128 MB files (this is an approximate size and can vary depending on dataset characteristics) for each table partition.

Default Value: Not Selected

Example: Selected

Auto Compact

Database Type is Databricks Lakehouse Platform.

Select this checkbox to allow DLP to compact small files being written into a Delta table.

Default Value: Not Selected

Example: Selected

Use Mounts

Checkbox

If this checkbox is selected, it provides three different sources to mount DBFS.

S3 Bucket Mounts

Use this field set to provide the information required to configure the S3 Database account. It consists of the following fields:

• S3 Access Key

• S3 Secret Key

• S3 Bucket

• Mount Point

S3 Access Key

String

Specify the S3 access key ID that you want to use for AWS authentication.

Default Value: N/A
Example: NAVRGGRV7EDCFVLKJH

S3 Secret Key

String

Specify the S3 secret key associated with the S3 Access-ID key listed in the S3 Access-key ID field.

Default Value: N/A
Example: 2RGiLmL/6bCujkKLaRuUJHY9uSDEjNYr+ozHRtg

S3 Bucket

String

Specify the name of the S3 bucket that you want to use for staging data to Snowflake. 

Default Value: N/A
Example: sl-bucket-ca

Mount Point

String

The directory within DBFS to mount the source. Data written to mount point paths (/mnt) is stored outside of the DBFS root, for example in GCS buckets that are mounted as DBFS paths. Mounts are performed during Pipeline Validation and Pipeline Execution. It will not fail as the path is already mounted.

In the case of ELT Load Snap (DataBricks only) the priority of the loading from paths is as follows:
Cloud Storage Path (ELT Load snap) →  DBFS Folder path (source for loading Databricks table) →  First non-empty mount provided under Use Mounts in ELT DLP Account.

Learn more at Defining mount points.

Blob Storage Mounts

Use this field set to provide the information required to configure the blob storage mounts. It consists of the following fields:

• Azure Blob Account Name

• Azure Blob Storage Key

• Container Name

• Mount Point

Azure Blob Account Name

Database Type is Databricks Lake Platform. External Location is Blob Storage or Azure Data Lake Gen2.

Enter your Account name to access the selected Azure external storage location. You must configure this field if you want to use the ELT Load Snap.

Default value: N/A

Example: adlsgen2v02

Azure Blob Storage Key

Database Type is Databricks Lake Platform. External Location is Blob Storage or Azure Data Lake Gen2. Azure Auth Type is Shared Access Signature, Managed Identity, or Service Principals.

Enter the storage access key value corresponding to the Azure account mentioned in Storage Account field. See COPY INTO (Transact-SQL) in Microsoft Docs for more information on the expected values for this field.

This field is needed only when you want to define the File Name Pattern in the ELT Load Snap to locate and read the file/s from the Azure Folder path. Else, leave it blank.

Default value: N/A

Example: ufv!befDIuf#fnb$KH&_hweuf

Container Name

Database Type is Databricks Lake Platform. External Location is Blob Storage or Azure Data Lake Gen2.

Enter the name of the container in the Azure storage account.

Azure Container does not apply to the Redshift database.

Default value: N/A

Example: hrdepartment

Mount Point

String

The directory within DBFS to mount the source. Data written to mount point paths (/mnt) is stored outside of the DBFS root, for example in GCS buckets that are mounted as DBFS paths. Mounts are performed during Pipeline validation and Pipeline execution. It will not fail as the path is already mounted.

In the case of ELT Load Snap (Databricks only) the priority of the loading from paths is as follows:
Cloud Storage Path (ELT Load snap) →  DBFS Folder path (source for loading Databricks table) →  First non-empty mount provided under Use Mounts in ELT DLP Account.

Learn more at Defining mount points.

Data Lake Storage Gen2 Mounts

Use this field set to provide the information required to configure the Data Lake Storage Gen2 Mounts. It consists of the following fields:

• Application Id

• ADLS Gen2 Oauth2 Secret Key

• ADLS Gen2 Tenant Id

• ADLS Gen2 Account Name

• ADLS Gen2 Container Name

• Mount Point

Application Id

Database Type is Databricks Lake Platform. External Location is Blob Storage or Azure Data Lake Gen2.

Enter the name of theapplication (client) ID for the Azure Active Directory application that you want to configure.

Default value: N/A

Example: 635f1f0e-2378bd-412683734777efa9ef

ADLS Gen2 Oauth2 Secret Key

Database Type is Databricks Lake Platform. External Location is Blob Storage or Azure Data Lake Gen2.

Enter the Oauth2 secret key which is part of ADLS Gen2 account authentication in the Azure portal.

Default value: N/A

Example: 7ed4587-a77f-43bf-94c5dfg

ADLS Gen2 Tenant Id

Database Type is Databricks Lake Platform. External Location is Blob Storage or Azure Data Lake Gen2.

Enter the Tenant ID obtained after creating the application in the Azure portal. 

Default value: N/A

Example: 2060aaafe-89d9-9514-eac46338ec05

ADLS Gen2 Account Name

Database Type is Databricks Lake Platform. External Location is Blob Storage or Azure Data Lake Gen2.