...
You can use this account type to connect Databricks Snaps with data sources that use the Databricks account with Azure Data Lake Storage (ADLS) Blob Storage as a source.
...
Limitations and Known Issues
None.
Account Settings
...
Info |
---|
|
Field Name | Field Type | Field Dependency | Description |
---|---|---|---|
Label* Default Value: N/A | String | None. | Specify a unique label for the account. |
Download JDBC Driver Automatically Default Value: Not Selected | Checkbox | None | Select this checkbox to allow the Snap account to download the certified JDBC Driver for DLP. The following fields are disabled when this checkbox is selected |
:
To use a JDBC Driver |
that you choose, clear this checkbox, upload (to SLDB), and choose the required JAR files in the JDBC JAR(s) and/or ZIP(s): JDBC Driver field. Use of Custom JDBC JAR version You can use a different JAR file version |
other than the recommended |
Spark JDBC and Databricks JDBC If you do not select this checkbox and use an older JDBC JAR file (older than version 2.6.25), ensure that you use:
| |||
JDBC URL* Default Value: N/A Example: jdbc:spark://adb-2409532680880038.18.azuredatabricks.net:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/2409532680880038/0326-212833-drier754;AuthMech=3; | String | None | Enter the JDBC driver connection string that you want to use in the syntax provided below, for connecting to your DLP instance. |
Learn more in Microsoft's JDBC and ODBC drivers and configuration parameters |
. jdbc:spark://dbc-ede87531-a2ce.cloud.databricks.com:443/default;transportMode=http;ssl=1;httpPath= Avoid passing Password inside the JDBC URL If you specify the password inside the JDBC URL, it is saved as it is and is not encrypted. |
Instead we recommend |
using |
the provided Password field |
to ensure that your password is encrypted. |
Use Token Based Authentication Default |
Value: Selected |
Deselected | Checkbox | None | Select this checkbox to use token-based authentication for connecting to the target database (DLP) instance. Activates the Token field. |
Token* Default |
Value: N/A | String | When the Use Token Based Authentication checkbox is selected. | Enter the token value for accessing the target database/folder path. |
Database name* Default |
Value: N/A | String | None | Enter the name of the database to use by default. This database is used if you do not specify one in the Databricks Select or Databricks Insert Snaps. |
Source/Target Location* | Dropdown | None | Select the source or target data warehouse into which the queries must be loaded, that is |
Azure Blob Storage. This activates the following fields:
|
Azure storage account name* Default value: N/A Example: |
testblob | String | When ADLS Blob Storage is selected as the source. | Enter the name with which Azure Storage was created. The Bulk Load Snap automatically appends the '.blob.core.windows.net' domain to the value of this property. |
Azure Container* Default |
Value: N/A | String |
When ADLS Blob Storage is selected as the source. | Enter the name of an existing Azure container. |
Azure folder* Default |
Value: N/A | String |
When ADLS Blob Storage is selected as the source. | Enter the name of an existing Azure folder in the container to be used |
for hosting files. |
Azure Auth Type Default |
Value: Shared Access Signature | Dropdown |
When ADLS Blob Storage is selected as the source. | Select the authorization type that you want to consider |
when setting up the account. Options available are:
|
|
SAS Token* Default |
Value: N/A Example: ?sv=2020-08-05&st=2020-08-29T22%3A18%3A26Z&se=2020-08-30T02%3A23%3A26Z&sr=b&sp=rw&sip=198.1.2.60-198.1.2.70&spr=https&sig=A%1DEFGH1Ijk2Lm3noI3OlWTjEg2tYkboXr1P9ZUXDtkk%3D | String |
When you select Shared Access Signature in the Azure Auth Type. | Enter the SAS token which is the part of the SAS URI associated with your Azure storage account. |
Learn more in Getting Started with SAS |
. |
Azure storage account key* Default |
Value: N/A | String | Appears when you select Storage account key in the Azure Auth Type. | Enter the access key ID associated with your Azure storage account. |
Advanced Properties | Other parameters that you want to specify to configure the account. This |
fieldset consists of the following fields:
|
URL Property Name
URL properties | Use this |
---|
fieldset to define the account parameter's name and its corresponding value. Click + to add the parameters and the corresponding values. Add each URL property-value pair in a separate row. |
| ||
URL property name |
---|
URL property value
URL property name
Default Value: N/A | N/A | None | Specify the name of the parameter for the URL property. |
---|---|---|---|
URL property value Default Value: N/A | N/A | None | Specify the value for the URL property parameter. |
Batch size* Default Value: N/A | Integer | None | Specify the number of queries that you want to execute at a time.
|
Fetch size* Default Value: 100 | Integer | None | Specify the number of rows a query must fetch for each execution. Large values could cause the server to run out of memory. |
Min pool size* Default Value: 3 | Integer | None | Specify the minimum number of idle connections that you want the pool to maintain at a time. |
Max pool size* Default Value: 15 | Integer | None | Specify the maximum number of connections that you want the pool to maintain at a time. |
Max life time* Default Value: 60 | Integer | None | Specify the maximum lifetime of a connection in the pool, in seconds.
Minimum value: 0 |
Idle Timeout* Default Value: 5 | Integer | None | Specify the maximum amount of time in seconds that a connection is allowed to sit idle in the pool. 0 (zero) indicates that idle connections are never removed from the pool. Minimum value: 0 |
Checkout timeout* Default Value: 10000 | Integer | None | Specify the maximum time in milliseconds you want the system to wait for a connection to become available when the pool is exhausted. If you specify 0, the Snap waits infinitely until the connection is available. Therefore, we recommend you not to specify 0 for Checkout Timeout. Minimum value: 0 |
Snap Pack History
Expand | ||||
---|---|---|---|---|
|
...
|
...
|
...
Related Links
...