...
Field Name | Field Type | Description | |||
---|---|---|---|---|---|
Label* Default Value: Databricks - Select | String | The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline. | |||
Database name Default Value: N/A | String/Expression/Suggestion | Enter your DLP database name for the SELECT statement to read the data from. | |||
Table name* Default Value: N/A | String/Expression/Suggestion | Enter your table name for the SELECT statement to read the data from. | |||
Where clause Default Value: N/A | String/Expression/Suggestion | Specify the WHERE clause (condition) for the SELECT statement to apply while retrieving the data from the DLP table. | |||
Order by | You can use this fieldset to define the order in which the retrieved rows are populated in the output. Click the Add fieldset row icon ( ) and specify one column name in each row to define your sort order. | ||||
Column names Default Value: None. | String/Expression/Suggestion | Click the Suggest icon ( ) and select a column name from the suggested list of column names in the table specified above.
| |||
Limit rows Default Value: None. | String/Expression | Specify the number of rows of data you want the Snap to retrieve from the table.
| |||
Output fields | You can use this fieldset to define the list of columns to be retrieved in the output. Click the Add fieldset row icon ( ) and specify one column name in each row. | ||||
Output field Default Value: None. | String/Expression/Suggestion | Click the Suggest icon ( ) and select a column name from the suggested list of column names in the table specified above.
| |||
Fetch Output Fields In Schema Default Value: Not selected | Checkbox | Select this checkbox to retrieve schema information from the table for only the output fields specified above. If not selected, the Snap retrieves schema information from all fields available in the table. | |||
Pass through Default Value: Selected | Checkbox | Select this checkbox to include the input document as-is under the | |||
Ignore empty result Default Value: Not selected | Checkbox | Select this checkbox to prevent the Snap from writing any document to its output view when no records are retrieved for the specified Snap configuration.
| |||
Match data types Default Value: Not selected | Checkbox | With at least one output field selected in the Output fields fieldset, select this checkbox to match the data types of the columns with those retrieved when no output field is specified (as in | |||
Number of Retries Default value: 0 Example: 3 Minimum value: 0 | Integer | Specifies the maximum number of retry attempts when the Snap fails to write. If the value is larger than 0, the Snap first downloads the target file into a temporary local file. If any error occurs during the download, the Snap waits for the time specified in the Retry interval and attempts to download the file again from the beginning. When the download is successful, the Snap streams the data from the temporary file to the downstream Pipeline. All temporary local files are deleted when they are no longer needed. Ensure that the local drive has sufficient free disk space to store the temporary local file. | |||
Retry Interval (seconds) Default value: 1 Example: 3 Minimum value: 1 | Integer | Specifies the minimum number of seconds the Snap must wait before each retry attempt. | |||
Staging Mode Default value: In Memory Example: On disk | Dropdown list | Select this property to specify the location where the in-flight records should be stored while using the Retry option. Options available are: On disk: The records are stored on the disk. In Memory: The records are stored in the internal memory. | |||
Manage Queued Queries Default value: Continue to execute queued queries when pipeline is stopped or if it fails. Example: Cancel queued queries when pipeline is stopped or if it fails | Dropdown list | Select this property to determine whether the Snap should continue or cancel the execution of the queued Databricks SQL queries when you stop the Pipeline. If you select Cancel queued queries when pipeline is stopped or if it fails, then the read queries under execution are cancelled, whereas the write type of queries under execution are not cancelled. Databricks internally determines which queries are safe to be cancelled and cancels those queries. Due to an issue with DLP, aborting an ELT Pipeline validation (with preview data enabled) causes only those SQL statements that retrieve data using bind parameters to get aborted while all other static statements (that use values instead of bind parameters) persist.
To avoid this issue, ensure that you always configure your Snap settings to use bind parameters inside its SQL queries. | |||
Snap Execution Default Value: Validate & Execute | Dropdown list | Select one of the three modes in which the Snap executes. Available options are:
|
...
Downloads
Info |
---|
|
Attachments | ||||||
---|---|---|---|---|---|---|
|
...