In this article
Overview
You can use this Snap to execute a job, check its status in Databricks, and, based on the job's status, complete or fail the pipeline. The Snap triggers the task to execute and then periodically checks its status. The Snap stops after the job is executed. However, if the pipeline is canceled before the task is finished, the Snap sends a request to stop the task.
Snap Type
The Databricks - Run Job Snap is a Write-type Snap.
Prerequisites
Valid client ID.
A valid account with the required permissions.
Support for Ultra Pipelines
Works in Ultra Pipelines.
Limitations and Known Issues
None.
Snap Views
Type | Format | Number of Views | Examples of Upstream and Downstream Snaps | Description |
---|---|---|---|---|
Input | Document Binary Binary or Document |
|
| Requires a valid task name, notebook path, and cluster-info. |
Output | Document Binary Binary or Document |
|
| Executes the selected notebook. |
Error | Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the pipeline by choosing one of the following options from the When errors occur list under the Views tab:
Learn more about Error handling in Pipelines. |
Snap Settings
Asterisk ( * ): Indicates a mandatory field.
Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.
Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.
Add icon ( ): Indicates that you can add fields in the field set.
Remove icon ( ): Indicates that you can remove fields from the field set.
Upload icon ( ): Indicates that you can upload files.
Field Name | Field Type | Description | |
---|---|---|---|
Label* Default Value: Databricks - Run Job | String | The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline. | |
Task name* Default Value: N/A | String/Expression | Specify the name of the task to perform the job. | |
Notebook path* Default Value: N/A | String/Expression | Specify the path of the saved notebook that will run in this job. | |
Cluster* Default Value: N/A | Specify the cluster to run the job. | ||
Parameter(s) | Use this field set to specify the parameters to run the job. | ||
Key* Default Value: N/A | String/Expression | Specify the parameter key. | |
Value* Default Value: N/A | String/Expression | Specify the parameter value. | |
Interval check (seconds)* Default Value: 10 | Integer/Expression | Specify the number of seconds to wait before checking the status of the task. | |
Snap Execution Default Value: Execute only | Dropdown list | Select one of the following three modes in which the Snap executes:
|
Troubleshooting
Error | Reason | Resolution |
---|---|---|
Account validation failed. | The Pipeline ended before the batch could complete execution due to a connection error. | Verify that the Refresh token field is configured to handle the inputs properly. If you are not sure when the input data is available, configure this field as zero to keep the connection always open. |
Examples
Excluding Fields from the Input Data Stream
We can exclude the unrequired fields from the input data stream by omitting them in the Input schema fieldset. This example demonstrates how we can use the <Snap Name> to achieve this result:
<screenshot of Pipeline/Snap and description>
Download this Pipeline.
Downloads
Download and import the Pipeline into SnapLogic.
Configure Snap accounts, as applicable.
Provide Pipeline parameters, as applicable.
Snap Pack History
Related Content
Endpoint Doc Link 1
Endpoint Doc Link 2