...
You can use this Snap to unload the result of a query to a file or files stored on the Snowflake stage, or on an external S3 workspace, or on Azure Storage Blob, if required. The target Snowflake table is not modified. Once the data is unloaded to the storage, users can get the file either using Snowflake GET or S3 file reader.
...
| Info |
|---|
Snowflake provides the option to use the Cross Account IAM into the external staging. You can adopt the cross-account access through Storage Integration option . With this setup, you don’t need to pass any credentials and access the storage only using the named stage or integration object. Learn more: Configuring Cross Account IAM Role Support for Snowflake Snaps. |
...
Snap Type
The Snowflake - Bulk Load Unload Snap is a Read-type Snap.
...
Works in Ultra Pipelines. However, we recommend that you not use this Snap in an Ultra Pipeline. Snowflake provides the option to use the Cross Account IAM into the external staging. You can adopt the cross account access through option Storage Integration. With this setup, you don’t need to pass any credentials around and access the storage only using the named stage or integration object. For more details: Configuring Cross Account IAM Role Support for Snowflake Snaps.
Known Issues
Because of performance issues, all Snowflake Snaps now ignore the Cancel queued queries when pipeline is stopped or if it fails option for Manage Queued Queries, even when selected. Snaps behave as though the default Continue to execute queued queries when the Pipeline is stopped or if it fails option were selected.
...
Snap Views
Type | Format | Number of Views | Examples of Upstream and Downstream Snaps | Description |
|---|---|---|---|---|
Input | Document |
|
| A JSON document containing parameterized inputs for the Snap’s configuration, if needed. |
Output | Document |
|
| A JSON document containing the unload request details and the result of the unload operation. |
Error | Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab. The available options are:
Learn more about Error handling in Pipelines. | |||
Snap Settings
| Info |
|---|
|
): Indicates a list that is dynamically populated based on the configuration.Field Name | Field Type | Description | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Label* Default Value: Snowflake - Unload | String | Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline. | ||||||||||||
Query* Default Value: N/A | String/Expression | Specify a SELECT query. The results of the query are unloaded. In most cases, it is worthwhile to unload data in sorted order by specifying an ORDER BY clause in the query; this approach will save the time Required. to sort the data when it is reloaded. | ||||||||||||
Staging location
| Dropdown list/Expression | Specify the staging location for the unload. The expected input for this should be a path to a file or filename-prefix. The options available include:
| ||||||||||||
Target Default Value: N/A | String/Expression | Specify the staging area where the unloaded file(s) are placed. If the staging location is external, it will be put under the S3 Bucket or Microsoft Azure Storage Blob specified for the Account. If the staging location is internal, the files will be placed in the user’s home folder. | ||||||||||||
File Format Type Default Value: None | Dropdown list | The format type for the unloaded file. The options available are None, CSV, and CUSTOMIZED. If CUSTOMIZED is chosen, then the customized format identifier must be specified. | ||||||||||||
Customized format identifier Default Value: None | String | Specify the file format object to use for unloading data from the table. The field is valid only when the File format type is specified as Customized. Otherwise, this will be ignored. | ||||||||||||
File format option
| String | Specify the file format option. Separate multiple options by using blank spaces and commas. You can use various file format options including binary format which passes through in the same way as other file formats. See File Format Type Options for more information. Before loading binary data into Snowflake, you must specify the binary encoding format, so that the Snap can decode the string type to binary types before loading into Snowflake. This can be done by specifying the following binary file format: BINARY_FORMAT=xxx (Where XXX = HEX|BASE64|UTF-8) However, the file you upload and download must be in similar formats. For instance, if you load a file in HEX binary format, you should specify the HEX format for download as well. | ||||||||||||
Copy options | ||||||||||||||
Overwrite Default Value: Deselected | Checkbox | If selected, the UNLOAD overwrites the existing files, if any, in the location where files are stored. If Deselected, the option does not remove the existing files or overwrite. | ||||||||||||
Generate single file Default Value: Deselected | Checkbox | If selected, the UNLOAD will generate a single file. If it is not selected, the filename prefix needs to be included in the path. | ||||||||||||
Max file size
| Integer | Specify the maximum size (in bytes) of each file to be generated in parallel per thread. The number should be greater than 0, If it is less than or equals 0, the Snap will use the default size for snowflake: 16000000 (16MB). | ||||||||||||
Encryption type Default Value: None | Dropdown list | Specify the type of encryption to be used on the data. The available encryption options are:
The KMS Encryption option is available only for S3 Accounts (not for Azure Accounts) with Snowflake. | ||||||||||||
KMS key Default Value: N/A | String/Expression | Specify the KMS key that you want to use for S3 encryption. For more information about the KMS key, see AWS KMS Overview and Using Server Side Encryption. This property applies only when you select Server-Side KMS Encryption in the Encryption Type field above. | ||||||||||||
Include column headings Default Value: Deselcted | Checkbox | If selected, the table column heading will be included in the generated files. If multiple files are generated, the heading will be included in every file. | ||||||||||||
Validation Mode
| Dropdown list | Select this mode for visually verifying the data before unloading it. If you select NONE, validation mode is disabled or, the unloaded data will not be written to the file. Instead, it will be sent as a response to the output. The options available include:
| ||||||||||||
Manage Queued Queries Default Value: Continue to execute queued queries when the pipeline is stopped or if it fails | Dropdown list | Select this property to decide whether the Snap should continue or cancel the execution of the queued Snowflake Execute SQL queries when you stop the pipeline. If you select Cancel queued queries when the pipeline is stopped or if it fails, then the read queries under execution are canceled, whereas the write queries under execution are not canceled. Snowflake internally determines which queries are safe to be canceled and cancels those queries. | ||||||||||||
Default Value: Execute only | Dropdown list |
| ||||||||||||
The Snap behaves the same on a Groundplex as it does in a Cloudplex.
...