Input 

Document

• Min: 1

• Max: 2

• Mapper

• Copy

Requires the table name where the data has to be inserted and the data flush interval (milliseconds) in which the data is pushed to the database.

Output

Document

• Min: 0

• Max: 1

• Mapper

• Filter

Inserts data in Snowflake tables in specified intervals.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the pipeline by choosing one of the following options from the When errors occur list under the Views tab:

• Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.

• Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

• Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Label*

Default Value: Snowflake – Snowpipe Streaming
Example: Stream Data

String

Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Schema name

Default Value: N/A
Example: “PUBLIC”

String/Expression/Suggestion

Specify the database schema name.

Table Name*

Default Value: N/A
Example: “PUBLIC”.”SNOWPIPESTREAMING”

String/Expression/Suggestion

Specify the table name on which the insert operation has to be executed.

Create table if not present

Default Value: Deselected

Checkbox

Select this checkbox if you want the table to be automatically created if it does not already exist.

Max client lag*

Default Value: 1000
Example: 1500
Max Value: 60,000

Integer/Expression/Suggestion

Specify the client data flush interval in milliseconds. Adjust this value based on the maximum latency your target system can handle (60,000 ms).

Snap execution

Default Value: 
Example: Validate & Execute

Dropdown list

Select one of the following three modes in which the Snap executes:

• Validate & Execute: Performs limited execution of the Snap and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime.

• Execute only: Performs full execution of the Snap during pipeline execution without generating preview data.

• Disabled: Disables the Snap and all Snaps that are downstream from it.

Schema name not found

The schema name is required for Snowpipe Streaming.

Provide a schema name in the Snap configuration.