On this Page
| Snap type: | Write | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Description: | This Snap allows you to execute arbitrary SQL. You can drop your database with it, so be careful. This Snap supports SQL Server 2008 or newer. Valid JSON paths that are defined in the WHERE clause for queries/statements will be substituted with values from an incoming document. Documents will be written to the error view if the document is missing a value to be substituted into the query/statement. If a SELECT query is executed, the query's results are merged into the incoming document and any existing keys will have their values overwritten. On the other hand, the original document is written if there are no results from the query. If an output view is available and an UPDATE/INSERT/MERGE/DELETE statement was executed, then the original document that was used to create the statement will be output with the status of the statement executed. Expected upstream Snaps: The dynamic variables used in the execute query can be defined by providing values upstream. The document generator Snaps like JSON Generator can be used in upstream. Expected downstream Snaps: The Snap will output one document for every record retrieved, hence any document processing Snap can be used downstream. While trying to access a column name that contains specific characters as supported by Azure SQL such as $, #, @ etc., such field names should be enclosed in the square brackets. Modes
| |||||||||
| Prerequisites: | [None] | |||||||||
| Limitations and Known Issues: | When the SQL statement property is an expression, the pipeline parameters are shown in the suggest, but not the input schema. | |||||||||
| Behavior Change |
If you have any existing Pipelines that are mapped with status key or previous description then those Pipelines will fail. So, you might need to revisit your Pipeline design. | |||||||||
| Configurations: | Account & AccessThis Snap uses account references created on the Accounts page of SnapLogic Manager to handle access to this endpoint. See Azure SQL Account for information on setting up this type of account. Views
| |||||||||
| Troubleshooting: | None | |||||||||
Settings | ||||||||||
Label* | Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline. | |||||||||
SQL statement* | Specifiy the SQL statement to execute on the server. There are two possible scenarios that you encounter when working with SQL statements in SnapLogic. Scenarios to successfully execute your SQL statements Scenario 1: Executing SQL statements without expressions
Examples:
Additionally, the JSON path is allowed only in the WHERE clause. If the SQL statement starts with SELECT (case-insensitive), the Snap regards it as a select-type query and executes once per input document. If not, it regards it as write-type query and executes in batch mode. Scenario 2: Executing SQL queries with expressions
Table name and column names must not be provided as bind parameters. Only values can be provided as bind parameters. Examples:
Single quotes in values must be escaped Any relational database (RDBMS) treats single quotes ( For example:
Default value: [None] | |||||||||
Pass through | If selected, the input document will be passed through to the output view under the key 'original'. This property applies only to the Execute Snaps with SELECT statement. Default value: Selected | |||||||||
Ignore empty result | If selected, no document will be written to the output view when a SELECT operation does not produce any result. If this property is not selected and the Pass through property is selected, the input document will be passed through to the output view. Default value: Not selected | |||||||||
Auto commit | Select one of the options for this property to override the state of the Auto commit property on the account. The Auto commit at the Snap-level has three values: True, False, and Use account setting. The expected functionality for these modes are:
Default value: Use account setting 'Auto commit' may be enabled for certain use cases if PostgreSQL JDBC driver is used in either Redshift, PostgreSQL or Generic JDBC Snap. But the JDBC driver may cause out of memory issues when Select statements are executed. In those cases, “Auto commit" in Snap property should be set to ‘False’ and the Fetch size in the “Account setting" can be increased for optimal performance. Behavior of DML Queries in Database Execute Snap when auto-commit is false DDL queries used in the Database Execute Snap will be committed by the Database itself, regardless of the Auto-commit setting. When Auto commit is set to false for the DML queries, the commit is called at the end of the Snap's execution. The Auto commit needs to be true in a scenario where the downstream Snap does depend on the data processed on an Upstream Database Execute Snap containing a DML query. When the Auto commit is set to the Use account setting on the Snap, the account level commit needs to be enabled. | |||||||||
Snap execution | Select one of the three modes in which the Snap executes. Available options are:
| |||||||||
.png?version=1&modificationDate=1500437287427&cacheVersion=1&api=v2)
.png?version=1&modificationDate=1500437286406&cacheVersion=1&api=v2)
Basic Use Case
The following pipeline describes how the Snap functions as a standalone Snap in a pipeline:
- Extract: The SQL statement, select * from <table_name>, extracts the Azure table data.
Typical Snap Configurations
The key configuration of the Snap lies in how you pass the SQL statement to read Azure records. As it applies in SnapLogic, you can pass SQL statements in the following manner:
Without Expression: Directly passing the required SQL statement in the Azure SQL Execute Snap.
With Expressions
- Values from an upstream Snap: JSON Generator Snap passing the values to be inserted on to the table on Azure.
- Pipeline Parameter: Pipeline parameter set to pass the required values to the Azure.
Use Case
The following describes a pipeline, with a broader business logic involving the ETL transformations, that shows how typically in an enterprise environment, an execute functionality is used. Pipeline download link below.
This pipeline reads and moves files from the SQL Server Database to the Azure SQL Database and the Azure SQL Execute Snap reads the newly loaded table on the Azure SQL instance.
Extract: The SQL Server Select Snap reads the data from the SQL Server Database.
Load: The Azure SQL Execute Snap inserts the data into a Azure SQL table.
Read: Another Execute Snap is used to read the data from the newly loaded table on the Azure SQL database.
Downloads