PostgreSQL - Bulk Upsert

PostgreSQL - Bulk Upsert

In this article

Overview

This Snap enables you to perform bulk insert or update operations (using the MERGE command) into the existing tables or any input data stream. The upsert operation updates existing rows if the specified value exists in the target table and inserts a new row if the specified value does not exist in the target table. You can define a delete condition to control whether records are updated, deleted, or inserted. If the condition is provided and met, the matching records are deleted.

postgresql-bulk-upsert.png

Prerequisites

A valid account with the required permissions. Learn more about Roles and Permissions.

Support for Ultra Pipelines

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

 

 

  • Min: 1

  • Max: 1

Requires the Upsert format and additional detail, if required, to insert new or update existing rows in bulk.

Output

Document

 

 

  • Min: 0

  • Max: 1

The output is in document view format. The data from the incoming document loaded into the destination table is the output of this Snap. It gives the load statistics after the operation is completed.

The output view also lists the number of rows that were updated, modified, or inserted in the target table.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Expression: JavaScript syntax to access SnapLogic Expressions to set field values dynamically (if enabled). If disabled, you can provide a static value. Learn more.

  • SnapGPT: Generates SnapLogic Expressions based on natural language using SnapGPT. Learn more.

  • Suggestion: Populates a list of values dynamically based on your Account configuration.

  • Upload: Uploads files. Learn more.

Learn more about the icons in the Snap Settings dialog.

Field Name

Field Type

Description

Field Name

Field Type

Description

Label*

 

 

String

The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Default ValuePostgreSQL Bulk Upsert
ExamplePostgreSQL Bulk Upsert

Schema name

String/Expression/Suggestion

The database schema name. Selecting a schema filters the Table name list to show only tables within that schema.

Default value: None.
Example: emp_bulk_upsert_schema

Table name

 

String/Expression/Suggestion

The table on which to execute the insert operation.

Default value: None.
Example: emp_mdm_master

Key columns*

Specify the conditions to check for existing entries in the target table.

Column*

String/Expression/Suggestion

Specify one or more columns from the target table to check for existing entries.

Delete upsert condition

 

String/Expression

Specify the delete condition to update, delete, or insert records.

When the delete upsert condition is not blank and the condition is met, the records are deleted.

Example: emp_name="Charlie"

Header provided

 

Checkbox

Select this checkbox if the header is included in the input schema.

HEADER, FORMAT, and ENCODING should not be provided as part of Additional COPY options.

Additional COPY options

Configure additional copy options, if any.

COPY options

String/Expression

Specify the COPY option.

Default value: None.
Example: DELIMITER '|', ESCAPE '`'

Snap Execution

Dropdown list

Select one of the following three modes in which the Snap executes:

  • Validate & Execute: Performs limited execution of the Snap based on Preview Document Count value in User settings and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime.

  • Execute only: Performs full execution of the Snap during pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Default ValueExecute only
Example: Validate & Execute

Troubleshooting

Error

Reason

Resolution

Error

Reason

Resolution

The Snap displays an error if you use an older version (below 15) of PostgreSQL.

An older version of PostgreSQL might exist. The MERGE command is available only from PostgreSQL version 15.

Use PostgreSQL DB version 15 to execute the MERGE command.

 

The MERGE command in PostgreSQL 15 version does not have a returning clause, which helps to output the number of records inserted/updated separately.

 

Key column name is required.

No key column(s) specified for checking for existing entries.

Please enter one or more key column names.

Key column name is not present in target table.

Incorrect key column(s) specified for checking for existing entries.

Please select one or more key column names from the suggestion box.

All columns in target table are key columns.

The merge will fail as all columns in the target table are key columns.

Please select one or more (but not all) key column names from the suggestion box.

Example

postgreSQL-validate.png

Prerequisites: A valid PostgreSQL account.

Delete, Insert, Update with Delete Upsert Condition using PostgreSQL Snaps

This example Pipeline demonstrates a complete database workflow that creates a PostgreSQL table, populates it with initial data, performs bulk upsert operations using CSV data, and retrieves the final results. The pipeline showcases database table management, data insertion, conditional updates, and data retrieval operations in a sequential manner.

  1. Configure the PostgreSQL - Multi Execute Snap to execute multiple SQL statements. This includes dropping any existing target table, creating a new table with columns for id, name, quantity, and price, and inserting initial sample data records: Apple, Banana, and Grapes records.

    postgremultiexecute.png
  2. Configure the CSV Generator Snap to generate CSV content containing product data with updated quantities and prices for both existing and new items.

  3. Configure the PostgreSQL - Bulk Upsert to perform bulk upsert operations on the target_table in the public schema.

    postgre-bulk-upsert.png
  4. After validation, this Snap performs bulk upsert operations on the target_table using the CSV data, with id as the key column, and includes a conditional delete operation that removes records where id equals 5 and name equals Grapes. The output displays the details of the updated, inserted, and deleted records.

    postgre-bulk-upsert-json.png

     

  5. After validation, all records from the target table are updated, and the data can be further used for other downstream operations.

    postgreselect-output.png

Downloads

  1. Download and import the pipeline into the SnapLogic application.

  2. Configure Snap accounts, as applicable.

  3. Provide pipeline parameters, as applicable.

  File Modified

File PostgreSQL_BulkUpsert_Example.slp

Jan 19, 2026 by Anand Vedam

Snap Pack History

PostgreSQL Snap Pack

Related Content