Postgres Dynamic Account

On this page

Overview

Use this account type to connect PostgreSQL Snaps with data sources that use PostgreSQL accounts.

Expression-enabled authentication fields, such as Username, Password, and Client Secret, support Secrets Management, a SnapLogic add-on that allows you to store endpoint credentials in a third-party secrets manager, such as AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault. During validation and execution, pipelines obtain the credentials directly from the secrets manager. Learn more: Configure Accounts to use secrets.

Prerequisites

None.

Account Settings

Parameter NameData TypeDescriptionDefault ValueExample 
LabelStringThe name for the account. We recommend that you update the account name if there is more than one account in your project.N/APostgreSQL_Account_Type
Authentication methodDropdown list

Select the authentication method to create your Postgres Account. The available options are:

  • User ID and password

  • IAM Auth

User ID and passwordIAM Auth
Account propertiesNA

Required. The account information required to create a connection to the PostgreSQL database.

N/AN/A

Hostname

String

Required. The PostgreSQL server's address.

This property is expression-enabled. For more information on the expression language, see Understanding Expressions in SnapLogic and Using Expressions. For information on Pipeline Parameters, see Pipeline Properties.

N/Aacc-postgresql-us-test-rsd.amazonaws.com

Port Number

Integer

Required. The port number to use to make the server connection.

This property is expression-enabled. For more information on the expression language, see Understanding Expressions in SnapLogic and Using Expressions. For information on Pipeline Parameters, see Pipeline Properties.

33061234

Database name

String/Expression

Required. The name of the database to which you want to connect.

This property is expression-enabled. For more information on the expression language, see Understanding Expressions in SnapLogic and Using Expressions. For information on Pipeline Parameters, see Pipeline Properties.

N/ATestDB

Username

String/Expr

Required. The username that is allowed to connect to the database. This username will be used as the default username when retrieving connections. The username must be valid in order to set up the data source.

This property is expression-enabled. For more information on the expression language, see Understanding Expressions in SnapLogic and Using Expressions. For information on Pipeline Parameters, see Pipeline Properties.

N/Atestaccount

Password

String

Required. The password associated with the username listed above. The password must be valid in order to set up the data source.

This property is expression-enabled. For more information on the expression language, see Understanding Expressions in SnapLogic and Using Expressions. For information on Pipeline Parameters, see Pipeline Properties.

N/AN/A

JDBC Jars

N/AThe list of JDBC jar files that must be loaded for the account to work.N/AN/A

JDBC Driver

String

The JDBC driver that must be loaded for the account to work. By default, this field has one row populated. Click the  icon to add more rows.

{Default Driver postgres 42.7.2.jar}postgresql-9.3-1104.jdbc4.jar

JDBC Driver Class

StringThe JDBC driver class that you want to use with the account.org.postgresql.DriverN/A
IAM propertiesAppears when you select the IAM Auth for the Authentication methodConfigure the IAM-releated properties for IAM authentication.
IAM roleCheckbox

Appears when you select the IAM Auth for the Authentication method.

Select this checkbox to use the IAM role stored in the Groundplex EC2 instance.

Note:

DeselectedSelected
AWS access key IDString/Expression

Appears when you select the IAM Auth for the  Authentication method and deselect IAM role checkbox.

Specify the AWS access key ID associated with your AWS account.

N/AASIAEXAMPLEAWS
AWS secret access keyString/Expression

Appears when you select the IAM Auth for the Authentication method and deselect IAM role checkbox.

Specify the Secret access key associated with your AWS authentication.

N/AwJalrXUtnFEMI/K7ENG/bPxRfiCEKEY
AWS security tokenString/Expression

Appears when you select the IAM Auth for the Authentication method and deselect IAM role checkbox.

Specify the Security Token used to authenticate your requests to AWS servicesOnly global Security Token Service (STS) regions are supported.

N/AFQoGZXIvYXdzEFYaDO/7V77y1HTi1
a0B7CK4AQpdKsNC6M/4X8e0fH4QE/...
AWS regionString/Expression

Appears when you select the IAM Auth for the Authentication method and select IAM role checkbox.

Specify the AWS region where the application is running.


N/Aus-east-1
Cross account IAM propertiesUse this field set to configure the cross-account access. Learn more about - Setting up Cross account IAM Role.
Role ARNString/ExpressionSpecify the Amazon Resource Name (ARN) of the role to assume.N/Aarn:aws:iam::61286491850:role/EC2_role
External IDString/ExpressionSpecify an External ID that might be required by the role to assumeN/A7542158

Configure SSH Tunnel


Checkbox

Select this checkbox if Snap needs to create an SSH tunnel to connect the Snaplex to the PostgreSQL server. After the operation is completed, the tunnel is closed.

If you select true, you must provide the configuration details of the SSH tunnel.

Learn more about Setting Up SSH Tunneling with PostgreSQL.

DeselectedSelected

SSH Auth Mode


String/Expression

Select an option to specify the mode for authenticating the user on the SSH tunnel. The available options are:

  • Password: Select this option if you want to use the configured SSH Hostname, SSH Username, and the SSH Password properties directly.

  • KeyFile (Private Key File): Select this option if you want to upload a file for the Key.

  • KeyFile (Private Key String): Select this option if you want to copy and paste the private key contents into the account.

This option is available only if the Configure SSH Tunnel checkbox is selected.

Do not enable expressions in this field. If you enable an expression, the other fields based on it might not be available.

PasswordKeyFile

 SSH Hostname

String/Expression

Specify either the IP address or the domain name of the SSH server to which you want to connect


None_SSH_Hostname

SSH Username


String/Expression

Specify the SSH username for connecting to the SSH tunnel. 

None_SSH_User

SSH Password


String/Expression

Specify the password for the specified SSH username for connecting to the SSH tunnel. This field is required if the SSH Auth Mode is Password.

None<Encrypted>

Private Key File URL


String/Expression

Specify the location of the keystore file. This field is required if the SSH Auth Mode is KeyFile (Private Key File).

The file can be in SLDB, on the host machine that is hosting the JCC, or at any other remote location. For a local file, click  to select the appropriate file using the file browser. You can also upload the file using any protocol such as https, ftp, sldb, and sftp.

None_PK_File

Private Key


String/Expression

Specify the private key for authentication. This field is required if the SSH Auth Mode is KeyFile (Private Key String).

None_PrivateKey

Private Key Passphrase 

String/Expression

Specify the password is to be used to access the Key File URL or to decrypt the Private Key. If no value is provided, the keystore password is used. This field is required if the SSH Auth Mode is KeyFile.

None:y<>6[[]gMssb^rM

SSH Port


Integer/Expression

Specify the SSH port to connect to the PostgreSQL Server.

Ensure that there are no port conflicts

None22222
Advanced PropertiesN/AAdvanced properties associated with the PostgreSQL account type.N/AN/A

Auto commit

Checkbox

Select this check box to commit batches as soon as they are executed. If the Snap fails, only the batch being executed at that moment is rolled back.

Deselect the check box to commit Snap execution output only after all the batches are executed. If the Snap fails, the entire transaction is rolled back unless the Snap finds invalid input data before it sends the insert request to the server and routes the error documents to the Error view.

SelectedN/A

Batch size

Integer

Required. The number of statements to execute at a time.

Select queries are not batched.

Using a large batch size could use up the JDBC placeholder limit of 2100.

5020

Fetch size

Integer

Required. The number of rows that you want the Snap to fetch at a time when executing a query.

Large values could cause the server to run out of memory.

10050

Min pool size

IntegerRequired. The minimum number of connections that a pool should maintain at a time.35

Max pool size

IntegerRequired. The maximum number of connections that a pool should maintain at a time.5030

Max life time

Integer

Required. The maximum lifetime (in minutes) of a connection in the pool. Ensure that the value you enter is less than any database or infrastructure-imposed connection time limit. A value of 0 indicates an infinite lifetime, subject to the Idle Timeout value.

An in-use connection is never retired. Connections are removed only after they are closed.

3050

Idle Timeout

IntegerRequired. The maximum amount of time (in minutes) for which a connection is allowed to sit idle in the pool. A value of 0 indicates that idle connections are never removed from the pool.510

Checkout timeout

IntegerRequired. The number of milliseconds for which the Snap waits for a connection to be available when the pool is exhausted. A value of 0 indicates an infinite timeout.
The Snap throws an exception after the wait time expires.
100008000
URL propertiesFieldsetUse this fieldset to configure the URL that connects to your account.N/AN/A

URL property name

StringThe name of the URL property.N/AsocketTimeout

URL property value

stringThe value of the URL property.N/A10

Troubleshooting

ErrorReasonResolution

JDBC driver hangs in case of a connection error

The default socketTime value in PostgreSQL accounts is 0. This causes the JDBC driver to hang if there are any connection errors.

To overcome this, configure the URL properties in Account settings with the following parameter and value:

URL property name: socketTimeout
URL property value: 10
A SocketTimeout value of 10 ensures that the JDBC driver returns no connections after 10 seconds.

Timeout issue

A connection failure does not suspend the running PostgreSQL Pipelines if the timeout value is lesser than the time to process the query. 

To overcome this issue, set the URL property socketTimeout with a longer time period than that of the longest expected query.

  • URL property name: socketTimeout
  • URL property value:<time in seconds>

SSL connection validation issue

When connecting to an SSL-enabled PostgreSQL account, the account fails in validation.

To overcome this, configure the URL properties in Account settings to establish an SSL connection with the following parameter and value:

  • URL property name: slmode
  • URL property values: require

Auto-commit with Execute Snaps

For a DB Execute Snap, assume that a stream of documents enters the input view of the Snap and the SQL statement property has JSON paths in the WHERE clause. If the number of documents is large, the Snap executes in more than one batches rather than executing one per each document. Each batch would contain a certain number of WHERE clause values. If Auto commit is turned on, a failure would only roll back the records in the current batch. If Auto commit is turned off, the entire operation would be rolled back. For a single execute statement (with no input view), the setting has no practical effect.