On this page

Overview

Use this account type to connect PostgreSQL Snaps with data sources that use PostgreSQL accounts.

Expression-enabled authentication fields, such as Username, Password, and Client Secret, support Secrets Management, a SnapLogic add-on that allows you to store endpoint credentials in a third-party secrets manager, such as AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault. During validation and execution, pipelines obtain the credentials directly from the secrets manager. Learn more: Configure Accounts to use secrets.

Prerequisites

None.

Account Settings

Parameter Name	Data Type	Description	Default Value	Example
Label*	String	The name for the account. We recommend that you update the account name if there is more than one account in your project.	N/A	PostgreSQL_Account_Type
Authentication method	Dropdown list	Select the authentication method to create your Postgres Account. The available options are: User ID and password IAM Auth	User ID and password	IAM Auth
Account properties*	NA	Required. The account information required to create a connection to the PostgreSQL database.	N/A	N/A
Hostname*	String /Expression	Required. The PostgreSQL server's address. This property is expression-enabled. For more information on the expression language, see Understanding Expressions in SnapLogic and Using Expressions. For information on Pipeline Parameters, see Pipeline Properties.	N/A	acc-postgresql-us-test-rsd.amazonaws.com
Port number*	Integer /Expression	Required. The port number to use to make the server connection. This property is expression-enabled. For more information on the expression language, see Understanding Expressions in SnapLogic and Using Expressions. For information on Pipeline Parameters, see Pipeline Properties.	3306	1234
Database name*	String/Expression	Required. The name of the database to which you want to connect. This property is expression-enabled. For more information on the expression language, see Understanding Expressions in SnapLogic and Using Expressions. For information on Pipeline Parameters, see Pipeline Properties.	N/A	TestDB
Username	String /Expression	The username that is allowed to connect to the database. This username will be used as the default username when retrieving connections. The username must be valid in order to set up the data source. This property is expression-enabled. For more information on the expression language, see Understanding Expressions in SnapLogic and Using Expressions. For information on Pipeline Parameters, see Pipeline Properties. The Username is mandatory for both the authentication types: User ID and password: Example, snapuser IAM Auth: Example, iamuser	N/A	snapuser
Password	String /Expression	Required. The password associated with the username listed above. The password must be valid in order to set up the data source. This property is expression-enabled. For more information on the expression language, see Understanding Expressions in SnapLogic and Using Expressions. For information on Pipeline Parameters, see Pipeline Properties.	N/A	N/A
JDBC Jars	N/A	The list of JDBC jar files that must be loaded for the account to work.	N/A	N/A
JDBC driver	String	The JDBC driver that must be loaded for the account to work. By default, this field has one row populated. Click the icon to add more rows.	{Default Driver postgres 42.7.2.jar}	postgresql-9.3-1104.jdbc4.jar
JDBC driver class	String	The JDBC driver class that you want to use with the account.	org.postgresql.Driver	N/A
IAM properties	Appears when you select the IAM Auth* for the Authentication method.* Configure the IAM-releated properties for IAM authentication.
IAM role	Checkbox	Appears when you select the IAM Auth* for the Authentication method.* Select this checkbox to use the IAM role stored in the Groundplex EC2 instance. Note: The IAM role is valid only in Groundplex nodes hosted in the EC2 environment. Learn more about Configuring an EC2 role for IAM Role in AWS S3 Account. When you choose IAM Auth as the Authentication method, you must provide the username. When you select the IAM role checkbox, the AWS access key, AWS secret key, and AWS security token fields are hidden.	Deselected	Selected
AWS access key ID	String/Expression	Appears when you select the IAM Auth* for the Authentication method and deselect IAM role checkbox.* Specify the AWS access key ID associated with your AWS account.	N/A	ASIAEXAMPLEAWS
AWS secret access key	String/Expression	Appears when you select the IAM Auth* for the Authentication method and deselect IAM role checkbox.* Specify the Secret access key associated with your AWS authentication.	N/A	wJalrXUtnFEMI/K7ENG/bPxRfiCEKEY
AWS security token	String/Expression	Appears when you select the IAM Auth* for the Authentication method and deselect IAM role checkbox.* Specify the Security Token used to authenticate your requests to AWS services. Only global Security Token Service (STS) regions are supported.	N/A	FQoGZXIvYXdzEFYaDO/7V77y1HTi1 a0B7CK4AQpdKsNC6M/4X8e0fH4QE/...
AWS region	String/Expression	Appears when you select the IAM Auth* for the Authentication method and select IAM role checkbox.* Specify the AWS region where the application is running.	N/A	us-east-1
Cross account IAM properties	Use this field set to configure the cross-account access. Learn more about - Setting up Cross account IAM Role.
Role ARN	String/Expression	Specify the Amazon Resource Name (ARN) of the role to assume.	N/A	arn:aws:iam::61286491850:role/EC2_role
External ID	String/Expression	Specify an External ID that might be required by the role to assume	N/A	7542158
Configure SSH Tunnel	Checkbox/Expression	Select this checkbox if Snap needs to create an SSH tunnel to connect the Snaplex to the PostgreSQL server. After the operation is completed, the tunnel is closed. If you select true, you must provide the configuration details of the SSH tunnel. Learn more about Setting Up SSH Tunneling with PostgreSQL.	Deselected	Selected
SSH auth mode*	Dropdown List/Expression	Select an option to specify the mode for authenticating the user on the SSH tunnel. The available options are: Password: Select this option if you want to use the configured SSH Hostname, SSH Username, and the SSH Password properties directly. KeyFile (Private Key File): Select this option if you want to upload a file for the Key. KeyFile (Private Key String): Select this option if you want to copy and paste the private key contents into the account. This option is available only if the Configure SSH Tunnel checkbox is selected. Do not enable expressions in this field. If you enable an expression, the other fields based on it might not be available.	Password	KeyFile
SSH hostname*	String/Expression	Specify either the IP address or the domain name of the SSH server to which you want to connect.	None	_SSH_Hostname
SSH username*	String/Expression	Specify the SSH username for connecting to the SSH tunnel.	None	_SSH_User
SSH password*	String/Expression	Specify the password for the specified SSH username for connecting to the SSH tunnel. This field is required if the SSH Auth Mode is Password.	None	<Encrypted>
Private key file URL*	String/Expression	Specify the location of the keystore file. This field is required if the SSH Auth Mode is KeyFile (Private Key File). The file can be in SLDB, on the host machine that is hosting the JCC, or at any other remote location. For a local file, click to select the appropriate file using the file browser. You can also upload the file using any protocol such as `https, ftp, sldb, and sftp.`	None	_PK_File
Private key	String/Expression	Specify the private key for authentication. This field is required if the SSH Auth Mode is KeyFile (Private Key String).	None	_PrivateKey
Private key passphrase	String/Expression	Specify the password is to be used to access the Key File URL or to decrypt the Private Key. If no value is provided, the keystore password is used. This field is required if the SSH Auth Mode is KeyFile.	None	:y<>6[[]gMssb^rM
SSH port*	Integer/Expression	Specify the SSH port to connect to the PostgreSQL Server. Ensure that there are no port conflicts	None	22222
Advanced properties	N/A	Advanced properties associated with the PostgreSQL account type.	N/A	N/A
Auto commit	Checkbox /Expression	Select this check box to commit batches as soon as they are executed. If the Snap fails, only the batch being executed at that moment is rolled back. Deselect the check box to commit Snap execution output only after all the batches are executed. If the Snap fails, the entire transaction is rolled back unless the Snap finds invalid input data before it sends the insert request to the server and routes the error documents to the Error view.	Selected	N/A
Batch size*	Integer /Expression	Required. The number of statements to execute at a time. Select queries are not batched. Using a large batch size could use up the JDBC placeholder limit of 2100.	50	20
Fetch size*	Integer /Expression	Required. The number of rows that you want the Snap to fetch at a time when executing a query. Large values could cause the server to run out of memory.	100	50
Min pool size	Integer	Required. The minimum number of connections that a pool should maintain at a time.	3	5
Max pool size*	Integer /Expression	Required. The maximum number of connections that a pool should maintain at a time.	50	30
Max lifetime (minutes)*	Integer /Expression	Required. The maximum lifetime (in minutes) of a connection in the pool. Ensure that the value you enter is less than any database or infrastructure-imposed connection time limit. A value of 0 indicates an infinite lifetime, subject to the Idle Timeout value. An in-use connection is never retired. Connections are removed only after they are closed.	30	50
Idle timeout (minutes)*	Integer/Expression	Required. The maximum amount of time (in minutes) for which a connection is allowed to sit idle in the pool. A value of 0 indicates that idle connections are never removed from the pool.	5	10
Checkout timeout (milliseconds)*	Integer/Expression	Required. The number of milliseconds for which the Snap waits for a connection to be available when the pool is exhausted. A value of 0 indicates an infinite timeout. The Snap throws an exception after the wait time expires.	10000	8000
URL properties	Fieldset	Use this fieldset to configure the URL that connects to your account.	N/A	N/A
URL property name	String/Expression	The name of the URL property.	N/A	socketTimeout
URL property value	String/Expression	The value of the URL property.	N/A	10

Troubleshooting

Error Reason Resolution

JDBC driver hangs in case of a connection error

The default socketTime value in PostgreSQL accounts is 0. This causes the JDBC driver to hang if there are any connection errors.

To overcome this, configure the URL properties in Account settings with the following parameter and value:

URL property name: socketTimeout

URL property value: 10

A SocketTimeout value of 10 ensures that the JDBC driver returns no connections after 10 seconds.

Timeout issue

A connection failure does not suspend the running PostgreSQL Pipelines if the timeout value is lesser than the time to process the query.

To overcome this issue, set the URL property socketTimeout with a longer time period than that of the longest expected query.

URL property name: socketTimeout
URL property value:<time in seconds>

SSL connection validation issue

When connecting to an SSL-enabled PostgreSQL account, the account fails in validation.

To overcome this, configure the URL properties in Account settings to establish an SSL connection with the following parameter and value:

URL property name: slmode
URL property values: require

Auto-commit with Execute Snaps

For a DB Execute Snap, assume that a stream of documents enters the input view of the Snap and the SQL statement property has JSON paths in the WHERE clause. If the number of documents is large, the Snap executes in more than one batches rather than executing one per each document. Each batch would contain a certain number of WHERE clause values. If Auto commit is turned on, a failure would only roll back the records in the current batch. If Auto commit is turned off, the entire operation would be rolled back. For a single execute statement (with no input view), the setting has no practical effect.

SnapLogic Documentation

PostgreSQL Dynamic Account

Overview

Prerequisites

Account Settings

Troubleshooting

Auto-commit with Execute Snaps