Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this article

Table of Contents
maxLevel2
absoluteUrltrue

Overview

You can use this account type to connect JDBC Snaps with data sources that use Generic Database account.

Prerequisites

None.

Limitations

None.

Known Issue

Generic JDBC Snaps connecting to Informix database through the Generic Database account can cause thread leaks due to JDBC driver implementations.

Workaround: To avoid these thread leaks, do the following:

  • Use the Informix JDBC driver version 4.50.4.1 or a higher version.
  • Configure the following URL properties in the account settings:
    • Url Property name: IFMXCONNECTION_CLEANER_THREADS
    • Url property value: 0

Account Settings

Image Removed

Note

Field names marked with an asterisk ( * ) are mandatory.

...

Field Name

...

Field Type

...

Description

...

Label*

...

String

...

In this article

Table of Contents
maxLevel2
absoluteUrltrue

Overview

You can use this account type to connect JDBC Snaps with data sources that use the Generic Database account.

...

Prerequisites

None.

Limitations

None.

Known Issues

Generic JDBC Snaps connecting to the Informix database through the Generic Database account can cause thread leaks because of JDBC driver implementations.

Workaround: To avoid these thread leaks, do the following:

  • Use the Informix JDBC driver 4.50.4.1 version or a later version.

  • Configure the following URL properties in the account settings:

    • Url Property name: IFMXCONNECTION_CLEANER_THREADS

    • Url property value: 0

Account Settings

Info
  • Asterisk ( * ): Indicates a mandatory field.

  • Suggestion icon (Image Added): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon (Image Added): Indicates whether the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon (Image Added): Indicates that you can add fields in the field set.

  • Remove icon (Image Added): Indicates that you can remove fields from the field set.

Field Name

Field Type

Field Dependency

Description

Label*


Default Value: N/A
Example: Generic Database Account

String

N/A

Specify a unique label for the account.


Account properties*

The information required to create a connection to the database.

JDBC JARs

Use this field set to define JDBC Drivers. This field set consists of the JDBC Driver field.

JDBC Driver*

Default Value: N/A
Examplevertica-jdk5-6.1.2-0.jar

String/Expression

N/A

Specify the JDBC driver to use. Click the Upload Image Addedicon to upload the required JDBC driver that must be used for the account.

  • Type 3 and Type 4 JDBC drivers are only supported. Type 4 JDBC driver is recommended. For more information on JDBC driver versions and downloads, see https://www.ibm.com/support/pages/db2-jdbc-driver-versions-and-downloads

  • You can upload the JDBC drivers through Designer or Manager and it is stored on a per project basis. That is, only users with access to that project can view the uploaded JDBC drivers. To provide access to all users of your org, place the driver in the /shared project.

  • We recommend you to use the db2jcc.jar driver for DB2 with your JDBC account. The DB2 version 11.5 supports only JDBC 4.0 Driver (db2jcc4.jar) and 11.1 version supports JDBC 3.0 driver (db2jcc.jar) and JDBC 4.0 Driver (db2jcc4.jar).

JDBC Driver Class*


Default Value: [None]
Example: com.vertica.jdbc.Driver

String/Expression

N/A

Specify the JDBC Driver class name to use.


JDBC URL

Default Value: [None]
Example: jdbc:vertica://Snaplogic.com/database

String/Expression

N/A

Specify the JDBC URL to use.


Username


Default Value: N/A
Example: Snapuser

String/Expression

N/A

Specify the database username to use.


Password

Default Value: N/A
Example: nb*#!@09

String/Expression

N/A

Specify the database password to use.


Database name

Default Value: Auto detect
Example: Oracle

String/Expression

N/A

Choose a database to which the account must be connected. The available options are:

  • Auto detect: If you select Auto detect, and if the Snap detects the target database as NetSuite, then the Limit rows field in the Generic JDBC - Select Snap is ignored.

  • PostgreSQL

  • Redshift

  • NetSuite: If you select NetSuite, then the Limit rows field in the Generic JDBC - Select Snap ignored irrespective of whether you provide a value or not.

  • MySQL

  • Oracle

  • SQL Server 2012

  • SQL Server 2008

  • SAPHana

  • Apache Hive

  • DB2

  • SQLMX

  • Apache Derby

  • Spark SQL

If you use PostgreSQL JDBC driver to connect to Redshift database, the Auto detect option automatically detects the PostgreSQL database instead of Redshift. The behavior of the JDBC Snaps is optimized for the selected database.

Test Query

Default Value: N/A
Example:

  • SQL Query:

    • SELECT * FROM customers WHERE age > 30

  • MongoDB Query:

    • db.customers.find({ age: { $gt: 30 } })

  • Redshift Query:

    • SELECT customer_id, order_date, total_amount
      FROM sales
      WHERE total_amount > 1000
      ORDER BY order_date DESC
      LIMIT 10;

String/Expression

Activates on selecting Auto detect for Database name.

Specify a custom query to validate the database connection.

  • The Test Query must be effective as this executes multiple times during the lifecycle of a connection. The efficiency of a Test Query directly affects the efficiency of the Pipeline execution. All databases do not support all test queries. Hence, use a query that supports your database.

  • Select 1 test query is supported only in H2, MySQL, PostgreSQL, and SQLite databases.

Configure SSH tunnel

Default Value: Deselected

Checkbox

N/A

Select this checkbox to connect to the database server through the SSH tunnel. After the operation is completed, the tunnel is closed.

Learn more about SSH Tunneling Testing with PostgreSQL.

SSH auth mode

Default Value: Password
Example: KeyFile

Dropdown list

N/A

Select an option to specify the mode for authenticating the user on the SSH tunnel. The available options are:

  • Password: Uses the SSH hostname, username, and password for authentication.

  • KeyFile (Private Key File): Uses the Private key file for authentication, which you can upload from SLDB into the account after setting up the SSH tunnel.

  • KeyFile (Private Key String): Uses the Private key string for authentication, which you can upload from SLDB into the account after setting up the SSH tunnel.

SSH hostname

Default Value: N/A
Example: 127.0.0.1

String/Expression

N/A

Specify the IP address or the domain name of the SSH server to which you want to connect.

SSH username

Default Value: N/A
Example: SSHUser

String/Expression

N/A

Specify the SSH username that is authorized to connect to the database. This username is used as the default username when retrieving connections.

The username must be valid to set up the data source.

SSH password

Default Value: N/A
Example: <Encrypted>

String/Expression

Appears if SSH Auth Mode is Password.

Specify the password for the SSH username for connecting to the SSH tunnel. 

Private key file URL

Default Value: N/A
Example: postgres-ssh.pem

String/Expression

Appears if SSH Auth Mode is KeyFile (Private Key File).

Specify the location of the private key file.

The file can be in SLDB, on the host machine that is hosting the JCC, or at any other remote location. Click the File browser icon to upload the file from your local system. You can also upload the file using any protocol such as HTTPS, FTP, SLDB, and SFTP.

Private key*

Default Value: N/A
Example:

Generic Database AccountAccount properties*The information required to create a connection to the database.

JDBC JARs

Use this field set to define JDBC Drivers. This field set consists of the JDBC Driver field.

JDBC Driver*String

Specify the JDBC driver to use. Click the Upload Image Removedicon to upload the JDBC driver that you want. Click Add Image Removed to add a row for specifying a JDBC driver. Specify each driver in a separate row.

Note
  • Type 3 and Type 4 JDBC drivers are only supported. Type 4 JDBC driver is recommended. For more information on JDBC driver versions and downloads, see https://www.ibm.com/support/pages/db2-jdbc-driver-versions-and-downloads
  • You can upload the JDBC drivers through Designer or Manager and it is stored on a per project basis. That is, only users with access to that project can view the uploaded JDBC drivers. To provide access to all users of your org, place the driver in the /shared project.
  • We recommend you to use the db2jcc.jar driver for DB2 with your JDBC account. The DB2 version 11.5 supports only JDBC 4.0 Driver (db2jcc4.jar) and 11.1 version supports JDBC 3.0 driver (db2jcc.jar) and JDBC 4.0 Driver (db2jcc4.jar).

Default Value: N/A
Examplevertica-jdk5-6.1.2-0.jar

JDBC Driver Class*

String

Specify the JDBC Driver class name to use.

Default Value: [None]
Examplecom.vertica.jdbc.Driver

JDBC URL

String

Specify the JDBC URL to use.

Default Value: [None]
Example: jdbc:vertica://Snaplogic.com/database

UsernameString

Specify the database username to use.

Default Value: N/A
ExampleSnapuser

PasswordString

Specify the database password to use.

Default Value: N/A
Example: k*#!@09

Database nameString

Choose a database to which the account must be connected. The available options are:

Auto detect

Info

If you select Autodetect and if the Snap detects the target database as Netsuite then the Limit rows field in the Generic JDBC - Select Snap is ignored.

  • PostgreSQL
  • Redshift
  • Netsuite

    Info

    If you select Netsuite then the Limit rows field in the Generic JDBC - Select Snap is ignored irrespective of whether you provide a value or not.

  • MySQL
  • Oracle
  • SQL Server 2012
  • SQL Server 2008
  • SAPHana
  • Apache Hive
  • DB2
  • SQLMX
  • Apache Derby
  • Spark SQL
  • If you use PostgreSQL JDBC driver to connect to Redshift database, the Auto detect option automatically detects the PostgreSQL database instead of Redshift. The behavior of the JDBC Snaps is optimized for the selected database.

    Default Value: Auto detect
    Example: Oracle

    Test QueryString

    Activates on selecting on selecting Auto detect for Database name.

    Specify a custom query to validate the database connection.

    Note
    • The Test Query must be effective as this executes multiple times during the lifecycle of a connection. The efficiency of a Test Query directly affects the efficiency of the Pipeline execution. All databases do not support all test queries. Hence, use a query that supports your database.
    • Select 1 test query is supported only in H2, MySQL, PostgreSQL, and SQLite databases.
    Default Value: N/A
    Example: Select 1
    Advanced properties*Min pool size*IntegerSpecify the minimum number of idle connections a pool will maintain at a time.
    Default Value: 0
    Example: 0
    Note

    If the size is set to non-zero, JCC restart is needed when the database account expires.

    Max pool size*IntegerSpecify the maximum number of idle connections a pool will maintain at a time.
    Default Value: 15
    Example: 10
    Max idle time*IntegerSpecify the number of seconds a connection will exist in the pool before being destroyed.
    Default Value: 60 
    Example: 300
    Checkout timeout*IntegerSpecify the number of milliseconds to wait for a connection to be available in the pool. A timeout of zero makes the Snap wait infinitely.  After a certain time, an exception is thrown and the Pipeline fails.
    Default Value: 10000
    Example:10000
    Url propertiesUse this field set to define URL properties to use in the JDBC URL.
    Url property nameString

    Specify the name for the URL property.

    Default Value: N/A
    Example: maxAllowedPacket

    Url property valueString

    Specify a value for the property name.

    Default Value: N/A
    Example1000

    Auto commitCheckboxSelect this checkbox to commit a batch immediately after the batch executes. So, only the current executing batch will be rolled back if the Snap fails.
    If you deselect, then a transaction is started for the Snap run and committed upon run success. The transaction is rolled back if the Snap fails.
    Default Value: SelectedFetch size*IntegerSpecify the number of records to retrieve from the DB at a time.
    Default Value: 100
    Example: 100
    Batch size*Integer

    Specify the number of query statements to execute at a time. SELECT queries are not batched.

    • If the Batch Size is one, the query is executed as-is, that is the Snap skips the batch (non-batch execution).
    • If the Batch Size is greater than one, the Snap performs the regular batch execution.
    Info

    If you are using AWS Athena database, the Batch size must be set to 1.

    Default Value: 50
    Example: 10

    ...

    See Also

    ...

    -----BEGIN RSA PRIVATE KEY-----

    ………………..

    -----END RSA PRIVATE KEY-----

    String/Expression

    Appears if SSH Auth Mode is KeyFile (Private Key String).

    Specify the private key for authentication.

    Private key passphrase

    Default Value: N/A
    Example: 3x@mpl3_P@ssw0rd!

     

    String/Expression

    Appears if SSH Auth Mode is KeyFile.

    Specify the passphrase that is to be used to decrypt the private key.

     

    SSH port

    Default Value: N/A
    Example: 222

    Integer/Expression

    N/A

    Specify the SSH port to connect to any of the following database servers:

    • PostgreSQL

    • Redshift

    • NetSuite

    • MySQL

    • Oracle

    • SQL Server 2012

    • SQL Server 2008

    • SAPHana

    • Apache Hive

    • DB2

    • SQLMX

    • Apache Derby

    • Spark SQL

    Info

    Ensure that there are no port conflicts.

    Advanced properties*


    Min pool size*

    Default Value: 0
    Example: 0

    Integer/Expression

    N/A

    Specify the minimum number of idle connections a pool will maintain at a time.

    If the size is set to non-zero, JCC restart is needed when the database account expires.


    Max pool size*

    Default Value: 15
    Example: 10

    Integer/Expression

    N/A

    Specify the maximum number of idle connections a pool will maintain at a time.


    Max idle time*

    Default Value: 60 
    Example:100

    Integer/Expression

    N/A

    Specify the number of minutes a connection must remain in the connection pool before being discarded.


    Checkout timeout*

    Default Value: 10000
    Example:10000

    Integer/Expression

    N/A

    Specify the number of milliseconds to wait for a connection to be available in the pool.

    If you provide 0, the Snap waits infinitely until the connection is available. Therefore, we recommend you not to specify 0 for Checkout Timeout.

    Url properties

    Use this field set to define URL properties to use in the JDBC URL.

    Url property name

    Default Value: N/A
    Example:

    • maxAllowedPacket

    • encrypt

    • selectMethod

    String

    N/A

    Specify a name for the URL property to be used by the account. For instance:

    • If you want the account to use trust certificates, specify the Url property name as trustServerCertificate and the value as true.

    • If you do not want the account to use encryption, specify the Url property name as encrypt and the value as false.

    • If you want to specify the method for selecting and navigating data in the database, use the Url property name as selectMethod, and specify the value as cursor.

    Url property value

    Default Value: N/A
    Example

    • 1000

    • false

    • cursor

    String

    N/A

    Specify a value for the URL property name.


    Auto commit

    Default Value: Selected

    Checkbox

    N/A

    Select one of the options for this property to override the state of the Auto commit property on the account. The Auto commit at the Snap-level has three values: TrueFalse, and Use account setting. The expected functionality for these modes are:

    •  True - The Snap will execute with auto-commit enabled regardless of the value set for Auto commit in the Account used by the Snap.

    •  False - The Snap will execute with auto-commit disabled regardless of the value set for Auto commit in the Account used by the Snap.

    • Use account setting - The Snap will execute with Auto commit property value inherited by the Account used by the Snap.

    • 'Auto commit' may be enabled for certain use cases if PostgreSQL jdbc driver is used in either Redshift, PostgreSQL or generic JDBC Snap. But the JDBC driver may cause out of memory issues when Select statements are executed. In those cases, “Auto commit" in Snap property should be set to ‘False’ and the Fetch size in the “Account setting" can be increased for optimal performance.

    • For a DB Execute Snap, assume that a stream of documents are passed to the input view of the Snap and the SQL statement property has JSON paths in the WHERE clause. If the number of documents are large, the Snap executes in more than one batch rather than executing one per each document. Each batch would contain a certain number of WHERE clause values. If Auto commit is turned on, a failure would only roll back the records in the current batch. If Auto commit is turned off, the entire operation would be rolled back. For a single execute statement (with no input view), the setting has no practical effect.

    Multiexcerpt include macro
    nameDDL Auto Commit
    pagePostgreSQL - Execute

    Fetch size*

    Default Value: 100
    Example: 100

    Integer/Expression

    N/A

    Specify the number of records to retrieve from the DB at a time.


    Batch size*

    Default Value: 50
    Example: 10

    Integer/Expression

    N/A

    Specify the number of query statements to execute at a time. SELECT queries are not batched.

    • If the Batch Size is one, the query is executed as-is, that is the Snap skips the batch (non-batch execution).

    • If the Batch Size is greater than one, the Snap performs the regular batch execution.

    If you are using AWS Athena database, the Batch size must be set to 1.

    Troubleshooting

    Error

    Reason

    Resolution

    Caused by: java.lang.AbstractMethodError

    Snap displays this error when the endpoint database driver lacks an implementation for the isValid() or any other method.

    To address this issue, consider the following troubleshooting methods:

    • Check database driver version: Ensure that you are using the latest version of the target database driver. Outdated drivers might lack support for certain methods.

    • Check documentation: Refer to the official documentation of the target database driver. Check if the isValid() method is supported and if any specific configurations are required.

    • Driver compatibility: Verify the compatibility of the database driver with the specific database version you are using. Incompatibility can lead to missing implementations for certain methods.

    • Alternative health check mechanisms: Explore alternative mechanisms to validate the database connection health, such as executing a test query or utilizing other available methods provided by the driver.

    • Community forums and support: Visit the official database driver community forums or seek support.

    Snap Pack History

    Expand

    Multiexcerpt include macro
    nameJDBC_SPH
    templateData[]
    pageJDBC Snap Pack
    addpanelfalse

    ...

    Related Content