In this article
Table of Contents | ||||
---|---|---|---|---|
|
Overview
You can use this account type to connect JDBC Snaps with data sources that use the Generic Database account.
Prerequisites
None.
Limitations
None.
Known Issue
Generic JDBC Snaps connecting to Informix database through the Generic Database account can cause thread leaks due to JDBC driver implementations.
Workaround: To avoid these thread leaks, do the following:
Use the Informix JDBC driver 4.50.4.1 version or a later version.
Configure the following URL properties in the account settings:
Url Property name: IFMXCONNECTION_CLEANER_THREADS
Url property value: 0
Account Settings
...
Info |
---|
|
...
Field Name
...
Field Type
...
Description
Label*
Default Value: N/A
Example: Generic Database Account
...
String
Specify a unique label for the account.
...
Account properties*
...
The information required to create a connection to the database.
...
JDBC JARs
...
Use this field set to define JDBC Drivers. This field set consists of the JDBC Driver field.
JDBC Driver*
Default Value: N/A
Example: vertica-jdk5-6.1.2-0.jar
...
String/Expression
...
Specify the JDBC driver to use. Click the Upload icon to upload the JDBC driver that must be used for the account.
Note |
---|
|
JDBC Driver Class*
Default Value: [None]
Example: com.vertica.jdbc.Driver
...
String/Expression
Specify the JDBC Driver class name to use.
JDBC URL
Default Value: [None]
Example: jdbc:vertica://Snaplogic.com/database
...
String/Expression
Specify the JDBC URL to use.
Username
...
String/Expression
Specify the database username to use.
Password
Default Value: N/A
Example: nb*#!@09
...
String/Expression
Specify the database password to use.
Database name
Default Value: Auto detect
Example: Oracle
...
String/Expression
...
Choose a database to which the account must be connected. The available options are:
Auto detect: If you select Autodetect and if the Snap detects the target database as NetSuite then the Limit rows field in the Generic JDBC - Select Snap is ignored.
PostgreSQL
Redshift
NetSuite: If you select NetSuite, then the Limit rows field in the Generic JDBC - Select Snap ignored irrespective of whether you provide a value or not.
MySQL
Oracle
SQL Server 2012
SQL Server 2008
SAPHana
Apache Hive
DB2
SQLMX
Apache Derby
Spark SQL
If you use PostgreSQL JDBC driver to connect to Redshift database, the Auto detect option automatically detects the PostgreSQL database instead of Redshift. The behavior of the JDBC Snaps is optimized for the selected database.
Test Query
Default Value: N/A
Example: Select 1
...
String/Expression
...
Activates on selecting Auto detect for Database name.
Specify a custom query to validate the database connection.
Note |
---|
|
...
Advanced properties*
Min pool size*
Default Value: 0
Example: 0
...
Integer/Expression
Specify the minimum number of idle connections a pool will maintain at a time.
Note |
---|
If the size is set to non-zero, JCC restart is needed when the database account expires. |
Max pool size*
Default Value: 15
Example: 10
...
Integer/Expression
...
Max idle time*
Default Value: 60
Example:100
...
Integer/Expression
...
Checkout timeout*
Default Value: 10000
Example:10000
...
Integer/Expression
...
Specify the number of milliseconds to wait for a connection to be available in the pool.
If you provide 0, the Snap waits infinitely until the connection is available. Therefore, we recommend you not to specify 0 for Checkout Timeout.
...
Url properties
...
Use this field set to define URL properties to use in the JDBC URL.
Url property name
Default Value: N/A
Example:
maxAllowedPacket
encrypt
selectMethod
...
String
...
Specify a name for the URL property to be used by the account. For instance:
If you want the account to use trust certificates, specify the Url property name as
trustServerCertificate
and the value astrue
.If you do not want the account to use encryption, specify the Url property name as
encrypt
and the value asfalse
.If you want to specify the method for selecting and navigating data in the database, use the Url property name as
selectMethod
, and specify the value ascursor
.
...
Url property value
Default Value: N/A
Example:
1000
false
cursor
...
String
Specify a value for the URL property name.
...
Auto commit
Default Value: Selected
...
Checkbox
...
Select this checkbox to commit a batch immediately after the batch executes. So, only the current executing batch will be rolled back if the Snap fails.
If you deselect, then a transaction is started for the Snap run and committed upon run success. The transaction is rolled back if the Snap fails
...
Fetch size*
Default Value: 100
Example: 100
...
Integer/Expression
...
Batch size*
Default Value: 50
Example: 10
...
Specify the number of query statements to execute at a time. SELECT queries are not batched.
If the Batch Size is one, the query is executed as-is, that is the Snap skips the batch (non-batch execution).
If the Batch Size is greater than one, the Snap performs the regular batch execution.
If you are using AWS Athena database, the Batch size must be set to 1.
...
Related Content
...
...
In this article
Table of Contents | ||||
---|---|---|---|---|
|
Overview
You can use this account type to connect JDBC Snaps with data sources that use the Generic Database account.
...
Prerequisites
None.
Limitations
None.
Known Issues
Generic JDBC Snaps connecting to the Informix database through the Generic Database account can cause thread leaks because of JDBC driver implementations.
Workaround: To avoid these thread leaks, do the following:
Use the Informix JDBC driver 4.50.4.1 version or a later version.
Configure the following URL properties in the account settings:
Url Property name: IFMXCONNECTION_CLEANER_THREADS
Url property value: 0
Breaking Change
Your existing pipelines that use JDBC Snaps to integrate with Oracle or Redshift databases using the bundled Oracle or Redshift JDBC drivers will stop functioning with the 438patches28052.
Workaround
You must manually upload the Oracle or Redshift JDBC driver in the JDBC Database account to run your pipelines successfully.
Download the drivers:
OJDBC: Maven Repository: com.oracle.database.jdbc » ojdbc6 » 11.2.0.4
Redshift: Maven Repository: com.amazon.redshift » redshift-jdbc42 » 2.1.0.11
Account Settings
Info |
---|
|
Field Name | Field Type | Field Dependency | Description | ||||||
---|---|---|---|---|---|---|---|---|---|
Label* Default Value: N/A | String | N/A | Specify a unique label for the account. | ||||||
Account properties* | The information required to create a connection to the database. | ||||||||
JDBC JARs | Use this field set to define JDBC Drivers. This field set consists of the JDBC Driver field. | ||||||||
JDBC Driver* Default Value: N/A | String/Expression | N/A | Specify the JDBC driver to use. Click the Upload icon to upload the required JDBC driver that must be used for the account.
| ||||||
JDBC Driver Class* Default Value: [None] | String/Expression | N/A | Specify the JDBC Driver class name to use. | ||||||
JDBC URL Default Value: [None] | String/Expression | N/A | Specify the JDBC URL to use. | ||||||
Username
| String/Expression | N/A | Specify the database username to use. | ||||||
Password Default Value: N/A | String/Expression | N/A | Specify the database password to use. | ||||||
Database name Default Value: Auto detect | String/Expression | N/A | Choose a database to which the account must be connected. The available options are:
If you use PostgreSQL JDBC driver to connect to Redshift database, the Auto detect option automatically detects the PostgreSQL database instead of Redshift. The behavior of the JDBC Snaps is optimized for the selected database. | ||||||
Test Query Default Value: N/A
| String/Expression | Activates on selecting Auto detect for Database name. | Specify a custom query to validate the database connection.
| ||||||
Configure SSH tunnel Default Value: Deselected | Checkbox | N/A | Select this checkbox to connect to the database server through the SSH tunnel. After the operation is completed, the tunnel is closed. Learn more about SSH Tunneling Testing with PostgreSQL. | ||||||
SSH auth mode Default Value: Password | Dropdown list | N/A | Select an option to specify the mode for authenticating the user on the SSH tunnel. The available options are:
| ||||||
SSH hostname Default Value: N/A | String/Expression | N/A | Specify the IP address or the domain name of the SSH server to which you want to connect. | ||||||
SSH username Default Value: N/A | String/Expression | N/A | Specify the SSH username that is authorized to connect to the database. This username is used as the default username when retrieving connections. The username must be valid to set up the data source. | ||||||
SSH password Default Value: N/A | String/Expression | Appears if SSH Auth Mode is Password. | Specify the password for the SSH username for connecting to the SSH tunnel. | ||||||
Private key file URL Default Value: N/A | String/Expression | Appears if SSH Auth Mode is KeyFile (Private Key File). | Specify the location of the private key file. The file can be in SLDB, on the host machine that is hosting the JCC, or at any other remote location. Click the File browser icon to upload the file from your local system. You can also upload the file using any protocol such as HTTPS, FTP, SLDB, and SFTP. | ||||||
Private key* Default Value: N/A -----BEGIN RSA PRIVATE KEY----- ……………….. -----END RSA PRIVATE KEY----- | String/Expression | Appears if SSH Auth Mode is KeyFile (Private Key String). | Specify the private key for authentication. | ||||||
Private key passphrase Default Value: N/A
| String/Expression | Appears if SSH Auth Mode is KeyFile. | Specify the passphrase that is to be used to decrypt the private key.
| ||||||
SSH port Default Value: N/A | Integer/Expression | N/A | Specify the SSH port to connect to any of the following database servers:
| ||||||
Advanced properties* | |||||||||
Min pool size* Default Value: 0 | Integer/Expression | N/A | Specify the minimum number of idle connections a pool will maintain at a time. If the size is set to non-zero, JCC restart is needed when the database account expires. | ||||||
Max pool size* Default Value: 15 | Integer/Expression | N/A | Specify the maximum number of idle connections a pool will maintain at a time. | ||||||
Max idle time* Default Value: 60 | Integer/Expression | N/A | Specify the number of minutes a connection must remain in the connection pool before being discarded. | ||||||
Checkout timeout* Default Value: 10000 | Integer/Expression | N/A | Specify the number of milliseconds to wait for a connection to be available in the pool. If you provide 0, the Snap waits infinitely until the connection is available. Therefore, we recommend you not to specify 0 for Checkout Timeout. | ||||||
Url properties | Use this field set to define URL properties to use in the JDBC URL. | ||||||||
Url property name Default Value: N/A
| String | N/A | Specify a name for the URL property to be used by the account. For instance:
| ||||||
Url property value Default Value: N/A
| String | N/A | Specify a value for the URL property name. | ||||||
Auto commit Default Value: Selected | Checkbox | N/A | Select one of the options for this property to override the state of the Auto commit property on the account. The Auto commit at the Snap-level has three values: True, False, and Use account setting. The expected functionality for these modes are:
| ||||||
Fetch size* Default Value: 100 | Integer/Expression | N/A | Specify the number of records to retrieve from the DB at a time. | ||||||
Batch size* Default Value: 50 | Integer/Expression | N/A | Specify the number of query statements to execute at a time. SELECT queries are not batched.
If you are using AWS Athena database, the Batch size must be set to 1. |
Troubleshooting
Error | Reason | Resolution |
---|---|---|
| Snap displays this error when the endpoint database driver lacks an implementation for the isValid() or any other method. | To address this issue, consider the following troubleshooting methods:
|
Snap Pack History
Expand | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
...