In this article
Table of Contents | ||||
---|---|---|---|---|
|
Overview
Use this account type to connect <Snap Pack name> Redshift Snaps with data sources that use <Account Type Name> . The Redshift Snaps support both Redshift Cluster and Redshift Serverless accounts.
Prerequisites
- Valid client ID.
- Valid tenant URL.
- ...
- ...
Limitations
Known Issues
- Supports only reading JSON files.
- ....
Account Settings
...
title | Documenting Fields Based On Data Type/UI Element |
---|
**Delete Before Publishing**
Choose from the following sentences to document specific field types.
...
- <State what the field contains>. Available options are: <bullet list of LoVs>
The Salesforce API to be used. Available options are:... - Select the <category> that you want to use. Available options are...
* Option 1<italicized>: <third person singular form of the verb>
* Option 2<italicized>: <third person singular form of the verb>
Select the API that you want to use. Available options are:
Bulk API: Sends the Snap execution request details as a bulk API call.
REST API: ... - Indicates.... Available options are:
Indicates how the Snap must be executed. Available options are:
* Validate & Execute: Performs limited execution of the Snap (up to 50 records) during Pipeline validation; performs full execution of the Snap (unlimited records) during Pipeline execution.
* Execute only: Performs full execution of the Snap during Pipeline execution; does not execute the Snap during Pipeline validation.
* Disabled: Disables the Snap and, by extension, its downstream Snaps.
Check boxes:
- If selected, <Snap behavior>.
If selected, an empty file is written when the incoming document has no data. - If selected, <behavior>. If not selected/Otherwise, <behavior>
Use "If not selected" if the first sentence is long.
If selected, the Snap uses the file path value as is. Otherwise, the Snap uses the file path value in the URL.
If selected, an empty file is written when the incoming document has empty data. If there is no incoming document at the input view of the Snap, no file is written regardless of the value of the property.
- Select to <action>
Use this if the behavior is binary. Either this or that, where the converse behavior is apparent/obvious.
Select to execute the Pipeline during validation.
Text Fields
- Describe what the field represents/contains. Additional details, as applicable, in a separate sentence. Include caveats such as the field being conditionally mandatory, limitations, etc.
The name of the account.
The account ID that you want to use to log in to the endpoint.
Required if IAM Role is selected.
Do not use this field if you are using batch processing.
Numeric Text Fields
- Describe what the field represents/contains. Additional details, as applicable, in a separate sentence. Include caveats such as the field being conditionally mandatory, limitations, etc. Include special values that impact the field's behavior as a bullet list.
The number of records in a batch.
The number of seconds for which you want the Snap to wait between retries.
The number of seconds for which the Snap waits between retries.
Use the following special values:
* 0: Disables batching.
* 1: Includes all documents in a single request.
...
Required. Unique user-provided label for the account.
...
Troubleshooting
...
Account validation failed
...
The Pipeline ended before the batch could complete execution due to a connection error.
Verify that the Refresh token field is configured to handle the inputs properly. If you are not sure when the input data is available, configure this field as zero to keep the connection always open.
See Also
...
Access to the folders in which you are creating an account.
Limitations and Known Issues
Multiexcerpt include macro | ||||||||
---|---|---|---|---|---|---|---|---|
|
Multiexcerpt include macro | ||||||||
---|---|---|---|---|---|---|---|---|
|
Account Settings
...
Field Name | Field Type | Description | |
---|---|---|---|
Label* Default Value: N/A | String | Specify a unique label for the account. | |
JDBC Driver Class Default Value: com.amazon.redshift.jdbc42.Driver | String/Expression | By default, the | |
JDBC JARs
| Use this field set to add a list of JDBC JAR files to be loaded. By default, the Redshift account is bundled with the JDBC v2.1.0.29 driver. However, you can add a custom JAR file. Click + to add a new row for each JDBC JAR file. | ||
JDBC Driver
Default value: N/A JDBC 42-2.1.0.29.jar | String | The Redshift Snap Pack is bundled with the default Redshift JDBC driver v2.1.0.29. Therefore, even if you do not provide a JDBC Driver, the account does not fail. | |
JDBC URL Default Value: N/A | String/Expression | Enter the URL of the JDBC database. | |
Account properties* | Configure the account properties to establish a connection to the database. | ||
Endpoint* Default Value: N/A | String/Expression | Enter the server's address to connect to. | |
Port Number* Default Value: 5439 | Numeric/Expression | Enter the database server's port to connect. | |
Database name* Default Value: N/A | String/Expression | Enter the database name to connect. | |
Username Default Value: N/A | String/Expression | Enter the username to connect to the database. The username will be used as the default username when retrieving connections. The username must be valid in order to set up the data source. | |
Password Default Value: N/A | String/Expression | Enter the password used to connect to the data source. The password will be used as the default password when retrieving connections. The password must be valid in order to set up the data source. | |
S3 Bucket Default Value: N/A | String/Expression | Enter the external S3 Bucket name residing in an external AWS account to use for staging data onto Redshift. This field is required when using Redshift Bulk Load, Redshift Bulk Upsert, Redshift S3 Upsert, and Redshift Unload Snaps. | |
S3 Folder Default Value: N/A | String/Expression | Enter the relative path to a folder in S3 Bucket. This is used as a root folder for staging data onto Redshift. | |
S3 Access-key ID Default Value: N/A | String/Expression | Enter the S3 Access key ID part of AWS authentication. This field is required when using Redshift Bulk Load, Redshift Bulk Upsert, Redshift S3 Upsert, and Redshift Unload Snaps if the IAM role checkbox is not selected. | |
S3 Secret key Default Value: N/A | String/Expression | Enter the S3 Secret key part of AWS Authentication. This field is required when using Redshift Bulk Load, Redshift Bulk Upsert, Redshift S3 Upsert, and Redshift Unload Snaps if the IAM role checkbox is not selected. | |
S3 Security Token Default Value: N/A | String/Expression | Specify the S3 security token part of the AWS Security Token Service (STS) authentication. It is not required unless a specific S3 credential is configured to require it. Note that only global STS regions are supported. This field is required when using Redshift Bulk Load, Redshift Bulk Upsert, Redshift S3 Upsert, and Redshift Unload Snaps if the IAM role checkbox is not selected. | |
IAM properties | Specify the IAM properties information for Redshift to communicate with IAM | ||
AWS account ID Default Value: N/A | String/Expression | Enter the ID of the Amazon Web Services account to be used for performing the bulk load operation. This field is required when using Redshift Bulk Load, Redshift Bulk Upsert, Redshift S3 Upsert, and Redshift Unload Snaps if the IAM role checkbox is selected. | |
IAM role name Default Value: N/A | String/Expression | Enter the name of the IAM role that has been assigned to the Redshift cluster to access the S3 bucket provided above. This field is required when using Redshift Bulk Load, Redshift Bulk Upsert, Redshift S3 Upsert, and Redshift Unload Snaps if the IAM role checkbox is selected. | |
Region name Default Value: N/A | String/Expression | Enter the name of the region the Redshift cluster. | |
Advanced properties | Specify advanced properties to support this account. | ||
Auto commit Default Value: Selected | Checkbox | Select this checkbox to enable the Snap to commit offsets automatically as messages are consumed and sent to the output view. | |
Batch size* Default Value: 50 | Integer/Expression | Enter the number of statements to execute at a time. Select queries are not batched. User-defined types are not supported if a JDBC driver with a version other than 11.2.0.4.0 is set. | |
Fetch size* Default Value:100 | Integer/Expression | Enter the number of rows to fetch at a time when executing a query. User-defined types are not supported if a JDBC driver with a version other than 11.2.0.4.0 is set. | |
Max pool size* Default Value: 50 | Integer/Expression | Enter the maximum number of connections a pool will maintain at the same time. Redshift Bulk Load/Bulk Upsert/S3 Upsert Snaps require a minimum of 2 connections per Snap in a pipeline. For example, if a pipeline has a Redshift Bulk Load Snap and an S3 Upsert Snap, then the pool size must be greater than or equal to 4 for successful execution. | |
Max life time* Default Value: 30 | Integer/Expression | Enter the maximum lifetime of a connection in the pool. Ensure that the value you enter is a few seconds shorter than any database or infrastructure-imposed connection time limit. A value of 0 indicates an infinite lifetime, subject to the Idle Timeout value. An in-use connection is never retired. Connections are removed only after they are closed. | |
Idle Timeout* Default Value: 5 | Integer/Expression | Enter the maximum amount of time a connection is allowed to sit idle in the pool. A value of 0 indicates that idle connections are never removed from the pool. | |
Checkout timeout* Default Value:1000 | Integer/Expression | Enter the number of milliseconds to wait for a connection to be available when the pool is exhausted. If you enter 0 for this value, the Snap waits infinitely until the connection is available. Therefore, we recommend you not enter 0 for Checkout Timeout. | |
URL Properties | Specify the URL properties associated with this account. | ||
URL property name Default Value: N/A | String/Expression | Enter the URL property name. | |
URL property value Default Value: N/A | String/Expression | Enter the URL property value. |
Insert excerpt | ||||||
---|---|---|---|---|---|---|
|