/
S3

S3

In this article

Overview

You can use this account type to connect S3 Snaps with data sources that use an Amazon S3 account.

Prerequisites

  • Either EC2 or ECS instance as a Groundplex. The IAM role is valid only in Groundplex nodes hosted in the EC2 or ECS environment.

  • JCC with the following global property set:jcc.jvm_options=-DIAM_CREDENTIAL_FOR_S3=TRUE

If you do not have an EC2 instance groundplex, you can authenticate your account using the Access Key ID and Secret Key. You can assume roles using the Cross account IAM role, which uses the IAM role specified in the settings. The Access Key ID and Secret Key need to be able to assume roles in the user specifications.

Account Settings

S3-account-overview.png
  • Asterisk (*): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon (): Indicates whether the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon (): Indicates that you can add fields in the fieldset.

  • Remove icon (): Indicates that you can remove fields from the fieldset.

Field Name

Field Type

Description

Field Name

Field Type

Description

Label*

 

Default Value: None
ExampleS3 Account

String

Specify a unique label for the account.

 

Access Key ID

 

Default Value: N/A
Example: CKIA2EP4BT3EYCWBKC

String/Expression

Specify a unique access key ID part of AWS authentication.

This field is required if the IAM role is disabled.

 

Secret Key

 

Default Value: N/A
Example: G9Hm2h5+PtSI7CnZO9KLVgyFPAc5ZTqC9uV94uPr

String/Expression

Specify the secret key part of AWS authentication.

This field is required if the IAM role is disabled.

 

Security Token


Default Value: N/A
Example: Djh198SnOKIA2

String/Expression

Specify the security token that is part of AWS Security Token Services (STS) credentials. Note that only global STS regions are supported.

IAM Role

 

Default value: Deselected

Checkbox

Select this checkbox to use the EC2 or ECS instance stored in the IAM role, instead of the normal AWS authentication to access the S3 bucket. The Access-key ID and Secret key fields are ignored in this case. Learn more about Configuring an EC2 role for IAM Role in AWS S3 Account.

The Amazon S3 Snaps automatically detect the Maximum session duration value for the Cross-Account IAM role (1 through 12 hours). The Snaps round down the value to the nearest hour. So, if the Snap administrator sets the Maximum session duration at 3 hours and 45 minutes, the Snaps read it as 3 hours. The Snaps also refresh the session before it expires. However, the automatic session refresh does not support the case of very large file upload or download that takes longer than the maximum session duration.

  • The IAM role is valid in Groundplex nodes hosted in the EC2 and ECS environments.

    • For EC2, set the global properties (Key and Value) on the Groundplex as shown below and restart the JCC: 
      jcc.jvm_options = -DIAM_CREDENTIAL_FOR_S3=TRUE

    • For ECS, update the task definition environment variables as follows:

[

{
                    "name": "LANG",
                    "value": "en_US.UTF-8"
                },
                {
                    "name": "LC_ALL",
                    "value": "en_US.UTF-8"
                },
                {
                    "name": "JAVA_TOOL_OPTIONS",
                    "value": "-Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8"
                }
            ]

 

  • The List, Read, and Write permissions are required as per the attached S3 policy for the IAM role stored on the EC2 and ECS instances. 

  • When you select the IAM Role checkbox and validate the account, an error is displayed, and the account is not validated. To use the account, provide valid Role ARN and External ID values, and then click the Apply button.

Cross Account IAM Role

Configure the properties required to perform cross-account access. Learn more about setting up Cross Account IAM Role.

When you use the Cross Account IAM Role in the Amazon S3 account, the ownerDisplayName and ownerID fields in the output document of the S3 Browser Snap may be empty.

Role ARN

 

Default Value: N/A
Example: arn:aws:s3:::sandbox-test-snap

String/Expression

Specify the Amazon Resource Name of the role to assume.

 

External ID

 

Default Value: N/A
Example: 321f248c-8f4a-21be-87c4-184c9f8e2d03

String/Expression

Specify an optional external ID that might be required by the role to assume.

 

Encryption

Configure the properties required for encryption.

Encryption Type

 

Default value: None
Example: Server-Side Encryption

Dropdown list

Specify the AWS Key Management Service key used to encrypt S3 objects. It can be the key ID or ARN. The available options are:

  • None: The files do not get encrypted using KMS encryption.

  • Server-Side Encryption: The S3 file is written and encrypted using the 256-bit Advanced Encryption Standard AAES256. For Snaps that read objects from S3, this field is not required, because encrypted data is automatically decrypted when data is read from S3.

  • Server-Side KMS Encryption: The output files on Amazon S3 are encrypted using this encryption with an Amazon S3-generated KMS key.

  • Client-Side KMS Encryption: The output files on Amazon S3 are encrypted using this encryption with a client-generated KMS key.

  • For Snaps that read objects from S3, this field is not required. 

  • For Snaps that write objects to S3, this is required for encryption types—Server-Side encryption and Client-Side encryption with AWS KMS-Managed Keys.

    • For Server-Side encryption, the key must be in the same region as the S3 bucket.

    • For Client-Side encryption, a key from any region can be used by using the key ARN value. If a key ID is used for Client-Side encryption, it defaults to the us-east-1 region.

KMS Key

 

Default value: None
Example: 28e3c2b6-74e2-4a3e-9890-6cd8e1c03661

String

Specify the AWS Key Management Service (KMS) key ID or ARN to be used for the S3 encryption. 

  • This is required only if the KMS Encryption type property is configured to use the encryption with KMS. Learn more about AWS KMS Overview and Using Server Side Encryption

  • For Snaps that write objects to S3, this is required for encryption types Server-Side encryption with AWS KMS-Managed Keys and Client-Side encryption with AWS KMS-Managed Keys.

    • For Server-Side encryption, the key must be in the same region as the S3 bucket.

    • For Client-Side encryption, a key from any region can be used by using the key ARN value. If a key ID is used for Client-Side encryption, it defaults to the us-east-1 region.

KMS Region

 

Default Value: N/A
Example: s3.us-east-2

String/Expression/Suggestion

Specify the AWS region where the KMS key is located. 

 

Bucket for account validation

 

Default Value: N/A
Example: project-backups-2025

String

Specify the bucket for validating the account.

This field is not required for most buckets, and if the bucket's region belongs to one of the default regions, you can leave it blank. However, this field is required if the account validation fails and if the bucket is in a recently added AWS region.

Troubleshooting

Error

Reason

Resolution

When authorizing an S3 account, if the IAM role checkbox is selected, the following error is displayed on clicking the Validate button.

"Failed to validate account."

When validating the S3 account if the IAM role checkbox is selected, an error is displayed.

Account validation is not supported when you select the IAM Role checkbox.
Ensure that you provide valid Role ARN and External ID values and then click Apply instead of Validate (on the account settings dialog) to authorize and use the account.

Snap Pack History

 


Related Content