...
You can use the AWS S3 account to connect the Binary Snaps with data sources that are in AWS S3.
Prerequisites
Valid permissions based on the Snap and intended operation.
EC2 instance as a Groundplex. The IAM role is valid only in Groundplex nodes hosted in the EC2 environment. Learn more about Configuring an EC2 role for IAM Role in AWS S3 Account.
JCC with the following global property set:jcc.jvm_options=-DIAM_CREDENTIAL_FOR_S3=TRUE
If you do not have an EC2 instance groundplex, then you can authenticate your account by using the Access Key ID and Secret Key. You can assume roles using the Cross account IAM role, that uses the IAM role specified in the settings. The Access Key ID and Secret Key need to have the ability to assume in the user specifications.
Account Settings
...
Info |
---|
Asterisk (*): Indicates a mandatory field. Suggestion icon (): Indicates a list that is dynamically populated based on the configuration. Expression icon (): Indicates whether the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic. Add icon (): Indicates that you can add fields in the field set. Remove icon (): Indicates that you can remove fields from the field set.
|
Field Name | Field Type | Description |
---|
Label* Default Value: None Example: AWS S3 Account | String | Specify a unique name for the account instance |
Access-key ID Default value: [None] Example: <Encrypted>
| String | Specify a unique access key ID part of AWS authentication. Note |
---|
The Access-key ID is required when the IAM role is disabled. |
|
Secret key Default value: [None] Example: <Encrypted>
| String | Specify the secret key part of AWS authentication Note |
---|
The Secret key is required when the IAM role is disabled. |
|
Server-side encryption Default Value: Deselected | Checkbox | If selected, the S3 file is written and encrypted using the 256-bit Advanced Encryption Standard AAES256. For Snaps that read objects from S3, this field is not required, as encrypted data is automatically decrypted when data is read from S3. |
KMS Encryption type Default Value: None Example: Server-Side KMS Encryption | String | Choose the encryption type from the following list. This field represents the AWS Key Management Service key used to encrypt S3 objects—it can be the key ID or ARN. None: The files do not get encrypted using KMS encryption. Server-Side KMS Encryption: The output files on Amazon S3 are encrypted with Amazon S3 generated KMS key. Client-Side KMS Encryption: The output files on Amazon S3 are encrypted with client generated KMS key.
|
using Server this is required. For Snaps that read objects from S3, this field is not required. For Server-Side encryption, the key must be in the same region as the S3 bucket. For Client-Side encryption, a key from any region can be used by using the key ARN value. If a key ID is used for Client-Side encryption, it defaults to the us-east-1 region.
|
KMS key Default Value: N/A Example: <Encrypted> | String | Specify the AWS Key Management Service (KMS) key ID or ARN to be used for the S3 encryption. This is only required if the KMS Encryption type property is configured to use the encryption with KMS. Learn more about the KMS key: AWS KMS Overview and Using Server Side Encryption. For Snaps that write objects to S3 using Server-Side encryption and Client-Side encryption with AWS KMS-Managed Keys this is required. For Snaps that read objects from S3, this field is not required. For Server-Side encryption, the key must be in the same region as the S3 bucket. For Client-Side encryption, a key from any region can be used by using the key ARN value. If a key ID is used for Client-Side encryption, it defaults to the us-east-1 region.
|
KMS region Default Value: N/A Example: s3.us-east-2 | String | Specify or select a name of the region to which the KMS key belongs. |
IAM role Default Value: Deselected
| Checkbox |
If selected, the IAM role Select this checkbox to use the Groundplex EC2 instance stored in the |
EC2 instance is usedIAM role, instead of the normal AWS authentication |
, to access the S3 bucket. The Access-key ID and Secret key fields are ignored in this case. |
To create EC2 role, see noteThis property In the Groundplex, add the following line to ., and also requires specific configuration. Set the global.properties file on the Groundplex as shown below and restart the JCC:
jcc.jvm_options = -DIAM_CREDENTIAL_FOR_S3=TRUE When you select the IAM Role checkbox
|
and validate the account, an error is displayed and validated. When you provide valid Role ARN and External ID values, then click Apply button to use the account. value: [None]Value: None Example: arn:aws:s3::test-bucket-sa-sl/* | String/Expression | Specify the Amazon Resource Name of the role to assume. |
External ID Default |
value: [None]Value: None Example: 321f248c-8f4a-21be-87c4-184c9f8e2d03 | String/Expression | Specify an external ID that might be required by the role to assume. |
Support IAM role max session duration Default Value: Deselected
| Checkbox | Select this checkbox when you want to extend the maximum session duration of an IAM role defined in AWS. On selecting this checkbox, the cross-account IAM role is assumed with the maximum session duration defined for the IAM role. Note |
---|
This checkbox is deselected by default. The default maximum session duration for an IAM role is one hour; however, you can define a custom duration between 1-12 hours. Learn how to increase the IAM role maximum session duration limit. We recommend that you select this checkbox if the maximum session duration of the IAM role is greater than an hour. |
|
Troubleshooting
...
Error
...
Reason
...
Resolution
...
When creating an S3 account, if you select the IAM role checkbox, then the following error is displayed on clicking the Validate button.
"Failed to validate account."
...
When you select the IAM role checkbox and validate the account, an error is displayed.
...
Ensure that you provide valid Role ARN and External ID values and then click Apply instead of Validate (on the AWS S3 account settings dialog) to use the account.
...
Multiexcerpt include macro |
---|
name | Troubleshooting_S3_Account |
---|
templateData | [] |
---|
page | S3 |
---|
addpanel | false |
---|
|
Multiexcerpt macro |
---|
hidden | false |
---|
name | AccPerm |
---|
fallback | false |
---|
|
Account PermissionsSnap | Snap Operation | Minimum S3 Permissions |
---|
S3 Account | | s3:ListAllMyBuckets | S3 File Writer
| | s3:PutObject | | s3:PutObject, s3:ListBucket | Write object tags. | s3:PutObject, s3:PutObjectTagging | Update the Access Control List (ACL). | s3:PutObject, s3:ListAllMyBuckets, s3:PutObjectAcl | Suggest list of buckets in the File name field. | s3:ListAllMyBuckets | Suggest S3 objects in File name field. | s3:ListBucket | S3 File Reader
| Read files. | s3:GetObject | Read versioning-enabled files. | s3:GetObject, s3:GetObjectVersion | Suggest list of buckets in the File field. | s3:ListAllMyBuckets | Suggest S3 objects in the File field. | s3:ListBucket | Suggest list of Version IDs. | s3:ListBucketVersions | Read object tags. | s3:GetObject, s3:GetObjectTagging | File Writer | | s3:PutObject | | s3:PutObject, s3:ListBucket | ZipFile Writer | Write file with 'File action'=OVERWRITE. | s3:PutObject | Write file with 'File action'=IGNORE or ERROR. | s3:PutObject, s3:ListBucket | File Reader | Read files. | s3:GetObject | ZipFile Reader | Read files. | s3:GetObject | Multi File Reader | Read one file only without wildcards. | s3:GetObject | Read files. Use wildcards. Include sub-folders.
| s3:GetObject, s3:ListBucket | Directory Browser | List files and directories. | s3:ListBucket | File Delete | Delete files. | s3:DeleteObject, s3:ListBucket | File Operation | Copy files. | s3:GetObject, s3:PutObject, s3:ListBucket | Move files. | s3:GetObject, s3:PutObject, s3:ListBucket, s3:DeleteObject | File Poller | Poll files. | s3:ListBucket |
See Learn more about Setting Permissions and Permissions for the Amazon S3 Bucket for more information. |
ACL permissions
ACL permission | Corresponding access policy permissions when the ACL permission is granted on a bucket | Corresponding access policy permissions when the ACL permission is granted on an object |
---|
READ
| s3:ListBucket , s3:ListBucketVersions , and s3:ListBucketMultipartUploads
| s3:GetObject and s3:GetObjectVersion
|
WRITE
| s3:PutObject
Bucket owner can create, overwrite, and delete any object in the bucket. Object owner has FULL_CONTROL over their objects.
In addition, when the grantee is the bucket owner, granting WRITE permission in a bucket ACL allows the s3:DeleteObjectVersion action to be performed on any version in that bucket. | Not applicable. |
READ_ACP
| s3:GetBucketAcl
| s3:GetObjectAcl and s3:GetObjectVersionAcl
|
WRITE_ACP
| s3:PutBucketAcl
| s3:PutObjectAcl and s3:PutObjectVersionAcl
|
FULL_CONTROL
| Equivalent to granting READ , WRITE , READ_ACP , and WRITE_ACP ACL permissions. Accordingly, this ACL permission maps to a combination of corresponding access policy permissions. | Equivalent to granting READ , READ_ACP , and WRITE_ACP ACL permissions. Accordingly, this ACL permission maps to a combination of corresponding access policy permissions. |
Insert excerpt |
---|
| Binary Snap Pack |
---|
| Binary Snap Pack |
---|
name | Binary_SPH |
---|
nopanel | true |
---|
|
...