Versions Compared

Old Version 21

changes.mady.by.user Mohammed Iqbal

Saved on

compared with

New Version Current

changes.mady.by.user Kalpana Malladi

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this article

Table of Contents
maxLevel2
absoluteUrltrue

Overview

Use You can use this account type to connect Hadoop Snaps with data sources that use AWS S3 accounts.

Info

The AWS S3 Account supports cross-account IAM Role. The Parquet Reader, Parquet Writer, ORC Reader, and ORC Writer Snaps support the cross-account IAM role.


Prerequisites

Multiexcerpt macro
nameawss3p
  • S3 accounts must have full access.
  • S3 ListAllMyBuckets permission is required for the S3 account to be validated successfully.

Limitations and Known Issues

None.

Account Settings

Image RemovedImage Added

Parameter Data TypeDescription
Default ValueExample 
Label*String


Multiexcerpt macro
nameAccount_Label
Required. Unique user-provided

Specify a unique label for the account. We recommend that you update the account name if there is more than one account of the same account type in your project.

Default Value: N/A
Example: S3 Account

Access-key IDString

Required when IAM role is deselected.

The

Specify the unique access key ID part of AWS authentication.

Default Value: N/A
Example: ASTPPGC2DCFDB5DW9GHI

Secret keyString

Required when IAM role is deselected.

The

Specify the secret key part of AWS authentication.

N

Default Value: N/A
Example: FGSDFG5465F4G6D5F4DFG5DFD5FGD5F5FGD58

Server-side encryption
Check box
Checkbox

Required for writing to S3.

Specifies that

Select this checkbox to enable the server-side encryption

be used

to use for the objects.

For

Learn more

information see 

, Protecting Data Using Server-Side Encryption with Amazon S3-Managed Encryption Keys.

Deselected

Default Value: Deselected
Example: N/A

IAM role
Check box
Checkbox

If

this check box is selected

you select this checkbox, the IAM role stored in the EC2 instance is used to access the S3 bucket.

Note
property
  • checkbox, ensure that the Access-key ID and Secret key fields are empty.
  • This
property
  • field is valid only in Groundplex nodes hosted in the EC2 environment.
    In the Groundplex, add the following line to global.properties and restart the JCC: 
    jcc.jvm_options = -DIAM_CREDENTIAL_FOR_S3=TRUE
  • Validation does not work when you select this
property is enabled
  • checkbox.
Deselected

Default Value: Deselected
Example: N/A

S3 RegionString

Specify the name of the region in which the S3 bucket resides. 


Warning
You need to specify S3 Region only if you have to access the S3 buckets in the cross-region or proxied cross-regions. If you leave this field blank and try to access cross-region S3 buckets, the Snap displays Bad requesterror.

Default Value: None
Example: us-east-2

IAM Role properties
N/A

Use this

fieldset

field set to enter information associated with the IAM Role.

Note

Use this

fieldset

field set only if you do not plan to provide the Access key ID and Secret key, and if IAM role, above, is selected.

Default Value: N/A

AWS account ID

String
The

Specify the Amazon Web Services account ID associated with the AWS S3 account that you want to use.

Default Value: N/A

IAM role name

String
The 

Specify the name of the IAM role that can access the AWS S3 account identified above.

Default Value: N/A

External ID

String/Expression

Specify an external ID that might be required by the role to assume.

Default Value: N/A

Example:74521369541

Region Endpoint name

String
The

Specify the endpoint name of the region to which the target AWS S3 bucket belongs.

Protocols supported: S3

Default Value: N/A
Example: s3.us-east-2.amazonaws.com

Troubleshooting

Error MessageReasonResolution
Failed to validate account: Invalid IAM role setting

Access-key ID and Secret key should be empty if IAM role is selected.

This means that you selected the IAM role check box but also provided access-key ID and secret key information.

Address the reported issue. Do not provide both IAM role and access-key details for the same account.
Failed to validate account
Failed to validate account
This typically means that your IAM role details are incorrect.
Verify if the provided credentials are correct.
Access key cannot be null.
Failed to validate account: The AWS Access Key Id you provided does not exist in our records.Access key is invalid.