AWS S3 Account for Hadoop
In this article
Overview
You can use this account type to connect Hadoop Snaps with data sources that use AWS S3 accounts.
The AWS S3 Account supports cross-account IAM Role. The Parquet Reader, Parquet Writer, ORC Reader, and ORC Writer Snaps support the cross-account IAM role.
Prerequisites
- S3 accounts must have full access.
- S3 ListAllMyBuckets permission is required for the S3 account to be validated successfully.
Limitations and Known Issues
None.
Account Settings
Parameter | Data Type | Description |
---|---|---|
Label* | String | Specify a unique label for the account. We recommend that you update the account name if there is more than one account of the same account type in your project. Default Value: N/A |
Access-key ID | String | Required when IAM role is deselected. Specify the unique access key ID part of AWS authentication. Default Value: N/A |
Secret key | String | Required when IAM role is deselected. Specify the secret key part of AWS authentication. Default Value: N/A |
Server-side encryption | Checkbox | Required for writing to S3. Select this checkbox to enable the server-side encryption to use for the objects. Learn more, Protecting Data Using Server-Side Encryption with Amazon S3-Managed Encryption Keys. Default Value: Deselected |
IAM role | Checkbox | If you select this checkbox, the IAM role stored in the EC2 instance is used to access the S3 bucket.
Default Value: Deselected |
S3 Region | String | Specify the name of the region in which the S3 bucket resides. You need to specify S3 Region only if you have to access the S3 buckets in the cross-region or proxied cross-regions. If you leave this field blank and try to access cross-region S3 buckets, the Snap displays Bad request error.Default Value: None |
IAM Role properties | Use this field set to enter information associated with the IAM Role. Use this field set only if you do not plan to provide the Access key ID and Secret key, and if IAM role, above, is selected. Default Value: N/A | |
AWS account ID | String | Specify the Amazon Web Services account ID associated with the AWS S3 account that you want to use. Default Value: N/A |
IAM role name | String | Specify the name of the IAM role that can access the AWS S3 account identified above. Default Value: N/A |
External ID | String/Expression | Specify an external ID that might be required by the role to assume. Default Value: N/A Example: 74521369541 |
Region Endpoint name | String | Specify the endpoint name of the region to which the target AWS S3 bucket belongs. Protocols supported: S3 Default Value: N/A |
Troubleshooting
Error Message | Reason | Resolution |
---|---|---|
Failed to validate account: Invalid IAM role setting | Access-key ID and Secret key should be empty if IAM role is selected. This means that you selected the IAM role check box but also provided access-key ID and secret key information. | Address the reported issue. Do not provide both IAM role and access-key details for the same account. |
Failed to validate account | Failed to validate account This typically means that your IAM role details are incorrect. | Verify if the provided credentials are correct. |
Access key cannot be null. | ||
Failed to validate account: The AWS Access Key Id you provided does not exist in our records. | Access key is invalid. |
Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.