S3 Browser
In this article
Overview
You can use this Snap to list the attributes of S3 objects in a specific bucket matching the prefix. It internally lists all S3 objects with the specified bucket name and prefix, and then applies the filter if any.
AWS S3 does not define any folder object and the '/' characters are part of the S3 object key names.
Snap Type
The S3 Browser Snap is a Read-type Snap that reads objects from S3.
Prerequisites
None.
Support for Ultra Pipelines
Works in Ultra Pipelines.
Limitations and Known Issues
None.
Snap Views
Type | Format | Number of Views | Examples of Upstream and Downstream Snaps | Description |
---|---|---|---|---|
Input | Document |
|
| Key/value pairs used to evaluate expression properties in this Snap. |
Output | Document
|
|
| The output displays the following attributes:
The attributes for each S3 object matching the specific Snap field values. For example, if you have the following S3 objects with the prefix
If the S3 Browser Snap has the following properties, then the Snap produces output documents for all the five S3 objects.
If the S3 Browser Snap has the following properties, then the Snap produces only one output document for foo/boo.
When you use the Cross Account IAM Role in the S3 account, the |
Error | Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:
Learn more about Error handling in Pipelines. |
Snap Settings
Field Name | Field Type | Field Dependency | Description |
---|---|---|---|
Label*
Default Value: S3 Browser | String | N/A | Specify a unique name for the Snap.
|
Bucket*
Default Value: N/A
| String/Expression/Suggestion | N/A | Specify or select the S3 bucket name, from where the S3 objects are to be listed. Bucket names are unique globally and can be accessed without the region name in most cases. If you cannot access a bucket name without its region name, you can specify the region information with the following syntax:
Note: If you enter an incorrect region name, but the bucket name is valid, the AWS S3 service might successfully access the bucket without errors.
|
Prefix Default Value: None
| String/Expression/Suggestion | N/A | Specify the prefix of S3 object key names. |
Filter (Glob) Default Value: **
| String/Expression | N/A | Specify the filter the Snap should use internally to list all S3 objects. |
Show Advanced Properties Default Value: Deselected | Checkbox | Displays Advanced Properties when you select this checkbox. | Select this checkbox to display the advanced properties. |
Maximum Object Keys Default Value: All
| String/Integer/Expression | Appears on selecting Show Advanced Properties checkbox. | Specify the maximum number of object keys at the output view. ‘All’ or an empty field produces all matching object key names. |
Ignore Empty Result Default Value: Selected | Checkbox | Appears on selecting Show Advanced Properties checkbox. | Select this checkbox if you want the Snap to produce no output when there is no matching S3 object key. Else, the Snap outputs a blank document when there are no matching S3 object keys.
|
Maximum Retries* Default Value: 3 | Integer/Expression | Appears on selecting Show Advanced Properties checkbox. | Specify the maximum number of retry attempts to perform in case of a temporary network loss.
|
Pass Through
Default Value: Deselected | Checkbox | Appears on selecting Show Advanced Properties checkbox. | Select this checkbox to pass the input document to the output view as a value under the '
|
Delimiter
| String/Expression | Appears on selecting Show Advanced Properties checkbox. | Specify an optional delimiter to produce object keys that contain the same string between the prefix and the first occurrence of the delimiter. A delimiter '/' causes the Snap to produce a nonrecursive object-key listing. |
Snap Execution Default Value: Validate & Execute | Dropdown list | N/A | Select one of the three modes in which the Snap executes. Available options are:
|
Example
Basic Use Case - Managing Data in S3
This basic use case demonstrates how to perfrom the following operations using the Amazon S3 Snaps:
Upload a single JSON file in the S3 object
Copy the file
List the S3 Object and its attributes
Download the file
Delete the file
Configure key-value pairs containing parameters to pass to the Pipeline during execution.
Configure the JSON Generator Snap with sample data as follows.
Configure the JSON Formatter Snap to format the data. You can view the JSON binary data in the output preview of the Snap.
Configure the S3 Upload Snap to upload the binary data to S3 objects residing in the S3 bucket.
Configure the S3 Copy Snap to copy the S3 object from source folder to target folder.
Configure the S3 Browser Snap to list the attributes for S3 objects.
Configure the S3 Download Snap to download S3 objects.
As the output is in binary format, you need to convert the binary data to document data using the Binary to Document Snap.
Configure the S3 Delete Snap to delete the S3 objects. You can view the status of the deleted object in the output preview.
Downloads
Snap Pack History
Related Links
Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.