Input 

Document

• Min: 0

• Max: 1

• Mapper

• JSON Parser

An upstream Snap is optional. Any document with key-value pairs to evaluate expression properties.

Output

Binary

• Min: 0

• Max: 1

• CSV Parser

• JSON Parser

• Mapper (binary input view)

• File Writer

An example of the output binary data is as following:

Image Modified

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

• Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.

• Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

• Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Label*

Default Value: S3 Select
Example: Select Invoices from Q4 2018

String

None

The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline.

Bucket*

Default Value: None
Example:

• mybucket

• mybucket@eu-west-1

• Example of S3 Bucket in S3 region:

  • mybucket@s3.eu-west-1.amazonaws.com

• Example of S3 VPC endpoint:

  • my-bucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

  • testBucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

String/Expression/Suggestion

None

Specify the S3 bucket name where S3 objects are located.

Object Key*

Default Value: None
Examples: 

• test.csv

• abc/test.json

• abc/xyz/test.xml

String/Expression/Suggestion

None

Specify the S3 object key name, which may include one or more forward-slash ('/') characters.

Select Query*

Default Value: None
Example: select * from S3Object

String/Expression

None

Enter a SELECT query to be executed on the S3 object. For a detailed description on using the SELECT command, refer to the Amazon documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-glacier-select-sql-reference-select.html

Input Data Format

Compression Type*

Default Value: None
Example: GZIP

String/Expression/Dropdown list

None

Select one of the following compression types: NONE, GZIP, or BZIP2.

Data format*

Default Value: CSV
Examples: JSON

String/Expression/Dropdown list

None

Select one of the data formats for the input data: CSV, JSON or Parquet.

Field Delimiter

Default Value: , (Comma)
Examples: 

• | (Pipe)

• \t (tab)

String/Expression

Appears when you select CSV for Data Format.

Enter the field delimiter character used in the input data.

Record Delimiter

Default Value: LF (new line)
Examples: CRLF

String/Expression/Suggestion

Appears when you select CSV for Data Format.

Enter the record delimiter used in the input data.

Quote Character

Default Value: " (double quote)
Example: ' (single quote)

String/Expression

Appears when you select CSV for Data Format.

Enter the quote character used in the input data.

Quote Escape Character

Default Value: \
Example: "

String/Expression

Appears when you select CSV for Data Format.

Enter the escape character used to escape quote characters inside the values, where values are wrapped around quote characters in the input data. Note that some CSV file uses double quote character to escape double quotes inside values. For example, "He said ""Yes""", which means: He said "Yes" when it is parsed.

File Header Info

Default Value: USE
Examples: 

• NONE

• IGNORE

Dropdown list

Appears when you select CSV for Data Format.

Specify whether to use the header information in the SELECT query.

Show Advanced Properties

Default Value: Deselected

Checkbox

None

Select this checkbox to display the advanced properties.
Deselect this checkbox to hide the properties.

Maximum Retries*

Default Value: 3
Example: 2

Integer/Expression

Appears when you select Show Advanced Properties checkbox.

Specify the maximum number of retry attempts to perform in case of a temporary network loss.

Snap Execution

Default Value: 
Example: Validate & Execute

Dropdown list

None

Select one of the following three modes in which the Snap executes:

• Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

• Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

• Disabled: Disables the Snap and all Snaps that are downstream from it.