Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • The Snap ignores escape characters when used along with any quote characters. When this happens, the resultant data cannot be parsed correctly using CSV Parser Snap which can cause an error. See Apache CSV Format Issue for details.

  • The Snap prepares the column header for the output view (and hence the CSV file) using the keys defined in the first record. This may result in ignoring any additional key passed in the subsequent records. We recommend that you pass values for a comprehensive set of all keys used in the input view, for the first record.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

  • Min: 1

  • Max: 1

  • CSV Generator

  • Mapper

Any document that contains data structured as key-value attributes.

Output

Document

  • Min: 1

  • Max: 1

  • File Writer

  • CSV Parser

CSV data formatted using specifications provided in the Snap's settings.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab. The available options are:

  • Stop Pipeline Execution: Stops the current pipeline execution when the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the rest of the records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

Info
  • Asterisk (*): Indicates a mandatory field.

  • Suggestion icon ((blue star)): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon ((blue star)): Indicates whether the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon ((blue star) ): Indicates that you can add fields in the field set.

  • Remove icon ((blue star)): Indicates that you can remove fields from the field set.

Field Name

Field Type

Description

Label*

Default ValueCSV Formatter
ExampleCSV_Formatter_temp

String

Specify a unique name for the Snap.

Binary header properties

Use this

fieldset

field set to add binary-header properties to the output data. These properties contain data related information that enables the system to interpret data.

Binary headers in a document can be accessed and used in the expression-enabled properties of downstream Snaps. For example, you can use binary headers to specify custom statuses associated with the output data.

Similarly, a 'content-location' property added to the binary header in this Snap can be referenced in the File name property of a File Writer Snap with the expression: $['content-location'].

Expression-enabled fields in Binary header properties are evaluated against the first input document only.

Expression

Default

value

ValueN/A
Exampletext/csv

String

Specify the value to be associated with a specific binary header property.

Field 2

Target Path

Default ValueN/A
Example$['content-location']

String

Specify the target JSON path where the value in the expression is written.

Quote character

Default

value

Value:  “
Example:  “

String

The character that you want to use as the escape character in the CSV document. For example, if you use double quotes (") as the escape character, then commas in the actual data will need to be escaped using double-quotes on both sides.

This property can be an expression. However, if the value associated with the expression contains more than one character, only the first character is used as the quote character.

Quote mode

Default

value

Value:  ALL
ExampleMINIMAL

Dropdown list

Select an option to specify how the quote character should be used in formatting the CSV data. Available options are:

  • ALLAll values are quoted.

  • MINIMAL: Values are minimally quoted.  For example, if a value includes a delimiter character, only that character is enclosed in the quote character.

  • NON_NUMERIC: All values except numbers are quoted. If values in the input documents are number objects, they are not quoted. If they are numeric strings, they are quoted.

  • NONE: No value is quoted. If a value includes a delimiter character, it is escaped with a '\' (backslash).

If the Quote character property is empty, the selection of this property is ignored and the Snap uses NONE for the Quote mode.

Delimiter

Default

value

Value:

  

, 

,


Example

Examples:

 

  • \t

,
  • \u0001

String/Expression

Specify the string or the character

that must

to be used as a delimiter in formatting the

CSV data. Tabs are common delimiters. To use a tab as the delimiter, enter \t.This field accepts any delimiter that can be a single character or a string

delimited data. Any combination of characters may be used, adhering to the following guidelines.

The input must be submitted with any control characters escaped. For example, \t

resolves to a tab. You can also use strings, such as |,|, abc*abc, or !*!.You can use an escaped Unicode string, such as \u0007, \u0021\u0022\u0021, or \u0021\t

(tab), \n (new line), or \\ (single backslash) must be escaped accordingly. Unicode characters must be specified using the Unicode escape sequence \uXXXX, where each X represents a hexadecimal digit (0-9, a-f) with all four hexadecimal digits defined.

Note

When using a single backslash (\) as a delimiter, it does not need to be escaped (\\). However, if you are using a multi-character delimiter that contains one or more backslashes (\), you must escape all backslashes (\\).

Use header

Default

value

ValueDeselected
ExampleSelected

Checkbox

Select this checkbox to indicate whether the column names in the Header fields property should be used to format the CSV data. If this checkbox is deselected, the key set of the first document data is used as a CSV header.

Note

Ensure that the CSV document header contains non-empty values for all columns.

Header fields

Default value

Use this field set to define the header values you want to use in the CSV output data.

Field names


Default ValueN/A
Example$name

String

Specify the field name for the header

values that you want to use in the output CSV data

.

Write CSV header

Default

value

ValueSelected


ExampleDeselected

Checkbox

Select this checkbox to indicate whether the header strings listed in the Header fields properties should be written to the output CSV data.

Default value: Selected.

Header size error policy

Default

value

ValueDefault
ExampleFail if record is larger than header

Dropdown list

Select an option to handle any header size errors. 

  • If the Use header field is not selected, then the Snap determines the CSV header from the key set of the first input document.

  • If the Write CSV header field is selected, then the Snap writes the CSV header to the output view binary stream.

The header size error condition occurs when any subsequent input document has additional column names which are not present in the header. To handle header size errors, you can select any of the following options:

  • Default: The Snap appends the additional values to

the CSV line.Trim record to fit header: The Snap writes the trimmed CSV line to the output view and ignores all additional values in the input document
  • the CSV line.

  • Fail if record is larger than header: The Snap writes the error document to the error view and continues to process the next input document. 

  • Trim record to fit header: The Snap writes the trimmed CSV line to the output view and ignores all additional values in the input document.

Ignore empty stream

Default

value

ValueDeselected

ExampleSelected

Checkbox

Select this

This checkbox

to indicate

determines whether

you want

the Snap

to

must or not ignore empty streams received at the input view during

Pipeline

pipeline execution.

If this option is selected, the Snap does

Select this checkbox to ignore empty input data and not produce any output stream

; else, the Snap writes

.

Deselect this checkbox to write an empty array to the output stream.

Character set

Default

value

ValueUFT-8
ExampleUTF-16LE

Dropdown list

Select the character set in which the input CSV data is encoded. The available options are:

  • UTF-8

  • UTF-16LE

  • UTF-16BE

  • ISO-LATIN-1

  • CESU-8

Write BOM

Default

value

ValueDeselected
ExampleSelected

Checkbox

Select this

This checkbox

to indicate

determines the behavior of the Snap when it starts to write the CSV output data.

If this option is selected, the Snap writes the

Select this checkbox to write the BOM (Byte Order Mark) for the character set selected in the Character-set encoding property; else, the Snap skips writing BOM.

Newline

Default

value

ValueLF
ExampleCR+LF

Dropdown list

Select an option to specify the newline characters that you want to use as a line break. The available options are: LF, CR+LF and CR.

Escape character

Default

value

Value\
Example\

String/Expression

Specify the escape character that is to be used when formatting rows. Only single characters are supported.

 As of 4.3.2, this property can be an expression

 You can also use the expression enabler to enter a value, which is evaluated with the values from the pipeline parameters.

Leave this

property

field empty if no escape character is used in the input CSV data.

Snap Execution

Default ValueValidate & Execute
Example: Disabled

Dropdown list

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Examples

...

Specify Line Feeds for Windows while Formatting Documents

Text files created on Windows have different line endings than files created on Linux. Windows uses the carriage return and line feed ("\r\n") as a line ending, while Linux uses just a line feed ("\n").

In this example, we create a Pipeline that reads CSV data and then formats it for Windows-style line feeds using the CSV Formatter Snap.

...

Download this Pipelinepipeline.

Expand
titleUnderstanding the pipeline

Use the CSV Generator Snap to supply CSV documents:

 


The raw output from the CSV Formatter Snap is as follows:

Notice how the double - quotes used in the first line of the input CSV data is escaped in the output document.

The CSV Formatter Snap formats the input data and produces an output in the Windows-style line feeds for Windows style. We selected the Newline Property with CR+LF to give the line feeds in Windows format. We also leave the Quote character field unchanged as '"'. This tells the Snap that all special characters have to be placed in quotes in the CSV output.

Note

The Newline property enables you to create output that works with the line feeds for Windows. Unless specified, the Snap's output defaults to Linux. 

Use the File Writer Snap to create a file using the output. Successful execution of the Pipeline displays the following output:

...

Create Filenames Using the CSV Formatter Snap

In this example, pick up CSV This example demonstrates how to use CSV Formatter Snap to (documents that contain details related to simple purchases purchase-related details and, depending on the value of the purchases, we ) to route them into two documents, picking up their names using the output of the CSV Formatter Snap.

...

Download this Pipelinepipeline.

Expand
titleUnderstanding the pipeline

For the purposes of this example, we are picking up CSV data from a CSV Generator Snap. You can also supply this data using a combination of File Reader and CSV Parser Snaps.

The CSV Generator Snap contains transaction details, as shown below:

We copy the output of the CSV Generator Snap and connect each copy to a Filter Snap, where we filter out transactions based on whether they are worth more or less than $40:

We now add a 'Bill_Type' binary header property to each set of filtered outputs, so each document is tagged appropriately:


We now have two records, one containing all transactions that are worth less than or equal to $40 , and another containing all transactions over $40. We now write each of these files to the SLDB using the File Writer Snap, configuring it to include this classification of transactions in the filename. We do this by using the expression $Bill_Type+".csv" expression in the File name property.

You can now see that two new files were created in the project: $Small.csv and $Large.csv:

Download this Pipeline.

...

Pass Execution Status Codes through Binary Header Properties of the CSV Formatter Snap

In this example, we shall send CSV data from ServiceNow into SnapLogic using a Triggered Task. We shall then see how you can use the CSV Formatter Snap in the triggered Pipeline pipeline to manipulate the execution status (error) codes generated during Pipeline pipeline execution.

...

Download this Pipeline.

Expand
titleUnderstanding the pipeline

For the example to work, the Pipeline pipeline triggered by the Task should lead to an error. We shall route the error output to an error view. We shall then use the error view to manipulate the Status header of the error document. Once this is done, the parent Task sends the updated error output back to ServiceNow, and we should see the updated status code in the ServiceNow Administration interface.

We need to perform the following tasks:

  1. Create a

Pipeline
  1. pipeline that accepts CSV data, generates an error, and manipulates the error data to update the status code.

  2. Create a Triggered Task that can trigger the Pipeline created in Step 1.

  3. Configure ServiceNow to send and receive CSV data from the Triggered Task created in Step 2.

Let us look at each of these tasks in some detail:

Creating

Create a

Pipeline

pipeline to accept CSV and manipulate error document data

This Pipeline is structured as shown in the screenshot above. Let us now look at its constituents:

Snap Name

Snap Type

Task Description

Settings (Click to expand screenshot)

Parse Data

CSV Parser

Parses streaming data from the parent Triggered Task.

Enter Faulty Mappings

Mapper

Creates problematic mappings to trigger an error in the downstream Snap.

Image RemovedImage Added

Change Status Codes

Mapper

Manipulates (or changes) the status code in the input document containing the error view from the Enter Faulty Mappings (Mapper) Snap.

In this example, we choose to update the status code to 603.

Update Headers

CSV Formatter

Adds the updated status code into the binary header of the output CSV document to be sent to ServiceNow.

Creating

Create a Triggered Task that sends CSV data into the TriggeredTask_CSVFormatter_Example Pipeline

To create a Triggered Task:

  1. Click the Create Task icon at the SnapLogic Designer toolbar. This displays the Create Task popup. Configure the Task as shown below:

    Image Modified

    Your Task is now configured. You now need to configure your ServiceNow account to send CSV data to this Task.

Configuring

Configure ServiceNow to send and receive CSV data from the Triggered Task

Note

This step can only work if you have a ServiceNow account.

To configure ServiceNow to send and receive CSV data from the Triggered Task created in Step 2:

  1. Log into your ServiceNow account and open the pane on the left of the page.

  2. Search for the string outbound in the Filter field displayed in the top-left section of the page. From the filtered list of options displayed, click REST Messages under the System Web Services section.

    Image Modified


    This displays the REST Messages page.

  3. Click New to create a new REST message and configure the message as shown below:

    Image Modified

  4. Click Submit to save your changes. This creates the REST call. You must now

need to
  1. configure the call with the data you want to send. To do so, click the REST call. This displays the details of the REST call. 

  2. Image Modified

    Scroll down to the HTTP Methods section and click

on
  1. the Default Get method created automatically for the call. This displays the call's details:

    Image Modified

  2. You need to send a POST call to the Task

,
  1. so that you can provide the content that must be processed as a result of the call. To do so, click the HTTP Method field and, from the drop-down list displayed, select POST. Once you make this selection, the Content field appears at the bottom of the page. 

  2. Enter the content you want to use for the call:

    Image Modified


  3. Click Test. This triggers the SnapLogic URL endpoint (This is the Task we created earlier in this example.) and displays the output of the Pipeline in the HTTP Status field: 603, as configured using the Mapper Snap, above.

    Image Modified

You can similarly use the CSV Formatter Snap to update binary header values associated with any input document.

Downloads

Info
  1. Download and import the Pipeline into SnapLogic.

  2. Configure Snap accounts as applicable.

  3. Provide Pipeline parameters as applicable.

Attachments
patterns*.slp,*.zip

Snap Pack History

Related

...

Content