...
On this Page
Table of Contents | ||||||
---|---|---|---|---|---|---|
|
...
Snap type:
Transform
...
Description:
This Snap takes an expression, evaluates it, and writes the result to the provided target path. If an expression fails to evaluate, use the Views tab to specify error handling. For more information on expressions, see Understanding Expressions in SnapLogic.
This Snap supports both binary and document data streams. The default input and output is document, but you can select Binary from the Views tab in the Snap's settings.
Structural Transformations
The following structural transformations from the Structure Snap are supported in the Mapper Snap:
- Move - A move is equivalent to doing a mapping without a pass-through. The source value is read from the input data and placed into the output data. Since pass-through is turned off, the input data is not copied to the output. Also, the source value is treated as an expression in the Mapper, but it is a JSONPath in the Structure Snap. A jsonPath() function was added to the expression language that can be used to execute a JSONPath on a given value. If pass-through is enabled, then you will probably have to delete the old value.
- Delete - Write a JSONPath in the source column and leave the target column blank.
- Update - All of the cases for update can be handled by writing the appropriate JSONPath. For example:
- Update value: target path = $last_name
- Update map: target = $address.first_name
- Update list: target = $names[(value.length)]
- The '(value.length)' evaluates to the current length of the array, so the new value will be placed there at the end.
- Update list of maps: target = $customers[*].first_name
- This translates into "write the value into the 'first_name' field in all elements of the 'customers' array".
- Update list of lists: target = $lists_of_lists[*][(value.length)]
For performance reasons, the Mapper does not make a copy of any arrays or objects written to the Target Path. If you write the same array or object to more than one target path and plan to modify the object, make the copy yourself. For example, given the array "$myarray" and the following mappings:
Code Block |
---|
$myarray -> $MyArray
$myarray -> $OtherArray |
Any future changes made to either "$MyArray" or "$OtherArray" are in the both arrays. In that case, make a copy of the array as shown below:
Code Block |
---|
$myarray -> $MyArray
[].concat($myarray) -> $OtherArray |
The same is true for objects, except you can make a copy using the ".extend()" method as shown below:
Code Block |
---|
$myobject -> $MyObject
$myobject.extend({}) -> $OtherObject |
...
[None]
...
Works in Ultra Pipelines.
...
Expressions used in this Snap, downstream of any Snowflake Snap, that evaluate to very large values such as EXP(900) are displayed in the input/output previews as Infinity
. However, you can see the exact values in the validation previews; hence, ignore this error.
...
Accounts are not used with this Snap.
...
This Snap has at most one document error view and produces zero or more documents in the view. If the Snap fails during the operation, an error document is sent to the error view containing the fields error, reason, original, resolution, and stacktrace:
Code Block |
---|
{ error: "$['SFDCID__c\"name'] is undefined" reason:
"$['SFDCID__c\"name'] was not found in the containing object." original: {[:{}
resolution: "Please check expression syntax and data types."
stacktrace: "com.Snaplogic.Snap.api.SnapDataException: ...
} |
Passing Binary Data
You would convert binary data to document data by preceding the Mapper Snap with the Binary-to-Document Snap. Likewise, to convert the document output of the Mapper Snap to binary data, you would add the Document-to-Binary Snap after the Mapper Snap.
Currently, you can do this transformation within the Mapper Snap itself. You set the Mapper Snap to take binary data as its input and output by using the $content expression.
Note | ||
---|---|---|
| ||
If you are only working with a binary stream as both input and output, you must set both source and target fields with $content, then manipulate the binary data using the Expression Builder. If you do not specify this mapping, then the binary stream from the binary input document is passed through unchanged. |
...
Settings
...
Label
...
Null-safe access
...
Enabled: Lets you set the target value to null in case the source path does not exist. For example $person.phonenumbers.pop() ->$ lastphonenumber may result in an error if person.phonenumbers does not exist on the source data. Enabling Null-safe access allows the Snap to write null to lastphonenumber instead of causing an error.
Disabled: Fails if the source path does not exist, ignores the record entirely, or writes the record to the error view depending on the setting of the error view property.
Pass through
Required. This setting determines if data should be passed through or not. If not selected, then only the data transformation results that are defined in the mapping section will appear in the output document and the input data will be discarded. If selected, then all of the original input data will be passed into the output document together with the data transformation results.
Note |
---|
This setting is impacted by Mapping Root. If Mapping Root is set to $ and Pass through is not selected, anything not mapped in the table will not pass through. However, if Mapping Root is set to $customer and Pass through is not selected, it will only apply to the items within the Mapping Root level. That means that anything above the Mapping Root level will pass though and items at the Mapping level that are not mapped in the table will not pass through. |
Default: Not selected.
When to always select Pass through
Always select Pass through if you plan to leave the Target path field blank in the Mapper Snap; otherwise, the Snap throws an error informing you that the field that you want to delete doesn't exist. This is expected behavior.
Say you have an input file that contains a number of attributes; but you need only two of these downstream. So, you connect a Mapper to the Snap supplying the input file, select the two attributes you need by listing them in the Expression fields, leave the Target path field blank, and select Pass through. When you execute the pipeline, the Mapper Snap evaluates the input documents/binary data and picks up the two attributes that you want, and passes the entire document/binary data through to the Target schema. From the list of available attributes in the Target Schema, the Mapper Snap picks up the two attributes you listed in the Expression fields, and passes them as output. However, if you hadn't selected the Pass through check box, the Target Schema would be empty, and the Mapper would display a No schema available
error.
Mapping Root
...
Required. This setting specifies the sub-section of the input data to be mapped. For more information, see Understanding the Mapping Root.
Default: $
...
Required. Expression and target to write the result of the expression. Expressions that are evaluated will remove the source targets at the end of the run. For example:
Expression | Target Path |
---|---|
$first.concat(" ", $last) | $full |
Note |
---|
Incoming fields from previous Snaps that are not expressly defined in the Mapping Table are passed through the Data Snap to the next Snap. However, when defining output fields in the Target Path, if the field name is the same as a field name that would otherwise "pass-through", the field in the mapping table wins and will override the output. |
See Understanding Expressions in SnapLogic for more information on the expression language and Using Expressions for usage guidelines.
Target Path Recommendation
Iris simplifies configuring the Target path property in the Mapper Snap by recommending suggestions for the Expression and Target path property mapping. To make these suggestions, Iris analyzes Expression and Target path mappings in other Pipelines in your Org and suggests the exact matches for the Expressions in your current Pipeline. The suggestions are displayed upon clicking
For example, you have the Expression $Emp.Emp_Personal.FirstName in one of your Pipelines. And you have set the Target path for this expression as $FirstName. Now, if you use the expression $Emp.Emp_Personal.FirstName in a new Pipeline, then Iris suggests $FirstName as one of the recommended Target paths. This helps you standardize the naming standards within your org.
The following video illustrates how Iris recommends Target path in a Mapper Snap:
...
Managing Numeric Inputs in Mapper Expressions
...
Say the value being passed from upstream for $num is 20.05. You would expect the value of $numnew to now be 120.05. But, when you execute the Snap, the value of $numnew is shown as 20.05100.
This happens because, as of now, the Mapper Snap reads all incoming data as strings, unless they are expressly listed as integers (INT) or decimals (FLOAT). So, to ensure that the upstream numeric data is appropriately interpreted, parse the data as a float. This will convert the numeric data into a decimal; and all calculations performed on the upstream data in the Mapper Snap will work as expected:
...
The value of $numnew is now shown as 120.05.
...
Multiexcerpt include macro | ||||
---|---|---|---|---|
|
...
Multiexcerpt include macro | ||||
---|---|---|---|---|
|
Mapping Table
The mapping table makes it easier to do the following:
- Determine which fields in a schema are mapped or unmapped.
- Create and manage a large mapping table through drag-and-drop.
- Search for specific fields.
For more information, see Using the Mapping Table.
Example
Removing Columns from Excel Files Using Mapper
In this example, you read an Excel file from the SLDB and remove columns that you do not need from the file. You then write the updated data back into the SLDB as a JSON file.
...
Parse the file using the Excel Parser Snap. You can preview the parsed data by clicking the icon.
...
In the Expression field, you enter the criteria that you want to use to remove the Discounts and Month Name columns.
Paste code macro |
---|
$.filter((value, key) => !key.match("Discounts|Month Number")) |
...
Example Data Output
Successful Mapping
If your source data looks like:
Code Block |
---|
{
"first_name": "John",
"last_name": "Smith",
"phone_num": "123-456-7890"
} |
And your mapping looks like:
- Expression: $first_name.concat(" ", $last_name)
- Target path: $full_name
Your outgoing data will look like:
Code Block |
---|
{
"full_name": "John Smith",
"phone_num": "123-456-7890"
} |
Unsuccessful Mapping
If your source data looks like:
Code Block |
---|
{
"first_name": "John",
"last_name": "Smith",
"phone_num": "123-456-7890"
} |
And your mapping looks like:
- Expression: $middle_name.concat(" ", $last_name)
- Target path: $full_name
An error will be thrown.
Example:
Escaping Special Characters in Source Data
This example demonstrates how you can use the Mapper Snap to customize source data containing special characters so that it is correctly read and interpreted by downstream Snaps.
In the sample Pipeline, custom JSON data is provided in the JSON Generator Snap, wherein the values of field1
and field10
include the special character (').
The output preview of the JSON Generator Snap displays the special character correctly:
Before sending this data to downstream Snaps, you may need to prefix the special characters with an escape character so that downstream Snaps correctly interpret these.
You can do this using the Expression field in the Mapper Snap. Based on the accepted escape characters in the endpoint, you can select from the following expressions:
...
Single quote (')
JSON:
$original.mapValues((value,key)=> value.toString().replaceAll("'","''"))
OR
$original.mapValues((value,key)=> value.toString().replaceAll("'","\''"))
CSV:
$[' Business-Name'].replace ("'","''")
...
Ampersand (&)
JSON:
$original.mapValues((value,key)=> value.toString().replaceAll("'","\&'"))
OR
$original.mapValues((value,key)=> value.toString().replaceAll("'","&'"))
CSV:
$[' Business-Name'].replace ("'","&'")
...
Backslash (\)
JSON:
$original.mapValues((value,key)=> value.toString().replaceAll("'","\\'"))
Info |
---|
Backslash is configured as an escape character in SnapLogic. Therefore, it must itself be escaped to be displayed as text. |
CSV:
$[' Business-Name'].replace ("'","\\'")
...
In this way, you can customize the data to be passed on to downstream Snaps using the Expression field in the Mapper Snap.
Refer to the Community discussion for more information.
See it in Action
The SnapLogic Data Mapper
...
In this article
Table of Contents | ||||||||
---|---|---|---|---|---|---|---|---|
|
Overview
Use this Snap to transform incoming data with the specific mappings and produce new output data (binary or document). The Mapper Snap evaluates an expression and writes the result to the target path. Use the Views tab to specify error handling if an expression fails to evaluate.
...
Pass Binary Data
To convert binary data to document data add the Binary-to-Document Snap upstream of the Mapper Snap. Similarly, add the Document-to-Binary Snap downstream of the Mapper Snap to convert the document output to binary data.
You can also transform binary data to document data in the Mapper Snap itself with the
$content
expression.
Binary Input and Output
If you only work with a binary stream as both input and output, you must set both source and target fields with
$content
, then use the Expression Builder to manipulate the binary data. If you do not specify this mapping, then the binary stream from the binary input document passes through with no change.
Snap Type
The Mapper Snap is a Transform-type Snap that transforms data and passes it to the downstream Snap.
Support for Ultra Pipelines
Works in Ultra Pipelines.
Limitations
The Mapper Snap does not support Base64URL decoding.
Expressions used in this Snap (downstream of any Snowflake Snaps) that evaluate to very large values such as
EXP(900)
displays asInfinity
in the input/output previews. However, the exact evaluated values are in the validation previews. Learn more: JavaScript Limitations in Displaying Numbers.
Snap Views
Type | Format | Number of Views | Examples of Upstream and Downstream Snaps | Description |
---|---|---|---|---|
Input |
|
| Binary-to-Document | This Snap can have at most one document or binary input view. If you do not specify an input view, the Snap generates a downstream flow of one row. By default, the Input type is Document. You can select Binary type if your input is in Binary format. |
Output |
|
| Any Document Snap | This Snap has exactly one document or binary output view. By default, the Output type is Document. You can select the Binary type to view the output in the Binary format. |
Error | Error handling is a generic way to handle errors without data loss or Snap execution failure. You can handle the errors that the Snap might encounter when running the pipeline with one of the following options from the When errors occur list under the Views tab. The available options are:
Learn more about Error handling in Pipelines. |
Snap Settings
Info |
---|
|
Field Name | Field Type | Description | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
Label* Default Value: Mapper | String | The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline. | ||||||||
Null-safe access Default Value: Deselected | Checkbox | Select this checkbox to set the target value to null if the source path does not exist. For example, | ||||||||
Pass through Default Value:Deselected | Checkbox | Select this checkbox so the Snap passes all the original input data into the output document with the data transformation results. If you deselect this checkbox, only the data transformation results in the mapping section appear in the output document and the input data is discarded.
| ||||||||
Transformations* | Use this field set to configure the settings for data transformations. | |||||||||
Mapping Root Default Value: $ | String/Suggestion | Specify the subsection of the input data to map. Learn more: Understand the Mapping Root. | ||||||||
Input Schema | Dropdown list | Select the input data (from the upstream Snap) that you want to transform. Drag the item you want to map and place it under the Mapping table. | ||||||||
Sorted Default Value: Selected | Checkbox | Select this checkbox to sort the input schema and the target schema. The sort options are:
| ||||||||
Mapping table* | Use this field set to specify the source path, expression, and target path columns used to map schema structure. The mapping table makes it easier to:
Learn more: Use the Mapping Table. | |||||||||
Expression* Default Value: N/A | String/Expression | Specify the function to use to transform the data. For example, combine, concatenate, or flatten. Expressions that are evaluated replace the source targets at the end of the pipeline runtime.
Learn More: Understand Expressions in SnapLogic and Use Expressions for usage guidelines.
| ||||||||
Target path Default Value: N/A | String/Suggestion | Specify the target JSON path where the value from the evaluated expression is written. For example, after evaluation of the Target Path Recommendation
For example, you have the Expression $Emp.Emp_Personal.FirstName in one of your pipelines. You set the Target path for this expression as $FirstName. Now, if you use the expression $Emp.Emp_Personal.FirstName in a new pipeline, then Iris recommends $FirstName as one of the Target paths to help you standardize the naming standards in your org. The following shows how Iris recommends the Target path in a Mapper Snap: | ||||||||
Snap Execution | Dropdown list | Select one of the three modes in which the Snap executes. Available options are:
|
Input and Output Preview
The Input Schema of the Mapper Snap displays the input strings from the upstream Snap. On validation, the Input Preview displays the preview of the input from the upstream Snap, and the Output Preview displays the output preview that passes to the downstream Snap. The following elements are available in the input and output previews:
...
Number | Element | Description |
---|---|---|
1 | Expand All | Expands the objects in the Input/Output. |
2 | Collapse All | Collapses the objects in the Input/Output. |
3 | Render whitespace | When you select this checkbox, the blank spaces (leading, trailing, or in the middle of a string) in an expression field renders as symbols in the output:
When you deselect this checkbox, the Snap renders blank spaces and tabs as-is. By default, the Render whitespace checkbox is selected. |
4 | Download | Downloads the JSON file. |
Data Output Example
Successful Mapping | ||||||
---|---|---|---|---|---|---|
If your source data looks like | And your mapping looks like | Your output data will look like | ||||
|
|
| ||||
Unsuccessful Mapping | ||||||
|
| An error displays. |
Mapper Examples
...
Watch the Data Mapper in Action
The Data Mapper
Widget Connector | ||||||||
---|---|---|---|---|---|---|---|---|
|
...
SnapLogic Best Practices: Data Transformations and Mappings
...
|
Best Practices: Data Transformations and Mappings
Widget Connector | ||||||||
---|---|---|---|---|---|---|---|---|
|
...
...
|
...
|
Attachments | ||
---|---|---|
|
...
Related Content
Insert excerpt | ||||||
---|---|---|---|---|---|---|
|