Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Parameter NameData TypeDescriptionDefault ValueExample 
LabelString
Excerpt Include
File Writer
File Writer
nopaneltrue
ELT TransformMap Data
Pass throughCheck boxSelect this to include the columns in the upstream Snap's query projection along with the existing columns that are specified in the mapping table.Not selectedSelected
Input Schema

Use this field set to select or drag and drop fields available in the input schema coming from the upstream Snap into the Mapping Table and define the expression, as needed for the transformation. 

This field set contains the following fields.

  • Select All
  • Deselect All
  • (Search/Find) box
  • All
Note
titleRendering Complex Data Types in Databricks Lakehouse Platform

Based on the data types of the fields in the input schema, this section of the Snap renders the preview of complex data types like map and struct as object data type and array as an array data type. It renders all other incoming data types as-is except for the values in binary fields are displayed as a base64 encoded string and as string data type.


Select AllHyperlink

Click this hyperlink to select (the check boxes of) for using all the fields from the input schema to define your transformation criteria in the Mapping table. Drag and drop the selected fields to populate all their field names in the Expression column.

N/AN/A
Deselect AllHyperlinkClick this hyperlink to deselect all the selected check boxes to restart selecting the fields needed to define your transformation criteria.N/AN/A
(Search/Find)StringStart entering a field name to filter the list by the entered keyword and select the fields needed for defining the transformation.N/AN/A
AllDrop-down list

Change the selection in this drop-down list to filter the list of fields displayed in the Input Schema. The available options are:

  • All. To view all fields available in the Input Schema.
  • Mapped. To view only the input fields that are selected/mapped in the Mapping table.
  • Unmapped. To view only the input fields that are not selected/mapped in the Mapping table.
AllSelected
Mapping Table

This field set enables you to specify the transformations you want to perform on the columns/records in the source table.

Each different transformation must be specified in a separate row. Click + to add a new row.

This field set consists of the following fields:

  • Expression
  • Target Path
Note

The output query contains only those columns that are specified here unless the Pass through check box is also selected.


ExpressionString/ExpressionThe column and the transformation operation that you want on that column.N/A

EMPLOYEE

ORDER_NUMBER + 1

Target PathString

The name to be assigned to the column.

Info

If the Target Path field is empty for a specified expression, the respective column is deleted from the output. See the Example below.


N/A

EMPLOYEE

ORDER_NUMBER_NEW

Output Schema

Use this field set to select or drag and drop fields available in the output schema coming from the downstream Snap into the Mapping Table and define the expression, as needed for the transformation. Output schema fields are populated upon validation of the Pipeline and this depends on the downstream Snap's configuration.

This field set contains the following fields.

  • Select All
  • Deselect All
  • (Search/Find) box
  • All
Note
titleRendering Complex Data Types in Databricks Lakehouse Platform

Based on the data types of the fields in the input schema, this section of the Snap renders the preview of complex data types like map and struct as object data type and array as an array data type. It renders all other incoming data types as-is except for the values in binary fields are displayed as a base64 encoded string and as string data type.


Select AllHyperlink

Click this hyperlink to select (the check boxes of) for mapping all the fields from the output schema with corresponding input field expressions (your transformation criteria) in the Mapping table. drag and drop the selected fields to populate all their field names in the Target Path column.

N/AN/A
Deselect AllHyperlinkClick this hyperlink to deselect all the selected check boxes to restart selecting the output fields needed to define your transformation criteria.N/AN/A
(Search/Find)StringStart entering a field name to filter the list by the entered keyword and select the fields to include in the transformation criteria.N/AN/A
AllDrop-down list

Change the selection in this drop-down list to filter the list of entries displayed in the Output Schema. The available options are:

  • All. To view all fields available in the Output Schema.
  • Mapped. To view only the target fields that are selected/mapped in the Mapping table.
  • Unmapped. To view only the target fields that are not selected/mapped in the Mapping table.
AllSelected
Input PreviewDisplay-onlyThis section of the Snap's Settings displays the preview (partial result) of the incoming data from the previous Snap in the Pipeline.N/AN/A
Output PreviewDisplay-onlyThis section of the Snap's Settings displays the preview (partial result) of applying the transformation criteria defined in the Mapping table.N/AN/A
Get preview dataCheck box
Multiexcerpt include macro
namegetpreviewdata
pageELT Intersect
Not selectedSelected

...