On this Page

Overview

This is a Transform type Snap that converts categorical fields into numeric fields. There are two available encoding options: Integer Encoding and One Hot Encoding.

SnapLogic Documentation > Categorical to Numeric > CatToNum.png

Input and Output

Expected input

First input view: A document containing categorical fields.
Second input view: A document containing data statistics computed by the Profile Snap.

Expected output: A document containing numeric field.

Expected upstream Snaps

First input view: A Snap that has a document output view. For example, Mapper, CSV Generator, or Categorical to Numeric.
Second input view: A sequence of File Reader and JSON Parser Snaps. These Snaps read the data statistics computed by the Profile Snap in another pipeline. It is required to select Value distribution in the Profile Snap and set Top value limit according to the number of unique values; or set to 0, which means unlimited.

Expected downstream Snaps: A Snap that has a document input view. For example, Mapper, JSON Formatter, or Type Converter.

Prerequisites

None.

Configuring Accounts

Accounts are not used with this Snap.

Configuring Views

Input	This Snap has exactly two document input views, the Data input view and the Profile input view.
Output	This Snap has exactly one document output view.
Error	This Snap has at most one document error view.

Troubleshooting

None

Limitations and Known Issues

None

Modes

Ultra Pipelines: Works in Ultra Pipelines.

Snap Settings

Label	Required. The name for the Snap. Modify this to be more specific, especially if there are more than one of the same Snap in the pipeline.
Policy	The preferences for fields and encoding methods. For each policy, select the input field with categorical values, the encoding method, and the result field.
Field	Required. The field that must be transformed. This is a suggestible property that lists all available fields in the input documents. Default value: None.
Rule	Required. The type of transformation to be performed on the selected field. Two options are available: Integer Encoding: Each unique value is mapped to an integer starting from 1. One Hot Encoding: Each unique value results in a field. The value is 1 if it matches the field name; otherwise, it is 0. Default value: Integer Encoding
Result field	Required. The result field that is used in the output map. If the variable in the Result field property is the same as the one in the Field property, the values are overwritten. If the Result field does not exist in the original input document, a new field is added. Default value: None.

Best Practices

We recommend you use the Sort Snap before the Categorical to Numeric Snap to prevent the pipeline from stalling when handling large datasets.
We recommend you use the Profile Snap in another (child/parent pipeline) pipeline and process the results to use with Categorical to Numeric Snap when handling large datasets.

Examples

Assigning Team based on Position

This pipeline demonstrates how you can use the Categorical to Numeric Snap to assign team number for employees (numeric) based on their position (categorical).
SnapLogic Documentation > Categorical to Numeric > C-N_Example_1.jpg

Download this pipeline.

In this example, the CSV Generator Snap contains employee data with categorical fields. The Copy Snap duplicates the data flow and feeds it into the Categorical to Numeric and Profile Snaps. The Profile Snap computes data statistics and sends them to the Categorical to Numeric Snap. The Categorical to Numeric Snap then converts the position values (categorical) into team numbers (numeric), based on the selected encoding methods.

The CSV Generator Snap contains employee data as shown below.

name,gender,position,experience
andy,male,technical_writer,11
jeff,male,technical_writer,22
mindy,female,instructional_designer,7
kathy,female,technical_writer,8
thomas,male,technical_writer,2
kim,female,technical_writer,9
bruce,male,technical_writer,4
john,male,technical_writer,10
jonah,male,instructional_designer,1
clark,male,instructional_designer,15
hal,male,technical_writer,7

The Categorical to Numeric Snap contains two policies that assign employees to groups based on their position. Integer Encoding maps each position into an integer ($team_numer). One Hot Encoding creates a field for each position ($team_instructional_designer and $team_technical_writer).

SnapLogic Documentation > Categorical to Numeric > Example_CatToNum.png

The Profile Snap computes the value distribution of the input data, which is required by Integer Encoding and One Hot Encoding.

SnapLogic Documentation > Categorical to Numeric > Example_Profile.png

As you can see, the value distribution of $position contains "technical_writer" and "instructional_designer". Integer Encoding assigns values 1 and 2 to these, while One Hot Encoding creates a field for each of them.

SnapLogic Documentation > Categorical to Numeric > C-N_Example_4-1.jpg

The preview output of Categorical to Numeric Snap is shown below. This Snap adds 3 fields to the output.

$team_number: This is the output of the first policy, where you use Integer Encoding to convert values in the $position field to numeric values.
$team_instructional_designer and $team_technical_writer: These two fields are created by the second Policy. The first field indicates the "instructional_designer" position, and the second field indicates the "technical_writer" position.