Scale
- Mohammed Iqbal
- Lakshmi Manda
- John Brinckwirth
On this Page
Overview
The Scale Snap is a Transform type Snap that scales numeric values in fields to specific ranges or applies statistical transformations. The Snap helps you with data preparation before applying a machine learning algorithm to the data. The Scale Snap supports the following four transformations:
- Scale to range [0,1]
- Scale to range [-1,1]
- Z-transformation
- Log-transformation
Input and Output
Expected input
- First input view: A document with numeric fields.
- Second input view: A document containing data statistics computed by the Profile Snap.
The Scale Snap processes the data in streams while the Profile Snap consumes all the data before it derives any statistics. Therefore, while using the Profile Snap:
- Build a pipeline with the Profile Snap to generate data statistics that are stored in the JSON format using the JSON Formatter and File Writer Snaps.
- Use the File Reader and JSON Parser Snaps to read statistics from the Profile Snap and feed the output data into the Scale Snap.
Expected output: A document with numeric fields that are transformed as per the transformation type.
Expected upstream Snaps
- First input view: A Snap that has a document output view. For example, Mapper, CSV Generator, or Categorical to Numeric.
- Second input view: A sequence of File Reader and JSON Parser Snaps. These Snaps read the data statistics computed by the Profile Snap in another pipeline. It is required to select Value distribution in the Profile Snap and set Top value limit according to the number of unique values; or set to 0, which means unlimited.
Expected downstream Snaps: A Snap that has a document input view. For example, Mapper, JSON Formatter, or Type Converter.
Prerequisites
None
Configuring Accounts
Accounts are not used with this Snap.
Configuring Views
Input | This Snap has exactly two document input views - the Data input view and the Profile input view.
|
|---|---|
| Output | This Snap has exactly one document output view. |
| Error | This Snap has at most one document error view. |
Troubleshooting
None
Limitations and Known Issues
None
Modes
- Ultra Pipelines: Works in Ultra Pipelines.
Snap Settings
| Label | Required. The name for the Snap. You can modify this to be specific, especially if you have more than one of the same Snap in your pipeline. |
|---|---|
| Policy | Specify your preferences for a field's transformation. Each policy contains an input field, transformation rule, and the result field. The Snap transforms the values in the input field and writes them to the result field. You can apply multiple transformations to the same input field. However, the result fields must be different. If the result field is the same as the input field, the Snap overwrites the input field with the result field. |
| Field | The field that must be transformed. This is a suggestible field that suggests all the fields in the dataset. The Snap displays an error message for non-numeric fields. Default value: None |
| Rule | The type of transformation to be performed on the selected field. The available options are:
For example, if you want to transform 35 with the following statistics:
The result for each transformation rule is as following:
|
| Result field | The result field to use in the output map. If the Result field is the same as Field, the values are overwritten. If the Result field does not exist in the original input document, a new field is added. Default value: None |
Snap Execution | Select one of the following three modes in which the Snap executes:
Default Value: Execute only |
Example
Normalizing Values
This pipeline demonstrates how to normalize values using the Scale Snap. An input dataset with 19 rows is passed to the Scale Snap. The dataset consists of fields with different ranges and we are scaling those fields into the same range of [0,1] to prepare this dataset before applying machine learning algorithm.
Download this pipeline.
The input is a Blood Transfusion Service Center dataset in CSV format that contains the following details: last_donation(month), total_donation, total_blood(cc), first_donation(month), and donate_in_mar_2017. The dataset is derived from here. Fields in this dataset are scaled into the range [0,1] using Scale Snap.
The input data preview is as follows:
We use the Type Converter Snap which is configured as the following to automatically convert values into the appropriate type:
The Auto checkbox is selected so the Snap automatically detects and converts the data types, as required.
The converted data from Type Converter Snap is passed to the Profile Snap to derive statistical details of the dataset. The Profile Snap is configured as follows:
Following is the output preview of the Profile Snap.
The data from Profile Snap is passed to the Scale Snap. The Scale Snap is configured as follows:
The output preview of the Scale Snap is:
You can see that all the values are scaled in the range [0,1].
Downloads
Important steps to successfully reuse Pipelines
- Download and import the pipeline into the SnapLogic application.
- Configure Snap accounts as applicable.
- Provide pipeline parameters as applicable.
Snap Pack History
Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.