Cross Validator (Regression)

On this Page

Overview

This is a Transform type Snap that performs K-fold Cross Validation on a regression dataset. Cross validation is a technique for evaluating ML algorithms by splitting the original dataset into K equally-sized chunks. K is the number of folds. In each of the K iterations, K-1 chunks are used to train the model while the last chunk is used as a test set. The average error and other statistics are computed to be used to select the most suitable algorithm for the dataset. 

In the settings, you can select the algorithm, specify parameters, and number of folds. If you want to perform K-fold Cross Validation on classification dataset, use the Cross Validator - Classification Snap instead. 

Input and Output

Expected input: The regression dataset.

Expected output: Statistical information about the performance of the selected algorithm on the dataset.

Expected upstream Snap: Any Snap that generates a dataset document. For example, CSV Generator, JSON Generator, or a combination of File Reader and JSON Parser.

Expected downstream Snap: CSV/JSON Formatter Snap and File Writer Snap can be used to write the output statistics to file.

Prerequisites

  • The data from upstream Snap must be in tabular format (no nested structure). 
  • This Snap automatically derives the schema (field names and types) from the first document. Therefore, the first document must not have any missing values.

Configuring Accounts

Accounts are not used with this Snap.

Configuring Views

Input

This Snap has exactly one document input view.
OutputThis Snap has exactly one document output view.
ErrorThis Snap has at most one document error view.

Troubleshooting

None.

Limitations and Known Issues

None.

Modes


Snap Settings


LabelRequired. The name for the Snap. Modify this to be more specific, especially if there are more than one of the same Snap in the pipeline.
Label field

Required. The label or output field in the dataset. This must be numeric. This is the field that the model will be trained to predict.

Default value: None

Example: $price

Algorithm

Required. The regression algorithm to be used to build the model. There are four regression algorithms available currently:

  • K-Nearest Neighbors
  • Linear Regression
  • Decision Stump
  • Random Forests

The implementations are from WEKA, an open source machine learning library in Java.

Default value: K-Nearest Neighbors

Option

The parameters to be applied on the selected algorithm. Each algorithm has a different set of parameters to be configured in this property. If this property is left blank, the default values are applied for all the parameters. If specifying multiple parameters, separate them with a comma ",".

See Options for Algorithms section below for details.

Default value: None

Examples:

  • batch_size = 120
  • batch_size = 120, collapse_tree = true
Fold

Required. The number of folds.

Minimum value: 2

Default value: 10

Use random seed

If selected, Random seed is applied to the randomizer in order to get reproducible results.

Default value: Selected

Random seed

RequiredNumber used as static seed for randomizer.

Default value: 12345

Snap Execution

Select one of the following three modes in which the Snap executes:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Default ValueExecute only

Example: Validate & Execute

Error rendering macro 'excerpt-include' : No link could be created for 'Trainer -- Regression'.


Example


Heating Load Prediction – Cross Validation

This pipeline demonstrates a typical cross validation exercise for a dataset before a model is trained to prediction the target field. The dataset is a record of various aspects of a building. The building's required heating load depends upon each of these aspects. The cross validation is to validate the model's ability to predict this heating load.

Download this pipeline.

 Understanding the pipeline

Input

The input is generated by the CSV Generator Snap and is composed of the following fields:

  • Relative Compactness
  • Surface Area
  • Wall Area
  • Roof Area
  • Overall Height
  • Orientation
  • Glazing Area
  • Glazing Area Distribution
  • Heating Load

This dataset has been sourced fromUCI Dataset Archive.

Objective

Use Cross Validator (Regression) Snap to evaluate how each ML algorithm performs in this dataset.

Data Preparation

This input document is passed through the Type Converter Snap that is configured to automatically detect and convert the data types. In any ML pipeline, you must first analyze the input document using the Profile Snap and the Type Inspector Snap to ensure that there are no null values or that the data types are accurate. This step is skipped in this example for simplicity's sake.

Below is a preview of the output from the Type Converter Snap:

Cross Validation

After preparing the data, the first thing to do is K-fold Cross Validation. Cross Validator (Regression) Snap takes the full dataset and randomly splits the dataset into training set and test set which are used to evaluate the selected ML algorithm.

Below is the configuration of the Cross Validator (Regression) Snap:

The output from this Snap is as shown below:

Optionally, you can write the output from the Cross Validator (Regression) Snap into a file using the downstream File Writer Snap.

Download this pipeline.

Additional Example

The following use case demonstrates a real-world scenario for using this Snap:

Downloads

Important steps to successfully reuse Pipelines

  1. Download and import the pipeline into the SnapLogic application.
  2. Configure Snap accounts as applicable.
  3. Provide pipeline parameters as applicable.

  File Modified

File Energy_Efficiency_Cross_Validation.slp

Nov 09, 2018 by Mohammed Iqbal

Snap Pack History

 Click to view/expand
Release Snap Pack VersionDateType  Updates

August 2024

main27765

 

Stable

Upgraded the org.json.json library from v20090211 to v20240303, which is fully backward compatible.

May 2024main26341 StableUpdated and certified against the current SnapLogic Platform release.
February 2024main25112 StableUpdated and certified against the current SnapLogic Platform release.
November 2023main23721Nov 8, 2023StableUpdated and certified against the current SnapLogic Platform release.

August 2023

main22460

Aug 16, 2023

Stable

Updated and certified against the current SnapLogic Platform release.

May 2023433patches21854 Latest

Fixed an issue with the Cross Validator (Classification) Snap where the native Windows DLL caused the Snaplex to stall

May 2023433patches21644 Latest

Improved an error message in the Remote Python Script Snap to explain the reason and resolution for the case where a Python script has errors.

May 2023

main21015 

Stable

Upgraded with the latest SnapLogic Platform release.

February 2023main19844 StableUpgraded with the latest SnapLogic Platform release.
November 2022main18944 Stable

Upgraded with the latest SnapLogic Platform release.

August 2022 main17386 Stable

Upgraded with the latest SnapLogic Platform release.

4.29429patches16809 Latest

Removed the log4j dependency from the ML Core Snaps due to security vulnerabilities.

4.29main15993 StableUpgraded with the latest SnapLogic Platform release.
4.28main14627 Stable

Upgraded with the latest SnapLogic Platform release.

4.27427patches13948 Latest

Fixed an issue with the following Snaps, where a deadlock occurred when data is loaded from both the input views.

4.27main12833 StableUpgraded with the latest SnapLogic Platform release.
4.26main11181 StableUpgraded with the latest SnapLogic Platform release.
4.25main9554 StableUpgraded with the latest SnapLogic Platform release.
4.24main8556 StableUpgraded with the latest SnapLogic Platform release.
4.23main7430 StableUpgraded with the latest SnapLogic Platform release.
4.22main6403 StableUpgraded with the latest SnapLogic Platform release.
4.21snapsmrc542 StableUpgraded with the latest SnapLogic Platform release.
4.20 Patchmlcore8770 Stable

Adds the log4j dependency to the ML Core Snaps to resolve the "Could not initialize class org.apache.log4j.LogManager" error. 

4.20snapsmrc535 StableUpgraded with the latest SnapLogic Platform release.
4.19snapsmrc528 StableUpgraded with the latest SnapLogic Platform release.
4.18snapsmrc523 StableUpgraded with the latest SnapLogic Platform release.
4.17 PatchALL7402 Latest

Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

4.17snapsmrc515
 
Latest
  • New Snap: Introducing the Clustering Snap that performs exploratory data analysis to find hidden patterns or groupings in data.
  • Enhanced the AutoML Snap. You can now:
    • Select algorithms to derive the top models.
    • Input the best model generated by another AutoML Snap from a previous execution.
    • View an interactive HTML report that contains statistics of up to 10 models.
  • Added the Snap Execution field to all Standard-mode Snaps. In some Snaps, this field replaces the existing Execute during preview check box.
4.16snapsmrc508
 
StableNew Snap: Introducing the AutoML Snap, which lets you automate the process of selecting machine learning algorithms and tuning hyperparameters. This Snap gives the best predictive model within the specified time limit.
4.15snapsmrc500
 
Stable
  • New Snap Pack. Perform data modeling operations such as model training, cross-validation, and model-based predictions. Additionally, you can also execute Python scripts remotely. Snaps in this Snap Pack are: 
    • Cross Validator -- Classification
    • Cross Validator -- Regression
    • Predictor -- Classification
    • Predictor -- Regression
    • Remote Python Script
    • Trainer -- Classification
    • Trainer -- Regression
  • Releases the Remote Python Executor account and the Remote Python Executor Dynamic account for the Remote Python Script Snap.