Skip to end of banner
Go to start of banner

Remote Python Script

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 27 Next »

On this Page

In this Section

Overview

The Remote Python Script Snap is a Transform type Snap that enables you to execute a Python script on a Remote Python Executor (RPE). The RPE can be installed on the same node as the JCC or elsewhere. The location of the RPE is specified in the account. See Remote Python Executor Installation for steps on installing the RPE.

The Snap has an additional Input and Output view. You can use the second Input view to pass a script to the Snap, in which case the default script within the Snap is ignored. This is useful in case the script is stored in external locations such as GitHub or SFTP server. The second Output view shows the standard output (STDOUT) redirected from the RPE. 

Input and Output

Expected input

  • First input view: The document upon which the Python script is to be executed. 
  • Second input view: The Python script to be executed. 

Expected output:

  • First output view: The output from the Python script's execution. 
  • Second output view: The STDOUT redirected from the RPE.

Expected upstream Snaps

  • First input view: Any Snap that has a document output view. For example, CSV Generator, or a combination of File Reader and CSV Parser.
  • Second input view: You can use Snaps that read script files from the SLDB or external locations such as GitHub. For example, a combination of File Reader and Binary to Document.

Expected downstream Snaps:

  • First output view: Any Snap that accepts a document input. For example, Mapper, or a combination of JSON Formatter and File Writer.
  • Second output view: Any Snap that accepts a document input. For example, Mapper, or a combination of Document to Binary and File Writer.

Prerequisites

The RPE should be installed and running before using this Snap. See Remote Python Executor Installation for steps to install the RPE.

Configuring Accounts

This Snap uses account references created on the Accounts page of SnapLogic Manager to handle access to this endpoint. See Remote Python Executor Account for information on setting up this type of account.

Configuring Views

Input

This Snap has at most two document input views.
OutputThis Snap has at least one and at most two document output views.
ErrorThis Snap has at most one document error view.

Troubleshooting

None.

Limitations and Known Issues

None.

Modes


Snap Settings


LabelRequired. The name for the Snap. Modify this to be more specific, especially if there are more than one of the same Snap in the pipeline.
Edit Script

Required. The Python script to be executed. Click Edit Script to open the script editor. The editor contains a default Python script, edit it as required or supply another script through the second input view. 

The default script writes the input documents to the $original field in the output. 

Default value: Default Python script as shown below:

Functions in the default script

There are three main functions:

  • snaplogic_init
  • snaplogic_process
  • snaplogic_final 

The first function (snaplogic_init) is executed before consuming input data. The second function (snaplogic_process) is called on each of the incoming documents. The last function (snaplogic_final) is processed after all incoming documents have been consumed by snaplogic_process.

Snap Execution

Select one of the three modes in which the Snap executes. Available options are:
  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.
  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.
  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Examples


Case Conversion

This example demonstrates using a Python script to convert all lower-case characters in an incoming document to upper-case characters.

Download this pipeline.

 Understanding the pipeline

The input to the Remote Pyhon Script Snap is a CSV document generated by the CSV Generator Snap. Below is a preview of the CSV Generator Snap's output:


The Remote Python Script Snap is configured with the following script:


When executed, the script converts all lower-case characters in $text to upper-case characters. This is shown in the output preview of the Remote Python Script Snap below:


Download this pipeline.

Additional Examples

The following use cases demonstrate a few real-world scenarios for using this Snap:

Downloads

  File Modified
You are not logged in. Any changes you make will be marked as anonymous. You may want to Log In if you already have an account.
No files shared here yet.
  • Drag and drop to upload or browse for files
  • Snap Pack History

     Click to view/expand
    Release Snap Pack VersionDateType  Updates
    February 2024main25112 StableUpdated and certified against the current SnapLogic Platform release.
    November 2023main23721Nov 8, 2023StableUpdated and certified against the current SnapLogic Platform release.

    August 2023

    main22460

    Aug 16, 2023

    Stable

    Updated and certified against the current SnapLogic Platform release.

    May 2023433patches21854 Latest

    Fixed an issue with the Cross Validator (Classification) Snap where the native Windows DLL caused the Snaplex to stall

    May 2023433patches21644 Latest

    Improved an error message in the Remote Python Script Snap to explain the reason and resolution for the case where a Python script has errors.

    May 2023

    main21015 

    Stable

    Upgraded with the latest SnapLogic Platform release.

    February 2023main19844 StableUpgraded with the latest SnapLogic Platform release.
    November 2022main18944 Stable

    Upgraded with the latest SnapLogic Platform release.

    August 2022 main17386 Stable

    Upgraded with the latest SnapLogic Platform release.

    4.29429patches16809 Latest

    Removed the log4j dependency from the ML Core Snaps due to security vulnerabilities.

    4.29main15993 StableUpgraded with the latest SnapLogic Platform release.
    4.28main14627 Stable

    Upgraded with the latest SnapLogic Platform release.

    4.27427patches13948 Latest

    Fixed an issue with the following Snaps, where a deadlock occurred when data is loaded from both the input views.

    4.27main12833 StableUpgraded with the latest SnapLogic Platform release.
    4.26main11181 StableUpgraded with the latest SnapLogic Platform release.
    4.25main9554 StableUpgraded with the latest SnapLogic Platform release.
    4.24main8556 StableUpgraded with the latest SnapLogic Platform release.
    4.23main7430 StableUpgraded with the latest SnapLogic Platform release.
    4.22main6403 StableUpgraded with the latest SnapLogic Platform release.
    4.21snapsmrc542 StableUpgraded with the latest SnapLogic Platform release.
    4.20 Patchmlcore8770 Stable

    Adds the log4j dependency to the ML Core Snaps to resolve the "Could not initialize class org.apache.log4j.LogManager" error. 

    4.20snapsmrc535 StableUpgraded with the latest SnapLogic Platform release.
    4.19snapsmrc528 StableUpgraded with the latest SnapLogic Platform release.
    4.18snapsmrc523 StableUpgraded with the latest SnapLogic Platform release.
    4.17 PatchALL7402 Latest

    Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

    4.17snapsmrc515
     
    Latest
    • New Snap: Introducing the Clustering Snap that performs exploratory data analysis to find hidden patterns or groupings in data.
    • Enhanced the AutoML Snap. You can now:
      • Select algorithms to derive the top models.
      • Input the best model generated by another AutoML Snap from a previous execution.
      • View an interactive HTML report that contains statistics of up to 10 models.
    • Added the Snap Execution field to all Standard-mode Snaps. In some Snaps, this field replaces the existing Execute during preview check box.
    4.16snapsmrc508
     
    StableNew Snap: Introducing the AutoML Snap, which lets you automate the process of selecting machine learning algorithms and tuning hyperparameters. This Snap gives the best predictive model within the specified time limit.
    4.15snapsmrc500
     
    Stable
    • New Snap Pack. Perform data modeling operations such as model training, cross-validation, and model-based predictions. Additionally, you can also execute Python scripts remotely. Snaps in this Snap Pack are: 
      • Cross Validator -- Classification
      • Cross Validator -- Regression
      • Predictor -- Classification
      • Predictor -- Regression
      • Remote Python Script
      • Trainer -- Classification
      • Trainer -- Regression
    • Releases the Remote Python Executor account and the Remote Python Executor Dynamic account for the Remote Python Script Snap.
    • No labels