Skip to end of banner
Go to start of banner

Exit

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

Snap type:

Read


Description:

This Snap forces a pipeline to stop with a failed status if it receives more records than the user-defined threshold.


Prerequisites:

[None]


Support and limitations:

Works in Ultra Task Pipelines.


Account: 

Accounts are not used with this Snap.


Views:
InputThis Snap has exactly one document input view.
OutputThis Snap has at most one document output view.
ErrorNo error view is defined for this Snap. Nothing is written to the error view in this Snap. The Snap must let the pipeline fail if it receives more than the number of records required.

Settings

Label


Required. The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Threshold limit



Required. The maximum number of input records to be received by the Snap.
Note: Though it is an expression property, it cannot take values from upstream Snaps. Only direct values or pipeline parameters are allowed.

Example: 10

Default value: 0


Exit error message


The message to be displayed when the error occurs.
Note: This is an expression property. In case the property value is blank, the default value message is shown for a better user experience.

Default value:  Exceeded threshold limit

Page lookup error: page "Anaplan Read" not found.

If you're experiencing issues please see our Troubleshooting Guide.

Page lookup error: page "Anaplan Read" not found.

If you're experiencing issues please see our Troubleshooting Guide.

Examples

Write up to Three Input Documents into SLDB and Reject All Additional Documents

In this example, you have a list of documents, and you want to filter the first six input documents based on the number they contain in a specific field. You want all odd numbers routed into one branch, and all the even numbers, into another. To do so, you create a parent and child Pipeline.

  • The parent Pipeline accepts multiple documents and triggers the child Pipeline for each document.
  • The child Pipeline routes incoming documents into two branches based on their contents. It then accepts only the first three documents in each branch and generates an error for each additional document.

Download these Pipelines

The Child Pipeline

You first design the child Pipeline.

  1. Add a Router Snap to the Pipeline and configure it to route all documents containing even numbers in the $num field into a branch labeled Even, and all documents containing odd numbers into a branch labeled Odd.


  2. Add an Exit Snap to each output view of the Router Snap and configure them both to pass three documents and throw an error when they receive more than three documents. Also, add an output view to each Exit Snap, so you can view the documents that were passed.


  3. Add a JSON Formatter Snap to the output view of each Exit Snap to convert the Exit Snap's output documents into a binary stream of data.

  4. Add a File Writer Snap to the output view of each JSON Formatter Snap and configure each to write out the data received as JSON files in SLDB. 
     

The Parent Pipeline

Design the parent Pipeline to accept multiple documents and trigger one child Pipeline execution for all the input documents, together.

  1. Add a JSON Generator Snap to the Pipeline and configure it send seven documents, in three of which the $num field contains even numbers. In the four remaining documents, $num contains odd numbers.


  2. Add a Pipeline Execute Snap to the Pipeline and configure it to execute the child Pipeline once for the entire collection of documents received from the JSON Generator Snap. To do so, select the Reuse executions to process documents check box. If you do not select this check box, then the child Pipeline will be executed once for each document coming in, and the Exit Snap will always receive only one document and never throw an error. For details on configuring the Pipeline Execute Snap, see Pipeline Execute.


  3. Save and execute the parent Pipeline. 
    1. The Pipeline Execute Snap sends all seven documents together to the child Pipeline for processing.
    2. The Router Snap routes the incoming documents into two branches, Even and Odd.
    3. The Exit Snap attached to the Even branch offers all three documents as output and does not throw any error.
    4. The Exit Snap attached to the Odd branch offers all three documents as output. It also sends an error back to the Pipeline Execute Snap, which triggered the child Pipeline.


    5. The two File Writer Snaps create two documents, Evens.JSON and Odds.JSON, in SLDB, each containing three documents.
       

Download these Pipelines

Downloads

  File Modified
You are not logged in. Any changes you make will be marked as anonymous. You may want to Log In if you already have an account.
No files shared here yet.
  • Drag and drop to upload or browse for files
  • Snap Pack History

     Click to view/expand
    ReleaseSnap Pack VersionDateTypeUpdates
    March 2024436patches25576 LatestFixed an issue with the Pipeline Execute Snap where a condition resulted in the output documents not being read.
    February 2024main25112 StableUpdated and certified against the current SnapLogic Platform release.
    November 2023main23721 StableUpdated and certified against the current SnapLogic Platform release.
    August 2023434patches22576 Latest

    Fixed an issue with the Pipeline Execute Snap that caused a null pointer exception during the retry when the input view was a Binary type. Retry is not supported for a Binary input view with this Snap.

    August 2023

    main22460

     


    Stable

    Updated and certified against the current SnapLogic Platform release.

    May 2023

    433patches22107

     Latest

    Fixed an issue with the Pipeline Execute Snap that caused it to acknowledge the Ultra pipeline message before the message reached the last Snap in the pipeline. Because of the early acknowledgment, the last Snap in the pipeline failed and the response could not be sent back.

    May 2023433patches21744 LatestFixed an issue with the Pipeline Execute Snap where a missing label produced a null pointer exception.
    May 2023

    433patches21196

     Latest

    Fixed an issue with the Pipeline Execute Snap where no input view produced a null pointer exception.

    May 2023433patches21040 LatestProvided a fix to ensure the Data Validator works with "unknown" data types.
    May 2023main21015 StableUpgraded with the latest SnapLogic Platform release.
    February 2023432patches20279 Latest

    The Pipeline Execute Snap now returns errors from child pipelines with a unique identifier.

    February 2023432patches19695 Latest
    Fixed an issue with the Pipeline Execute Snap that was causing Unable to create a deep copy of the object errors.

    As of  we have discovered an issue with this patch affecting the Pipeline Execute Snap. If you have pipelines leveraging the Pipeline Execute Snap, we recommend you NOT use this patch and instead use patch 432patches20279.

    February 2023main19844 StableUpgraded with the latest SnapLogic Platform release.
    November 2022main18944 StableUpgraded with the latest SnapLogic Platform release.
    September 2022430patches18070 LatestThe Pipeline Execute Snap with binary output that is used in a low-latency feed Ultra Pipeline now works as expected. The requests are now acknowledged correctly.
    August 2022main17386 StableUpgraded with the latest SnapLogic Platform release.

    4.29

    main15993

       

    Stable

    Upgraded with the latest SnapLogic Platform release.

    4.28 Patch428patches15026 Latest

    Fixed an issue with HTTP Router Snap where the Snap did not work as intended for a GET request that used a Cloud URL when the HTTP Router was the first Snap in the triggered task Pipeline.

    4.28main14627 StableEnhanced the Filter Snap with the Null-safe access checkbox. Select this checkbox to consider the valid values in the filter expression and remove the missing or empty data instead of treating them as null.
    4.27 Patch427patches13726
     
    Latest

    Enhanced the Filter Snap with the Null-safe access checkbox. Select this checkbox to consider the missing data or empty value as null when accessing the source path.

    4.27main12833
     
    StableEnhanced the Gate Snap with the Output Document Content field that enables you to indicate the count of input documents and the specific (first or last) document to include in the output.
    4.26main11181
     
    Stable

    Added Binary Copy Snap to copy a binary stream to the Snap’s output views. You can use this Snap if you want to send the same information to multiple endpoints.

    4.25 Patch425patches10571
     
    Latest

    Enhanced the performance of Binary Router Snap via more efficient IO utilization.

    4.25main9554
     
    Stable

    Enhanced the HTTP Router Snap to support Ultra Pipelines.

    4.24main8556
     
    Stable

    Added the Ignore empty stream checkbox to the Gate Snap Settings

    4.23main7430
     
    Stable

    Enhances the Data Validator Snap by making the Constraint value field as optional under the Constraints.

    4.22main6403
     
    StableUpgraded with the latest SnapLogic Platform release.
    4.21 Patch                     flow8855-Latest

    Fixes the Binary Router Snap's failure in Ultra Pipelines by preventing a deep-copy (performing a copy of the original objects recursively) of binary data headers.

    4.21snapsmrc542-StableUpgraded with the latest SnapLogic Platform release.
    4.20snapsmrc535-Latest
    • Adds a new Snap, Gate, which enables you to consolidate multiple input streams into a single output document where each input stream is displayed as a label containing the list of records that were received by that input view.
    • The Pipeline Execute Snap now provides the following capabilities:
      • Specify the Snaplex on which to run child Pipelines. You can now select the Snaplex on which to run the child Pipelines through the Execute On and Snaplex Path fields. The default option is executing the child Pipeline on the same Snaplex node as the Parent, but you can also specify a different Snaplex through the drop-down list, which includes the option to manually enter a Snaplex name.
      • Configure retry logic for child Pipeline executions. You can now configure retry logic for child Pipeline executions though the following fields: Number of Retries, Retry Interval, and Timeout fields.
    4.19snaprsmrc528-StableUpgraded with the latest SnapLogic Platform release.
    4.18 Patchflow7743-Latest

    Added a property Validate full JSON path to the Data Validator Snap to fix an issue wherein a field validation failure against a constraint does not produce an error.

    4.18snapsmrc523-StableUpgraded with the latest SnapLogic Platform release.
    4.17 Patchflow7637-Latest

    Fixed an issue with the Data Validator Snap wherein the output fails to report all but one of the violations when there are multiple constraints of the Required type.

    4.17 PatchALL7402-Latest

    Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

    4.17snapsmrc515-Latest

    Added the Snap Execution field to all Standard-mode Snaps. In some Snaps, this field replaces the existing Execute during preview check box.

    4.16snapsmrc508-Latest

    Pipeline Execute Snap: Added the capability to select either document (previously supported) or binary data (new) for your input and output Views.

    4.15 Patchflow6263-Latest

    Improved the error messages to include the source path of the constraint for data validation failures.

    4.15snapsmrc500-StableUpgraded with the latest SnapLogic Platform release.
    4.14snapsmrc490-StableUpgraded with the latest SnapLogic Platform release.
    4.13snapsmrc486-StableUpgraded with the latest SnapLogic Platform release.
    4.12 Patchflow4895-Latest

    Fixed an issue in the Data Validator Snap that made multiple copies of lineage on the error path.

    4.12snapsmrc480-Stable

    Resolved an issue with the Union Snap that caused excessive CPU utilization.

    4.11 Patchflow4461-Latest

    Fixed an issue with the Pipeline Execute Snap that could cause the parent pipeline to freeze when the child pipeline fails.

    4.11 Patchflow4292 -Latest

    Resolved an issue with the Union Snap in Ultra Mode that spikes the CPU utilization to ~30% during startup of 120 instances.

    4.11snapsmrc465-StableUpgraded with the latest SnapLogic Platform release.
    4.10snapsmrc414-StableUpgraded with the latest SnapLogic Platform release.
    4.9.0 Patchflow3094-Latest

    Addressed an issue in Binary Router where the Snap did not wait until all threads were completed to write data to the output views.

    4.9.0 Patchflow3320-Latest

    Data Validator Snap - fixed Type constraint feature.

    4.9snapsmrc405-StableUpgraded with the latest SnapLogic Platform release.
    4.8.0snapsmrc398
    Stable

    Filter: Snap-aware error handling policy enabled for Spark mode. This ensures the error handling specified on the Snap is used.

    4.7.0 Patchflow2598
    Latest

    Addressed a possible race condition when reusing pipelines with a Pipeline Execute Snap.

    4.7.0flow2297
    Stable

    Fixed an issue with Ultra pipeline when invoking, using Pipeline Execute, a child pipeline that contains a non-Ultra compatible Snap.

    4.7.0snapsmrc382
    Stable
    • Recommendation: The Pipeline Execute Snap is meant to eventually replace ForEach and Task Execute. It is suggested you use Pipeline Execute for all new pipelines and update existing pipelines at your earliest convenience.
    • Updated the Pipeline Execute Snap with Execute during preview field.
    4.6.0 Patchflow1909
    Latest

    Resolved an issue with Union Snap slowing down pipelines.

    4.6.0snapsmrc362
    Stable
    • The following Snaps now support error view in Spark mode: Copy, Union.
    • ForEach: The value for the Maximum instances property is limited to a maximum of 100. A value higher than 100 will automatically be limited down to 100 during pipeline execution.
    • Pipeline Execute: Snaplex property added to let you specify the Snaplex on which to run the child pipeline.
    • Resolved an issue in Task Execute Snap that caused an internal server error (status code: 500).
    • Resolved an issue in Pipeline Execute Snap that did not show parameter suggestions when the pipeline is selected from another project.
    • Resolved an issue in Copy Snap that showed errors on the UI while the Javascript was being executed.
    • Resolved an issue in Filter Snap that failed with "could not filter the document" if upstream data for ServiceNow Query.
    4.5.1flow1574
    Latest
    • Resolved an issue with Pipeline Execute not able to load pipeline selected from the pipeline picker property.
    • Enhanced error handling in Pipeline Execute for when Reuse is enabled and the pipeline path is an expression whose result is not constant.
    • Resolved an issue in Pipeline Execute not including the runtime ID and state in error documents.
    4.5.0snapsmrc344
    Stable
    • NEW! Pipeline Execute introduced in this release. This Snap provides a way to flexibly and efficiently execute pipelines and flowing documents into and out of these executions.

    • Resolved an issue in Task Execute Snap to ensure JAVA 7 and JAVA 8 Snaplex give the same output.

    • Resolved an issue with the Router Snap routing to the wrong output when used in Spark mode

    4.4.0

    Stable
    • Resolved an issue with Exit Snap hanging during validation if threshold is not met.

    • Spark support added to the Copy, Filter, Router, and Union Snaps.

    4.3.2

    Latest
    • Resolved an issue with Task Execute where java.math.BigInteger cannot be cast to java.lang.String exception if input document variable was a numeric type.

    • Resolved an issue with Task Execute failing with expression enabled.

    • Resolved an issue with data preview not working in the ForEach Snap.

    • Resolved an issue with Execute during Preview functionality broken in the ForEach Snap.

    • Resolved an issue with ForEach Snap not representing the output schema correctly during preview.

    4.3.1

    Latest

    Resolved an issue with the ForEach poll interval being inconsistent.

    4.3.0

    Stable
    • In Data Validator, the pattern option now works with partial matches.

    • Exit Snap 

      • Error handling improved to handle messages that are non-strings.

      • Exit Snap now supports expressions in the Exit error message property.

    • ForEach Snap now generates output documents when run. Output documents will also be produced if a child pipeline called by the ForEach is manually stopped.

    4.2.2

    Latest
    • Resolved an issue in ForEach where a recursive pipeline in synchronous mode was unable to finish execution.

    • Resolved an issue in Data Validator not working for partial matches.



    August 7, 2015 (2015.25/4.2.1)
    • NEW! Exit Snap introduced in this release. This Snap forces a pipeline to stop with a failed status if it receives more records than the user-defined threshold.

    • Resolved error in ForEach Snap where an exposed error view triggered a pipeline failure.



    June 27, 2015 (2015.22)

    The Router Snap now supports even distribution across all output views if no expressions are defined. With this change, Routes is no longer a required field.



    June 6, 2015 (2015.20)
    • Task Execute should have Execute on preview
    • Directory Browser Snap did not render output data when running from "Windows" Groundplex.


    May 15, 2015

    ForEach: Error was not thrown on validation if the child pipeline was not provided.



    May 2, 2015
    • Binary Router: enhanced error handling
    • Filter: bug fixes
    • Router: bug fixes


    December 20, 2014

    Updated Snaps may include new or changed functionality or be otherwise improved.

    • Data Validator: Type constraint added.
    • Union Snap: Preserve order was removed.


    July/Summer 2014
    • NEW! Binary Router Snap introduced in this release.
    • NEW! Task Execute Snap introduced in this release.


    June 30, 2014
    • Data Validator (Beta Release). This Snap validates incoming documents and their attributes against constraints you define.
    • ForEach: Addressed an issue with parallel execution not working properly.


    April 2014

    ForEach Snap updated to support Snaplex selection and pipeline parameters.



    January 2014

    NEW! ForEach Snap introduced in this release.



    November 2013
    • NEW! Head Snap introduced in this release.
    • NEW! Tail Snap introduced in this release.


    Initial Release (June 2013)
    • Router Snap introduced in this release.
    • Copy Snap introduced in this release.
    • Union Snap introduced in this release.
    • Filter Snap introduced in this release.

    • No labels