Record Replay

Owned by Subhajit Sengupta
Last updated: Jun 05, 2024 by Kalpana Malladi
2 min read

On this Page

Snap type:

Transform


Description:

This Snap records the input documents from Triggered or Ultra Task requests and replays them in Designer so that you can work with real preview data. When developing a new triggered or Ultra pipeline, you should create a pipeline with this Snap at the head, create the task, and send a sample request to the task to be recorded. Note that for an Ultra Task, you should disable the task so that the recording can be saved. Now that there is a recording saved, you can continue to develop your pipeline with real data.


Prerequisites:

[None]


Support and limitations:Works in Ultra Pipelines.
Account: 

Accounts are not used with this Snap.


Views:
InputThis Snap has exactly one document input view. It should be used as the unlinked input view in a Triggered or Ultra Task.
OutputThis Snap has exactly one document output view.
ErrorThis Snap has at most one document error view and produces zero or more documents in the view.

Settings

Label


Required. The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Record File Path


The path to the file where the recorded documents are stored.
Example: (with expression toggle (=) on) 'sldb:///' + Snap.instanceId + '.json'
Default value: (with expression toggle (=) on) 'sldb:///%s-%s-%s.json'.sprintf(pipe.label, encodeURIComponent(Snap.label), Snap.instanceId)


Count

Required. The number of documents to record. The last 'Count' documents are kept and written to the 'Record File Path' when the pipeline completes.


Replay-Only

Select this option to just replay recorded documents, not record new documents.
Default value: Not selected


Snap Execution



Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Default Value: Execute only
Example: Validate & Execute

Examples

Generating an HTML Page with Weather Data from a SOAP Service

In this example, this Ultra Pipeline generates an HTML page with weather data from a SOAP service.

 


The pipeline will lookup your location based on your IP address using a rest service and then contact the soap service.  The generated page also includes an input for specifying the postal code manually if you want to query another location.

The Record Replay Snap records the input documents from the feed requests and replays them in Designer so you can work with real Suggest data.

 

The Router Snap validates the request and determines whether the geoIP lookup is needed from the REST Get Snap. A SOAP Snap is used to get the weather data.

 

The XML Generator generates the HTML page, a Mapper Snap does some minor data updates, and a Document to Binary Snap returns the HTML page to the client instead of a JSON-encoded string.

  


 

Downloads

Important steps to successfully reuse Pipelines

  1. Download and import the pipeline into the SnapLogic application.
  2. Configure Snap accounts as applicable.
  3. Provide pipeline parameters as applicable.

  File Modified

File weather.slp

Apr 06, 2017 by Diane Miller

See Also


 Click to view/expand
ReleaseSnap Pack VersionDateType Updates
August 2024main27765 StableUpgraded the org.json.json library from v20090211 to v20240303, which is fully backward compatible.
May 2024437patches26643 Latest
  • Fixed an issue with the Sort Snap that displayed an error when estimating the size of the input document provided by the upstream S3 Browser Snap.
  • Fixed an issue with the Parquet Formatter Snap that was unable to route errors to the error view.
May 2024437patches26453 Latest

  • Added expression support to the Skip lines field in the CSV Parser Snap to enable passing pipeline parameters and upstream values. 

  • Fixed an issue with the XML Parser Snap that caused an error when using the Splitter option in the Snap settings. 

May 2024main26341 Stable
  • Added Parquet Parser and Parquet Formatter Snaps to the Transform Snap Pack:
    • Parquet Parser: Reads the binary Parquet data and writes document data to the output.
    • Parquet Formatter: Reads the document data and writes it to the output in binary Parquet format.

  • Enhanced the JSON Splitter Snap to capture metadata and lineage information from the input document.

February 2024436patches25564 Latest

Fixed an issue with the JSON Formatter Snap that generated incorrect schema.

February 2024436patches25292 Latest

Fixed an out-of-memory error issue with the Aggregate Snap. This Snap no longer performs the presort for the input documents.

If the input documents are unsorted and GROUP-BY fields are used, you must use the Sort Snap upstream of the Aggregate Snap to presort the input document stream and set the Sorted stream field Ascending or Descending to prevent the out-of-memory error. However, if the total size of input documents is expected to be relatively small compared to the available memory, then Sort Snap is not required upstream.

Learn more about presorting unsorted input documents to be processed by the Aggregate Snap.

February 2024main25112 StableUpdated and certified against the current SnapLogic Platform release.
November 2023435patches24802 LatestFixed an issue with the Excel Parser Snap that caused a null pointer exception when the input data was an Excel file that did not contain a StylesTable.
November 2023435patches24481 Latest

Fixed an issue with the Aggregate Snap where the Snap was unable to produce the desired number of output documents when the input was unsorted and the GROUP-BY fields field set was used.

November 2023435patches24094 Latest

Fixed a deserialization issue for a unique function in the Aggregate Snap.

November 2023main23721 StableUpdated and certified against the current SnapLogic Platform release.
August 2023434patches23076 LatestFixed an issue with the Binary to Document Snap where an empty input document with Ignore Empty Stream selected caused the Snap to stop executing.
August 2023434patches23034 Latest
  • Fixed an issue with the Transform Snap Pack that caused an error when the input file was a binary JSON file that contained a string value of more than 20,000,000 characters.

  • Fixed a memory issue with the Aggregate Snap that occurred when using GROUP-BY fields.

August 2023434patches22705 Latest

Fixed an issue with the JSON Splitter Snap that caused the pipeline to terminate with excessive memory usage on the Snaplex node after the 4.33 GA upgrade. The Snap now consumes less memory.

August 2023main22460 StableUpdated and certified against the current SnapLogic Platform release.
May 2023433patches22431 Latest
  • Fixed an issue with the Excel Multi Sheet Formatter Snap that caused it to produce binary output data when there was no input document and Ignore empty stream was selected.
  • Introduced the following new Snaps:

    • GeoJSON Parser: Parses geospatial data from binary data input and outputs the contents as a GeoJSON document downstream.

    • WKT Parser: Parses geospatial data from binary data input and outputs the contents as a WKT (Well Known Text) document downstream.

May 2023433patches21779 Latest

The Decrypt Field and Encrypt Field Snaps now support CTR (Counter mode) for the AES (Advanced Encryption Standard) block cipher algorithm.

May 2023433patches21586 Latest

The Decrypt Field Snap now supports the decryption of various encrypted fields on providing a valid decryption key.

May 2023433patches21461 Latest

The following Transform Snaps include new fields to improve memory management: Aggregate, Group By Fields, Group By N,