Suggest

Some Snaps can provide suggestions through introspection. If you want to take data from one source and send it to another, you need a Transform Snap in between to map the data from one schema to the other. Short of looking up the schema by directly logging into each source, use the Transform Snap's Suggest functionality to provide lists of fields that make up the source and target schemas for easier mapping.

Caching helps in using the preview data from the previous Suggest run instead of generating the same data for every run, thereby reducing the time spent to generate preview data during pipeline development. When cache is enabled for the Suggest run, the Snaps that remain unchanged from previous runs do not process any data in the current run. Instead, the Suggest URIs and schema are copied from the cached data. However, any changed downstream Snap connected immediately to the cached upstream Snap reads the preview data of the upstream Snap for its execution. All the downstream Snaps following a changed Snap are also invalidated for using cache and hence are executed afresh.  

However, the cache is not used for Snaps with binary output. Any binary output view stores only 8K of encrypted preview data in the cloud and continues to stream remaining data until the downstream document Snap gets enough data to output 50 documents for its preview. Because the size of data needed to generate 50 documents can be more than just 8K, the preview data for the binary output Snap from a previous run is not useful. In other words, the 8K data from the cache might not necessarily generate 50 documents in the downstream document output Snap. To handle this case, the caching is enabled only for document Snaps. If there is a binary output Snap preceding the document output Snap, and the document output Snap is changed, then the cache is invalidated for the upstream binary output Snap as well. 

Following are the scenarios when the suggest run does not use cache: 

• Retry button is clicked. 
• A different Snaplex is selected. 
• Any references of the pipeline, such as child pipelines or accounts, are changed. 
• Previous runtime is missing for unforeseen reasons. 

Cache for a pipeline is completely deleted from the servers when: 

• The Retry button is clicked. 
• The User logs out.
• A Pipeline is deleted.