In this Article
The pipeline Execution Statistics pop-up provides information on the pipeline status when executed. Likewise, when validating a pipeline, you can view Pipeline Validation Statistics.
You can view Pipeline Execution Statistics from the following locations:
As a pipeline executes, the statistics are updated periodically so that you can monitor its progress. If the execution is successful, the Snaps should transition from Started to Completed.
Snap Statistics tab | Extra Details tab |
---|---|
The Snap Statistics tab summarizes pipeline execution time, including fractional components representing seconds. The Extra Details tab gives a precise timestamp of pipeline start and completion times, aiding in understanding execution duration.
You might notice a difference between the pipeline execution time displayed in the Snap Statistics tab and the Extra Details tab on the pipeline dashboard. For example, the Snap Statistics tab displays pipeline execution completion time as 2.30 minutes, and the Extra Details tab shows completion time from Jan 25 at 1:37:54 am till Jan 25 at 1:40:12 am.
The time displayed in the Snap Statistics tab does not represent minutes and seconds in the traditional sense. Instead, it includes a fractional component representing a portion of a second. For instance, in this example, the 0.30 fraction of the time translates to 30/100*60 = 18 seconds. The actual execution time for the pipeline would be 2 minutes and 18 seconds. This aligns with the completion time indicated in the extra details tab.
This tab displays the overall statistics for each Snap on the left and the statistics for the Snap's input, output, and error views on the right. You can change the order of the Snaps in the table by selecting Dataflow or Alphabetical on the top right. The Alphabetical listing orders statistics by the Snap name, while Dataflow orders statistics by placing adjacent Snaps together in the table (though if two or more Snaps are at the same depth in a pipeline, they are shown in random order because of the algorithms used to sort them).
Selecting Dataflow displays the order of data flow in the current instance of the pipeline, which may differ from the data flow at the time when the pipeline was executed. |
Owners of child pipelines can access runtime logs and Snap statistics of the parent pipelines if one or more of the following is true:
|
If the pipeline fails, you see inline which Snap failed.
If a Snap writes to its error view, the row in the table will be highlighted in red:
Warnings might display for Snaps if a potential problem is detected that should be addressed. For example, pipelines that use a deprecated version of the Join Snap display a warning, as shown in the following image:
pipeline executions with warnings show up in the Dashboard with a warning icon next to their execution status for easy identification:
When you execute a pipeline on the Designer canvas, you can view the states of execution.
When a Snap encounters warnings, the warning icon displays on the Snap, as shown in the following image:
When a Snap encounters errors, the error icon displays on the Snap, as shown in the following image:
The columns for each Snap are as follows:
The Duration bar charts provide an overview of how long the pipeline takes during the processing of each document. The width of the chart represents the overall run time of the Snap and is broken down into three sections with different colors:
Input wait time as green.
Execution time as blue.
Output wait time as purple.
The input time and output time are how long the Snap takes waiting for its neighbor Snaps to do their processing. The execution time is how long the Snap takes to process documents. You can mouse over the Duration bar to see the exact amount of time that the Snap takes waiting or executing.
Each Snap has Views that accept or produce documents to be processed. Each view has its own statistics display for the number of documents processed and the rate at which the documents flow through the view.
The Bytes column populated for Snaps that have binary views and displays the total amount of data that has passed through the view.
The Documents column shows how many structured or binary documents flowed through the view.
Binary views can send multiple documents, for example, a ZipFile Reader Snap will output a separate binary document for each file in the Zip. |
The Rate column is a rough approximation of how quickly the documents flow through the view. If the Snap has binary views, the byte rate is also displayed.
When you mouse over the resource columns in a row, the row highlights and a dropdown will appear to show even more information. The top half of the dropdown has sections which display the average usage of a resource for every document that is processed, a chart showing the last minute of activity, and the total usage of the resource over the full run-time of the Snap. The bottom line shows the version and build information for the Snap class.
The Memory Allocated displays the number of bytes that are allocated by the Snap.
This number does not reflect the amount of memory that was freed, and so it does not represent the peak memory usage of the Snap. Therefore, you should not necessarily rely on using this metric to estimate the required size of a Snaplex node. Rather, this number provides an insight into how much memory had to be allocated to process all of the documents. For example, if the total allocated was 5MB and the Snap processed 32 documents, then Snap allocated roughly 164KB per document. When combined with the other statistics, this number can help you identify the potential causes of performance issues. |
|
Pipeline Parameters
While this tab appears for all pipelines, it only displays information for those pipelines that are run either by a task or by another Snap (such as the pipeline Execute Snap). Values are returned only for those parameters that have Capture selected in the pipeline Properties dialog.
The Extra Details tabs provides basic information about the pipeline and its execution, including details about pipeline type and run policy.
In the following image, the Extra Details tab displays information about an Ultra pipeline Task.
In the following image, the Extra Details tab displays information about a Resumable pipeline.
The Extra Details tab provides additional information about the pipeline run, including:
The Request History for Ultra tasks. These statistics display the last 100 documents processed in the Ultra pipeline.
You can also collect the Ultra history statistics using the pipeline Monitoring API. |
Some statistics in common with those from the Ultra Request report are identical.
Receive Time. The time that the request is received.
Processing Time. The total time for the Ultra pipeline to process the documents associated with this request. For example, if the input document is copied, this time includes all the time needed to process each copy.
Extra Path. The path of the Ultra pipeline task.
State Transition. These statistics display when multiple runs of an Ultra pipeline occur.
State. Current state of the Ultra pipeline.
Timestamp. The time that the most recent Ultra pipeline execution occurred.
Username. The username of the person invoking the Ultra pipeline.