Skip to end of banner
Go to start of banner

Creating Ultra Tasks

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 17 Next »

In this article

Overview

You can create an Ultra Task after you have created the Pipeline in SnapLogic Designer. You can create a Task in Designer or Manager. When you create a Task in Designer, the Pipeline is pre-selected when you open the Create Task dialog. When you create an Ultra Task in Manager, you must select a Pipeline to associate with the Task.

Creating a Task includes the following tasks and topics:

Creating an Ultra Task

  1. Open the Create Task dialog doing one of the two following methods:
    • In Designer, click  to expand the toolbar, and click   to display the drop-down menu, then select Task > Ultra to display the dialog:

      OR
    • In Manager, navigate to the target Project, click  to display the drop-down menu, then select Task > Ultra to display the dialog.


  2. In the dialog, make the following required selections:

    1. Name. Enter a name for the Ultra Pipeline Task. This name is used in the URL when the Pipeline has unconnected inputs/outputs.

      Unless otherwise noted, the names of any asset or project is limited to UTF-8 alphanumeric characters and these punctuation characters !"#$%&'()*+,-.:;<=>?@[\]^_`{|}~.

    2. Pipeline. Select the Pipeline to run from the Pipeline drop-down menu. This Pipeline is triggered when the Ultra Task is called. If applicable, edit any of the parameter values from the Pipeline.

    3. Snaplex. Select the Snaplex on which to run the Pipeline from the Snaplex drop-down menu. The Snaplex must have a FeedMaster associated with it.

  3. Change the optional settings and click Save.
  4. To run the Ultra Task, navigate in Manager to the Project Folder where the Ultra Task resides, and complete the following substeps.

    1. Click the Tasks tab in the Asset Toolbar menu.
    2. Select the target Task and click Enable.
  5. To invoke your Ultra Task, see Invoking Ultra Pipelines as Tasks

Once invoked, the Pipeline is started on the Snaplex using the Ultra run policy specified in the Task. The Ultra Pipeline consumes messages from the FeedMaster and processes the documents. The Pipeline restarts automatically after the Task or the Pipeline itself is modified and saved—including known child Pipelines and accounts that are not computed dynamically using an expression.

You can open the Task Details page for your Task by clicking View Details:

On the Task Details page, you can view Pipeline execution information for any instance by clicking Run Details.

To monitor your Ultra Task, go to the Dashboard Pipeline Wall and monitor the execution of the Task.

Notifications

You can create email notifications based on the status of the Ultra Task. If your Org supports Slack notifications, you can add Slack channels or recipients. 

To add recipients for Task notifications:

  • In Notifications, enter each recipient's email address, separated by commas.

  • If Slack notifications are configured for your Org, then the following fields support dropdown lists containing your Slack channels and users. For each field, you can also type to invoke suggested recipients. 
    • Direct Message(s). Select Slack recipients from the dropdown list. 
    • Channels. Select Slack channels as recipients from the dropdown list.

  • Select the status event for which Notifications are sent from the When Task has selector:
    • Started. The Task is activated.
    • Completed. The document was process.
    • Failed. The Pipeline did not execute.
    • Stopped. The Pipeline has stopped abruptly and the Task is no longer running.

Recipients receive notifications based on the selections in the When Task has field.

Ultra Task Configuration Options 

You can configure the following settings for your Ultra Pipeline Task. Each field has a default value, except where indicated.

Instances per Snaplex

You can specify how many instances of the Pipeline should be running on a Snaplex. For example, to achieve redundancy, you can configure an Ultra instance per document.

The default value is one.

If the Snaplex has five nodes and you ask for ten, it starts two on each node. The algorithm for distributing executions on nodes is based on how many of Ultra Pipelines are running already. Each instance connects to the input queue, and receives notifications of message delivery as they come in. When multiple instances are running, the message is delivered to a single instance.

If the Pipeline/node fails for any reason, then the message is unacknowledged in the queue, so when the consumer is recognized as no longer connected, the message goes back to the queue and is picked up by another instance. In the correct configuration, messages should not be lost.

Alias

For high availability and disaster recovery contexts, you can provide an alias for the Task in the Alias field. When an Ultra in one region fails, a Snaplex in another region can run the Ultra Task by invoking another Task with the Alias. Two Tasks must have the same Alias. The alias must be in the same Project but be deployed to different Snaplex instances.

Bearer Token

The Bearer Token is auto-generated for you to provide as the HTTP Authorization header. Alternatively, you can clear this field and have the Pipeline authenticate requests itself.


Max Failures 

You can indicate the maximum number of consecutive failures in a short period of time by entering a value in the Max Failures field. When this threshold is reached, the Task is automatically disabled. 

The default value is ten document processing failures.

Set to zero if the Task should never be disabled. 

Max In-Flight

You can prevent an excess of documents queued on a single, slow execution.  Once the execution reaches this threshold of documents being processed, requests are distributed to other instances of the Ultra Pipeline. 

Set the Max In-Flight value to the maximum number of documents that can be processed by an instance at any one time. The default value is 200 documents.

This value can be set to one, but doing so affects performance, since a value of one would disable the prefetching of documents. Set to a higher value if Task executions take a long time.

Pipeline Parameters

Pipeline parameters are supported. If you choose a Pipeline that contains Pipeline parameters to run in the Ultra Task, then these Pipeline parameters also appear in the Create Task window, where you can modify them. In the following example, the parameter keys path, full, and table name are displayed as fields with their entries as the corresponding values. 

In the full field, the values contain nested JSON.

Change the Run Policy

When you create an Ultra Task, Ultra is the chosen run policy by default. 

If you change the Run Policy setting to Triggered or Scheduled, this changes the underlying Task type, in which case you must re-configure the Task.


  • No labels