Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this article

Table of Contents
maxLevel3

Overview

...

Ultra Tasks define the configuration to run an Ultra Pipeline on a Snaplex. When you activate an Ultra Task, the associated Snaplex starts one or more Pipeline instances that continuously process documents from a Feedmaster or another message queue. If you modify assets related to an active Ultra Task, in some cases, the Snaplex restarts the Pipeline instances automatically.

For high availability and disaster recovery, you can run a Pipeline in multiple regions. To do this, create multiple Ultra Tasks for the same Pipeline using the same alias. Assign each Ultra Task to a Snaplexex in a different region. When an Ultra Pipeline in one region fails, a Snaplex in another region can run the Ultra Task by invoking the Ultra Task with the same alias. Learn more about the deployment architecture for providing high availability and disaster recovery.

You can tune Ultra Task performance by specifying the number of Pipeline instances to run and the maximum number of documents that one instance should process at a time. Also, you can specify how many times a Pipeline can fail before the Snaplex disables the Ultra Task. Configuration Options describes the fields that control these behaviors: Instances (per Snaplex), Max In-Flight, and Maximum Failures.

Instances Per Snaplex

When you create an Ultra Task

...

Creating an Ultra Task includes the following topics:

Creating an Ultra Task

...

, you have the option to load balance processing across nodes. The number of instances controls how many pipelines are started to process the Ultra requests. Since the pipelines are automatically started and are always running, the number of pipelines needs to be balanced. Having too many instances uses up slots, resulting in extra load on the node. Having too few could result in delays for requests if the request volume is higher than running instances.

Manual

As a default option, this selection sets the number of pipeline instances to start, regardless of the number of JCC nodes. Use this setting in Ultra Tasks where the pipeline limits the number of concurrent calls from downstream systems.

For example:

An Ultra Pipeline calls a SOAP service and the service can process five API calls at one time => Set the number of instances to 5 for the Snaplex.

Instances per Node

This selection sets the number of pipeline instances per node. Use this setting during times when the request volume on the Ultra Task is predictably steady and when there are no limits on the number of concurrent requests that the downstream systems can process.

This setting does not guarantee 2 instances per node, but 8 nodes in total. The balancing mechanism is designed to distribute them evenly.

For example:

Setting this to 2 runs two pipeline instances on each JCC node. Setting this to four runs eight instances start.

This setting suits both types of Ultra Pipelines, providing redundancy through scaling.

AutoScaling

Scale the number of pipeline instances based on the number of requests being processed. This option can be used when the request rate is expected to be variable, with low and peak volumes being significantly different.

Info

Always-on Ultra Pipelines do not support AutoScaling because they do not require a FeedMaster.

Prerequisites

  • A Snaplex with one or more Feedmaster nodes.

  • Write permissions for the Project Space.

  • A valid Pipeline.

Workflow

...

Limitation

The names of assets, projects, or project spaces are restricted to UTF-8 alphanumeric characters and certain special characters, excluding the comma.

Creating an Ultra Task

  1. From Designer or Manager, open the Create Task dialog:

    • In Designer with the Pipeline open, in the toolbar, click More Tools. Click Create Task and select Ultra Task:

      Image Added

    • In Manager, from the left navigation, select the Project containing the Pipeline. In the toolbar, click Create and select Task > Ultra Task.

      Image Added

      The Create Task dialog displays:

      Image Added

  2. Edit the required fields:

    1. Name. Enter a name. This name is used in the URL when the Pipeline has unconnected inputs/outputs.

...

    1.  
      The names of assets, projects, or project spaces are limited to UTF-8 alphanumeric characters and these

...

    1. special characters except for comma: !"

...

    1. $%&'()*+

...

    1. -.:;<=

...

    1. >@[

...

    1. ]^_`{|}~.

    2. Pipeline.

...

    1. If not already selected, click the drop-down menu

...

    1. and select the Pipeline to be triggered by the Ultra Task

...

    1. .

...

Once activated, the associated Pipeline is started on the Snaplex using the Ultra run policy that is specified in the Task dialog window. The Ultra Pipeline consumes messages from the FeedMaster and processes input documents. The Pipeline restarts automatically after the Ultra Task or the Pipeline itself is modified and saved—including known child Pipelines and accounts that are not computed dynamically using an expression.

Viewing Ultra Task Details

After you create and enable your Ultra Task, you can view details about the Task.

...

Notifications

...

    1.  

      If you choose a Pipeline that contains parameters, the parameters display in the Create Task dialog, where you can modify them. In the following example, the parameter keys path, full, and table name are displayed, with their entries as the corresponding values. In the full field, the values contain nested JSON.

      Image Added

    2. Snaplex. From the drop-down menu, select a Snaplex that has a Feedmaster. 

  1. Modify the applicable optional fields as described in Configuration Options.

  2. Click Save.

  3. (Optional) To create an API in API Manager, click Save and create API. Learn more about creating APIs from Ultra Tasks.

Configuration Options

Field

Description

Notifications

Enables email notifications based on the status of the Scheduled Task. If your Org supports Slack notifications, you can add Slack channels or recipients.

...

 

To use notifications:

  1. In Notifications, enter each recipient's email address, separated by commas.

  2. If Slack notifications are configured for your Org,

...

  1. the following fields

...

  1. display. You can select from the dropdown list or start typing a name to find it in the list:

    • Direct Message(s). Select

...

    • individuals to receive notifications.

    • Channels. Select

...

    • channels to receive notifications.

  1. Select the status

...

  1. that triggers notifications from the When Task has selector:

    • Started. The Task is activated.

    • Completed. The Task is completed.

    • Failed. The associated Pipeline did not execute.

    • Stopped. The associated Pipeline has stopped abruptly and the Task is no longer running.

...

Recipients receive notifications based on the selections in the When Task has field.

Ultra Task Configuration Options

You can configure the following settings for your Ultra Task. Each field has a default value, except where indicated.

Instances per Snaplex

You can specify how many instances of the Pipeline should be running on a Snaplex. For example, to achieve redundancy, you can configure an Ultra instance per document.

The default value is one instance per Snaplex.

If the Snaplex has five nodes and ten Ultra Task instances are actively running, then the Snaplex starts two instances on each node. The algorithm for distributing executions on nodes is based on how many Ultra Task Pipelines are running already. Each instance connects to the input queue, and receives notifications of message delivery as they come in.

When multiple instances are running, the message is delivered to a single instance. If the Pipeline or node fails for any reason, then the message is unacknowledged in the queue. Consequently, when the consumer is recognized as no longer being connected, the message returns to the queue and is picked up by another instance. In the correct configuration, messages should not be lost.

Alias

...

Number of Instances

Select from the dropdown to scale the running instances for the Ultra Tasks.

Info

 Important

Ensure you have all the recent updates to Ultra enhancements to use the Autoscaling feature.

  • Manual per snaplex: Specifies the number of instances per Snaplex equal to or greater than 1.

    • Default: 1

    • Minimum: 1

    • Maximum: 1000

  • Manual per node: Specifies the number of instances per node, which must be in the range of 1-99 inclusively. The total number of instances is based on available nodes. By default, the instance value is set to 1.

  • Autoscale based on feedmaster queue: (Advanced option) Specifies the minimum and the maximum numbers of instances per Snaplex, which must be in the range of 1-99 inclusively. By default, the instance values are set to 1.

    You can also view details of the Auto-scaling events in the Activity log under the Snaplex tab with information on the previous and current configurations.

Note

This configuration applies only to Response-and-request Ultra Pipeline Tasks. Ultra Auto-scaling setting is not available for Polling Ultra Pipeline Tasks.

Alias

Use an alias to run an Ultra Task in multiple regions for high availability and disaster recovery. Use the same alias in multiple Ultra Tasks that run the same Pipeline. When an Ultra Task in one region fails, a Snaplex in another region can run the

...

Pipeline by invoking the Ultra Task that shares the Alias name. The Ultra Task that shares the Alias must be in the same project folder as the primary Ultra Task, but be deployed to different Snaplex instances.

Bearer Token

...

An automatically generated token for the HTTP Authorization header. If you clear this field,

...

authentication is not required. You can also

...

set the

...

Max Failures 

You can indicate the maximum number of consecutive failures in a short period of time by entering a value in the Max Failures field. When this threshold is reached, the Task is automatically disabled. 

The default value is ten document processing failures.

Set this value to zero if the Task should never be disabled. 

Max In-Flight

...

value yourself. 

Note

To authenticate the application successfully, ensure to remove the special characters from the token when adding or updating the values.

Anchor
max_failures
max_failures
Maximum Failures

Specifies an optional threshold when the pipeline fails to start. If the number of failures reaches the threshold, the Snaplex disables the Ultra Task. The default value is 10. The maximum value is 1000.

Set the value to 0 to remove the limitation on pipeline failures.

During development, the pipeline can be in an invalid state and fail to start multiple times. Therefore, in this scenario, we recommend that you set the value to 0 until the pipeline is ready for production.

Max In-Flight

Specifies a threshold for documents to be processed on a single Pipeline instance. Once the execution reaches the threshold, the Snaplex distributes to other instances of the Ultra Pipeline

...

. This enables you to prevent an excessive number of documents from being queued on a single, slow execution.

Set the Max In-Flight value to the maximum number of documents that can be processed by

...

one instance at

...

a time.

The default value is 200 documents.

...

Default: 200

Minimum: 1

Maximum: 1000

Set this value higher if the Ultra Task executions take a long time.

Pipeline Parameters

You can specify Pipeline parameters. If you choose a Pipeline that contains Pipeline parameters to run in the Ultra Task, then these Pipeline parameters also appear in the Create Task window, where you can modify them. In the following example, the parameter keys path, full, and table name are displayed, with their entries as the corresponding values. 

Image Removed

In the full field, the values contain nested JSON.

A value of 1 disables the prefetching of documents and can negatively affect performance. 

Editing an Ultra Task

  1. Click Manager.

  2. From the left navigation, select the Project Space to which the Ultra Task belongs.

  3. Select the Tasks tab.

  4. Click the Ultra Task name. The Edit Update Task dialog appears.

  5. Edit the Ultra Task and click Save to close the dialog.

Changing the Run Policy

When you create an Ultra Task, Ultra is set as the

...

Run Policy by default.

...

If you change the Run Policy setting to Triggered or Scheduled,

...

 you must re-configure the Task.

Viewing Ultra Task Execution Details

After you create and enable your Ultra Task, you can view execution details.

...

  1. In Manager, navigate to the Ultra Task and select Details from the dropdown menu:

    Image Added
  2. On the Task Details page, view execution statistics by clicking Run Details:

    Image Added

Related content