Expression Libraries

Created by Diane Miller
Last updated: Jun 11, 2021 by John BrinckwirthVersion comment

In this Page

Overview

Expression libraries are files (.expr) that contain one or more expressions that can be imported into a Pipeline for use across all expression properties. Libraries can be useful if you have static metadata or common functions that you wish to reference in a Snap's expression properties, but may require a strong familiarity with JavaScript syntax. 

When a Pipeline is executed, the contents of all of the expression library files are evaluated and the results are stored as properties in the lib global variable. Note that the values that make up the libraries are static and cannot be changed during Pipeline execution.

You can create a library file using your favorite text editor to build the expression. You can make an object literal containing all the values and functions to use. 

Example 1

{
    prefix: 'test',
    prefixer: x => this.prefix + x
}

Placing this content in a file named "helpers.expr" and uploading it to your project will make it available for use in Pipelines. 

  • The expression library can have a single top-level object, which in turn can have multiple child objects.
  • Block comments starting with '/*' and ending with '*/' are supported.

To import a library into a Pipeline:

  1. Open the Pipeline Properties dialog in the Designer and find the Expression Libraries table near the bottom. 
  2. Add a row to that table and then click the file browser icon to select the file to import.

After clicking Save, all imported libraries will be available for use through the lib variable in the expression language under the name of the file without the extension. For example, to reference the "prefix" property in the "helpers.expr" library, you would write:

lib.helpers.prefix


And, to call the "prefixer" function, you would write:

lib.helpers.prefixer($.name)


If you would like to explicitly set the name of the library in the lib variable, you can fill out the "As" column in the Expression Libraries table in the Pipeline Properties dialog. 

Example 2

The following expression library can be used to generate the start date and end date of the previous month in multiple formats.

{
    // Convert the given Date-with-time object into a Date with only the year and month
    to_month_date: x => Date.UTC(x.getFullYear(), x.getMonth() - 1),

    // Get the start of the previous month based on the given date or the current time if no parameters are given
    prev_month_start: x => this.to_month_date((x || Date.now()).minusMonths(1)),

    // Get the end time of the previous month based on the given date or the current time if no parameters are given
    prev_month_end: x => this.to_month_date((x || Date.now()).withDayOfMonth(1)).minusSeconds(1),
    prev_month_start_string: x => this.prev_month_start(x).toString().split('T')[0],
    prev_month_end_string: x => this.prev_month_end(x).toString().split('T')[0],
    prev_month_start_epoch: x => this.prev_month_start(x).getTime(),
    prev_month_end_epoch: x => this.prev_month_end(x).getTime()
}


Example 3

In the following example, the spread operator …other can be used to indicate that zero or more parameters can be accepted. As a result, the CONCAT function has two or more parameters. This function returns the concatenation of all parameters.

CONCAT: (x, y, ...other) => other.reduce((accum, curVal) => accum + curVal, x.toString() + y.toString()


The 'this' Variable

You may have noticed in the example expression library listed above that the function references the this variable. When defining an object literal, the this variable is set to the object being constructed so that you can reference other functions and data values. The overlay works like the "merge()" method on objects where existing properties are updated instead of replacing the entire object.

Built-in Variables

The following variables are available for use within the expressions in the library:

  • __path__ - The path to this library as it was listed in the imports for the Pipeline.
  • __name__ - The name of this library as it was derived or given in the imports for the Pipeline.

Overlaying Libraries

The values in one library can be overlaid over another by giving them the same name (that is, setting the "As" column in the import list to the same value). This feature allows you to create a library with default values/functions that can then be specialized by importing other libraries that override the default values.

As an example, if the following two code blocks were in separate files and imported with the name "stuff":

{
  "num": 1,
  "str": "Hello, World"
}


{
  "num": 2
}

The resulting value of library would be as follows since the "num" value in the second file would override the value in the first file:

{
  "num": 2,
  "str": "Hello, World"
}

Example

In this example, we have a set of status updates from certain account types. For example, let us assume the following JSON:

[
    {
        "id": "123792837",
        "status": "Done",
        "type": "Checking"
    },
    {
        "id": "543793437",
        "status": "Done",
        "type": "Savings"
    },
    {
        "id": "136792898",
        "status": "some-error",
        "type": "Home Loan"
    }
]


We want to move this data to another system that uses different values for status and account type, thus implementing a rule that:

  • for attribute "status"
    • if status = "Done" then make the status = "COMPLETED"
    • if status contains "error" then make the status = "ERROR"
  • for attribute "type"
    • if type = "Checking" then make the type = "CHK"
    • if type  = "Savings"  then make the type = "SAV"
    • if type  = "Home Loan"  then make the type = "MOR"

We create an expression library file, demo.expr, that contains the following:

{
	typeMap: {
		Checking: "CHK",
		Savings: "SAV",
		"Home Loan": "MOR"
	},

	convertStatus: x => (x == "Done" ? "COMPLETED" : (x.indexOf("error") != -1 ? "ERROR" : "UNKNOWN"))
}

Where lines 2 to 6 map that attribute type and line 8 sets the status.

The demo.expr file is added to our Pipeline through the Expression Libraries section of the Pipeline Properties.

In a Mapper Snap, we create the following expressions:

  • lib.demo.convertStatus($status) to a Target path of $updated_status. This uses the lib variable to call convertStatus property in the demo library to use on the incoming $status field.
  • lib.demo.typeMap[$type] to a Target path of $updated_type. This uses the lib variable to call typeMap property in the demo library to use on the incoming $type field.

The resulting output from the Mapper looks like the following:

See Also