In this article
You can use the REST Get Snap to execute an HTTP Get method on a REST API service endpoint.
None.
Works in Ultra Task Pipelines.
The REST GET Snap Link HTTP response header maybe be malformed compared to other third-party applications like POSTMAN.
View Type | View Format | Number of Views | Examples of Upstream and Downstream Snaps | Description |
---|---|---|---|---|
Input | Document |
|
| Each input document offers details associated with the data required from the target RESTful server. |
Output | Document |
|
| Each output document contains data retrieved from the RESTful web server. If the Snap fails during the operation, it sends an error document to the error view containing the fields error, reason, resolution, and stacktrace. However, you must enable the Error view to view the error document. |
Error | Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:
Learn more about Error handling in Pipelines. |
Field Name | Field Type | Description | |||
---|---|---|---|---|---|
Label*
| String | Specify a unique name for the Snap. | |||
Service URL* Default Value: N/A | String/Expression | Specify the service endpoint URL of REST API. You can provide the URL in one of the following ways:
For example:
For example: For Snaps using AWS Signature V4 accounts, you can use the canonical name (CNAME) for the URI so it's not necessary for the URL to end with amazonaws.com or have the region and service provided in it. However, if you are using the CNAME you must provide it in the AWS Region and Service Name fields in the AWS Signature V4 account. The host name in the CNAME must be equal to the bucket name. For example: New URI with CNAME: Here, the bucket name is The Snap finds the value at the JSON path $.widget.id in the input data and replaces "%s" in the Service URL with the value. You can connect File Reader and JSON Parser Snaps upstream of a REST Put Snap and prepare the following JSON file for the File Reader Snap:
| |||
Process array Default Value: Deselected | Checkbox | Select to produce a stream of documents with one record in each document.
| |||
Trust all certificates Default Value: Deselected | Checkbox | ||||
Follow redirects
| Checkbox | ||||
Query parameters Default Value: N/A | |||||
Query parameter Default Value: N/A | String | The name of the query parameter. | |||
Query parameter value Default Value: N/A | String/Expression | The value that you want to assign to the parameter. | |||
HTTP Header | |||||
Key Default Value: N/A | String/Expression | The name of the HTTP header. | |||
Value Default Value: N/A | String/Expression | The value that you want to assign to the HTTP header. | |||
Response entity type Default Value: DEFAULT | Dropdown list | Select one of the following response entity types you want the Snap to display in the output document.:
| |||
Cookie Policy Default Value: Ignore Cookies | Dropdown list | Select a Cookie Policy from the following options:
| |||
Read timeout Default Value: 900 | Integer | ||||
Connection timeout Default Value: 30 | Integer | ||||
Maximum request attempts Default Value: 5 | Integer | ||||
Retry request interval Default Value: 3 | Integer | Specify the time in seconds to wait before retrying the request. | |||
Retry Policy Default Value: Connection errors | String | Select how you want to handle connection and error responses from the following options:
| |||
Normalize URL | Checkbox |
| |||
Has next Default Value: N/A | String/Expression | Enter the expression that must be evaluated on the output document to true or false, to indicate whether there are more records available. The Snap continues to read the next page until this expression evaluates to false in the new set of output documents.
To see this property in action, see Retrieving Contact Information from HubSpot or Retrieving Contact Information from HubSpot Using Upstream Values, below.
| |||
Next URL Default Value: N/A | String/Expression | Enter the expression that is evaluated on the output document to a URL that the Snap must use to get the next page. This property is used for the REST Get pagination feature. The Next URL has two components, which are structured as follows:
You can create the Next URL in either of the following ways:
For additional information, see Retrieving Contact Information from HubSpot or Retrieving Contact Information from HubSpot Using Upstream Values.
| |||
Pagination interval(s) Default Value: 0 | Integer | The number of seconds for the Snap to wait before attempting to get the next page. | |||
Show all headers Default Value: Deselected | Checkbox | ||||
Snap Execution Default Value: Validate & Execute | String |
When making REST API calls to a specific endpoint using the GET method, there might be a lot of results to return. We use the pagination settings to fetch all the records/documents on multiple pages. For example, if the GET API call returns 20 documents by default, to get all other pages/records, we use Has Next field and get the list of all records until the last record.
The response can provide gzipped data. In that case, the Snap can be configured to process it using the header parameter:
HTTP Header
Key | Value |
---|---|
Accept-Encoding | gzip |
Accept | */* |
The stream is then uncompressed while parsing the data and making it available on the output view.
Error | Reason | Resolution | |
---|---|---|---|
Failed to execute HTTP request. | The Service URL must have a protocol, such as http://, https://. | Check the Snap properties. Specifically, check your entry in the Service URL field. | |
<Service_URL>: Name or service not known. This basically means that the service URL you entered is not accessible. | |||
Failure: Validation errors: property_map.settings.serviceUrl.value: Could not compile expression: <Service_URL> | The snap settings are not correct. Encountered extraneous input ':' at line 1:5; expecting={<EOF>, '||', '&&', '^', '==', '!=', '>', '<', '>=', '<=', '+', '-', '*', '/', '%', '?', 'instanceof', 'in', 'matches', '[', '(', '.', PropertyRef}. This means that you have turned expressions on () while entering string value into the Service URL field. | Fix the snap configuration and try executing again. Please check expression syntax. Ensure that your service URL is a proper expression; or turn off the expression control ().
| |
URL Parse Exception - 403 | The Service URL path might be containing any of the following special characters: | To escape the special characters, use the global function encodeURIComponent on any variables that might contain special characters so that they are encoded properly. | |
Too many Requests - 429 | There are too many requests to REST endpoint. | Wait for the retry to succeed. Rest related Snaps extract the response header and automatically retry when they encounter status 429. By default, the retry interval (Retry-After) is specified in the HTTP response header. If no value is available for Retry-After, then the Snap’s Retry Interval value is used. |
In this example, we use the Has next field to retrieve the list of users created most recently in HubSpot, then separate the user information from the other details in the API response.
Step 1: Configure the REST Get Snap to retrieve user information from HubSpot. Configure the REST Get Snap without the Pagination details - Has Next and Next URL.
Step 2: Validate the Snap. The Snap displays the output with only 20 records and has more value as true indicating that there are more records.
Step 3: Next, configure the Has Next and Next URL fields as follows:
REST Get configuration details: The following configuration details are key to understanding this example:
Field | Value | Description |
---|---|---|
Service URL | https://api.hubapi.com/contacts/v1/lists/all/contacts/all?hapikey=demo | This is a demo API offered by HubSpot, which returns details of all contacts created in the demo account, in descending order. This API only scrolls back 30 days in time. |
Has next | $entity['has-more'] | The REST Get Snap lists 20 documents by default. If there are more contacts created during the 30 days preceding the call, this API returns true for the key has-more. |
Next URL | "https://api.hubapi.com/contacts/v1/lists/all/contacts/all?hapikey=demo&vidOffset=" + $entity['vid-offset'] | This is the URL that the Snap calls to retrieve the next set of 20 documents. In this URL, |
Step 4: Execute the Snap—it lists out API response entities in sets of 20, until the last contact information is listed out, and the has-more
field changes to false:
Step 5: Configure the JSON Splitter Snap. The output from the REST Get Snap contains a lot of details, but for this example we only want user details, which are available in the entity.contacts
array in the API response. Use the JSON Splitter Snap to isolate the information.
The output of the JSON Splitter Snap contains only the information we need:
In this example, we use values from the upstream Snap in the Has next property of the REST Get Snap to retrieve the list of users created most recently in HubSpot.
Step 1: Configure the JSON Generator Snap. Enter the endpoint URL as follows:
[ { "url" : "https://api.hubapi.com/contacts/v1/lists/all/contacts/all?hapikey=demo&vidOffset" } ] |
Execute the Snap—it gets the following output:
Step 2: Configure the REST Get Snap to retrieve user information from HubSpot.
The following configuration details are critical to understand this example:
Field | Value | Description |
---|---|---|
Service URL | https://api.hubapi.com/contacts/v1/lists/all/contacts/all?hapikey=demo | This is a demo API offered by HubSpot, which lists out a few details of their most recently created users in descending order. This API only scrolls back 30 days in time. |
Has next | $entity['has-more'] | The Get all contacts API offers 20 documents by default. If there are more contacts created during the 30 days preceding the call, this API returns true for the key has-more. |
Next URL | $original.url+ "=" + $entity['vid-offset'] | This is the expression that uses the value received from the upstream JSON Generator Snap to create the URL that the Snap calls to retrieve the next set of documents. In this URL, |
Step 2: Execute the Sna. It lists out API response entities in sets of 20, until the last contact information is listed out, and the has-more
field changes to false.
You can use the REST Get Snap to get information from the Pipeline Monitoring API. For example, to return the status of executed pipelines for the specified time within the org Snaplogic, the GET statement of GET https://elastic.snaplogic.com/api/1/rest/public/runtime/snaplogic?start=1430377200000&end=1430420399000 can be passed through the REST Get Snap as shown below:
See the Use Case example Pipeline: Connecting to Rally with REST.
The REST Get Snap in the above example pipeline gets the data for a JIRA issue created and updated in the preceding Snaps in the pipeline.
The following JSON data is the input document to the Snap:
{ "response": { "headers": { "strict-transport-security": "max-age=315360000;includeSubdomains", "x-content-type-options": "nosniff", "x-seraph-loginreason": "OK", "connection": "keep-alive", "set-cookie": "atlassian.xsrf.token=BWCK-SR48-JN93-ESFC|c127207e6dcb6f113eb45ec4d1dc463829af8dc2|lin; Path=/; Secure", "content-type": "application/json;charset=UTF-8", "date": "Thu, 15 Jan 2015 18:49:00 GMT", "x-arequestid": "648x1343x1", "x-asen": "SEN-2848532", "content-length": "0", "x-asessionid": "1g8lldp", "cache-control": "no-cache, no-store, no-transform", "x-ausername": "admin", "server": "nginx" }, "statusLine": { "reasonPhrase": "No Content", "statusCode": 204, "protoVersion": "HTTP/1.1" }, "entity": null }, "original": { "id": "17392", "update": { "comment": [ { "add": { "body": "Bug has been fixed." } } ] } } } |
Click this link to see the output provided by the Snap: Output Data
{ "response": { "headers": { "strict-transport-security": "max-age=315360000;includeSubdomains", "x-content-type-options": "nosniff", "x-seraph-loginreason": "OK", "connection": "keep-alive", "set-cookie": "atlassian.xsrf.token=BWCK-SR48-JN93-ESFC|c127207e6dcb6f113eb45ec4d1dc463829af8dc2|lin; Path=/; Secure", "content-type": "application/json;charset=UTF-8", "date": "Thu, 15 Jan 2015 18:49:00 GMT", "x-arequestid": "648x1343x1", "x-asen": "SEN-2848532", "content-length": "0", "x-asessionid": "1g8lldp", "cache-control": "no-cache, no-store, no-transform", "x-ausername": "admin", "server": "nginx" }, "statusLine": { "reasonPhrase": "No Content", "statusCode": 204, "protoVersion": "HTTP/1.1" }, "entity": null }, "original": { "id": "17392", "update": { "comment": [ { "add": { "body": "Bug has been fixed." } } ] } } } Click this link to see the output provided by the Snap: Output Data { "headers": { "strict-transport-security": "max-age=315360000;includeSubdomains", "x-content-type-options": "nosniff", "vary": "Accept-Encoding", "x-seraph-loginreason": "OK", "connection": "keep-alive", "set-cookie": "atlassian.xsrf.token=BWCK-SR48-JN93-ESFC|1640969cad157d3859fba4971c79e09b38f27b21|lin; Path=/; Secure", "transfer-encoding": "chunked", "content-type": "application/json;charset=UTF-8", "date": "Thu, 15 Jan 2015 18:49:02 GMT", "x-arequestid": "649x1344x1", "x-asen": "SEN-2848532", "x-asessionid": "1nuotcy", "cache-control": "no-cache, no-store, no-transform", "x-ausername": "admin", "server": "nginx" }, "statusLine": { "reasonPhrase": "OK", "statusCode": 200, "protoVersion": "HTTP/1.1" }, "entity": { "expand": "renderedFields,names,schema,transitions,operations,editmeta,changelog", "id": "17392", "self": "https://Snaplogic.atlassian.net/rest/api/2/issue/17392", "key": "JIRASNAP-4312", "fields": { "issuetype": { "self": "https://Snaplogic.atlassian.net/rest/api/2/issuetype/2", "id": "2", "description": "A new feature of the product, which has yet to be developed.", "iconUrl": "https://Snaplogic.atlassian.net/images/icons/issuetypes/newfeature.png", "name": "New Feature", "subtask": false }, "timespent": null, "project": { "self": "https://Snaplogic.atlassian.net/rest/api/2/project/10000", "id": "10000", "key": "JIRASNAP", "name": "Jira Snap for SnapLogic 4.0", "avatarUrls": { "48x48": "https://Snaplogic.atlassian.net/secure/projectavatar?pid=10000&avatarId=10011", "24x24": "https://Snaplogic.atlassian.net/secure/projectavatar?size=small&pid=10000&avatarId=10011", "16x16": "https://Snaplogic.atlassian.net/secure/projectavatar?size=xsmall&pid=10000&avatarId=10011", "32x32": "https://Snaplogic.atlassian.net/secure/projectavatar?size=medium&pid=10000&avatarId=10011" } }, "fixVersions": [], "aggregatetimespent": null, "resolution": null, "customfield_10500": null, "customfield_10501": null, "resolutiondate": null, "workratio": -1, "lastViewed": null, "watches": { "self": "https://Snaplogic.atlassian.net/rest/api/2/issue/JIRASNAP-4312/watchers", "watchCount": 1, "isWatching": true }, "created": "2015-01-15T10:48:57.605-0800", "customfield_10020": null, "customfield_10021": "Not Started", "customfield_10100": null, "priority": { "self": "https://Snaplogic.atlassian.net/rest/api/2/priority/3", "iconUrl": "https://Snaplogic.atlassian.net/images/icons/priorities/major.png", "name": "Major", "id": "3" }, "customfield_10101": null, "customfield_10300": "0|1009kw:", "labels": [], "customfield_10016": null, "customfield_10017": null, "customfield_10018": null, "customfield_10019": null, "timeestimate": null, "aggregatetimeoriginalestimate": null, "versions": [], "issuelinks": [], "assignee": null, "updated": "2015-01-15T10:49:00.553-0800", "status": { "self": "https://Snaplogic.atlassian.net/rest/api/2/status/10001", "description": "", "iconUrl": "https://Snaplogic.atlassian.net/images/icons/statuses/open.png", "name": "To Do", "id": "10001", "statusCategory": { "self": "https://Snaplogic.atlassian.net/rest/api/2/statuscategory/2", "id": 2, "key": "new", "colorName": "blue-gray", "name": "To Do" } }, "components": [], "timeoriginalestimate": null, "description": "Create, update and delete JIRA issues using the JIRA Create, Put and Delete Snaps", "customfield_10012": null, "customfield_10013": null, "customfield_10014": null, "customfield_10015": null, "timetracking": {}, "customfield_10007": null, "customfield_10008": null, "attachment": [], "aggregatetimeestimate": null, "summary": "Test JIRA Snaps (create/update/delete a Jira issue)", "creator": { "self": "https://Snaplogic.atlassian.net/rest/api/2/user?username=admin", "name": "admin", "key": "admin", "emailAddress": "pnarayan@Snaplogic.com", "avatarUrls": { "48x48": "https://Snaplogic.atlassian.net/secure/useravatar?avatarId=10122", "24x24": "https://Snaplogic.atlassian.net/secure/useravatar?size=small&avatarId=10122", "16x16": "https://Snaplogic.atlassian.net/secure/useravatar?size=xsmall&avatarId=10122", "32x32": "https://Snaplogic.atlassian.net/secure/useravatar?size=medium&avatarId=10122" }, "displayName": "Snap admin [Administrator]", "active": true, "timeZone": "America/Los_Angeles" }, "subtasks": [], "reporter": { "self": "https://Snaplogic.atlassian.net/rest/api/2/user?username=admin", "name": "admin", "key": "admin", "emailAddress": "pnarayan@Snaplogic.com", "avatarUrls": { "48x48": "https://Snaplogic.atlassian.net/secure/useravatar?avatarId=10122", "24x24": "https://Snaplogic.atlassian.net/secure/useravatar?size=small&avatarId=10122", "16x16": "https://Snaplogic.atlassian.net/secure/useravatar?size=xsmall&avatarId=10122", "32x32": "https://Snaplogic.atlassian.net/secure/useravatar?size=medium&avatarId=10122" }, "displayName": "Snap admin [Administrator]", "active": true, "timeZone": "America/Los_Angeles" }, "customfield_10000": null, "aggregateprogress": { "progress": 0, "total": 0 }, "customfield_10001": null, "customfield_10002": null, "customfield_10200": null, "customfield_10003": null, "customfield_10400": null, "environment": null, "duedate": null, "progress": { "progress": 0, "total": 0 }, "comment": { "startAt": 0, "maxResults": 1, "total": 1, "comments": [ { "self": "https://Snaplogic.atlassian.net/rest/api/2/issue/17392/comment/15458", "id": "15458", "author": { "self": "https://Snaplogic.atlassian.net/rest/api/2/user?username=admin", "name": "admin", "key": "admin", "emailAddress": "pnarayan@Snaplogic.com", "avatarUrls": { "48x48": "https://Snaplogic.atlassian.net/secure/useravatar?avatarId=10122", "24x24": "https://Snaplogic.atlassian.net/secure/useravatar?size=small&avatarId=10122", "16x16": "https://Snaplogic.atlassian.net/secure/useravatar?size=xsmall&avatarId=10122", "32x32": "https://Snaplogic.atlassian.net/secure/useravatar?size=medium&avatarId=10122" }, "displayName": "Snap admin [Administrator]", "active": true, "timeZone": "America/Los_Angeles" }, "body": "Bug has been fixed.", "updateAuthor": { "self": "https://Snaplogic.atlassian.net/rest/api/2/user?username=admin", "name": "admin", "key": "admin", "emailAddress": "pnarayan@Snaplogic.com", "avatarUrls": { "48x48": "https://Snaplogic.atlassian.net/secure/useravatar?avatarId=10122", "24x24": "https://Snaplogic.atlassian.net/secure/useravatar?size=small&avatarId=10122", "16x16": "https://Snaplogic.atlassian.net/secure/useravatar?size=xsmall&avatarId=10122", "32x32": "https://Snaplogic.atlassian.net/secure/useravatar?size=medium&avatarId=10122" }, "displayName": "Snap admin [Administrator]", "active": true, "timeZone": "America/Los_Angeles" }, "created": "2015-01-15T10:49:00.550-0800", "updated": "2015-01-15T10:49:00.550-0800" } ] }, "votes": { "self": "https://Snaplogic.atlassian.net/rest/api/2/issue/JIRASNAP-4312/votes", "votes": 0, "hasVoted": false }, "worklog": { "startAt": 0, "maxResults": 20, "total": 0, "worklogs": [] } } }, "original": { "response": { "headers": { "strict-transport-security": "max-age=315360000;includeSubdomains", "x-content-type-options": "nosniff", "x-seraph-loginreason": "OK", "connection": "keep-alive", "set-cookie": "atlassian.xsrf.token=BWCK-SR48-JN93-ESFC|c127207e6dcb6f113eb45ec4d1dc463829af8dc2|lin; Path=/; Secure", "content-type": "application/json;charset=UTF-8", "date": "Thu, 15 Jan 2015 18:49:00 GMT", "x-arequestid": "648x1343x1", "x-asen": "SEN-2848532", "content-length": "0", "x-asessionid": "1g8lldp", "cache-control": "no-cache, no-store, no-transform", "x-ausername": "admin", "server": "nginx" }, "statusLine": { "reasonPhrase": "No Content", "statusCode": 204, "protoVersion": "HTTP/1.1" }, "entity": null }, "original": { "id": "17392", "update": { "comment": [ { "add": { "body": "Bug has been fixed." } } ] } } } } |
For Eloqua, set:
Has Next: ($total - ($page*$pageSize)) > 0
Next URL: “/assets/campaigns?count=2&page=” + (resultdoc.page +1)
For Marketo, set:
Has Next: resultdoc.moreResult
Next URL: “/rest/v1/activities.json?nextPageToken=” + resultdoc.nextPageToken + ”&activityTypeIds=1&activityTypeIds=12”
In this Pipeline, we request for a service URL 'https://mockbin.com/bin/f580f547-310a-44da-bec2-e43890eebe26' using the Rest Get Snap, where the REST endpoint returns the HTTP response with multiple cookies under the same header. We connected the JSON Splitter Snap to view the cookies under the header 'set-cookie'.
We enter the JSON Path as $headers['set-cookie'] in the JSON Snap
The successful execution of the Pipeline displays the below output view in which we can see the two cookies, {JSESSIONID:first, Path:/} and {ORA_OD_OSI:second, Path:/} respectively under the header'set-cookie'.
The following example demonstrates how the Snap displays the output when you deselect the Process array checkbox.
Configure the REST Get Snap as follows:
Validate the Snap. The Snap displays the data as-is in the output.
The following example demonstrates how the Snap displays the output when you select the Process array checkbox.
Configure the REST Get Snap as follows:
Validate the Snap. The Snap parses the JSON input and displays the output as follows:
SnapLogic Tips and Tricks: REST Snap Compression Capabilities
REST GET and the SnapLogic Public APIs for Pipeline Executions
Connecting SaaS providers with SnapLogic's OAuth-enabled REST Snaps
SnapLogic Community Articles: