Page Comparison

Input 

Binary or Document

• Min: 0

• Max: 1

• Mapper

• File Reader

• JSON Parser

Each input document includes one of the following:

• Details associated with the data required from the target RESTful server.

• Details associated with the data that must be replaced or created in the target RESTful server.

• Object ID to be deleted at the REST API service endpoint.

Output

Binary or Document

• Min: 1

• Max: 1

• Mapper

• Union

• Copy

• Join

• Filter

Each output document contains one of the following:

• Data retrieved from the RESTful web server.

• Response data from the REST API service endpoint detailing the status of the request.

• Details related to the business objects.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

• Stop Pipeline Execution: Stops the current Pipeline execution if the Snap encounters an error.

• Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

• Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Label*

Default Value: HTTP Client
Example: HTTP Client

String

None.

Specify a unique name for the Snap.

Request Method*

Default Value: GET
Example: HEAD

String/Expression/Suggestion

None.

Specify a custom method or choose one of the following HTTP request methods:

• DELETE

• GET

• HEAD

• OPTIONS

• PATCH

• POST

• PUT

• TRACE

URI*

Default Value: N/A
Example: https://elastic.snaplogic.com/api/1/rest/
public/runtime/snaplogic?
start=1230377200000&end=1320450398700

String/Expression

None.

Specify the service endpoint URL of the REST API. You can provide the URL in one of the following ways:

• As a JavaScript expression in expression language. For example, http://coresnapsqa-v4.s3.us-east-2.amazonaws.com/S3Account/'+encodeURIComponent("Special_char_owner_!3)12.json")

• As a plain string that you should enclose in double quotes (""). For example, "https://www.example.com/slm/webservice/v2.0/defect/%s".replace("%s", $.widget.id)

You can use the canonical name (CNAME) for URI so it's not necessary for the URL to end with amazonaws.com or have the region and service provided in it. However, if you are using the CNAME you must provide it in the AWS Region and Service Name fields in the AWS Signature V4 account.

The host name in the CNAME must be equal to the bucket name.
For example:
Previous URI: https://jsmith-bucket.nia3.snaplogic.net.s3.us-west-1.amazonaws.com/HttpClientTest/test1
New URI with CNAME: https://jsmith-sltest.nia3.snaplogic.net/HttpClientTest/test1
Here bucket name is jsmith-bucket.nia3.snaplogic.net

The Snap finds the value at the JSON path "$.widget.id" in the input data and replaces "%s" in the URL with the value. You can connect the File Reader and JSON Parser Snaps upstream of the HTTP Client Snap and prepare the following JSON file to the File Reader Snap:

[
 { "widget": { "id": "111", "name": "foo" } }
 { "widget": { "id": "555", "name": "bar" } }
]
"https://www.example.com/data/v27.0/sobjects/Account"

Query Parameters

Use this field set to add query parameters to be included in the request. This is optional and can either be defined directly in the URL, as separate parameters or both. 

Parameter Name

Default Value: N/A
Example: id

String/Expression

N/A

Specify the name of the query parameter.

Parameter Value

Default Value:N/A
Example: $widget.id

String/Expression

N/A

Specify the value to assign to the parameter.

HTTP Headers

Use this field set to create the HTTP header key-value pairs required for defining the headers for different types of input (JSON, PDF, DOCX, and so on). If you want to specify only content-type headers, you can configure the Multipart Content-Type property instead. 

Header Name

Default Name: N/A
Example:
APIKey
content-type

String/Expression/Suggestion

N/A

Specify the name of the HTTP header. Alternatively, select one of the following options from the suggestion list:

• Content-Type

• Accept

• User-Agent

• Content-Length

• Authorization

• Connection

• Accept-Encoding

Header Value

Default Value: N/A 
Example: uyfikhEugk2iKnJWPvG8Rv8sL73BgM2C
application/json

String/Expression

N/A

Specify the value for the HTTP header.

Enable debug

Default Value: Selected

Checkbox

N/A

Select this checkbox to include the information about the executed request and response in the output document (under the_debug key that helps you with debugging).

• When you select the Enable debug checkbox, the information about the executed request and response is written to both error and output documents (under the _debugkey) and it also displays a Lint warning in case of errors.

• When you deselect the Enable debug checkbox, the history is neither written to error document nor output document and does not display any Lint warning.

Use form encoding for spaces

Default Value: Deselected

Checkbox

N/A

Select this checkbox to enable the Snap to encode any space characters in query parameters using the plus sign (+) instead of the percent-encoding (%20).

On selecting this checkbox, the Snap preserves whatever encoding has already been made. So, if you place a plus symbol in the query portion URL or percent-encodes (%20), the Snap preserves the same.

Learn more about encoding of query parameters.

Prevent URL encoding

Default Value: Deselected

Checkbox

N/A

Select this checkbox to prevent the Snap from automatic URL encoding of the input URI. Enable this setting to use an already encoded URI.

When you deselect this checkbox, the following are some of the characters that are automatically encoded by the Snap:

                              (in PATH segment)   (in QUERY segment)
Character name    Character   URL Encoded value   URL Encoded value
-----------------------------------------------------------------
Space                         %20                 %20 
Exclamation       !           !                   %21
Double Quote      "           %22                 %22
Pound             #           %23                 %23
Dollar Sign       $           $                   %24
Percent           %           %25                 %25
Ampersand         &           &                   %26
Single Quote      '           '                   %27
Left Parenthesis  (           (                   %28
Right Parenthesis )           )                   %29
Asterisk          *           *                   *
Plus              +           +                   %2B
Comma             ,           ,                   %2C
Hyphen            -           -                   -
Period (dot)      .           .                   .
Forward Slash     /           /                   %2F
Colon             :           :                   %3A
Semicolon         ;           ;                   %3B
Left-angle        <           %3C                 %3C
Equals            =           =                   %3D
Right-angle       >           %3E                 %3E
Question Mark     ?           %3F                 %3F 
At Sign           @           @                   %40
Left Bracket      [           %5B                 %5B
Backslash         \           %5C                 %5C
Right Bracket     ]           %5D                 %5D
Caret             ^           %5E                 %5E
Underscore        _           _                   _
Backtick          `           %60                 %60
Left Brace        {           %7B                 %7B
Vertical Bar      |           %7C                 %7C
Right Brace       }           %7D                 %7D
Tilde             ~           ~                   %7E

HTTP Entity

An HTTP entity is the message body you can send in a request or receive from a response. An entity represents both binary and character content. The POST and PUT methods use entities.

Entity Type

Default Value: none
Example: multipart

Dropdown list

N/A

Choose one of the following entity types to include in the request body.

• none: Select this option if you do not want to use any body in your message.

• multipart: This option enables you to send (form data) large binary or non-ASCII text to the server.

• x-www-form-urlencoded: This option enables you to send the URL-encoded data to the server. You must specify the key-value pairs to enable the Snap to send the encoded data in the request body to the server.

• raw: This option enables you to send raw data without modifying it. It supports the following file types:

  • Text

  • Javascript

  • JSON

  • HTML

  • XML

• binary: This option enables you send binary data in the response body.

Multipart Subtype

Default Value: form-data
Example: form-data

String/Expression

Appears on selecting multipart for Entity Type.

Specify the MIME subtype for multipart, which specifies the nature of the message parts and their interrelationship.

Multipart Form

Appears on selecting multipart for Entity Type.

Use this field set to configure the fields required to upload multiple files and text. Learn more about Multipart Upload. By default, the Snap Settings display the fields required to upload a file. The data that is sent to the server using this entity type uses the following encoding pattern:

• The control names and values are escaped. All space characters are replaced by the ‘+', and reserved characters follow the RFC 17.38 notations.

• An equal symbol, ‘=', separates the key and value, and the key-value pairs use ‘&' as a separator.

Type

Default Value: FILE
Example: TEXT

Dropdown list

Appears on selecting multipart for Entity Type.

Choose either of the following multipart types to initiate the upload:

• FILE

• TEXT

Key

Default Value: N/A
Example: file

String/Expression

Appears on selecting multipart for Entity Type.

Specify the key for the multipart form-data needed to upload a file.

The HTTP Client uses multipart entity to achieve the file upload. The form-data of its multipart entity contains key-value pairs. Upload and multipart file key depend on the service endpoint.

Value

Default Value: N/A
Example: 

• FILE: persons.csv

• TEXT: “sldb://file.txt”, “sampletext”

String/Expression

Appears on selecting multipart for Entity Type.

Specify the file or text to be uploaded.

If Multipart Type is FILE, the following are applicable:

• The file protocols supported for file values are 'sldb:///' and 'file:///'.

• If the file path is a relative path, it is considered an SLDB file.

• If this field is not empty, HTTP Entity and Batch size fields are ignored and the Multipart Key field is required.

• If the value is an expression, the input document is used to evaluate the expression. Each input document results in one file upload.

• This field does not support wildcard or glob patterns.

If Multipart Type is a TEXT, then:

• The value should be expression-enabled and enclosed in double quotes.

Content Type

Default Value: N/A
Example: text/csv

String/Expression/Suggestion

Appears on selecting multipart for Entity Type.

Select the content type headers for the data in the body of the multipart HTTP request.

The available options are:

• application/octet-stream

• text/plain

• application/json

• application/xml

• text/csv

• text/html

• image/jpeg

Form

Use this field set to define key-value pairs for the URL-encoded form entity type.

Key

Default Value: N/A
Example: id

String/Expression

Appears on selecting x-www-form-urlencoded for Entity Type.

Specify the key of the form entry.

Value

Default Value: N/A
Example: $widget

String/Expression

Appears on selecting x-www-form-urlencoded for Entity Type.

Specify the value for the key defined above.

Raw Entity

Default Value: $
Example: $

String/Expression

Appears on selecting raw for Entity Type.

Specify the body to send in the request. If the raw entity is from an input document, the document is converted into JSON.

File

Default Value: N/A
Example:

String/Expression

Appears on selecting binary for Entity Type.

Specify or upload the file to send in the Post payload.

Pagination

Properties to define pagination settings. Learn more about Pagination.

Has Next

Default Value: N/A
Example: $entity['has-more']

String/Expression

N/A

Specify the boolean expression to indicate whether there is a next page. Learn more about Has Next in action, Retrieving Contact Information from HubSpot Using Upstream Values .

Total Pages to Fetch

Default Value: N/A
Example: 5

Integer/Expression

N/A

Specify the maximum number of pages to fetch. The Snap stops fetching the next page if either the maximum number of pages is reached or the Has Next condition evaluates to false.

Override URI

Default Value: Deselected

Checkbox

N/A

Select this checkbox to enable the URI to be overridden with the specified Next URI for each pagination request.

Reuse request parameters

Default Value: Deselected

Checkbox

N/A

Select this checkbox to enable the Snap to reuse the specified URI parameters in the pagination requests. You can use the Override Parameters table to overrirde some of them.

Override Headers

Define the headers to be overidden or added in the next pagination request. Headers with the same name are overwritten and new headers are appended. If the value is empty, the header is removed. The header value is evaluated against the response document.

Name

String/Expression/Suggestion

N/A

Specify the name of the header to be updated or added.

Value

String/Expression

N/A

Specify the value of the header to be udpated or added.

Pagination Interval (seconds)

Default Value: 0
Example: 5

Integer

N/A

Specify the interval in seconds between the requests for each page.

Client Settings

Additional settings for the HTTP Client.

Connection

Properties related to the connection to be established.

Cookie Policy

Default Value: RFC Lax
Example: RFC Strict

Dropdown list

N/A

Select a Cookie Policy from the following options:

• Browser Compatibility: This policy is compatible with different servers even if they are not completely standards-compliant. If you have issues when parsing cookies, use this policy.

• Ignore Cookies: This cookie policy ignores all cookies. Use this policy to prevent the HTTP Client from accepting and sending cookies.

• RFC Strict: This policy uses the set-cookie header

• RFC Lax: This policy uses set-cookie and set-cookie2 for parsing.

Trust all certificates

Default Value: Deselected

Checkbox

N/A

Select this checkbox to trust all certificates, such as self-signed certificates.  

Read Timeout (seconds)

Default Value: 900
Example: 60

Integer

N/A

Specify the number of seconds the Snap must wait before terminating the request because of a failure to read from the target service.

Connection Timeout (seconds)

Default Value: 30
Example: 60

Integer

N/A

Specify the number of seconds for which the Snap must wait before aborting the request due to a failure to establish a connection to the target endpoint or service.

Retry

Properties for handling retries.

Retry Condition

Default Value: Connection errors
Example: All errors

Dropdown list

N/A

Select one of the following options to handle connection and error responses:

• Connection errors: Occurs when a client fails to establish a secure connection to the server.

• Connection and auth errors (401/403): Occurs when either the user authorization fails or access to the page/resource is forbidden.

• Connection and client errors (4xx): Occurs when the request for the resource contains bad syntax, or when the resource is not found (404).

• Connection and server errors (5xx): Occurs when the server is unable to complete a request.

• All errors: Applies to all of the above-listed errors.

Maximum Attempts

Default Value: 5
Example: 3

Integer

N/A

Specify the maximum number of attempts the Snap must make to receive a response. If the attempts do not result in a response, the Snap terminates the request.

Retry Interval (seconds)

Default Value: 3
Example: 10

Integer

N/A

Specify the number of seconds the Snap must wait between two successive requests. A retry happens only when the previous attempt resulted in an exception.

Automatically retry on 429 responses

Default Value: Selected

Checkbox

N/A

Select this checkbox to automatically retry the 429 Too Many Requests.

Use Retry-After header for 429

Default Value: Selected

Checkbox

N/A

Select this checkbox to enable the Snap to follow the Retry-After response header (if present) to retry the 429 rersponses.

Redirects

Properties for handling redirections.

Follow redirects

Default Value: Selected

Checkbox

N/A

Select this checkbox to accept the response and redirect the request.

Follow original HTTP method

Default Value: Deselected

Checkbox

Appears on selecting the Follow redirects checkbox.

Select this checkbox to enable the Snap to use the original HTTP method to do the redirect request. Otherwise, the GET method is used.

Maximum Number of Redirects

Default Value: 10
Example: 5

Integer/Expression

N/A

Specify the maximum number of redirects to allow.

Allow relative redirect

Default Value: Selected

Checkbox

N/A

Select this checkbox to determine whether relative redirects should be allowed. The HTTP specification requires the location value to be an absolute URI.

Allow circular redirect

Default Value: Deselected

Checkbox

N/A

Select this checkbox to allow circular redirects to the same location.

Output

Properties that define the display of the output.

Response Type

Default Value: JSON
Example: Text

Dropdown list

Choose one of the following response types:

• JSON

• Text

• Raw

Extract entity as Documents

Default Value: Selected

Checkbox

Appears on selecting JSON for Response Type.

Select this checkbox to extract the response entity as an output document as per the specified JSON path. If the extracted value is a list, it is split.

Write unsuccessful response to Error View

Default Value: Deselected

Checkbox

N/A

Select this checkbox to treat all unsuccessful responses (308, 4xx, and 5xx) as errors and write them to the error view.

Include current cookies

Default Value: Selected

Checkbox

N/A

Select this checkbox to include current cookies in each output document.

Snap Execution

Default Value: Execute only
Example: Validate & Execute

Dropdown list

N/A

Select one of the following three modes in which the Snap executes:

• Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

• Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

• Disabled: Disables the Snap and all Snaps that are downstream from it.

Stack trace error in the Snap:

"Caused by: java.io.EOFException at java.base/java.util.zip.GZIPInputStream.readUByte(Unknown Source)

The HTTP Client Snap sends Accept-Encoding is "gzip, deflate" in the request by default .

Provide Accept-Encoding = * in the HTTP header to prevent the error.

Failed to execute the HTTP request.

The Service URL must have a protocol, such as http://, https://.

Check the Snap properties. Specifically, check your entry in the Service URL field.

<Service_URL>: Name or service not known. This means that the service URL you entered is not accessible.

Failure: Validation errors: property_map.settings.serviceUrl.value: Could not compile expression: <Service_URL>

The Snap settings are not correct. Encountered extraneous input ':' at line 1:5; expecting={<EOF>, '||', '&&', '^', '==', '!=', '>', '<', '>=', '<=', '+', '-', '*', '/', '%', '?', 'instanceof', 'in', 'matches', '[', '(', '.', PropertyRef}.

This means that you turned expressions on () when entering the string value into the Service URL field.

Fix the Snap configuration and try executing again. Check the expression syntax. Ensure that your service URL is a proper expression; or turn off the expression control (). 

Learn more about Using Expressions.

The provided URI is invalid.

Illegal character in schema name.

Verify that the URI and parameters that you provide are syntactically correct.

Status codes

URL Parse Exception - 403

The Service URL path might be containing any of the following special characters: !, =, %, #, $, ^&()_¢äâêîôûñç¡¿ÉÙËǨ°¸ðø©¢¾A+²½µ®§÷¶þ 

To escape the special characters, use the global function encodeURIComponent on any variables that might contain special characters so that they are encoded properly.

status code = 404

Phrase = NOT FOUND, refer to the error_entity field in the error view document for more details.

This means that your service URL does not point to the correct endpoint.

Check the values of Snap properties. Specifically, check your service URL.

status code = 404

Phrase = NOT FOUND, refer to the error_entity field in the error view document for more details.

This means that your service URL does not point to the endpoint that it should.

Check the values of Snap properties. Specifically, check your service URL.

status code = 406

NOT ACCEPTABLE.

This means that the endpoint is not receiving any details from upstream Snaps.

Ensure that you have valid references in the HTTP entity field.

status code = 409

CONFLICT, refer to the error_entity field in the error view document for more details. This happens when you already have the resource that you want to create in the target endpoint.

Verify the values in the Snap configuration.

Too many Requests - 429

There are too many requests to the endpoint.

Wait for the retry to succeed. The HTTP Client Snap extracts the response header and automatically retries when it encounters status 429.

By default, the retry interval (Retry-After) is specified in the HTTP response header. If no value is available for Retry-After, then the Snap’s Retry Interval value is used.