In this article

1 Overview
- 1.1 Why use HTTP Client Snap in your pipelines?
- 1.2 Snap Type
- 1.3 Prerequisites
- 1.4 Support for Ultra Pipelines
- 1.5 Limitations
- 1.6 Behavior Change
- 1.7 Known Issues
2 Snap Views
3 Snap Settings
4 Troubleshooting
5 Examples
6 Snap Pack History

Overview

The HTTP Client Snap is an advanced REST Snap that enables you to send HTTP requests to a server and receive corresponding HTTP responses. It integrates all commonly used HTTP methods, including DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT, and TRACE. This Snap allows 50 concurrent connections per route and a total of 500 connections in the pool for all routes.

This Snap uses the Apache HTTP client5 library and strictly follows RFC 3986. According to Section 2.2 of the RFC, all reserved characters in the URI path must be encoded.

Why use HTTP Client Snap in your pipelines?

When integrating APIs and services, you can use either the HTTP Client Snap or the REST Snaps. While both Snaps facilitate HTTP requests, the HTTP Client Snap offers multiple benefits. We highly recommend that you use the HTTP Client Snap instead of the REST Snaps for connecting to REST-based APIs and applications.

Enhanced performance and efficiency

Optimized for faster execution of HTTP requests by supporting efficient bulk processing capabilities.
Asynchronous support for improved throughput and reduced execution time.

Learn more: Example: Send records in a batch using a POST method

Extended protocol and payload support

Supports advanced HTTP methods (for example, PATCH and custom methods).
Better handling of raw HTTP responses and non-standard API interactions with improved support for non-JSON payloads (for example, XML and multipart form-data).

Learn more: Example: Update data in the endpoint using a PATCH method

Greater flexibility and customization

Provides additional control over request construction and modification by supporting a wider range of authentication methods and header configurations.
Enables seamless integration with APIs requiring specific payload formats.

Learn more: Configuring API Suite Accounts

Ideal for comprehensive use cases

Designed for complex integrations that require fine-grained control over HTTP requests.
Designed for scenarios where REST Snaps' predefined configurations are restrictive.
Useful for APIs with non-standard authentication or response structures.

Learn more: HTTP Client Examples

Improved debugging and error handling

Provides detailed logging and troubleshooting options through advanced error-handling and granular retries and timeouts.

Learn more: HTTP Client#Troubleshooting

Pagination support due to its flexibility and configurability

Supports various pagination methods:
- Page-based pagination (page or offset parameters in the URL)
- Cursor-based pagination (next page token or cursor from API response)
- Header-based pagination (pagination information provided in response headers)
Dynamic token extraction: The HTTP Client Snap enables you to dynamically extract the next page token from the API response (headers, body, or even a nested field) and use it in the subsequent request. The REST Snap Pack is more rigid in handling custom pagination mechanisms.
Full control over API calls: You can define the pagination limit, offset, cursor, or token placement directly within the HTTP Client Snap’s request settings. This is useful when working with APIs that require non-standard pagination handling.

Learn more:

Snap Type

The HTTP Client Snap is a Transform-type Snap that transforms the data passed through the HTTP request.

Prerequisites

A valid account with the required permissions.

Support for Ultra Pipelines

Works in Ultra Pipelines.

Limitations

If a Triggered Task Cloud URL is the URI field value in the HTTP Client Snap, then during pipeline validation, repeatedly clicking might cause the pipeline validation to hang. This happens because the HTTP Client Snap is making an API call to trigger the associated pipeline every time the button is clicked.
Workaround: Only click once, then wait for the pipeline validation to stop before clicking the button again.
The HTTP Client Snap hides the Authorization headers in the debug information for Basic Authentication and Digest Authentication Accounts because of security considerations.

If you specify the Authorization headers manually in the Snap or through the Secured Headers Account, then the headers are displayed in the debug information, but the values are hidden as follows:

Behavior Change

HTTP Client Snaps in a pipeline no longer share a connection pool; each Snap now maintains its own dedicated pool. This change ensures that SSL certificates are isolated per Snap, following the best practices recommended by Apache HttpClient 5. We recommend that you configure a separate account for each HTTP Client Snap to avoid connection or certificate conflicts.
With 4.41 GA main31019 Snap Pack version, the HTTP Client Snap that uses reserved characters in the Request API might fail, as HttpClient5 library encodes all the reserved characters as part of RFC 3986.
We recommend that you select the Prevent URL encoding checkbox to prevent encoding of special characters in the URL and run the pipeline.
With the main31019 version, the HTTP Client Snap does not normalize URLs by default. If you need to normalize a URL, you must perform this as a preliminary step (by replacing double slashes with a single slash in the path) for the URL preparation process.
With the 433patches21307 update, pagination evaluation occurs before sending the output to the output view. Therefore, if snap.out.totalCount is part of the Has next or any other pagination properties, the Snap will not include the count in the output corresponding to the request that produced the output document.

Known Issues

This Snap fails with the Name may not be null error when the key-value pair values are empty for the x-www-form-urlencoded Entity Type.
The AWS Sig V4 Account does not support Virtual Private Cloud for the HTTP Client Snap.

Snap Views

Type	Format	Number of Views	Examples of Upstream and Downstream Snaps	Description

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input

Binary or Document

Min: 0
Max: 1

Mapper
File Reader
JSON Parser

Each input document includes one of the following:

Details associated with the data required from the target RESTful server.
Details associated with the data that must be replaced or created in the target RESTful server.
Object ID to be deleted at the REST API service endpoint.

Output

Binary or Document

Min: 0
Max: 1

Mapper
Union
Copy
Join
Filter

Each output document contains one of the following:

Data retrieved from the RESTful web server.
Response data from the REST API service endpoint detailing the status of the request.
Details related to the business objects.

When you include a second output view, the Snap uses this view to output the debug information by capturing the request and response.
The special characters, if any, are encoded by the Snap.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

Stop Pipeline Execution: Stops the current Pipeline execution if the Snap encounters an error.
Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.
Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

When you select the Enable debug checkbox, the information about the executed request and response is written to the error view in case of errors.

Learn more about Error handling in Pipelines.

Snap Settings

Asterisk ( * ): Indicates a mandatory field.
Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.
Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.
Add icon ( ): Indicates that you can add fields in the field set.
Remove icon ( ): Indicates that you can remove fields from the field set.
Upload icon ( ): Indicates that you can upload files.

Field Name			Field Type	Field Dependency	Description

Field Name		Field Type	Field Dependency	Description
Label* Default Value: HTTP Client Example: HTTP Client		String	None.	Specify a unique name for the Snap.
Request Method* Default Value: GET Example: HEAD		String/Expression/Suggestion	None.	Specify a custom method or choose one of the following HTTP request methods: DELETE GET HEAD OPTIONS PATCH POST PUT TRACE
URI* Default Value: N/A Example: https://elastic.snaplogic.com/api/1/rest/ public/runtime/snaplogic? start=1230377200000&end=1320450398700		String/Expression	None.	Specify the service endpoint URL of the REST API. You can provide the URL in one of the following ways: As a JavaScript expression in expression language. For example, `http://coresnapsqa-v4.s3.us-east-2.amazonaws.com/S3Account/'+encodeURIComponent("Special_char_owner_!3)12.json")` As a plain string that you should enclose in double quotes (""). For example, `"https://www.example.com/slm/webservice/v2.0/defect/%s".replace("%s", $.widget.id)` You can use the canonical name (CNAME) for the URI, so it's not necessary for the URL to end with `amazonaws.com` or have the region and service provided in it. However, if you are using the CNAME you must provide it in the AWS Region and Service Name fields in the AWS Signature V4 account. The host name in the CNAME must be equal to the bucket name. For example: Previous URI: `https://jsmith-bucket.nia3.snaplogic.net.s3.us-west-1.amazonaws.com/HttpClientTest/test1` New URI with CNAME: `https://jsmith-sltest.nia3.snaplogic.net/HttpClientTest/test1` Here bucket name is `jsmith-bucket.nia3.snaplogic.net` The Snap finds the value at the JSON path "$.widget.id" in the input data and replaces "%s" in the URL with the value. You can connect the File Reader and JSON Parser Snaps upstream of the HTTP Client Snap and prepare the following JSON file to the File Reader Snap: `[ { "widget": { "id": "111", "name": "foo" } } { "widget": { "id": "555", "name": "bar" } } ] "https://www.example.com/data/v27.0/sobjects/Account"` The URL for this Snap must be valid. If the URL contains any special characters, such as `!, =, %21, $,` and `^`, the Snap displays an exception error. You can escape the special characters (using expression language) using one of the following methods: Use the global function encodeURIComponent on any variables that might contain special characters so that they are encoded properly. For example, `'http://coresnapsqa-v4.s3.us-east-2.amazonaws.com/S3Account/'+encodeURIComponent(_fileName)` If you are aware of the code, use the code directly in the URL field. For example, `http://coresnapsqa-v4.s3.us-east-2.amazonaws.com/S3Account/Special_char_owner_%213%2912.json`; where `<%213%2912>` is the code. We recommend that you use the former (encodeURIComponent) method to escape the special characters.
Query Parameters	Use this field set to add query parameters to be included in the request. This is optional and can either be defined directly in the URL, as separate parameters or both.
	Parameter Name Default Value: N/A Example: id	String/Expression	N/A	Specify the name of the query parameter.
	Parameter Value Default Value:N/A Example: $widget.id	String/Expression	N/A	Specify the value to assign to the parameter.
HTTP Headers	Use this field set to create the HTTP header key-value pairs required for defining the headers for different types of input (JSON, PDF, DOCX, and so on). If you want to specify only content-type headers, you can configure the Multipart Content-Type property instead.
	Header Name Default Name: N/A Example: APIKey content-type	String/Expression/Suggestion	N/A	Specify the name of the HTTP header. Alternatively, select one of the following options from the suggestion list: Content-Type: You must specify the `content-type` in the header for an `OPTIONS` request method. Accept User-Agent Content-Length Authorization Connection Accept-Encoding This Snap accepts default value `/` for the `Accept` header that enables the Snap to handle all forms of response content types.
	Header Value Default Value: N/A Example: uyfikhEugk2iKnJWPvG8Rv8sL73BgM2C application/json	String/Expression	N/A	Specify the value for the HTTP header.
Enable debug Default Value: Selected		Checkbox	N/A	Select this checkbox to include the information about the executed request and response in the output document (under the`_debug` key that helps you with debugging). When you select the Enable debug checkbox, the information about the executed request and response is written to both error and output documents (under the `_debug`key) and it also displays a Lint warning in case of errors. When you deselect the Enable debug checkbox, the history is neither written to error document nor output document and does not display any Lint warning. Using debug mode increases memory usage because of the retention of historical requests. So, for long-running or large input pipelines, we recommend you to deselect the Enable debug checkbox to avoid memory issues.
Use form encoding for spaces Default Value: Deselected		Checkbox	N/A	Select this checkbox to enable the Snap to encode any space characters in query parameters using the plus sign (+). By default, the Snap encodes space characters using percent-encoding (`%20`). For example, when the base HTTP Get URI is: https://httpbin.org/anything/!"$%&'()*+,-.:;<+>?@[\]^_`{\|}~ after encoding, the URI will be: `GET /anything/%21%22%24%25%26%27%28%29%2A%2B%2C-.:%3B%3C%2B%3E?%40%5B%5C%5D%5E_%60%7B%7C%7D~` Selecting this checkbox preserves whatever encoding has already been made. So, if you place a plus symbol in the query portion URL or percent-encodes (`%20`), the Snap preserves the same. Learn more about encoding of query parameters.
Prevent URL encoding Default Value: Deselected		Checkbox	N/A	Select this checkbox to prevent the Snap from automatically URL-encoding of the input URI. Enable this setting to use an already encoded URI. URL encoding encodes the same character in different encodings based on its location within the URL. The specific segments of the URL that apply here are the PATH portion and the QUERY portion: `URI = scheme:[//authority][/path][?query][#fragment]` When you deselect this checkbox, the following are some of the characters that are automatically encoded by the Snap: Better Code Block
HTTP Entity	An HTTP entity is the message body you can send in a request or receive from a response. An entity represents both binary and character content. The POST and PUT methods use entities.
HTTP Entity	Entity Type Default Value: none Example: multipart	Dropdown list	N/A	Choose one of the following entity types to include in the request body. none: Select this option if you do not want to use any `body` in your message. multipart: This option enables you to send (form data) large binary or non-ASCII text to the server. Learn more about uploading a multipart file. x-www-form-urlencoded: This option enables you to send the URL-encoded data to the server. You must specify the key-value pairs to enable the Snap to send the encoded data in the request body to the server. raw: This option enables you to send raw data without modifying it. It supports the following file types: Text Javascript JSON HTML XML binary: This option enables you to send binary data in the response body. It supports files from any external system, such as S3 and WASB. Learn more.
Multipart Subtype Default Value: form-data Example: form-data		String/Expression	Appears on selecting multipart* for Entity Type.*	Specify the MIME subtype for multipart, which specifies the nature of the message parts and their interrelationship.
Multipart Form	Appears on selecting multipart* for Entity Type.* Use this field set to configure the fields required to upload multiple files and text. Learn more about Multipart Upload. By default, the Snap Settings display the fields required to upload a file. The data that is sent to the server using this entity type uses the following encoding pattern: The control names and values are escaped. All space characters are replaced by the ‘+', and reserved characters follow the RFC 3986 notations. An equal symbol, ‘=', separates the key and value, and the key-value pairs use ‘&' as a separator.
	Type Default Value: FILE Example: TEXT	Dropdown list	Appears on selecting multipart* for Entity Type.*	Choose either of the following multipart types to initiate the upload: FILE TEXT
	Key Default Value: N/A Example: file	String/Expression	Appears on selecting multipart* for Entity Type.*	Specify the key for the multipart form-data needed to upload a file. The HTTP Client uses multipart entity to achieve the file upload. The form-data of its multipart entity contains key-value pairs. Upload and multipart file key depend on the service endpoint.