This article describes the procedure for migrating your existing REST pipelines and accounts to the HTTP Client Snap and accounts. Refer to this example pipeline for migrating the REST pipeline and account.
You can use the Parent pipeline to bulk migrate the REST pipelines/accounts to HTTP Client Snap/accounts.
You can use the Parent pipeline to bulk migrate the REST pipelines/accounts to HTTP Client Snap/accounts.
Valid access to the Org and Project space.
Optional. Identify the number of REST pipelines and accounts to be migrated. You can use the Scan pipeline to scan the project paths where the REST pipelines are located.
Save a version of the pipelines containing the REST Snaps. You can use the automated Version creation pipeline to create a version of the given pipeline, if the Action type is “update".
If you need to update the existing pipelines, set the Action type to update in your migration pipeline configuration.
If you need to create new pipelines and accounts, set the Action type to create in your migration pipeline configuration.
Also, create a new project and copy the assets, such as files, to the same project folder.
The migration tool includes the following pipelines:
Scan pipeline: This pipeline scans the specified project path for all the available REST pipelines and accounts and provides both lists.
Version creation pipeline: This pipeline automatically creates versions for all the REST pipelines in a given project. Hence, we recommend that you use this pipeline if you want to update your existing pipelines because it assists you in the migration process by creating backup versions of the affected pipelines.
Child pipelines: Includes the following pipelines for migrating REST accounts and pipelines:
Account migration: REST to HTTP Client Account migration
Pipeline migration: REST to HTTP Client Pipeline migration
Parent pipeline: This pipeline runs the child pipelines to migrate REST pipelines and accounts in bulk to the HTTP Client.
Step 1: Use the Scan pipeline to identify the REST pipeline and account paths by scanning the specified Project path.
Description | Pipeline Overview | Download the pipeline |
|---|---|---|
The SnapLogic Metadata Snaps check for the metadata information, which the pipeline identifies the REST pipelines/accounts and writes their path to the output. Input
Output
|  |
Step 2: Use the Version creation pipeline to automatically create a version of all the REST pipelines in a given project. Therefore, we highly recommend you to use this pipeline before you begin the migration. This ensures that there is a backup of the original pipeline in case you encounter any issues during migration.
For creating a version, the HTTP Client Snap (Creating version) must be configured with the Basic Authentication Account with the user who has sufficient permission to create a version for the specified pipeline.
For creating a version, the HTTP Client Snap (Creating version) must be configured with the Basic Authentication Account with the user who has sufficient permission to create a version for the specified pipeline.
Description | Pipeline Overview | Download the pipeline |
|---|---|---|
This pipeline scans all pipelines in a project, identifies those containing REST snaps, and creates new versions for them. Input
Output
 |  |
Step 3:
a) Use the REST to HTTP Client Account migration pipeline to read and transform the specified REST account into an HTTP Client-supported account.
b) Use the REST to HTTP Client Pipeline migration pipeline to read the metadata and transform it to HTTP Client metadata.
The child pipelines can migrate only one account or one pipeline at a time.
The child pipelines can migrate only one account or one pipeline at a time.
Description | Pipeline Overview | Download the pipeline |
|---|---|---|
Accounts migration pipeline: This pipeline identifies the REST accounts and maps them to the compatible HTTP Client accounts. Learn more about the mapping of the accounts. Input:
Output: Using the Script (Mapping Metadata) Snap the pipeline converts the REST accounts to the compatible HTTP Client accounts in the destination folder. |  | |
REST pipeline migration pipeline: This pipeline identifies the REST Snaps in a given pipeline and maps their properties to those of the HTTP Client Snap. Learn more about mapping the Snap settings. Input:
For the For the
Output:
|  |
Step 4: Use the parent pipeline to migrate the pipelines/accounts to HTTP Client.
Note that this pipeline migrates the pipelines/accounts in bulk.
Note that this pipeline migrates the pipelines/accounts in bulk.
Description | Pipeline Overview | Download the pipeline |
|---|---|---|
This pipeline routes to pipeline flow or account flow based on the migration type specified in the JSON Generator Snap. It scans the project path for REST pipelines and accounts and sends them to the Pipeline Execute Snap, which executes the child pipeline. Input:
If your environment supports a larger pool size, you can increase the size to speed up the migration. If your environment supports a larger pool size, you can increase the size to speed up the migration. Output: The migrated pipelines are saved in Manager, and the errors are routed to the error view, and written to a file. |  |
The errors that occur during this migration are routed to the Error view, and those errors, in turn, are routed to a file in SLDB that can be debugged later. Refer to the Troubleshooting section for more details.
The errors that occur during this migration are routed to the Error view, and those errors, in turn, are routed to a file in SLDB that can be debugged later. Refer to the Troubleshooting section for more details.
We highly recommend that you use the Create Action type when migrating your accounts/pipelines. Because, the Update action has a limitation. When you use the Update action, the account type still displays as REST type in the Manager despite being migrated to HTTP Client, as the Manager doesn’t fetch the account type information from the account metadata. |
The following fields are not available in the HTTP Client Snap:
Batch Size: To work with input documents in a batch using the HTTP Client, refer to this example.
Process array
Normalize URL: The HTTP Client normalizes the URL by default at the backend.
There is no property available to give the upload file Name for both single-file and multi-file uploads; the HTTP Client Snap uses the uploaded file name internally.
There is no property available for the Upload transfer request type, so by default, the HTTP Client Snap calculates content length internally.
Accounts:
As part of the migration, we are copying the active REST account token to the HTTP Client token for all the token-based accounts. As long as the token is valid and the target endpoint contains the valid redirect URI (https://elastic.snaplogic.com/api/1/rest/admin/oauth2callback/apisuite), the accounts will work as expected.
REST In-memory OAuth2 accounts will be migrated to HTTP Client OAuth2 account by considering Header authenticated=true and Auth endpoint=token endpoint URL to avoid validation errors.
The script migrates, Dynamic OAuth2 accounts to Secure Header account in HTTP Client.
Important:
For all OAuth2 accounts, the OAuth2 app in the endpoint, such as Microsoft, must be configured with HTTP Client’s Redirect URI: (https://elastic.snaplogic.com/api/1/rest/admin/oauth2callback/apisuite)
Important:
For all OAuth2 accounts, the OAuth2 app in the endpoint, such as Microsoft, must be configured with HTTP Client’s Redirect URI: (https://elastic.snaplogic.com/api/1/rest/admin/oauth2callback/apisuite)
The HTTP Client Snap writes an output document after evaluating Pagination Properties. Therefore, when snap.out.totalCount is used, it is not evaluated correctly. We have addressed this limitation as part of the script, by updating the snap.out.totalCount to snap.out.totalCount+1 for pagination properties.
For multipart-related file uploads, in the REST Post snap, when an entity is not provided, the Snap constructs the JSON body based on the filename of the selected file. This is automated in the HTTP Client Snap, where the script automatically generates a new row in HTTP Client snap with entity fileName in case of multipart related.
The REST Snap outputs the response as part of the response object, but the structure of the response is different for the HTTP Client. So, this conversion is addressed in the upstream Snap.
View names between REST and HTTP Client have changed. As part of the script, we are making sure that output and error views are connected properly.
In REST Snaps, we encode the query params using +, but in HTTP Client, by default, we use %20. As part of the script, we enabled the Use form encoding for spaces checkbox in the HTTP Client to ensure the same behavior as REST Snaps.
When the Pagination URL is provided, the Page size is set to 0 in the HTTP Client Get request.
When single file upload and multi-file upload are selected and the Upload body type is multipart form-data, two upload requests occur as part of the REST POST snap, but after migration, only one request is sent that includes both single file upload and multi-file upload.
Error | Resolution |
|---|---|
If you encounter a Script error, for example:
| Contact customer support at support@snaplogic.com for assistance. |
Metadata error: Errors while reading the pipeline metadata. | Ensure, the source pipeline is working as expected. |
Pipeline error: Error while creating the Migration pipeline. |
|
If a source pipeline/account contains an * (asterisk), the pipeline might skip migrating that pipeline, as the SnapLogic Metadata Read Snap fails to read that path. | Remove the * (asterisk) from the pipeline name and retry. |
