Pipeline: Delete Files in SnapLogic

In this Page

Scenario

I want to use a pipeline to delete a selection of files from a particular project in SnapLogic. These files all start with the same first few characters.

Requirements

Snaps Used

For this scenario, the following Snaps are used:

  • REST Get
  • Structure
  • JSON Splitter
  • Filter
  • REST Delete
  • Union
  • JSON Formatter
  • File Writer

Configuration

  1. Create a new pipeline by clicking the (+) tab.
  2. Add and configure a REST Get Snap.
    • Set the Service URL to the path for your project, such as https://elastic.snaplogic.com/api/1/rest/asset/list/Snaplogic/projects/myproject
      To find the URL for your project:
      1. Go to Manager and select an item under your project.
      2. If using Chrome, right-click on that object and select Inspect Element. The link to that object will give you the path to that file under https://elastic.snaplogic.com, such as /api/1/rest/asset/list/snaplogic/projects/myproject/file.
    • Select Process array.
    • Under Accounts, configure the Snap to use Basic Authentication credentials.
  3. Add and configure a Structure Snap to extract the "entity" from the output of the REST Get Snap.
    • Add a row in the Mapping table as follows:
      Source path: $entity.response_map.entries
      Operation: move
      Target path: $entries
  4. Add and configure a JSON Splitter Snap to split the output into a single document per entry
    • Set JSON path to: $entries
  5. Add and configure a Filter Snap to filter assets starting with the same characters.
    • With the Filter expression toggle (=) selected, set the value to:
      $name.startsWith("filename")
      where filename is the actual start of the files you want to delete.
  6. Add and configure a second Filter Snap to verify you will only be deleting assets of type File from the project.
    • With the Filter expression toggle (=) selected, set the value to:
      $asset_type == "File"
  7. Add and configure a REST Delete Snap to delete the files and route errors to the error view.
    • Set the Service URL to:
      'https://elastic.snaplogic.com/api/1/rest/slfs/Snaplogic/projects/myproject/' + $name
    • In Views, set Error to route error data to error view.
    • Under Accounts, configure the Snap to use the same Basic Authentication credentials.
  8. Add and configure a Union Snap to union the output of the two output flows.
    • In Views, add a second Input view. Set the first input view to catch the error view from the REST Delete Snap and the second view as the success path from the Snap.
    • Leave Preserve Order to Unsorted.
  9. Add and configure a JSON Formatter Snaps.
    • Select Ignore empty stream.
  10. Add and configure a File Writer Snap.
    • Enter a value for File name.
    • Set File action to Overwrite.