In this article

Overview

You can use this Snap to ...delete the specified file, group of files, or directory from the supplied path and protocol in the Hadoop Distributed File System (HDFS), Azure Blob File System (ABFS), Windows Azure Storage Blob (WASB) and Azure Data Lake (ADL).

...

Snap Type

The <Snap name> Hadoop Distributed File System (HDFS) Delete Snap is a Read/Write/Parse/Transform/Flow/Formatwrite-type Snap that reads/fetches/writes/parses/executes/transforms/calls/creates… .

Prerequisites

• Valid client ID.

• A valid account with the required permissions.

None.

Support for Ultra Pipelines

...

Supports Ultra Pipelines. 

...

Works in Ultra Pipelines if....

...

Does not support Ultra Pipelines. 

Limitations

None.

Limitations and Known Issues

None.

Snap Views

Snap Settings

...

{"features":["title","icon","rounded","collapsible","border"],"title":"Documenting Fields Based On Data Type/UI Element","titleColor":"#000000","titleColorBG":"#ff5c30","titleSize":"14","titleBold":true,"titleItalic":false,"titleUnderline":false,"lozenge":"Hot stuff","lozengeColor":"#172B4D","lozengeColorBG":"#fff0b3","height":200,"panelPadding":12,"panelColor":"#172B4D","panelColorBG":"#FFFFFF","borderColor":"#ff5c30","borderRadius":3,"borderStyle":"solid","borderWidth":1,"icon":"editor/","iconPrimary":"#FFFFFF","iconSecondary":"#0052CC","newMacro":false}

data
Info
**Delete Before Publishing** Choose from the following sentences to document specific field types. Drop-down lists/Option Buttons (radio buttons): You must list the LoV and describe them if their meaning isn't apparent. In this case, format the LoV in italics, regular font for the LoV's description. In either case, list the LoVs as a bullet list. <State what the option should do in this field>. The available options are: <bullet list of LoVs> Specify the Salesforce API to be used. The available options are:... Select the <category> that you want to use. Available options are... * Option 1<italicized>. <third person singular form of the verb> * Option 2<italicized>. <third person singular form of the verb> Select the API that you want to use. Available options are: Bulk API. Sends the Snap execution request details as a bulk API call. REST API. ... OR Select one of the three following modes in which the Snap executes: * Validate & Execute. Performs limited execution of the Snap and generates a data preview during Pipeline validation, then performs full execution of the Snap (unlimited records) during Pipeline runtime. * Execute only. Performs full execution of the Snap during Pipeline execution without generating preview data. * Disabled. Disables the Snap and all Snaps downstream from it. Check boxes: If selected, <Snap behavior>. If selected, an empty file is written when the incoming document has no data. If selected, <behavior>. If not selected/Otherwise, <behavior> Use "If not selected" if the first sentence is long. If selected, the Snap uses the file path value as is. Otherwise, the Snap uses the file path value in the URL. If selected, an empty file is written when the incoming document has empty data. If there is no incoming document at the input view of the Snap, no file is written regardless of the value of the property. Select to <action> Use this if the behavior is binary. Either this or that, where the converse behavior is apparent/obvious. Select to execute the Pipeline during validation. Text Fields Describe what the user shall specify in this field. Additional details, as applicable, in a separate sentence. Include caveats such as the field being conditionally mandatory, limitations, etc. Enter the name for new account. Specify the account ID to use to log in to the endpoint. Required if IAM Role is selected. Do not use this field if you are using batch processing. Numeric Text Fields Describe what the field represents/contains. Additional details, as applicable, in a separate sentence. Include caveats such as the field being conditionally mandatory, limitations, etc. Include special values that impact the field's behavior as a bullet list. The number of records in a batch. The number of seconds for which you want the Snap to wait between retries. The number of seconds for which the Snap waits between retries. Use the following special values: * 0: Disables batching. * 1: Includes all documents in a single request. Notes in field descriptions Confluence’s new editor does not allow nesting of most macros inside another macro, especially the Note/Alert/Warning/Info (Panel) macros inside a table macro and Excerpt macros inside Expand or Panel Macro+ macros. So, as a workaround use the Footnotes approach as mentioned below: Assign numbers at the Note locations in the form of follow through phrases like See Note 2 below this table. or such. Add your Notes---an appropriate Note/Alert/Warning/Info (Panel) macro---immediately below the macro (for example, Table macro) beginning the content with the corresponding number assigned.

Info

Asterisk ( * ): Indicates a mandatory field. Suggestion icon (): Indicates a list that is dynamically populated based on the configuration. Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic. Add icon ( ): Indicates that you can add fields in the fieldset. Remove icon ( ): Indicates that you can remove fields from the fieldset. Upload icon ( ): Indicates that you can upload files.

...

Field Name

...

Field Type

...

Field Dependency

...

Description

Label*

Default Value: ELT Database Account
Example: ELT RS Account

...

String

...

None.

The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline.

Number of records

Default Value: 
Example:

...

String/Expression

...

Sampling Type is Number of records.

...

Enter the number of records to output.

...

fieldset Name

Specify advanced parameters that you want to include in the request.

...

Field 1*

Default Value: <value> or None.
Example: <value>

...

String

...

Debug mode check box is not selected.

...

Field 2

Default Value: <value> or None.
Example: <value>

...

String

...

Asterisk ( * ): Indicates a mandatory field. Suggestion icon (): Indicates a list that is dynamically populated based on the configuration. Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic. Add icon ( ): Indicates that you can add fields in the fieldset. Remove icon ( ): Indicates that you can remove fields from the fieldset. Upload icon ( ): Indicates that you can upload files.

Field Name	Field Type	Description
Label*   Default Value: HDFS delete Example: Hadoop delete	String	The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.
Directory   Default Value: hdfs://<hostname>:<port>/ Example: hdfs://ec2-54-198-212-134.compute-1.amazonaws.com:8020/user/john/input/	String/Expression/Suggestion	Specify the URL for the HDFS directory. It should start with the HDFS file protocol in the following format: hdfs://<hostname>:<port>/<path to directory>/ wasb:///<container name>/<path to directory>/ wasbs:///<container name>/<path to directory>/ adl://<container name>/<path to directory>/  abfs(s):///filesystem/<path>/ abfs(s)://filesystem@accountname.endpoint/<path> The Directory property is used only in the Suggest operation. When you click the Suggestion icon, the Snap displays a list of subdirectories under the specific directory. It generates the list by applying the value specified in the File Filter property.
File filter   Default Value: * Example: ?	String/Expression	Specify the Glob filter pattern. A file filter is a criteria to include or exclude specific files when processing data in HDFS. Info Use glob patterns to display a list of directories or files when you click the Suggest icon in the Directory or File property. A complete glob pattern is formed by combining the value of the Directory property with the Filter property. If the value of the Directory property does not end with "/", the Snap appends one so that the value of the Filter property is applied to the directory specified by the Directory property. Expand title Glob Pattern Interpretation Rules The following rules are used to interpret glob patterns: The * character matches zero or more characters of a name component without crossing directory boundaries. For example, the *.csv pattern matches a path representing a file name ending in .csv, and *.* matches all file names containing a period.The ** characters match zero or more characters across directories. Therefore, it matches all files or directories in the current directory and its subdirectories. For example, /home/** matches all files and directories in the /home/ directory. The ? character matches exactly one character of a name component. For example, 'foo.?' matches file names that start with 'foo.' and are followed by a single-character extension. The \ character is used to escape characters that would otherwise be interpreted as special characters. For example, the expression \\ matches a single backslash, and \{ matches a left brace. The ! character is used to exclude matching files from the output.  The [ ] characters form a bracket expression that matches a single character of a name component out of a set of characters. For example, '[abc]' matches 'a', 'b', or 'c'. The hyphen (-) may be used to specify a range, so '[a-z]' specifies a range that matches from 'a' to 'z' (inclusive). These forms can be mixed, so '[abce-g]' matches 'a', 'b', 'c', 'e', 'f' or 'g'. If the character after the [ is a !, it is used for negation, so '[!a-c]' matches any character except 'a', 'b', or 'c'. The '*', '?', and '\' characters match within a bracket expression. The '-' character matches itself if it is the first character within the brackets, or the first character after the !, if negating. The '{ }' characters are a group of sub-patterns where the group returns a match if any sub-pattern in the group matches the contents of a target directory. The ',' character is used to separate sub-patterns. Groups cannot be nested. For example, the pattern '*.{csv, json}' matches file names ending with '.csv' or '.json'. Leading dot characters in a file name are treated as regular characters in match operations. For example, the '*' glob pattern matches the file name ".login". All other characters match themselves. Examples: '*.csv' matches all files with a CSV extension in the current directory only. '**.csv' matches all files with a csv extension in the current directory and all its subdirectories. *[!{.pdf,.tmp}] excludes all files with the extension PDF or TMP.
File   Default Value: N/A Example: sample.csv tmp/another.csv $filename	String/Expression/Suggestion	Specify the file name or a relative path to a file under the directory specified in the Directory property. It should not start with a URL separator "/".  The value of the File property depends on the name of the directory specified in the Directory property and the criterion specified in the File filter property.
User Impersonation   Default Value: Deselected	Checkbox	Select this checkbox to enable user impersonation. Hadoop allows you to configure proxy users to access HDFS on behalf of other users; this is called impersonation. When user impersonation is enabled on the Hadoop cluster, any jobs submitted using a proxy are executed with the impersonated user's existing privilege levels rather than those of the superuser associated with the cluster. For more information on user impersonation in this Snap, refer to the section on User Impersonation below.
Delete Directory Default Value: Deselected	Checkbox/Expression	Select this checkbox to enable you to delete all the paths in the specified directory.
Number Of Retries   Default Value: 0 Example: 12	Integer/Expression	Specify the maximum number of attempts to be made to receive a response.  Note The request is terminated if the attempts do not result in a response. Retry operation, which attempts to receive a response, occurs only when the Snap loses the connection with the server.
Retry Interval (seconds)   Default Value: 1 Example: 30	Integer/Expression	Specify the time interval between two successive retry requests. A retry happens only when the previous attempt resulted in an exception.
Snap Execution Default Value: Execute Only Example: Validate & Execute	Dropdown list

N/A

Select one of the following three modes in which the Snap executes: Validate & Execute: Performs limited execution of the Snap

,

and generates a data preview during

Pipeline

pipeline validation. Subsequently, it performs full execution of the Snap (unlimited records) during

Pipeline

pipeline runtime. Execute only: Performs

full

complete execution of the Snap during

Pipeline

pipeline execution without generating preview data. Disabled: Disables the Snap and all Snaps that are downstream from it.

Troubleshooting

Examples

Excluding Fields from the Input Data Stream

We can exclude the unrequired fields from the input data stream by omitting them in the Input schema fieldset. This example demonstrates how we can use the <Snap Name> to achieve this result:

<screenshot of Pipeline/Snap and description>

...

Deleting multiple JSON files from Azure Data Lake Storage

In the given scenario, multiple JSON files with file names containing special characters are created for uploading to the Azure Data Lake Storage.

...

Configure the HDFS Writer Snap with specific details, such as the destination directory where the files should be added in the Azure Data Lake Storage. You can see that the file is written to the Azure Data Lake Storage in the output preview.

Snap configuration	Output preview
Image Added	Image Added

You can delete the same file from the Azure Data Lake Storage with the HDFS delete Snap.

Snap Configuration	Output preview
Image Added	Image Added

Downloads

...

Snap Pack History

...

Insert excerpt

XYZ

Hadoop Snap Pack

XYZ

Hadoop Snap Pack nopanel true

Related Content

• Endpoint Doc Link 1

• Endpoint Doc Link 2

• Endpoint Doc Link 3

• SnapLogic Glossary HDFS Writer

• Hadoop Snap Pack

• Getting Started with SnapLogic

• Snap Support for Ultra Pipelines