Directory Browser

In this article

Overview

  • This Snap browses a given directory path from various sources (such as FTP, FTPS, SFTP, SLDB and S3) and lists the attributes (such as Name, Type, Size, Last Modification Time) of the contents (files or sub-folders).

Important

We plan to introduce additional S3 features exclusively in Amazon S3 Snaps, while Binary Snaps with S3 support will not contain these updates. Therefore, we recommend you to use the Amazon S3 Snap Pack for all your S3 operations within your pipelines. However, Binary Snaps will be retained as is to maintain backward compatibility, but be aware that we will no longer provide S3 support for the Binary Snaps.

Learn more: Migration from Binary Snaps to Amazon S3 Snaps.

Snap Type

The Directory Browser Snap is a Read-type Snap.

Prerequisites

IAM Roles for Amazon EC2

The 'IAM_CREDENTIAL_FOR_S3' feature is used to access S3 files from EC2 Groundplex, without Access-key ID and Secret key in the AWS S3 account in the Snap. The IAM credential stored in the EC2 metadata is used to gain access rights to the S3 buckets. To enable this feature, set the Global properties (Key-Value parameters) and restart the JCC:
jcc.jvm_options = -DIAM_CREDENTIAL_FOR_S3=TRUE

This feature is supported in the EC2-type Groundplex only. Learn more.  


Connect to FTP server:

To connect to the FTP server that needs to reuse the session for data transfer over TLS protocol, add:

-DFTPS_SSL_TLS_PROTOCOL=TLSV1.2  (or) TLSV1.3property as a JVM option under the Global properties of the Node Properties tab:

Support for Ultra Pipelines 

Works in Ultra Pipelines

Limitations

None.

Known Issues

  • This Snap Pack does not natively support SHA1-based algorithms to connect to SFTP endpoints. With the August 2023 GA release, you can now leverage the properties specified in the Configuration settings for Snaps to add support for ones that are disabled on your Snaplex. 

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

  • Min: 0

  • Max: 1

  • Mapper

  • S3 Copy

Directory Path to be browsed and File Filter Pattern to be applied. It contains values for the directory path to be browsed and the GLOB filter to be applied to select the contents.

Output

Document


  • Min: 1

  • Max: 1

  • CSV Formatter

  • File Writer

The attributes (name, size, and so on) of the contents of the directory path matching the filter pattern. This Snap provides the various attributes (such as Name, Type, Size, Owner, Last Modification Time) of the contents of the given directory path.  Only those contents are selected that match the given GLOB filter.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop Pipeline Execution: Stops the current pipeline execution when the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk (*): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon (  ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon (): Indicates that you can add fields in the fieldset.

  • Remove icon (): Indicates that you can remove fields from the fieldset.

  • Upload icon (): Indicates that you can upload a file.

Field

Field Type

Description

Label*


Default Value: Directory Browser
ExampleDirectory Browser

String

Specify a unique name for the Snap.

Directory 

Default Value: N/A

Examples

  • Read the file from a location of the storage: adl://storename/folder/filename 

  • If the Snap is executed in the Windows Groundplex and needs to access D: drive): file:///D:/testFolder/ 

  • ftp://ftp.mock.com/vehicles

  • If the bucket name is 'testBucket': gs:///testBucket/testDir/ 

  • project-name

  • s3:///<S3_bucket_name>@s3.<region_name>.amazonaws.com/<path>

  • File in VPC: s3:///snaplogic-bucket@bucket.vpce-0e340d8d5b0eae90a-k9wbom46-us-west-2a.s3.us-west2.vpce.amazonaws.com

  • sftp://sftp.mock.com/students/grades/

  • share

  • sldb:///shared

  • sldb:///project-name

  • If the target folder is /share/folder in server: smb://server/share/folder/ 

  • If the name of the container is 'Snaplogic': wasb:///Snaplogic/testDir/
    or wasbs:///Snaplogic/testDir/ 

String/Expression

Specify the URL for the data source (directory) that should begin with a protocol. Following are the protocols that are currently supported:

  • sldb:

  • s3:

  • ftp:

  • ftps:

  • sftp:

  • hdfs:

  • wasb:

  • wasbs:

  • smb:

  • gs:

  • adl:

  • file:

File filter

Default Value: N/A
Example:

  • *.txt

  • ab????xx.*x

  • *.[jJ][sS][oO][nN]

String/Expression

Specify the Glob pattern for filtering the files. The Glob pattern can be applied to select the contents (files/sub-folders) of the directory. You cannot recursively navigate through the directory structures.

Ignore empty result

Default Value: Selected

Checkbox

Select this checkbox to enable the Snap to not write a document to the output view when the result is empty.

Deselect this checkbox in the following cases:

  • If the Snap receives an input document, the input document is passed through to the output view.

  • If the Snap does not receive an input document, an empty document is written to the output view.

Display update time and owner for S3 subdirectories


Default Value: Selected

Checkbox

Applicable only for S3 directory.

If selected, the Snap produces Update time and Owner attributes in the output documents for S3 subdirectories. However, it may take a long time or even freeze if there are very large number of files and subdirectories in the file structure under the specified S3 directory. This is because the Snap has to completely examine the S3 objects under the directory recursively in order to get Update time and Owner attributes for subdirectories.

If not selected, the Snap does not produce Update time and Owner attributes in the output documents for S3 subdirectories. Snap output is faster since it examines only S3 files and subdirectories immediately under the specified S3 directory.

Number of retries


Default Value: 0
Example:  3

String/Expression

Specify the maximum number of retry attempts in case of a network failure.

Minimum value: 0

Retry interval (seconds)


Default Value:1
Example: 3

String/Expression

Specify the minimum number of seconds the Snap must wait before attempting recovery from a network failure.

Minimum value: 1

Advanced properties

Use this field set to define specific settings for directory browsing. 

Properties

SAS URI

Dropdown list



Specify the URI of the Shared Access Storage (SAS) you need to access. You can generate the SAS URI either from the SAS Generator Snap or from the Azure portal → Shared access signature.

The supported SAS types are:

  • Service SAS on a container
  • Service SAS on blob
  • Ensure that the URI is specified in the format described here.
  • If you provide SAS URI in this field, then:
    • the Primary access key given in the account settings is overridden while authentication. If you do not provide the SAS URI, the Snap considers the Primary access key in the account settings.
    • only this URL is used and the Snap ignores the SAS URI settings that you have configured in the associated account.


Default value
: N/A

Example: https://myaccount.blob.core.windows.net/sascontainer/sasblob.txt?sv=2015-04-05&st=2015-04-
29T22%3A18%3A26Z&se=2015-04-30T02%3A23%3A26Z&sr=b&sp=rw&sip=168.1.5.60
-168.1.5.70&spr=https&sig=Z%2FRHIX5Xcg0Mq2rqI3OlWTjEg2tYkboXr1P9ZUXDtkk%3D

Use fast file-type check for wasb/wasbs protocols

String/Expression

Specifies whether the Snap can use a fast file-type check while browsing through a given file/directory path (for WASB and WASBS file protocols).

Set this field of the Snap to true to save execution times when working with WASB and WASBS file protocols.

Default value: N/A

Snap execution results may not be accurate when using this field. Set this field to true if longer execution times are not a constraint.


Snap Execution


Default ValueValidate & Execute
Example: Execute only


Dropdown list

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.



 Click to view/expand


Release Snap Pack VersionDateTypeUpdates
August 2024438patches28197 Latest
  • Fixed an issue in the Binary Snap Pack that caused intermittent errors on the Groundplex nodes when using the SMB protocol. The errors occurred when multiple IPs were returned for the same address, and one of the IPs was unreachable by the SMB server
  • Fixed an issue in the Directory Browser Snap where a mount error was displayed if the hostname contained an underscore, resulting in an empty hostname in the SMB URL.

August 2024

main27765

 

Stable

Upgraded the org.json.json library from v20090211 to v20240303, which is fully backward compatible.

May 2024437patches27146 LatestEnhanced the Azure Storage Account with Managed Identity, which provides applications with an automatically managed identity for connecting to resources that support Microsoft Entra ID authentication.
May 2024437patches26873 LatestFixed an issue with the File Poller Snap where the Snap applied a case-sensitive filter in the Windows operating system.
May 2024437patches26592 Latest

Enhanced the Decompress Snap to support encrypted and unencrypted ZIP and 7z files through the new File Password Account type.

May 2024main26341 Stable

The Azure Data Lake Account has been removed from the Binary Snap Pack because Microsoft retired the Azure Data Lake Storage Gen1 protocol on February 29, 2024. We recommend replacing your existing Azure Data Lake Account with other Azure Accounts.

February 2024436patches25711 Latest

Fixed an SMB (server message block) connectivity issue within the Binary Snap Pack, as the incorrect name provided by the SMB client did not match the Windows cluster virtual name, affecting the SPN (service principal name) connection.

February 2024436patches25332 Latest

Fixed a null pointer exception in the Public Key Account for Binary Snap Pack when the Public Key field value is null. Now, the account displays a configuration exception for a null value.

February 2024436patches25241 Latest

Fixed an issue with the File Writer Snap, which partially writes the file for some FTPS servers by pausing for seconds before closing the output stream as specified in the global property ftpsDelayBeforeClosing.

February 2024436patches25161 LatestFixed an issue with the File Poller Snap that displayed an exception when an unauthorized character ':' was used in the Windows Snaplex.
February 2024main25112 StableUpdated and certified against the current SnapLogic Platform release.
November 2023435patches24525 LatestFixed an issue with the File Poller Snap that caused it to poll recursively for files in the root directory of the S3 bucket.
November 2023435patches23780 Latest

Fixed an issue with the Binary Snaps that failed to build a data connection when connecting to the FTP server over FTPS protocol.

November 2023main23721 Stable

Updated and certified against the current SnapLogic Platform release.

August 2023434patches23502 LatestFixed an issue with the File Operation Snap (SFTP protocol) where the error message did not display during a move operation, although the existing file was available in the target path and the Error if exists checkbox was selected.
August 2023434patches23302 Latest

Fixed an issue that occurred when node properties were used to override default algorithm specifications for SFTP operations.

August 2023434patches22976 LatestFixed an issue that caused account credentials to be visible in the stack trace of some failed SFTP operations.
August 2023434patches22842 Latest

Fixed an issue that caused NTLM authentication issues when trying to access SMB servers.

August 2023434patches22639 Latest

The JSON key field in the Binary Google Service Account supports JSON strings. You can upload the JSON key either from SLDB or dynamically pass the value using a pipeline parameter or access values from Secrets Manager.

August 2023

main22460

 


Stable

Updated and certified against the current SnapLogic Platform release.

May 2023433patches22297 Latest
  • Fixed an issue with the PGP Sign Snap that caused an error when using an encryption subkey for signing.
  • Fixed an issue with the File Writer Snap where the file was unable to validate when the File action field was set to IGNORE for the WASB and WASBs protocols.
May 2023

433patches21913

 Latest
  • Dynamic ports are supported for the SMB file protocol.
  • Fixed an issue with the File Writer Snap that caused slow performance when writing large files and the Flush interval was set to a positive value.
May 2023433patches21870 LatestAdded the PGP Sign Snap, which allows binary data to be signed using PGP.
May 2023

433patches21645

 Latest

With the 4.33 GA release, support for some algorithms for SFTP connection negotiation is removed for improved security and because we’ve updated the library used to connect to SFTP sources. With Snap Pack version 433patches21645, you can modify the global properties. Refer to the Configuration Settings for Snaps documentation for details about how to revert to the previous settings.

May 2023433patches21576 Latest

Fixed the issues with the AES Encrypt and AES Decrypt Snaps, where the Snaps previously did not include the error stack trace in the error view. The Snaps now provide detailed information in case of any issues or errors.

May 2023433patches21482 Latest
  • Fixed an issue that caused a String index out of range error with the SFTP protocol in the File Writer Snap when it attempted to create a top-level directory.

  • Enhanced the PGP Encrypt Snap with the Encryption key ID field, which allows you to specify the key ID for encrypting the data. If you do not specify an encryption key ID, the Snap uses the primary key in the public key (master key).

Starting from version 433patches21482, the PGP Encrypt Snap will no longer support encryption with an expired key. To ensure continued support for encryption, we highly recommend extending the expiration of your PGP key.

May 2023433patches21291 LatestFixed an issue with the Multi File Reader Snap where it failed with the error S3 object not found when the Snap found no matching file to read and the Folder/File property value did not end with a forward slash (/).
May 2023433patches21179 Latest
  • Fixed an issue with the File Delete Snap where the Snap failed with a 404 Not Found error when trying to delete files from an Amazon S3 bucket. This issue occurred only with the Identity and Access Management (IAM) role in an Amazon AWS S3 Account.

  • Fixed an issue where Binary Snaps could not handle region information for the Amazon S3 file protocol, which resulted in an error.

May 2023main21015 Stable

The Key passphrase field in the Private Key Account now supports expressions, allowing dynamic evaluation using pipeline parameters when the expression button is enabled.

February 2023

432patches20458

 Latest

Fixed an issue where the ZipFile Read and ZipFIle Write Snaps failed to display the input schema for the File or File name field when using an expression.

February 2023432patches20431 Latest
  • Fixed an issue where the File Writer Snap would not retry on completing the writing of the file.

  • Added a configuration warning message when using the WASBS protocol with AzCopy, as it supports only the HTTPS protocol.
February 2023432patches20349 Latest

The JSCH library has been upgraded to version 0.2.7.

February 2023main19844 StableUpgraded with the latest SnapLogic Platform release.
November 2022431patches18977 Latest

The PGP Decrypt Snap now allows you to skip the signature verification when you face an issue with the signature in the encrypted file.

November 2022main18944 StableUpgraded with the latest SnapLogic Platform release.
September 2022430patches17933 Latest
  • The File Delete Snap now passes the input document to the error view correctly.

  • The AWS S3 and S3 Dynamic accounts now support the maximum session duration of an IAM role defined in AWS.

August Patch

430patches17292

 Stable and Latest

Fixed an issue with the Directory Browser Snap, which failed with a null pointer exception error when connecting to the SFTP server containing a port number.

The Binary Snap Pack is deployed as both the latest and stable distribution. We recommend that you use this version for your Org when using the recommended Snaplex version (main-13269 - 4.30 GA).

August 2022main17386 Stable
  • The File Operation Snap supports moving data from a local node to an Azure blob through the AZ Copy utility.

  • The Azure Storage Account includes the Request Size (MB) field to set the buffer limit before writing to Azure storage to enhance the performance.

  • The SSH Auth Account supports dynamic values for the following fields that allow you to use Pipeline parameters.

    • Username

    • Private key

    • Key passphrase

4.29 Patch

429patches16569

 Latest
4.29 Patch429patches15842 Latest
  • Improved the tooltip for the File action field in the ZipFile Write Snap.
  • Enhanced the SSH Auth Account with Expression enabler () for the following fields that allow the use of Pipeline parameters to populate the properties.
    • Username
    • Private key
    • Key passphrase
4.29main15993 Stable

Enhanced the following Snaps and Accounts that use SnapLogic File System (SLFS) to display Lint warnings in Pipelines when using incorrect file paths to read and write files. We recommend that you use the File Upload :Upload_icon: icon to specify the file path.

4.28 Patch428patches14919 Latest
  • Fixed an issue with the Directory Browser Snap where the Snap failed while parsing the value in the Amazon S3 client region.
  • Enhanced the File Writer Snap with the Prevent URL encoding checkbox to manage the URL encoding. Select this checkbox to prevent the encoding of the file path URL automatically. This allows the Snap to use the file path value as-is. Deselect this checkbox to automatically encode the characters in the File URL.
  • Fixed an issue with the ZipFile Write Snap where the Snap displayed a null pointer exception when the path in the upstream File Reader Snap started with "../".
  • Fixed an issue in the File Reader Snap where the Snap displayed an SMB Mount error when trying to connect to Mount Share. This Snap now supports SMB version 3.1.1.
  • Fixed an issue with the File Writer Snap where the Snap did not display an error when an error occurred while writing a file.
4.28 Patch

428patches14736

 Latest

Fixed an issue in the File Writer, S3 File Writer, and ZipFile Write Snaps where the Snaps failed with AccessControlListNotSupported error when using an external AWS account to write to the bucket if it has the bucket-owner-enforced setting enabled. With this fix, you can select the None option (the default value), from the AWS Canned ACL dropdown list, indicating that no Canned ACL request is set when writing a new file to prevent the error.

4.28main14627 Stable
  • Enhanced the File Operation Snap to support Azure Data Lake Storage (ADLS) Gen2 protocol for moving and copying files in the Azure Blob File System (ABFS).

  • Fixed an issue with the File Poller Snap to enhance performance of the polling operations that allows the Snap to poll the files without stopping.

  • Enhanced the File Writer, S3 File Writer, and ZipFile Write Snaps with the AWS Canned ACL (Access Control List) permission mode that enables you to use Canned ACLs when creating the S3 Object. This feature allows you to grant full access of the object to the S3 bucket owner.
4.27 Patch427patches14283 Latest
  • Fixed an issue with the File Writer Snap where the Server Message Block (SMB) mount failed to connect unless the Pipeline was restarted.

  • Enhanced the S3 File Writer Snap with the Suggest fully-qualified file names checkbox that allows you to use fully-qualified file names in the File name Suggestions list for gov cloud instances.

  • Fixed an issue with the File Poller Snap to handle indefinite polling operations without stopping in the process of execution.

  • Fixed an issue with the ZipFile Write Snap where the ZIP extension files were downloaded with an incorrect file extension.

  • Fixed an issue with the Zip File Read Snap where an archive failed to uncompress.

  • Fixed an issue with the File Writer Snap when a file greater than 10 MB failed to write when uploading to Google Storage.

  • Upgraded the Google Cloud Storage Java libraries from version 1.105.1 to version 2.2.2.

4.27 Patch

427patches13004

 Latest

Fixed an issue with the SMB Account where the Snaps could not re-establish a lost connection to the SMB mount point.

4.27main12833Stable
  • Enhanced the File Operation Snap with a new field Azcopy absolute path that enables the Snap to make a single API call to move an entire directory. This field supports directory and wildcard characters for Azure blob storage for all three operations: Copy, Move and Rename.
  • Enhanced the Azure Storage Account with a new option called SAS URI in the Auth Type dropdown. This enables you to select either Access Key or the SAS URI while configuring the account settings.
4.26 Patch426patches12288Latest

Fixed a memory leak issue when using HDFS protocol in Binary Snaps.

4.26 Patch426patches11922LatestFixed an issue with the File Writer Snap, where the Snap failed to open a channel while accessing through a SFTP protocol.
4.26 Patch426patches11529Latest
  • Fixed an issue with the Decompress Snap, where the Snap prematurely uncompressed multi-stream BZIP2 files.
  • Enhanced the PGP Encrypt Snap with improved memory usage and also fixed an issue with this Snap, where the Snap failed to encrypt large size files due to the lack of memory.
4.26 main11181
StableUpgraded with the latest SnapLogic Platform release.
4.25 Patch

425patches10994

 
Latest
  • Removed duplicate strings in the request’s User Agent header sent by  S3 File Writer and S3 File Reader Snaps to display the string without any duplicates.
  • Enhanced the Directory Browser Snap to process the files with the owner field for the SFTP protocol. Upon validation, the output of the Snap fetches the owner’s name, which was previously displayed as unknown.
  • Fixed an issue with the File Writer Snap where the Snap fails to connect to the SFTP endpoint using the AWS Transfer Family.
  • Fixed an issue where integration Pipelines fail to connect the SFTP Server with the Binary Snap Pack when upgraded to the SnapLogic 4.25 release version (main9554).
4.25 Patch425patches10217
 
Latest
  • Fixed the following issues in the PGP Decrypt Snap caused due to the Snap patch 425patches9581:
    • The Snap writes an empty document to the output view when there is an error in the Snap. It will now decrypt the document and write it to the output view only if it succeeds.
    • The Snap fails to decrypt any non-signed encrypted file more than 500 bytes. It will now decrypt both non-signed and signed encrypted files. 
    • Removed the Project ID field from the Google Service Account, since the JSON Key already contains information about the Project ID. 
4.25 Patch425patches9581
 
Latest

Fixed an issue with the PGP Decrypt Snap where the Snap writes an empty document to the output view when there is an error in the Snap. It will now decrypt the document and write it to the output view only if it succeeds.

As of  we have discovered an issue with this Binary patch affecting the PGP Decrypt Snap. In case your Pipelines use the PGP Decrypt Snap, we recommend you to NOT use this patch. Else, your Pipelines will likely fail. We're working on a fix and will be deploying a Snap patch soon.

4.25 Patch425patches9564
 
Latest

Enhanced the SFTP session in Binary Snaps to support the SFTP servers which close SFTP sessions slowly.

4.25main9554
 
Stable

Enhanced the S3 File Writer Snap with an ability to write S3objects with Access Control List (ACL) details in a single API call.

In the 4.23 release, the S3 File Writer Snap used to write an S3 file and then set ACL if the ACL property was not empty. If the file write was successful and the ACL setting failed, both the output document in the output view and the error document in the error view was displayed. This issue has been resolved in the 4.24 release by enabling the Snap to configure ACL while writing the file. Therefore, the Snap produces an error document in the error view if the ACL configuration fails, irrespective of whether the file write action passes or fails.

4.24 Patch424patches9044-Latest

Enhanced the S3 File Writer to use an MD5 checksum that automatically checks for data integrity and corruption while uploading the file.  

4.24 Patch424patches9020 
 
Latest

Fixed the file operation error of copying large S3 files in the File Operation Snap by supporting the transfer of zero-byte S3 files while uploading a file.

4.24 Patch424patches8876
 
Latest
  • Fixed the missing library error in Binary Snap Pack when running Hadoop Pipelines in JDK11 runtime.
  • Enhanced the Binary Snap Pack by adding a new account Two-Factor Auth Account that supports two layers of authentication for Secure File Transfer Protocol (SFTP). The two secure layers of verification to grant access include the Basic Auth (username/password) and the SSH Key (username/private key) methods.


4.24main8556
 
StableUpgraded with the latest SnapLogic Platform release.
4.23 Patch423patches8453
 
Latest

Fixed an issue in the File Writer Snap with the ADL protocol, to perform the correct action when OVERWRITE is selected from the File Action drop-down field.

4.23 Patch423patches8368-Latest

Fixed an encryption issue with the File Operation Snap by implementing both SSE-KMS and SSE-S3 supports to the Snap.

4.23 Patch423patches8368-Latest

Fixed an issue in the Snap Pack by removing the plaintext credentials in file URL from JCC log messages.

4.23 Patch423patches8099-Latest

Enhanced the Snap Pack by upgrading SMB client library.

4.23 Patch423patches7958-Latest

Fixed an issue with the File Writer Snap by avoiding overwriting of documents when using appending the data to a CSV file in SFTP.

4.23 Patch423patches7958-Latest

Rolled back the recent patch to Binary Snap Pack (File Writer Snap - 423patches7923) that might have induced SSL handshake failures/terminations in existing Pipelines. 

4.23 Patch423patches7923
Latest
  • Fixed the SFTP connection failure issue in File Writer Snap by
    • Ensuring that the BouncyCastleProvider is the first in the list of providers in the Security object.
    • Upgrading the BouncyCastle version to 1.67.

Do not use the 423patches7923 build for the File Writer Snap, because it causes connectivity issues with the Snaplex nodes.

4.23 Patch423patches7795-Latest

Fixed an issue with File Writer Snap by skipping the validation of the SAS URI having write permission, if the SAS URI uses an access policy.

4.23 Patch423patches7660-Latest

Enhanced the File Poller Snap by adding a field, Exit on first matches, which when set to true stops the Snap from executing after the first file paths matching the filter are written to the output view. If the field is not configured or is set to false (default value), then the Snap continues to poll the directory until the Polling timeout is reached.

4.23main7430-Stable

Enhanced the ZipFile Write Snap to provide the ability to change the default content-type (application/octet-stream) for any file that is stored into the Azure Blob Storage (WASB: or WASBS: file protocol). Hence, the files that the ZipFile Write Snap writes into the Blob Storage can be saved with content-type asapplication/zipor any other type instead of application/octet-streamalways.

You can now prevent the Snap from writing files always to Azure Blob Storage (WASB: or WASBS: file protocol) with the application/octet-stream content-type. To do so, configure a binary header property in an upstream Snap, such as JSON Formatter or Mapper, to match the file's content – application/zipapplication/java-archive and so on; and pass it as the $content-type (case-sensitive) for the file.

Binary header property in upstream Snap

4.22 Patch422patches6947-Latest

Fixed the issue of the hanging thread in the SMB accounts.

4.22 Patch422patches6950-Latest

Enhanced the S3 Dynamic Account by allowing to use the Security token field in Directory Browser, File Operation, File Poller, Multi File Reader, and File Delete Snaps. 

4.22 Patch422patches6629-Latest

Fixed the ZipFile Write Snap error by allowing the number of retries to be set greater than 0.

4.22 Patch422patches6542-Latest

Fixed SMB hanging threads by upgrading SMB client library.

4.22main6403-Latest
  • Added the Cross Account IAM Role feature to the existing accounts, AWS S3 Account and S3 Dynamic Account, to enable the AWS account user to access the resources of another AWS account for a specified time without sharing the login credentials.
  • Enhanced the File Operation Snap by adding a new property Simple file operation that allows moving or copying a file, through FTP protocol, irrespective of the account's access permissions on source and target directories.
4.21 Patch421patches6136-Latest

Fixed the issue with Multi File Reader Snap that continues to look for a directory even when FileSystem information is unavailable.

4.21 Patch421patches6090-Latest

Fixed the 'Connection pool shut down' error in S3 File Writer Snap by keeping the AmazonS3Client class object open until all the documents are processed.

4.21 Patch 421patches6077-Latest

Fixed the UnknownHostException error in Binary Snaps due to intermittent failures in FileSystemProvider registration, by replacing ServiceLoader with direct instantiations of FileSystemProvider class objects.

4.21 Patch421patches5994-Latest

Fixed an error in Multi File Reader and File Delete Snaps during Pipeline execution where the Snaps fail to retrieve the temporary files from the given S3 location.

4.21 Patch 421patches5963-Latest

Enhanced the Directory Browser Snap by making the File filter field optional.

4.21 Patch421patches5975-Latest

Fixed the Directory Browser Snap that fails with an Unable to create filesystem object for smb:// error, by rolling back the SMB client library to the prior working version as in snapsmrc542.

4.21 Patch421patches5871-Latest

When configuring the SSH Auth account type, SSH passphrase is now optional, whereas it was a required field earlier. 

4.21 Patchbinary8849-Latest

Enhanced the Directory Browser Snap by adding a new advanced property Use fast file-type check for wasb/wasbs protocols for quick Snap execution. 

4.21 Patchbinary8843-Latest

Fixed the issue in PGP Encrypt Snap where decrypting its output file using a third-party PGP software adds a random suffix and TMP extension in the file name. This issue occurs only when using the PGP Encrypt Snap in a Windows-based Groundplex.

4.21snapsmrc542-StableUpgraded with the latest SnapLogic Platform release.
4.20 Patchbinary8801-Latest
  • Fixed the Decompress Snap wherein the Snap stops decompressing at 64MB without displaying any error, when decompressing using the GZIP scheme.
  • Fixed an issue in the PGP Encrypt Snap so that third-party decryption tools do not receive a .tmp file from the Snap.   
4.20 Patchbinary8791-Latest

Fixed the ZipFile Write Snap wherein the Snap fails to execute if it finds different zip file paths when processing multiple input binary data.

4.20 Patchbinary8785-Latest
  • Added the User-defined object metadata and Object tags properties in the S3 File Writer Snap and the Get Object Tags property in the S3 File Reader Snaps to allow the inclusion of user-defined metadata and object tags.
  • Fixed a regression caused by Patch binary8775 wherein the File Reader and S3 File Reader Snaps fail to read S3 files with an account that does not include GetObjectTagging permission.
4.20 Patchbinary8775-Latest

Fixed the File Writer Snap to allow the deletion of temporary Google Storage files from the /tmp directory of the Groundplex if Pipeline execution stops before completion. 

4.20 Patchbinary8741-Latest
  • Fixed the ZipFile Write Snap wherein the Snap gives a Null Pointer Exception error if no input data is received by it during execution. 
  • Fixed the Email Sender & Reader Snaps where the Snaps fail to execute due to resource exhaustion on the Snaplex.
4.20 Patchbinary8710-Latest
  • Fixed the ZipFile Write Snap where the Snap generates files with incorrect names when the Use input view label checkbox is selected.
  • Fixed the File Writer Snap where the Snap fails intermittently when there is a network disruption.
  • Fixed the File Reader Snap where the Snap generates an output view, along with the error view, if there is an error when using the Google Service account.
4.20snapsmrc535-StableUpgraded with the latest SnapLogic Platform release.
4.19 Patchbinary8369-Latest

Fixed the following issues associated with the SMB account wherein:

  • Pipelines with multiple concurrent SMB connections hang intermittently.
  • Directories are not deleted completely.
4.19snaprsmrc528-Latest
  • The Key Passphrase field is now a mandatory field for the SSH Auth account. Ensure that you provide a passphrase when adding/editing accounts. Else, you will likely lose all updates related to the account and Pipeline.
  • Raises the minimum buffer size in the S3 File Writer Snap to 6 MB.
4.18 Patchbinary8768-Latest
  • Fixed the File Reader, File Writer, and Directory Browser Snaps to resolve 0x103 (STATUS_PENDING) and 0xc000022d (STATUS_RETRY) errors for SMB file operations.
4.18 Patchbinary8046-Latest

Enhanced the Snap Pack to support AWS SDK 1.11.634 to fix the NullPointerException issue in the AWS SDK. This issue occurred in AWS-related Snaps that had HTTP or HTTPS proxy configured without a username and/or password. 

4.18 Patch MULTIPLE7884-Latest

Fixed an issue with the PostgreSQL grammar to better handle the single quote characters.

4.18 Patchbinary7860-Latest

Fixed an issue with the File Writer Snap wherein the Snap is intermittently unable to set correct permissions.

4.18 Patch

binary7851

-Latest
  • Added new field, Advanced properties, in the File Reader, File Writer, File Delete, ZipFile Read, ZipFile Write, Directory Browser, Multi File Reader, File Poller, and File Operation Snaps. Use this field to configure the SAS URI to access Azure Storage. Supported SAS types are Service SAS on container, Service SAS on blob, and Account SAS.
  • Added new Snap, SAS Generator. This Snap enables you to generate Shared Access Storage (SAS) URIs for accessing Azure Storage containers and blobs.
4.18 Patchbinary7746-Latest

Fixed an issue with the SMB file protocol wherein Binary Snaps are unable to create a native thread using an SMB file path.

4.18 PatchMULTIPLE7778-Latest

Updated the AWS SDK library version to default to the Signature Version 4 Signing process for API requests across all regions.

4.18 Patchbinary7720-Latest

Fixed an issue with the ZipFile Read Snap wherein the Snap fails to extract files from a zip file heavier than 100MB when the number of retries is greater than zero.

4.18snapsmrc523-Latest

Added a new account type, Google Service Account, which enables you to isolate workflows from individual users.

4.18snapsmrc523-Latest

Added a new account type, Google Service Account, which enables you to isolate workflows from individual users.

4.17 Patch binary7589-Latest

Fixed an issue with the File Reader Snap wherein SMB fails to provide multiple Snaps with concurrent access.

4.17 Patch binary7487-Latest

Fixed an issue with the File Writer Snap wherein the Overwrite File action for SMB files fails on subsequent executions.

4.17 PatchALL7402-Latest

Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

4.17 Patch binary7381-Latest

Fixed an issue with SMB accounts wherein login to the server fails if password contains a plus (+) sign.

4.17 Patchbinary7366-Latest

Added support for multiple slashes as a prefix for SMB file URLs.

4.17 Patchbinary7280-Latest

Added a new field, File Delete Action, to the File Delete Snap that handles missing target files.

4.17 Patchsnapsmrc515-Latest
4.16 Patchbinary7049-Latest

Fixed an issue with the File Writer Snap wherein the Snap writes FTPS files partially. Two new global properties, ftpsDelayBeforeClosing and ftpDelayBeforeClosing, enable you to add delays to the FTP file protocol to handle the partial write. 

4.16 Patchbinary7049-Latest

Fixed an issue with the File Writer Snap wherein the Snap writes FTPS files partially. Two new global properties, ftpsDelayBeforeClosing and ftpDelayBeforeClosing, enable you to add delays to the FTP file protocol to handle the partial write. 

4.16 Patchbinary6975-Latest

Added a new property, Blob endpoint, for Binary Accounts, which enables you to specify the Azure Blob Storage endpoint that you want to use with an Azure Storage account. 

4.16snapsmrc508-Latest

Added a new property Only Output on Change in the File Poller Snap. This property helps avoid redundancy by not displaying the same matching file in every polling result unless the file is changed.

4.15 Patchbinary6744-Latest

Fixed a timeout error while writing to S3 in the S3 File Writer Snap.

4.15 Patchbinary6414-Latest

Fixed an issue wherein the hostname specified in the Directory Browser path failed to appear as a path in the output document.

4.15 Patchbinary6414-Latest

Fixed an issue wherein the hostname specified in the Directory Browser path failed to appear as a path in the output document.

4.15snapsmrc500-Latest

Added a new property, Binary Header Properties, to the CSV Formatter Snap. This property can be configured with the binary headers and their values in the incoming document.  

4.14 PatchMULTIPLE5732-Latest

Fixed an issue with S3 file reads getting aborted intermittently because of incomplete consumption of input stream.

4.14snapsmrc490-Latest

Enhanced the Basic Auth and Symmetric Crypto accounts to support pipeline parameters. Using pipeline parameters, Basic Auth now supports passing username and password, and Symmetric Crypto now supports passing secret key dynamically.

4.13 Patch

binary5372

-Latest

Updated the File Operation Snap for S3 to support files larger than 5GB by using an API for multipart upload copy.

4.13 Patch binary5315-Latest

Added a new property "Enable staging" to enable the File Reader and S3 File Reader Snaps to perform as expected even in a slow network connection to AWS S3.

4.13snapsmrc486-Latest

Added AWS Key Management Service (KMS) encryption support to AWS S3 and S3 Dynamic accounts in the Binary Snap Pack.

4.12binary5044-Latest

Fixed an issue with the S3 File Writer Snap, which corrupts while writing files if retry value is set to 1.

4.12 Patchbinary4882-Latest

Fixed issue caused in the File Operation Snap due to the rollback after the 4.12 release that impacted the S3 and FTP protocols. 

4.12 Patchbinary4840-Latest

Fixed an issue in the File Delete Snap, wherein if no account is given for the SFTP protocol, then the error did not display.

4.12 Patchbinary4797-Latest

Added a new property 'Number of retries' in the File Writer Snap to fix interoperability issues with a server software.

4.12 Patchbinary4736-Latest

Fixed the issue where File Writer Snap does not write files to Windows Serv-U FTP server, while not displaying any error message.

4.12snapsmrc480-Latest
  • Added new Snaps - File Operation Snap and S3 File Reader Snap.
  • Updated S3 File Writer Snap to provide suggestions for S3 buckets and file names (versioning support).
4.11 Patchbinary4424-Latest

Resolved an Issue with SFTP server where the pipeline stopped responding while writing to the Tumbleweed server (SFTP server). Fixed the code in a way that the SFTP server now flushes out data after each write.

4.11 Patchbinary4328-Latest

Fixed an issue with the File Delete Snap that was unable to delete the file with a size greater than 1MB.

4.11 Patchsnapsmrc465-Latest
  • Added the new Snaps, Multipart Reader and Multipart Writer.
  • Updated the S3 File Writer Snap with Maximum upload threads property to support the performance.
  • The File Reader and the ZipFile Reader Snaps now show the progress message during the pipeline runtime.
4.11binary4019-Stable

Resolved an issue with the File Writer Snap writing to an FTP Server that may prematurely close the connection; this patch ensures the write has fully completed before disconnecting.

4.10 Patchbinary3930-Latest

Directory Browser - When "Ignore empty result" is not selected, the "original" passthrough document is now returned even when the S3 directory is empty.

4.10 Patchbinary3840-Latest

S3OutputStream handles failures.

4.10snapsmrc414-Stable

No updates made. Automatic rebuild with a platform release.

4.9 Patchbinary3331-Latest

Directory Browser Snap - fixed NPE issue, "File filter" property, and "Display update time and owner for S3 subdirectories" property.

4.9 Patchbinary3183-Latest

Fixed SFTP connection leak when "Validate after write" is checked for File Write Snap.

4.9 Patchbinary3170-Latest

Addressed the following issue in Directory Browser: "Update time" being returned "unknown" for directories within S3 objects

4.9 Patchbinary3081-Latest

Added ability to sign PGP encrypted files.

4.9 Patchbinary3060-Latest

Addressed an issue in Directory Browser and fixed issue in writing the original document to output view when reading from S3.

4.9 Patchbinary3022-Latest

Addressed an issue regarding an array out of bounds exception when IAM Role is enabled.

4.9 Patchbinary3000-Latest
  • Restore the update date date-time format for using Directory Browser with S3. A new property Display update time and owner name for s3 subfolders was added that, when enabled, will calculate update times and owners on AWS S3 subfolders.
  • Added the IAM support to the AWS S3 account.
4.9Snapsmrc405-Stable

No updates made. Automatic rebuild with a platform release.

4.8 PatchSnapsmrc405-Latest

Addressed an issue with Directory Browser taking too lont to start listing files from S3 bucket and then failing with partial results due to CPU spikes.

4.8Snapsmrc398-Latest
  • Info tab added to accounts.

  • Streaming performance improvement while writing to an S3 bucket using S3 File Writer. Updated the S3 File Writer Snap with a Buffer size(MB) property.
  • The 'content-location' in the output header of Binary Snaps was updated to correctly reflect the location (for example, sldb:/// instead of sldb://). This may impact your Pipeline if you are using this information in the following Snaps: File Reader, AES Encrypt, AES Decrypt, BlowFish Encrypt, Blowfish Decrypt, Compress, Decompress, File Writer, Multi File Reader, PGP Encrypt, PGP Decrypt, TwoFish Encrypt, TwoFish Decrypt, or S3 File Writer.
4.7.0 Patchbinary2346-Latest

ZipFile Read Snap: Added support for reading 7z archive formats created using Windows machines.

4.7.0 Patchbinary2337-Latest

Added Encrypted filename property to PGP Encrypt Snap.

4.7Snapsmrc382-Latest
  • Updated the Snap Account settings with a new Account Type, Azure Data Lake.
  • Updated the Snaps- Directory Browser, File Writer, File Reader with the account Type, Azure Data Lake.
  • Updated the File Reader and ZipFile Read Snaps with the property, Prevent URL encoding.
  • Spark mode support added to S3 File Writer for AWS S3.
  • Spark mode support added to File Reader for AWS S3.
4.6.0 Patchbinary2110-Latest

File Writer: fixed an issue on Windows Snaplexes that prevented creating directories that are not present.

4.6.0 Patchbinary2009-Latest

File Writer: fixed an issue on Windows Snaplexes that prevented creating directories that are not present.

4.6.0 Patchbinary1898-Latest

File Reader and ZipFile Reader Snaps: Added a new "Prevent URL encoding" setting to allow users to disable automatic URL encoding of file paths. This allows pre-encoded URLs and file paths to be used as-is.

4.6.0 Patch