Enhanced the following Snaps and Accounts that use SnapLogic File System (SLFS) to display Lint warnings in Pipelines when using incorrect file paths to read and write files. We recommend that you use the File Upload :Upload_icon: icon to specify the file path.

• File Writer

• File Reader

• File Delete

• HDFS Reader

• HDFS Writer

• HDFS ZipFile Reader

• HDFS ZipFile Writer

• S3 File Writer

• S3 File Reader

• ORC Reader

• ORC Write

• ZipFile Read

• ZipFile Write

• Google Service Account

• Two-Factor Auth Account

• Fixed an issue with the Directory Browser Snap where the Snap failed while parsing the value in the Amazon S3 client region.
• Enhanced the File Writer Snap with the Prevent URL encoding checkbox to manage the URL encoding. Select this checkbox to prevent the encoding of the file path URL automatically. This allows the Snap to use the file path value as-is. Deselect this checkbox to automatically encode the characters in the File URL.
• Fixed an issue with the ZipFile Write Snap where the Snap displayed a null pointer exception when the path in the upstream File Reader Snap started with "../".
• Fixed an issue in the File Reader Snap where the Snap displayed an SMB Mount error when trying to connect to Mount Share. This Snap now supports SMB version 3.1.1.
• Fixed an issue with the File Writer Snap where the Snap did not display an error when an error occurred while writing a file.

428patches14736

Fixed an issue in the File Writer, S3 File Writer, and ZipFile Write Snaps where the Snaps failed with AccessControlListNotSupported error when using an external AWS account to write to the bucket if it has the bucket-owner-enforced setting enabled. With this fix, you can select the None option (the default value), from the AWS Canned ACL dropdown list, indicating that no Canned ACL request is set when writing a new file to prevent the error.

• Enhanced the File Operation Snap to support Azure Data Lake Storage (ADLS) Gen2 protocol for moving and copying files in the Azure Blob File System (ABFS).

• Fixed an issue with the File Poller Snap to enhance performance of the polling operations that allows the Snap to poll the files without stopping.

• Enhanced the File Writer, S3 File Writer, and ZipFile Write Snaps with the AWS Canned ACL (Access Control List) permission mode that enables you to use Canned ACLs when creating the S3 Object. This feature allows you to grant full access of the object to the S3 bucket owner.

• Fixed an issue with the File Writer Snap where the Server Message Block (SMB) mount failed to connect unless the Pipeline was restarted.

• Enhanced the S3 File Writer Snap with the Suggest fully-qualified file names checkbox that allows you to use fully-qualified file names in the File name Suggestions list for gov cloud instances.

• Fixed an issue with the File Poller Snap to handle indefinite polling operations without stopping in the process of execution.

• Fixed an issue with the ZipFile Write Snap where the ZIP extension files were downloaded with an incorrect file extension.

• Fixed an issue with the Zip File Read Snap where an archive failed to uncompress.

• Fixed an issue with the File Writer Snap when a file greater than 10 MB failed to write when uploading to Google Storage.

• Upgraded the Google Cloud Storage Java libraries from version 1.105.1 to version 2.2.2.

427patches13004

Fixed an issue with the SMB Account where the Snaps could not re-establish a lost connection to the SMB mount point.

• Enhanced the File Operation Snap with a new field Azcopy absolute path that enables the Snap to make a single API call to move an entire directory. This field supports directory and wildcard characters for Azure blob storage for all three operations: Copy, Move and Rename.
• Enhanced the Azure Storage Account with a new option called SAS URI in the Auth Type dropdown. This enables you to select either Access Key or the SAS URI while configuring the account settings.

Fixed a memory leak issue when using HDFS protocol in Binary Snaps.

• Fixed an issue with the Decompress Snap, where the Snap prematurely uncompressed multi-stream BZIP2 files.
• Enhanced the PGP Encrypt Snap with improved memory usage and also fixed an issue with this Snap, where the Snap failed to encrypt large size files due to the lack of memory.

425patches10994

• Removed duplicate strings in the request’s User Agent header sent by  S3 File Writer and S3 File Reader Snaps to display the string without any duplicates.
• Enhanced the Directory Browser Snap to process the files with the owner field for the SFTP protocol. Upon validation, the output of the Snap fetches the owner’s name, which was previously displayed as unknown.
• Fixed an issue with the File Writer Snap where the Snap fails to connect to the SFTP endpoint using the AWS Transfer Family.
• Fixed an issue where integration Pipelines fail to connect the SFTP Server with the Binary Snap Pack when upgraded to the SnapLogic 4.25 release version (main9554).

• Fixed the following issues in the PGP Decrypt Snap caused due to the Snap patch 425patches9581:
  • The Snap writes an empty document to the output view when there is an error in the Snap. It will now decrypt the document and write it to the output view only if it succeeds.
  • The Snap fails to decrypt any non-signed encrypted file more than 500 bytes. It will now decrypt both non-signed and signed encrypted files. 
  • Removed the Project ID field from the Google Service Account, since the JSON Key already contains information about the Project ID. 

Enhanced the SFTP session in Binary Snaps to support the SFTP servers which close SFTP sessions slowly.

Enhanced the S3 File Writer to use an MD5 checksum that automatically checks for data integrity and corruption while uploading the file.  

Fixed the file operation error of copying large S3 files in the File Operation Snap by supporting the transfer of zero-byte S3 files while uploading a file.

• Fixed the missing library error in Binary Snap Pack when running Hadoop Pipelines in JDK11 runtime.
• Enhanced the Binary Snap Pack by adding a new account Two-Factor Auth Account that supports two layers of authentication for Secure File Transfer Protocol (SFTP). The two secure layers of verification to grant access include the Basic Auth (username/password) and the SSH Key (username/private key) methods.

Fixed an issue in the File Writer Snap with the ADL protocol, to perform the correct action when OVERWRITE is selected from the File Action drop-down field.

Fixed an encryption issue with the File Operation Snap by implementing both SSE-KMS and SSE-S3 supports to the Snap.

Fixed an issue in the Snap Pack by removing the plaintext credentials in file URL from JCC log messages.

Enhanced the Snap Pack by upgrading SMB client library.

Fixed an issue with the File Writer Snap by avoiding overwriting of documents when using appending the data to a CSV file in SFTP.

Rolled back the recent patch to Binary Snap Pack (File Writer Snap - 423patches7923) that might have induced SSL handshake failures/terminations in existing Pipelines. 

Fixed an issue with File Writer Snap by skipping the validation of the SAS URI having write permission, if the SAS URI uses an access policy.

Enhanced the File Poller Snap by adding a field, Exit on first matches, which when set to true stops the Snap from executing after the first file paths matching the filter are written to the output view. If the field is not configured or is set to false (default value), then the Snap continues to poll the directory until the Polling timeout is reached.

Fixed the issue of the hanging thread in the SMB accounts.

Enhanced the S3 Dynamic Account by allowing to use the Security token field in Directory Browser, File Operation, File Poller, Multi File Reader, and File Delete Snaps. 

Fixed the ZipFile Write Snap error by allowing the number of retries to be set greater than 0.

Fixed SMB hanging threads by upgrading SMB client library.

• Added the Cross Account IAM Role feature to the existing accounts, AWS S3 Account and S3 Dynamic Account, to enable the AWS account user to access the resources of another AWS account for a specified time without sharing the login credentials.
• Enhanced the File Operation Snap by adding a new property Simple file operation that allows moving or copying a file, through FTP protocol, irrespective of the account's access permissions on source and target directories.

Fixed the issue with Multi File Reader Snap that continues to look for a directory even when FileSystem information is unavailable.

Fixed the 'Connection pool shut down' error in S3 File Writer Snap by keeping the AmazonS3Client class object open until all the documents are processed.

Fixed the UnknownHostException error in Binary Snaps due to intermittent failures in FileSystemProvider registration, by replacing ServiceLoader with direct instantiations of FileSystemProvider class objects.

Fixed an error in Multi File Reader and File Delete Snaps during Pipeline execution where the Snaps fail to retrieve the temporary files from the given S3 location.

Enhanced the Directory Browser Snap by making the File filter field optional.

Fixed the Directory Browser Snap that fails with an Unable to create filesystem object for smb:// error, by rolling back the SMB client library to the prior working version as in snapsmrc542.

When configuring the SSH Auth account type, SSH passphrase is now optional, whereas it was a required field earlier. 

Enhanced the Directory Browser Snap by adding a new advanced property Use fast file-type check for wasb/wasbs protocols for quick Snap execution. 

Fixed the issue in PGP Encrypt Snap where decrypting its output file using a third-party PGP software adds a random suffix and TMP extension in the file name. This issue occurs only when using the PGP Encrypt Snap in a Windows-based Groundplex.

• Fixed the Decompress Snap wherein the Snap stops decompressing at 64MB without displaying any error, when decompressing using the GZIP scheme.
• Fixed an issue in the PGP Encrypt Snap so that third-party decryption tools do not receive a .tmp file from the Snap.   

Fixed the ZipFile Write Snap wherein the Snap fails to execute if it finds different zip file paths when processing multiple input binary data.

• Added the User-defined object metadata and Object tags properties in the S3 File Writer Snap and the Get Object Tags property in the S3 File Reader Snaps to allow the inclusion of user-defined metadata and object tags.
• Fixed a regression caused by Patch binary8775 wherein the File Reader and S3 File Reader Snaps fail to read S3 files with an account that does not include GetObjectTagging permission.

Fixed the File Writer Snap to allow the deletion of temporary Google Storage files from the /tmp directory of the Groundplex if Pipeline execution stops before completion. 

• Fixed the ZipFile Write Snap wherein the Snap gives a Null Pointer Exception error if no input data is received by it during execution. 
• Fixed the Email Sender & Reader Snaps where the Snaps fail to execute due to resource exhaustion on the Snaplex.

• Fixed the ZipFile Write Snap where the Snap generates files with incorrect names when the Use input view label checkbox is selected.
• Fixed the File Writer Snap where the Snap fails intermittently when there is a network disruption.
• Fixed the File Reader Snap where the Snap generates an output view, along with the error view, if there is an error when using the Google Service account.

Fixed the following issues associated with the SMB account wherein:

• Pipelines with multiple concurrent SMB connections hang intermittently.
• Directories are not deleted completely.

• The Key Passphrase field is now a mandatory field for the SSH Auth account. Ensure that you provide a passphrase when adding/editing accounts. Else, you will likely lose all updates related to the account and Pipeline.
• Raises the minimum buffer size in the S3 File Writer Snap to 6 MB.

• Fixed the File Reader, File Writer, and Directory Browser Snaps to resolve 0x103 (STATUS_PENDING) and 0xc000022d (STATUS_RETRY) errors for SMB file operations.

Enhanced the Snap Pack to support AWS SDK 1.11.634 to fix the NullPointerException issue in the AWS SDK. This issue occurred in AWS-related Snaps that had HTTP or HTTPS proxy configured without a username and/or password. 

Fixed an issue with the PostgreSQL grammar to better handle the single quote characters.

Fixed an issue with the File Writer Snap wherein the Snap is intermittently unable to set correct permissions.

binary7851

• Added new field, Advanced properties, in the File Reader, File Writer, File Delete, ZipFile Read, ZipFile Write, Directory Browser, Multi File Reader, File Poller, and File Operation Snaps. Use this field to configure the SAS URI to access Azure Storage. Supported SAS types are Service SAS on container, Service SAS on blob, and Account SAS.
• Added new Snap, SAS Generator. This Snap enables you to generate Shared Access Storage (SAS) URIs for accessing Azure Storage containers and blobs.

Fixed an issue with the SMB file protocol wherein Binary Snaps are unable to create a native thread using an SMB file path.

Updated the AWS SDK library version to default to the Signature Version 4 Signing process for API requests across all regions.

Fixed an issue with the ZipFile Read Snap wherein the Snap fails to extract files from a zip file heavier than 100MB when the number of retries is greater than zero.

Added a new account type, Google Service Account, which enables you to isolate workflows from individual users.

Added a new account type, Google Service Account, which enables you to isolate workflows from individual users.

Fixed an issue with the File Reader Snap wherein SMB fails to provide multiple Snaps with concurrent access.

Fixed an issue with the File Writer Snap wherein the Overwrite File action for SMB files fails on subsequent executions.

Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

Fixed an issue with SMB accounts wherein login to the server fails if password contains a plus (+) sign.

Added support for multiple slashes as a prefix for SMB file URLs.

Added a new field, File Delete Action, to the File Delete Snap that handles missing target files.

• Added two properties, Number of retries and Retry interval. Use these to manage retry attempts to the following Snaps in case of network failures: Directory Browser, File Delete, File Poller, File Reader, File Writer, Multi File Reader, S3 File Reader, S3 File Writer, ZipFile Read, and ZipFile Write.
• Enhanced the Snap Pack to support SMB versions 2.0 and 3.0 for the File Delete, File Poller, File Reader, File Writer, Multi File Reader, ZipFile Read, ZipFile Write, Email Reader, Email Sender Snaps.
• Added the Snap Execution field to all Standard-mode Snaps. In some Snaps, this field replaces the existing Execute during preview check box.

Fixed an issue with the File Writer Snap wherein the Snap writes FTPS files partially. Two new global properties, ftpsDelayBeforeClosing and ftpDelayBeforeClosing, enable you to add delays to the FTP file protocol to handle the partial write. 

Fixed an issue with the File Writer Snap wherein the Snap writes FTPS files partially. Two new global properties, ftpsDelayBeforeClosing and ftpDelayBeforeClosing, enable you to add delays to the FTP file protocol to handle the partial write. 

Added a new property, Blob endpoint, for Binary Accounts, which enables you to specify the Azure Blob Storage endpoint that you want to use with an Azure Storage account. 

Added a new property Only Output on Change in the File Poller Snap. This property helps avoid redundancy by not displaying the same matching file in every polling result unless the file is changed.

Fixed a timeout error while writing to S3 in the S3 File Writer Snap.

Fixed an issue wherein the hostname specified in the Directory Browser path failed to appear as a path in the output document.

Fixed an issue wherein the hostname specified in the Directory Browser path failed to appear as a path in the output document.

Added a new property, Binary Header Properties, to the CSV Formatter Snap. This property can be configured with the binary headers and their values in the incoming document.  

Fixed an issue with S3 file reads getting aborted intermittently because of incomplete consumption of input stream.

Enhanced the Basic Auth and Symmetric Crypto accounts to support pipeline parameters. Using pipeline parameters, Basic Auth now supports passing username and password, and Symmetric Crypto now supports passing secret key dynamically.

binary5372

Updated the File Operation Snap for S3 to support files larger than 5GB by using an API for multipart upload copy.

Added a new property "Enable staging" to enable the File Reader and S3 File Reader Snaps to perform as expected even in a slow network connection to AWS S3.

Added AWS Key Management Service (KMS) encryption support to AWS S3 and S3 Dynamic accounts in the Binary Snap Pack.

Fixed an issue with the S3 File Writer Snap, which corrupts while writing files if retry value is set to 1.

Fixed issue caused in the File Operation Snap due to the rollback after the 4.12 release that impacted the S3 and FTP protocols. 

Fixed an issue in the File Delete Snap, wherein if no account is given for the SFTP protocol, then the error did not display.

Added a new property 'Number of retries' in the File Writer Snap to fix interoperability issues with a server software.

Fixed the issue where File Writer Snap does not write files to Windows Serv-U FTP server, while not displaying any error message.

• Added new Snaps - File Operation Snap and S3 File Reader Snap.
• Updated S3 File Writer Snap to provide suggestions for S3 buckets and file names (versioning support).

Resolved an Issue with SFTP server where the pipeline stopped responding while writing to the Tumbleweed server (SFTP server). Fixed the code in a way that the SFTP server now flushes out data after each write.

Fixed an issue with the File Delete Snap that was unable to delete the file with a size greater than 1MB.

• Added the new Snaps, Multipart Reader and Multipart Writer.
• Updated the S3 File Writer Snap with Maximum upload threads property to support the performance.
• The File Reader and the ZipFile Reader Snaps now show the progress message during the pipeline runtime.

Resolved an issue with the File Writer Snap writing to an FTP Server that may prematurely close the connection; this patch ensures the write has fully completed before disconnecting.

Directory Browser - When "Ignore empty result" is not selected, the "original" passthrough document is now returned even when the S3 directory is empty.

S3OutputStream handles failures.

No updates made. Automatic rebuild with a platform release.

Directory Browser Snap - fixed NPE issue, "File filter" property, and "Display update time and owner for S3 subdirectories" property.

Fixed SFTP connection leak when "Validate after write" is checked for File Write Snap.

Addressed the following issue in Directory Browser: "Update time" being returned "unknown" for directories within S3 objects

Added ability to sign PGP encrypted files.

Addressed an issue in Directory Browser and fixed issue in writing the original document to output view when reading from S3.

Addressed an issue regarding an array out of bounds exception when IAM Role is enabled.

• Restore the update date date-time format for using Directory Browser with S3. A new property Display update time and owner name for s3 subfolders was added that, when enabled, will calculate update times and owners on AWS S3 subfolders.
• Added the IAM support to the AWS S3 account.

No updates made. Automatic rebuild with a platform release.

Addressed an issue with Directory Browser taking too lont to start listing files from S3 bucket and then failing with partial results due to CPU spikes.

• Info tab added to accounts.

• Streaming performance improvement while writing to an S3 bucket using S3 File Writer. Updated the S3 File Writer Snap with a Buffer size(MB) property.
• The 'content-location' in the output header of Binary Snaps was updated to correctly reflect the location (for example, sldb:/// instead of sldb://). This may impact your Pipeline if you are using this information in the following Snaps: File Reader, AES Encrypt, AES Decrypt, BlowFish Encrypt, Blowfish Decrypt, Compress, Decompress, File Writer, Multi File Reader, PGP Encrypt, PGP Decrypt, TwoFish Encrypt, TwoFish Decrypt, or S3 File Writer.

ZipFile Read Snap: Added support for reading 7z archive formats created using Windows machines.

Added Encrypted filename property to PGP Encrypt Snap.

• Updated the Snap Account settings with a new Account Type, Azure Data Lake.
• Updated the Snaps- Directory Browser, File Writer, File Reader with the account Type, Azure Data Lake.
• Updated the File Reader and ZipFile Read Snaps with the property, Prevent URL encoding.
• Spark mode support added to S3 File Writer for AWS S3.
• Spark mode support added to File Reader for AWS S3.

File Writer: fixed an issue on Windows Snaplexes that prevented creating directories that are not present.

File Writer: fixed an issue on Windows Snaplexes that prevented creating directories that are not present.

File Reader and ZipFile Reader Snaps: Added a new "Prevent URL encoding" setting to allow users to disable automatic URL encoding of file paths. This allows pre-encoded URLs and file paths to be used as-is.

Resolved an issue with Directory Browser Snap not freeing up connections in Windows Groundplex.

• Binary Snaps that work with AWS S3 accounts can now configure their accounts with IAM Role.
• Enhanced the File Writer Snap with an option to create a directory if the specified directory path does not exist, the newly added field, Create directory if not present.
• Updated based on the currently supported account types.  The supported account types are: Basic Auth, AWS S3, S3 Dynamic, SSH Auth, SMB, Azure Storage, and Google Storage.

• File Reader, ZipFile Read, Multi File Reader Snaps are updated with the Execute during preview field to execute the Snap during the Save operation so that the output view can produce the preview data. For more information, see File Reader, ZipFile Read, & Multi File Reader.
• File Delete & File Writer Snaps are updated with an option to validate file deletion & file writing after Snap execution. For more information, see File Delete & File Writer.
• ZipFile Write Snap is updated with an option, Use input view label, to enable sequential file naming of files added to zip file. For more information, see ZipFile Write.
• Fixed an error in File Reader Snap that exposed account credentials in an error message.
• Fixed an error in File Writer & File Delete Snap to ensure consistent error handling for all the protocols.

• File Reader, File Writer, S3 File Writer, ZipFile Read, Directory Browser, Multi File Reader, File Poller, File Delete, ZipFile Write, HDFS Reader and HDFS Writer updated to support IAM Roles for Amazon EC2.
• Support for Google Storage added to File Reader, File Writer, Directory Browser, Multi File Reader, File Poller, File Delete, ZipFile Read, and ZipFile Write Snaps.
• Resolved an issue in File Writer Snap that occurred when writing to an SFTP file with an SSH account.
• Resolved an issue in Multi File Reader Snap that occurred when adding error views and performing binary data streaming.
• Doc Enhancement: Example added to File Delete Snap.

• A new Snap property Write empty file has been added to File Writer and S3 File Writer Snaps to allow for creation of zero byte files. 
• Resolved an exception in File Writer when 'file name' has no value.  
• ZipFile Read: "zip-filename" was added to the header of the output binary document.

• Resolved an issue with File Writer failing to timeout when attempting to connect to an SFTP location that was unreachable.
• Resolved an issue with File Reader failing validation is FTP connection password had special characters.
• Support for WASB and WASBS in Directory Browser, Multi File Reader, File Delete, File Poller, ZipFile Read, and ZipFile Write Snaps.

• Feature: Enhanced File Delete Snap to support S3 protocol.
• Resolved an issue with smb pathname failing when directory name contained a space.
• Resolved an issue with a pipeline  showing as successful when writing to a project to which it had no access.
• Resolved an issue with Directory Browser not returning any data when target files were not present. The option, Ignore empty result, now allows you to determine whether or not to output an empty document of no data is passed through.

• File Reader and File Writer now support Azure Blob as a file protocol.
• Support for specifying another Windows drive path added to Directory Browser, File Poller, Multi File Reader, File Reader, and File Writer.
• File Delete no longer displays user credentials in error messages.
• Resolved an issue with File Poller failing to properly determine directories versus files in an S3 path.

4.2.2

• Resolved an issue with the ZipFile Read Snap failing to decompress a zip file.
• Resolved an issue with MultiFile Reader not able to read UNC-syntax local files in Windows Groundplex.
• Resolved an SFTP I/O error with File Read.
• Resolved an issue in SFTP pooling when a large number of connections are opened for both pooling enabled and disabled.
• Resolved an issue in File Poller where setting Polling timeout to a negative number below -1 caused the Snap to run forever.
• Resolved an issue in AES Snap with 256 bit keys.
• Resolved a file size issue in File Writer with the Dropbox FTP site.

• NEW! File Poller Snap. This Snap polls a directory looking for files matching the specified pattern.
• PGP Decrypt now supports files that are encrypted and signed.
• Resolved Directory Browser unable to support file protocol on Windows Groundplex
• Resolved Multi File Reader File protocol is not working for Windows Groundplex

• Support for S3 Server-side encryption. See AWS S3 Account at Configuring Binary Accounts.

• ZipFile Read Snap hangs when the File property is evaluated to a binary object too large to be a file URL
• Directory Browser: Unexpected results when using SFTP protocol paths
• SMB ClearText credentials visible for incorrect connection string

• File writer Snap - Null pointer exception resolved - Better reporting on ftps errors
• Directory Browser Snap listed wrong path for S3 sub-directories

• Directory Browser: added pass-through capability
• File Delete: resolved an error writing to network stream
• File Writer: The output view field "fileAction" has changed to "result", web HDFS enhancements

• Compress and Decompress: Resolved an issue with pipeline preview not limiting the amount of data available in preview.
• File Writer: Improved error messages.

Multi File Reader: Resolved an issue were some files were read twice if Include subfolders was selected when using the S3 protocol.

Tooltip cleanup

File Writer now lets you set file permissions.

NEW! ZipFile Read and Write

File Writer Snap updated to allow for appending, overwriting, ignoring or erroring out if the file already exists.

• NEW! Encrypt and Decrypt Snaps for AES
• NEW! Encrypt and Decrypt Snaps for Blowfish
• NEW! Encrypt and Decrypt Snaps for Twofish
• NEW! Directory Browser
• Execute during preview was added to the File Writer Snap.

An output view was added to File Writer.

• NEW! File Delete Snap
• NEW! PGP Decrypt and PGP Encrypt

• NEW! Compress and Decompress
  
• Writer Snaps can now be enabled to write during preview.
• Reader Snaps allow you to pass through data from the input view. The output of the reader will provide a new document element holding the original data from the input view (under original).

There are now separate Snaps for reading a single file into SnapLogic and reading multiple files (File Reader and Multi File Reader).

Initial release. Read/write support for popular data formats: Binary, CSV, HTML, JSON