On this Page

Before you Begin

You may need to clear your browser cache before you log into the latest SnapLogic Intelligent Integration Platform (IIP). See the applicable documentation for your browser:

• Chrome
• Firefox

You may see an Upgrade Notice dialog informing that you will be logged out immediately to complete the update process. Because the SnapLogic platform consists of multiple applications, you are likely to see this message a few times.

Accounts and Platform Updates

Some accounts may have a fixed time for refresh tokens, such as Google accounts that must be refreshed every hour. If that refresh needs to occur when the platform is down for an update, the refresh does not occur. To prevent these accounts from failing after a new platform deployment, we recommend you to refresh your accounts before the designated downtime. If the platform is down for longer than an hour, you will need to refresh those accounts once the platform is back online.

Platform Enhancements

General

• Introducing Data Catalog as a Service. The catalog service adds a Table asset type which can store data location and schema information for files in an external file system. In conjunction with the Catalog Insert and Catalog Query Snaps, this feature provides data integrators with the tools required to interconnect and analyze enterprise metadata. For eXtremeplex users, this feature includes using the Catalog Writer and Catalog Reader Snaps (see Big Data Snaps). You can gain visibility of data (Table and Column Structure) stored in the Data Lake (usually in an AWS S3 account) for building Capture/Ingest/Conform/Refine Pipelines in a single UI.

• Enhanced support in Runtime Archive to enable access to logs of all pipeline executions for auditing purposes. Prior to this release, logs for only 100 child pipelines per parent pipeline were archived; this limitation has been removed.

• Added High Availability configurations for Triggered Tasks. Users relying on running their pipelines via the Task method can now deploy Triggered Tasks to multiple Snaplexes using one of the Override URLs to achieve fault tolerance in the event of a failure. 

• Added the API Usage Detail feature to API Meters in Manager, enabling administrators to monitor API workflows in their Org to prevent exceeding usage. You can now track API usage both daily and concurrently both summarized in pie graphs, along with a table of the top ten API uses.

• Enhanced the recent SnapLogic release process to enable admin users to automatically update their respective Orgs immediately when a new release is available. Starting with the SnapLogic 4.16 release (around mid-February, 2019), you will be able to configure their SnapLogic org to update automatically to 4.16 and later in the Manager tab. The current process of manually updating all Orgs 35 days after GA will continue. 

• Enhanced the API download functionality by giving an option to download tasks (APIs) in .yaml or .json formats for viewing in Swagger.

• Added two API parameters. For the GET /api/1/rest/public/snaplex/config/<path:snaplexpath> API call, you can set an expiration time in hours for how long the URL is valid. For the POST /api/1/rest/public/users API, the returned document now has the basic_user field, which displays the last user login. 

UI Enhancements

Manager

• You can now extract enterprise metadata from AWS S3 and store them as Table Assets in Manager. Metadata is stored in Tables accessible via projects; alternatively, you can search under Assets in Manager. Once opened, you can view partition keys and schema of the table data.
• API Meters in Manager enables you to monitor API usage. Clicking Usage Detail displays a table of the top ten most run API calls, as well as pie graphs summarizing the usage. Both daily and concurrent views are available. 
• Set your version of SnapLogic to upgrade automatically to the next available release of SnapLogic via the Auto Update Check box. With 4.15, this feature enables an automatic update to the 4.16 release.
• Enhanced the Activity Log UI, enabling you to view the number of log entries available, and view log entries in sets of 100.

Dashboard

• Updated the Snaplex health icons in the Dashboard Health Wall page to the following:
  
  •  Up and running
  •  Alert
  •  Not running
  •  Error status for nodes

Snap Enhancements

New Snap Packs

SnapLogic Data Science (Machine Learning): Introducing SnapLogic Data Science Snaps that enable you to conduct machine learning operations on your datasets. Using these Snap Packs, you can conduct data cleaning, data modeling, and data analytics. The Data Science solution includes the following: 

• ML Data Preparation Snap Pack: Perform preparatory operations on datasets such as data type transformation, data cleanup, sampling, shuffling, and scaling. Snaps in this Snap Pack are: 
  • Categorical to Numeric: Convert categorical columns into numeric columns by using integer coding or one hot encoding.
  • Clean Missing Values: Replace missing values in datasets by dropping or imputing values.
  • Date Time Extractor: Extract components from datetime objects.
  • Numeric to Categorical: Convert numeric columns into categorical columns by using custom ranging or binning.
  • Sample: Generate sample datasets from an input dataset using sampling algorithms.
  • Scale: Scale values in columns to specify ranges or apply statistical transformations.
  • Shuffle: Randomize the order of the row data in the dataset.
  • Type Converter: Determine types of values in columns. There are four supported types: integer, floating point, text, and datetime.

• ML Core Snap Pack: Perform data modeling operations such as model training, cross-validation, and model-based predictions. Additionally, you can also execute Python scripts remotely. Snaps in this Snap Pack are: 
  • Cross Validator - Classification: Cross validate classification data models using algorithms and choose the best possible algorithm for the dataset.
  • Cross Validator - Regression: Cross validate regression data models using algorithms and choose the best possible algorithm for the dataset.
  • Predictor - Classification: Predict the unlabeled data in a classification dataset using a data model.
  • Predictor - Regression: Predict the unlabeled data in a regression dataset using a data model.
  • Trainer - Classification: Train/Generate a data model for a classification dataset using algorithms.
  • Trainer - Regression: Train/Generate a data model for a regression dataset using algorithms.
  • Remote Python Script: Execute Pythons scripts remotely on the Python server.

• ML Analytics Snap Pack: Perform analytic operations such as data profiling. Snaps in this Snap Pack are:
  • Profile: Generate statistical information for the input dataset.
  • Type Inspector: Analyze the data types present in the dataset.

Read more about Data Science Snaps in Getting Started with ML.

EDI Snap Pack: Integrate with SnapLogic B2B gateway and create/read EDI transaction data. Snaps in this Snap Pack are:

• Get EDI Transactions: Fetch EDI transaction information for a specified transaction ID.
• Create EDI Transaction: Create an EDI transaction for the specified Partner.

Users can automate reporting activities and creation of EDI transactions using these Snaps in Ultra tasks. Users can also generate analytics information to aid in their decision-making efforts by using the Analytic Snap Packs in conjunction with these Snaps. 

Workday Prism: Provides the ability to integrate data in Workday Prism to generate self-service Analytical reports that empower HR and finance teams to make business decisions. This Snap Pack includes the following Snap:

• Bulk Load Snap: Use this Snap to upload multiple records/files in the SnapLogic pipeline from multiple sources (such as ERP, CRM, databases, and Workday HCM) into Workday Prism, and derive insightful visual analytics on the data.

Data Catalog Snap Pack: Added the following Snaps:

• Catalog Insert: Insert metadata information into metadata catalogs tables. You can also use this Snap to further enrich your metadata catalog with additional details, ensuring that you have all the meta-information you need to get actionable insights from your data.
• Catalog Query: Query metadata information from the metadata catalog. You can use this Snap to retrieve metadata from tables in your catalog and select the elements that you want to reuse. For example, you could use a combination of the Catalog Query and Catalog Insert Snaps to retrieve metadata from one set of tables and create or enrich other tables in your catalog.

New Snaps in Existing Snap Packs

Snowflake Snap Pack: Added the following Snaps:

• Snowflake Multi Execute: Execute multiple Data Modifying Language (DML) and Data Definition Language (DDL) queries in the Snowflake database. 
• Snowflake SCD2: Create SCD2 type field-historization in Snowflake tables.

Users can leverage these metadata Snaps by just referring to the table names without needing to know the actual file locations.

Hadoop Snap Pack: Added the following Snaps:

• HDFS ZipFile Writer: Create and write ZIP files directly into the Hadoop file system. Use this Snap to collect files from various locations, zip them all together, and insert them directly into the Hadoop file system at one go.
• HDFS ZipFile Reader: Read the contents of ZIP files that are located in the Hadoop file system. This Snap enables you to read all the ZIP files in a Hadoop directory that match specific criteria and make the unzipped files available to downstream Snaps.
  

Updated Snap Packs

Analytics

• Anaplan Account: Anaplan is extending the deadline to migrate to Certificate Authority (CA) certificates to December, 2019. To support this, on December 8, 2018, Anaplan will update their production certificate. This will replace validation for Anaplan certificates and existing Anaplan certificates will cease to work once this certificate has been issued. Anaplan still recommends that customers update their integration clients to a version that supports Certificate Authentication 2.0 which uses a CA certificate. To support Anaplan v2.0 API, we have enhanced the Anaplan Account configuration.
  

  Depending on the authentication method you use, perform the steps mentioned in the table below: Authentication method Action required with the updated Anaplan Snap Pack Basic username and password No action required CA certificate Update the Anaplan Account settings and enter relevant information in External certificate file, External certificate contents, and External private key properties. Click here for details. Self-signed Anaplan Certificate Once you upgrade your Snaplexes to 4.15, you must roll back to the previous version of the Anaplan Snap Pack. Anaplan certificate expires on December 8, 2018. You must download the new certificate and update the Anaplan Snap account with the new certificate.

  
  

• Anaplan Snap Pack: Added retry capability to Anaplan Action, Read, Write, and Upload Snaps that allow you to handle connection timeouts during Pipeline execution.

Binary

• Binary Snap Pack: Added a new property, Binary Header Properties, to the CSV Formatter Snap. This property can be configured with the binary headers and their values in the incoming document. 

Core Snap

• REST Snap Pack: Added the ability to upload multiple files using REST Post, Put, and Patch Snaps.

Data

• Hadoop Snap Pack: Enhanced the Parquet Reader and Parquet Writer Snaps to support the Data Catalog Snap Pack.
• Hive Snap Pack: Added Zookeeper support to the Configuring Hive Accounts. You can now set up your Hive account using Zookeeper.

Database

• Azure SQL Snap Pack: Added two new properties, Schema source and Use type default, to the PolyBase Bulk Load Snap that lets you specify the schema source and handle any missing values in the input documents.
• Snowflake Snap Pack:
  • Enhanced the Snowflake Snap Pack to reflect Azure certification.
  • Enhanced the Snowflake Bulk Upsert Snap to improve the Snap performance. 

Enterprise 

• JMS Snap Pack: Added HornetQ support for the JMS Producer and JMS Consumer Snaps.

Known Issues

• Parquet Writer: 
  • Unable to partition data when the Parquet schema has all the required fields.
  • Unable to generate a schema from the source when the input columns contain 'null' value in the first document.
• REST Post: Unable to support UTF-8 characters for Filename to be used property. Instead, the value renders as question  marks.
• Redshift Multi Execute: SQL binding does not work with Expression OFF.
• Teradata FastExport: Special characters appear in output after exporting table data is exported.

Limitations

• Hive Snap Pack: The Hive Snap Pack does not validate with Apache Hive JDBC v1.2.1 jars or earlier because of a defect in Hive. HDP 2.6.3 and HDP 2.6.1 run on Apache Hive JDBC v1.2.1 jars. To validate Snaps that must work with HDP 2.6.3 and HDP 2.6.1, use JDBC v2.0.0 jars. 

Big Data Enhancements

General

• Copy an eXtremeplex: Enables you to reuse existing configuration, while creating a new eXtremeplex in the Manager tab.
• Schema Source Options: You now have the option to ingest the Schema in Spark SQL 2.x CSV and JSON parser Snaps via Inferred Schema (automatically) or from a Hive Metastore (selected by the user), in addition to entering it manually.

• Added external validation functionality to Hadooplex: Validating pipelines externally as a regular Spark job enables users to execute multiple Spark Pipelines concurrently as opposed to a single Pipeline currently from a single Hadooplex container. Ensure that there are sufficient resources in the Hadoop cluster to run multiple Pipelines concurrently.

New Snaps in Spark 2.x Snap Pack

Spark SQL 2.x Snap Pack: Added the following new Snaps:

• Avro Formatter: Enables you to format data input from an upstream Snap in the Avro data format.
• Avro Parser: Parse Avro data to pass on to n downstream Snap in a different data format.
• JSON Formatter: Format data input from an upstream Snap in the JSON data format.
• JSON Parser: Parse Avro data to pass on to a downstream Snap in a different data format.
• Catalog Writer: Part of the Data Catalog service, this Snap enables you to format data per the selected format type and insert metadata into the target table path.
• Catalog Reader: Queries a table as well as reads and parses metadata information to the downstream table.

Known Issue

• When creating tables using the Catalog Reader and Catalog Writer Snaps, any user can overwrite the contents of the table or add it to any project regardless of permissions.

UAT Delta

4.15 Dot Releases