Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Table of Contents
minLevel1
maxLevel2
absoluteUrltrue

Use Case 1 (Creating and Sending Invoices to an EDI Application)

Problem

...

Problem

Querying huge amount of data from an external location, such as Azure Blob storage can be a rigorous task as well as time-consuming if file size is high. The resulting data can be inaccurate and there is a possibility of data loss.

Solution

Using the XYZ Azure Synapse SQL Snap Pack, you can automate the process of creating and sending invoices to an EDI application. This Pipeline demonstrates how this can be done...querying process for loading bulk data. This solution is efficient as it is easy to query your storage data and is cost-effective as well as the data processing works on the pay-as-you-go model. Learn more about the Azure Synapse Analytics pricing.

...

Download this solution. 

Understanding the Solution

Prerequisites:

...

Steps

Configuration

Validated Output

Step 1: Configure the Azure Synapse SQL - Bulk Load Snap.

a. The BulkLoad_TC13_VK target table into which the data from Blob Storage should load.

b. The File Name Pattern /*.csv loads only the files names with .csv file extension.

c. The Copy Argument With MAXERRORS=1000 ignores 1000 record errors and continues with the Snap execution.

d. Validate the Pipeline. You can view the following query in the output.

COPY INTO "dbo"."BulkLoad_TC13_VK" FROM 'https://snpqadatalake1.blob.core.windows.net/snpqadatalake1file/vamsinew/ID1ID2CSV (1).csv','https://snpqadatalake1.blob.core.windows.net/snpqadatalake1file/vamsinew/IDFile (1).csv' WITH ( FILE_TYPE = 'CSV' ,CREDENTIAL = (***hidden***),MAXERRORS = 1000 )

Image Added

Image Added

Step 2: Configure the JSON Formatter Snap to format the output into JSON data.

Image Added

Step 3: Configure the File Writer Snap to write the file to SLDB.

Image Added

Step 4: Execute the Pipeline.

Downloads

Attachments
patterns*.slp, *.zip

...