In this article
| Table of Contents | ||||
|---|---|---|---|---|
|
Overview
Use this Snap to read data from upstream Snaps and prepare them for writing into Adobe Enterprise Platform (AEP) as JSON or Parquet files.
Prerequisites
- The schema that you want to use with this Snap should exist in the Adobe Experience Platform. If you do not have the schema ready, create it using the Adobe Experience Platform user interface before using this Snap.
Limitations and Known Issues
- While the Schema Name, Dataset Name, and Connectors Object Name fields are expression enabled, their purpose is to read Pipeline parameter values. This Snap does not read values from upstream Snaps. You can, however, add this Snap in Pipelines with other Snaps to execute them in a sequence.
Snap Input and Output
| Input/Output | Type of View | Number of Views | Compatible Upstream and Downstream Snaps | Description |
|---|---|---|---|---|
| Input | Document |
|
| Each input document contains documents that must be prepared for writing to Adobe Experience Platform. |
| Output | Document |
|
| Each output document lists out the absolute path to the reformatted output file. |
Snap Settings
| Parameter Name | Data Type | Description | Default Value | Example | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Label | String | Required. The name for the Snap. Modify this to be more specific, especially if there is more than one of the same Snap in the Pipeline. | Adobe Experience Platform File Generator | Read and Reformat Documents from S3 | ||||||||||||
| Schema Name | String | Required. The name of the schema to be used to create the output Parquet or JSON file. This schema should already have been created in AEP.
| N/A | Snaplogic Demo Experience Event Schema | ||||||||||||
| Schema Type | String | Required. The schema type that you want to use to create the output file. Click to select from the following options:
| Parquet | JSON | ||||||||||||
| String |
| Execute only | Validate and Execute |
Troubleshooting
| Error | Reason | Resolution |
|---|---|---|
| Invalid Snap configuration | Property Schema Name is null or missing. This typically means that either you did not specify the schema that you want to use or misspelled the schema name. | Provide a vaild valid schema name. |
| Unable to load the private key for the given alias | Keystore file not found at the given keystore path. | This error appears when the account doesn't have valid values. Ensure that the keystore path, keystore passphrase, private key alias, and private key passphrase are correct. Also, you must upload the keystore in the Keystore Path that you specify. |
| Unable to obtain access token | This message appears when the endpoint account details are incorrect. | Check your Organization ID, Technical Account ID, Client Client ID, Client Client Secret Key details and details and try again. |
Examples
Writing JSON Documents from S3 as Parquet Files in AEP
In this example, you retrieve a JSON document from your AWS S3 account and write it as a parquet file in a target AWS S3 database.
You design your Pipeline as follows:
S3 File Reader
You add an S3 File Reader Snap to the Canvas and configure it to retrieve the file that you want to reformat for AEP.
You attach a JSON Parser Snap to the S3 File Reader, so you can work with the contents of the file in the Pipeline and view its contents.
Map fields to schema
You can now modify the contents of the file in SnapLogic and make it compliant with the AEP schema that you want to use. You use a Mapper Snap to do so.
- You use a random ID generator function to assign an ID ($_id) to each document.
- You place all the data in each incoming document into <tenant_id> (for example, _snaplogicinc).
- You use the Date.now function to create a $timestamp field associated with each document.
The preview the output of the Mapper Snap to check whether documents are structured as you wanted them. You see that each output document contains three high-level fields:
- A document ID, labeled "_id".
- The original data, labeled "_snaplogicinc".
- A timestamp, labeled "timestamp".
Adobe Experience Platform File Generator
You add an Adobe Experience Platform File Generator Snap, and configure it to create an AEP-compliant Parquet file using the documents coming in from the Mapper Snap.
Once execution succeeds, the Snap output lists the location of the Parquet file created.
Read file from disk
You use a File Reader Snap to read the Parquet file created by the AEP File Generator Snap.
The output of the Snap cannot be read, as it is in Parquet format; but you can see that the Read operation succeeded.
Write file to S3
You use an S3 File Writer to write the file to an S3 directory.
Once the write operation is successful, the Snap lists out details associated with the file written.
Downloads
| Attachments | ||
|---|---|---|
|
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|