...
You can use this Snap to convert documents to the Parquet format and write the data to HDFS, ADL (Azure Data Lake), ABFS (Azure Data Lake Blob File Storage Gen 2), WASB (Azure storage), or an S3 bucket. This Snap supports a nested schema such as LIST and MAP. You can also use this Snap to write schema information to the Catalog Insert Snap.
This Snap supports HDFS, ADL (Azure Data Lake), ABFS (Azure Data Lake Blob File Storage Gen 2), WASB (Azure storage), and S3 protocols.
...
Download
hadoop.dll
andwinutils.exe
​https://github.com/cdarlint/winutils/tree/master/hadoop-3.2.2/bin (SnapLogic’s Hadoop version is 3.2.2)Create a temporary directory.
Place the
hadoop.dll
andwinutils.exe
files in this path:c:\winutils\bin
Set the environment variable
HADOOP_HOME
to point toc:\winutils
Add
c:\winutils\bin
to the environment variable PATH as shown below:Add the JVM options in the Windows Snaplex:
jcc.jvm_options= -Djava.library.path=Cc:\winutils\testbin
If you already have an existing
jvm_options
, then add:"-Djava.library.path=C:\winutils\testbin"
after the space.
For example:jcc.jvm_options = -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000 -Djava.library.path=Cc:\winutils\testbin
Restart the JCC for configurations to take effect.
...