Snap Compatibility
Snaps in the Hadoop Snap Pack work with different accounts and protocols per the following table:
Snap | AWS S3 Account for Hadoop | AWS S3 Dynamic Account | Azure Storage for Hadoop | Azure Data Lake | Azure Data Lake Storage Gen 2 OAuth2 | Kerberos | Protocols Supported |
---|---|---|---|---|---|---|---|
Hadoop Directory Browser | ✖ | ✖ | ✖ | ✖ | ✔ | ✔ |
|
HDFS Reader | ✖ | ✖ | ✔ | ✔ | ✔ | ✔ |
|
HDFS Writer | ✖ | ✖ | ✔ | ✔ | ✔ | ✔ | WASB, WASBS, ABFS, HDFS, ADL |
HDFS ZipFile Reader | ✖ | ✖ | ✖ | ✖ | ✔ | ✔ | ABFS, HDFS |
HDFS ZipFile Writer | ✖ | ✖ | ✖ | ✖ | ✔ | ✔ | ABFS, HDFS |
ORC Reader | ✔ | ✔ | ✖ | ✖ | ✔ | ✖ | AWS S3, WASB, WASBS, ABFS, HDFS |
ORC Writer | ✔ | ✔ | ✖ | ✖ | ✔ | ✖ | AWS S3, ABFS, HDFS |
Parquet Reader | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | AWS S3, WASB, WASBS, ABFS, HDFS, ADL |
Parquet Writer | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | AWS S3, WASB, WASBS, ABFS, HDFS, ADL |
RC File Formatter, RC File Parser, Sequence Formatter, Sequence Parser | ✖ | ✖ | ✖ | ✖ | ✖ | ✖ | N/A |
Configuring Hadoop Accounts
You can configure Hadoop accounts in SnapLogic using either the Designer or Manager.
Using SnapLogic Designer
Selecting an existing account
SnapLogic organizes and displays all accounts to which you have access, sorting them by account type and location. To select an existing account:
- Click the
- Click .
Creating an account
- Click Add Account in the Account Reference dialog.
- Select the Location in which you want to create the account, select the account type, and click Continue. The Add Account dialog associated with the account type appears.
Enter the required account details. For detailed guidance on how to provide information associated with each account type, use the following links:
Child pages (Children Display) Info Enter additional information on this account in the Notes field of the Info tab. This will help you–and other users–understand the purpose of the account, especially if there are multiple accounts of the same type.
- Click Validate to verify the account, if the account type supports validation.
- Click Apply to complete configuring the binary account.
Using SnapLogic Manager
Use Manager to create accounts without associating them immediately with Pipelines.
Note |
---|
Accounts in SnapLogic are associated with projects. You can use accounts created in other projects only if you have at least Read access to them. |
In the left pane, browse to the project in which you want to create the account and click Create > Account > Binary, followed by the appropriate account type.
- The Create Account dialog associated with the selected account type appears.
- Repeat the steps numbered 3 through 5 in the previous section.
Warning |
---|
Avoid updating account credentials while Pipelines using that account are executing. This may lead to unexpected results, including locking your account. |
Insert excerpt | ||||||
---|---|---|---|---|---|---|
|