Azure Data Lake Storage Gen2 Account Configuration in Azure Portal to use in Binary Snap Pack
Overview
You can use this account type to connect the Snaps from Binary Snap Pack with data sources that use the Azure Data Lake Gen2 Storage account. An Azure storage platform includes data services like Blobs, Files, Queues, Tables, and Disks. This article guides you through creating a storage account using the Azure portal.
Prerequisites
An Azure account with a valid subscription to create the Storage Account.
Key steps
The following steps help you to create and register your application in the Azure Data Lake Storage Gen2 account to be used in the Snap.
Create the Azure Data Lake Storage Gen2 Storage Account
Check Access control (IAM) for the Storage Account
Select an app based on the permission
Locate the Tenant ID and Client ID for the registered app
Locate the Client and secrets ID value for the registered app
Create the Azure Data Lake Storage Gen2 Storage Account
Log in to the Microsoft Azure Portal.
Select Create a resource under Azure services on the Portal Home page or hover over the Storage accounts icon and click + Create.
Select Storage accounts and specify the required details under each tab for the storage accounts. Refer to the storage details table for the configuration settings.
Basic: Project and Instance details.
Advanced: Security and Blob Storage details.
Networking: Network connectivity and routing details.
Data protection: Recovery and Tracking details.
Encryption: Encryption details.
Tags: Tag details.
Review: Complete details of the Storage account.
Specify the Project and Instance details as highlighted under the Basics tab.
In the Advanced tab, add details related to Security.
In the Networking tab, specify connectivity, endpoints, and routing details.
In the Data protection tab, add details related to account Recovery and Tracking.
In the Encryption tab, add details related to the encryption type.
You can specify Resource Manager tags in the Tags tab to help organize your Azure resources.
Click Review+Create to pass the validation.
If the validation passes, click Create to deploy the Resource.
Select Go to Resource, to view the details of the storage account.
The hierarchical namespace needs to be enabled to be a Gen2 account. Currently Binary Snaps support the Gen1 and Gen2 account. Refer to the screenshot provided below to enable the hierarchical namespace option.
Storage account details
Tab Name | Field Name | Type | Description |
---|---|---|---|
Basics: Project details | Subscription | String | Required. Select the subscription to create the new storage account. |
Basics: Instance details | Resource group | String | Required. You can create a new resource group for this storage account, or select an existing one. Learn more about , Resource groups. |
Basics: Instance details | Storage account name | String | Specify a unique name for your storage account. Storage account names must be between 3 and 24 characters in length and may contain numbers and lowercase letters only. |
Basics: Instance details | Region | Drop-down | Select the appropriate region for your storage account from the drop-down list. For more information, refer to Regions and Availability Zones in Azure. All the regions are not supported by all types of storage accounts or redundancy configurations. For more information, refer to Azure Storage redundancy. |
Basics: Instance details | Performance | Radio button | Select the appropriate performance as per the purpose of your accounts. The options are:
|
Basics: Instance details | Redundancy | Drop-down | Select your desired redundancy configuration. The options available are:
If you select a geo-redundant configuration (GRS or GZRS), your data is replicated to a data center in a different region. |
Basics: Instance details | Make read access to data available in the event of regional unavailability. | Checkbox | Select this checkbox to get the read access to data in the secondary region. |
Advanced Security: Enables you to configure the security settings which will impact the storage account. | Require secure transfer for REST API operations | Checkbox | Optional. Require secure transfer to ensure that incoming requests to this storage account are made only via HTTPS (default). Recommended for optimal Security. For more information, refer to Require secure transfer to ensure secure connections. |
| Allow enabling public access on containers. | Checkbox | Optional. Recommended to selectively enable public access on specific containers. |
| Enable storage account key access | Checkbox | Optional. When enabled, this setting allows clients to authorize requests to the storage account using either the account access keys or an Azure Active Directory (Azure AD) account (default). For more information, refer to Prevent Shared Key authorization for an Azure Storage account. |
| Default to Azure Active Directory authorization in the Azure portal | Checkbox | Optional. When enabled, the Azure portal authorizes data operations with the user's Azure AD credentials by default. For more information, refer to Default to Azure AD authorization in the Azure portal. |
| Minimum TLS version | Drop-down | Required. Select the minimum version of Transport Layer Security (TLS) for incoming requests to the storage account. The default value is TLS version 1.2. For more information, refer to Enforce a minimum required version of Transport Layer Security (TLS) for requests to a storage account. |
| Permitted scope for copy operations | Drop-down | When this property is enabled, the Azure portal authorizes requests to blobs, queues, and tables with Azure Active Directory by default. |
| Enable hierarchical namespace | Checkbox | Optional. For more information, refer to Introduction to Azure Data Lake Storage Gen2. |
| Enable SFTP | Checkbox | Optional. Enable the use of Secure File Transfer Protocol (SFTP) to securely transfer of data over the internet. For more information, refer to Secure File Transfer (SFTP) protocol support in Azure Blob Storage. |
| Enable network file system v3 | Checkbox | Optional. Network File System Protocol is enablef for the storage account created. For more information, refer to Network File System (NFS) 3.0 protocol support in Azure Blob storage. |
| Allow cross-tenant replication | Checkbox | Required. By default, users with appropriate permissions can configure object replication across Azure AD tenants. For more information, refer to Prevent replication across Azure AD tenants. |
| Enable large file shares | Checkbox | Optional. Available only for standard file shares with the LRS or ZRS redundancies. |
| Access tier | Radio button | Optional. Hot and Cold based on frequency of usage. |
Networking | Network access | Radio button | Required. By default, incoming network traffic is routed to the public endpoint for your storage account. |
Routing preference | Radio button | Required. The network routing preference specifies how network traffic is routed to the public endpoint of your storage account from clients over the internet. | |
Data protection | Enable point-in-time restore for containers | Checkbox | Provides protection against accidental deletion or corruption by enabling you to restore block blob data to an earlier state. For more information, refer to Point-in-time restore for block blobs. |
Enable soft delete for blobs
| Checkbox | Optional. Soft delete enables you to recover blobs that were previously marked for deletion, including blobs that were overwritten. | |
Enable soft delete for blobs | Checkbox | Optional. Blob soft delete protects an individual blob, snapshot, or version from accidental deletes or overwrites by maintaining the deleted data in the system for a specified retention period. | |
Enable soft delete for containers | Checkbox | Container soft delete protects a container and its contents from accidental deletes by maintaining the deleted data in the system for a specified retention period. For more information, refer to Soft delete for containers (preview). | |
Enable soft delete for file shares | Checkbox | Optional. Soft delete for file shares protects a file share and its contents from accidental deletes by maintaining the deleted data in the system for a specified retention period. For more information, refer to Prevent accidental deletion of Azure file shares. | |
Enable versioning for blobs | Checkbox | Optional. Blob versioning automatically saves the state of a blob in a previous version when the blob is overwritten. For more information, refer to Blob versioning. | |
Enable blob change feed | Checkbox | Optional. The blob change feed provides transaction logs of all changes to all blobs in your storage account, as well as to their metadata. For more information, refer to Change feed support in Azure Blob Storage. | |
Enable version-level immutability support | Checkbox | Optional. Enable support for immutability policies that are scoped to the blob version. For more information, refer to Enable version-level immutability support on a storage account. | |
Encryption | Encryption type | Radio button | By default, data in the storage account is encrypted by using Microsoft-managed keys. You can rely on Microsoft-managed keys for the encryption of your data, or you can manage encryption with your own keys. The options available are:
|
Encryption | Enable support for customer-managed keys | Radio button | By default, customer managed keys can be used to encrypt only blobs and files. Set this option to All service types (blobs, files, tables, and queues) to enable support for customer-managed keys for all services. You are not required to use customer-managed keys if you choose this option. |
Encryption | Enable infrastructure encryption | Checkbox | Required if Encryption type field is set to Customer-managed keys. The available options are:
|
Encryption | User-assigned identity | Radio button | Required if Encryption type field is set to Customer-managed keys. If you are configuring customer-managed keys at create time for the storage account, you must provide a user-assigned identity to use for authorizing access to the key vault. |
Encryption | Enable infrastructure encryption | Checkbox | Optional. By default, infrastructure encryption is not enabled. |
Tags | Name Value Resource | Dropdown | Optional. Tags are name/value pairs that enable you to categorize resources and helps in billing purpose by the same tag to multiple resources and resource groups. Learn more about Tags. |
Review | NA | Labels | Provides details of all the tabs to review and proceed to create the Resource (storage account). |
Check Access to the Resource
From the navigation panel, select Access control (IAM) to check the apps that have permission to access the storage account created.
Create an Azure Portal Application for the Storage Account
From the left navigation menu on the home page, select the App registration option.
Under Overview, register a new app or use an existing application. From the search option, select the registered app and make a note of the Application (Client) ID and Tenant ID.
For a new application, on the Overview page, select Add > App registration. On the Register an application page, specify the Name of the application and click Register. The Application (client) ID is required for the new account.
Under Manage, click Certificates & secrets.
On the Certificates & secrets page, click + New client secret.
On the Add a client secret page, enter the Description and Expires and click Add. Make a note of the Client & secret value that is generated.
Specify Credentials in the Snap Account
Navigate to the Snap of your choice and set up the account with the following details noted in the earlier steps.
Tenant ID
Access ID
Secret key
Click Save and validate the Snap with the Azure Data Lake Gen2 account.
Related Content
Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.