Azure Data Lake Storage Gen2 Account Configuration in Azure Portal to use in Binary Snap Pack

Overview

You can use this account type to connect the Snaps from Binary Snap Pack with data sources that use the Azure Data Lake Gen2 Storage account. An Azure storage platform includes data services like Blobs, Files, Queues, Tables, and Disks. This article guides you through creating a storage account using the Azure portal.

Prerequisites

An Azure account with a valid subscription to create the Storage Account.

Key steps

The following steps help you to create and register your application in the Azure Data Lake Storage Gen2 account to be used in the Snap.

Create the Azure Data Lake Storage Gen2 Storage Account
Check Access control (IAM) for the Storage Account
Select an app based on the permission
Locate the Tenant ID and Client ID for the registered app
Locate the Client and secrets ID value for the registered app

Create the Azure Data Lake Storage Gen2 Storage Account

Log in to the Microsoft Azure Portal.
Select Create a resource under Azure services on the Portal Home page or hover over the Storage accounts icon and click + Create.
Select Storage accounts and specify the required details under each tab for the storage accounts. Refer to the storage details table for the configuration settings.
1. Basic: Project and Instance details.
2. Advanced: Security and Blob Storage details.
3. Networking: Network connectivity and routing details.
4. Data protection: Recovery and Tracking details.
5. Encryption: Encryption details.
6. Tags: Tag details.
7. Review: Complete details of the Storage account.
Specify the Project and Instance details as highlighted under the Basics tab.
In the Advanced tab, add details related to Security.
In the Networking tab, specify connectivity, endpoints, and routing details.
In the Data protection tab, add details related to account Recovery and Tracking.
In the Encryption tab, add details related to the encryption type.
You can specify Resource Manager tags in the Tags tab to help organize your Azure resources.
Click Review+Create to pass the validation.
If the validation passes, click Create to deploy the Resource.
Select Go to Resource, to view the details of the storage account.

The hierarchical namespace needs to be enabled to be a Gen2 account. Currently Binary Snaps support the Gen1 and Gen2 account. Refer to the screenshot provided below to enable the hierarchical namespace option.

Storage account details:

Tab Name	Field Name	Type	Description
Basics: Project details	Subscription	String	Required. Select the subscription to create the new storage account.
Basics: Instance details	Resource group	String	Required. You can create a new resource group for this storage account, or select an existing one. Learn more about , Resource groups.
Basics: Instance details	Storage account name	String	Specify a unique name for your storage account. Storage account names must be between 3 and 24 characters in length and may contain numbers and lowercase letters only.
Basics: Instance details	Region	Drop-down	Select the appropriate region for your storage account from the drop-down list. For more information, refer to Regions and Availability Zones in Azure. All the regions are not supported by all types of storage accounts or redundancy configurations. For more information, refer to Azure Storage redundancy.
Basics: Instance details	Performance	Radio button	Select the appropriate performance as per the purpose of your accounts. The options are: Standard: Select for general-purpose v2 storage accounts. This is a default option. This type of account is recommended by Microsoft. Premium: Select this option for scenarios requiring low latency. Under Premium, select the Premium account type to create. The following types of premium storage accounts are available: Block blobs: Recommended for scenarios with high transaction rates or that use smaller objects or require consistently low storage latency. File shares: Recommended for enterprise or high-performance scale applications. Use this account type if you want a storage account that supports both Server Message Block (SMB) and NFS file shares. Page blobs: Recommended for random read and write operations.
Basics: Instance details	Redundancy	Drop-down	Select your desired redundancy configuration. The options available are: LRS: Locally redundant storage - Recommended for non-critical scenarios. Low cost option with basic protection against server rack and drive failures. GRS: Geo-redundant storage - Recommended for backup scenarios. Intermediate option with failover capabilities in a secondary region. ZRS: Zone-redundant storage - Provides protection against data center level failures. Recommended for high availability scenarios. GZRS: Geo-zone-redundant storage - Optimal data protection storage that includes the offerings of both GRS and ZRS. Recommended for critical data scenarios. If you select a geo-redundant configuration (GRS or GZRS), your data is replicated to a data center in a different region.
Basics: Instance details	Make read access to data available in the event of regional unavailability.	Checkbox	Select this checkbox to get the read access to data in the secondary region. This option is available only for Geo-redundant storage and Geo-zone-redundant storage.
Advanced Security: Enables you to configure the security settings which will impact the storage account.	Require secure transfer for REST API operations	Checkbox	Optional. Require secure transfer to ensure that incoming requests to this storage account are made only via HTTPS (default). Recommended for optimal Security. For more information, refer to Require secure transfer to ensure secure connections.
	Allow enabling public access on containers.	Checkbox	Optional. Recommended to selectively enable public access on specific containers.
	Enable storage account key access	Checkbox	Optional. When enabled, this setting allows clients to authorize requests to the storage account using either the account access keys or an Azure Active Directory (Azure AD) account (default). For more information, refer to Prevent Shared Key authorization for an Azure Storage account.
	Default to Azure Active Directory authorization in the Azure portal	Checkbox	Optional. When enabled, the Azure portal authorizes data operations with the user's Azure AD credentials by default. For more information, refer to Default to Azure AD authorization in the Azure portal.
	Minimum TLS version	Drop-down	Required. Select the minimum version of Transport Layer Security (TLS) for incoming requests to the storage account. The default value is TLS version 1.2. For more information, refer to Enforce a minimum required version of Transport Layer Security (TLS) for requests to a storage account.
	Permitted scope for copy operations	Drop-down	When this property is enabled, the Azure portal authorizes requests to blobs, queues, and tables with Azure Active Directory by default.
	Enable hierarchical namespace	Checkbox	Optional. For more information, refer to Introduction to Azure Data Lake Storage Gen2. To use this storage account for Azure Data Lake Storage Gen2, enable the checkbox.
	Enable SFTP	Checkbox	Optional. Enable the use of Secure File Transfer Protocol (SFTP) to securely transfer of data over the internet. For more information, refer to Secure File Transfer (SFTP) protocol support in Azure Blob Storage.
	Enable network file system v3	Checkbox	Optional. Network File System Protocol is enablef for the storage account created. For more information, refer to Network File System (NFS) 3.0 protocol support in Azure Blob storage.
	Allow cross-tenant replication	Checkbox	Required. By default, users with appropriate permissions can configure object replication across Azure AD tenants. For more information, refer to Prevent replication across Azure AD tenants.
	Enable large file shares	Checkbox	Optional. Available only for standard file shares with the LRS or ZRS redundancies.
	Access tier	Radio button	Optional. Hot and Cold based on frequency of usage.
Networking	Network access	Radio button	Required. By default, incoming network traffic is routed to the public endpoint for your storage account.
Networking	Routing preference	Radio button	Required. The network routing preference specifies how network traffic is routed to the public endpoint of your storage account from clients over the internet.
Data protection	Enable point-in-time restore for containers	Checkbox	Provides protection against accidental deletion or corruption by enabling you to restore block blob data to an earlier state. For more information, refer to Point-in-time restore for block blobs. Enabling point-in-time restore also enables blob versioning, blob soft delete, and blob change feed.
	Enable soft delete for blobs	Checkbox	Optional. Soft delete enables you to recover blobs that were previously marked for deletion, including blobs that were overwritten.
	Enable soft delete for blobs	Checkbox	Optional. Blob soft delete protects an individual blob, snapshot, or version from accidental deletes or overwrites by maintaining the deleted data in the system for a specified retention period.
	Enable soft delete for containers	Checkbox	Container soft delete protects a container and its contents from accidental deletes by maintaining the deleted data in the system for a specified retention period. For more information, refer to Soft delete for containers (preview). Microsoft recommends enabling container soft delete for your storage accounts and setting a minimum retention period of seven days.
	Enable soft delete for file shares	Checkbox	Optional. Soft delete for file shares protects a file share and its contents from accidental deletes by maintaining the deleted data in the system for a specified retention period. For more information, refer to Prevent accidental deletion of Azure file shares. Microsoft recommends enabling soft delete for file shares for Azure Files workloads and setting a minimum retention period of seven days.
	Enable versioning for blobs	Checkbox	Optional. Blob versioning automatically saves the state of a blob in a previous version when the blob is overwritten. For more information, refer to Blob versioning. Microsoft recommends enabling blob versioning for optimal data protection for the storage account.
	Enable blob change feed	Checkbox	Optional. The blob change feed provides transaction logs of all changes to all blobs in your storage account, as well as to their metadata. For more information, refer to Change feed support in Azure Blob Storage.
	Enable version-level immutability support	Checkbox	Optional. Enable support for immutability policies that are scoped to the blob version. For more information, refer to Enable version-level immutability support on a storage account.
Encryption	Encryption type	Radio button	By default, data in the storage account is encrypted by using Microsoft-managed keys. You can rely on Microsoft-managed keys for the encryption of your data, or you can manage encryption with your own keys. The options available are: Microsoft-managed keys Customer-managed keys
Encryption	Enable support for customer-managed keys	Radio button	By default, customer managed keys can be used to encrypt only blobs and files. Set this option to All service types (blobs, files, tables, and queues) to enable support for customer-managed keys for all services. You are not required to use customer-managed keys if you choose this option.
Encryption	Enable infrastructure encryption	Checkbox	Required if Encryption type field is set to Customer-managed keys. The available options are: Select a key vault and key: Provides you an option to navigate to the key vault and key that you wish to use. Enter key from URI: Provides you an option with a field to enter the key URI and the subscription.
Encryption	User-assigned identity	Radio button	Required if Encryption type field is set to Customer-managed keys. If you are configuring customer-managed keys at create time for the storage account, you must provide a user-assigned identity to use for authorizing access to the key vault.
Encryption	Enable infrastructure encryption	Checkbox	Optional. By default, infrastructure encryption is not enabled.
Tags	Name Value Resource	Dropdown	Optional. Tags are name/value pairs that enable you to categorize resources and helps in billing purpose by the same tag to multiple resources and resource groups. Learn more about Tags.
Review	NA	Labels	Provides details of all the tabs to review and proceed to create the Resource (storage account).

Check Access to the Resource

From the navigation panel, select Access control (IAM) to check the apps that have permission to access the storage account created.

Create an Azure Portal Application for the Storage Account

From the left navigation menu on the home page, select the App registration option.
Under Overview, register a new app or use an existing application. From the search option, select the registered app and make a note of the Application (Client) ID and Tenant ID.
For a new application, on the Overview page, select Add > App registration. On the Register an application page, specify the Name of the application and click Register. The Application (client) ID is required for the new account.
Under Manage, click Certificates & secrets.
On the Certificates & secrets page, click + New client secret.
On the Add a client secret page, enter the Description and Expires and click Add. Make a note of the Client & secret value that is generated.

Specify Credentials in the Snap Account

Navigate to the Snap of your choice and set up the account with the following details noted in the earlier steps.
- Tenant ID
- Access ID
- Secret key
Click Save and validate the Snap with the Azure Data Lake Gen2 account.

To upgrade an existing storage account to Gen2 Account, select Settings > Data Lake Gen2 upgrade option from the Azure portal. Binary snaps currently support both Gen2 and Gen1 (ADL & Blob storage) accounts.

Related Content

Storage account overview.