Azure Data Lake

Prev Next

Overview

Replicate Preview enables efficient data movement into Azure Data Lake Storage Gen2, allowing it to be configured as a scalable target for analytical workloads. Users can quickly define replication pipelines by selecting source tables and configuring target storage paths, enabling structured data to be delivered to the data lake with minimal setup and operational overhead. The platform simplifies pipeline creation while ensuring consistent and reliable data delivery into cloud-based storage.

Note

Azure SQL Database is currently supported as both a source and target endpoint in Replicate Preview, limited to Snapshot (full load) replication mode, with Change Data Capture (CDC) and streaming replication not yet supported.

Connection Considerations

If you do not have a connection configured, refer to Connecting to Azure Data Lake Storage Gen2 for more information.

When configuring and using a connection to Azure Data Lake Storage Gen2, consider the following:

Configuration

Details

Authentication Method

Ensure the selected authentication method (for example, Service Principal or Managed Identity) is correctly configured. Invalid client credentials or expired secrets are a common cause of connection failures.

Permissions and Access Control

Verify that the configured identity has the required permissions on the storage account and container. At a minimum, the identity must have read/write access (for example, Storage Blob Data Contributor role).

Filesystem (Container) and Path Configuration

Confirm that the target filesystem (container) exists and that the specified directory path is valid. Replication jobs will fail if the path does not exist or if permissions are insufficient.

Storage Account Configuration

Ensure that the storage account is configured with Hierarchical Namespace (HNS) enabled, as required for Azure Data Lake Storage Gen2.

Network and Firewall Settings

Verify that the storage account allows access from Replicate Preview. If firewall rules or private endpoints are configured, ensure that the required network access is permitted.

File Format and Output Structure

Understand how data will be written to the data lake (for example, file format and folder structure). Ensure downstream systems are aligned with the expected output.

Naming Conventions

Ensure that generated folder and file names comply with Azure storage naming rules. Avoid unsupported characters that may cause write failures.

In Replicate Preview, configure your workspace using this system datastore and start replicating data. Refer to Configure Workspace: Datastores for more information.