Azure Data Lake Storage Gen2

Datafly Signal writes batches of first-party events to ADLS Gen2 — a Blob Storage account with hierarchical namespace enabled — ideal for Spark, Databricks, and Synapse workloads.

Why route Azure Data Lake through Datafly Signal

• The data lands in infrastructure you own. Signal writes to your Azure Data Lake directly from your own deployment. Datafly operates no intermediary and never holds a copy of your event data.
• Client IP and user agent are recorded as event context. Both come from the original visitor request rather than from your server, so geography and device analysis done in the warehouse reflects the actual visitor.
• One consent decision governs every destination. Azure Data Lake is gated by the same consent state as the rest of your stack, so a withdrawal applies everywhere at once.

Prerequisites

Before configuring Azure Data Lake Storage Gen2 in Signal, you need an Azure account with a Storage Account that has hierarchical namespace enabled, and a container for storing events.

Configuration

Authentication

Choosing an Authentication Method makes that method’s credentials required: the connection cannot be saved until they are filled in. For the Entra ID methods (service principal, managed identity, workload identity), assign Storage Blob Data Contributor on the storage account or container; no storage key is stored in Signal. The Helm values for workload identity are in Azure Blob Storage: Authentication. If auth_method is not set, Signal uses the first credential filled in, as it did before the setting existed.

Signal writes through the storage account’s Blob endpoint (<account>.blob.core.windows.net), which accepts writes to hierarchical-namespace accounts. If the account is only reachable through a private endpoint, create the private endpoint for the blob sub-resource.

Before you connect

• The container must already exist. Signal writes files into it but does not create it.
• Storage firewall. If the account’s firewall is on, allow the delivery workers’ outbound IP addresses, or reach the account through a private endpoint for the blob sub-resource.
• Shared Key disabled. If the account disallows Shared Key authorization, access keys, key-based connection strings and account SAS tokens are refused: use a Microsoft Entra ID method or a user delegation SAS.
• ACLs instead of an RBAC role. With POSIX access control lists on a hierarchical-namespace account, the identity needs Execute on every parent directory of the prefix, and Write and Execute on the directory the files go into.
• Azure Government and Azure China work only with a connection string: the other methods write to <account>.blob.core.windows.net.

Signal Setup

Quick Setup

1. Navigate to Integrations in the sidebar.
2. Open the Integration Library tab.
3. Find Azure Data Lake Storage Gen2 or filter by Cloud Storage.
4. Click Install, select a variant if available, and fill in the required fields.
5. Click Install Integration to create the integration with a ready-to-use default blueprint.

API Setup

curl -X POST http://localhost:8084/v1/admin/integration-catalog/azure_data_lake/install \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Azure Data Lake Storage Gen2",
    "variant": "default",
    "config": {
      "account_name": "dataflydatalake",
      "container": "datafly-events",
      "prefix": "events/",
      "auth_method": "service_principal",
      "tenant_id": "YOUR_TENANT_ID",
      "client_id": "YOUR_APP_CLIENT_ID",
      "client_secret": "YOUR_CLIENT_SECRET"
    },
    "delivery_mode": "server_side"
  }'

Presets

The blueprint ships the same eight presets as the other file-storage integrations. Each writes one named field per documented event-spec property (canonical names first, the legacy names as fallbacks), plus a global block under @root with the envelope fields, UTM parameters, the analytics and marketing consent flags, and email, phone and names as normalised SHA-256 digests (email_sha256, phone_sha256, first_name_sha256, last_name_sha256, name_sha256; phone shaped to E.164 first). A raw field is a deliberate edit to the installed blueprint. Every preset also stores the consent event’s consent_category, consent_action, consent_message and consent_valid_until.

Schema

Signal writes newline-delimited JSON by default, or Parquet (Snappy-compressed) or CSV. CSV and Parquet use either the canonical 16-column layout (message_id, event_type, event_name, … properties_json, context_json) or, with schema_mode set to blueprint, typed columns from your blueprint output, with event_name taken from the blueprint’s vendor event name. See File formats and schema for the column reference.

Files use Hive-style partitions of the event time, which Spark, Databricks Auto Loader and Synapse read directly:

<prefix>/yyyy=2026/mm=09/dd=15/hh=10/<pipeline_id>/<message_id>.parquet

Stored event context

Alongside the fields your blueprint maps (where email, phone, names and address are stored as normalised SHA-256 by default), each file row carries the event’s collected context exactly as Signal received it. The blueprint does not filter or hash the context: this is deliberate (decision 2026-09-23), so the context is complete for attribution, fraud and debugging queries, and it is listed here so you can decide what to keep.

In each NDJSON line these are the context object, with anonymous_id, user_id, device_id, message_id, event_id, click_ids and collected_ids beside the mapped properties. In CSV and Parquet with the canonical schema they are the ip_address, user_agent, page_url, page_path and page_referrer columns and, in full, the context_json column. In blueprint schema mode only the columns you declare are written.

The IP address, device and advertising IDs, and cookie and click identifiers are personal data under UK and EU GDPR. To keep any of them out of storage, strip, mask or hash the field for every integration with the organisation data layer, apply URL redaction to page URLs, or remove it in the installed blueprint; and set the storage’s retention to match your privacy notice. See Personal data by integration.

Consent

ADLS Gen2 is a first-party destination in your own Azure subscription. The default blueprint forwards all events. Apply consent-aware filtering or partitioning via pipeline transforms on context.consent if needed.

Testing

1. Enable the integration in Signal and trigger a test event on your website.
2. In the Azure portal, open your Storage Account > Containers > your container.
3. Navigate through the path prefix directory structure to find event files.
4. Download a file and inspect the event data.
5. In Signal, check the Live Events view to confirm delivery status shows as successful.

Troubleshooting

See also

• Azure Blob Storage
• Azure Synapse Analytics
• Databricks (SQL Warehouse)
• Amazon S3