Azure Event Hubs

Datafly Signal publishes first-party events to Azure Event Hubs for real-time data streaming and ingestion at scale, with native integration to Azure Stream Analytics, Functions, and other Azure services.

Why route Azure Event Hubs through Datafly Signal

The data lands in infrastructure you own. Signal writes to your Azure Event Hubs directly from your own deployment. Datafly operates no intermediary and never holds a copy of your event data.
Client IP and user agent are recorded as event context. Both come from the original visitor request rather than from your server, so geography and device analysis done in the warehouse reflects the actual visitor.
Presets for eight verticals. General (the default: page, identify and the key cross-vertical events), Retail, Insurance, Travel, Media & Streaming, Publishing, B2B / SaaS and Automotive, each writing one named field per documented event property. Email, phone and names are written only as SHA-256 digests unless you deliberately change the installed blueprint.
One consent decision governs every destination. Azure Event Hubs is gated by the same consent state as the rest of your stack, so a withdrawal applies everywhere at once.

Prerequisites

Before configuring Azure Event Hubs in Signal, you need an Azure account with an Event Hubs namespace, an event hub, and a connection string with Send permission.

Create an Azure Account

If you don’t already have one, sign up at azure.microsoft.com.

Create an Event Hubs Namespace

In the Azure portal, search for Event Hubs and click Create.
Select your Subscription and Resource group.
Enter a Namespace name (e.g. datafly-events-ns). Names must be globally unique.
Select a Location close to your Signal deployment.
Choose a Pricing tier:
- Basic — 1 consumer group, 100 brokered connections.
- Standard — 20 consumer groups, 1000 brokered connections, Kafka support (recommended).
- Premium — dedicated resources, higher limits.
Set Throughput Units (Standard) or Processing Units (Premium) based on expected volume.
Click Review + Create > Create.

Create an Event Hub

Open your Event Hubs namespace in the Azure portal.
Click + Event Hub at the top.
Enter a Name (e.g. datafly-events).
Set Partition Count (2-32, default 4). More partitions allow higher parallel consumption.
Set Message Retention (1-7 days on Standard tier).
Click Create.

Get the Connection String

In your Event Hubs namespace, go to Settings > Shared access policies.
Click on the RootManageSharedAccessKey policy (or create a new policy with only Send permission for least-privilege access).
Copy the Connection string - primary key.

For production, create a dedicated shared access policy with only Send permission rather than using the root key. Go to Shared access policies > Add > check only Send.

Configuration

Field	Type	Required	Description
`connection_string`	secret	Yes	Connection string of a shared access policy with Send. A policy on the event hub itself (least privilege, as Microsoft recommends) gives a string ending in `EntityPath=<hub>`; a namespace-level policy gives one without it.
`event_hub_name`	string	Yes	The event hub to send to. Also accepts `event_hub`. With an `EntityPath` connection string it may be left blank or must name the same hub; a different hub is refused, naming the one the string is scoped to.
`namespace`	string	No	A label for the event debugger; the connection string already names the namespace.

Microsoft Entra ID (managed identity) authentication is not supported yet (#135).

Signal Setup

Quick Setup

Navigate to Integrations in the sidebar.
Open the Integration Library tab.
Find Azure Event Hubs or filter by Streaming.
Click Install, select a variant if available, and fill in the required fields.
Click Install Integration to create the integration with a ready-to-use default blueprint.

API Setup

curl -X POST http://localhost:8084/v1/admin/integration-catalog/azure_event_hubs/install \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Azure Event Hubs",
    "variant": "default",
    "config": {
      "namespace": "datafly-events-ns",
      "event_hub": "datafly-events",
      "connection_string": "Endpoint=sb://datafly-events-ns.servicebus.windows.net/;SharedAccessKeyName=SendPolicy;SharedAccessKey=YOUR_KEY"
    },
    "delivery_mode": "server_side"
  }'

What Signal writes

Each event is one Event Hubs event (AMQP) whose body is the Signal event as JSON:

Field	Contents
`event`, `type`	Event name (`purchase`, `page`, …) and type (`track`, `page`, `identify`, …).
`anonymous_id`, `user_id`, `device_id`	Visitor identifiers (`user_id` and `device_id` only when set).
`message_id`, `event_id`	Signal’s message ID (use it to deduplicate) and the client event ID.
`timestamp`, `sent_at`, `received_at`	RFC 3339 UTC times: event, client send, Signal receipt.
`org_id`, `pipeline_id`, `vendor_type`	Where the event came from and which destination type wrote it.
`context`	The collected context as sent: `ip`, `user_agent`, `locale`, `page`, `campaign`, `consent`, `geo`, `device`, `library`, and any partner attributes. Written in full; the blueprint does not filter it (#1414).
`click_ids`, `collected_ids`	Ad click IDs and ID-provider values, when present.
`properties`	Only the fields the blueprint maps (below).

Partition key: user_id, else anonymous_id, so one visitor’s events stay on one partition in order.
Size: 1 MB per publish (Standard tier and above).
Delivery: at least once; deduplicate on message_id.

Only what the blueprint maps is written. The properties object holds the fields your installed blueprint maps and nothing else (the row contract): the global block under @root (copies of the envelope fields, UTM parameters, the analytics and marketing consent flags, and email_sha256, phone_sha256, first_name_sha256, last_name_sha256, name_sha256), and the event section’s fields at the top level (transaction_id, value, currency, items, …). Identify traits and client properties the blueprint does not map are never written. An event with no section in the blueprint still arrives, with the @root block only: add a section for any other event whose properties you need.

A purchase from an identified visitor, as written by the default (General) preset. The raw email, phone and name in the visitor’s identify traits are not in it; only their digests are. Empty context objects are trimmed here for brevity.

{
  "anonymous_id": "anon-s1-7f3a",
  "user_id": "crm-4711",
  "event": "purchase",
  "type": "track",
  "properties": {
    "@root": {
      "anonymous_id": "anon-s1-7f3a",
      "email_sha256": "86e0b9e56c17cc4d12387e1949b85053fbe73bc3ce5a1188713a9d300cc6133d",
      "event_id": "evt-s1-2",
      "event_name": "purchase",
      "event_timestamp": "2026-09-23T09:30:02.000Z",
      "event_type": "track",
      "first_name_sha256": "81f8f6dde88365f3928796ec7aa53f72820b06db8664f5fe76a7eb13e24546a2",
      "ip_address": "203.0.113.9",
      "last_name_sha256": "4f887a0887ca68593c8acd4e83e0b6bc5f42cd5aff7d3a159041c84230ed1cf6",
      "locale": "en-GB",
      "message_id": "msg-s1-azure_event_hubs-2",
      "name_sha256": "ed37d99b1445238af3386f81a77a2caf3fffc0cd610be39b4f3ba53943dc66bf",
      "page_url": "https://shop.example/checkout",
      "phone_sha256": "a8acc3a90a7b4e4dc65e93db9240ed26523050ef754d63b75b5161de76781436",
      "user_agent": "Mozilla/5.0 (Macintosh)",
      "user_id": "crm-4711",
      "utm_medium": "cpc",
      "utm_source": "google"
    },
    "coupon": "AUTUMN10",
    "currency": "GBP",
    "items": [{ "item_id": "SKU-1", "item_name": "Trail shoe", "price": 80, "quantity": 1 }],
    "shipping": 4.5,
    "tax": 14.08,
    "transaction_id": "SIG-ORDER-1001",
    "value": 84.5
  },
  "context": {
    "ip": "203.0.113.9",
    "user_agent": "Mozilla/5.0 (Macintosh)",
    "locale": "en-GB",
    "page": { "url": "https://shop.example/checkout", "title": "Checkout", "path": "/checkout" },
    "campaign": { "source": "google", "medium": "cpc" }
  },
  "timestamp": "2026-09-23T09:30:02Z",
  "sent_at": "2026-09-23T09:30:02Z",
  "message_id": "msg-s1-azure_event_hubs-2",
  "org_id": "org-s1",
  "pipeline_id": "pl-s1",
  "received_at": "2026-09-23T09:30:02Z",
  "event_id": "evt-s1-2",
  "vendor_type": "azure_event_hubs"
}

The message layout on this page was produced by Signal’s Event Hubs writer against the Azure Event Hubs emulator on 2026-09-23 (batch S1 review), not written from memory.

Stored event context

Alongside the fields your blueprint maps (where email, phone, names and address are stored as normalised SHA-256 by default), each message carries the event’s collected context exactly as Signal received it. The blueprint does not filter or hash the context: this is deliberate (decision 2026-09-23), so the context is complete for attribution, fraud and debugging queries, and it is listed here so you can decide what to keep.

Stored	What it contains
Visitor request	IP address, user agent, `Accept-Language`, locale and timezone
Page	URL including its query string, path, title and referrer
Campaign	UTM source, medium, campaign, term and content
Consent	The visitor’s consent state
Location	GeoIP country, region and city, and the network (ASN)
Device	Screen size; for app events the OS and version, manufacturer, model, advertising ID and IDFV when the app collected them
Vendor IDs	Cookie and click identifiers in `vendor_ids` (for example the `_fbp` browser ID, `gclid`, `ttclid`, the GA client ID)
Session and library	Session ID, number and engagement time; the collector’s name and version
Partner attributes	Profile attributes from partner enrichment (`partner_attrs`)

Each message is the event JSON: context, click_ids and collected_ids, and the anonymous_id, user_id, device_id, message_id and event_id identifiers, sit beside the mapped properties.

The IP address, device and advertising IDs, and cookie and click identifiers are personal data under UK and EU GDPR. To keep any of them out of storage, strip, mask or hash the field for every integration with the organisation data layer, apply URL redaction to page URLs, or remove it in the installed blueprint; and set the storage’s retention to match your privacy notice. See Personal data by integration.

The integration’s consent category in Signal decides which events reach Azure Event Hubs; events that fail it are never written. Every message also carries the visitor’s consent state (context.consent, and consent_analytics / consent_marketing in @root) so downstream consumers can apply their own rules.

Testing

Enable the integration in Signal and trigger a test event on your website.
In the Azure portal, open your Event Hubs namespace > your event hub.
Check the Monitoring section for incoming messages and throughput metrics.
Use Process data > Explore (or Azure Stream Analytics) to inspect individual events.
In Signal, check the Live Events view to confirm delivery status shows as successful.

Troubleshooting

Problem	Solution
Events not appearing in the event hub	Verify the namespace, event hub name, and connection string are correct.
`Unauthorized` (401)	The connection string is invalid or the shared access policy lacks Send permission. Verify the policy and regenerate the key if needed.
`MessagingEntityNotFound`	The event hub does not exist in the namespace. Verify the event hub name.
`QuotaExceeded`	You have exceeded the throughput units for the namespace. Increase throughput units or enable auto-inflate.
High latency	Check the number of throughput units. Each unit supports 1 MB/s ingress. Increase units for higher throughput.
Connection string format error	The connection string must start with `Endpoint=sb://`. With an `EntityPath=...` string, `event_hub_name` must be blank or name that same hub.

Visit Azure Event Hubs documentation for full API reference, partitioning strategies, and consumer group setup.