Azure Event Hubs

Datafly Signal publishes first-party events to Azure Event Hubs for real-time data streaming and ingestion at scale, with native integration to Azure Stream Analytics, Functions, and other Azure services.

Prerequisites

Before configuring Azure Event Hubs in Signal, you need an Azure account with an Event Hubs namespace, an event hub, and a connection string with Send permission.

Create an Azure Account

If you don’t already have one, sign up at azure.microsoft.com.

Create an Event Hubs Namespace

In the Azure portal, search for Event Hubs and click Create.
Select your Subscription and Resource group.
Enter a Namespace name (e.g. datafly-events-ns). Names must be globally unique.
Select a Location close to your Signal deployment.
Choose a Pricing tier:
- Basic — 1 consumer group, 100 brokered connections.
- Standard — 20 consumer groups, 1000 brokered connections, Kafka support (recommended).
- Premium — dedicated resources, higher limits.
Set Throughput Units (Standard) or Processing Units (Premium) based on expected volume.
Click Review + Create > Create.

Create an Event Hub

Open your Event Hubs namespace in the Azure portal.
Click + Event Hub at the top.
Enter a Name (e.g. datafly-events).
Set Partition Count (2-32, default 4). More partitions allow higher parallel consumption.
Set Message Retention (1-7 days on Standard tier).
Click Create.

Get the Connection String

In your Event Hubs namespace, go to Settings > Shared access policies.
Click on the RootManageSharedAccessKey policy (or create a new policy with only Send permission for least-privilege access).
Copy the Connection string - primary key.

For production, create a dedicated shared access policy with only Send permission rather than using the root key. Go to Shared access policies > Add > check only Send.

Configuration

Field	Type	Required	Description
`namespace`	string	Yes	The Event Hubs namespace name.
`event_hub`	string	Yes	The name of the specific event hub to send events to. Also accepts `event_hub_name`.
`connection_string`	secret	Yes	The namespace-level or event-hub-scoped connection string with Send permission.

Signal Setup

Quick Setup

Navigate to Integrations in the sidebar.
Open the Integration Library tab.
Find Azure Event Hubs or filter by Streaming.
Click Install, select a variant if available, and fill in the required fields.
Click Install Integration to create the integration with a ready-to-use default blueprint.

API Setup

curl -X POST http://localhost:8084/v1/admin/integration-catalog/azure_event_hubs/install \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Azure Event Hubs",
    "variant": "default",
    "config": {
      "namespace": "datafly-events-ns",
      "event_hub": "datafly-events",
      "connection_string": "Endpoint=sb://datafly-events-ns.servicebus.windows.net/;SharedAccessKeyName=SendPolicy;SharedAccessKey=YOUR_KEY"
    },
    "delivery_mode": "server_side"
  }'

Schema

Each event is sent as one Event Hubs event:

Body — JSON document of the canonical event envelope: event_id, type, event, anonymous_id, user_id, timestamp, received_at, sent_at, context, properties, traits, source_id, integration_id.
PartitionKey — anonymous_id (or user_id if set), giving consistent per-user partition placement.

Event Hubs is a first-party destination in your own Azure subscription. The default blueprint forwards all events. Apply consent filtering via Stream Analytics queries or in pipeline transforms on context.consent if needed.

Testing

Enable the integration in Signal and trigger a test event on your website.
In the Azure portal, open your Event Hubs namespace > your event hub.
Check the Monitoring section for incoming messages and throughput metrics.
Use Process data > Explore (or Azure Stream Analytics) to inspect individual events.
In Signal, check the Live Events view to confirm delivery status shows as successful.

Troubleshooting

Problem	Solution
Events not appearing in the event hub	Verify the namespace, event hub name, and connection string are correct.
`Unauthorized` (401)	The connection string is invalid or the shared access policy lacks Send permission. Verify the policy and regenerate the key if needed.
`MessagingEntityNotFound`	The event hub does not exist in the namespace. Verify the event hub name.
`QuotaExceeded`	You have exceeded the throughput units for the namespace. Increase throughput units or enable auto-inflate.
High latency	Check the number of throughput units. Each unit supports 1 MB/s ingress. Increase units for higher throughput.
Connection string format error	The connection string must start with `Endpoint=sb://`. Ensure you copied the full string from Shared access policies.

Visit Azure Event Hubs documentation for full API reference, partitioning strategies, and consumer group setup.