IntegrationsStreamingAzure Event Hubs

Azure Event Hubs

Datafly Signal delivers events to Azure Event Hubs for real-time data streaming and event ingestion at scale, with native integration to Azure Stream Analytics, Functions, and other Azure services.

Prerequisites

Before configuring Azure Event Hubs in Signal, you need an Azure account with an Event Hubs namespace, an event hub, and a connection string with Send permission.

Create an Azure Account

If you don’t already have one, sign up at azure.microsoft.com.

Create an Event Hubs Namespace

  1. In the Azure portal, search for Event Hubs and click Create.
  2. Select your Subscription and Resource group.
  3. Enter a Namespace name (e.g. datafly-events-ns). Names must be globally unique.
  4. Select a Location close to your Signal deployment.
  5. Choose a Pricing tier:
    • Basic — 1 consumer group, 100 brokered connections.
    • Standard — 20 consumer groups, 1000 brokered connections, Kafka support (recommended).
    • Premium — dedicated resources, higher limits.
  6. Set Throughput Units (Standard) or Processing Units (Premium) based on expected volume.
  7. Click Review + Create > Create.

Create an Event Hub

  1. Open your Event Hubs namespace in the Azure portal.
  2. Click + Event Hub at the top.
  3. Enter a Name (e.g. datafly-events).
  4. Set Partition Count (2-32, default 4). More partitions allow higher parallel consumption.
  5. Set Message Retention (1-7 days on Standard tier).
  6. Click Create.

Get the Connection String

  1. In your Event Hubs namespace, go to Settings > Shared access policies.
  2. Click on the RootManageSharedAccessKey policy (or create a new policy with only Send permission for least-privilege access).
  3. Copy the Connection string - primary key.

For production, create a dedicated shared access policy with only Send permission rather than using the root key. Go to Shared access policies > Add > check only Send.

Configuration

FieldTypeRequiredDescription
namespacestringYesThe Event Hubs namespace name.
event_hub_namestringYesThe name of the specific event hub to send events to.
connection_stringsecretYesThe namespace-level connection string with Send permission.

Signal Setup

Quick Setup

  1. Navigate to Integrations in the sidebar.
  2. Open the Integration Library tab.
  3. Find Azure Event Hubs or filter by Cloud Storage.
  4. Click Install, select a variant if available, and fill in the required fields.
  5. Click Install Integration to create the integration with a ready-to-use default blueprint.

API Setup

curl -X POST http://localhost:8084/v1/admin/integration-catalog/azure_event_hubs/install \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Azure Event Hubs",
    "variant": "default",
    "config": {
      "namespace": "datafly-events-ns",
      "event_hub_name": "datafly-events",
      "connection_string": "Endpoint=sb://datafly-events-ns.servicebus.windows.net/;SharedAccessKeyName=SendPolicy;SharedAccessKey=YOUR_KEY"
    },
    "delivery_mode": "server_side"
  }'

Testing

  1. Enable the integration in Signal and trigger a test event on your website.
  2. In the Azure portal, open your Event Hubs namespace > your event hub.
  3. Check the Monitoring section for incoming messages and throughput metrics.
  4. Use Process data > Explore (or Azure Stream Analytics) to inspect individual events.
  5. In Signal, check the Live Events view to confirm delivery status shows as successful.

Troubleshooting

ProblemSolution
Events not appearing in the event hubVerify the namespace, event hub name, and connection string are correct.
Unauthorized (401)The connection string is invalid or the shared access policy lacks Send permission. Verify the policy and regenerate the key if needed.
MessagingEntityNotFoundThe event hub does not exist in the namespace. Verify the event hub name.
QuotaExceededYou have exceeded the throughput units for the namespace. Increase throughput units or enable auto-inflate.
High latencyCheck the number of throughput units. Each unit supports 1 MB/s ingress. Increase units for higher throughput.
Connection string format errorThe connection string must start with Endpoint=sb://. Ensure you copied the full string from Shared access policies.

Visit Azure Event Hubs documentation for full API reference, partitioning strategies, and consumer group setup.

Amazon SQSConfluent Cloud