IntegrationsStreamingRabbitMQ

RabbitMQ

Datafly Signal publishes first-party events to RabbitMQ exchanges over AMQPS for flexible, open-source message brokering with advanced routing, dead-letter queues, and broad protocol support.

Why route RabbitMQ through Datafly Signal

  • The data lands in infrastructure you own. Signal writes to your RabbitMQ directly from your own deployment. Datafly operates no intermediary and never holds a copy of your event data.
  • Client IP and user agent are recorded as event context. Both come from the original visitor request rather than from your server, so geography and device analysis done in the warehouse reflects the actual visitor.
  • Presets for eight verticals. General (the default: page, identify and the key cross-vertical events), Retail, Insurance, Travel, Media & Streaming, Publishing, B2B / SaaS and Automotive, each writing one named field per documented event property. Email, phone and names are written only as SHA-256 digests unless you deliberately change the installed blueprint.
  • One consent decision governs every destination. RabbitMQ is gated by the same consent state as the rest of your stack, so a withdrawal applies everywhere at once.

Prerequisites

Before configuring RabbitMQ in Signal, you need a RabbitMQ server (self-hosted or managed), an exchange, and a routing key configuration.

Set Up a RabbitMQ Server

You have several options:

Option A: CloudAMQP (Managed)

  1. Sign up at cloudamqp.com.
  2. Create a new instance and select a plan and region.
  3. Note the AMQP URL from the instance details (e.g. amqps://user:pass@host/vhost).

Option B: Amazon MQ for RabbitMQ

  1. In the AWS console, go to Amazon MQ > Create broker.
  2. Select RabbitMQ as the engine type.
  3. Configure the broker and note the AMQP endpoint.

Option C: Self-Hosted RabbitMQ

  1. Install RabbitMQ using the official guide.
  2. Start the RabbitMQ server.
  3. Enable the management plugin for the web UI: rabbitmq-plugins enable rabbitmq_management.
  4. Access the management UI at http://hostname:15672.

Create a Virtual Host (Optional)

If you want to isolate Signal events:

  1. In the RabbitMQ management UI, go to Admin > Virtual Hosts > Add a new virtual host.
  2. Enter a name (e.g. datafly).
  3. Click Add virtual host.

Create an Exchange

  1. In the management UI, go to Exchanges > Add a new exchange.
  2. Enter a Name (e.g. datafly.events).
  3. Choose the Type:
    • topic — routes messages based on routing key patterns (recommended for flexibility).
    • direct — routes messages to queues with an exact routing key match.
    • fanout — routes messages to all bound queues (ignores routing key).
  4. Set Durable to Yes to survive broker restarts.
  5. Click Add exchange.

Create a Queue and Binding

  1. Go to Queues > Add a new queue.
  2. Enter a Name (e.g. datafly-events-queue).
  3. Set Durable to Yes.
  4. Click Add queue.
  5. Click on the queue, scroll to Bindings, and add a binding:
    • From exchange: datafly.events
    • Routing key: events (or a pattern like events.# for topic exchanges)

Create a User for Signal

  1. In the management UI, go to Admin > Users > Add a user.
  2. Enter a Username (e.g. datafly_signal) and Password.
  3. Set Tags to empty (no management access needed).
  4. Click Add user.
  5. Click on the user and set Permissions for the virtual host: Configure .*, Write .*, Read .* (or restrict to specific exchanges).

Get the AMQP URL

The AMQP connection URL format is:

amqps://username:password@hostname:5671/vhost
  • Port 5671 for AMQPS (TLS), port 5672 for AMQP (plaintext).
  • URL-encode the virtual host if it contains special characters (e.g. / becomes %2F).
⚠️

Always use amqps:// (with TLS) in production. The amqp:// protocol transmits credentials and data in cleartext.

Configuration

FieldTypeRequiredDescription
amqp_urlsecretYesThe AMQP URL, amqp://user:pass@host:5672/vhost or amqps://...:5671/vhost. Also accepts url.
exchangestringNoThe exchange to publish to. Leave empty for the default exchange, where the routing key is the queue name.
routing_keystringYesThe routing key: the queue name on the default exchange, or the key your bindings match.

Signal Setup

Quick Setup

  1. Navigate to Integrations in the sidebar.
  2. Open the Integration Library tab.
  3. Find RabbitMQ or filter by Streaming.
  4. Click Install, select a variant if available, and fill in the required fields.
  5. Click Install Integration to create the integration with a ready-to-use default blueprint.

API Setup

curl -X POST http://localhost:8084/v1/admin/integration-catalog/rabbitmq/install \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "RabbitMQ",
    "variant": "default",
    "config": {
      "url": "amqps://datafly_signal:your_password@rabbitmq.example.com:5671/datafly",
      "exchange": "datafly.events",
      "routing_key": "events"
    },
    "delivery_mode": "server_side"
  }'

What Signal writes

Each event is one AMQP 0-9-1 message whose body is the Signal event as JSON:

FieldContents
event, typeEvent name (purchase, page, …) and type (track, page, identify, …).
anonymous_id, user_id, device_idVisitor identifiers (user_id and device_id only when set).
message_id, event_idSignal’s message ID (use it to deduplicate) and the client event ID.
timestamp, sent_at, received_atRFC 3339 UTC times: event, client send, Signal receipt.
org_id, pipeline_id, vendor_typeWhere the event came from and which destination type wrote it.
contextThe collected context as sent: ip, user_agent, locale, page, campaign, consent, geo, device, library, and any partner attributes. Written in full; the blueprint does not filter it (#1414).
click_ids, collected_idsAd click IDs and ID-provider values, when present.
propertiesOnly the fields the blueprint maps (below).
  • Properties: content_type application/json, delivery_mode 2 (persistent), message_id = the Signal message ID, type = the event type (track, page, …).
  • Headers: datafly_event_name.
  • Exchange and routing key: as configured.
  • Delivery: mandatory publish with publisher confirms; Signal counts an event delivered when the broker acks it and has not returned it.
⚠️

A message that no queue is bound to receive is returned by RabbitMQ (312 NO_ROUTE): Signal treats it as a permanent failure, naming the routing key and exchange, and dead-letters the event rather than counting it delivered. Check the exchange, routing key and bindings before you enable the integration, and bind a durable queue so messages survive a broker restart.

Only what the blueprint maps is written. The properties object holds the fields your installed blueprint maps and nothing else (the row contract): the global block under @root (copies of the envelope fields, UTM parameters, the analytics and marketing consent flags, and email_sha256, phone_sha256, first_name_sha256, last_name_sha256, name_sha256), and the event section’s fields at the top level (transaction_id, value, currency, items, …). Identify traits and client properties the blueprint does not map are never written. An event with no section in the blueprint still arrives, with the @root block only: add a section for any other event whose properties you need.

A purchase from an identified visitor, as written by the default (General) preset. The raw email, phone and name in the visitor’s identify traits are not in it; only their digests are. Empty context objects are trimmed here for brevity.

{
  "anonymous_id": "anon-s1-7f3a",
  "user_id": "crm-4711",
  "event": "purchase",
  "type": "track",
  "properties": {
    "@root": {
      "anonymous_id": "anon-s1-7f3a",
      "email_sha256": "86e0b9e56c17cc4d12387e1949b85053fbe73bc3ce5a1188713a9d300cc6133d",
      "event_id": "evt-s1-2",
      "event_name": "purchase",
      "event_timestamp": "2026-09-23T09:30:02.000Z",
      "event_type": "track",
      "first_name_sha256": "81f8f6dde88365f3928796ec7aa53f72820b06db8664f5fe76a7eb13e24546a2",
      "ip_address": "203.0.113.9",
      "last_name_sha256": "4f887a0887ca68593c8acd4e83e0b6bc5f42cd5aff7d3a159041c84230ed1cf6",
      "locale": "en-GB",
      "message_id": "msg-s1-rabbitmq-2",
      "name_sha256": "ed37d99b1445238af3386f81a77a2caf3fffc0cd610be39b4f3ba53943dc66bf",
      "page_url": "https://shop.example/checkout",
      "phone_sha256": "a8acc3a90a7b4e4dc65e93db9240ed26523050ef754d63b75b5161de76781436",
      "user_agent": "Mozilla/5.0 (Macintosh)",
      "user_id": "crm-4711",
      "utm_medium": "cpc",
      "utm_source": "google"
    },
    "coupon": "AUTUMN10",
    "currency": "GBP",
    "items": [{ "item_id": "SKU-1", "item_name": "Trail shoe", "price": 80, "quantity": 1 }],
    "shipping": 4.5,
    "tax": 14.08,
    "transaction_id": "SIG-ORDER-1001",
    "value": 84.5
  },
  "context": {
    "ip": "203.0.113.9",
    "user_agent": "Mozilla/5.0 (Macintosh)",
    "locale": "en-GB",
    "page": { "url": "https://shop.example/checkout", "title": "Checkout", "path": "/checkout" },
    "campaign": { "source": "google", "medium": "cpc" }
  },
  "timestamp": "2026-09-23T09:30:02Z",
  "sent_at": "2026-09-23T09:30:02Z",
  "message_id": "msg-s1-rabbitmq-2",
  "org_id": "org-s1",
  "pipeline_id": "pl-s1",
  "received_at": "2026-09-23T09:30:02Z",
  "event_id": "evt-s1-2",
  "vendor_type": "rabbitmq"
}

The message layout on this page was produced by Signal’s RabbitMQ writer against a RabbitMQ 4 broker on 2026-09-23 (batch S1 review), not written from memory.

Stored event context

Alongside the fields your blueprint maps (where email, phone, names and address are stored as normalised SHA-256 by default), each message carries the event’s collected context exactly as Signal received it. The blueprint does not filter or hash the context: this is deliberate (decision 2026-09-23), so the context is complete for attribution, fraud and debugging queries, and it is listed here so you can decide what to keep.

StoredWhat it contains
Visitor requestIP address, user agent, Accept-Language, locale and timezone
PageURL including its query string, path, title and referrer
CampaignUTM source, medium, campaign, term and content
ConsentThe visitor’s consent state
LocationGeoIP country, region and city, and the network (ASN)
DeviceScreen size; for app events the OS and version, manufacturer, model, advertising ID and IDFV when the app collected them
Vendor IDsCookie and click identifiers in vendor_ids (for example the _fbp browser ID, gclid, ttclid, the GA client ID)
Session and librarySession ID, number and engagement time; the collector’s name and version
Partner attributesProfile attributes from partner enrichment (partner_attrs)

Each message is the event JSON: context, click_ids and collected_ids, and the anonymous_id, user_id, device_id, message_id and event_id identifiers, sit beside the mapped properties.

The IP address, device and advertising IDs, and cookie and click identifiers are personal data under UK and EU GDPR. To keep any of them out of storage, strip, mask or hash the field for every integration with the organisation data layer, apply URL redaction to page URLs, or remove it in the installed blueprint; and set the storage’s retention to match your privacy notice. See Personal data by integration.

Consent

The integration’s consent category in Signal decides which events reach RabbitMQ; events that fail it are never written. Every message also carries the visitor’s consent state (context.consent, and consent_analytics / consent_marketing in @root) so downstream consumers can apply their own rules.

Testing

  1. Enable the integration in Signal and trigger a test event on your website.
  2. In the RabbitMQ management UI, go to Queues and select the queue bound to your exchange.
  3. Check the Messages count — it should increase.
  4. Click Get messages to inspect the message payload.
  5. In Signal, check the Live Events view to confirm delivery status shows as successful.

Troubleshooting

ProblemSolution
Events not appearing in the queueVerify the AMQP URL, exchange name, and routing key are correct. Check that a queue is bound to the exchange with a matching routing key.
ACCESS_REFUSEDThe username or password is incorrect, or the user lacks permissions on the virtual host. Verify credentials and user permissions.
Connection refused / timeoutCheck that the RabbitMQ server is accessible on port 5671 (TLS) or 5672 (plaintext). Verify firewall rules.
NOT_FOUND - no exchangeThe exchange does not exist. Create it in the management UI or via rabbitmqadmin.
Messages published but not in queueThe exchange exists but no queue is bound with the routing key. Add a binding from the exchange to the queue.
TLS handshake failureIf using amqps://, ensure the RabbitMQ server has a valid TLS certificate and the CA is trusted by Signal.
Virtual host not foundThe virtual host in the URL does not exist. Create it or update the URL. Remember to URL-encode the vhost name.

Visit RabbitMQ documentation for full AMQP reference, exchange types, and clustering guides.

See also

Apache KafkaIndex