IntegrationsStreamingRabbitMQ

RabbitMQ

Datafly Signal publishes first-party events to RabbitMQ exchanges over AMQPS for flexible, open-source message brokering with advanced routing, dead-letter queues, and broad protocol support.

Why route RabbitMQ through Datafly Signal

  • The data lands in infrastructure you own. Signal writes to your RabbitMQ directly from your own deployment. Datafly operates no intermediary and never holds a copy of your event data.
  • Client IP and user agent are recorded as event context. Both come from the original visitor request rather than from your server, so geography and device analysis done in the warehouse reflects the actual visitor.
  • One consent decision governs every destination. RabbitMQ is gated by the same consent state as the rest of your stack, so a withdrawal applies everywhere at once.

Prerequisites

Before configuring RabbitMQ in Signal, you need a RabbitMQ server (self-hosted or managed), an exchange, and a routing key configuration.

Set Up a RabbitMQ Server

You have several options:

Option A: CloudAMQP (Managed)

  1. Sign up at cloudamqp.com.
  2. Create a new instance and select a plan and region.
  3. Note the AMQP URL from the instance details (e.g. amqps://user:pass@host/vhost).

Option B: Amazon MQ for RabbitMQ

  1. In the AWS console, go to Amazon MQ > Create broker.
  2. Select RabbitMQ as the engine type.
  3. Configure the broker and note the AMQP endpoint.

Option C: Self-Hosted RabbitMQ

  1. Install RabbitMQ using the official guide.
  2. Start the RabbitMQ server.
  3. Enable the management plugin for the web UI: rabbitmq-plugins enable rabbitmq_management.
  4. Access the management UI at http://hostname:15672.

Create a Virtual Host (Optional)

If you want to isolate Signal events:

  1. In the RabbitMQ management UI, go to Admin > Virtual Hosts > Add a new virtual host.
  2. Enter a name (e.g. datafly).
  3. Click Add virtual host.

Create an Exchange

  1. In the management UI, go to Exchanges > Add a new exchange.
  2. Enter a Name (e.g. datafly.events).
  3. Choose the Type:
    • topic — routes messages based on routing key patterns (recommended for flexibility).
    • direct — routes messages to queues with an exact routing key match.
    • fanout — routes messages to all bound queues (ignores routing key).
  4. Set Durable to Yes to survive broker restarts.
  5. Click Add exchange.

Create a Queue and Binding

  1. Go to Queues > Add a new queue.
  2. Enter a Name (e.g. datafly-events-queue).
  3. Set Durable to Yes.
  4. Click Add queue.
  5. Click on the queue, scroll to Bindings, and add a binding:
    • From exchange: datafly.events
    • Routing key: events (or a pattern like events.# for topic exchanges)

Create a User for Signal

  1. In the management UI, go to Admin > Users > Add a user.
  2. Enter a Username (e.g. datafly_signal) and Password.
  3. Set Tags to empty (no management access needed).
  4. Click Add user.
  5. Click on the user and set Permissions for the virtual host: Configure .*, Write .*, Read .* (or restrict to specific exchanges).

Get the AMQP URL

The AMQP connection URL format is:

amqps://username:password@hostname:5671/vhost
  • Port 5671 for AMQPS (TLS), port 5672 for AMQP (plaintext).
  • URL-encode the virtual host if it contains special characters (e.g. / becomes %2F).
⚠️

Always use amqps:// (with TLS) in production. The amqp:// protocol transmits credentials and data in cleartext.

Configuration

FieldTypeRequiredDescription
urlsecretYesThe AMQP connection URL (e.g. amqps://user:pass@host:5671/vhost). Also accepts amqp_url.
exchangestringYesThe exchange name to publish messages to.
routing_keystringNoThe routing key for published messages. Defaults to events.

Signal Setup

Quick Setup

  1. Navigate to Integrations in the sidebar.
  2. Open the Integration Library tab.
  3. Find RabbitMQ or filter by Streaming.
  4. Click Install, select a variant if available, and fill in the required fields.
  5. Click Install Integration to create the integration with a ready-to-use default blueprint.

API Setup

curl -X POST http://localhost:8084/v1/admin/integration-catalog/rabbitmq/install \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "RabbitMQ",
    "variant": "default",
    "config": {
      "url": "amqps://datafly_signal:your_password@rabbitmq.example.com:5671/datafly",
      "exchange": "datafly.events",
      "routing_key": "events"
    },
    "delivery_mode": "server_side"
  }'

Schema

Each event is published as one AMQP message:

  • Body — JSON document of the canonical event envelope: event_id, type, event, anonymous_id, user_id, timestamp, received_at, sent_at, context, properties, traits, source_id, integration_id.
  • content_type — application/json.
  • delivery_mode — 2 (persistent), so messages survive broker restarts when bound to durable queues.
  • routing_key — configured value (default events).

Consent

RabbitMQ is a first-party destination under your control. The default blueprint forwards all events. Apply consent filtering via downstream consumers or in pipeline transforms on context.consent if needed.

Testing

  1. Enable the integration in Signal and trigger a test event on your website.
  2. In the RabbitMQ management UI, go to Queues and select the queue bound to your exchange.
  3. Check the Messages count — it should increase.
  4. Click Get messages to inspect the message payload.
  5. In Signal, check the Live Events view to confirm delivery status shows as successful.

Troubleshooting

ProblemSolution
Events not appearing in the queueVerify the AMQP URL, exchange name, and routing key are correct. Check that a queue is bound to the exchange with a matching routing key.
ACCESS_REFUSEDThe username or password is incorrect, or the user lacks permissions on the virtual host. Verify credentials and user permissions.
Connection refused / timeoutCheck that the RabbitMQ server is accessible on port 5671 (TLS) or 5672 (plaintext). Verify firewall rules.
NOT_FOUND - no exchangeThe exchange does not exist. Create it in the management UI or via rabbitmqadmin.
Messages published but not in queueThe exchange exists but no queue is bound with the routing key. Add a binding from the exchange to the queue.
TLS handshake failureIf using amqps://, ensure the RabbitMQ server has a valid TLS certificate and the CA is trusted by Signal.
Virtual host not foundThe virtual host in the URL does not exist. Create it or update the URL. Remember to URL-encode the vhost name.

Visit RabbitMQ documentation for full AMQP reference, exchange types, and clustering guides.

See also

Apache KafkaIndex