Custom Webhook

Datafly Signal can deliver events to any HTTP endpoint using the custom webhook integration. This is useful for connecting to internal systems, CRMs, custom data pipelines, or any third-party API not covered by a dedicated integration.

Overview

The webhook integration sends a configurable HTTP request for each event (or batch of events). You control the URL, HTTP method, headers, authentication, and payload format.

Configuration

Field	Required	Default	Description
`url`	Yes	—	The HTTP endpoint URL to deliver events to. Must be HTTPS in production.
`method`	No	`POST`	HTTP method: `POST` or `PUT`.
`headers`	No	`{}`	Custom HTTP headers as key-value pairs.
`authentication`	No	`none`	Authentication method: `none`, `bearer`, `basic`, or `hmac`.
`payload_format`	No	`canonical`	Payload format: `canonical` (Datafly event schema) or `custom` (Handlebars template).
`batch_enabled`	No	`false`	Whether to batch multiple events into a single request.
`batch_size`	No	`100`	Maximum number of events per batch (when batching is enabled).
`batch_interval_ms`	No	`5000`	Maximum time to wait before flushing a batch (milliseconds).
`timeout_ms`	No	`30000`	Request timeout in milliseconds.
`retry_enabled`	No	`true`	Whether to retry failed deliveries.
`max_retries`	No	`5`	Maximum number of retry attempts.

Management UI Setup

Go to Integrations > Add Integration > Custom Webhook.
Enter the destination url.
Configure authentication, headers, and payload format as needed.
Optionally enable batching.
Click Save.

Management API Setup

curl -X POST http://localhost:8084/v1/admin/integrations \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "source_id": "src_abc123",
    "vendor": "webhook",
    "name": "CRM Webhook",
    "enabled": true,
    "config": {
      "url": "https://api.example.com/events",
      "method": "POST",
      "authentication": "bearer",
      "auth_token": "your_api_token",
      "headers": {
        "X-Custom-Header": "my-value"
      },
      "payload_format": "canonical"
    }
  }'

Authentication Methods

None

No authentication is added to the request. Use this only for endpoints that handle authentication via other means (e.g. IP allowlisting, VPN).

{
  "config": {
    "url": "https://internal.example.com/events",
    "authentication": "none"
  }
}

Bearer Token

Adds an Authorization: Bearer {token} header to each request.

{
  "config": {
    "url": "https://api.example.com/events",
    "authentication": "bearer",
    "auth_token": "your_api_token"
  }
}

Resulting header:

Authorization: Bearer your_api_token

Basic Authentication

Adds an Authorization: Basic {base64(username:password)} header to each request.

{
  "config": {
    "url": "https://api.example.com/events",
    "authentication": "basic",
    "auth_username": "your_username",
    "auth_password": "your_password"
  }
}

Resulting header:

Authorization: Basic eW91cl91c2VybmFtZTp5b3VyX3Bhc3N3b3Jk

HMAC Signature

Computes an HMAC-SHA256 signature of the request body and includes it in a configurable header. This allows the receiving endpoint to verify that the request originated from Datafly Signal and was not tampered with.

{
  "config": {
    "url": "https://api.example.com/events",
    "authentication": "hmac",
    "hmac_secret": "your_shared_secret",
    "hmac_header": "X-Signature-256"
  }
}

Resulting header:

X-Signature-256: sha256=a1b2c3d4e5f6...

Verification example (Node.js):

const crypto = require('crypto');
 
function verifySignature(body, signature, secret) {
  const expected = 'sha256=' + crypto
    .createHmac('sha256', secret)
    .update(body, 'utf8')
    .digest('hex');
  return crypto.timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(expected)
  );
}

⚠️

Always use HMAC authentication when your webhook endpoint is publicly accessible. This prevents unauthorized parties from sending fake events to your endpoint.

Payload Format

Canonical Format

The default canonical format sends events in the Datafly canonical event schema:

{
  "event_id": "evt_abc123def456",
  "type": "track",
  "event": "Order Completed",
  "anonymous_id": "anon_123",
  "user_id": "user_456",
  "timestamp": "2026-01-29T14:30:00Z",
  "context": {
    "page": {
      "url": "https://example.com/checkout/confirmation",
      "title": "Order Confirmation",
      "referrer": "https://example.com/checkout"
    },
    "ip": "203.0.113.50",
    "user_agent": "Mozilla/5.0 ...",
    "locale": "en-US"
  },
  "properties": {
    "order_id": "ORD-001",
    "total": 129.99,
    "currency": "USD",
    "products": [
      {
        "product_id": "SKU-A",
        "name": "Widget",
        "price": 49.99,
        "quantity": 2
      }
    ]
  },
  "traits": {
    "email": "john@example.com",
    "name": "John Doe"
  }
}

Custom Template

The custom format uses a Handlebars template to transform the event into any JSON structure your endpoint expects. Define the template in the payload_template field:

{
  "config": {
    "url": "https://api.crm.example.com/contacts/events",
    "payload_format": "custom",
    "payload_template": "{ \"contact_email\": \"{{traits.email}}\", \"action\": \"{{event}}\", \"metadata\": { \"order_id\": \"{{properties.order_id}}\", \"revenue\": {{properties.total}}, \"source\": \"datafly\" } }"
  }
}

Rendered payload:

{
  "contact_email": "john@example.com",
  "action": "Order Completed",
  "metadata": {
    "order_id": "ORD-001",
    "revenue": 129.99,
    "source": "datafly"
  }
}

Template variables use dot notation to access nested fields: {{properties.order_id}}, {{context.page.url}}, {{traits.email}}. Missing fields render as empty strings.

Batching

When batch_enabled is true, multiple events are buffered and sent in a single request as a JSON array:

{
  "config": {
    "url": "https://api.example.com/events/batch",
    "batch_enabled": true,
    "batch_size": 50,
    "batch_interval_ms": 10000
  }
}

A batch is flushed when either condition is met:

The number of buffered events reaches batch_size.
The time since the first buffered event exceeds batch_interval_ms.

Batch payload:

[
  { "event_id": "evt_001", "event": "Page Viewed", "..." : "..." },
  { "event_id": "evt_002", "event": "Product Added", "..." : "..." },
  { "event_id": "evt_003", "event": "Order Completed", "..." : "..." }
]

Retry and Timeout

Setting	Default	Description
`timeout_ms`	`30000`	Request timeout. If the endpoint does not respond within this time, the request is considered failed.
`retry_enabled`	`true`	Enable/disable retries for this webhook.
`max_retries`	`5`	Maximum retry attempts (uses exponential backoff per the global retry policy).

Retries are triggered for:

HTTP 5xx responses (server errors)
HTTP 429 responses (rate limited)
Network errors (connection refused, timeout, DNS failure)

Retries are not triggered for:

HTTP 4xx responses (except 429) — these indicate a client error that will not be resolved by retrying.

If the webhook endpoint consistently returns 4xx errors, check the payload format and authentication configuration. The Management UI shows recent delivery errors on the integration detail page.

Use Cases

Internal Analytics Pipeline

Send all events to an internal service for custom processing:

{
  "vendor": "webhook",
  "name": "Internal Analytics",
  "config": {
    "url": "https://analytics.internal.example.com/ingest",
    "authentication": "bearer",
    "auth_token": "internal_api_key",
    "batch_enabled": true,
    "batch_size": 200,
    "batch_interval_ms": 5000
  }
}

CRM Integration

Push lead events to a CRM with a custom payload:

{
  "vendor": "webhook",
  "name": "CRM Leads",
  "config": {
    "url": "https://api.crm.example.com/leads",
    "authentication": "basic",
    "auth_username": "crm_user",
    "auth_password": "crm_pass",
    "payload_format": "custom",
    "payload_template": "{ \"email\": \"{{traits.email}}\", \"name\": \"{{traits.name}}\", \"source\": \"website\", \"event\": \"{{event}}\" }"
  },
  "event_filter": {
    "event_names": ["Lead Generated", "Signed Up"]
  }
}

Slack Notifications

Send conversion events to a Slack webhook for real-time notifications:

{
  "vendor": "webhook",
  "name": "Slack Purchase Alerts",
  "config": {
    "url": "https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXX",
    "authentication": "none",
    "payload_format": "custom",
    "payload_template": "{ \"text\": \"New purchase: {{properties.order_id}} for {{properties.currency}} {{properties.total}} by {{traits.email}}\" }"
  },
  "event_filter": {
    "event_names": ["Order Completed"]
  }
}

Rate Limits

Setting	Default
`rate_limit_rps`	`50`
`rate_limit_burst`	`100`

Adjust based on your endpoint’s capacity. For internal services with high throughput, increase these values as needed.

Snowflake Axon