Webhooks

The Webhook endpoint allows Datafly Signal to receive events from third-party services such as payment processors, CRMs, form builders, and other platforms that support outbound webhooks.

Endpoint

POST /v1/webhook

How It Works

1. You configure a webhook source in the Management UI, specifying the third-party service and payload mapping
2. The third-party service sends HTTP POST requests to your webhook URL
3. The Ingestion Gateway verifies the request signature, maps the payload to the canonical event format, and publishes to Kafka

Third-party service → POST /v1/webhook → Signature verification
  → Payload mapping → Kafka raw-events

Each webhook source gets a unique URL with an embedded source identifier:

https://data.example.com/v1/webhook?source=wh_abc123

Authentication

Webhook requests are authenticated using the source’s signature verification method. Each webhook source type has its own verification scheme:

The signing secret for each webhook source is configured in the Management UI when you create the source.

Payload Mapping

Third-party services send data in their own format. Datafly maps these payloads to the canonical event format using JSON path expressions configured per webhook source.

Configuration

Payload mapping is defined in the webhook source configuration:

{
  "source_type": "stripe",
  "event_type_path": "$.type",
  "mappings": {
    "event": "$.type",
    "userId": "$.data.object.customer",
    "properties.amount": "$.data.object.amount",
    "properties.currency": "$.data.object.currency",
    "properties.payment_id": "$.data.object.id",
    "properties.status": "$.data.object.status",
    "timestamp": "$.created"
  },
  "event_name_map": {
    "payment_intent.succeeded": "Payment Completed",
    "customer.subscription.created": "Subscription Started",
    "customer.subscription.deleted": "Subscription Cancelled"
  }
}

JSON Path Expressions

The mapping uses JSON path expressions to extract values from the incoming payload:

Event Name Mapping

The event_name_map translates vendor-specific event types into human-readable event names. If an incoming event type is not in the map, the raw event type string is used as-is.

Example: Stripe Webhook

1. Create the webhook source

In the Management UI, create a new webhook source with type “Stripe” and enter your Stripe webhook signing secret.

2. Configure payload mapping

source_type: stripe
event_type_path: "$.type"
timestamp_path: "$.created"
timestamp_format: unix_seconds
mappings:
  event: "$.type"
  userId: "$.data.object.customer"
  properties.amount: "$.data.object.amount"
  properties.currency: "$.data.object.currency"
  properties.payment_id: "$.data.object.id"
  properties.invoice_id: "$.data.object.invoice"
event_name_map:
  payment_intent.succeeded: "Payment Completed"
  invoice.paid: "Invoice Paid"
  customer.subscription.created: "Subscription Started"
  customer.subscription.updated: "Subscription Updated"
  customer.subscription.deleted: "Subscription Cancelled"
  charge.refunded: "Refund Issued"

3. Register the URL in Stripe

Add your webhook URL in the Stripe Dashboard under Developers > Webhooks:

https://data.example.com/v1/webhook?source=wh_abc123

4. Incoming Stripe event

When Stripe sends an event:

{
  "id": "evt_1234567890",
  "type": "payment_intent.succeeded",
  "created": 1740493800,
  "data": {
    "object": {
      "id": "pi_abc123",
      "customer": "cus_xyz789",
      "amount": 9999,
      "currency": "usd",
      "invoice": "in_def456",
      "status": "succeeded"
    }
  }
}

Datafly maps this to:

{
  "type": "track",
  "event": "Payment Completed",
  "userId": "cus_xyz789",
  "properties": {
    "amount": 9999,
    "currency": "usd",
    "payment_id": "pi_abc123",
    "invoice_id": "in_def456"
  },
  "timestamp": "2025-02-25T14:30:00.000Z",
  "context": {
    "source": "webhook",
    "webhook_source": "stripe",
    "webhook_event_id": "evt_1234567890"
  }
}

Example: CRM Updates (HubSpot)

source_type: hubspot
event_type_path: "$.subscriptionType"
mappings:
  event: "$.subscriptionType"
  userId: "$.objectId"
  properties.property_name: "$.propertyName"
  properties.property_value: "$.propertyValue"
event_name_map:
  contact.propertyChange: "Contact Updated"
  contact.creation: "Contact Created"
  deal.propertyChange: "Deal Updated"
  deal.creation: "Deal Created"

Example: Form Submissions (Custom)

For a custom form service that sends a simple JSON payload:

source_type: custom
signature_header: "X-Webhook-Signature"
signature_method: hmac_sha256
event_type_path: null
mappings:
  event: "'Form Submitted'"
  userId: "$.email"
  properties.form_id: "$.form_id"
  properties.form_name: "$.form_name"
  properties.fields: "$.submission"
  timestamp: "$.submitted_at"

Response

The webhook endpoint always returns a 200 OK response to the calling service, even if the payload cannot be mapped. This prevents the third-party service from retrying indefinitely.

{
  "success": true
}

Mapping errors and signature verification failures are logged and visible in the Management UI’s event debugger.

Supported Sources

These are pre-built webhook source types with built-in signature verification and default payload mappings:

Best Practices

• Always configure signature verification. This prevents unauthorised event injection.
• Use event name mapping to translate vendor-specific event types into consistent, human-readable names across your analytics.
• Test with the event debugger in the Management UI before going live. Send a test event from the third-party service and verify the mapping output.
• Keep the webhook URL secret. Treat it like an API key. If compromised, rotate the webhook source in the Management UI.
• Monitor delivery in the source dashboard. The Management UI shows delivery success rates, latency, and error details per webhook source.