Integrations

Datafly Signal delivers events server-to-server to vendor APIs. The Integration Library provides 260+ pre-built vendor integrations with one-click setup, industry-specific variants, and full customisation.

How Integrations Work

When you set up an integration, you create an integration template and assign it to one or more pipelines. Events collected by Datafly.js flow through the full pipeline before reaching the vendor:

Datafly.js → Signal collection endpoint
  → Organisation-level processing (consent, bots, PII, identity)
  → Pipeline transformations
  → Vendor API

1. The Signal applies your organisation processing enrichments and per-pipeline transformations.
2. Processed events are routed to the appropriate delivery layer for each integration.
3. The Signal for that vendor formats the payload to match the vendor’s API specification and sends it server-to-server.

Each vendor has its own Signal implementation that understands the target API’s authentication, payload format, rate limits, and error handling.

Integration Library

The Integration Library is a browsable catalog of 260+ pre-built vendor integrations. Each catalog entry includes:

• Configuration fields with labels, types, and help text
• Industry variants (e.g. Retail, Travel, Media) with tuned default configurations
• One-click install that creates a template and first revision automatically

Using the Library (Management UI)

1. Navigate to Integrations in the sidebar.
2. Click the Add Integration button (from My Integrations) or switch to the Integration Library tab.
3. Browse by category or search for a vendor. Each vendor card shows its brand logo for quick recognition.
4. Click Install on the vendor you want.
5. Choose a variant (if available), select a Blueprint for a pre-built configuration (if one exists for your vertical), fill in the required credentials, and click Install Integration.

Blueprints

Blueprints are pre-built V2 schema-mapping configurations for common vendor + vertical combinations. When installing an integration that has Blueprints available, you can select one to start with a ready-made configuration instead of building from scratch.

Available Blueprint verticals include Retail, Travel, and Media, with pre-mapped event types (purchase, add_to_cart, page_view, etc.) and field mappings specific to each vendor’s API format.

All Blueprint mappings are fully editable after install — they serve as a starting point, not a constraint.

Custom Integrations

For vendors not in the library, or for internal APIs, you can create a Custom Integration from the My Integrations tab. Custom integrations support:

• Freeform vendor display name
• Full V2 schema-mapping builder access
• Webhook delivery to any HTTP endpoint

Using the Library (API)

# List the catalog
curl -X GET "http://localhost:8084/v1/admin/integration-catalog?category=advertising" \
  -H "Authorization: Bearer YOUR_TOKEN"
 
# Install from catalog
curl -X POST http://localhost:8084/v1/admin/integration-catalog/meta_capi/install \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Meta CAPI Production",
    "variant": "retail",
    "config": {
      "pixel_id": "1234567890",
      "access_token": "your_access_token"
    },
    "delivery_mode": "server_side"
  }'

See the Integration Catalog API for full documentation.

Templates & Revisions

Integrations use a template + revision model for versioned configuration management:

• Template: An org-level reusable integration definition (e.g. “Meta CAPI Production”)
• Revision: An immutable configuration snapshot (v1, v2, v3…)
• Pipeline Integration: An assignment of a template at a specific revision to a pipeline

Template "Meta CAPI Production"
  ├── Revision v1 — initial setup
  ├── Revision v2 — updated access token
  └── Revision v3 — increased batch size (current)

Pipeline "Marketing Website" → uses v3
Pipeline "Blog" → uses v2

To change configuration, you create a new revision. This ensures a complete audit trail and allows different pipelines to use different versions.

See the Integration Templates API and Pipeline Integrations API for full documentation.

V2 Schema-Mapping Builder

The integration builder uses a schema-aware point-and-click interface organised into four categories:

Each mapping row maps a source field (from the canonical Datafly event schema) to a vendor field (from the vendor’s API schema), with optional transforms (lowercase, sha256, to_unix_seconds, etc.).

The builder includes:

• Autocomplete for both source and vendor fields from known schemas
• Coverage indicator showing how many required/optional vendor fields are mapped
• Code editor with YAML ↔ visual sync for advanced users
• Preview with dry-run against sample events (see below)

The legacy V1 step-based builder is retained for backward compatibility. New integrations default to V2.

Testing a blueprint

The Testing tab runs an event through the blueprint you are editing, unsaved changes included, and shows the fields it produced and the exact HTTP body the vendor would receive. Paste any of these into the Sample Event box:

• one event as Datafly.js sends it, copied from a /v1/batch or /v1/t request in your browser’s network tab;
• the whole batch request body (the first event is previewed, and the panel says so);
• an event from the Debugger or the event store;
• one of the built-in samples.

Events are read exactly as the collection endpoint reads them. Fields that Signal adds after an event is received, such as the IP address, geo lookup, session, minted vendor IDs and the visitor’s profile traits ($traits), are filled in with fixed placeholder values for the test and listed under Filled in for this test. Identifiers the pasted event carries, such as collectedIds and clickIds, are used as they are. A mapping marked required only fails the preview when the field is genuinely absent from the pasted event.

Only send mapped fields

The blueprint is the complete transformation: a vendor receives the fields your blueprint maps, plus the fixed fields its API requires (credentials, static parameters, the event name), and nothing else. A property the blueprint does not map, such as a raw form field, an extra product attribute, or identity data attached during enrichment, is not sent. This is the Only send mapped fields setting on the integration’s Connection tab, and it is on by default.

• Integrations created before this became the default keep forwarding unmapped properties so that upgrading changes nothing they send. The Connection tab and the Testing tab say so, and the delivery log names every unmapped field the integration forwards. Turn the setting on once the Testing tab shows that everything the vendor needs is mapped. This earlier behaviour is deprecated and will be removed in a future release.
• Personal data is never forwarded unmapped. Whatever the setting, a property that is a plain email, phone number, first, last or full name, date of birth, street address or similar personal field (the fields the event specifications say must never be sent raw), and on an identify event the visitor’s name, reaches a vendor only when the blueprint maps it: hashed for platforms that match on it, plain only for tools that act on it. A value that is already a hash is forwarded as before. The visitor’s stored email is attached as $traits.email, never as a property (see Profile Attachment).
• Webhooks forward the whole event by default, because that is their purpose, apart from the personal data above: since 2026-09-23 a webhook that relies on forwarding unmapped properties no longer receives a plain email, phone, name, date of birth or address unless its blueprint maps it (see Custom Webhook). The setting can still be turned on for a webhook.
• Warehouses, databases, files and streams, and any endpoint that carries fields in its URL, always send only mapped fields, whatever the setting says.
• To forward unmapped properties deliberately, turn the setting off for that integration. The choice is recorded on the connection as strict_output_contract: false.

The Testing tab builds the vendor payload under the same setting as live delivery and lists the unmapped fields it removed or forwarded.

Consent Categories

When a consent category is configured on an integration, the Signal reads context.consent.canonical.<category> and drops the event if the required category is not granted. Datafly.js populates the consent object from your CMP — see Canonical Categories.

{
  "context": {
    "consent": {
      "categories": {
        "necessary": true,
        "statistics": true,
        "marketing": false
      },
      "canonical": {
        "essential": true,
        "analytics": true,
        "marketing": false
      }
    }
  }
}

In this example, an integration requiring marketing would not receive this event because canonical.marketing is false.

Retry Behaviour

A delivery that fails with a network error, a timeout, a 429 or a 5xx is retried. A request that fails briefly is retried a few times within seconds; if the delivery still fails, it is retried on a schedule:

A failure that waiting cannot fix, such as a 4xx other than 429 (the vendor rejected the request), is not retried. When the schedule is spent, or the failure is permanent, the event goes to the dead letter queue.

Retry policy per destination

Each destination can change how it retries, in the integration’s Connection settings under Delivery, or through the Management API as keys of the integration’s connection config. A setting left empty keeps the default above. The settings are stored with the integration’s revision, so a change takes effect when the pipeline is published, like any other integration change.

The status settings and the request timeout apply to destinations reached over HTTP, which covers the API destinations in the catalogue; the schedule, maximum retries and window apply to every destination. For a custom HTTP destination, request_timeout takes precedence over its timeout_ms setting.

A setting outside these rules is refused when the integration is saved, with a message naming the setting and the rule it breaks.

Redirects

A vendor endpoint that answers with a redirect is followed only when the redirect stays on the same host and keeps the request method (a moved path, or http to https on the same host). A redirect to a different host, or one that would turn a POST into a GET, means the vendor did not accept the event: a retired pixel host that forwards to the vendor’s home page would otherwise look like a successful delivery. Signal does not follow it, does not retry, and sends the event to the dead letter queue with the redirect’s status code and Location in the error.

Dead Letter Queue

An event whose retries are spent, or whose failure is permanent, waits in the dead letter queue until an operator replays or discards it. Nothing expires automatically. Each entry is one event and one destination, so a replay sends the event only to the destination that failed.

• Search, replay and discard entries on the Dead Letter Queue page of the Management UI, or through the Management API.
• Entries still on the retry schedule show as retrying, entries waiting for an operator as pending.

Rate Limiting

Each destination is sent to no faster than its vendor’s rate limit, shared by every delivery worker, so a deployment with several workers still keeps to one quota. When the limit is reached, events wait their turn: nothing is dropped or counted as failed, delivery is only delayed.

Set them in the integration’s Connection settings under Delivery, or as keys of its connection config.

Supported integrations

Every destination below ships with a pre-built blueprint and is included in every deployment. Filter by name or category, or press Ctrl/⌘ + K for site-wide search.