ProcessingEvent Filtering

Event Filtering

⚠️

Coming Soon — Event filtering is under active development and not yet available in the Management UI. The pipeline engine already supports filter steps (see Pipeline Configuration); the features described here add a higher-level, UI-driven filtering layer on top of that foundation.

Event filtering lets you control which events are delivered to each integration based on field values. Instead of writing pipeline filter steps by hand, you define filter rules in the Management UI and they are applied automatically before transformation runs.

How It Works

Filters are evaluated after the Organisation Data Layer has enriched the event, but before the Pipeline Transformation Engine runs. If an event does not pass the filters for a given integration, it is dropped for that integration only — other integrations are unaffected.

Incoming event
  → Organisation Data Layer (enrichment, identity)
  → Event Filtering          ← you are here
  → Pipeline Transformation Engine
  → Delivery

There are two levels of filtering:

LevelScopeExample
Global filtersApply to every event routed to the integration”Only send events where traits.country equals GB
Event-level filtersApply to a specific event type within the integration”Only send Purchase events where properties.revenue is set”

Event-level filters are checked after global filters. If a global filter drops the event, event-level filters are never evaluated.

Filter Operators

Each filter rule consists of a field path, an operator, and (for most operators) a value.

OperatorDescriptionValue requiredExample
is_setField exists and is not nullNotraits.email is set
is_not_setField is null or missingNotraits.email is not set
equalsField exactly matches valueYesproperties.currency equals USD
not_equalsField does not match valueYesevent.type not equals identify
containsString field contains substringYesproperties.url contains /checkout
matchesField matches a regular expressionYestraits.email matches ^.*@example\.com$

All string comparisons are case-sensitive. If you need case-insensitive matching, use the matches operator with the (?i) regex flag — for example (?i)^purchase$.

Logic Groups

Filter rules are combined using logic groups. Each group specifies whether all rules must pass (AND logic) or any rule must pass (OR logic).

all (AND)

Every rule in the group must evaluate to true for the event to pass.

filters:
  logic: all
  rules:
    - field: traits.email
      operator: is_set
    - field: properties.currency
      operator: equals
      value: "USD"

This means: deliver the event only if the user has an email and the currency is USD.

any (OR)

At least one rule in the group must evaluate to true for the event to pass.

filters:
  logic: any
  rules:
    - field: properties.source
      operator: equals
      value: "web"
    - field: properties.source
      operator: equals
      value: "mobile"

This means: deliver the event if the source is web or mobile.

Nesting Groups

Groups can be nested to express more complex logic:

filters:
  logic: all
  rules:
    - field: traits.email
      operator: is_set
    - logic: any
      rules:
        - field: properties.country
          operator: equals
          value: "GB"
        - field: properties.country
          operator: equals
          value: "US"

This means: email must be set and country must be either GB or US.

Evaluation Order

When an event is routed to an integration, filters are evaluated in this order:

  1. Global filters — checked first. If the event does not pass, it is dropped immediately.
  2. Event-level filters — checked next, only for the matching event type. If no event-level filter is defined for this event type, the event passes by default.
  3. Pipeline transformation — the pipeline steps run on the filtered event.
  4. Delivery — the transformed payload is published to the delivery topic.
Event arrives for integration

  ├─ Global filters pass?
  │    No  → drop event
  │    Yes ↓
  ├─ Event-level filter defined for this event type?
  │    No  → proceed (event passes by default)
  │    Yes → Event-level filters pass?
  │           No  → drop event
  │           Yes ↓
  ├─ Pipeline transformation

  └─ Delivery

Examples

Don’t send to Braze if email is empty

Set a global filter on the Braze integration so that no events are delivered unless the user’s email address is present.

# Global filter on the Braze integration
filters:
  logic: all
  rules:
    - field: traits.email
      operator: is_set

Any event routed to Braze where traits.email is null or missing will be silently dropped.

Only send purchase events if opted_in_marketing is true

Set an event-level filter on the integration for Purchase events only.

# Event-level filter for "Purchase" events
event_filters:
  Purchase:
    logic: all
    rules:
      - field: traits.opted_in_marketing
        operator: equals
        value: true

Other event types (e.g. Page View, Add to Cart) are unaffected by this filter and continue to be delivered normally.

Combined global and event-level filters

# Global: only process events from the UK
filters:
  logic: all
  rules:
    - field: context.geo.country
      operator: equals
      value: "GB"
 
# Event-level: for Purchase events, also require revenue > 0
event_filters:
  Purchase:
    logic: all
    rules:
      - field: properties.revenue
        operator: is_set

Blueprint Default Filters

Blueprints can include default filter configurations. When an integration is created from a blueprint, the blueprint’s filters are applied automatically. Users can then customise or override them.

# In a blueprint definition
defaults:
  filters:
    logic: all
    rules:
      - field: traits.email
        operator: is_set
  event_filters:
    Purchase:
      logic: all
      rules:
        - field: properties.order_id
          operator: is_set
        - field: properties.revenue
          operator: is_set

Blueprint default filters are a starting point. When you edit filters on an integration that was created from a blueprint, your changes take priority. Updating the blueprint later does not overwrite customised filters on existing integrations.

Relationship to Pipeline Filter Steps

Event filtering and pipeline filter steps serve different purposes:

FeatureEvent FilteringPipeline Filter Steps
Configured viaManagement UI (no code)Pipeline YAML/JSON
EvaluatedBefore the pipeline runsDuring pipeline execution
Operates onCanonical Datafly eventPartially transformed payload
Best forBusiness rules (who gets what)Technical routing (vendor format constraints)

You can use both together. Event filtering handles the high-level “should this integration receive this event?” decision, while pipeline filter steps handle vendor-specific requirements during transformation.