Custom Code

Datafly Signal supports sandboxed JavaScript execution within the processing pipeline. Custom code enables complex business logic that cannot be expressed through the declarative pipeline configuration alone.

Overview

Custom code runs in two contexts:

Context	When It Runs	Scope
Organisation-level	Step 10 of the Org Data Layer	Runs once per event, before pipeline processing
Pipeline-level	As a step within a specific integration’s pipeline	Runs per event per integration, during pipeline processing

Both contexts use the same sandboxed runtime and API surface.

Runtime Environment

Custom code executes inside a V8 isolate — the same JavaScript engine used in Chrome and Node.js, but in a stripped-down sandbox with no access to the host system.

Limits

Resource	Limit
Execution time	50ms per invocation
Memory	8 MB per isolate
Code size	64 KB per script
Network access	None
File system access	None
`require()` / `import`	Not available

⚠️

Custom code has no network or filesystem access. It cannot make HTTP requests, read files, or access environment variables. All data must be passed in via the event object and helper functions.

Supported JavaScript Features

The runtime supports ES2022 syntax including:

const, let, arrow functions
Template literals, destructuring, spread operators
async/await (but no external I/O to await)
Map, Set, WeakMap, WeakSet
Array.prototype.at(), Object.hasOwn()
structuredClone()
JSON.parse(), JSON.stringify()
String.prototype.replaceAll()
Regular expressions

API Reference

Entry Point

Your script must export a process function that receives the event and returns the modified event:

function process(event, context) {
  // Transform the event
  return event;
}

Parameter	Type	Description
`event`	object	The canonical event object (mutable)
`context`	object	Read-only context with org config and helpers

Return Value

Return	Behaviour
Modified event object	Event continues through the pipeline
`null`	Event is dropped (not sent to any integration)
Thrown error	Event is sent to the dead-letter topic

The `event` Object

The event object contains the full canonical event with all enrichments from preceding Org Data Layer steps:

function process(event, context) {
  // Event type: "track", "page", "identify", "group"
  event.type;
 
  // Event name (track events only)
  event.event;
 
  // User identity
  event.userId;
  event.anonymousId;
 
  // Event data
  event.properties; // track/page events
  event.traits; // identify/group events
 
  // Enriched context
  event.context.page; // { url, title, referrer, path }
  event.context.geo; // { country, region, city, ... }
  event.context.device; // { type, browser, os, ... }
  event.context.session; // { id, eventIndex, isNew }
  event.context.ip;
  event.context.userAgent;
 
  // Identity data
  event.vendorIds; // { ga4_client_id, fbp, ttp, ... }
  event.clickIds; // { gclid, fbclid, ... }
 
  return event;
}

The `context` Object

The context object provides read-only access to organisation configuration and helper functions:

function process(event, context) {
  // Organisation ID
  context.orgId;
 
  // Source ID
  context.sourceId;
 
  // Organisation-level settings (key-value pairs)
  context.settings; // { "brand": "acme", "region": "us", ... }
 
  // Helper functions
  context.helpers.hash.sha256("value"); // SHA-256 hash
  context.helpers.hash.md5("value"); // MD5 hash
  context.helpers.base64.encode("value"); // Base64 encode
  context.helpers.base64.decode("encoded"); // Base64 decode
  context.helpers.uuid(); // Generate UUID v4
  context.helpers.now(); // Current Unix timestamp (seconds)
  context.helpers.log("message"); // Log to event debugger
 
  return event;
}

context.helpers.log() writes to the event debugger in the Management UI, not to stdout. Use it for debugging during development; remove or gate it behind a setting for production.

Examples

Event Name Normalisation

Convert all event names to a consistent snake_case format:

function process(event, context) {
  if (event.event) {
    event.event = event.event
      .trim()
      .toLowerCase()
      .replace(/\s+/g, "_")
      .replace(/[^a-z0-9_]/g, "");
  }
  return event;
}

Custom Revenue Calculation

Compute total revenue from line items when the revenue property is not provided:

function process(event, context) {
  if (event.event === "Purchase" && !event.properties.revenue) {
    const products = event.properties.products || [];
    const total = products.reduce((sum, product) => {
      const price = product.price || 0;
      const quantity = product.quantity || 1;
      return sum + price * quantity;
    }, 0);
 
    event.properties.revenue = Math.round(total * 100) / 100;
    event.properties.currency = event.properties.currency || "USD";
  }
  return event;
}

UTM Parameter Parsing

Extract UTM parameters from the page URL and add them to the event properties:

function process(event, context) {
  const url = event.context?.page?.url;
  if (!url) return event;
 
  try {
    const searchParams = new URL(url).searchParams;
    const utmParams = [
      "utm_source",
      "utm_medium",
      "utm_campaign",
      "utm_term",
      "utm_content",
    ];
 
    const campaign = {};
    for (const param of utmParams) {
      const value = searchParams.get(param);
      if (value) {
        campaign[param.replace("utm_", "")] = value;
      }
    }
 
    if (Object.keys(campaign).length > 0) {
      event.context.campaign = {
        ...event.context.campaign,
        ...campaign,
      };
    }
  } catch {
    // Invalid URL, skip
  }
 
  return event;
}

Custom PII Detection

Apply business-specific PII rules that go beyond the built-in patterns:

function process(event, context) {
  const piiFields = [
    "properties.customer_notes",
    "properties.support_transcript",
    "properties.feedback",
  ];
 
  for (const fieldPath of piiFields) {
    const parts = fieldPath.split(".");
    let obj = event;
    for (let i = 0; i < parts.length - 1; i++) {
      obj = obj?.[parts[i]];
    }
 
    const lastKey = parts[parts.length - 1];
    if (obj && typeof obj[lastKey] === "string") {
      // Redact email addresses
      obj[lastKey] = obj[lastKey].replace(
        /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g,
        "[REDACTED_EMAIL]"
      );
 
      // Redact phone numbers
      obj[lastKey] = obj[lastKey].replace(
        /\+?[\d\s\-().]{10,}/g,
        "[REDACTED_PHONE]"
      );
    }
  }
 
  return event;
}

Conditional Event Dropping

Drop events that match specific criteria (e.g. internal traffic):

function process(event, context) {
  // Drop events from internal IP ranges
  const internalRanges = ["10.", "172.16.", "192.168.", "203.0.113."];
  const ip = event.context?.ip || "";
  if (internalRanges.some((range) => ip.startsWith(range))) {
    return null; // Drop the event
  }
 
  // Drop test events in production
  if (event.properties?.is_test === true) {
    return null;
  }
 
  return event;
}

Data Enrichment from Settings

Use organisation-level settings to enrich events:

function process(event, context) {
  // Add organisation metadata to every event
  event.properties._org_brand = context.settings.brand || "unknown";
  event.properties._org_region = context.settings.region || "unknown";
 
  // Map product categories using org-level config
  const categoryMap = JSON.parse(context.settings.category_map || "{}");
  if (event.properties.category_id) {
    event.properties.category_name =
      categoryMap[event.properties.category_id] || "Other";
  }
 
  return event;
}

Pipeline-Level Custom Code

Custom code can also run as a step within an integration’s pipeline. The syntax is the same but defined inline in the pipeline configuration:

steps:
  - type: map
    mappings:
      client_id: "event.vendorIds.ga4_client_id"
 
  - type: custom_code
    code: |
      function process(event, context) {
        // Complex mapping logic that can't be expressed declaratively
        if (event.properties.products) {
          event._items = event.properties.products.map((p, i) => ({
            item_id: p.product_id,
            item_name: p.name,
            price: p.price,
            index: i,
          }));
        }
        return event;
      }
 
  - type: map
    mappings:
      events[0].params.items: "event._items"

Error Handling

Scenario	Behaviour
Script throws an error	Event is sent to the dead-letter topic with the error message
Script exceeds 50ms timeout	Execution is terminated; event is sent to dead-letter topic
Script exceeds 8MB memory	Isolate is killed; event is sent to dead-letter topic
Script returns `undefined`	Event proceeds unchanged (treated as pass-through)
Script returns `null`	Event is dropped intentionally

Errors are visible in the Management UI’s event debugger with the full error message and stack trace.

⚠️

Custom code errors affect only the single event being processed. Other events continue processing normally. However, if a script consistently fails (e.g. a syntax error), every event will be sent to the dead-letter topic. Monitor the dead-letter topic and event debugger after deploying new custom code.

Best Practices

Keep scripts focused. Each script should do one thing well. If you have multiple unrelated transformations, use separate scripts or pipeline steps.
Handle missing data gracefully. Always use optional chaining (?.) and fallback values. Events may not always have every field.
Avoid side effects. The same event may be processed multiple times in failure-recovery scenarios. Ensure your code is idempotent.
Test with dry-run. Use the pipeline dry-run API or the Management UI’s test runner before deploying custom code to production.
Use context.helpers.log() sparingly. Each log call writes to the event debugger. Excessive logging can slow down the debugger UI.
Prefer declarative steps where possible. Map, filter, transform, and compute steps are faster and easier to maintain than custom code. Reserve custom code for logic that truly requires it.

Expressions Overview