Amazon Redshift

Datafly Signal writes events directly into a Redshift table so you can analyse first-party behaviour alongside the rest of your AWS data estate.

Why route Amazon Redshift through Datafly Signal

  • The data lands in infrastructure you own. Signal writes to your Amazon Redshift directly from your own deployment. Datafly operates no intermediary and never holds a copy of your event data.
  • Personal data is hashed on the way in. Email, phone (shaped to E.164 first), first and last name, full name and postcode are normalised (trim and lowercase) and hashed with SHA-256 inside your own deployment, so the table holds email_sha256, phone_sha256 and the other digests rather than raw personal data. A raw column is an opt-in edit to the installed blueprint.
  • Client IP and user agent are recorded as event context. Both come from the original visitor request rather than from your server, so geography and device analysis done in the warehouse reflects the actual visitor.
  • Presets for Retail, Insurance, Travel, Media & Streaming, Publishing, B2B / SaaS, Automotive and General. Each vertical preset names one field per property in the matching event spec, with canonical names (transaction_id, value, items), legacy aliases (order_id, revenue, products) read automatically, and numeric properties typed as numbers. Every other event still lands with its properties.
  • One consent decision governs every destination. Amazon Redshift is gated by the same consent state as the rest of your stack, so a withdrawal applies everywhere at once.

Prerequisites

Before configuring Amazon Redshift in Signal, you need an AWS account with a Redshift cluster, a target schema and table, and a database user with INSERT privileges.

Create an AWS Account

If you don’t already have one, sign up at aws.amazon.com.

Create a Redshift Cluster

  1. Open the Redshift console.
  2. Click Create cluster.
  3. Enter a Cluster identifier (e.g. datafly-analytics).
  4. Choose a node type and number of nodes based on your expected data volume.
  5. Set a Admin user name and Admin user password.
  6. Under Network and security, configure the VPC and subnet group.
  7. Click Create cluster.
  8. Once the cluster is available, note the Endpoint (e.g. datafly-analytics.abc123.us-east-1.redshift.amazonaws.com) and Port (default 5439).

Configure Network Access

  1. In the Redshift cluster details, click the VPC security group link.
  2. Add an Inbound rule allowing TCP traffic on port 5439 from the IP addresses or CIDR range of your Signal deployment.
  3. If Signal runs outside AWS, enable Publicly accessible on the cluster (under Modify > Network and security).
⚠️

If your cluster is publicly accessible, restrict the security group inbound rules to only the IP addresses of your Signal infrastructure. Never allow 0.0.0.0/0 access to a production cluster.

Create a Schema and Table

Connect to the cluster using the Redshift Query Editor or a SQL client (e.g. psql, DBeaver):

CREATE SCHEMA datafly;
 
CREATE TABLE datafly.events (
  message_id      VARCHAR(64),
  event_type      VARCHAR(20),
  event_name      VARCHAR(256),
  anonymous_id    VARCHAR(64),
  user_id         VARCHAR(256),
  ip_address      VARCHAR(64),
  user_agent      VARCHAR(1024),
  page_url        VARCHAR(4096),
  page_path       VARCHAR(2048),
  page_referrer   VARCHAR(4096),
  org_id          VARCHAR(64),
  pipeline_id     VARCHAR(64),
  event_timestamp TIMESTAMP,
  received_at     TIMESTAMP,
  properties_json VARCHAR(65535),
  context_json    VARCHAR(65535)
)
DISTKEY(anonymous_id)
SORTKEY(event_timestamp);

Signal inserts exactly these column names. Using anonymous_id as the distribution key and event_timestamp as the sort key optimises joins by visitor and time-range queries. Signal binds properties_json and context_json as JSON text without JSON_PARSE, so a SUPER column would hold each document as a single string value (AWS: without JSON_PARSE, Redshift “treats the value as a single string”) and navigation such as properties_json.transaction_id would return NULL. Keep them VARCHAR and parse at query time, or build a view with JSON_PARSE. A row whose JSON exceeds 65,535 bytes is rejected. Native SUPER support is tracked in #1419.

Create a Database User for Signal

Create a dedicated user with limited privileges:

CREATE USER datafly_signal PASSWORD 'your_secure_password';
GRANT USAGE ON SCHEMA datafly TO datafly_signal;
GRANT INSERT ON TABLE datafly.events TO datafly_signal;

Configuration

FieldTypeRequiredDescription
hoststringYesThe Redshift cluster endpoint hostname.
portstringYesThe Redshift port. Defaults to 5439.
databasestringYesThe database name.
schema_namestringYesThe schema containing the target table (default public).
table_namestringYesThe target table (for example events).
ssl_modestringNoPostgreSQL SSL mode; defaults to require.
usernamestringYesThe database username for authentication.
passwordsecretYesThe database password for authentication.

Signal Setup

Quick Setup

  1. Navigate to Integrations in the sidebar.
  2. Open the Integration Library tab.
  3. Find Amazon Redshift or filter by Warehouse.
  4. Click Install, select a variant if available, and fill in the required fields.
  5. Click Install Integration to create the integration with a ready-to-use default blueprint.

API Setup

curl -X POST http://localhost:8084/v1/admin/integration-catalog/amazon_redshift/install \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Amazon Redshift",
    "variant": "default",
    "config": {
      "host": "datafly-analytics.abc123.us-east-1.redshift.amazonaws.com",
      "port": "5439",
      "database": "analytics",
      "schema_name": "datafly",
      "table_name": "events",
      "username": "datafly_signal",
      "password": "your_secure_password"
    },
    "delivery_mode": "server_side"
  }'

Schema

Signal inserts one row per event over these columns:

ColumnRedshift typeNotes
message_idVARCHAR(64)Unique per delivered event.
event_typeVARCHAR(20)track, page, identify, etc.
event_nameVARCHAR(256)Signal event name (snake_case).
anonymous_idVARCHAR(64)First-party visitor identifier. Good DISTKEY.
user_idVARCHAR(256)Logged-in user identifier (nullable).
ip_address, user_agentVARCHARClient IP and user agent.
page_url, page_path, page_referrerVARCHARPage context.
org_id, pipeline_idVARCHAR(64)Signal identifiers.
event_timestampTIMESTAMPClient event time. Use as SORTKEY.
received_atTIMESTAMPTime Signal received the event.
properties_jsonVARCHAR(65535)The event as the blueprint shaped it, as JSON text.
context_jsonVARCHAR(65535)Page, device and consent context, as JSON text.

Where mapped fields land

The table layout is fixed: the envelope columns above plus properties_json. The blueprint shapes properties_json: event fields from the preset (for example transaction_id, value, insurance_type, policy_term_months) sit at its top level, and the preset’s envelope block (identity digests, UTM, consent flags) sits under @root:

SELECT event_timestamp,
       JSON_EXTRACT_PATH_TEXT(properties_json, 'transaction_id')        AS policy_number,
       JSON_EXTRACT_PATH_TEXT(properties_json, 'value')::FLOAT          AS premium,
       JSON_EXTRACT_PATH_TEXT(properties_json, '@root', 'email_sha256') AS email_sha256
FROM datafly.events
WHERE event_name = 'purchase';

Signal does not write mapped fields to their own typed columns yet. Create views over properties_json for the columns you query often.

Stored event context

Alongside the fields your blueprint maps (where email, phone, names and address are stored as normalised SHA-256 by default), each row carries the event’s collected context exactly as Signal received it. The blueprint does not filter or hash the context: this is deliberate (decision 2026-09-23), so the context is complete for attribution, fraud and debugging queries, and it is listed here so you can decide what to keep.

StoredWhat it contains
Visitor requestIP address, user agent, Accept-Language, locale and timezone
PageURL including its query string, path, title and referrer
CampaignUTM source, medium, campaign, term and content
ConsentThe visitor’s consent state
LocationGeoIP country, region and city, and the network (ASN)
DeviceScreen size; for app events the OS and version, manufacturer, model, advertising ID and IDFV when the app collected them
Vendor IDsCookie and click identifiers in vendor_ids (for example the _fbp browser ID, gclid, ttclid, the GA client ID)
Session and librarySession ID, number and engagement time; the collector’s name and version
Partner attributesProfile attributes from partner enrichment (partner_attrs)

They are written to the ip_address, user_agent, page_url, page_path and page_referrer columns and, in full, to the context_json column, beside anonymous_id and user_id. Click and collected IDs outside the context are stored only where the blueprint maps them.

The IP address, device and advertising IDs, and cookie and click identifiers are personal data under UK and EU GDPR. To keep any of them out of storage, strip, mask or hash the field for every integration with the organisation data layer, apply URL redaction to page URLs, or remove it in the installed blueprint; and set the storage’s retention to match your privacy notice. See Personal data by integration.

Redshift is a first-party destination under your control. The default blueprint forwards all events. Apply consent filtering inside your pipeline transforms or via downstream materialised views on the preset’s consent flags (JSON_EXTRACT_PATH_TEXT(properties_json, '@root', 'consent_analytics')) or context_json if needed.

Testing

  1. Enable the integration in Signal and trigger a test event on your website.
  2. Connect to the Redshift cluster and query the target table:
SELECT * FROM datafly.events ORDER BY event_timestamp DESC LIMIT 10;
  1. Verify event rows are appearing with correct data.
  2. In Signal, check the Live Events view to confirm delivery status shows as successful.

Troubleshooting

ProblemSolution
Events not appearing in RedshiftVerify the host, port, database, schema, and table name are correct.
Connection refused / timeoutCheck that the security group allows inbound traffic on port 5439 from Signal’s IP addresses. Verify the cluster is publicly accessible if Signal runs outside the VPC.
permission denied for schemaThe database user lacks USAGE on the schema. Run GRANT USAGE ON SCHEMA datafly TO datafly_signal;.
permission denied for relationThe database user lacks INSERT on the table. Run GRANT INSERT ON TABLE datafly.events TO datafly_signal;.
SSL connection errorsRedshift requires SSL by default. Ensure your Signal deployment trusts the Amazon root CA certificates.
Slow insertsSignal inserts one row per event, which Redshift handles poorly at high volume (AWS recommends COPY or multi-row inserts). For high-volume pipelines, land events in Amazon S3 and load them with COPY.
Credential errorsVerify the username and password are correct. Check that the user has not been locked out.

Visit Amazon Redshift documentation for full SQL reference and cluster management guides.

See also