Amazon Redshift
Datafly Signal writes events directly into a Redshift table so you can analyse first-party behaviour alongside the rest of your AWS data estate.
Why route Amazon Redshift through Datafly Signal
- The data lands in infrastructure you own. Signal writes to your Amazon Redshift directly from your own deployment. Datafly operates no intermediary and never holds a copy of your event data.
- Personal data is hashed on the way in. Email, phone (shaped to E.164 first), first and last name, full name and postcode are normalised (trim and lowercase) and hashed with SHA-256 inside your own deployment, so the table holds
email_sha256,phone_sha256and the other digests rather than raw personal data. A raw column is an opt-in edit to the installed blueprint. - Client IP and user agent are recorded as event context. Both come from the original visitor request rather than from your server, so geography and device analysis done in the warehouse reflects the actual visitor.
- Presets for Retail, Insurance, Travel, Media & Streaming, Publishing, B2B / SaaS, Automotive and General. Each vertical preset names one field per property in the matching event spec, with canonical names (
transaction_id,value,items), legacy aliases (order_id,revenue,products) read automatically, and numeric properties typed as numbers. Every other event still lands with its properties. - One consent decision governs every destination. Amazon Redshift is gated by the same consent state as the rest of your stack, so a withdrawal applies everywhere at once.
Prerequisites
Before configuring Amazon Redshift in Signal, you need an AWS account with a Redshift cluster, a target schema and table, and a database user with INSERT privileges.
Create an AWS Account
If you don’t already have one, sign up at aws.amazon.com.
Create a Redshift Cluster
- Open the Redshift console.
- Click Create cluster.
- Enter a Cluster identifier (e.g.
datafly-analytics). - Choose a node type and number of nodes based on your expected data volume.
- Set a Admin user name and Admin user password.
- Under Network and security, configure the VPC and subnet group.
- Click Create cluster.
- Once the cluster is available, note the Endpoint (e.g.
datafly-analytics.abc123.us-east-1.redshift.amazonaws.com) and Port (default5439).
Configure Network Access
- In the Redshift cluster details, click the VPC security group link.
- Add an Inbound rule allowing TCP traffic on port
5439from the IP addresses or CIDR range of your Signal deployment. - If Signal runs outside AWS, enable Publicly accessible on the cluster (under Modify > Network and security).
If your cluster is publicly accessible, restrict the security group inbound rules to only the IP addresses of your Signal infrastructure. Never allow 0.0.0.0/0 access to a production cluster.
Create a Schema and Table
Connect to the cluster using the Redshift Query Editor or a SQL client (e.g. psql, DBeaver):
CREATE SCHEMA datafly;
CREATE TABLE datafly.events (
message_id VARCHAR(64),
event_type VARCHAR(20),
event_name VARCHAR(256),
anonymous_id VARCHAR(64),
user_id VARCHAR(256),
ip_address VARCHAR(64),
user_agent VARCHAR(1024),
page_url VARCHAR(4096),
page_path VARCHAR(2048),
page_referrer VARCHAR(4096),
org_id VARCHAR(64),
pipeline_id VARCHAR(64),
event_timestamp TIMESTAMP,
received_at TIMESTAMP,
properties_json VARCHAR(65535),
context_json VARCHAR(65535)
)
DISTKEY(anonymous_id)
SORTKEY(event_timestamp);Signal inserts exactly these column names. Using anonymous_id as the distribution key and event_timestamp as the sort key optimises joins by visitor and time-range queries. Signal binds properties_json and context_json as JSON text without JSON_PARSE, so a SUPER column would hold each document as a single string value (AWS: without JSON_PARSE, Redshift “treats the value as a single string”) and navigation such as properties_json.transaction_id would return NULL. Keep them VARCHAR and parse at query time, or build a view with JSON_PARSE. A row whose JSON exceeds 65,535 bytes is rejected. Native SUPER support is tracked in #1419.
Create a Database User for Signal
Create a dedicated user with limited privileges:
CREATE USER datafly_signal PASSWORD 'your_secure_password';
GRANT USAGE ON SCHEMA datafly TO datafly_signal;
GRANT INSERT ON TABLE datafly.events TO datafly_signal;Configuration
| Field | Type | Required | Description |
|---|---|---|---|
host | string | Yes | The Redshift cluster endpoint hostname. |
port | string | Yes | The Redshift port. Defaults to 5439. |
database | string | Yes | The database name. |
schema_name | string | Yes | The schema containing the target table (default public). |
table_name | string | Yes | The target table (for example events). |
ssl_mode | string | No | PostgreSQL SSL mode; defaults to require. |
username | string | Yes | The database username for authentication. |
password | secret | Yes | The database password for authentication. |
Signal Setup
Quick Setup
- Navigate to Integrations in the sidebar.
- Open the Integration Library tab.
- Find Amazon Redshift or filter by Warehouse.
- Click Install, select a variant if available, and fill in the required fields.
- Click Install Integration to create the integration with a ready-to-use default blueprint.
API Setup
curl -X POST http://localhost:8084/v1/admin/integration-catalog/amazon_redshift/install \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Amazon Redshift",
"variant": "default",
"config": {
"host": "datafly-analytics.abc123.us-east-1.redshift.amazonaws.com",
"port": "5439",
"database": "analytics",
"schema_name": "datafly",
"table_name": "events",
"username": "datafly_signal",
"password": "your_secure_password"
},
"delivery_mode": "server_side"
}'Schema
Signal inserts one row per event over these columns:
| Column | Redshift type | Notes |
|---|---|---|
message_id | VARCHAR(64) | Unique per delivered event. |
event_type | VARCHAR(20) | track, page, identify, etc. |
event_name | VARCHAR(256) | Signal event name (snake_case). |
anonymous_id | VARCHAR(64) | First-party visitor identifier. Good DISTKEY. |
user_id | VARCHAR(256) | Logged-in user identifier (nullable). |
ip_address, user_agent | VARCHAR | Client IP and user agent. |
page_url, page_path, page_referrer | VARCHAR | Page context. |
org_id, pipeline_id | VARCHAR(64) | Signal identifiers. |
event_timestamp | TIMESTAMP | Client event time. Use as SORTKEY. |
received_at | TIMESTAMP | Time Signal received the event. |
properties_json | VARCHAR(65535) | The event as the blueprint shaped it, as JSON text. |
context_json | VARCHAR(65535) | Page, device and consent context, as JSON text. |
Where mapped fields land
The table layout is fixed: the envelope columns above plus properties_json. The blueprint shapes properties_json: event fields from the preset (for example transaction_id, value, insurance_type, policy_term_months) sit at its top level, and the preset’s envelope block (identity digests, UTM, consent flags) sits under @root:
SELECT event_timestamp,
JSON_EXTRACT_PATH_TEXT(properties_json, 'transaction_id') AS policy_number,
JSON_EXTRACT_PATH_TEXT(properties_json, 'value')::FLOAT AS premium,
JSON_EXTRACT_PATH_TEXT(properties_json, '@root', 'email_sha256') AS email_sha256
FROM datafly.events
WHERE event_name = 'purchase';Signal does not write mapped fields to their own typed columns yet. Create views over properties_json for the columns you query often.
Stored event context
Alongside the fields your blueprint maps (where email, phone, names and address are stored as normalised SHA-256 by default), each row carries the event’s collected context exactly as Signal received it. The blueprint does not filter or hash the context: this is deliberate (decision 2026-09-23), so the context is complete for attribution, fraud and debugging queries, and it is listed here so you can decide what to keep.
| Stored | What it contains |
|---|---|
| Visitor request | IP address, user agent, Accept-Language, locale and timezone |
| Page | URL including its query string, path, title and referrer |
| Campaign | UTM source, medium, campaign, term and content |
| Consent | The visitor’s consent state |
| Location | GeoIP country, region and city, and the network (ASN) |
| Device | Screen size; for app events the OS and version, manufacturer, model, advertising ID and IDFV when the app collected them |
| Vendor IDs | Cookie and click identifiers in vendor_ids (for example the _fbp browser ID, gclid, ttclid, the GA client ID) |
| Session and library | Session ID, number and engagement time; the collector’s name and version |
| Partner attributes | Profile attributes from partner enrichment (partner_attrs) |
They are written to the ip_address, user_agent, page_url, page_path and page_referrer columns and, in full, to the context_json column, beside anonymous_id and user_id. Click and collected IDs outside the context are stored only where the blueprint maps them.
The IP address, device and advertising IDs, and cookie and click identifiers are personal data under UK and EU GDPR. To keep any of them out of storage, strip, mask or hash the field for every integration with the organisation data layer, apply URL redaction to page URLs, or remove it in the installed blueprint; and set the storage’s retention to match your privacy notice. See Personal data by integration.
Consent
Redshift is a first-party destination under your control. The default blueprint forwards all events. Apply consent filtering inside your pipeline transforms or via downstream materialised views on the preset’s consent flags (JSON_EXTRACT_PATH_TEXT(properties_json, '@root', 'consent_analytics')) or context_json if needed.
Testing
- Enable the integration in Signal and trigger a test event on your website.
- Connect to the Redshift cluster and query the target table:
SELECT * FROM datafly.events ORDER BY event_timestamp DESC LIMIT 10;- Verify event rows are appearing with correct data.
- In Signal, check the Live Events view to confirm delivery status shows as successful.
Troubleshooting
| Problem | Solution |
|---|---|
| Events not appearing in Redshift | Verify the host, port, database, schema, and table name are correct. |
| Connection refused / timeout | Check that the security group allows inbound traffic on port 5439 from Signal’s IP addresses. Verify the cluster is publicly accessible if Signal runs outside the VPC. |
permission denied for schema | The database user lacks USAGE on the schema. Run GRANT USAGE ON SCHEMA datafly TO datafly_signal;. |
permission denied for relation | The database user lacks INSERT on the table. Run GRANT INSERT ON TABLE datafly.events TO datafly_signal;. |
| SSL connection errors | Redshift requires SSL by default. Ensure your Signal deployment trusts the Amazon root CA certificates. |
| Slow inserts | Signal inserts one row per event, which Redshift handles poorly at high volume (AWS recommends COPY or multi-row inserts). For high-volume pipelines, land events in Amazon S3 and load them with COPY. |
| Credential errors | Verify the username and password are correct. Check that the user has not been locked out. |
Visit Amazon Redshift documentation for full SQL reference and cluster management guides.