Amazon RDS
Datafly Signal writes first-party events directly to an Amazon RDS instance over the native database protocol. Signal connects to RDS exactly as any application would — there is no Data API or special proxy involved.
The engine setting selects the writer:
postgres: RDS for PostgreSQL or Aurora PostgreSQL, over the PostgreSQL wire protocol.mysql: RDS for MySQL, RDS for MariaDB or Aurora MySQL, over the MySQL wire protocol.sqlserver: RDS for SQL Server, over the same TDS protocol as Azure SQL.
RDS for Oracle and RDS for Db2 are not supported.
Why route Amazon RDS through Datafly Signal
- The data lands in infrastructure you own. Signal writes to your Amazon RDS directly from your own deployment. Datafly operates no intermediary and never holds a copy of your event data.
- Personal data is hashed on the way in. Email, phone (shaped to E.164 first), first, last and full name and postcode are normalised (trimmed and lower-cased) and hashed with SHA-256 inside your own deployment, so the table holds
email_sha256,phone_sha256and the other digests rather than the raw values. Storing a raw value is a deliberate edit to the installed blueprint. - Only what the blueprint maps is stored. The properties Signal writes are exactly the fields the blueprint maps. Identify traits and client properties the blueprint does not map are never written, so a stray email in an event property cannot land in Amazon RDS in plain text.
- Presets for Retail, Insurance, Travel, Media & Streaming, B2B / SaaS, Automotive and General. Each vertical preset names one field per property in the matching event spec, with canonical names (
transaction_id,value,items), legacy aliases (order_id,revenue,products) read automatically, numbers typed as numbers and currency codes upper-cased. - Client IP and user agent are recorded as event context. Both come from the original visitor request rather than from your server, so geography and device analysis done in the warehouse reflects the actual visitor.
- One consent decision governs every destination. Amazon RDS is gated by the same consent state as the rest of your stack, so a withdrawal applies everywhere at once.
Prerequisites
Before configuring Amazon RDS in Signal you need an AWS account with an RDS instance running, a target schema/table, a database user, and network connectivity from Signal to the instance.
Create an RDS Instance
- Open the RDS console.
- Click Create database.
- Choose the engine: PostgreSQL, MySQL, MariaDB, SQL Server, Aurora PostgreSQL, or Aurora MySQL.
- Pick a deployment option (Single-AZ for dev, Multi-AZ for production), instance class, and storage settings.
- Set the Master username and Master password.
- Configure VPC, subnet group, and public accessibility as needed.
- Click Create database.
Note the Endpoint (e.g. datafly-db.abc123.us-east-1.rds.amazonaws.com) and Port (5432 / 3306 / 1433 depending on engine) once the instance is available.
Configure Network Access
- Open the RDS instance’s VPC security group.
- Add an Inbound rule allowing the relevant port from the IP addresses or CIDR range of your Signal deployment.
- If Signal runs outside the VPC, enable Publicly accessible on the instance.
Never allow 0.0.0.0/0 access to a production database. Restrict inbound rules to your Signal infrastructure’s IP addresses or use a VPC peering / PrivateLink connection.
Create a Database, Schema, and Table
Connect to the instance with psql, mysql, or sqlcmd and create the target objects. Use the table definition from the matching engine guide:
Create a Database User for Signal
Create a least-privilege user with INSERT on the target table (plus SELECT on its message_id column for PostgreSQL and SQL Server). Example for PostgreSQL:
CREATE USER datafly_signal WITH PASSWORD 'your_secure_password';
GRANT USAGE ON SCHEMA datafly TO datafly_signal;
GRANT INSERT ON TABLE datafly.events TO datafly_signal;
GRANT SELECT (message_id) ON TABLE datafly.events TO datafly_signal;PostgreSQL needs SELECT on message_id for the ON CONFLICT (message_id) insert and SQL Server needs it for the existence check before the insert; MySQL needs INSERT only. See the grants on the PostgreSQL, MySQL and Azure SQL pages.
Configuration
Install the integration once and pick the engine; all three engines take the same fields and write the same columns.
| Field | Type | Required | Description |
|---|---|---|---|
engine | select | Yes | postgres (default), mysql or sqlserver. Selects the writer. |
host | string | Yes | The RDS endpoint hostname or IP address (for SQL Server, server is also accepted). |
port | string | No | Database port. Blank uses the chosen engine’s default (5432 PostgreSQL, 3306 MySQL, 1433 SQL Server); another engine’s default port is treated as blank, so an integration switched to MySQL or SQL Server does not keep 5432. Set it only for a non-default port. |
database | string | Yes | The target database name. |
schema_name | string | No | PostgreSQL (default public) and SQL Server (default dbo); ignored for MySQL. |
table_name | string | Yes | The target table name. |
username | string | Yes | Database username with INSERT on the table. |
password | secret | Yes | Database password. |
ssl_mode | string | No | PostgreSQL only: libpq sslmode (disable, require, verify-ca, verify-full). Defaults to require. |
tls_mode | string | No | MySQL only: true verifies the server certificate and host name, against ca_certificate when set (otherwise the system roots, which do not include the managed service’s CA); verify-ca verifies the chain against ca_certificate but not the host name; skip-verify and preferred encrypt without verifying; false or empty connects without TLS. |
encrypt | string | No | SQL Server only: true (default), false or disable. With true and false the server certificate is verified (false still encrypts the login), against ca_certificate when set, otherwise the system roots. disable connects without encryption and cannot be combined with ca_certificate. |
ca_certificate | textarea | No | MySQL and SQL Server: the PEM bundle of the certificate authority that signed the server’s certificate (the Amazon RDS global bundle), so the connection is encrypted and verified. Not used by PostgreSQL. |
Signal Setup
Quick Setup
- Navigate to Integrations in the sidebar.
- Open the Integration Library tab.
- Find Amazon RDS or filter by Database.
- Click Install, choose the Database Engine (
postgres,mysqlorsqlserver), and fill in the required fields. - Click Install Integration to create the integration with a ready-to-use default blueprint.
API Setup (PostgreSQL example)
curl -X POST http://localhost:8084/v1/admin/integration-catalog/amazon_rds/install \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Amazon RDS (PostgreSQL)",
"variant": "postgres",
"config": {
"engine": "postgres",
"host": "datafly-db.abc123.us-east-1.rds.amazonaws.com",
"port": "5432",
"database": "datafly_events",
"schema_name": "datafly",
"table_name": "events",
"username": "datafly_signal",
"password": "your_secure_password",
"ssl_mode": "require"
},
"delivery_mode": "server_side"
}'Schema
Signal writes the same sixteen columns on every engine (PostgreSQL, MySQL and SQL Server); the engine field selects the writer. Create the table with the DDL on the matching engine page and use its query examples:
- PostgreSQL: table and queries (
message_idmust be the primary key: inserts useON CONFLICT (message_id) DO NOTHING) - MySQL: table and queries (inserts use
INSERT IGNORE) - SQL Server: table and queries (inserts skip an existing
message_id)
| Column | Contents |
|---|---|
message_id | Unique per delivered event; the idempotency key (a redelivery is not inserted twice). |
event_type | track, page, identify, and so on. |
event_name | Signal event name (snake_case), for example purchase. |
anonymous_id | First-party visitor identifier. |
user_id | Logged-in user identifier; NULL when absent. |
ip_address, user_agent | Client IP and user agent from the visitor’s request. |
page_url, page_path, page_referrer | Page context. |
org_id, pipeline_id | Signal identifiers. |
event_timestamp | Client event time (UTC). |
received_at | Time Signal received the event (UTC). |
properties_json | The fields the blueprint mapped, as JSON (below). |
context_json | The event context as JSON: page, device, campaign, consent, IP and user agent. |
What the properties hold
The blueprint decides what properties_json contains, and only mapped fields are written (identify traits and unmapped client properties never are):
- The event’s own fields from the preset sit at the top level, for example
transaction_id,value,currency,items,insurance_typeorpolicy_term_months. - The preset’s global block sits under
@root: copies of the envelope (event_name,event_type,user_id,anonymous_id,event_id,message_id,event_timestamp,page_url,page_referrer,ip_address,user_agent,locale,timezone,source_id),utm_source,utm_medium,utm_campaign, the consent flagsconsent_analyticsandconsent_marketing, and the hashed identityemail_sha256,phone_sha256,first_name_sha256,last_name_sha256,name_sha256. identifyaddsname_sha256andpostal_code_sha256at the top level.- An event the preset does not list stores its envelope and the
@rootblock only.
For example, on PostgreSQL:
SELECT event_timestamp,
properties_json->>'transaction_id' AS transaction_id,
(properties_json->>'value')::numeric AS value,
properties_json->>'currency' AS currency,
properties_json->'@root'->>'email_sha256' AS email_sha256
FROM datafly.events
WHERE event_name = 'purchase'
ORDER BY event_timestamp DESC
LIMIT 10;Signal does not write mapped fields to their own columns; create views over properties_json for the fields you query often. What else a row carries outside the blueprint (the whole event context) is under review in #1414.
Stored event context
Alongside the fields your blueprint maps (where email, phone, names and address are stored as normalised SHA-256 by default), each row carries the event’s collected context exactly as Signal received it. The blueprint does not filter or hash the context: this is deliberate (decision 2026-09-23), so the context is complete for attribution, fraud and debugging queries, and it is listed here so you can decide what to keep.
| Stored | What it contains |
|---|---|
| Visitor request | IP address, user agent, Accept-Language, locale and timezone |
| Page | URL including its query string, path, title and referrer |
| Campaign | UTM source, medium, campaign, term and content |
| Consent | The visitor’s consent state |
| Location | GeoIP country, region and city, and the network (ASN) |
| Device | Screen size; for app events the OS and version, manufacturer, model, advertising ID and IDFV when the app collected them |
| Vendor IDs | Cookie and click identifiers in vendor_ids (for example the _fbp browser ID, gclid, ttclid, the GA client ID) |
| Session and library | Session ID, number and engagement time; the collector’s name and version |
| Partner attributes | Profile attributes from partner enrichment (partner_attrs) |
They are written to the ip_address, user_agent, page_url, page_path and page_referrer columns and, in full, to the context_json column, beside anonymous_id and user_id. Click and collected IDs outside the context are stored only where the blueprint maps them.
The IP address, device and advertising IDs, and cookie and click identifiers are personal data under UK and EU GDPR. To keep any of them out of storage, strip, mask or hash the field for every integration with the organisation data layer, apply URL redaction to page URLs, or remove it in the installed blueprint; and set the storage’s retention to match your privacy notice. See Personal data by integration.
Consent
Amazon RDS is a first-party destination under your control, gated by the same consent state as every other destination. Each stored event also records the visitor’s consent at collection time: the preset writes consent_analytics and consent_marketing under @root in the properties (properties_json), and the full consent object is in the context. Use them in views or queries if your governance needs a per-row consent filter.
Testing
-
Enable the integration in Signal and trigger a test event on your website.
-
Connect to the RDS instance and query the target table:
SELECT message_id, event_type, event_name, event_timestamp FROM datafly.events ORDER BY event_timestamp DESC LIMIT 10; -
Verify that event rows are appearing with correct data.
-
In Signal, check the Live Events view to confirm delivery status shows as successful.
Troubleshooting
| Problem | Solution |
|---|---|
| Connection refused / timeout | Check the VPC security group allows inbound traffic on the relevant port from Signal’s IP addresses. Verify the instance is publicly accessible if Signal runs outside the VPC. |
| Authentication failure | Verify the database username and password. Check that the user has not been locked out. |
permission denied | The user lacks INSERT on the table, SELECT on its message_id column (PostgreSQL and SQL Server) or USAGE on the schema (PostgreSQL). Grant the required permissions. |
relation/table does not exist | Verify the database, schema, and table names. Use fully-qualified names. |
| SSL connection errors | Most RDS engines require SSL. PostgreSQL: ssl_mode=require. MySQL and SQL Server: paste the Amazon RDS global CA bundle into ca_certificate (with tls_mode true, or encrypt true); without it the RDS certificate authority is not in the system roots and verification fails. tls_mode=skip-verify encrypts without verifying the server. |
| Slow inserts | Add indexes that match your query patterns; verify the instance class is not bottlenecked. |
Visit Amazon RDS documentation for full engine-specific configuration guides.