Amazon DynamoDB
Datafly Signal writes first-party events as items into a DynamoDB table — fully managed, serverless NoSQL with single-digit millisecond performance at any scale.
Why route Amazon DynamoDB through Datafly Signal
- The data lands in infrastructure you own. Signal writes to your Amazon DynamoDB directly from your own deployment. Datafly operates no intermediary and never holds a copy of your event data.
- Personal data is hashed on the way in. Email, phone (shaped to E.164 first), first, last and full name and postcode are normalised (trimmed and lower-cased) and hashed with SHA-256 inside your own deployment, so the table holds
email_sha256,phone_sha256and the other digests rather than the raw values. Storing a raw value is a deliberate edit to the installed blueprint. - Only what the blueprint maps is stored. The properties Signal writes are exactly the fields the blueprint maps. Identify traits and client properties the blueprint does not map are never written, so a stray email in an event property cannot land in DynamoDB in plain text.
- Presets for Retail, Insurance, Travel, Media & Streaming, B2B / SaaS, Automotive and General. Each vertical preset names one field per property in the matching event spec, with canonical names (
transaction_id,value,items), legacy aliases (order_id,revenue,products) read automatically, numbers typed as numbers and currency codes upper-cased. - Client IP and user agent are recorded as event context. Both come from the original visitor request rather than from your server, so geography and device analysis done in the warehouse reflects the actual visitor.
- One consent decision governs every destination. Amazon DynamoDB is gated by the same consent state as the rest of your stack, so a withdrawal applies everywhere at once.
Prerequisites
Before configuring Amazon DynamoDB in Signal, you need an AWS account with a DynamoDB table and IAM credentials. Follow the steps below.
Create an AWS Account
If you don’t already have one, sign up at aws.amazon.com. Ensure billing is configured and you have console access.
Create a DynamoDB Table
- Open the DynamoDB console.
- Click Create table.
- Enter a Table name (e.g.
datafly_events). - Set the Partition key to
message_id(type: String) and add no sort key: Signal writes each item bymessage_idalone, and a table with a sort key rejects the write. For time-range queries, add a global secondary index later (for example partition keyevent_name, sort keyevent_timestamp). - Leave the default settings (on-demand capacity mode is recommended for unpredictable workloads) or choose provisioned capacity if you have predictable traffic.
- Click Create table.
On-demand capacity mode automatically scales to handle your traffic. If you expect consistent high throughput, provisioned capacity with auto-scaling may be more cost-effective.
Create an IAM User for Signal
- Open the IAM console.
- Go to Users > Create user.
- Enter a username (e.g.
datafly-signal-dynamodb). - Select Attach policies directly.
- Click Create policy and use the following JSON:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:PutItem",
"dynamodb:BatchWriteItem"
],
"Resource": "arn:aws:dynamodb:us-east-1:123456789012:table/datafly_events"
}
]
}- Replace the region, account ID, and table name with your values.
- Attach the policy to the user.
Generate Access Keys
- On the IAM user detail page, go to Security credentials.
- Under Access keys, click Create access key.
- Select Application running outside AWS.
- Click Create access key.
- Copy the Access Key ID and Secret Access Key immediately — the secret is only shown once.
Store these credentials securely. If you lose the secret access key, you must create a new key pair.
Configuration
| Field | Type | Required | Description |
|---|---|---|---|
table_name | string | Yes | The DynamoDB table name to write items to. |
region | select | Yes | The AWS region where the DynamoDB table is located. |
access_key_id | secret | Yes | AWS access key ID with dynamodb:PutItem and dynamodb:BatchWriteItem. |
secret_access_key | secret | Yes | Secret access key for the access key ID. |
session_token | secret | No | Optional STS session token for temporary credentials. |
The table’s partition key must be message_id (String) with no sort key: Signal always writes the item keyed on message_id, and the blueprint does not change the key.
Signal Setup
Quick Setup
- Navigate to Integrations in the sidebar.
- Open the Integration Library tab.
- Find Amazon DynamoDB or filter by Database.
- Click Install, select a variant if available, and fill in the required fields.
- Click Install Integration to create the integration with a ready-to-use default blueprint.
API Setup
curl -X POST http://localhost:8084/v1/admin/integration-catalog/amazon_dynamodb/install \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Amazon DynamoDB",
"variant": "default",
"config": {
"table_name": "datafly_events",
"region": "us-east-1",
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
},
"delivery_mode": "server_side"
}'Schema
Each event is written as one item with PutItem; a redelivered event replaces its own item (same message_id). Every attribute is a string (S):
| Attribute | Contents |
|---|---|
message_id | Partition key. Unique per delivered event. |
event_type | track, page, identify, and so on. |
event_name | Signal event name (snake_case), for example purchase. |
anonymous_id | First-party visitor identifier. |
user_id | Logged-in user identifier; absent when not known. |
event_timestamp | Client event time, ISO 8601 UTC with milliseconds (2026-09-23T10:00:00.000Z). |
received_at | Time Signal received the event, same format. |
org_id, pipeline_id | Signal identifiers. |
properties | The fields the blueprint mapped, as JSON text (below). |
context | The event context as JSON text: page, device, campaign, consent, IP and user agent. |
An item cannot exceed 400 KB, which a single event does not approach.
What the properties hold
The blueprint decides what properties contains, and only mapped fields are written (identify traits and unmapped client properties never are):
- The event’s own fields from the preset sit at the top level, for example
transaction_id,value,currency,items,insurance_typeorpolicy_term_months. - The preset’s global block sits under
@root: copies of the envelope (event_name,event_type,user_id,anonymous_id,event_id,message_id,event_timestamp,page_url,page_referrer,ip_address,user_agent,locale,timezone,source_id),utm_source,utm_medium,utm_campaign, the consent flagsconsent_analyticsandconsent_marketing, and the hashed identityemail_sha256,phone_sha256,first_name_sha256,last_name_sha256,name_sha256. identifyaddsname_sha256andpostal_code_sha256at the top level.- An event the preset does not list stores its envelope and the
@rootblock only.
properties is JSON text, so parse it after reading (for example with jq):
aws dynamodb get-item --table-name datafly_events \
--key '{"message_id": {"S": "<message_id>"}}' \
--query 'Item.properties.S' --output text | jq '{transaction_id, value, currency, email_sha256: .["@root"].email_sha256}'Mapped fields stay inside properties; they are not lifted to top-level fields. What else a row carries outside the blueprint (the whole event context) is under review in #1414.
Stored event context
Alongside the fields your blueprint maps (where email, phone, names and address are stored as normalised SHA-256 by default), each document carries the event’s collected context exactly as Signal received it. The blueprint does not filter or hash the context: this is deliberate (decision 2026-09-23), so the context is complete for attribution, fraud and debugging queries, and it is listed here so you can decide what to keep.
| Stored | What it contains |
|---|---|
| Visitor request | IP address, user agent, Accept-Language, locale and timezone |
| Page | URL including its query string, path, title and referrer |
| Campaign | UTM source, medium, campaign, term and content |
| Consent | The visitor’s consent state |
| Location | GeoIP country, region and city, and the network (ASN) |
| Device | Screen size; for app events the OS and version, manufacturer, model, advertising ID and IDFV when the app collected them |
| Vendor IDs | Cookie and click identifiers in vendor_ids (for example the _fbp browser ID, gclid, ttclid, the GA client ID) |
| Session and library | Session ID, number and engagement time; the collector’s name and version |
| Partner attributes | Profile attributes from partner enrichment (partner_attrs) |
They are stored as the document’s context field (Bigtable: the context column), using the event context’s JSON field names, beside anonymous_id and user_id. Click and collected IDs outside the context are stored only where the blueprint maps them.
The IP address, device and advertising IDs, and cookie and click identifiers are personal data under UK and EU GDPR. To keep any of them out of storage, strip, mask or hash the field for every integration with the organisation data layer, apply URL redaction to page URLs, or remove it in the installed blueprint; and set the storage’s retention to match your privacy notice. See Personal data by integration.
Consent
DynamoDB is a first-party destination under your control, gated by the same consent state as every other destination. Each stored event also records the visitor’s consent at collection time: the preset writes consent_analytics and consent_marketing under @root in the properties (properties), and the full consent object is in the context. Use them in views or queries if your governance needs a per-row consent filter.
Testing
- Enable the integration in Signal and trigger a test event on your website.
- Open the DynamoDB console and navigate to your table.
- Click Explore table items to verify that event items are appearing.
- Check the
message_idpartition key and confirm the event payload is stored correctly. - In Signal, check the Live Events view to confirm delivery status shows as successful.
Troubleshooting
| Problem | Solution |
|---|---|
| Events not appearing in DynamoDB | Verify the table name, region, and partition key match your DynamoDB table configuration. |
AccessDeniedException | The IAM user lacks dynamodb:PutItem or dynamodb:BatchWriteItem permission on the target table. Update the IAM policy. |
ResourceNotFoundException | The table does not exist in the specified region. Check the table name and region. |
ValidationException | The table’s key schema is not message_id (String) alone, for example a different partition key name or a sort key. Recreate the table with partition key message_id and no sort key. |
ProvisionedThroughputExceededException | Your table is in provisioned mode and has hit its write capacity. Switch to on-demand mode or increase provisioned WCU. |
| Credential errors | Verify the access key ID and secret access key are correct and the IAM user has not been deactivated. |
Visit Amazon DynamoDB documentation for full API reference and best practices.