Google Cloud Storage

Datafly Signal writes batches of first-party events as objects into a GCS bucket — ready for query with BigQuery external tables, Dataproc, or any analytics framework that reads GCS.

Why route Google Cloud Storage through Datafly Signal

• The data lands in infrastructure you own. Signal writes to your Google Cloud Storage directly from your own deployment. Datafly operates no intermediary and never holds a copy of your event data.
• Client IP and user agent are recorded as event context. Both come from the original visitor request rather than from your server, so geography and device analysis done in the warehouse reflects the actual visitor.
• Presets for eight verticals. General (the default: page, identify and the key cross-vertical events), Retail, Insurance, Travel, Media & Streaming, Publishing, B2B / SaaS and Automotive, each writing one named field per documented event property. Email, phone and names are written only as SHA-256 digests unless you deliberately change the installed blueprint.
• One consent decision governs every destination. Google Cloud Storage is gated by the same consent state as the rest of your stack, so a withdrawal applies everywhere at once.

Prerequisites

Before configuring Google Cloud Storage in Signal, you need a GCP project with a GCS bucket and a service account with the Storage Object Creator role.

Configuration

Signal Setup

Quick Setup

1. Navigate to Integrations in the sidebar.
2. Open the Integration Library tab.
3. Find Google Cloud Storage or filter by Cloud Storage.
4. Click Install, select a variant if available, and fill in the required fields.
5. Click Install Integration to create the integration with a ready-to-use default blueprint.

API Setup

curl -X POST http://localhost:8084/v1/admin/integration-catalog/google_cloud_storage/install \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Google Cloud Storage",
    "variant": "default",
    "config": {
      "bucket_name": "datafly-events-production",
      "project_id": "datafly-analytics",
      "service_account_json": "{\"type\": \"service_account\", ...}",
      "prefix": "events/"
    },
    "delivery_mode": "server_side"
  }'

What Signal writes

By default each event is one object (set batch_size to consolidate events into one file per partition-hour). Object names use Hive-style partitions of the event time in UTC:

gs://<bucket>/<prefix>/yyyy=2026/mm=09/dd=23/hh=09/<pipeline_id>/<message_id>.ndjson

The extension and content type follow file_format: .ndjson (application/x-ndjson), .csv (text/csv) or .parquet (application/vnd.apache.parquet).

JSON (default)

One line per event: the Signal event as JSON, the same document the streaming destinations send.

CSV and Parquet, canonical schema

A header row and one row per event in the 16 fixed columns the SQL warehouses use, all text (timestamps ISO-8601 in UTC): message_id, event_type, event_name, anonymous_id, user_id, ip_address, user_agent, page_url, page_path, page_referrer, org_id, pipeline_id, event_timestamp, received_at, properties_json (the mapped properties object above) and context_json. A purchase (digests shortened):

message_id,event_type,event_name,anonymous_id,user_id,ip_address,user_agent,page_url,page_path,page_referrer,org_id,pipeline_id,event_timestamp,received_at,properties_json,context_json
msg-s1-amazon_s3-2,track,purchase,anon-s1-7f3a,crm-4711,203.0.113.9,Mozilla/5.0 (Macintosh),https://shop.example/checkout,/checkout,,org-s1,pl-s1,2026-09-23T09:30:02.000Z,2026-09-23T09:30:02.000Z,"{""@root"":{""anonymous_id"":""anon-s1-7f3a"",""email_sha256"":""86e0b9e5..."", ...},""currency"":""GBP"",""items"":[...],""transaction_id"":""SIG-ORDER-1001"",""value"":84.5}","{""ip"":""203.0.113.9"",""page"":{...},""campaign"":{...}}"

Blueprint schema mode

With schema_mode set to blueprint, each file has one typed column per field instead (File formats and schema). Inferred columns are message_id, event_name and event_timestamp plus the event section’s top-level fields. The global block’s fields (identifiers, digests, UTM, consent) sit under @root and are not inferred as columns; declare them with an @root. source:

message_id
event_name
event_timestamp:timestamp
user_id:string:@root.user_id
email_sha256:string:@root.email_sha256
transaction_id
value:double
currency

which writes, for the same purchase:

message_id,event_name,event_timestamp,user_id,email_sha256,transaction_id,value,currency
msg-s1-google_cloud_storage-2,purchase,2026-09-23T09:30:02.000Z,crm-4711,86e0b9e56c17cc4d12387e1949b85053fbe73bc3ce5a1188713a9d300cc6133d,SIG-ORDER-1001,84.5,GBP

Stored event context

Alongside the fields your blueprint maps (where email, phone, names and address are stored as normalised SHA-256 by default), each file row carries the event’s collected context exactly as Signal received it. The blueprint does not filter or hash the context: this is deliberate (decision 2026-09-23), so the context is complete for attribution, fraud and debugging queries, and it is listed here so you can decide what to keep.

In each NDJSON line these are the context object, with anonymous_id, user_id, device_id, message_id, event_id, click_ids and collected_ids beside the mapped properties. In CSV and Parquet with the canonical schema they are the ip_address, user_agent, page_url, page_path and page_referrer columns and, in full, the context_json column. In blueprint schema mode only the columns you declare are written.

The IP address, device and advertising IDs, and cookie and click identifiers are personal data under UK and EU GDPR. To keep any of them out of storage, strip, mask or hash the field for every integration with the organisation data layer, apply URL redaction to page URLs, or remove it in the installed blueprint; and set the storage’s retention to match your privacy notice. See Personal data by integration.

Consent

The integration’s consent category in Signal decides which events reach Google Cloud Storage; events that fail it are never written. Every row also carries the visitor’s consent state (context.consent / context_json, and consent_analytics / consent_marketing in @root) so downstream queries can apply their own rules.

Testing

1. Enable the integration in Signal and trigger a test event on your website.
2. Open the Cloud Storage console and navigate to your bucket.
3. Browse to the prefix path and verify that event files are appearing.
4. Click on a file to download and inspect the event data.
5. In Signal, check the Live Events view to confirm delivery status shows as successful.

Troubleshooting

See also

• Google BigQuery
• Google Pub/Sub
• Amazon S3
• Azure Blob Storage