Google Firestore

Datafly Signal delivers events to Google Cloud Firestore for flexible, scalable NoSQL document storage with real-time sync capabilities and offline support.

Prerequisites

Before configuring Google Firestore in Signal, you need a GCP project with a Firestore database, a target collection, and a service account.

Create a GCP Account and Project

Sign up at cloud.google.com if you don’t already have an account.
Create a new project or select an existing one in the GCP Console.
Note the Project ID.

Create a Firestore Database

Go to the Firestore console.
If prompted, click Create database.
Choose the Firestore mode:
- Native mode — recommended for Signal. Provides real-time listeners, offline support, and document-based queries.
- Datastore mode — optimised for server-side workloads, no real-time features.
Select a Location close to your Signal deployment.
Click Create database.

⚠️

The Firestore mode cannot be changed after creation. Choose Native mode for the most flexibility.

Define a Collection

Firestore collections are created automatically when the first document is written. However, you can optionally create the collection structure in advance:

In the Firestore console, click Start collection.
Enter the Collection ID (e.g. events).
Add a sample document or click Auto-ID and Save (you can delete this placeholder later).

Signal will create documents in the specified collection automatically. Each event becomes a separate document with the message_id as the document ID.

Create a Service Account

Go to IAM & Admin > Service Accounts > Create Service Account.
Enter a name (e.g. datafly-signal-firestore).
Grant the Cloud Datastore User role (roles/datastore.user) — this role covers both Firestore Native and Datastore mode.
Click Done.

Generate a Service Account Key

Click on the service account.
Go to Keys > Add Key > Create new key > JSON.
The key file will download. Store it securely.

⚠️

Store the JSON key file securely. Do not commit it to version control.

Configuration

Field	Type	Required	Description
`project_id`	string	Yes	The Google Cloud project ID that contains the Firestore database.
`collection`	string	Yes	The Firestore collection name to write documents to.
`service_account_json`	secret	Yes	The full JSON key file content for a GCP service account with Firestore permissions.

Signal Setup

Quick Setup

Navigate to Integrations in the sidebar.
Open the Integration Library tab.
Find Google Firestore or filter by Cloud Storage.
Click Install, select a variant if available, and fill in the required fields.
Click Install Integration to create the integration with a ready-to-use default blueprint.

API Setup

curl -X POST http://localhost:8084/v1/admin/integration-catalog/google_firestore/install \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Google Firestore",
    "variant": "default",
    "config": {
      "project_id": "datafly-analytics",
      "collection": "events",
      "service_account_json": "{\"type\": \"service_account\", ...}"
    },
    "delivery_mode": "server_side"
  }'

Testing

Enable the integration in Signal and trigger a test event on your website.
Open the Firestore console and navigate to the events collection.
Verify that event documents are appearing with the correct fields and data.
Click on individual documents to inspect the full event payload.
In Signal, check the Live Events view to confirm delivery status shows as successful.

Troubleshooting

Problem	Solution
Events not appearing in Firestore	Verify the project ID, collection name, and service account JSON are correct.
`Permission denied` (403)	The service account lacks the Cloud Datastore User role. Add it in IAM & Admin > IAM.
`NOT_FOUND`	The Firestore database has not been created in this project. Create it in the Firestore console.
Invalid service account JSON	Ensure you pasted the complete JSON key file content.
`RESOURCE_EXHAUSTED`	You have exceeded Firestore write quotas. Check the quotas page and consider batching.
Wrong Firestore mode	If the database is in Datastore mode, the collection structure differs. Verify the mode in the Firestore console.
Document size limit	Individual Firestore documents cannot exceed 1 MB. Check the event payload size.

Visit Google Firestore documentation for full API reference, security rules, and query best practices.

Google Cloud SQL Google Spanner