Deployment
Datafly Signal supports three deployment models depending on your stage of development and compliance requirements.
Deployment Options
| Option | Use Case | Infrastructure |
|---|---|---|
| Docker Compose | Local development and testing | Your workstation |
| Kubernetes | Production, managed by Datafly | Cloud Kubernetes cluster |
| Customer-Hosted | Production, customer’s own VPC | Customer’s cloud account |
Cloud-Specific Guides
Step-by-step deployment guides for each cloud provider, including infrastructure provisioning, secrets management, TLS, and verification:
| Cloud | Kubernetes | Kafka | Redis | PostgreSQL | Guide |
|---|---|---|---|---|---|
| AWS | EKS | MSK | ElastiCache | RDS | ALB Ingress, Secrets Manager |
| GCP | GKE | Confluent Cloud | Memorystore | Cloud SQL | GCE Ingress, Secret Manager |
| Azure | AKS | Event Hubs | Azure Cache | Azure DB | nginx/AGIC, Key Vault |
Use the Sizing Calculator to determine the right infrastructure tier for your event volume.
Architecture
All deployment models run the same set of services:
+-----------------------+
| Management UI |
| (Next.js, :3000) |
+----------+------------+
|
+----------v------------+
| Management API |
| (Go, :8084) |
+----------+------------+
|
Browser -----> +-----------v-----------+
| Ingestion Gateway |
| (Go, :8080) |
+-----------+-----------+
|
+------v------+
| Kafka |
+------+------+
|
+-----------v-----------+
| Event Processor |
| (Go, :8081) |
+-----------+-----------+
|
+------v------+
| Kafka |
+------+------+
|
+-----------v-----------+
| Delivery Workers |
| (Go, :8082) |
+-----------+-----------+
|
Vendor APIs
+------------------+ +------------------+
| Identity Hub | | PostgreSQL |
| (Go, :8083) | | Redis |
+------------------+ +------------------+Infrastructure Requirements
All deployment models require three infrastructure services:
| Service | Purpose | Production Recommendation |
|---|---|---|
| Apache Kafka | Event streaming between services | AWS MSK, Confluent Cloud, or self-managed |
| Redis | Caching (pipeline keys, sessions, rate limits) | AWS ElastiCache, GCP Memorystore, or self-managed |
| PostgreSQL | Configuration, users, audit logs | AWS RDS, GCP Cloud SQL, or self-managed |
Database Migrations
Regardless of deployment model, database migrations must be run before starting services after any upgrade. See the Migrations page for details.
Single-Tenant Isolation
Datafly Signal uses a single-tenant deployment model: each customer gets their own isolated deployment. In Kubernetes, this means one namespace per customer with dedicated service instances, Kafka topics, and database schemas.
Single-tenant isolation ensures that customer data never co-mingles. Each deployment is independently scalable, upgradeable, and configurable.
Next Steps
- Getting started locally? Begin with Docker Compose.
- Going to production? Read the Kubernetes deployment guide, then follow the guide for your cloud: AWS, GCP, or Azure.
- Sizing your deployment? Use the Sizing Calculator to get node specs, managed service tiers, and cost estimates.
- Compliance requirements? Review Customer-Hosted deployment.
- Setting up monitoring? See Observability for Prometheus metrics and Grafana dashboards.
- Upgrading? Follow the Upgrades guide. Migrations run automatically with the Helm chart.
- Disaster recovery? Configure Backup & DR for your database.
- Having issues? Check Troubleshooting for common problems and solutions.