Encryption

Datafly Signal encrypts data at every layer — in transit between all components and at rest in storage. This page details the encryption mechanisms used throughout the platform.

Encryption in Transit

Browser to Ingestion Gateway

All event data sent from Datafly.js to the Ingestion Gateway is encrypted using TLS 1.2+. The customer’s subdomain (e.g., data.example.com) is served with a valid TLS certificate.

Service to Service

Internal communication between Datafly services within a cluster is secured at the network level. For deployments that require service mesh encryption (e.g., compliance with zero-trust network policies), Datafly is compatible with Istio and Linkerd mTLS.

All infrastructure connections (message queue, cache, database) support TLS. In production deployments with managed cloud services, TLS is enabled by default.

Service to Vendor APIs

All Delivery Worker connections to vendor APIs use TLS 1.2+. Vendor API endpoints universally require HTTPS. The Delivery Worker validates the vendor’s TLS certificate and rejects connections with invalid or expired certificates.

Encryption at Rest

Vendor API Credentials

Vendor API credentials (API secrets, access tokens, OAuth refresh tokens) stored in the database are encrypted using AES-256-GCM before being written to storage. The encryption key is provided via an environment variable and should be stored in a secrets manager.

When a Delivery Worker needs credentials at delivery time, it decrypts them in memory. Credentials are never written to disk in plaintext.

Database

In production, the database should use Transparent Data Encryption (TDE):

TDE encrypts the entire database at the storage layer, protecting against physical disk theft or snapshot exfiltration.

Message Queue

In production message queue deployments:

Cross-Domain Identity Tokens

The Identity Hub encrypts cross-domain identity tokens to securely transfer visitor identity between domains. Tokens are short-lived and single-use, preventing replay attacks.

See Cross-Domain Identity for configuration details.

PII Hashing

When PII hashing is enabled in the Org Data Layer, specified fields are hashed with SHA-256 before being delivered to vendor APIs. Values are lowercased and trimmed before hashing to ensure consistent results.

This allows vendors like Meta and Google to perform identity matching using hashed identifiers without receiving raw PII. The fields to hash are configurable per organisation.

Key Management

All encryption keys and secrets should be stored in a dedicated secrets manager (e.g. AWS Secrets Manager, HashiCorp Vault, Kubernetes Secrets).

Enterprise Key Management (KMS)

For enterprise and regulated environments, Signal supports envelope encryption with cloud-managed Key Management Services. The deployment encryption key (DEK) is wrapped by a cloud-managed Key Encryption Key (KEK) — the plaintext DEK only exists in memory at runtime.

Supported KMS Providers

Configuration

Configure via signal.yaml or environment variables:

encryption:
  provider: "gcp-kms"
  require_encryption: true
 
  gcp_kms:
    key_resource_name: "projects/my-project/locations/global/keyRings/signal/cryptoKeys/dek"

Or using environment variables:

Mandatory Encryption

When require_encryption: true, all services that handle PII will refuse to start without a valid encryption key. This prevents accidental plaintext storage in production.

Build Tags (Cloud Providers)

Cloud KMS providers are compiled in via Go build tags to avoid unnecessary SDK dependencies:

# Include all cloud providers (recommended for enterprise)
RUN go build -tags "gcpkms,awskms,azurekms,vaultkms" -o /app ./cmd/...
 
# Include only GCP (lighter build)
RUN go build -tags "gcpkms" -o /app ./cmd/...

The local provider is always available and requires no build tags.

Key Rotation

Signal supports zero-downtime key rotation using a dual-key mechanism. All ciphertext includes a version byte, allowing the system to identify which key was used for encryption.

Rotation Procedure

Step 1 — Deploy the new key alongside the old:

# Set the new key as primary, old key as fallback
ENCRYPTION_KEY=<new-key>
ENCRYPTION_KEY_PREVIOUS=<old-key>

Perform a rolling restart of all services. New data will be encrypted with the new key. Old data decrypts using the fallback key.

Step 2 — Re-encrypt existing data:

# Trigger background re-encryption via the Management API
curl -X POST https://api.example.com/v1/admin/crypto/re-encrypt \
  -H "Authorization: Bearer <token>"
 
# Monitor progress
curl https://api.example.com/v1/admin/crypto/re-encrypt \
  -H "Authorization: Bearer <token>"

The re-encryption job runs in the background, decrypting each record with the old key and re-encrypting with the new key. Progress is reported via the GET endpoint.

Step 3 — Remove the old key:

Once re-encryption is complete (verified by the status endpoint), remove ENCRYPTION_KEY_PREVIOUS and perform a final rolling restart.

Rotation Cadence

Crypto Health Check

All services expose a GET /health/crypto endpoint that verifies encryption is operational:

{
  "status": "healthy",
  "provider": "gcp-kms",
  "key_version": 1,
  "dual_key_mode": false,
  "checks": {
    "key_valid": true,
    "round_trip": true
  }
}

This endpoint is integrated into the Kubernetes readiness probe — a service with a failing crypto health check will be removed from the load balancer.

Audit Logging

All cryptographic operations are recorded in the audit log:

Audit logs never contain plaintext keys, PII, or raw ciphertext — only metadata (key versions, record IDs, operation types, timestamps, and actor identity).

Redis TLS

Redis connections support TLS for encrypting cached data in transit:

redis:
  address: "redis.internal:6380"
  tls:
    enabled: true
    ca_cert_path: "/certs/ca.crt"

Or via environment variables:

All major managed Redis services (ElastiCache, Memorystore, Azure Cache) support TLS.