Management APIOverview

Management API

The Management API is a Go REST + WebSocket service that powers the Datafly Signal management interface. It provides programmatic access to all platform configuration: pipelines, integrations (catalog, templates, revisions), transformations, users, and audit logs.

Base URL

All endpoints are prefixed with:

https://{cluster}/api/v1/admin

In local development, the Management API runs on port 8084:

http://localhost:8084/v1/admin

Authentication

All requests must include a valid JWT access token in the Authorization header:

Authorization: Bearer {access_token}

Obtain tokens via the authentication endpoints. Access tokens expire after 15 minutes; use the refresh endpoint to obtain new ones.

Content Type

All request and response bodies use JSON:

Content-Type: application/json

Rate Limiting

The API enforces a rate limit of 100 requests per minute per authenticated user. When the limit is exceeded, the API returns a 429 Too Many Requests response with a Retry-After header indicating when the client can retry.

{
  "error": {
    "code": "RATE_LIMITED",
    "message": "Rate limit exceeded. Retry after 32 seconds."
  }
}

Pagination

List endpoints use cursor-based pagination. Pass a cursor and limit as query parameters:

GET /v1/admin/sources?cursor=eyJpZCI6MTAwfQ&limit=50
ParameterTypeDefaultDescription
cursorstringOpaque cursor from a previous response’s next_cursor field
limitinteger50Number of items per page (max 100)

Paginated responses include a next_cursor field. When next_cursor is null, there are no more results.

{
  "data": [ ... ],
  "next_cursor": "eyJpZCI6MTUwfQ",
  "has_more": true
}

Error Format

All error responses follow a consistent format:

{
  "error": {
    "code": "INVALID_PIPELINE_KEY",
    "message": "The provided pipeline key is not valid or has been revoked."
  }
}
FieldTypeDescription
error.codestringMachine-readable error code (see Error Codes)
error.messagestringHuman-readable description of the error

HTTP Status Codes

StatusMeaning
200Success
201Resource created
204Success, no content (delete operations)
400Bad request (invalid JSON, missing fields)
401Unauthorized (missing or invalid token)
403Forbidden (insufficient permissions)
404Resource not found
409Conflict (duplicate resource)
422Unprocessable entity (validation error)
429Rate limited
500Internal server error

Sections

SectionDescription
AuthenticationLogin, token refresh, JWT format
PipelinesManage data collection pipelines, pipeline keys, JS collector builds
Integration CatalogBrowse and install from the integration library
Integration TemplatesManage org-level integration templates and immutable revisions
Pipeline IntegrationsAssign integrations to pipelines at specific revisions
Sources (Legacy)Legacy source management API
Integrations (Legacy)Legacy integration configuration API
TransformationsDefine and test event transformations
Data LayerOrg-wide data layer configuration
Users & RBACUser management and role-based access
Audit LogsQuery the audit trail

The Management API also exposes a WebSocket endpoint at ws://{cluster}/v1/admin/debug/live for real-time event debugging. Connect with a valid JWT to stream events as they flow through the pipeline.