System Overview
Synapse Relay is a monorepo containing three application services, a set of infrastructure dependencies, and a policy layer. The three services are independently deployable and communicate only through well-defined HTTP interfaces and shared database state.
Data Flow
Understanding the data flow is the key to understanding the security guarantees. Every piece of data that enters the system is handled according to one consistent rule: the original may flow to the tool, but only the redacted version is ever stored.
This means the audit log contains no PII by construction, not by policy enforcement. An attacker with read access to the database cannot reconstruct personal information from the logs because it was never written there. The redaction happens in the gateway's request path, before the logging async goroutine is invoked.
The only place original data flows is the forwarded request to the upstream MCP server. The tool needs the real data to function. The gateway passes it through unchanged in that direction only.
The Request Path in Detail
A tool call enters the gateway as a JSON-RPC 2.0 POST request. The gateway reads the entire body into memory — up to 4MB, configurable — before any processing begins. This is a deliberate choice: streaming processing of a partially-read body creates race conditions in the PII detection step.
The request is parsed to determine the JSON-RPC method. Only tools/call triggers the full pipeline. All other MCP methods — tools/list, resources/list, and others — are proxied directly to the upstream server without interception. This keeps latency minimal for non-sensitive operations.
For tools/call, the pipeline executes in sequence, short-circuiting at the first failure: rate limit → policy → PII scan → approval gate → upstream forward → PII scan on response → async log.
Go Gateway
The gateway is a single Go binary compiled to a scratch container image with no OS dependencies. It starts in under 100 milliseconds and handles tens of thousands of concurrent connections without tuning. The choice of Go over Node.js for this layer was deliberate: the gateway is on every request's hot path and needs deterministic low latency, not flexibility.
The rate limiter uses Redis sorted sets implementing a sliding window algorithm. Each key is rl:{tenant_id}:{tool_name}. A pipeline of four Redis commands atomically removes expired entries, counts current members, adds the new timestamp, and sets expiry. The entire check-and-increment is atomic from the gateway's perspective.
The OPA client uses a 2-second timeout. If OPA is unavailable, the gateway fails open — the call is allowed, and the failure is logged. This is the correct behavior for an operational system: a policy engine outage should not bring down production traffic. The inverse — failing closed — is appropriate only for security-critical contexts where the organization has explicitly accepted the availability tradeoff.
Node.js API
The API handles everything that requires business logic: authentication, MCP server generation, analytics queries, connector imports, and the WebSocket live feed. It communicates with the gateway only through shared database state and an internal HTTP endpoint — there is no direct gateway-to-API call in the hot path.
MCP server generation is asynchronous by design. The API creates a generating record in PostgreSQL and returns 202 Accepted immediately. The actual LLM call runs in setImmediate so the HTTP response is not blocked. When generation completes, the record transitions to ready and a WebSocket event is broadcast to all connected dashboard clients in the same tenant.
vLLM Model Server
vLLM uses PagedAttention to manage GPU memory without fragmentation, and continuous batching to process multiple requests simultaneously without fixed batch sizes. The result is near-linear throughput scaling with concurrent users — unlike naive inference servers that process requests one at a time.
| Model | VRAM Required | Generation Speed | Quality |
|---|---|---|---|
| Qwen2.5-Coder-7B-Instruct | 16GB | ~1.5s per server | Good |
| Qwen2.5-Coder-32B-Instruct | 40GB (single) | ~4s per server | Excellent |
| Qwen2.5-Coder-32B-Instruct | 24GB×2 (TP=2) | ~2s per server | Excellent |
| DeepSeek-Coder-V2 | 80GB×2 (TP=2) | ~3s per server | Best in class |
OPA Policy Engine
Open Policy Agent runs as a sidecar container exposing its REST API on port 8181. Policies are Rego files mounted as a volume — changes to policy files are reloaded without restarting OPA. The gateway evaluates policy by POSTing to /v1/data/synapse/relay with the full call context as input.
The input document includes: tenant ID, tenant plan, tool name, MCP server ID, client ID, current request rate, and allowed rate. The output document includes three boolean fields: allow, requires_approval, and redact_pii. The gateway acts on all three.
PII Redaction
Microsoft Presidio runs as two microservices: an analyzer (detection) and an anonymizer (replacement). The gateway serializes the tool call arguments to JSON, POSTs to the analyzer, receives a list of entity spans with types and confidence scores, then POSTs the original text plus those spans to the anonymizer, which returns the redacted text.
All entities are replaced with their type token — <EMAIL>, <PHONE>, <US_SSN>, etc. The type is preserved for audit and analytics purposes. A reviewer can see that PII was present without being able to reconstruct what it was.
If Presidio is unavailable, the gateway falls back to a compiled regex layer covering the most common entity types. This fallback is not as accurate as the ML-based analyzer but provides a meaningful safety net.
Database Schema
Eight tables, designed for append-mostly workloads. Tool calls are never updated after creation — status transitions are expressed as new column values, not row replacements. The primary indexes are on (tenant_id, created_at DESC) for the live feed query and on status filtered to pending states for the approval queue.
| Table | Purpose | Key columns |
|---|---|---|
| tenants | Organization isolation unit | id, slug, plan |
| users | Auth and RBAC | id, tenant_id, role, password_hash |
| api_keys | Programmatic access tokens | key_hash, key_prefix, scopes |
| mcp_servers | Generated and imported servers | source_type, status, endpoint_url, tools, generated_code |
| tool_calls | Immutable audit log | tool_name, redacted_input, redacted_output, latency_ms, pii_detected |
| approval_requests | Human review queue | tool_call_id, status, expires_at, reviewer_id |
| policies | Stored OPA policy config | rego_source, enabled |
| connectors | Composio/Zapier imports | provider, config, mcp_server_id |
Metrics Reference
| Metric | Type | Labels |
|---|---|---|
| synapse_relay_tool_calls_total | Counter | tool_name, mcp_server, status, tenant_id |
| synapse_relay_tool_call_duration_ms | Histogram | tool_name, mcp_server |
| synapse_relay_policy_decisions_total | Counter | result, tool_name, tenant_id |
| synapse_relay_pii_detections_total | Counter | entity_type, tenant_id |
| synapse_relay_pending_approvals | Gauge | — |
| synapse_relay_rate_limit_hits_total | Counter | target_type, tenant_id |
| synapse_relay_active_connections | Gauge | — |
| synapse_relay_gateway_requests_total | Counter | method, path, status_code |
| synapse_relay_gateway_request_duration_ms | Histogram | method, path |