Skip to main content

Message Queue Isolation

Message queues introduce asynchronous communication between microservices, typically via a message queue broker or service. Since communication is indirect, one might consider including the broker and producers in sandboxes when working on a message queue consumer microservice, to ensure the sandbox consumer does not interfere with the baseline by consuming messages intended for the baseline.

However, this is often impractical in terms of setup time and resource usage. Signadot provides more effective alternatives for isolating message queue traffic in sandboxes.

General Approaches to Message Queue Isolation​

1. Message-Level Routing (Routing API)​

Many message queues (e.g., Kafka) support multiple consumers. In these scenarios, each consumer can receive all messages. To isolate sandboxes, you can:

  1. Decorate messages with a routing key in the header or message attribute.
  2. Send all messages to all consumers (e.g., Kafka consumer groups).
  3. Configure each consumer to process or ignore messages based on the routing key.

This approach is lightweight, flexible, and works with RouteGroups. It allows sharing the message queue broker between the baseline and sandboxes without interference.

Signadot provides gRPC and REST APIs, and library code to support this methodology. See this working demo.

2. Topic/Queue-Based Isolation​

Alternatively, use ResourcePlugins to create dedicated topics or queues for each sandbox. This requires only topic/queue name configuration for producers and consumers, but is less flexible than message-level routing.

System-Specific Integrations​

For detailed guidance on implementing message queue isolation with specific systems, see the following pages: