RabbitMQ Transporter
The RabbitMQ transport speaks AMQP 0-9-1 over a topic exchange. Publishers address a subject, the broker decides which queues want it, and adding a consumer needs no change on the publisher. Queues are the unit of work distribution: every replica of a service consumes the same queue, so the broker load-balances between them, while two different services use two different queues and both see the message. The one thing to get right before anything else is the queue name — the default is a durable, named, shared queue for a reason.
See Microservices Overview for patterns, handlers and the envelope.
Delivery guarantees
This is the durable transport of the set. Three defaults produce at-least-once delivery, and each is worth understanding:
- Publisher confirms are on. A bare AMQP publish is fire-and-forget even towards
the broker: it is a one-way frame, so a message the broker rejected or dropped
looks exactly like one it accepted. With confirms,
Publishwaits for the broker's verdict and returns an error when the broker nacks — a full disk, a failed quorum write. SetDisableConfirmsto trade that for throughput. - Acknowledgement is manual. A message is acked after the handler succeeds, so a
consumer crash mid-handler redelivers rather than loses.
AutoAckacknowledges on delivery instead, which makes the transport at-most-once — the broker forgets the message as soon as it hits the socket. - A failed handler nacks without requeue. A message that fails deterministically
therefore cannot spin the consumer in a hot loop. Configure
DeadLetterExchangeto keep those messages instead of discarding them.
A publisher confirm proves the broker accepted a message. It never proves anybody
was bound to receive it. Set Mandatory to have the broker return unroutable
messages; returns are logged rather than surfaced from Publish, because an
unroutable message is both Returned and Acked and there is no single verdict to
report.
Handlers should still be idempotent. At-least-once means exactly that: a redelivery after a crash between the handler's database write and its ack is normal operation, not a bug.
Minimal working setup
A server:
package main
import (
"github.com/gin-gonic/gin"
"github.com/nika-framework/nika"
"github.com/nika-framework/nika/common/microservice"
"github.com/nika-framework/nika/common/microservice/transport/rabbitmq"
"myapp/src"
)
func main() {
app := nika.NewApp()
microservice.Setup(app, microservice.Config{
Transport: rabbitmq.MustNew(rabbitmq.Options{
URL: "amqp://guest:guest@localhost:5672/",
Queue: "users.workers", // distinct per service
DeadLetterExchange: "nika.dlx",
}),
Middleware: []gin.HandlerFunc{nika.RecoveryMiddleware()},
})
app.LoadModule(src.NewAppModule())
app.Listen(":3001")
}
Handlers declare transport:"rabbitmq":
type UserController struct {
Create func(*gin.Context) `transport:"rabbitmq" pattern:"user_created"`
FindOne func(*gin.Context) `transport:"rabbitmq" pattern:"user_find"`
}
A client, handling the error:
transport, err := rabbitmq.New(rabbitmq.Options{
URL: "amqp://guest:guest@localhost:5672/",
Queue: "orders.workers",
})
if err != nil {
return fmt.Errorf("rabbitmq transport: %w", err)
}
client, err := microservice.SetupClient(app, transport)
if err != nil {
return err
}
New does not dial. Connections are established lazily and re-established after a
drop, so constructing a transport never fails because the broker happens to be
restarting, and a wiring mistake surfaces as a configuration error rather than as a
timeout. A dial failure is logged with the credentials stripped from the URL.
Options
| Field | Default | Purpose |
|---|---|---|
URL |
— | AMQP dial string, e.g. amqp://guest:guest@localhost:5672/. Required unless Conn is set. Ignored when Conn is set. |
Conn |
nil |
Reuse an existing *amqp.Connection. Close closes this transport's channels but never a connection it did not dial. |
Exchange |
"nika" (DefaultExchange) |
The topic exchange to publish to and bind against. |
ExchangeType |
"topic" (DefaultExchangeType) |
Anything else disables broker-side pattern routing and leaves all filtering to the Router. |
Queue |
"nika.workers" (DefaultQueue) |
This service's queue. Give every service a distinct name — see below. |
Durable |
Bool(true) |
Declare a topology that survives a broker restart. A *bool, because the safe value is not the zero value; use rabbitmq.Bool(false) to opt out. |
AutoDelete |
false |
Delete the queue when its last consumer disconnects. Leave it false — see below. |
QueueArgs |
nil |
Extra queue x-arguments (x-max-length, x-queue-type, x-message-ttl, …). DeadLetterExchange is merged into these. |
DeadLetterExchange |
"" |
Declares the queue with x-dead-letter-exchange, so a nacked message is republished there instead of discarded. |
Prefetch |
32 (DefaultPrefetch) |
QoS prefetch count, applied per consumer. |
Concurrency |
Prefetch |
Deliveries handled at once by this transport. |
AutoAck |
false |
Acknowledge on delivery instead of after the handler. Makes delivery at-most-once. |
Requeue |
false |
Make a failed handler nack with requeue=true. See the poison-message section. |
Mandatory |
false |
Ask the broker to return a message that matched no queue. Returns are logged. |
DisableConfirms |
false |
Turn off publisher confirms, so Publish returns as soon as the frame is written. |
ReplyTimeout |
microservice.DefaultRequestTimeout (10s) |
Request deadline when the caller passes none. |
TLSConfig |
nil |
Used when URL has the amqps scheme. |
Heartbeat |
10s (DefaultHeartbeat) |
AMQP heartbeat interval. AMQP over a NAT or a load balancer dies silently without one. |
Logger |
slog.Default() |
Receives malformed-message and reconnect events. |
Concurrency defaults to Prefetch because a prefetch larger than the concurrency
just parks messages in a client-side buffer, where they are invisible to the broker's
queue-depth metrics.
The queue name is the expensive mistake
An anonymous auto-delete queue silently drops everything published during a deploy
An anonymous, auto-delete queue exists only while its consumer is connected. Every message published during a deploy, a crash loop or a broker reconnect is dropped by the broker with no error anywhere: the publisher's confirm succeeds, the exchange matches no queue, and the message is gone. A durable named queue keeps accumulating while the service is down and drains when it returns.
That is why Queue defaults to "nika.workers" — durable, named, shared — rather
than to a server-generated name, and why Durable is Bool(true) rather than a plain
bool whose zero value would be false. Leave AutoDelete alone unless the queue
really is disposable.
Two more consequences of "the queue is the unit of distribution":
- Every service sharing an
Exchangemust set a distinctQueue. Two services on one queue compete for messages instead of both receiving them, so roughly half of each service's messages go to the other one. - Every replica of one service shares its queue. That is what makes scaling out divide the work instead of multiplying it — the opposite of the broadcast default on NATS without a queue group.
Wildcard patterns force a # binding
An AMQP topic binding key looks like a Nika pattern and is not. AMQP topic keys are
dot-separated words, and its metacharacters occupy a whole word: * matches
exactly one word, # matches zero or more words. A word that merely contains a
metacharacter — user_* — is not a wildcard at all; RabbitMQ compares it literally,
so binding user_* receives nothing except a message literally routed to user_*.
Nika wildcards are character-level: user_* matches user_23 and user_created.
There is therefore no AMQP binding key that expresses "any suffix within one word",
which is the common case in this framework. The resolution:
- A literal pattern binds to its own routing key, so the broker filters and the queue only sees traffic it can handle.
- Any wildcard pattern forces one catch-all
#binding, and the core Router does the character-level filtering in-process.
The cost is real and worth stating: the queue then receives every message published
to the exchange, including subjects owned by other services sharing it, which the
Router answers with a 404 / PATTERN_NOT_FOUND envelope. A warning is logged when
this happens. Give each service its own Exchange — or its own vhost — when that
matters.
When the catch-all is required, the literal binding keys are dropped from the plan.
That is an efficiency choice, not a correctness requirement: RabbitMQ guarantees that
for every queue a message is routed to, the queue receives exactly one copy of that
message, so keeping both a # and a literal binding on the same queue would not
duplicate deliveries. # simply already subsumes the literals, which makes the extra
bindings pure bookkeeping in the broker's binding table.
Patterns AMQP cannot carry
These are rejected before the transport touches the network, since a pattern AMQP cannot express is a programming error rather than a broker problem:
| Rejected | Why |
|---|---|
. in a pattern |
The AMQP word separator. user.created would be two words and would start matching * bindings the author never intended. Use _. |
# in a pattern |
The AMQP multi-word wildcard. Allowing it would let a pattern quietly subscribe to the whole exchange. |
| whitespace or a control character | Not usable in a key. |
| over 255 bytes | The AMQP short-string limit for a routing or binding key. |
Publishing to a wildcard subject is rejected too — a published subject must be literal.
Prefetch is what balances work across replicas
Without a QoS prefetch the broker pushes the whole queue at whichever consumer
connected first, and every other replica idles with an empty queue while the first one
drowns. Prefetch defaults to 32 and is applied per consumer (global=false), which
is what makes the balance work — a connection-wide budget would not.
Poison messages
A handler failure nacks without requeue by default. That is deliberate: a requeued message goes back to the head of the queue and is redelivered immediately, so a message that fails deterministically — a malformed payload, a bug, a missing row — is redelivered forever at the speed of the CPU. That is the classic AMQP poison-message outage.
The supported way to keep those messages is a dead-letter exchange:
rabbitmq.MustNew(rabbitmq.Options{
URL: "amqp://guest:guest@localhost:5672/",
Queue: "users.workers",
DeadLetterExchange: "nika.dlx",
})
A nacked message is then republished there instead of discarded, and an operator can inspect and replay it without the consumer ever retrying in a loop.
Changing DeadLetterExchange on an existing queue
AMQP queue arguments are immutable. Changing this value requires deleting and
redeclaring the queue; a mismatched redeclaration fails the channel with
PRECONDITION_FAILED.
Turn Requeue on only when your handler failures are genuinely transient, and pair it
with DeadLetterExchange plus a queue x-delivery-limit so the broker itself caps the
retries. A retry counter derived from the x-death header is deliberately not
implemented: x-death only exists after the broker has already dead-lettered the
message, so counting it in the consumer reimplements broker policy in application code
and silently stops working when an operator changes the dead-letter topology. Retry
limits belong in the queue definition.
A message whose bytes do not decode is rejected without requeue whatever Requeue
says, because bytes that do not parse now will not parse on redelivery either — that
is a guaranteed infinite loop, not a retry.
Request/reply
Request uses the standard AMQP RPC pattern: one long-lived exclusive reply queue per
client process, correlation by CorrelationId, and a single consumer demultiplexing
into a map of waiters. The reply queue is established — declared and consumed —
before anything is published, so a peer that answers instantly cannot beat the
client to its own mailbox.
The request carries an Expiration equal to its timeout, so the broker drops it once
the caller's deadline has passed. A backed-up queue then does not hand a consumer work
nobody is listening for any more.
A lost connection while waiting fails the request at once rather than after the full timeout, because a reply can no longer arrive over a dead channel.
Testing
Handlers need no broker. nikatest.NewMicroservice(t) runs the whole message stack
over the in-memory transport:
ms := nikatest.NewMicroservice(t)
ms.LoadModule(src.NewAppModule())
ms.Send("user_created", CreateUserDto{Name: "Ada"}).
ExpectStatus(201).
ExpectJSONPath("data.name", "Ada")
See Testing.
The routing translation — the part of an AMQP integration most likely to be wrong, and the hardest to observe at runtime, since a bad binding fails by receiving nothing rather than by erroring — is unit-tested without a broker. The rest of the suite needs a live broker and is behind a build tag:
RABBITMQ_URL=amqp://guest:guest@localhost:5672/ \
go test -race -tags rabbitmq_integration ./common/microservice/transport/rabbitmq/
When to use it, and when not to
Use RabbitMQ when messages are work that must not be lost: a queue that keeps accumulating while a consumer is down, per-message acknowledgement, per-replica work sharing, and a dead-letter exchange for the ones that will never succeed. It is the right default for command-style traffic — "charge this card", "send this email", "resize this image" — where a dropped message needs a human to notice.
Do not use it as an event log. A message is gone once acked, so there is no replay and no second consumer group arriving later to read history — that is Kafka. Do not reach for it for latency-sensitive RPC either: it works, but each call costs a broker hop in each direction, where NATS and gRPC are direct. And if a missed message is genuinely acceptable, Redis or NATS will do the same job with less operational surface.